Re: [oVirt Jenkins] check_gerrit_ovirt_org - Build # 20653 - Failure!

From: "Itamar Heim" <iheim(a)redhat.com&gt; To: "Jenkins oVirt Server" <jenkins(a)ovirt.org&gt; Cc: infra(a)ovirt.org, "Robert Middleswarth" <robert(a)middleswarth.net&gt;, "Eyal Edri" <eedri(a)redhat.com&gt; Sent: Thursday, August 16, 2012 1:03:56 PM Subject: Re: [oVirt Jenkins] check_gerrit_ovirt_org - Build # 20653 - Failure! gerrit has been recycled again. from reading about this, there are several possible issues causing this: 1. large patches submitted 2. large patches being viewed in side by side diff all translate to memory issues or a bug in jetty causing the machine to fill the log with: [2012-08-16 03:45:24,764] WARN org.eclipse.jetty.util.log : Dispatched Failed! 3. there are several postmaster processes occupying a lot of memory, could be large patches also affect us with the new full patch in email (gerrit 2.5 has a limitation on size of patches to be emailed iirc) 4. robert/eyal - can you please verify your versions are at least these: "Just as a follow-up, we have a couple of Jenkins instances talking to gerrit via the Gerrit Trigger plugin. It appears that after upgrading each of the Jenkins instances and the trigger plugin that the issues have cleared up. The versions we're now running are (in case it helps anyone in the future): * Gerrit 2.3

* Jenkins 1.461

* Jenkins GIT plugin 1.1.7

* Gerrit Trigger 2.5.1 "

from: https://groups.google.com/forum/?fromgroups#!topic/repo-discuss/TVhBSBHEJ... On 08/16/2012 12:54 PM, Jenkins oVirt Server wrote: > Project: http://jenkins.ovirt.org/job/check_gerrit_ovirt_org/ > Build: http://jenkins.ovirt.org/job/check_gerrit_ovirt_org/20653/ > Build Number: 20653 > Build Status: Failure > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #20653 > > > > ----------------- > Failed Tests: > ----------------- > No tests ran. > > ------------------ > Build Log: > ------------------ > Started by timer > Building remotely on rhel_62_slave_01 in workspace > /home/jenkins/workspace/check_gerrit_ovirt_org > No emails were triggered. > [check_gerrit_ovirt_org] $ /bin/sh -xe > /tmp/hudson1221775034538770259.sh > + rm index.html > + rm 'gitweb?p=test.git' > + wget -v -d gerrit.ovirt.org --timeout 60 -t 3 > Setting --timeout (timeout) to 60 > Setting --tries (tries) to 3 > DEBUG output created by Wget 1.12 on linux-gnu. > > --2012-08-16 05:51:04-- http://gerrit.ovirt.org/ > Resolving gerrit.ovirt.org... 107.22.212.69 > Caching gerrit.ovirt.org => 107.22.212.69 > Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected. > Created socket 3. > Releasing 0x000000000105b860 (new refcount 1). > > ---request begin--- > GET / HTTP/1.0 > User-Agent: Wget/1.12 (linux-gnu) > Accept: */* > Host: gerrit.ovirt.org > Connection: Keep-Alive > > ---request end--- > HTTP request sent, awaiting response... Read error (Connection > timed out) in headers. > Closed fd 3 > Retrying. > > --2012-08-16 05:52:05-- (try: 2) http://gerrit.ovirt.org/ > Found gerrit.ovirt.org in host_name_addresses_map (0x105b860) > Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected. > Created socket 3. > Releasing 0x000000000105b860 (new refcount 1). > > ---request begin--- > GET / HTTP/1.0 > User-Agent: Wget/1.12 (linux-gnu) > Accept: */* > Host: gerrit.ovirt.org > Connection: Keep-Alive > > ---request end--- > HTTP request sent, awaiting response... Read error (Connection > timed out) in headers. > Closed fd 3 > Retrying. > > --2012-08-16 05:53:07-- (try: 3) http://gerrit.ovirt.org/ > Found gerrit.ovirt.org in host_name_addresses_map (0x105b860) > Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected. > Created socket 3. > Releasing 0x000000000105b860 (new refcount 1). > > ---request begin--- > GET / HTTP/1.0 > User-Agent: Wget/1.12 (linux-gnu) > Accept: */* > Host: gerrit.ovirt.org > Connection: Keep-Alive > > ---request end--- > HTTP request sent, awaiting response... Read error (Connection > timed out) in headers. > Closed fd 3 > Giving up. > > Build step 'Execute shell' marked build as failure > IRC notifier plugin: Sending notification to: #ovirt-jenkins > Email was triggered for: Failure > Sending email for trigger: Failure >