This is a multi-part message in MIME format.
--------------080001040203070702070306
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
On 08/16/2012 06:03 AM, Itamar Heim wrote:
gerrit has been recycled again.
from reading about this, there are several possible issues causing this:
1. large patches submitted
2. large patches being viewed in side by side diff
all translate to memory issues or a bug in jetty causing the machine
to fill the log with:
[2012-08-16 03:45:24,764] WARN org.eclipse.jetty.util.log :
Dispatched Failed!
3. there are several postmaster processes occupying a lot of memory,
could be large patches also affect us with the new full patch in email
(gerrit 2.5 has a limitation on size of patches to be emailed iirc)
4. robert/eyal - can you please verify your versions are at least these:
"Just as a follow-up, we have a couple of Jenkins instances talking to
gerrit via the Gerrit Trigger plugin. It appears that after upgrading
each of the Jenkins instances and the trigger plugin that the issues
have cleared up. The versions we're now running are (in case it helps
anyone in the future):
* Gerrit 2.3
* Jenkins 1.461
1.466.1
<
http://jenkins-ci.org/>
* Jenkins GIT plugin 1.1.7
1.1.22
* Gerrit Trigger 2.5.1 "
2.5.2
from:
https://groups.google.com/forum/?fromgroups#!topic/repo-discuss/TVhBSBHEJ...
On 08/16/2012 12:54 PM, Jenkins oVirt Server wrote:
> Project:
http://jenkins.ovirt.org/job/check_gerrit_ovirt_org/
> Build:
http://jenkins.ovirt.org/job/check_gerrit_ovirt_org/20653/
> Build Number: 20653
> Build Status: Failure
> Triggered By: Started by timer
>
> -------------------------------------
> Changes Since Last Success:
> -------------------------------------
> Changes for Build #20653
>
>
>
> -----------------
> Failed Tests:
> -----------------
> No tests ran.
>
> ------------------
> Build Log:
> ------------------
> Started by timer
> Building remotely on rhel_62_slave_01 in workspace
> /home/jenkins/workspace/check_gerrit_ovirt_org
> No emails were triggered.
> [check_gerrit_ovirt_org] $ /bin/sh -xe /tmp/hudson1221775034538770259.sh
> + rm index.html
> + rm 'gitweb?p=test.git'
> + wget -v -d
gerrit.ovirt.org --timeout 60 -t 3
> Setting --timeout (timeout) to 60
> Setting --tries (tries) to 3
> DEBUG output created by Wget 1.12 on linux-gnu.
>
> --2012-08-16 05:51:04--
http://gerrit.ovirt.org/
> Resolving gerrit.ovirt.org... 107.22.212.69
> Caching
gerrit.ovirt.org => 107.22.212.69
> Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected.
> Created socket 3.
> Releasing 0x000000000105b860 (new refcount 1).
>
> ---request begin---
> GET / HTTP/1.0
> User-Agent: Wget/1.12 (linux-gnu)
> Accept: */*
> Host:
gerrit.ovirt.org
> Connection: Keep-Alive
>
> ---request end---
> HTTP request sent, awaiting response... Read error (Connection timed
> out) in headers.
> Closed fd 3
> Retrying.
>
> --2012-08-16 05:52:05-- (try: 2)
http://gerrit.ovirt.org/
> Found
gerrit.ovirt.org in host_name_addresses_map (0x105b860)
> Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected.
> Created socket 3.
> Releasing 0x000000000105b860 (new refcount 1).
>
> ---request begin---
> GET / HTTP/1.0
> User-Agent: Wget/1.12 (linux-gnu)
> Accept: */*
> Host:
gerrit.ovirt.org
> Connection: Keep-Alive
>
> ---request end---
> HTTP request sent, awaiting response... Read error (Connection timed
> out) in headers.
> Closed fd 3
> Retrying.
>
> --2012-08-16 05:53:07-- (try: 3)
http://gerrit.ovirt.org/
> Found
gerrit.ovirt.org in host_name_addresses_map (0x105b860)
> Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected.
> Created socket 3.
> Releasing 0x000000000105b860 (new refcount 1).
>
> ---request begin---
> GET / HTTP/1.0
> User-Agent: Wget/1.12 (linux-gnu)
> Accept: */*
> Host:
gerrit.ovirt.org
> Connection: Keep-Alive
>
> ---request end---
> HTTP request sent, awaiting response... Read error (Connection timed
> out) in headers.
> Closed fd 3
> Giving up.
>
> Build step 'Execute shell' marked build as failure
> IRC notifier plugin: Sending notification to: #ovirt-jenkins
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
--
Thanks
Robert Middleswarth
@rmiddle (twitter/IRC)
--------------080001040203070702070306
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 08/16/2012 06:03 AM, Itamar Heim
wrote:<br>
</div>
<blockquote cite="mid:502CC58C.3060207@redhat.com"
type="cite">gerrit
has been recycled again.
<br>
from reading about this, there are several possible issues causing
this:
<br>
1. large patches submitted
<br>
2. large patches being viewed in side by side diff
<br>
<br>
all translate to memory issues or a bug in jetty causing the
machine to fill the log with:
<br>
[2012-08-16 03:45:24,764] WARN org.eclipse.jetty.util.log :
Dispatched Failed!
<br>
<br>
3. there are several postmaster processes occupying a lot of
memory, could be large patches also affect us with the new full
patch in email (gerrit 2.5 has a limitation on size of patches to
be emailed iirc)
<br>
<br>
4. robert/eyal - can you please verify your versions are at least
these:
<br>
<br>
"Just as a follow-up, we have a couple of Jenkins instances
talking to
<br>
gerrit via the Gerrit Trigger plugin. It appears that after
upgrading
<br>
each of the Jenkins instances and the trigger plugin that the
issues
<br>
have cleared up. The versions we're now running are (in case it
helps
<br>
anyone in the future):
<br>
<br>
* Gerrit 2.3
<br>
</blockquote>
<br>
<blockquote cite="mid:502CC58C.3060207@redhat.com"
type="cite">*
Jenkins 1.461
<br>
</blockquote>
1.466.1<br>
<a
href="http://jenkins-ci.org/"></a>
<blockquote cite="mid:502CC58C.3060207@redhat.com"
type="cite">*
Jenkins GIT plugin 1.1.7
<br>
</blockquote>
1.1.22<br>
<blockquote cite="mid:502CC58C.3060207@redhat.com"
type="cite">*
Gerrit Trigger 2.5.1 "
<br>
</blockquote>
2.5.2<br>
<blockquote cite="mid:502CC58C.3060207@redhat.com"
type="cite">
<br>
from:
<br>
<a class="moz-txt-link-freetext"
href="https://groups.google.com/forum/?fromgroups#!topic/repo-discus...
<br>
<br>
<br>
On 08/16/2012 12:54 PM, Jenkins oVirt Server wrote:
<br>
<blockquote type="cite">Project:
<a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/check_gerrit_ovirt_org/">...
<br>
Build:
<a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/check_gerrit_ovirt_org/20653/&quo...
<br>
Build Number: 20653
<br>
Build Status: Failure
<br>
Triggered By: Started by timer
<br>
<br>
-------------------------------------
<br>
Changes Since Last Success:
<br>
-------------------------------------
<br>
Changes for Build #20653
<br>
<br>
<br>
<br>
-----------------
<br>
Failed Tests:
<br>
-----------------
<br>
No tests ran.
<br>
<br>
------------------
<br>
Build Log:
<br>
------------------
<br>
Started by timer
<br>
Building remotely on rhel_62_slave_01 in workspace
/home/jenkins/workspace/check_gerrit_ovirt_org
<br>
No emails were triggered.
<br>
[check_gerrit_ovirt_org] $ /bin/sh -xe
/tmp/hudson1221775034538770259.sh
<br>
+ rm index.html
<br>
+ rm 'gitweb?p=test.git'
<br>
+ wget -v -d
gerrit.ovirt.org --timeout 60 -t 3
<br>
Setting --timeout (timeout) to 60
<br>
Setting --tries (tries) to 3
<br>
DEBUG output created by Wget 1.12 on linux-gnu.
<br>
<br>
--2012-08-16 05:51:04-- <a class="moz-txt-link-freetext"
href="http://gerrit.ovirt.org/">http://gerrit.ovirt.org/<...
<br>
Resolving gerrit.ovirt.org... 107.22.212.69
<br>
Caching
gerrit.ovirt.org => 107.22.212.69
<br>
Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected.
<br>
Created socket 3.
<br>
Releasing 0x000000000105b860 (new refcount 1).
<br>
<br>
---request begin---
<br>
GET / HTTP/1.0
<br>
User-Agent: Wget/1.12 (linux-gnu)
<br>
Accept: */*
<br>
Host:
gerrit.ovirt.org
<br>
Connection: Keep-Alive
<br>
<br>
---request end---
<br>
HTTP request sent, awaiting response... Read error (Connection
timed out) in headers.
<br>
Closed fd 3
<br>
Retrying.
<br>
<br>
--2012-08-16 05:52:05-- (try: 2) <a class="moz-txt-link-freetext"
href="http://gerrit.ovirt.org/">http://gerrit.ovirt.org/<...
<br>
Found
gerrit.ovirt.org in host_name_addresses_map (0x105b860)
<br>
Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected.
<br>
Created socket 3.
<br>
Releasing 0x000000000105b860 (new refcount 1).
<br>
<br>
---request begin---
<br>
GET / HTTP/1.0
<br>
User-Agent: Wget/1.12 (linux-gnu)
<br>
Accept: */*
<br>
Host:
gerrit.ovirt.org
<br>
Connection: Keep-Alive
<br>
<br>
---request end---
<br>
HTTP request sent, awaiting response... Read error (Connection
timed out) in headers.
<br>
Closed fd 3
<br>
Retrying.
<br>
<br>
--2012-08-16 05:53:07-- (try: 3) <a class="moz-txt-link-freetext"
href="http://gerrit.ovirt.org/">http://gerrit.ovirt.org/<...
<br>
Found
gerrit.ovirt.org in host_name_addresses_map (0x105b860)
<br>
Connecting to gerrit.ovirt.org|107.22.212.69|:80... connected.
<br>
Created socket 3.
<br>
Releasing 0x000000000105b860 (new refcount 1).
<br>
<br>
---request begin---
<br>
GET / HTTP/1.0
<br>
User-Agent: Wget/1.12 (linux-gnu)
<br>
Accept: */*
<br>
Host:
gerrit.ovirt.org
<br>
Connection: Keep-Alive
<br>
<br>
---request end---
<br>
HTTP request sent, awaiting response... Read error (Connection
timed out) in headers.
<br>
Closed fd 3
<br>
Giving up.
<br>
<br>
Build step 'Execute shell' marked build as failure
<br>
IRC notifier plugin: Sending notification to: #ovirt-jenkins
<br>
Email was triggered for: Failure
<br>
Sending email for trigger: Failure
<br>
<br>
</blockquote>
<br>
<br>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Thanks
Robert Middleswarth
@rmiddle (twitter/IRC)
</pre>
</body>
</html>
--------------080001040203070702070306--