i think this happens when gerrit server is too overloaded, and can't re= turn an answer for more than 10min (default value). latest git plugin allows us to set a different value than 10min.
for 1st action, i'd change those failing jobs to use maybe 20min instea= d, and see if it's fixed. can you provide list of links for the failing jobs you saw?
for longer run, i'd think on how to reduce/manage load on gerrit server= , maybe using zuul. [1] david - any thoughts?
[1] http://ci.openstack.org/zuul/
----- Original Message -----
From: "Michal Skrivanek" <michal.skrivanek@redhat.com> To: infra@ovirt.org Sent: Thursday, June 5, 2014 5:36:59 PM Subject: Jenkins issue (findbugs)
----- Forwarded Message ----- From: "Frantisek Kobzik" <fkobzik@redhat.com> To: infra@ovirt.org Sent: Thursday, June 5, 2014 11:23:12 AM Subject: Jenkins issue (findbugs)
Hi guys,
I have an issue with jenkins build and I was advised to contact infra= =2E The console output is below (Similar issue also happens with other ki= nds of jobs).
I'm seeing this issue quite often (~50 % of pushes), is there anythin= g you can do about it?
Thanks, Franta.
9:47:36 ERROR: Timeout after 10 minutes 09:47:36 FATAL: Failed to fetch from git://gerrit.ovirt.org/ovirt-eng= ine 09:47:37 hudson.plugins.git.GitException: Failed to fetch from git://gerrit.ovirt.org/ovirt-engine 09:47:37 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:623) 09:47:37 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:85=
09:47:37 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:880) 09:47:37 at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:11=
09:47:37 at hudson.model.AbstractProject.checkout(AbstractProject.java:1320) 09:47:37 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(Abs=
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --57GaMnHJUE8iAksGB9GERqTGcdbIQoH4D Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thu 05 Jun 2014 05:10:48 PM CEST, Eyal Edri wrote: 5) 8) tractBuild.java:609)
09:47:37 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)=
09:47:37 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.j= ava:518) 09:47:37 at hudson.model.Run.execute(Run.java:1700) 09:47:37 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 09:47:37 at hudson.model.ResourceController.execute(ResourceController.java:88) 09:47:37 at hudson.model.Executor.run(Executor.java:231) 09:47:37 Caused by: hudson.plugins.git.GitException: Command "git fet= ch --tags --progress git://gerrit.ovirt.org/ovirt-engine refs/changes/14/28314/3 --prune" returned status code 143: 09:47:37 stdout: 9:47:37 stderr: remote: Counting objects: 136 remote: Counting objects: 375 remote: Counting objects: 582 . etc. . 09:47:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitA= PIImpl.java:1276) 09:47:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCreden= tials(CliGitAPIImpl.java:1146) 09:47:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImp= l.java:87) 09:47:37 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl= =2Ejava:254) 09:47:37 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandle= r$1.call(RemoteGitImpl.java:153) 09:47:37 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandle= r$1.call(RemoteGitImpl.java:146) 09:47:37 at hudson.remoting.UserRequest.perform(UserRequest.java:118= ) 09:47:37 at hudson.remoting.UserRequest.perform(UserRequest.java:48)=
09:47:37 at hudson.remoting.Request$2.run(Request.java:328) 09:47:37 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecut= orService.java:72) 09:47:37 at java.util.concurrent.FutureTask.run(FutureTask.java:262)=
09:47:37 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.= java:1145) 09:47:37 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor= =2Ejava:615) 09:47:37 at java.lang.Thread.run(Thread.java:744) .
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Maybe we can start investigating on create mirrors for gerrit to pull=20 from the slaves (also downstream if able) at least for read operations,=20 that will leverage a lot of load from the jenkins slaves (if not all of=20 it) -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --57GaMnHJUE8iAksGB9GERqTGcdbIQoH4D Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTkIqsAAoJEEBxx+HSYmnD53wH/jdr2N5aY3R0ESVaVp80IyBb 0g/jlvBY3BoKe1fdpWh0PCXRkI/uh3sUGO0NmVD1KHJgXd8+00RlZYnh/HIQrdTf sQzkIs5J54WQusvVA1CXeMJdWEPhDWv4+haeS+DtsITj+5VI86q8WP9siYpRy26j v3WmHeEBBxBCmHZNtKi0Nu0GG7/5IcBtFneyxpDBCvrs9hbe+BS14ScczaCAC1nq NVEUrC5h7tLPDxblXzu7hbrbUxFYj341D9jMrgQgYFIpnYB+WVn8XKB6jDK0GoV1 y0OAO1pSRvAli2BS/zxkSm2LemktmVh8k6o1odKb5katzq67ZhCtthPBgqAL3zg= =GhCF -----END PGP SIGNATURE----- --57GaMnHJUE8iAksGB9GERqTGcdbIQoH4D--