--E13BgyNx05feLLmH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 02/10, Eyal Edri wrote:
it seems that that slave isn't responsive to ssh, so it might got some sort of an infra issue. =20 i think we should consider adding some sort of verification process for s= laves. something that will run nightly or before each job if it's fast enough. =20 we can think on checking - ping - ssh - git clone.. =20 david, what do you think? might reduce a lot of false positive failures.
Adding something like that requires tampering jenkins internals (or wrappin= g a job inside a job or similar), so it's not easy doing it per-job. Jenkins itself should be doing connectivity checks periodically and take out slaves of the pool if unreachable, unresponsive (if the ping is not fast enough) or the disk/swap is filling up. The slave log shows that it was connected after the job ran, I'll try to fi= gure out what happened before with it
=20 e. =20 ----- Original Message -----
From: "Yevgeny Zaspitsky" <yzaspits@redhat.com> To: "David Caro" <dcaroest@redhat.com>, "Eyal Edri" <eedri@redhat.com> Cc: infra@ovirt.org Sent: Tuesday, February 10, 2015 6:15:40 PM Subject: Re: Jenkins failures =20 Here is an example for a git failure on a Jenkins node: http://jenkins.ovirt.org/job/ovirt-engine_master_find-bugs_gerrit/26316= /console =20 On 05/02/15 16:07, David Caro wrote:
Also take into account that monday/tuesday we had a major outage on j= enkins and all the slaves behaved unreliably if working at all.
On 02/05, Eyal Edri wrote:
Hi,
we'd be more than happy to help and fix those issues. can you please provide links and info on specific failures so we can= debug them?
also, you're welcome also to open a ticket to our ticketing system [= 1] to track a specific item. keep in mind the infra team is limited in resources, so not all tick= ets might be solves quickly, especially if a major outage (like we had this week) is in progress.
[1] https://fedorahosted.org/ovirt/newticket
/e
----- Original Message -----
From: "Yevgeny Zaspitsky" <yzaspits@redhat.com> To: infra@ovirt.org Sent: Thursday, February 5, 2015 3:59:34 PM Subject: Jenkins failures
Hi All,
Lately I barely get any valuable input from the Jenkins CI builds o= n my patches. Throughout the last week most of the builds finished with different Jenkins failures. The reasons were:
* git failure * lack of permission to mkdir * failure to retrieve artifacts from the artifactory * unexpected shutdown
Such a high rate of failures makes the value of the builds very low= and causes me to spend my time on understanding whether it's my fault o= r not.
I'd be very thankful and happier if Jenkins reliability was improve= d.
Regards, Yevgeny
* English - detected * English * Hebrew * Russian
* English * Hebrew * Russian
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra =20 =20
--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --E13BgyNx05feLLmH Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJU2kvzAAoJEEBxx+HSYmnDP1oH/jJZCa33WhupSBNZiKrNWYdv NYCDNYySJhlRnYdLZPQPLHapUClJHiYlxGIkPzgxla1JMYZAcF5zmkvfmluBD4Eg QakhCWapn01W6BGp+h9o3mf5MkY5j6niF1Cjm5mBcBozS5tKviGq3C5YU4cTZosN hxB31ju6462UQkhxEMj1qsLlBs8N+AYdlbslKzofs/c6GXbyaiAQsISEIB5mXLg0 k0KomZibSbd8WtX1M73OLxH7craJk/B/zM1Xqg/4Qo/UnvVmCwt17ji6ER0aYkCd igFTljY1Hv/e/tmquWgTt35ShPiih/a0vB18s8bUZMMrP6bxX4Jw3sVNYRMXBRY= =rq3f -----END PGP SIGNATURE----- --E13BgyNx05feLLmH--