Re: [ovirt-devel] Check-merged is broken since we added network test to it

בתאריך 23 בדצמ׳ 2016 12:46 PM,‏ "Dan Kenigsberg" <danken(a)redhat.com&gt; כתב: On Thu, Dec 22, 2016 at 5:45 PM, Yaniv Bronheim <ybronhei(a)redhat.com&gt; wrote: wrote: "post hoc ergo propter hoc" is a fallacy. We had a horrible week CI-wise; check-merged keept failing due to "Groovy thread" exception which is an internal Jenkins thingy. Please give the CI team the credit of being able to eliminate the Jenkins and environment related issues. We have runs that pre-date this week's issues and still show that behaviour. for example: http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/692/console The fact that the job crashes during the groovy code is not an indicator that the issue is in that code. That exact same code runs for every single job in the system without failing, so it means that something in this job probably creates conditions that prevent it from running. Actually we know more then that. Looking carefully at the job logs you can see the groovy fails to run because the slave disconnects just before Jenkins tries to run it (By now you should be familiar with OVIRT-938). Again, all the CI code that runs before, after and during this job is also used in other jobs that do not fail in the same manner - With this I want to Yaniv to ask him to help figuring out if the check_merged.sh code may be doing anything that may cause this. Our conversation led to this email. Trying to wave this this as a "Groovy failure" at this point is not helping. The CI team is not going to come up with a magic solution to this one without you guys` help. Now that's over, but a recent test failed on http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/778/console 11:55:54 RuntimeError: Failed to run reposync 3 times for repoid: ovirt-master-snapshot-static-el7, aborting. It is wrong to pick up on the first patch that happened to see the "Groovy thread" exception.