<div dir="ltr">I believe that we always deployed the mom rpms from patches but we just didn't know about it until now when we have a real patch breaking VDSM.<div>I've sent [1] to split created & merged jobs for mom so we'll deploy only the merged ones, this should deploy only mom rpms created from merged commits. </div><div><br></div><div>I've deleted the bad mom from the repo and running tests again to see if it works.</div><div><br></div><div>[1] <a href="https://gerrit.ovirt.org/#/c/63152/1">https://gerrit.ovirt.org/#/c/63152/1</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 1, 2016 at 5:53 PM, Eyal Edri <span dir="ltr"><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I think I know what the problem is (not 100% sure).<div>I see the experimental job [1] is taking mom rpms from an unmerged patch with -1 [2], so if this patch has a bug then it deployed broken mom to the experimental repo and failing tests.</div><div>I'm trying to exclude the created jobs for mom out of this workflow.</div><div><br></div><div>[1] <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/1283/" target="_blank">http://jenkins.ovirt.org/<wbr>job/test-repo_ovirt_<wbr>experimental_master/1283/</a></div><div>[2] <a href="https://gerrit.ovirt.org/#/c/63130/" target="_blank">https://gerrit.ovirt.org/#<wbr>/c/63130/</a></div></div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">On Thu, Sep 1, 2016 at 3:32 PM, Barak Korren <span dir="ltr"><<a href="mailto:bkorren@redhat.com" target="_blank">bkorren@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi there,<br>
<br>
The oVirt "experimental" system tests form "master and 4.0 started<br>
failing today at approximately 9:20 ([1], [2]) at the AddHost stage.<br>
<br>
I suspect the failure is due to a mom build that was performed today<br>
at 9:05 ([3], [4]). This is because the following can be found in the<br>
VDSM logs ([5], [6]):<br>
<br>
MainThread::ERROR::2016-09-01 06:18:41,076::vdsm::151::vds::<wbr>(run)<br>
Exception raised<br>
Traceback (most recent call last):<br>
File "/usr/share/vdsm/vdsm", line 149, in run<br>
serve_clients(log)<br>
File "/usr/share/vdsm/vdsm", line 104, in serve_clients<br>
cif = clientIF.getInstance(irs, log, scheduler)<br>
File "/usr/share/vdsm/clientIF.py", line 204, in getInstance<br>
cls._instance = clientIF(irs, log, scheduler)<br>
File "/usr/share/vdsm/clientIF.py", line 105, in __init__<br>
self._prepareMOM()<br>
File "/usr/share/vdsm/clientIF.py", line 273, in _prepareMOM<br>
self.mom = MomClient(momconf)<br>
File "/usr/lib/python2.7/site-packa<wbr>ges/vdsm/momIF.py", line 46, in __init__<br>
raise MomNotAvailableError()<br>
MomNotAvailableError<br>
MainThread::INFO::2016-09-01 06:18:41,077::vdsm::153::vds::<wbr>(run)<br>
Stopping threads<br>
MainThread::INFO::2016-09-01 06:18:41,077::vdsm::164::vds::<wbr>(run) Exiting<br>
<br>
<br>
Please have a look.<br>
Thanks,<br>
Barak.<br>
<br>
<br>
[1]: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/1283/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/t<wbr>est-repo_ovirt_experimental_ma<wbr>ster/1283/</a><br>
[2]: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/1077/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/t<wbr>est-repo_ovirt_experimental_4.<wbr>0/1077/</a><br>
[3]: <a href="http://jenkins.ovirt.org/job/mom_master_create-rpms-el7-x86_64_created/15/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/m<wbr>om_master_create-rpms-el7-x86_<wbr>64_created/15/</a><br>
[4]: <a href="http://jenkins.ovirt.org/job/mom_4.0_create-rpms-el7-x86_64_created/11/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/m<wbr>om_4.0_create-rpms-el7-x86_64_<wbr>created/11/</a><br>
[5]: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/1283/artifact/exported-artifacts/basic_suite_master.sh-fc24/exported-artifacts/test_logs/basic_suite_master/post-002_bootstrap.py/lago_basic_suite_master_host0/_var_log_vdsm/vdsm.log" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/t<wbr>est-repo_ovirt_experimental_ma<wbr>ster/1283/artifact/exported-ar<wbr>tifacts/basic_suite_master.sh-<wbr>fc24/exported-artifacts/test_<wbr>logs/basic_suite_master/post-<wbr>002_bootstrap.py/lago_basic_<wbr>suite_master_host0/_var_log_<wbr>vdsm/vdsm.log</a><br>
[6]: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/1077/artifact/exported-artifacts/basic_suite_4.0.sh-fc23/exported-artifacts/test_logs/basic_suite_4.0/post-002_bootstrap.py/lago_basic_suite_4_0_host0/_var_log_vdsm/vdsm.log" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/t<wbr>est-repo_ovirt_experimental_4.<wbr>0/1077/artifact/exported-artif<wbr>acts/basic_suite_4.0.sh-fc23/<wbr>exported-artifacts/test_logs/<wbr>basic_suite_4.0/post-002_boots<wbr>trap.py/lago_basic_suite_4_0_<wbr>host0/_var_log_vdsm/vdsm.log</a><br>
<span><font color="#888888"><br>
--<br>
Barak Korren<br>
<a href="mailto:bkorren@redhat.com" target="_blank">bkorren@redhat.com</a><br>
RHEV-CI Team<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div></div></div><span class="HOEnZb"><font color="#888888">-- <br><div data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>Eyal Edri<br>Associate Manager</div><div>RHV DevOps<br>EMEA ENG Virtualization R&D<br>Red Hat Israel<br><br>phone: <a href="tel:%2B972-9-7692018" value="+97297692018" target="_blank">+972-9-7692018</a><br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>Eyal Edri<br>Associate Manager</div><div>RHV DevOps<br>EMEA ENG Virtualization R&D<br>Red Hat Israel<br><br>phone: +972-9-7692018<br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div>
</div>