<div dir="ltr">I think that the missing packages issue you've mentioned is not persistent and I don't see any more tests failing on it anymore, I assume it was a change in the repo or something similar. But the original error that I've reported about is persistent and it's failing OST since yesterday. We must find the root cause for those errors as every patch to ovirt-engine is based on some faulty commit and it won't be deployed to the tested repo. I'll mention though that tests that don't include new engine patches (for example tests that include only new VDSM) pass OST.</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Oct 18, 2017 at 3:54 PM, Allon Mureinik <span dir="ltr"><<a href="mailto:amureini@redhat.com" target="_blank">amureini@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace">These patches are part of the engine backend - the failure happens WAY beforehand, they don't look related as far as I can tell.</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">Subsequent failures of the same suite seem to have different errors, e.g.:</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace"><pre style="color:rgb(0,0,0)">Traceback (most recent call last):
File "/tmp/ovirt-y8uX2TlUiq/<wbr>pythonlib/otopi/context.py", line 133, in _executeMethod
method['method']()
File "/tmp/ovirt-y8uX2TlUiq/otopi-<wbr>plugins/otopi/packagers/<wbr>yumpackager.py", line 256, in _packages
if self._miniyum.<wbr>buildTransaction():
File "/tmp/ovirt-y8uX2TlUiq/<wbr>pythonlib/otopi/miniyum.py", line 920, in buildTransaction
raise yum.Errors.YumBaseError(msg)
YumBaseError: [u'vdsm-4.20.3-202.git6826cec.<wbr>el7.centos.x86_64 requires libvirt-daemon-kvm >= 3.2.0-14.el7_4.3', u'10:qemu-kvm-ev-2.9.0-16.el7_<wbr>4.5.1.x86_64 requires ipxe-roms-qemu >= 20170123-1.git4e85b27.el7_4.1'<wbr>]</pre><pre style="color:rgb(0,0,0)"><br></pre><pre style="color:rgb(0,0,0)">Has something changed in the way OST sets up repos/hosts?</pre></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Oct 18, 2017 at 2:40 PM, Daniel Belenky <span dir="ltr"><<a href="mailto:dbelenky@redhat.com" target="_blank">dbelenky@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi all,<div><font face="arial, helvetica, sans-serif"><br></font></div><div><div><font face="arial, helvetica, sans-serif"><b>The following test is failing:</b> <span style="text-decoration-line:underline;color:rgb(51,51,51)">002_bootstrap.verify_<wbr>add_hosts</span></font></div><div><span style="color:rgb(51,51,51)"><font face="arial, helvetica, sans-serif"><b><a href="http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/3166/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/" target="_blank">All logs from failing job</a></b></font></span></div><div><font color="#333333" face="arial, helvetica, sans-serif"><b>Only 2 engine patches participated in the test, so the suspected patches are:</b></font></div><div><ol><li><a href="https://gerrit.ovirt.org/#/c/82542/2" target="_blank"><b style="font-family:arial,helvetica,sans-serif">https://gerrit.ovirt.org/#/c/8<wbr>2542/2</b><br></a></li><li><b style="font-family:arial,helvetica,sans-serif"><a href="https://gerrit.ovirt.org/#/c/82545/3" target="_blank">https://gerrit.ovirt.org/#/c/8<wbr>2545/3</a></b><br></li></ol><div>Due to the fact that when this error first introduced we had another error, the CI can't automatically detect the specific patch.</div></div><div><font face="arial, helvetica, sans-serif"><b><br></b></font></div><div><font face="arial, helvetica, sans-serif"><b>Error snippet from logs: </b></font><b style="color:rgb(0,0,0);white-space:pre-wrap"><a href="http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/3166/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/lago-basic-suite-master-engine/_var_log/ovirt-engine/host-deploy/ovirt-host-deploy-ansible-20171015165106-lago-basic-suite-master-host-0-74ed9407.log/*view*/" target="_blank">ovirt-host-deploy-ansibl<wbr>e log</a> (Full log)</b></div><div><pre style="color:rgb(0,0,0);word-wrap:break-word;white-space:pre-wrap">TASK [ovirt-host-deploy-firewalld : Enable firewalld rules] ********************
failed: [lago-basic-suite-master-host-<wbr>0] (item={u'service': u'glusterfs'}) => {"changed": false, "failed": true, "item": {"service": "glusterfs"}, "msg": "ERROR: Exception caught: org.fedoraproject.FirewallD1.E<wbr>xception: INVALID_SERVICE: 'glusterfs' not among existing services Permanent and Non-Permanent(immediate) operation, Services are defined by port/tcp relationship and named as they are in /etc/services (on most systems)"}</pre></div><div><br></div><div><b>Error from HOST 0 firewalld log: <a href="http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-queue-tester/3166/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/lago-basic-suite-master-host-0/_var_log/firewalld/*view*/" target="_blank">lago-basic-suite-master-h<wbr>ost-0/_var_log/firewalld/</a> (Ful<wbr>l log)</b></div><div><pre style="color:rgb(0,0,0);word-wrap:break-word;white-space:pre-wrap">2017-10-15 16:51:24 ERROR: INVALID_SERVICE: 'glusterfs' not among existing services</pre></div><span class="m_-1143403738827743543HOEnZb"><font color="#888888"><div>-- <br></div><div class="m_-1143403738827743543m_1301092226419385229gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span>DANIEL</span> <span>BELENKY</span></p><p style="margin:0px;padding:0px"></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><span>RHV DEVOPS</span></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><table style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium" border="0"><tbody><tr><td width="100px"><a href="https://red.ht/sig" target="_blank"><img width="90" height="auto"></a></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div>
</font></span></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span>DANIEL</span> <span>BELENKY</span></p><p style="margin:0px;padding:0px"></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><span>RHV DEVOPS</span></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><table style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium" border="0"><tbody><tr><td width="100px"><a href="https://red.ht/sig" target="_blank"><img src="https://www.redhat.com/files/brand/email/sig-redhat.png" width="90" height="auto"></a></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div>
</div>