The error in vdsm.log Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices dev_object.setup() File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/graphics.py", line 63, in setup net_api.create_libvirt_network(display_network, self.conf['vmId']) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in create_libvirt_network libvirt.create_network(netname, user_reference) File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 94, in create_network if not is_libvirt_network(netname): File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 159, in is_libvirt_network netname = LIBVIRT_NET_PREFIX + netname TypeError: cannot concatenate 'str' and 'NoneType' objects 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed (vm:659) The tests last passed on Mar 28. Did a recent patch break this? The full build logs at http://jenkins.ovirt.org/job/ ovirt_master_hc-system-tests/52/artifact/exported-artifacts/test_logs/ thanks sahina

On Tue, Apr 4, 2017 at 9:28 AM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > Job's still failing on master. > Could this be related to network patches that got merged on Mar 28, for > instance https://gerrit.ovirt.org/#/c/74390/ ? > > On Thu, Mar 30, 2017 at 11:41 AM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >> >> The error in vdsm.log >> >> Traceback (most recent call last): >> File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices >> dev_object.setup() >> File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/graphics.py", >> line 63, in setup >> net_api.create_libvirt_network(display_network, self.conf['vmId']) >> File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in >> create_libvirt_network >> libvirt.create_network(netname, user_reference) >> File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line >> 94, in create_network >> if not is_libvirt_network(netname): >> File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line >> 159, in is_libvirt_network >> netname = LIBVIRT_NET_PREFIX + netname >> TypeError: cannot concatenate 'str' and 'NoneType' objects >> 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] >> (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed >> (vm:659) >> >> The tests last passed on Mar 28. Did a recent patch break this? >> >> The full build logs at >> http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/52/artifact/exp... >> >> thanks >> sahina Sorry Sahina for having missed your email. Indeed, it seems that Eddy's topic branch of creating libvirt networks just-in-time causes the failure. Do note that the topic branch is quite long, and reverting parts of it might take a little while. Petr, can you make hc-system-tests green again? Regards, Dan.

I've merged the fix of https://gerrit.ovirt.org/#/c/75134/ With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why. Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/

? On Tue, Apr 4, 2017 at 2:11 PM, Petr Horacek <phoracek(a)redhat.com&gt; wrote: > Hello Sahina, > > I think I have a fix for that. Can I somehow trigger the test with > VDSM refspec/custom RPMs? > > Thanks, > Petr > > 2017-04-04 8:52 GMT+02:00 Dan Kenigsberg <danken(a)redhat.com&gt;: >> On Tue, Apr 4, 2017 at 9:28 AM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >>> Job's still failing on master. >>> Could this be related to network patches that got merged on Mar 28, for >>> instance https://gerrit.ovirt.org/#/c/74390/ ? >>> >>> On Thu, Mar 30, 2017 at 11:41 AM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >>>> >>>> The error in vdsm.log >>>> >>>> Traceback (most recent call last): >>>> File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices >>>> dev_object.setup() >>>> File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/ graphics.py", >>>> line 63, in setup >>>> net_api.create_libvirt_network(display_network, self.conf['vmId']) >>>> File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in >>>> create_libvirt_network >>>> libvirt.create_network(netname, user_reference) >>>> File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line >>>> 94, in create_network >>>> if not is_libvirt_network(netname): >>>> File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line >>>> 159, in is_libvirt_network >>>> netname = LIBVIRT_NET_PREFIX + netname >>>> TypeError: cannot concatenate 'str' and 'NoneType' objects >>>> 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] >>>> (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed >>>> (vm:659) >>>> >>>> The tests last passed on Mar 28. Did a recent patch break this? >>>> >>>> The full build logs at >>>> http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/ 52/artifact/exported-artifacts/test_logs/ >>>> >>>> thanks >>>> sahina >> >> Sorry Sahina for having missed your email. Indeed, it seems that >> Eddy's topic branch of creating libvirt networks just-in-time causes >> the failure. >> Do note that the topic branch is quite long, and reverting parts of it >> might take a little while. >> >> Petr, can you make hc-system-tests green again? >> >> Regards, >> Dan.

On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > > > On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken(a)redhat.com&gt; wrote: >> >> I've merged the fix of https://gerrit.ovirt.org/#/c/75134/ >> >> With it, the hc-basic suit no longer fail - it hangs for hours, and I >> don't know why. >> >> Sahina, can you look at >> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/ > > > The gluster setup and HE install does take around 15-20 minutes. Looks like > you aborted in between? What does the log say? http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console I aborted the job after 1 hour and 47 minutes.

On Thu, Apr 6, 2017 at 3:02 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > > > On Thu, Apr 6, 2017 at 2:24 PM, Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > >> On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >> > >> > >> > On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken(a)redhat.com&gt; >> wrote: >> >> >> >> I've merged the fix of https://gerrit.ovirt.org/#/c/75134/ >> >> >> >> With it, the hc-basic suit no longer fail - it hangs for hours, and I >> >> don't know why. >> >> >> >> Sahina, can you look at >> >> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/p >> arameters/ >> > >> > >> > The gluster setup and HE install does take around 15-20 minutes. Looks >> like >> > you aborted in between? >> >> >> What does the log say? >> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console >> >> I aborted the job after 1 hour and 47 minutes. >> > > Sorry, missed that. I could not make out much from logs apart from the > fact that it's stuck on starting gluster services. Since there are no > gluster logs available from the host VMs, cannot dig further. > Which logs are needed? I thought we collect everything from /var/log ?

Y. > > > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel >

On Thu, Apr 6, 2017 at 3:02 PM, Sahina Bose <sabose(a)redhat.com&gt; wrote: > > > On Thu, Apr 6, 2017 at 2:24 PM, Dan Kenigsberg <danken(a)redhat.com&gt; wrote: >> >> On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose(a)redhat.com&gt; wrote: >> > >> > >> > On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken(a)redhat.com&gt; >> > wrote: >> >> >> >> I've merged the fix of https://gerrit.ovirt.org/#/c/75134/ >> >> >> >> With it, the hc-basic suit no longer fail - it hangs for hours, and I >> >> don't know why. >> >> >> >> Sahina, can you look at >> >> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/ >> > >> > >> > The gluster setup and HE install does take around 15-20 minutes. Looks >> > like >> > you aborted in between? >> >> >> What does the log say? >> http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console >> >> I aborted the job after 1 hour and 47 minutes. > > > Sorry, missed that. I could not make out much from logs apart from the fact > that it's stuck on starting gluster services. Since there are no gluster > logs available from the host VMs, cannot dig further. Can you check by running the hc suite on-premise? I believe that we've solved the network-related bug, but I do not have a proof, as there's something else broken there.