
The error in vdsm.log Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices dev_object.setup() File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/graphics.py", line 63, in setup net_api.create_libvirt_network(display_network, self.conf['vmId']) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in create_libvirt_network libvirt.create_network(netname, user_reference) File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 94, in create_network if not is_libvirt_network(netname): File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 159, in is_libvirt_network netname = LIBVIRT_NET_PREFIX + netname TypeError: cannot concatenate 'str' and 'NoneType' objects 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed (vm:659) The tests last passed on Mar 28. Did a recent patch break this? The full build logs at http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/52/artifact/export... thanks sahina

Job's still failing on master. Could this be related to network patches that got merged on Mar 28, for instance https://gerrit.ovirt.org/#/c/74390/ ? On Thu, Mar 30, 2017 at 11:41 AM, Sahina Bose <sabose@redhat.com> wrote:
The error in vdsm.log
Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices dev_object.setup() File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/graphics.py", line 63, in setup net_api.create_libvirt_network(display_network, self.conf['vmId']) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in create_libvirt_network libvirt.create_network(netname, user_reference) File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 94, in create_network if not is_libvirt_network(netname): File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 159, in is_libvirt_network netname = LIBVIRT_NET_PREFIX + netname TypeError: cannot concatenate 'str' and 'NoneType' objects 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed (vm:659)
The tests last passed on Mar 28. Did a recent patch break this?
The full build logs at http://jenkins.ovirt.org/job/ ovirt_master_hc-system-tests/52/artifact/exported-artifacts/test_logs/
thanks sahina

On Tue, Apr 4, 2017 at 9:28 AM, Sahina Bose <sabose@redhat.com> wrote:
Job's still failing on master. Could this be related to network patches that got merged on Mar 28, for instance https://gerrit.ovirt.org/#/c/74390/ ?
On Thu, Mar 30, 2017 at 11:41 AM, Sahina Bose <sabose@redhat.com> wrote:
The error in vdsm.log
Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices dev_object.setup() File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/graphics.py", line 63, in setup net_api.create_libvirt_network(display_network, self.conf['vmId']) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in create_libvirt_network libvirt.create_network(netname, user_reference) File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 94, in create_network if not is_libvirt_network(netname): File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 159, in is_libvirt_network netname = LIBVIRT_NET_PREFIX + netname TypeError: cannot concatenate 'str' and 'NoneType' objects 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed (vm:659)
The tests last passed on Mar 28. Did a recent patch break this?
The full build logs at http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/52/artifact/export...
thanks sahina
Sorry Sahina for having missed your email. Indeed, it seems that Eddy's topic branch of creating libvirt networks just-in-time causes the failure. Do note that the topic branch is quite long, and reverting parts of it might take a little while. Petr, can you make hc-system-tests green again? Regards, Dan.

Hello Sahina, I think I have a fix for that. Can I somehow trigger the test with VDSM refspec/custom RPMs? Thanks, Petr 2017-04-04 8:52 GMT+02:00 Dan Kenigsberg <danken@redhat.com>:
On Tue, Apr 4, 2017 at 9:28 AM, Sahina Bose <sabose@redhat.com> wrote:
Job's still failing on master. Could this be related to network patches that got merged on Mar 28, for instance https://gerrit.ovirt.org/#/c/74390/ ?
On Thu, Mar 30, 2017 at 11:41 AM, Sahina Bose <sabose@redhat.com> wrote:
The error in vdsm.log
Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices dev_object.setup() File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/graphics.py", line 63, in setup net_api.create_libvirt_network(display_network, self.conf['vmId']) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in create_libvirt_network libvirt.create_network(netname, user_reference) File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 94, in create_network if not is_libvirt_network(netname): File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 159, in is_libvirt_network netname = LIBVIRT_NET_PREFIX + netname TypeError: cannot concatenate 'str' and 'NoneType' objects 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed (vm:659)
The tests last passed on Mar 28. Did a recent patch break this?
The full build logs at http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/52/artifact/export...
thanks sahina
Sorry Sahina for having missed your email. Indeed, it seems that Eddy's topic branch of creating libvirt networks just-in-time causes the failure. Do note that the topic branch is quite long, and reverting parts of it might take a little while.
Petr, can you make hc-system-tests green again?
Regards, Dan.

I've merged the fix of https://gerrit.ovirt.org/#/c/75134/ With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why. Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/ ? On Tue, Apr 4, 2017 at 2:11 PM, Petr Horacek <phoracek@redhat.com> wrote:
Hello Sahina,
I think I have a fix for that. Can I somehow trigger the test with VDSM refspec/custom RPMs?
Thanks, Petr
2017-04-04 8:52 GMT+02:00 Dan Kenigsberg <danken@redhat.com>:
On Tue, Apr 4, 2017 at 9:28 AM, Sahina Bose <sabose@redhat.com> wrote:
Job's still failing on master. Could this be related to network patches that got merged on Mar 28, for instance https://gerrit.ovirt.org/#/c/74390/ ?
On Thu, Mar 30, 2017 at 11:41 AM, Sahina Bose <sabose@redhat.com> wrote:
The error in vdsm.log
Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices dev_object.setup() File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/graphics.py", line 63, in setup net_api.create_libvirt_network(display_network, self.conf['vmId']) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in create_libvirt_network libvirt.create_network(netname, user_reference) File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 94, in create_network if not is_libvirt_network(netname): File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py", line 159, in is_libvirt_network netname = LIBVIRT_NET_PREFIX + netname TypeError: cannot concatenate 'str' and 'NoneType' objects 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed (vm:659)
The tests last passed on Mar 28. Did a recent patch break this?
The full build logs at http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/52/artifact/export...
thanks sahina
Sorry Sahina for having missed your email. Indeed, it seems that Eddy's topic branch of creating libvirt networks just-in-time causes the failure. Do note that the topic branch is quite long, and reverting parts of it might take a little while.
Petr, can you make hc-system-tests green again?
Regards, Dan.

On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken@redhat.com> wrote:
I've merged the fix of https://gerrit.ovirt.org/#/c/75134/
With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why.
Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/
The gluster setup and HE install does take around 15-20 minutes. Looks like you aborted in between?
?
Hello Sahina,
I think I have a fix for that. Can I somehow trigger the test with VDSM refspec/custom RPMs?
Thanks, Petr
2017-04-04 8:52 GMT+02:00 Dan Kenigsberg <danken@redhat.com>:
On Tue, Apr 4, 2017 at 9:28 AM, Sahina Bose <sabose@redhat.com> wrote:
Job's still failing on master. Could this be related to network patches that got merged on Mar 28, for instance https://gerrit.ovirt.org/#/c/74390/ ?
On Thu, Mar 30, 2017 at 11:41 AM, Sahina Bose <sabose@redhat.com> wrote:
The error in vdsm.log
Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2016, in _setup_devices dev_object.setup() File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/
graphics.py",
line 63, in setup net_api.create_libvirt_network(display_network, self.conf['vmId']) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 90, in create_libvirt_network libvirt.create_network(netname, user_reference) File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py",
94, in create_network if not is_libvirt_network(netname): File "/usr/lib/python2.7/site-packages/vdsm/network/libvirt.py",
On Tue, Apr 4, 2017 at 2:11 PM, Petr Horacek <phoracek@redhat.com> wrote: line line
159, in is_libvirt_network netname = LIBVIRT_NET_PREFIX + netname TypeError: cannot concatenate 'str' and 'NoneType' objects 2017-03-29 22:58:39,559-0400 ERROR (vm/d71bdf4e) [virt.vm] (vmId='d71bdf4e-1eb3-4762-bd0e-05bb9f5e43ef') The vm start process failed (vm:659)
The tests last passed on Mar 28. Did a recent patch break this?
The full build logs at http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/ 52/artifact/exported-artifacts/test_logs/
thanks sahina
Sorry Sahina for having missed your email. Indeed, it seems that Eddy's topic branch of creating libvirt networks just-in-time causes the failure. Do note that the topic branch is quite long, and reverting parts of it might take a little while.
Petr, can you make hc-system-tests green again?
Regards, Dan.

On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken@redhat.com> wrote:
I've merged the fix of https://gerrit.ovirt.org/#/c/75134/
With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why.
Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/
The gluster setup and HE install does take around 15-20 minutes. Looks like you aborted in between?
What does the log say? http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console I aborted the job after 1 hour and 47 minutes.

On Thu, Apr 6, 2017 at 2:24 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken@redhat.com>
I've merged the fix of https://gerrit.ovirt.org/#/c/75134/
With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why.
Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/
The gluster setup and HE install does take around 15-20 minutes. Looks
wrote: like
you aborted in between?
What does the log say? http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console
I aborted the job after 1 hour and 47 minutes.
Sorry, missed that. I could not make out much from logs apart from the fact that it's stuck on starting gluster services. Since there are no gluster logs available from the host VMs, cannot dig further.

On Thu, Apr 6, 2017 at 3:02 PM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 2:24 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken@redhat.com>
I've merged the fix of https://gerrit.ovirt.org/#/c/75134/
With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why.
Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/
The gluster setup and HE install does take around 15-20 minutes. Looks
wrote: like
you aborted in between?
What does the log say? http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console
I aborted the job after 1 hour and 47 minutes.
Sorry, missed that. I could not make out much from logs apart from the fact that it's stuck on starting gluster services. Since there are no gluster logs available from the host VMs, cannot dig further.
Which logs are needed? I thought we collect everything from /var/log ? Y.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

On Thu, Apr 6, 2017 at 5:41 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Thu, Apr 6, 2017 at 3:02 PM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 2:24 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken@redhat.com>
I've merged the fix of https://gerrit.ovirt.org/#/c/75134/
With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why.
Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/p
arameters/
The gluster setup and HE install does take around 15-20 minutes. Looks
wrote: like
you aborted in between?
What does the log say? http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console
I aborted the job after 1 hour and 47 minutes.
Sorry, missed that. I could not make out much from logs apart from the fact that it's stuck on starting gluster services. Since there are no gluster logs available from the host VMs, cannot dig further.
Which logs are needed? I thought we collect everything from /var/log ?
I think logs are collected only for the tests? This is prior to running the tests in tests-scenario
Y.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

On Thu, Apr 6, 2017 at 3:02 PM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 2:24 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken@redhat.com> wrote:
I've merged the fix of https://gerrit.ovirt.org/#/c/75134/
With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why.
Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/
The gluster setup and HE install does take around 15-20 minutes. Looks like you aborted in between?
What does the log say? http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console
I aborted the job after 1 hour and 47 minutes.
Sorry, missed that. I could not make out much from logs apart from the fact that it's stuck on starting gluster services. Since there are no gluster logs available from the host VMs, cannot dig further.
Can you check by running the hc suite on-premise? I believe that we've solved the network-related bug, but I do not have a proof, as there's something else broken there.

I started the basic-hc test with latest VDSM and it passed OK: http://jenkins.ovirt.org/job/ovirt-system-tests_manual/205/ Are we good? 2017-04-06 14:21 GMT+02:00 Dan Kenigsberg <danken@redhat.com>:
On Thu, Apr 6, 2017 at 3:02 PM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 2:24 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Apr 6, 2017 at 11:31 AM, Sahina Bose <sabose@redhat.com> wrote:
On Thu, Apr 6, 2017 at 1:31 PM, Dan Kenigsberg <danken@redhat.com> wrote:
I've merged the fix of https://gerrit.ovirt.org/#/c/75134/
With it, the hc-basic suit no longer fail - it hangs for hours, and I don't know why.
Sahina, can you look at http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/parameters/
The gluster setup and HE install does take around 15-20 minutes. Looks like you aborted in between?
What does the log say? http://jenkins.ovirt.org/job/ovirt-system-tests_manual/199/console
I aborted the job after 1 hour and 47 minutes.
Sorry, missed that. I could not make out much from logs apart from the fact that it's stuck on starting gluster services. Since there are no gluster logs available from the host VMs, cannot dig further.
Can you check by running the hc suite on-premise?
I believe that we've solved the network-related bug, but I do not have a proof, as there's something else broken there.

On Thu, Apr 6, 2017 at 5:27 PM, Petr Horacek <phoracek@redhat.com> wrote:
I started the basic-hc test with latest VDSM and it passed OK:
http://jenkins.ovirt.org/job/ovirt-system-tests_manual/205/
Are we good?
"we" as in "vdsm networking" are good. "We" as devel@ovirt still have a riddle regarding why hc-basic tends to get stuck.
participants (4)
-
Dan Kenigsberg
-
Petr Horacek
-
Sahina Bose
-
Yaniv Kaul