Python segfault in while running vdsm unittests - first aid
Hi all, I can confirm what Francesco found - the failure is easily reproduced (100%) when running make check with the NOSE_WITH_XUNIT=1, and when using latest nose version from git. This issue make the failing CI job not only useless, but harmful, wasting everyone time. I suggest to remove the NOSE_WITH_XNUNIT flag until nose is fixed. To be more specific - in the shell script running the tests: 1. Disable xunit support # Nose 1.3.0 and later segatult with this flag #export NOSE_WITH_XUNIT=1 2. Remove the Publish JUNit test result report, looking for **/nosetests.xml Better have working tests with bad jenkins integration, then broken tests. I changed the configuration on jenkins and the builds are successful now :-) Nir
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "infra" <infra@ovirt.org> Cc: "Dan Kenigsberg" <danken@redhat.com>, "Eyal Edri" <eedri@redhat.com>, "Francesco Romani" <fromani@redhat.com>, "Allon Mureinik" <amureini@redhat.com> Sent: Monday, June 9, 2014 2:44:37 PM Subject: Python segfault in while running vdsm unittests - first aid
Hi all,
I can confirm what Francesco found - the failure is easily reproduced (100%) when running make check with the NOSE_WITH_XUNIT=1, and when using latest nose version from git.
This issue make the failing CI job not only useless, but harmful, wasting everyone time.
I suggest to remove the NOSE_WITH_XNUNIT flag until nose is fixed.
is there a bug open on nose, any newer version that has a fix for it?
To be more specific - in the shell script running the tests:
1. Disable xunit support
# Nose 1.3.0 and later segatult with this flag #export NOSE_WITH_XUNIT=1
2. Remove the Publish JUNit test result report, looking for **/nosetests.xml
Better have working tests with bad jenkins integration, then broken tests.
I changed the configuration on jenkins and the builds are successful now :-)
thanks!
Nir
----- Original Message ---
From: "Eyal Edri" <eedri@redhat.com> To: "Nir Soffer" <nsoffer@redhat.com> Cc: "infra" <infra@ovirt.org>, "Dan Kenigsberg" <danken@redhat.com>, "Francesco Romani" <fromani@redhat.com>, "Allon Mureinik" <amureini@redhat.com> Sent: Monday, June 9, 2014 1:54:24 PM Subject: Re: Python segfault in while running vdsm unittests - first aid
I can confirm what Francesco found - the failure is easily reproduced (100%) when running make check with the NOSE_WITH_XUNIT=1, and when using latest nose version from git.
This issue make the failing CI job not only useless, but harmful, wasting everyone time.
I suggest to remove the NOSE_WITH_XNUNIT flag until nose is fixed.
is there a bug open on nose, any newer version that has a fix for it?
Here is the bug: https://github.com/nose-devs/nose/issues/817 No further activitity (neither on the bug nor on the repo) since my report -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
----- Original Message -----
From: "Francesco Romani" <fromani@redhat.com> To: "Eyal Edri" <eedri@redhat.com> Cc: "Nir Soffer" <nsoffer@redhat.com>, "infra" <infra@ovirt.org>, "Dan Kenigsberg" <danken@redhat.com>, "Allon Mureinik" <amureini@redhat.com> Sent: Monday, June 9, 2014 3:03:13 PM Subject: Re: Python segfault in while running vdsm unittests - first aid
----- Original Message ---
From: "Eyal Edri" <eedri@redhat.com> To: "Nir Soffer" <nsoffer@redhat.com> Cc: "infra" <infra@ovirt.org>, "Dan Kenigsberg" <danken@redhat.com>, "Francesco Romani" <fromani@redhat.com>, "Allon Mureinik" <amureini@redhat.com> Sent: Monday, June 9, 2014 1:54:24 PM Subject: Re: Python segfault in while running vdsm unittests - first aid
I can confirm what Francesco found - the failure is easily reproduced (100%) when running make check with the NOSE_WITH_XUNIT=1, and when using latest nose version from git.
This issue make the failing CI job not only useless, but harmful, wasting everyone time.
I suggest to remove the NOSE_WITH_XNUNIT flag until nose is fixed.
is there a bug open on nose, any newer version that has a fix for it?
Here is the bug: https://github.com/nose-devs/nose/issues/817
No further activitity (neither on the bug nor on the repo) since my report
But I think this is a bug in python, either in the core or in the xml module used by nose. nose probably only trigger the issue in the underlying xml code written in c.
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: "Eyal Edri" <eedri@redhat.com>, "infra" <infra@ovirt.org>, "Dan Kenigsberg" <danken@redhat.com>, "Allon Mureinik" <amureini@redhat.com> Sent: Monday, June 9, 2014 2:11:33 PM Subject: Re: Python segfault in while running vdsm unittests - first aid
[...]
is there a bug open on nose, any newer version that has a fix for it?
Here is the bug: https://github.com/nose-devs/nose/issues/817
No further activitity (neither on the bug nor on the repo) since my report
But I think this is a bug in python, either in the core or in the xml module used by nose. nose probably only trigger the issue in the underlying xml code written in c.
I fully agree with you that the core issue is probably elsewhere, most likely where you are suggesting. Unfortunately I don't know yet where exactly, so as starting point I notified the nose devs. Also, I wasn't yet able to pinpoint the segfault. So far AFAIU looks like a memory corruption which later let the VM crash badly. -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
On 06/09/2014 03:22 PM, Francesco Romani wrote:
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: "Eyal Edri" <eedri@redhat.com>, "infra" <infra@ovirt.org>, "Dan Kenigsberg" <danken@redhat.com>, "Allon Mureinik" <amureini@redhat.com> Sent: Monday, June 9, 2014 2:11:33 PM Subject: Re: Python segfault in while running vdsm unittests - first aid
[...]
is there a bug open on nose, any newer version that has a fix for it?
Here is the bug: https://github.com/nose-devs/nose/issues/817
No further activitity (neither on the bug nor on the repo) since my report
But I think this is a bug in python, either in the core or in the xml module used by nose. nose probably only trigger the issue in the underlying xml code written in c.
I fully agree with you that the core issue is probably elsewhere, most likely where you are suggesting. Unfortunately I don't know yet where exactly, so as starting point I notified the nose devs.
Also, I wasn't yet able to pinpoint the segfault. So far AFAIU looks like a memory corruption which later let the VM crash badly.
do you get the segfault that was introduced since the toolTests.py that way?? i don't get segfault on latest commit -- Yaniv Bronhaim.
----- Original Message -----
From: "ybronhei" <ybronhei@redhat.com> To: "Francesco Romani" <fromani@redhat.com>, "Nir Soffer" <nsoffer@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Monday, June 9, 2014 10:51:56 PM Subject: Re: Python segfault in while running vdsm unittests - first aid
On 06/09/2014 03:22 PM, Francesco Romani wrote:
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: "Eyal Edri" <eedri@redhat.com>, "infra" <infra@ovirt.org>, "Dan Kenigsberg" <danken@redhat.com>, "Allon Mureinik" <amureini@redhat.com> Sent: Monday, June 9, 2014 2:11:33 PM Subject: Re: Python segfault in while running vdsm unittests - first aid
[...]
is there a bug open on nose, any newer version that has a fix for it?
Here is the bug: https://github.com/nose-devs/nose/issues/817
No further activitity (neither on the bug nor on the repo) since my report
But I think this is a bug in python, either in the core or in the xml module used by nose. nose probably only trigger the issue in the underlying xml code written in c.
I fully agree with you that the core issue is probably elsewhere, most likely where you are suggesting. Unfortunately I don't know yet where exactly, so as starting point I notified the nose devs.
Also, I wasn't yet able to pinpoint the segfault. So far AFAIU looks like a memory corruption which later let the VM crash badly.
do you get the segfault that was introduced since the toolTests.py that way?? i don't get segfault on latest commit
You don't get the segfault because I disabled the xunit nose plugin. See the first message in this thread. Nir
participants (4)
-
Eyal Edri -
Francesco Romani -
Nir Soffer -
ybronhei