[VDSM] Tests failing because of ordering dependencies
Hi all, Trying to run vdsm tests via tox (so correct nose is used automatically), some of the tests fail. The failure are all about ordering expectations, which look wrong. Please check and fix your tests. Thanks, Nir ---- 18:04:10 ====================================================================== 18:04:10 FAIL: test_parseVolumeStatus (gluster_cli_tests.GlusterCliTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/gluster_cli_tests.py", line 1121, in test_parseVolumeStatus 18:04:10 self._parseVolumeStatusClients_test() 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/gluster_cli_tests.py", line 449, in _parseVolumeStatusClients_test 18:04:10 self.assertEquals(status.keys(), ['bricks', 'name']) status.keys() order is not undefined. 18:04:10 AssertionError: Lists differ: ['name', 'bricks'] != ['bricks', 'name'] 18:04:10 18:04:10 First differing element 0: 18:04:10 name 18:04:10 bricks 18:04:10 18:04:10 - ['name', 'bricks'] 18:04:10 + ['bricks', 'name'] 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testSetPolicyParameters (momTests.MomPolicyTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/momTests.py", line 118, in testSetPolicyParameters 18:04:10 self.assertEqual(api.last_policy_content, expected) 18:04:10 AssertionError: "(set a 5)\n(set b True)\n(set c 'test')" != "(set a 5)\n(set c 'test')\n(set b True)" Nothing obvious, need deeper checking. 18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:01:56,902 INFO (MainThread) [MOM] Preparing MOM interface (momIF:49) 18:04:10 2016-10-12 18:01:56,903 INFO (MainThread) [MOM] Using named unix socket /tmp/tmpqOQZvm/test_mom_vdsm.sock (momIF:58) 18:04:10 --------------------- >> end captured logging << --------------------- 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: test_disk_virtio_cache (vmStorageTests.DriveXMLTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmStorageTests.py", line 84, in test_disk_virtio_cache 18:04:10 self.check(vm_conf, conf, xml, is_block_device=False) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmStorageTests.py", line 222, in check 18:04:10 self.assertXMLEqual(drive.getXML().toxml(), xml) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <total_iops_sec>800</total_iops_sec> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk> 18:04:10 18:04:10 Expected: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 <total_iops_sec>800</total_iops_sec> Order of these elements differ, need to check why. 18:04:10 </iotune> 18:04:10 </disk> 18:04:10 18:04:10 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testCpuXML (vmTests.TestVm) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmTests.py", line 434, in testCpuXML 18:04:10 self.assertXMLEqual(find_xml_element(xml, "./cputune"), cputuneXML) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" /> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 </cputune> 18:04:10 18:04:10 Expected: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" /> Same 18:04:10 </cputune> 18:04:10 18:04:10 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testSetIoTune (vmTests.TestVm) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmTests.py", line 936, in testSetIoTune 18:04:10 self._xml_sanitizer(expected_xml)) 18:04:10 AssertionError: '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><write_bytes_sec>1</write_bytes_sec><total_bytes_sec>0</total_bytes_sec><read_bytes_sec>2</read_bytes_sec></iotune></disk>' != '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><read_bytes_sec>2</read_bytes_sec><write_bytes_sec>1</write_bytes_sec><total_bytes_sec>0</total_bytes_sec></iotune></disk>' 18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:03:34,110 INFO (MainThread) [virt.vm] vmId=`TESTING`::New device XML for hda: <disk device="hdd" snapshot="no" type="block"> 18:04:10 <source dev="/dev/dummy"/> 18:04:10 <target bus="ide" dev="hda"/> 18:04:10 <iotune> 18:04:10 <write_bytes_sec>1</write_bytes_sec> 18:04:10 <total_bytes_sec>0</total_bytes_sec> 18:04:10 <read_bytes_sec>2</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk> Seems that this test is not using assertXMLEqual so we don't get meaningful error like the tests above. 18:04:10 (vm:2687) 18:04:10 --------------------- >> end captured logging << ---------------------
If you want to reproduce the errors, try this patch: https://gerrit.ovirt.org/65404 On Wed, Oct 12, 2016 at 9:14 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
Trying to run vdsm tests via tox (so correct nose is used automatically), some of the tests fail.
The failure are all about ordering expectations, which look wrong.
Please check and fix your tests.
Thanks, Nir
----
18:04:10 ====================================================================== 18:04:10 FAIL: test_parseVolumeStatus (gluster_cli_tests.GlusterCliTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/gluster_cli_tests.py", line 1121, in test_parseVolumeStatus 18:04:10 self._parseVolumeStatusClients_test() 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/gluster_cli_tests.py", line 449, in _parseVolumeStatusClients_test 18:04:10 self.assertEquals(status.keys(), ['bricks', 'name'])
status.keys() order is not undefined.
18:04:10 AssertionError: Lists differ: ['name', 'bricks'] != ['bricks', 'name'] 18:04:10 18:04:10 First differing element 0: 18:04:10 name 18:04:10 bricks 18:04:10 18:04:10 - ['name', 'bricks'] 18:04:10 + ['bricks', 'name'] 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testSetPolicyParameters (momTests.MomPolicyTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/momTests.py", line 118, in testSetPolicyParameters 18:04:10 self.assertEqual(api.last_policy_content, expected) 18:04:10 AssertionError: "(set a 5)\n(set b True)\n(set c 'test')" != "(set a 5)\n(set c 'test')\n(set b True)"
Nothing obvious, need deeper checking.
18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:01:56,902 INFO (MainThread) [MOM] Preparing MOM interface (momIF:49) 18:04:10 2016-10-12 18:01:56,903 INFO (MainThread) [MOM] Using named unix socket /tmp/tmpqOQZvm/test_mom_vdsm.sock (momIF:58) 18:04:10 --------------------- >> end captured logging << --------------------- 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: test_disk_virtio_cache (vmStorageTests.DriveXMLTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmStorageTests.py", line 84, in test_disk_virtio_cache 18:04:10 self.check(vm_conf, conf, xml, is_block_device=False) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmStorageTests.py", line 222, in check 18:04:10 self.assertXMLEqual(drive.getXML().toxml(), xml) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <total_iops_sec>800</total_iops_sec> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk> 18:04:10 18:04:10 Expected: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 <total_iops_sec>800</total_iops_sec>
Order of these elements differ, need to check why.
18:04:10 </iotune> 18:04:10 </disk> 18:04:10 18:04:10 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testCpuXML (vmTests.TestVm) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmTests.py", line 434, in testCpuXML 18:04:10 self.assertXMLEqual(find_xml_element(xml, "./cputune"), cputuneXML) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" /> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 </cputune> 18:04:10 18:04:10 Expected: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" />
Same
18:04:10 </cputune> 18:04:10 18:04:10 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testSetIoTune (vmTests.TestVm) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmTests.py", line 936, in testSetIoTune 18:04:10 self._xml_sanitizer(expected_xml)) 18:04:10 AssertionError: '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><write_bytes_sec>1</write_bytes_sec><total_bytes_sec>0</total_bytes_sec><read_bytes_sec>2</read_bytes_sec></iotune></disk>' != '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><read_bytes_sec>2</read_bytes_sec><write_bytes_sec>1</write_bytes_sec><total_bytes_sec>0</total_bytes_sec></iotune></disk>' 18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:03:34,110 INFO (MainThread) [virt.vm] vmId=`TESTING`::New device XML for hda: <disk device="hdd" snapshot="no" type="block"> 18:04:10 <source dev="/dev/dummy"/> 18:04:10 <target bus="ide" dev="hda"/> 18:04:10 <iotune> 18:04:10 <write_bytes_sec>1</write_bytes_sec> 18:04:10 <total_bytes_sec>0</total_bytes_sec> 18:04:10 <read_bytes_sec>2</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk>
Seems that this test is not using assertXMLEqual so we don't get meaningful error like the tests above.
18:04:10 (vm:2687) 18:04:10 --------------------- >> end captured logging << ---------------------
On Wed, Oct 12, 2016 at 09:16:08PM +0300, Nir Soffer wrote:
If you want to reproduce the errors, try this patch: https://gerrit.ovirt.org/65404
On Wed, Oct 12, 2016 at 9:14 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
Trying to run vdsm tests via tox (so correct nose is used automatically), some of the tests fail.
The failure are all about ordering expectations, which look wrong.
Please check and fix your tests.
Thanks, Nir
----
18:04:10 ====================================================================== 18:04:10 FAIL: test_parseVolumeStatus (gluster_cli_tests.GlusterCliTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/gluster_cli_tests.py", line 1121, in test_parseVolumeStatus 18:04:10 self._parseVolumeStatusClients_test() 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/gluster_cli_tests.py", line 449, in _parseVolumeStatusClients_test 18:04:10 self.assertEquals(status.keys(), ['bricks', 'name'])
status.keys() order is not undefined.
I've never seen this in Python 2, but since Python 3 uses a different hash function, this specific issue is handled in https://gerrit.ovirt.org/#/c/65008/
On Wed, Oct 12, 2016 at 9:14 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Trying to run vdsm tests via tox (so correct nose is used automatically), some of the tests fail.
The failure are all about ordering expectations, which look wrong.
tox -e tests tests create: /vdsm/.tox/tests tests installdeps: nose==1.3.7 tests installed: nose==1.3.7 tests runtests: PYTHONHASHSEED='1382456816' tox uses random seed on each run, exposing bad code assuming order of unordered things. To repeat tests failures, you probably want to use the same seed, this can be done like this: tox -e tests --hashseed SEED See https://docs.python.org/2/using/cmdline.html#envvar-PYTHONHASHSEED Nir
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "devel" <devel@ovirt.org>, "Adam Litke" <alitke@redhat.com>, "Dan Kenigsberg" <danken@redhat.com>, "Francesco Romani" <fromani@redhat.com>, "Piotr Kliczewski" <pkliczew@redhat.com>, "Martin Sivak" <msivak@redhat.com> Sent: Wednesday, October 12, 2016 8:14:11 PM Subject: [VDSM] Tests failing because of ordering dependencies
Hi all,
Trying to run vdsm tests via tox (so correct nose is used automatically), some of the tests fail.
The failure are all about ordering expectations, which look wrong.
Please check and fix your tests.
Thanks, Nir
18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:01:56,902 INFO (MainThread) [MOM] Preparing MOM interface (momIF:49) 18:04:10 2016-10-12 18:01:56,903 INFO (MainThread) [MOM] Using named unix socket /tmp/tmpqOQZvm/test_mom_vdsm.sock (momIF:58) 18:04:10 --------------------- >> end captured logging << --------------------- 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: test_disk_virtio_cache (vmStorageTests.DriveXMLTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmStorageTests.py", line 84, in test_disk_virtio_cache 18:04:10 self.check(vm_conf, conf, xml, is_block_device=False) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmStorageTests.py", line 222, in check 18:04:10 self.assertXMLEqual(drive.getXML().toxml(), xml) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <total_iops_sec>800</total_iops_sec> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk> 18:04:10 18:04:10 Expected: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 <total_iops_sec>800</total_iops_sec>
Order of these elements differ, need to check why.
Most likely because we build the xml like (vdsm/virt/vmdevices/storage.py) def _getIotuneXML(self): iotune = vmxml.Element('iotune') for key, value in self.specParams['ioTune'].iteritems(): iotune.appendChildWithArgs(key, text=str(value)) return iotune and iteritems() ordering is not defined
---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmTests.py", line 434, in testCpuXML 18:04:10 self.assertXMLEqual(find_xml_element(xml, "./cputune"), cputuneXML) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" /> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 </cputune> 18:04:10 18:04:10 Expected: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" />
Same
(vdsm/virt/vmxml.py) def appendCpus(self): # lot of code # CPU-pinning support # see http://www.ovirt.org/wiki/Features/Design/cpu-pinning if 'cpuPinning' in self.conf: cputune = Element('cputune') cpuPinning = self.conf.get('cpuPinning') for cpuPin in cpuPinning.keys(): cputune.appendChildWithArgs('vcpupin', vcpu=cpuPin, cpuset=cpuPinning[cpuPin]) self.dom.appendChild(cputune)
18:04:10 </cputune> 18:04:10 18:04:10 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testSetIoTune (vmTests.TestVm) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/vmTests.py", line 936, in testSetIoTune 18:04:10 self._xml_sanitizer(expected_xml)) 18:04:10 AssertionError: '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><write_bytes_sec>1</write_bytes_sec><total_bytes_sec>0</total_bytes_sec><read_bytes_sec>2</read_bytes_sec></iotune></disk>' != '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><read_bytes_sec>2</read_bytes_sec><write_bytes_sec>1</write_bytes_sec><total_bytes_sec>0</total_bytes_sec></iotune></disk>' 18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:03:34,110 INFO (MainThread) [virt.vm] vmId=`TESTING`::New device XML for hda: <disk device="hdd" snapshot="no" type="block"> 18:04:10 <source dev="/dev/dummy"/> 18:04:10 <target bus="ide" dev="hda"/> 18:04:10 <iotune> 18:04:10 <write_bytes_sec>1</write_bytes_sec> 18:04:10 <total_bytes_sec>0</total_bytes_sec> 18:04:10 <read_bytes_sec>2</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk>
Seems that this test is not using assertXMLEqual so we don't get meaningful error like the tests above.
Fixed the reporting: https://gerrit.ovirt.org/65417 For the sloppy virt tests, I think the right call is to fix the tests rather than the code. The code is not wrong, because the ordering of the attributes is not relevant. We "just" need tests which can interpret XML on a deeper level. -- Francesco Romani Red Hat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
For short lists using sorted output will make the code easier to debug and test. בתאריך 13 באוק׳ 2016 11:03 לפנה״צ, "Francesco Romani" <fromani@redhat.com> כתב:
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "devel" <devel@ovirt.org>, "Adam Litke" <alitke@redhat.com>, "Dan Kenigsberg" <danken@redhat.com>, "Francesco Romani" <fromani@redhat.com>, "Piotr Kliczewski" <pkliczew@redhat.com>, "Martin Sivak" <msivak@redhat.com> Sent: Wednesday, October 12, 2016 8:14:11 PM Subject: [VDSM] Tests failing because of ordering dependencies
Hi all,
Trying to run vdsm tests via tox (so correct nose is used automatically), some of the tests fail.
The failure are all about ordering expectations, which look wrong.
Please check and fix your tests.
Thanks, Nir
18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:01:56,902 INFO (MainThread) [MOM] Preparing MOM interface (momIF:49) 18:04:10 2016-10-12 18:01:56,903 INFO (MainThread) [MOM] Using named unix socket /tmp/tmpqOQZvm/test_mom_vdsm.sock (momIF:58) 18:04:10 --------------------- >> end captured logging << --------------------- 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: test_disk_virtio_cache (vmStorageTests.DriveXMLTests) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/ vdsm/tests/vmStorageTests.py", line 84, in test_disk_virtio_cache 18:04:10 self.check(vm_conf, conf, xml, is_block_device=False) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/ vdsm/tests/vmStorageTests.py", line 222, in check 18:04:10 self.assertXMLEqual(drive.getXML().toxml(), xml) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/ vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <total_iops_sec>800</total_iops_sec> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk> 18:04:10 18:04:10 Expected: 18:04:10 <disk device="disk" snapshot="no" type="file"> 18:04:10 <source file="/path/to/volume" /> 18:04:10 <target bus="virtio" dev="vda" /> 18:04:10 <shareable /> 18:04:10 <serial>54-a672-23e5b495a9ea</serial> 18:04:10 <driver cache="writethrough" error_policy="enospace" io="threads" name="qemu" type="qcow2" /> 18:04:10 <iotune> 18:04:10 <read_bytes_sec>6120000</read_bytes_sec> 18:04:10 <total_iops_sec>800</total_iops_sec>
Order of these elements differ, need to check why.
Most likely because we build the xml like
(vdsm/virt/vmdevices/storage.py)
def _getIotuneXML(self): iotune = vmxml.Element('iotune') for key, value in self.specParams['ioTune'].iteritems(): iotune.appendChildWithArgs(key, text=str(value)) return iotune
and iteritems() ordering is not defined
---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/ vdsm/tests/vmTests.py", line 434, in testCpuXML 18:04:10 self.assertXMLEqual(find_xml_element(xml, "./cputune"), cputuneXML) 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/ vdsm/tests/testlib.py", line 253, in assertXMLEqual 18:04:10 (actualXML, expectedXML)) 18:04:10 AssertionError: XMLs are different: 18:04:10 Actual: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" /> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 </cputune> 18:04:10 18:04:10 Expected: 18:04:10 <cputune> 18:04:10 <vcpupin cpuset="2-3" vcpu="1" /> 18:04:10 <vcpupin cpuset="0-1" vcpu="0" />
Same
(vdsm/virt/vmxml.py)
def appendCpus(self): # lot of code
# CPU-pinning support # see http://www.ovirt.org/wiki/Features/Design/cpu-pinning if 'cpuPinning' in self.conf: cputune = Element('cputune') cpuPinning = self.conf.get('cpuPinning') for cpuPin in cpuPinning.keys(): cputune.appendChildWithArgs('vcpupin', vcpu=cpuPin, cpuset=cpuPinning[cpuPin]) self.dom.appendChild(cputune)
18:04:10 </cputune> 18:04:10 18:04:10 18:04:10 18:04:10 ====================================================================== 18:04:10 FAIL: testSetIoTune (vmTests.TestVm) 18:04:10 ---------------------------------------------------------------------- 18:04:10 Traceback (most recent call last): 18:04:10 File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/ vdsm/tests/vmTests.py", line 936, in testSetIoTune 18:04:10 self._xml_sanitizer(expected_xml)) 18:04:10 AssertionError: '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><write_bytes_sec>1</write_bytes_sec>< total_bytes_sec>0</total_bytes_sec><read_bytes_sec>2</ read_bytes_sec></iotune></disk>' != '<disk device="hdd" snapshot="no" type="block"><source dev="/dev/dummy"/><target bus="ide" dev="hda"/><iotune><read_bytes_sec>2</read_bytes_sec>< write_bytes_sec>1</write_bytes_sec><total_bytes_sec>0</ total_bytes_sec></iotune></disk>' 18:04:10 -------------------- >> begin captured logging << -------------------- 18:04:10 2016-10-12 18:03:34,110 INFO (MainThread) [virt.vm] vmId=`TESTING`::New device XML for hda: <disk device="hdd" snapshot="no" type="block"> 18:04:10 <source dev="/dev/dummy"/> 18:04:10 <target bus="ide" dev="hda"/> 18:04:10 <iotune> 18:04:10 <write_bytes_sec>1</write_bytes_sec> 18:04:10 <total_bytes_sec>0</total_bytes_sec> 18:04:10 <read_bytes_sec>2</read_bytes_sec> 18:04:10 </iotune> 18:04:10 </disk>
Seems that this test is not using assertXMLEqual so we don't get meaningful error like the tests above.
Fixed the reporting: https://gerrit.ovirt.org/65417
For the sloppy virt tests, I think the right call is to fix the tests rather than the code. The code is not wrong, because the ordering of the attributes is not relevant. We "just" need tests which can interpret XML on a deeper level.
-- Francesco Romani Red Hat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
participants (3)
-
Dan Kenigsberg -
Francesco Romani -
Nir Soffer