Random jenkins failures

Nir Soffer nsoffer at redhat.com
Wed Jan 13 21:42:25 UTC 2016 

On Wed, Jan 13, 2016 at 3:37 PM, Eyal Edri <eedri at redhat.com> wrote:
> Looks like loop device issues.
> Nir - didn't you say you have a patch to fix this?

The patch was merge today:
https://gerrit.ovirt.org/51614/

It fixes incorrect umount in two tests, that may cause stale loop
devices effecting
other tests on same slave.


>
> In any case i think rebooting the slave fix this.
>
> E.
>
> On Wed, Jan 13, 2016 at 3:08 PM, Vinzenz Feenstra <vfeenstr at redhat.com>
> wrote:
>>
>>
>> On Jan 13, 2016, at 1:20 PM, Vinzenz Feenstra <vfeenstr at redhat.com> wrote:
>>
>> I have just submitted a set of 4 patches where 1 patch unit tests failed
>> with the pasted text below. Those patches are absolutely unrelated to those
>> failures.
>>
>>
>> Please check into those issues - Thanks
>>
>>
>> It happened again
>> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/2186/console
>>
>> 13:06:47
>> ======================================================================
>> 13:06:47 ERROR: testLoopMount (mountTests.MountTests)
>> 13:06:47
>> ----------------------------------------------------------------------
>> 13:06:47 Traceback (most recent call last):
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/mountTests.py",
>> line 128, in testLoopMount
>> 13:06:47     m.mount(mntOpts="loop")
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 225, in mount
>> 13:06:47     return self._runcmd(cmd, timeout)
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 241, in _runcmd
>> 13:06:47     raise MountError(rc, ";".join((out, err)))
>> 13:06:47 MountError: (32, ';mount: /tmp/tmpl2jG_h: failed to setup loop
>> device: No such file or directory\n')
>> 13:06:47 -------------------- >> begin captured logging <<
>> --------------------
>> 13:06:47 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /sbin/mkfs.ext2 -F /tmp/tmpl2jG_h (cwd None)
>> 13:06:47 Storage.Misc.excCmd: DEBUG: SUCCESS: <err> = 'mke2fs 1.42.13
>> (17-May-2015)\n'; <rc> = 0
>> 13:06:47 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /usr/bin/mount -o loop /tmp/tmpl2jG_h /var/tmp/tmpRslb5M (cwd None)
>> 13:06:47 --------------------- >> end captured logging <<
>> ---------------------
>> 13:06:47
>> 13:06:47
>> ======================================================================
>> 13:06:47 ERROR: testSymlinkMount (mountTests.MountTests)
>> 13:06:47
>> ----------------------------------------------------------------------
>> 13:06:47 Traceback (most recent call last):
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/mountTests.py",
>> line 150, in testSymlinkMount
>> 13:06:47     m.mount(mntOpts="loop")
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 225, in mount
>> 13:06:47     return self._runcmd(cmd, timeout)
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 241, in _runcmd
>> 13:06:47     raise MountError(rc, ";".join((out, err)))
>> 13:06:47 MountError: (32, ';mount: /var/tmp/tmpTeUZUl/backing.img: failed
>> to setup loop device: No such file or directory\n')
>> 13:06:47 -------------------- >> begin captured logging <<
>> --------------------
>> 13:06:47 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /sbin/mkfs.ext2 -F /var/tmp/tmpTeUZUl/backing.img (cwd None)
>> 13:06:47 Storage.Misc.excCmd: DEBUG: SUCCESS: <err> = 'mke2fs 1.42.13
>> (17-May-2015)\n'; <rc> = 0
>> 13:06:47 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /usr/bin/mount -o loop /var/tmp/tmpTeUZUl/link_to_image
>> /var/tmp/tmpTeUZUl/mountpoint (cwd None)
>> 13:06:47 --------------------- >> end captured logging <<
>> ---------------------
>> 13:06:47
>> 13:06:47
>> ======================================================================
>> 13:06:47 ERROR: test_getDevicePartedInfo
>> (parted_utils_tests.PartedUtilsTests)
>> 13:06:47
>> ----------------------------------------------------------------------
>> 13:06:47 Traceback (most recent call last):
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testValidation.py",
>> line 97, in wrapper
>> 13:06:47     return f(*args, **kwargs)
>> 13:06:47   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/parted_utils_tests.py",
>> line 61, in setUp
>> 13:06:47     self.assertEquals(rc, 0)
>> 13:06:47 AssertionError: 1 != 0
>> 13:06:47 -------------------- >> begin captured logging <<
>> --------------------
>> 13:06:47 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 dd if=/dev/zero
>> of=/tmp/tmp7dS7VS bs=100M count=1 (cwd None)
>> 13:06:47 root: DEBUG: SUCCESS: <err> = '1+0 records in\n1+0 records
>> out\n104857600 bytes (105 MB) copied, 0.350029 s, 300 MB/s\n'; <rc> = 0
>> 13:06:47 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 losetup -f --show
>> /tmp/tmp7dS7VS (cwd None)
>> 13:06:47 root: DEBUG: FAILED: <err> = 'losetup: /tmp/tmp7dS7VS: failed to
>> set up loop device: No such file or directory\n'; <rc> = 1
>>
>>
>>
>>
>>
>> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/2168/console
>>
>>
>>
>> 12:15:21
>> 12:15:21
>> ======================================================================
>> 12:15:21 ERROR: testLoopMount (mountTests.MountTests)
>> 12:15:21
>> ----------------------------------------------------------------------
>> 12:15:21 Traceback (most recent call last):
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/mountTests.py",
>> line 128, in testLoopMount
>> 12:15:21     m.mount(mntOpts="loop")
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 225, in mount
>> 12:15:21     return self._runcmd(cmd, timeout)
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 241, in _runcmd
>> 12:15:21     raise MountError(rc, ";".join((out, err)))
>> 12:15:21 MountError: (32, ';mount: /tmp/tmpsDTh9u: failed to setup loop
>> device: No such file or directory\n')
>> 12:15:21 -------------------- >> begin captured logging <<
>> --------------------
>> 12:15:21 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /sbin/mkfs.ext2 -F /tmp/tmpsDTh9u (cwd None)
>> 12:15:21 Storage.Misc.excCmd: DEBUG: SUCCESS: <err> = 'mke2fs 1.42.13
>> (17-May-2015)\n'; <rc> = 0
>> 12:15:21 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /usr/bin/mount -o loop /tmp/tmpsDTh9u /var/tmp/tmpTPj7t6 (cwd None)
>> 12:15:21 --------------------- >> end captured logging <<
>> ---------------------
>> 12:15:21
>> 12:15:21
>> ======================================================================
>> 12:15:21 ERROR: testSymlinkMount (mountTests.MountTests)
>> 12:15:21
>> ----------------------------------------------------------------------
>> 12:15:21 Traceback (most recent call last):
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/mountTests.py",
>> line 150, in testSymlinkMount
>> 12:15:21     m.mount(mntOpts="loop")
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 225, in mount
>> 12:15:21     return self._runcmd(cmd, timeout)
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
>> line 241, in _runcmd
>> 12:15:21     raise MountError(rc, ";".join((out, err)))
>> 12:15:21 MountError: (32, ';mount: /var/tmp/tmpBpSrGA/backing.img: failed
>> to setup loop device: No such file or directory\n')
>> 12:15:21 -------------------- >> begin captured logging <<
>> --------------------
>> 12:15:21 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /sbin/mkfs.ext2 -F /var/tmp/tmpBpSrGA/backing.img (cwd None)
>> 12:15:21 Storage.Misc.excCmd: DEBUG: SUCCESS: <err> = 'mke2fs 1.42.13
>> (17-May-2015)\n'; <rc> = 0
>> 12:15:21 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
>> /usr/bin/mount -o loop /var/tmp/tmpBpSrGA/link_to_image
>> /var/tmp/tmpBpSrGA/mountpoint (cwd None)
>> 12:15:21 --------------------- >> end captured logging <<
>> ---------------------
>> 12:15:21
>> 12:15:21
>> ======================================================================
>> 12:15:21 ERROR: test_getDevicePartedInfo
>> (parted_utils_tests.PartedUtilsTests)
>> 12:15:21
>> ----------------------------------------------------------------------
>> 12:15:21 Traceback (most recent call last):
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testValidation.py",
>> line 97, in wrapper
>> 12:15:21     return f(*args, **kwargs)
>> 12:15:21   File
>> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/parted_utils_tests.py",
>> line 61, in setUp
>> 12:15:21     self.assertEquals(rc, 0)
>> 12:15:21 AssertionError: 1 != 0
>> 12:15:21 -------------------- >> begin captured logging <<
>> --------------------
>> 12:15:21 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 dd if=/dev/zero
>> of=/tmp/tmpNOvAvX bs=100M count=1 (cwd None)
>> 12:15:21 root: DEBUG: SUCCESS: <err> = '1+0 records in\n1+0 records
>> out\n104857600 bytes (105 MB) copied, 0.373591 s, 281 MB/s\n'; <rc> = 0
>> 12:15:21 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 losetup -f --show
>> /tmp/tmpNOvAvX (cwd None)
>> 12:15:21 root: DEBUG: FAILED: <err> = 'losetup: /tmp/tmpNOvAvX: failed to
>> set up loop device: No such file or directory\n'; <rc> = 1
>> 12:15:21 --------------------- >> end captured logging <<
>> ---------------------
>> 12:15:21
>>
>>
>> _______________________________________________
>> Infra mailing list
>> Infra at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>
>>
>>
>> _______________________________________________
>> Infra mailing list
>> Infra at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>
>
>
>
> --
> Eyal Edri
> Associate Manager
> EMEA ENG Virtualization R&D
> Red Hat Israel
>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)

More information about the Infra mailing list