Tested failing because of missing loop devices

Eyal Edri eedri at redhat.com
Wed Dec 23 15:51:30 UTC 2015 

On Wed, Dec 23, 2015 at 5:41 PM, Nir Soffer <nsoffer at redhat.com> wrote:

> On Wed, Dec 23, 2015 at 5:11 PM, Eyal Edri <eedri at redhat.com> wrote:
> > I'm guessing this will e solved by running it on lago?
> > Isn't that what yaniv is working on now?
> >
> > or these are unit tests and not functional?
> >
> > e.
> >
> > On Wed, Dec 23, 2015 at 4:48 PM, Dan Kenigsberg <danken at redhat.com>
> wrote:
> >>
> >> On Wed, Dec 23, 2015 at 03:21:31AM +0200, Nir Soffer wrote:
> >> > Hi all,
> >> >
> >> > We see too many failures of tests using loop devices. Is it possible
> >> > that we run tests
> >> > concurrently on the same slave, using all the available loop devices,
> or
> >> > maybe
> >> > creating races between different tests?
> >> >
> >> > It seems that we need new decorator for disabling tests on the CI
> >> > slaves, since this
> >> > environment is too fragile.
> >> >
> >> > Here are some failures:
> >> >
> >> > 01:10:33
> >> > ======================================================================
> >> > 01:10:33 ERROR: testLoopMount (mountTests.MountTests)
> >> > 01:10:33
> >> > ----------------------------------------------------------------------
> >> > 01:10:33 Traceback (most recent call last):
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/mountTests.py",
> >> > line 128, in testLoopMount
> >> > 01:10:33     m.mount(mntOpts="loop")
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
> >> > line 225, in mount
> >> > 01:10:33     return self._runcmd(cmd, timeout)
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
> >> > line 241, in _runcmd
> >> > 01:10:33     raise MountError(rc, ";".join((out, err)))
> >> > 01:10:33 MountError: (32, ';mount: /tmp/tmpZuJRNk: failed to setup
> >> > loop device: No such file or directory\n')
> >> > 01:10:33 -------------------- >> begin captured logging <<
> >> > --------------------
> >> > 01:10:33 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
> >> > /sbin/mkfs.ext2 -F /tmp/tmpZuJRNk (cwd None)
> >> > 01:10:33 Storage.Misc.excCmd: DEBUG: SUCCESS: <err> = 'mke2fs 1.42.13
> >> > (17-May-2015)\n'; <rc> = 0
> >> > 01:10:33 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
> >> > /usr/bin/mount -o loop /tmp/tmpZuJRNk /var/tmp/tmpJO52Xj (cwd None)
> >> > 01:10:33 --------------------- >> end captured logging <<
> >> > ---------------------
> >> > 01:10:33
> >> > 01:10:33
> >> > ======================================================================
> >> > 01:10:33 ERROR: testSymlinkMount (mountTests.MountTests)
> >> > 01:10:33
> >> > ----------------------------------------------------------------------
> >> > 01:10:33 Traceback (most recent call last):
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/mountTests.py",
> >> > line 150, in testSymlinkMount
> >> > 01:10:33     m.mount(mntOpts="loop")
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
> >> > line 225, in mount
> >> > 01:10:33     return self._runcmd(cmd, timeout)
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/vdsm/storage/mount.py",
> >> > line 241, in _runcmd
> >> > 01:10:33     raise MountError(rc, ";".join((out, err)))
> >> > 01:10:33 MountError: (32, ';mount: /var/tmp/tmp1UQFPz/backing.img:
> >> > failed to setup loop device: No such file or directory\n')
> >> > 01:10:33 -------------------- >> begin captured logging <<
> >> > --------------------
> >> > 01:10:33 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
> >> > /sbin/mkfs.ext2 -F /var/tmp/tmp1UQFPz/backing.img (cwd None)
> >> > 01:10:33 Storage.Misc.excCmd: DEBUG: SUCCESS: <err> = 'mke2fs 1.42.13
> >> > (17-May-2015)\n'; <rc> = 0
> >> > 01:10:33 Storage.Misc.excCmd: DEBUG: /usr/bin/taskset --cpu-list 0-1
> >> > /usr/bin/mount -o loop /var/tmp/tmp1UQFPz/link_to_image
> >> > /var/tmp/tmp1UQFPz/mountpoint (cwd None)
> >> > 01:10:33 --------------------- >> end captured logging <<
> >> > ---------------------
> >> > 01:10:33
> >> > 01:10:33
> >> > ======================================================================
> >> > 01:10:33 ERROR: test_getDevicePartedInfo
> >> > (parted_utils_tests.PartedUtilsTests)
> >> > 01:10:33
> >> > ----------------------------------------------------------------------
> >> > 01:10:33 Traceback (most recent call last):
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/testValidation.py",
> >> > line 97, in wrapper
> >> > 01:10:33     return f(*args, **kwargs)
> >> > 01:10:33   File
> >> >
> >> >
> "/home/jenkins/workspace/vdsm_master_check-patch-fc23-x86_64/vdsm/tests/parted_utils_tests.py",
> >> > line 61, in setUp
> >> > 01:10:33     self.assertEquals(rc, 0)
> >> > 01:10:33 AssertionError: 1 != 0
> >> > 01:10:33 -------------------- >> begin captured logging <<
> >> > --------------------
> >> > 01:10:33 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 dd if=/dev/zero
> >> > of=/tmp/tmpasV8TD bs=100M count=1 (cwd None)
> >> > 01:10:33 root: DEBUG: SUCCESS: <err> = '1+0 records in\n1+0 records
> >> > out\n104857600 bytes (105 MB) copied, 0.368498 s, 285 MB/s\n'; <rc> =
> >> > 0
> >> > 01:10:33 root: DEBUG: /usr/bin/taskset --cpu-list 0-1 losetup -f
> >> > --show /tmp/tmpasV8TD (cwd None)
> >> > 01:10:33 root: DEBUG: FAILED: <err> = 'losetup: /tmp/tmpasV8TD: failed
> >> > to set up loop device: No such file or directory\n'; <rc> = 1
> >> > 01:10:33 --------------------- >> end captured logging <<
> >> > ---------------------
> >> >
> >>
> >> I've reluctantly marked another test as broken in
> >> https://gerrit.ovirt.org/50484
> >> due to a similar problem.
> >> Your idea of @brokentest_ci decorator is slightly less bad - at least we
> >> do not ignore errors in this test when run on non-ci platforms.
>
> Sadly the correct name should be @brokenci :-)
>

Not sure i agree, it should be broken tests, not making sure to run
isolated and not affecting the env :)


>
> Nir
>



-- 
Eyal Edri
Associate Manager
EMEA ENG Virtualization R&D
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20151223/fecfa278/attachment.html>

More information about the Infra mailing list