[ovirt-devel] Re: EL8 builds fail to mount loop device - kernel too old?

You may be running out of loop devices, if that's the case, you need to manually mknod them, see https://github.com/oVirt/ovirt-node-ng-image/blob/56a2797b5ef84bd56ab95fd... On Thursday, October 10, 2019, Nir Soffer <nsoffer(a)redhat.com <mailto:nsoffer@redhat.com>> wrote: On Wed, Oct 9, 2019 at 11:56 PM Nir Soffer <nsoffer(a)redhat.com <mailto:nsoffer@redhat.com>> wrote: I'm trying to run imageio tests on el8 mock, and the tests fail early when trying to create storage for testing: [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-ext4-mount Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. [userstorage] INFO Creating file /var/tmp/imageio-storage/file-512-ext4-mount/file [userstorage] INFO Creating backing file /var/tmp/imageio-storage/file-512-xfs-backing [userstorage] INFO Creating loop device /var/tmp/imageio-storage/file-512-xfs-loop [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-xfs-mount mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad option, bad superblock on /dev/loop4, missing codepage or helper program, or other error. Traceback (most recent call last): File "/usr/local/bin/userstorage", line 10, in <module> sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 42, in main create(cfg) File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 52, in create b.create() File "/usr/local/lib/python3.6/site-packages/userstorage/file.py", line 47, in create self._mount.create() File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 53, in create self._mount_loop() File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 94, in _mount_loop ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path]) File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs', '/var/tmp/imageio-storage/file-512-xfs-loop', '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit status 32. https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//ar... <https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//ar... Same code runs fine in Travis: https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863 <https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863> And also locally on Fedora 29: $ ../jenkins/mock_configs/mock_runner.sh -C ../jenkins/mock_configs -p el8 ... ## Wed Oct  9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64 ##      took 85 seconds ##      rc = 0 My guess is that we run el8 jobs on el7 hosts with old kernels (Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.) Here is info from failed builds: DEBUG buildroot.py:503: kernel version == 3.10.0-693.11.6.el7.ppc64le https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art... <https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art... DEBUG buildroot.py:503:  kernel version == 3.10.0-957.12.1.el7.x86_64 https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art... <https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art... and successful builds: DEBUG buildroot.py:503:  kernel version == 5.1.18-200.fc29.x86_64 My laptop Runtime kernel version: 4.15.0-1032-gcp https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863 <https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863> This issue will affects vdsm, using similar code to create storage for testing. For vdsm the issue is more tricky, since it requires same host/distro:  49   - tests-py37:  50             runtime-requirements:  51               host-distro: same So I think we need: - make slaves with newer kernel (fc29? fc30?) with the el8 mock env - add el8 slaves for running el8 mock with "host-distro: same"

If we don't have a solution we need to disable el8 tests, or continue testing without storage, which will skip about 200 tests. Nir