[JIRA] (OVIRT-2813) EL8 builds fail to mount loop device - kernel
too old?
by Marcin Sobczyk (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2813?page=com.atlassian.jir... ]
Marcin Sobczyk commented on OVIRT-2813:
---------------------------------------
Hi,
On 10/10/19 6:14 AM, Yuval Turgeman wrote:
> You may be running out of loop devices, if that's the case, you need
> to manually mknod them, see
> https://github.com/oVirt/ovirt-node-ng-image/blob/56a2797b5ef84bd56ab95fd...
>
> On Thursday, October 10, 2019, Nir Soffer <nsoffer(a)redhat.com
> <mailto:nsoffer@redhat.com>> wrote:
>
> On Wed, Oct 9, 2019 at 11:56 PM Nir Soffer <nsoffer(a)redhat.com
> <mailto:nsoffer@redhat.com>> wrote:
>
> I'm trying to run imageio tests on el8 mock, and the tests
> fail early when trying to create storage
> for testing:
>
> [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-ext4-mount
> Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
> [userstorage] INFO Creating file /var/tmp/imageio-storage/file-512-ext4-mount/file
> [userstorage] INFO Creating backing file /var/tmp/imageio-storage/file-512-xfs-backing
> [userstorage] INFO Creating loop device /var/tmp/imageio-storage/file-512-xfs-loop
> [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-xfs-mount
> mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad option, bad superblock on /dev/loop4, missing codepage or helper program, or other error.
> Traceback (most recent call last):
> File "/usr/local/bin/userstorage", line 10, in <module>
> sys.exit(main())
> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 42, in main
> create(cfg)
> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 52, in create
> b.create()
> File "/usr/local/lib/python3.6/site-packages/userstorage/file.py", line 47, in create
> self._mount.create()
> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 53, in create
> self._mount_loop()
> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 94, in _mount_loop
> ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path])
> File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
> raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs', '/var/tmp/imageio-storage/file-512-xfs-loop', '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit status 32.
>
> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//ar...
> <https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//ar...>
>
>
> Same code runs fine in Travis:
> https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863
> <https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863>
>
>
> And also locally on Fedora 29:
> $ ../jenkins/mock_configs/mock_runner.sh -C
> ../jenkins/mock_configs -p el8
> ...
> ## Wed Oct 9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64
> ## took 85 seconds
> ## rc = 0
>
>
> My guess is that we run el8 jobs on el7 hosts with old kernels
> (Suggestion: Use Linux kernel >= 3.18 for improved stability
> of the metadata and journal checksum features.)
>
>
> Here is info from failed builds:
>
> DEBUG buildroot.py:503: kernel version == 3.10.0-693.11.6.el7.ppc64le
> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art...
> <https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art...>
>
> DEBUG buildroot.py:503: kernel version == 3.10.0-957.12.1.el7.x86_64
> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art...
> <https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/art...>
> and successful builds:
>
> DEBUG buildroot.py:503: kernel version == 5.1.18-200.fc29.x86_64
> My laptop
>
> Runtime kernel version: 4.15.0-1032-gcp
> https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863
> <https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863>
>
>
> This issue will affects vdsm, using similar code to create
> storage for testing.
>
> For vdsm the issue is more tricky, since it requires same
> host/distro:
>
> 49 - tests-py37:
> 50 runtime-requirements:
> 51 host-distro: same
>
> So I think we need:
> - make slaves with newer kernel (fc29? fc30?) with the el8
> mock env
> - add el8 slaves for running el8 mock with "host-distro: same"
>
AFAIK Anton was working on that and we should have el8-native ci slaves
maybe even today.
+Anton
>
> If we don't have a solution we need to disable el8 tests, or
> continue testing without
> storage, which will skip about 200 tests.
>
> Nir
>
> EL8 builds fail to mount loop device - kernel too old?
> ------------------------------------------------------
>
> Key: OVIRT-2813
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2813
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: Nir Soffer
> Assignee: infra
>
> I'm trying to run imageio tests on el8 mock, and the tests fail early when
> trying to create storage
> for testing:
> [userstorage] INFO Creating filesystem
> /var/tmp/imageio-storage/file-512-ext4-mount
> Suggestion: Use Linux kernel >= 3.18 for improved stability of the
> metadata and journal checksum features.
> [userstorage] INFO Creating file
> /var/tmp/imageio-storage/file-512-ext4-mount/file
> [userstorage] INFO Creating backing file
> /var/tmp/imageio-storage/file-512-xfs-backing
> [userstorage] INFO Creating loop device
> /var/tmp/imageio-storage/file-512-xfs-loop
> [userstorage] INFO Creating filesystem
> /var/tmp/imageio-storage/file-512-xfs-mount
> mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad
> option, bad superblock on /dev/loop4, missing codepage or helper
> program, or other error.
> Traceback (most recent call last):
> File "/usr/local/bin/userstorage", line 10, in <module>
> sys.exit(main())
> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py",
> line 42, in main
> create(cfg)
> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py",
> line 52, in create
> b.create()
> File "/usr/local/lib/python3.6/site-packages/userstorage/file.py",
> line 47, in create
> self._mount.create()
> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py",
> line 53, in create
> self._mount_loop()
> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py",
> line 94, in _mount_loop
> ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path])
> File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
> raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs',
> '/var/tmp/imageio-storage/file-512-xfs-loop',
> '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit
> status 32.
> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//ar...
> Same code runs fine in Travis:
> https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863
> And also locally on Fedora 29:
> $ ../jenkins/mock_configs/mock_runner.sh -C ../jenkins/mock_configs -p el8
> ...
> ## Wed Oct 9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64
> ## took 85 seconds
> ## rc = 0
> My guess is that we run el8 jobs on el7 hosts with old kernels
> (Suggestion: Use Linux kernel >= 3.18 for improved stability of the
> metadata and journal checksum features.)
> This issue will affects vdsm, using similar code to create storage for
> testing.
> For vdsm the issue is more tricky, since it requires same host/distro:
> 49 - tests-py37:
> 50 runtime-requirements:
> 51 host-distro: same
> So I think we need:
> - make slaves with newer kernel (fc29? fc30?) with the el8 mock env
> - add el8 slaves for running el8 mock with "host-distro: same"
> If we don't have a solution we need to disable el8 tests, or continue
> testing without
> storage, which will skip about 200 tests.
> Nir
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100111)
4 years, 7 months
[JIRA] (OVIRT-2813) EL8 builds fail to mount loop device - kernel
too old?
by Yuval Turgeman (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2813?page=com.atlassian.jir... ]
Yuval Turgeman commented on OVIRT-2813:
---------------------------------------
You may be running out of loop devices, if that's the case, you need to
manually mknod them, see
https://github.com/oVirt/ovirt-node-ng-image/blob/56a2797b5ef84bd56ab95fd...
On Thursday, October 10, 2019, Nir Soffer <nsoffer(a)redhat.com> wrote:
> On Wed, Oct 9, 2019 at 11:56 PM Nir Soffer <nsoffer(a)redhat.com> wrote:
>
>> I'm trying to run imageio tests on el8 mock, and the tests fail early
>> when trying to create storage
>> for testing:
>>
>> [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-ext4-mount
>> Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
>> [userstorage] INFO Creating file /var/tmp/imageio-storage/file-512-ext4-mount/file
>> [userstorage] INFO Creating backing file /var/tmp/imageio-storage/file-512-xfs-backing
>> [userstorage] INFO Creating loop device /var/tmp/imageio-storage/file-512-xfs-loop
>> [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-xfs-mount
>> mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad option, bad superblock on /dev/loop4, missing codepage or helper program, or other error.
>> Traceback (most recent call last):
>> File "/usr/local/bin/userstorage", line 10, in <module>
>> sys.exit(main())
>> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 42, in main
>> create(cfg)
>> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 52, in create
>> b.create()
>> File "/usr/local/lib/python3.6/site-packages/userstorage/file.py", line 47, in create
>> self._mount.create()
>> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 53, in create
>> self._mount_loop()
>> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 94, in _mount_loop
>> ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path])
>> File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
>> raise CalledProcessError(retcode, cmd)
>> subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs', '/var/tmp/imageio-storage/file-512-xfs-loop', '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit status 32.
>>
>> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-
>> patch/1593//artifact/check-patch.el8.ppc64le/mock_logs/
>> script/stdout_stderr.log
>>
>>
>> Same code runs fine in Travis:
>> https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863
>>
>>
>> And also locally on Fedora 29:
>> $ ../jenkins/mock_configs/mock_runner.sh -C ../jenkins/mock_configs -p
>> el8
>> ...
>> ## Wed Oct 9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64
>> ## took 85 seconds
>> ## rc = 0
>>
>>
>> My guess is that we run el8 jobs on el7 hosts with old kernels
>> (Suggestion: Use Linux kernel >= 3.18 for improved stability of the
>> metadata and journal checksum features.)
>>
>
> Here is info from failed builds:
>
> DEBUG buildroot.py:503: kernel version == 3.10.0-693.11.6.el7.ppc64le
> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-
> patch/1593/artifact/check-patch.el8.ppc64le/mock_logs/init/root.log
>
> DEBUG buildroot.py:503: kernel version == 3.10.0-957.12.1.el7.x86_64
> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-
> patch/1593/artifact/check-patch.el8.x86_64/mock_logs/init/root.log
>
> and successful builds:
>
> DEBUG buildroot.py:503: kernel version == 5.1.18-200.fc29.x86_64
> My laptop
>
> Runtime kernel version: 4.15.0-1032-gcp
> https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863
>
>
> This issue will affects vdsm, using similar code to create storage for
>> testing.
>>
>> For vdsm the issue is more tricky, since it requires same host/distro:
>>
>> 49 - tests-py37:
>> 50 runtime-requirements:
>> 51 host-distro: same
>>
>> So I think we need:
>> - make slaves with newer kernel (fc29? fc30?) with the el8 mock env
>> - add el8 slaves for running el8 mock with "host-distro: same"
>>
>> If we don't have a solution we need to disable el8 tests, or continue
>> testing without
>> storage, which will skip about 200 tests.
>>
>> Nir
>>
>>
> EL8 builds fail to mount loop device - kernel too old?
> ------------------------------------------------------
>
> Key: OVIRT-2813
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2813
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: Nir Soffer
> Assignee: infra
>
> I'm trying to run imageio tests on el8 mock, and the tests fail early when
> trying to create storage
> for testing:
> [userstorage] INFO Creating filesystem
> /var/tmp/imageio-storage/file-512-ext4-mount
> Suggestion: Use Linux kernel >= 3.18 for improved stability of the
> metadata and journal checksum features.
> [userstorage] INFO Creating file
> /var/tmp/imageio-storage/file-512-ext4-mount/file
> [userstorage] INFO Creating backing file
> /var/tmp/imageio-storage/file-512-xfs-backing
> [userstorage] INFO Creating loop device
> /var/tmp/imageio-storage/file-512-xfs-loop
> [userstorage] INFO Creating filesystem
> /var/tmp/imageio-storage/file-512-xfs-mount
> mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad
> option, bad superblock on /dev/loop4, missing codepage or helper
> program, or other error.
> Traceback (most recent call last):
> File "/usr/local/bin/userstorage", line 10, in <module>
> sys.exit(main())
> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py",
> line 42, in main
> create(cfg)
> File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py",
> line 52, in create
> b.create()
> File "/usr/local/lib/python3.6/site-packages/userstorage/file.py",
> line 47, in create
> self._mount.create()
> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py",
> line 53, in create
> self._mount_loop()
> File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py",
> line 94, in _mount_loop
> ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path])
> File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
> raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs',
> '/var/tmp/imageio-storage/file-512-xfs-loop',
> '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit
> status 32.
> https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//ar...
> Same code runs fine in Travis:
> https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863
> And also locally on Fedora 29:
> $ ../jenkins/mock_configs/mock_runner.sh -C ../jenkins/mock_configs -p el8
> ...
> ## Wed Oct 9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64
> ## took 85 seconds
> ## rc = 0
> My guess is that we run el8 jobs on el7 hosts with old kernels
> (Suggestion: Use Linux kernel >= 3.18 for improved stability of the
> metadata and journal checksum features.)
> This issue will affects vdsm, using similar code to create storage for
> testing.
> For vdsm the issue is more tricky, since it requires same host/distro:
> 49 - tests-py37:
> 50 runtime-requirements:
> 51 host-distro: same
> So I think we need:
> - make slaves with newer kernel (fc29? fc30?) with the el8 mock env
> - add el8 slaves for running el8 mock with "host-distro: same"
> If we don't have a solution we need to disable el8 tests, or continue
> testing without
> storage, which will skip about 200 tests.
> Nir
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100111)
4 years, 7 months
