October 2019 - Infra - oVirt List Archives

[JIRA] (OVIRT-2813) EL8 builds fail to mount loop device - kernel too old?
by Nir Soffer (oVirt JIRA) 09 Oct '19

09 Oct '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2813?page=com.atlassian.jira.… ] Nir Soffer commented on OVIRT-2813: ----------------------------------- On Wed, Oct 9, 2019 at 11:56 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > I'm trying to run imageio tests on el8 mock, and the tests fail early when > trying to create storage > for testing: > > [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-ext4-mount > Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. > [userstorage] INFO Creating file /var/tmp/imageio-storage/file-512-ext4-mount/file > [userstorage] INFO Creating backing file /var/tmp/imageio-storage/file-512-xfs-backing > [userstorage] INFO Creating loop device /var/tmp/imageio-storage/file-512-xfs-loop > [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-xfs-mount > mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad option, bad superblock on /dev/loop4, missing codepage or helper program, or other error. > Traceback (most recent call last): > File "/usr/local/bin/userstorage", line 10, in <module> > sys.exit(main()) > File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 42, in main > create(cfg) > File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 52, in create > b.create() > File "/usr/local/lib/python3.6/site-packages/userstorage/file.py", line 47, in create > self._mount.create() > File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 53, in create > self._mount_loop() > File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 94, in _mount_loop > ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path]) > File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call > raise CalledProcessError(retcode, cmd) > subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs', '/var/tmp/imageio-storage/file-512-xfs-loop', '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit status 32. > > > https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//arti… > > > Same code runs fine in Travis: > https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863 > > > And also locally on Fedora 29: > $ ../jenkins/mock_configs/mock_runner.sh -C ../jenkins/mock_configs -p el8 > ... > ## Wed Oct 9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64 > ## took 85 seconds > ## rc = 0 > > > My guess is that we run el8 jobs on el7 hosts with old kernels > (Suggestion: Use Linux kernel >= 3.18 for improved stability of the > metadata and journal checksum features.) > Here is info from failed builds: DEBUG buildroot.py:503: kernel version == 3.10.0-693.11.6.el7.ppc64le https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/artif… DEBUG buildroot.py:503: kernel version == 3.10.0-957.12.1.el7.x86_64 https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593/artif… and successful builds: DEBUG buildroot.py:503: kernel version == 5.1.18-200.fc29.x86_64 My laptop Runtime kernel version: 4.15.0-1032-gcp https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863 This issue will affects vdsm, using similar code to create storage for > testing. > > For vdsm the issue is more tricky, since it requires same host/distro: > > 49 - tests-py37: > 50 runtime-requirements: > 51 host-distro: same > > So I think we need: > - make slaves with newer kernel (fc29? fc30?) with the el8 mock env > - add el8 slaves for running el8 mock with "host-distro: same" > > If we don't have a solution we need to disable el8 tests, or continue > testing without > storage, which will skip about 200 tests. > > Nir > > > EL8 builds fail to mount loop device - kernel too old? > ------------------------------------------------------ > > Key: OVIRT-2813 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2813 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > I'm trying to run imageio tests on el8 mock, and the tests fail early when > trying to create storage > for testing: > [userstorage] INFO Creating filesystem > /var/tmp/imageio-storage/file-512-ext4-mount > Suggestion: Use Linux kernel >= 3.18 for improved stability of the > metadata and journal checksum features. > [userstorage] INFO Creating file > /var/tmp/imageio-storage/file-512-ext4-mount/file > [userstorage] INFO Creating backing file > /var/tmp/imageio-storage/file-512-xfs-backing > [userstorage] INFO Creating loop device > /var/tmp/imageio-storage/file-512-xfs-loop > [userstorage] INFO Creating filesystem > /var/tmp/imageio-storage/file-512-xfs-mount > mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad > option, bad superblock on /dev/loop4, missing codepage or helper > program, or other error. > Traceback (most recent call last): > File "/usr/local/bin/userstorage", line 10, in <module> > sys.exit(main()) > File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", > line 42, in main > create(cfg) > File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", > line 52, in create > b.create() > File "/usr/local/lib/python3.6/site-packages/userstorage/file.py", > line 47, in create > self._mount.create() > File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", > line 53, in create > self._mount_loop() > File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", > line 94, in _mount_loop > ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path]) > File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call > raise CalledProcessError(retcode, cmd) > subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs', > '/var/tmp/imageio-storage/file-512-xfs-loop', > '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit > status 32. > https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//arti… > Same code runs fine in Travis: > https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863 > And also locally on Fedora 29: > $ ../jenkins/mock_configs/mock_runner.sh -C ../jenkins/mock_configs -p el8 > ... > ## Wed Oct 9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64 > ## took 85 seconds > ## rc = 0 > My guess is that we run el8 jobs on el7 hosts with old kernels > (Suggestion: Use Linux kernel >= 3.18 for improved stability of the > metadata and journal checksum features.) > This issue will affects vdsm, using similar code to create storage for > testing. > For vdsm the issue is more tricky, since it requires same host/distro: > 49 - tests-py37: > 50 runtime-requirements: > 51 host-distro: same > So I think we need: > - make slaves with newer kernel (fc29? fc30?) with the el8 mock env > - add el8 slaves for running el8 mock with "host-distro: same" > If we don't have a solution we need to disable el8 tests, or continue > testing without > storage, which will skip about 200 tests. > Nir -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100111)

1 0

[JIRA] (OVIRT-2813) EL8 builds fail to mount loop device - kernel too old?
by Nir Soffer (oVirt JIRA) 09 Oct '19

09 Oct '19

Nir Soffer created OVIRT-2813: --------------------------------- Summary: EL8 builds fail to mount loop device - kernel too old? Key: OVIRT-2813 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2813 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Nir Soffer Assignee: infra I'm trying to run imageio tests on el8 mock, and the tests fail early when trying to create storage for testing: [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-ext4-mount Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features. [userstorage] INFO Creating file /var/tmp/imageio-storage/file-512-ext4-mount/file [userstorage] INFO Creating backing file /var/tmp/imageio-storage/file-512-xfs-backing [userstorage] INFO Creating loop device /var/tmp/imageio-storage/file-512-xfs-loop [userstorage] INFO Creating filesystem /var/tmp/imageio-storage/file-512-xfs-mount mount: /var/tmp/imageio-storage/file-512-xfs-mount: wrong fs type, bad option, bad superblock on /dev/loop4, missing codepage or helper program, or other error. Traceback (most recent call last): File "/usr/local/bin/userstorage", line 10, in <module> sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 42, in main create(cfg) File "/usr/local/lib/python3.6/site-packages/userstorage/__main__.py", line 52, in create b.create() File "/usr/local/lib/python3.6/site-packages/userstorage/file.py", line 47, in create self._mount.create() File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 53, in create self._mount_loop() File "/usr/local/lib/python3.6/site-packages/userstorage/mount.py", line 94, in _mount_loop ["sudo", "mount", "-t", self.fstype, self._loop.path, self.path]) File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['sudo', 'mount', '-t', 'xfs', '/var/tmp/imageio-storage/file-512-xfs-loop', '/var/tmp/imageio-storage/file-512-xfs-mount']' returned non-zero exit status 32. https://jenkins.ovirt.org/job/ovirt-imageio_standard-check-patch/1593//arti… Same code runs fine in Travis: https://travis-ci.org/nirs/ovirt-imageio/jobs/595794863 And also locally on Fedora 29: $ ../jenkins/mock_configs/mock_runner.sh -C ../jenkins/mock_configs -p el8 ... ## Wed Oct 9 23:37:22 IDT 2019 Finished env: el8:epel-8-x86_64 ## took 85 seconds ## rc = 0 My guess is that we run el8 jobs on el7 hosts with old kernels (Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.) This issue will affects vdsm, using similar code to create storage for testing. For vdsm the issue is more tricky, since it requires same host/distro: 49 - tests-py37: 50 runtime-requirements: 51 host-distro: same So I think we need: - make slaves with newer kernel (fc29? fc30?) with the el8 mock env - add el8 slaves for running el8 mock with "host-distro: same" If we don't have a solution we need to disable el8 tests, or continue testing without storage, which will skip about 200 tests. Nir -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100111)

1 0

[CQ]: 103369, 13 (ovirt-provider-ovn) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 09 Oct '19

09 Oct '19

A system test invoked by the "ovirt-master" change queue including change 103369,13 (ovirt-provider-ovn) failed. However, this change seems not to be the root cause for this failure. Change 103930,1 (ovirt-provider-ovn) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 103930,1 (ovirt-provider-ovn) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/103369/13 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/103930/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16420/

1 0

[CQ]: 103406, 5 (ovirt-provider-ovn) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 09 Oct '19

09 Oct '19

A system test invoked by the "ovirt-master" change queue including change 103406,5 (ovirt-provider-ovn) failed. However, this change seems not to be the root cause for this failure. Change 103930,1 (ovirt-provider-ovn) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 103930,1 (ovirt-provider-ovn) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/103406/5 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/103930/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16416/

1 0

[CQ]: 103257, 5 (ovirt-guest-agent) failed "ovirt-master" system tests
by oVirt Jenkins 09 Oct '19

09 Oct '19

Change 103257,5 (ovirt-guest-agent) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/103257/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16411/

1 0

[CQ]: 103983,3 (safelease) failed "ovirt-master" system tests
by oVirt Jenkins 09 Oct '19

09 Oct '19

Change 103983,3 (safelease) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/103983/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16405/

1 0

Please remove 4.2 from your stdci.yaml files
by Dafna Ron 09 Oct '19

09 Oct '19

Hi, We have several unstable jobs which are caused because 4.2 is no longer available for CQ. Please remove 4.2 from your stdci.yaml files to avoid getting unstable jobs. Thanks, Dafna

2 2

[CQ]: 103368, 9 (ovirt-provider-ovn) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 09 Oct '19

09 Oct '19

A system test invoked by the "ovirt-master" change queue including change 103368,9 (ovirt-provider-ovn) failed. However, this change seems not to be the root cause for this failure. Change 103930,1 (ovirt-provider-ovn) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 103930,1 (ovirt-provider-ovn) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/103368/9 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/103930/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16401/

1 0

[CQ]: 103981, 2 (safelease) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 09 Oct '19

09 Oct '19

A system test invoked by the "ovirt-master" change queue including change 103981,2 (safelease) failed. However, this change seems not to be the root cause for this failure. Change 99736,5 (safelease) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 99736,5 (safelease) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/103981/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/99736/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16396/

1 0

[CQ]: 103387, 8 (ovirt-provider-ovn) failed "ovirt-4.3" system tests, but isn't the failure root cause
by oVirt Jenkins 09 Oct '19

09 Oct '19

A system test invoked by the "ovirt-4.3" change queue including change 103387,8 (ovirt-provider-ovn) failed. However, this change seems not to be the root cause for this failure. Change 103368,9 (ovirt-provider-ovn) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 103368,9 (ovirt-provider-ovn) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/103387/8 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/103368/9 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2255/

1 0