November 2019 - Infra - oVirt List Archives

Build failed in Jenkins: system-sync_mirrors-fedora-updates-fc29-x86_64 #1105
by jenkins＠jenkins.phx.ovirt.org 13 Nov '19

13 Nov '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc29-x86_64…> Changes: ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc29-x86_64…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress --prune http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision edf61c5ec106227d6342b6e12e43f5d437a3727c (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f edf61c5ec106227d6342b6e12e43f5d437a3727c Commit message: "pipeline-loader: Allocate containers from code" > git rev-list --no-walk edf61c5ec106227d6342b6e12e43f5d437a3727c # timeout=10 [system-sync_mirrors-fedora-updates-fc29-x86_64] $ /bin/bash -xe /tmp/jenkins1059516635071455121.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror fedora-updates-fc29 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror fedora-updates-fc29 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror fedora-updates-fc29 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=fedora-updates-fc29 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs fedora-updates-fc29 yum + local repo_name=fedora-updates-fc29 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/fedora-updates-fc29 /var/www/html/repos/yum/fedora-updates-fc29/base + check_yum_sync_needed fedora-updates-fc29 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=fedora-updates-fc29 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/fedora-updates-fc29 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync fedora-updates-fc29 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=fedora-updates-fc29 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=fedora-updates-fc29 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/fedora-updates-fc29/base --norepopath --newest-only --urls --quiet Error setting up repositories: Cannot retrieve metalink for repository: fedora-updates-fc29. Please verify its path and try again + reposync_out= Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 1007
by jenkins＠jenkins.phx.ovirt.org 13 Nov '19

13 Nov '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/1007//artifact/exported-… Cheers, Jenkins

1 0

[CQ]: 104572, 2 (imgbased) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 12 Nov '19

12 Nov '19

A system test invoked by the "ovirt-master" change queue including change 104572,2 (imgbased) failed. However, this change seems not to be the root cause for this failure. Change 104533,2 (imgbased) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 104533,2 (imgbased) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/104572/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/104533/2 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16863/

1 0

[JIRA] (OVIRT-2828) Re: [ovirt-devel] Re: Check patch failure in vdsm
by Nir Soffer (oVirt JIRA) 12 Nov '19

12 Nov '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2828?page=com.atlassian.jira.… ] Nir Soffer commented on OVIRT-2828: ----------------------------------- On Tue, Nov 12, 2019 at 7:30 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > > On Tue, Nov 12, 2019 at 5:34 PM Miguel Duarte de Mora Barroso > <mdbarroso(a)redhat.com> wrote: > > > > On Mon, Nov 11, 2019 at 11:12 AM Eyal Shenitzky <eshenitz(a)redhat.com> wrote: > > > > > > Hi, > > > > > > I encounter the following error for py36 test in vdsm check patch: > > > > > > ... > > > ... > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall > > > 14:03:37 res = hook_impl.function(*args) > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda> > > > 14:03:37 firstresult=hook.spec.opts.get("firstresult") if hook.spec else False, > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec > > > 14:03:37 return self._inner_hookexec(hook, methods, kwargs) > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__ > > > 14:03:37 return self._hookexec(self, self.get_hookimpls(), kwargs) > > > 14:03:37 File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 82, in main > > > 14:03:37 return config.hook.pytest_cmdline_main(config=config) > > > 14:03:37 File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/bin/pytest", line 8, in <module> > > > 14:03:37 sys.exit(main()) > > > 14:03:37 [Inferior 1 (process 22145) detached] > > > 14:03:37 ============================================================= > > > 14:03:37 = Terminating watched process = > > > 14:03:37 ============================================================= > > > 14:03:37 PROFILE {"command": ["python", "py-watch", "600", "pytest", "-m", "not (slow or stress)", "--durations=10", "--cov=vdsm.storage", "--cov-report=html:htmlcov-storage-py37", "--cov-fail-under=62", "storage"], "cpu": 39.921942808919184, "elapsed": 604.4699757099152, "idrss": 0, "inblock": 1693453, "isrss": 0, "ixrss": 0, "majflt": 2, "maxrss": 331172, "minflt": 5606489, "msgrcv": 0, "msgsnd": 0, "name": "storage-py37", "nivcsw": 139819, "nsignals": 0, "nswap": 0, "nvcsw": 187576, "oublock": 2495645, "start": 1573386812.7961884, "status": 143, "stime": 118.260961, "utime": 123.055197} > > > 14:03:37 ERROR: InvocationError for command /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/bin/python profile storage-py37 python py-watch 600 pytest -m 'not (slow or stress)' --durations=10 --cov=vdsm.storage --cov-report=html:htmlcov-storage-py37 --cov-fail-under=62 storage (exited with code 143) > > > > > > > > > Is there any known issue? > > > > Anyone able to pitch in ? I think something similar is happening in > > [0], also on check-patch [1]. > > > > [0] - https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-pa… > > Yes it looks the same issue. > > > [1] - https://gerrit.ovirt.org/#/c/104274/ > > Jenkins slaves are very slow recently. I suspect we run too many jobs > concurrently or using > too many virtual cpus. I hope this will avoid the random failures: https://gerrit.ovirt.org/c/104629/ > Re: [ovirt-devel] Re: Check patch failure in vdsm > ------------------------------------------------- > > Key: OVIRT-2828 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2828 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > On Tue, Nov 12, 2019 at 5:34 PM Miguel Duarte de Mora Barroso > <mdbarroso(a)redhat.com> wrote: > > > > On Mon, Nov 11, 2019 at 11:12 AM Eyal Shenitzky <eshenitz(a)redhat.com> wrote: > > > > > > Hi, > > > > > > I encounter the following error for py36 test in vdsm check patch: > > > > > > ... > > > ... > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall > > > 14:03:37 res = hook_impl.function(*args) > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda> > > > 14:03:37 firstresult=hook.spec.opts.get("firstresult") if hook.spec else False, > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec > > > 14:03:37 return self._inner_hookexec(hook, methods, kwargs) > > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__ > > > 14:03:37 return self._hookexec(self, self.get_hookimpls(), kwargs) > > > 14:03:37 File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 82, in main > > > 14:03:37 return config.hook.pytest_cmdline_main(config=config) > > > 14:03:37 File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/bin/pytest", line 8, in <module> > > > 14:03:37 sys.exit(main()) > > > 14:03:37 [Inferior 1 (process 22145) detached] > > > 14:03:37 ============================================================= > > > 14:03:37 = Terminating watched process = > > > 14:03:37 ============================================================= > > > 14:03:37 PROFILE {"command": ["python", "py-watch", "600", "pytest", "-m", "not (slow or stress)", "--durations=10", "--cov=vdsm.storage", "--cov-report=html:htmlcov-storage-py37", "--cov-fail-under=62", "storage"], "cpu": 39.921942808919184, "elapsed": 604.4699757099152, "idrss": 0, "inblock": 1693453, "isrss": 0, "ixrss": 0, "majflt": 2, "maxrss": 331172, "minflt": 5606489, "msgrcv": 0, "msgsnd": 0, "name": "storage-py37", "nivcsw": 139819, "nsignals": 0, "nswap": 0, "nvcsw": 187576, "oublock": 2495645, "start": 1573386812.7961884, "status": 143, "stime": 118.260961, "utime": 123.055197} > > > 14:03:37 ERROR: InvocationError for command /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/bin/python profile storage-py37 python py-watch 600 pytest -m 'not (slow or stress)' --durations=10 --cov=vdsm.storage --cov-report=html:htmlcov-storage-py37 --cov-fail-under=62 storage (exited with code 143) > > > > > > > > > Is there any known issue? > > > > Anyone able to pitch in ? I think something similar is happening in > > [0], also on check-patch [1]. > > > > [0] - https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-pa… > Yes it looks the same issue. > > [1] - https://gerrit.ovirt.org/#/c/104274/ > Jenkins slaves are very slow recently. I suspect we run too many jobs > concurrently or using > too many virtual cpus. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100114)

1 0

[JIRA] (OVIRT-2828) Re: [ovirt-devel] Re: Check patch failure in vdsm
by Nir Soffer (oVirt JIRA) 12 Nov '19

12 Nov '19

Nir Soffer created OVIRT-2828: --------------------------------- Summary: Re: [ovirt-devel] Re: Check patch failure in vdsm Key: OVIRT-2828 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2828 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Nir Soffer Assignee: infra On Tue, Nov 12, 2019 at 5:34 PM Miguel Duarte de Mora Barroso <mdbarroso(a)redhat.com> wrote: > > On Mon, Nov 11, 2019 at 11:12 AM Eyal Shenitzky <eshenitz(a)redhat.com> wrote: > > > > Hi, > > > > I encounter the following error for py36 test in vdsm check patch: > > > > ... > > ... > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall > > 14:03:37 res = hook_impl.function(*args) > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda> > > 14:03:37 firstresult=hook.spec.opts.get("firstresult") if hook.spec else False, > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec > > 14:03:37 return self._inner_hookexec(hook, methods, kwargs) > > 14:03:37 File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__ > > 14:03:37 return self._hookexec(self, self.get_hookimpls(), kwargs) > > 14:03:37 File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 82, in main > > 14:03:37 return config.hook.pytest_cmdline_main(config=config) > > 14:03:37 File "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/bin/pytest", line 8, in <module> > > 14:03:37 sys.exit(main()) > > 14:03:37 [Inferior 1 (process 22145) detached] > > 14:03:37 ============================================================= > > 14:03:37 = Terminating watched process = > > 14:03:37 ============================================================= > > 14:03:37 PROFILE {"command": ["python", "py-watch", "600", "pytest", "-m", "not (slow or stress)", "--durations=10", "--cov=vdsm.storage", "--cov-report=html:htmlcov-storage-py37", "--cov-fail-under=62", "storage"], "cpu": 39.921942808919184, "elapsed": 604.4699757099152, "idrss": 0, "inblock": 1693453, "isrss": 0, "ixrss": 0, "majflt": 2, "maxrss": 331172, "minflt": 5606489, "msgrcv": 0, "msgsnd": 0, "name": "storage-py37", "nivcsw": 139819, "nsignals": 0, "nswap": 0, "nvcsw": 187576, "oublock": 2495645, "start": 1573386812.7961884, "status": 143, "stime": 118.260961, "utime": 123.055197} > > 14:03:37 ERROR: InvocationError for command /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py37/bin/python profile storage-py37 python py-watch 600 pytest -m 'not (slow or stress)' --durations=10 --cov=vdsm.storage --cov-report=html:htmlcov-storage-py37 --cov-fail-under=62 storage (exited with code 143) > > > > > > Is there any known issue? > > Anyone able to pitch in ? I think something similar is happening in > [0], also on check-patch [1]. > > [0] - https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-pa… Yes it looks the same issue. > [1] - https://gerrit.ovirt.org/#/c/104274/ Jenkins slaves are very slow recently. I suspect we run too many jobs concurrently or using too many virtual cpus. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100114)

1 0

[CQ]: 104490, 4 (vdsm) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 12 Nov '19

12 Nov '19

A system test invoked by the "ovirt-master" change queue including change 104490,4 (vdsm) failed. However, this change seems not to be the root cause for this failure. Change 104040,10 (vdsm) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 104040,10 (vdsm) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/104490/4 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/104040/10 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16858/

1 0

[CQ]: 104491, 4 (vdsm) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 12 Nov '19

12 Nov '19

A system test invoked by the "ovirt-master" change queue including change 104491,4 (vdsm) failed. However, this change seems not to be the root cause for this failure. Change 104040,10 (vdsm) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 104040,10 (vdsm) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/104491/4 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/104040/10 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/16855/

1 0

[JIRA] (OVIRT-2823) Workspace deleted while job was running
by Anton Marchukov (oVirt JIRA) 12 Nov '19

12 Nov '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2823?page=com.atlassian.jira.… ] Anton Marchukov reassigned OVIRT-2823: -------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > Workspace deleted while job was running > --------------------------------------- > > Key: OVIRT-2823 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2823 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Sandro Bonazzola > Assignee: Evgheni Dereveanchin > > missing workspace > /home/jenkins/workspace/ovirt-release_standard-check-patch/ovirt-release on > vm0013.workers-phx.ovirt.org > caused job to fail at > https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-release_standard… > There's something deleting workspaces while they are in use. > Please investigate. > -- > Sandro Bonazzola -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100114)

1 0

[JIRA] (OVIRT-2823) Workspace deleted while job was running
by Anton Marchukov (oVirt JIRA) 12 Nov '19

12 Nov '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2823?page=com.atlassian.jira.… ] Anton Marchukov edited comment on OVIRT-2823 at 11/12/19 12:25 PM: ------------------------------------------------------------------- there is jenkins job system-cleanuo-workspaces, might be related to it. We need to make sure this job is disabled and then removed. was (Author: amarchuk): there is jenkins job system-cleanuo-workspaces, might be related to it. We need to make sure tis job is removed. > Workspace deleted while job was running > --------------------------------------- > > Key: OVIRT-2823 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2823 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Sandro Bonazzola > Assignee: infra > > missing workspace > /home/jenkins/workspace/ovirt-release_standard-check-patch/ovirt-release on > vm0013.workers-phx.ovirt.org > caused job to fail at > https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-release_standard… > There's something deleting workspaces while they are in use. > Please investigate. > -- > Sandro Bonazzola -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100114)

1 0

[JIRA] (OVIRT-2823) Workspace deleted while job was running
by Anton Marchukov (oVirt JIRA) 12 Nov '19

12 Nov '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2823?page=com.atlassian.jira.… ] Anton Marchukov commented on OVIRT-2823: ---------------------------------------- there is jenkins job system-cleanuo-workspaces, might be related to it. We need to make sure tis job is removed. > Workspace deleted while job was running > --------------------------------------- > > Key: OVIRT-2823 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2823 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Sandro Bonazzola > Assignee: infra > > missing workspace > /home/jenkins/workspace/ovirt-release_standard-check-patch/ovirt-release on > vm0013.workers-phx.ovirt.org > caused job to fail at > https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-release_standard… > There's something deleting workspaces while they are in use. > Please investigate. > -- > Sandro Bonazzola -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100114)

1 0