September 2017 - Infra - oVirt List Archives

[CQ]: 82390, 1 (ovirt-engine-dashboard) failed "ovirt-master" system tests
by oVirt Jenkins 29 Sep '17

29 Sep '17

Change 82390,1 (ovirt-engine-dashboard) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/82390/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2944/

1 0

[ OST Failure Report ] [ oVirt master ] [ 28-09-2017 ] [ 004_basic_sanity.disk_operations ]
by Dafna Ron 29 Sep '17

29 Sep '17

This is a multi-part message in MIME format. --------------D32DAC9929FA84F802C5B88B Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, We had a failure in OST for basic disk operation. from what I can see in the log, there is an issue deleting a snapshot image and the issue is related to the live storage migration. However, a second pair of eyes would help since I can see engine reporting the error on failed delete while vdsm was not actually reporting a failed delete, at least not as ERROR. Also I don't think the patch that is reported as the cause was actually the cause. I greped the image from the log: http://pastebin.test.redhat.com/520150 * Link to suspected patches: * https://gerrit.ovirt.org/#/c/82293/2 * Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2906 Link to all logs: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2906/artifact/ (Relevant) error snippet from the log: <error> 2017-09-27 16:51:02,760-04 ERROR [org.ovirt.engine.core.bll.DestroyImageCheckCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-6) [8981a0f4-3c02-4a2d-b6ba-f21214b0ebd4] The following images were not removed: [be38d24d-33b9-43f0- 9ab4-15f0334e2ebe] </error> * --------------D32DAC9929FA84F802C5B88B Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> Hi, We had a failure in OST for basic disk operation. from what I can see in the log, there is an issue deleting a snapshot image and the issue is related to the live storage migration. However, a second pair of eyes would help since I can see engine reporting the error on failed delete while vdsm was not actually reporting a failed delete, at least not as ERROR. Also I don't think the patch that is reported as the cause was actually the cause. I greped the image from the log: <a class="moz-txt-link-freetext" href="http://pastebin.test.redhat.com/520150">http://pastebin.test.redhat.com/520150</a> <pre wrap=""> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <pre wrap=""><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/82293/2">https://gerrit.ovirt.org/#/c/82293/2</a></pre> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2906">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2906</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2906/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2906/artifact/</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error> 2017-09-27 16:51:02,760-04 ERROR [org.ovirt.engine.core.bll.DestroyImageCheckCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-6) [8981a0f4-3c02-4a2d-b6ba-f21214b0ebd4] The following images were not removed: [be38d24d-33b9-43f0- 9ab4-15f0334e2ebe] <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error> </pre> </body> </html> --------------D32DAC9929FA84F802C5B88B--

1 2

[oVirt Jenkins] standard-enqueue - Build #5331 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 29 Sep '17

29 Sep '17

------=_Part_848_849090528.1506688767265 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/standard-enqueue/5331/ Build Name: #5331 Build Description: Gerrit: 82382 - cockpit-ovirt (4.2) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/82382 - title: Allowing user to select any disk type and creating gdeploy config file - project: cockpit-ovirt - branch: ovirt-4.2 - author: Sandro Bonazzola <sbonazzo(a)redhat.com> ------=_Part_848_849090528.1506688767265--

1 0

[CQ]: 82379, 1 (ovirt-engine-metrics) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 29 Sep '17

29 Sep '17

A system test invoked by the "ovirt-master" change queue including change 82379,1 (ovirt-engine-metrics) failed. However, this change seems not to be the root cause for this failure. Change 82329,5 (ovirt-engine-metrics) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 82329,5 (ovirt-engine-metrics) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/82379/1 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/82329/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2941/

1 0

Build failed in Jenkins: system-sync_mirrors-centos-updates-el7-x86_64 #829
by jenkins＠jenkins.phx.ovirt.org 29 Sep '17

29 Sep '17

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/changes/13/75913/5:patch --prune > git rev-parse origin/patch^{commit} # timeout=10 > git rev-parse patch^{commit} # timeout=10 Checking out Revision 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f (patch) Commit message: "Exclude big packages from mirrors" > git config core.sparsecheckout # timeout=10 > git checkout -f 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f > git rev-list 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f # timeout=10 [system-sync_mirrors-centos-updates-el7-x86_64] $ /bin/bash -xe /tmp/jenkins5464824030326316176.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1478, in _commonLoadRepoXML self._revertOldRepoXML() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1323, in _revertOldRepoXML os.rename(old_data['old_local'], old_data['local']) OSError: [Errno 2] No such file or directory Build step 'Execute shell' marked build as failure

1 1

[ OST Failure Report ] [ oVirt master ] [ 28-09-2017 ] [ 003_00_metrics_bootstrap.configure_metrics ]
by Dafna Ron 29 Sep '17

29 Sep '17

This is a multi-part message in MIME format. --------------CCBA598F31714CCB8129AF90 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, We had failure in OST last night which seems to be related to two fluentd configuration patchs. I found that the error is actually in the deployment and can be found in the lago log. I am not sure it's related but I am also seeing unexpected errors in the vdsm log for getDeviceList: http://pastebin.test.redhat.com/520514 We apparently calling getDeviceList + checkStatus=true with no guid. ** *Link to suspected patches: * * **https://gerrit.ovirt.org/#/c/82329/ * * ** *https://gerrit.ovirt.org/#/c/81551/* Link to Job: *http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/* *http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/* Link to all logs: *http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/artifact/** **http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/artifact/** *** * *(Relevant) error snippet from the log: * * <error> TASK [ovirt_fluentd/client : Install non-ssl client] *************************** fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'fluentd_num_threads' is undefined"} fatal: [lago-basic-suite-master-host-0]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'fluentd_num_threads' is undefined"} fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'fluentd_num_threads' is undefined"} </error> * --------------CCBA598F31714CCB8129AF90 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> Hi, We had failure in OST last night which seems to be related to two fluentd configuration patchs. I found that the error is actually in the deployment and can be found in the lago log. I am not sure it's related but I am also seeing unexpected errors in the vdsm log for getDeviceList: <a class="moz-txt-link-freetext" href="http://pastebin.test.redhat.com/520514">http://pastebin.test.redhat.com/520514</a> We apparently calling getDeviceList + checkStatus=true with no guid. <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/82329/">https://gerrit.ovirt.org/#/c/82329/</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/81551/">https://gerrit.ovirt.org/#/c/81551/</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/artifact/</a> <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/artifact/</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error> TASK [ovirt_fluentd/client : Install non-ssl client] *************************** fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'fluentd_num_threads' is undefined"} fatal: [lago-basic-suite-master-host-0]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'fluentd_num_threads' is undefined"} fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'fluentd_num_threads' is undefined"} <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error> </body> </html> --------------CCBA598F31714CCB8129AF90--

3 3

[oVirt Jenkins] ovirt-node-ng_ovirt-4.1-snapshot_build-artifacts-el7-x86_64 - Build # 268 - Failure!
by jenkins＠jenkins.phx.ovirt.org 29 Sep '17

29 Sep '17

------=_Part_828_280597221.1506602431684 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.1-snapshot_build-artifac… Build: http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.1-snapshot_build-artifac… Build Number: 268 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #268 [Yuval Turgeman] Copying os-release from node image to stage2 ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_828_280597221.1506602431684--

1 1

Build failed in Jenkins: system-sync_mirrors-centos-updates-el7-x86_64 #827
by jenkins＠jenkins.phx.ovirt.org 29 Sep '17

29 Sep '17

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/changes/13/75913/5:patch --prune > git rev-parse origin/patch^{commit} # timeout=10 > git rev-parse patch^{commit} # timeout=10 Checking out Revision 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f (patch) Commit message: "Exclude big packages from mirrors" > git config core.sparsecheckout # timeout=10 > git checkout -f 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f > git rev-list 4b0fe3e0c9fba26cdbaafe2b29fddd3411225d6f # timeout=10 [system-sync_mirrors-centos-updates-el7-x86_64] $ /bin/bash -xe /tmp/jenkins6223189949086079841.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML if self._latestRepoXML(local): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1443, in _latestRepoXML repomd = self.metalink_data.repomd File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 916, in <lambda> metalink_data = property(fget=lambda self: self._getMetalink(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 912, in _getMetalink self._metalink = metalink.MetaLinkRepoMD(self.metalink_filename) File "/usr/lib/python2.7/site-packages/yum/metalink.py", line 189, in __init__ raise MetaLinkRepoErrorParseFail, "File %s is not XML" % filename yum.metalink.MetaLinkRepoErrorParseFail: File /home/jenkins/mirrors_cache/fedora-updates-fc25/metalink.xml is not XML Build step 'Execute shell' marked build as failure

1 1

oVirt infra daily report - unstable production jobs - 458
by jenkins＠jenkins.phx.ovirt.org 29 Sep '17

29 Sep '17

------=_Part_836_1112201525.1506639606004 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/458//artifact/exported-a… Cheers, Jenkins ------=_Part_836_1112201525.1506639606004 Content-Type: text/html; charset=us-ascii; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 28/09/2017 </th></tr><tr><th colspan=2 class=sub> <a href="http://jenkins.ovirt.org/">00 Unstable Critical</a> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/">ovirt-system-tests_he-basic-suite-4.1</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_836_1112201525.1506639606004--

1 0

[CQ]: 81551, 6 (ovirt-engine-metrics) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 28 Sep '17

28 Sep '17

A system test invoked by the "ovirt-master" change queue including change 81551,6 (ovirt-engine-metrics) failed. However, this change seems not to be the root cause for this failure. Change 82329,5 (ovirt-engine-metrics) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 82329,5 (ovirt-engine-metrics) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/81551/6 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/82329/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/

1 0