April 2018 - Infra - oVirt List Archives

Gerrit refrence not getting added to Bug
by Ravi Shankar Nori 08 Apr '18

08 Apr '18

Hi, Looks like there is an issue with automation Gerrit reference is not being automatically added to Bugzilla when a patch is submitted to gerrit. Thanks Ravi

2 1

Build failed in Jenkins: system-sync_mirrors-centos-updates-el7-x86_64 #1443
by jenkins＠jenkins.phx.ovirt.org 08 Apr '18

08 Apr '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 12408d1e905e40783b966b5c73293f29a2a2ad1b (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 12408d1e905e40783b966b5c73293f29a2a2ad1b Commit message: "infra-ansible: re-align" > git rev-list --no-walk 12408d1e905e40783b966b5c73293f29a2a2ad1b # timeout=10 [system-sync_mirrors-centos-updates-el7-x86_64] $ /bin/bash -xe /tmp/jenkins8387130590586695659.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-el7' Build step 'Execute shell' marked build as failure

1 1

[CQ]: 84981,13 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 07 Apr '18

07 Apr '18

Change 84981,13 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/84981/13 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6716/

1 0

[CQ]: 85701, 20 (ovirt-provider-ovn) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 06 Apr '18

06 Apr '18

A system test invoked by the "ovirt-master" change queue including change 85701,20 (ovirt-provider-ovn) failed. However, this change seems not to be the root cause for this failure. Change 89581,3 (ovirt-provider-ovn) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 89581,3 (ovirt-provider-ovn) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/85701/20 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/89581/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6714/

1 0

OST Failure - Weekly update [30/03/2018-06/04/2018]
by Dafna Ron 06 Apr '18

06 Apr '18

Hello, I would like to update on this week's failures and OST current status. We had a few failures this week and some are still on-going issues. We had two unrelated issues failing the test: 006_migrations.prepare_migration_attachments_ipv6 This failure was periodical and is causing failure for several changes. This issue is still on-going and seems to be pointing to this change as the root cause: https://gerrit.ovirt.org/#/c/89812/1 - examples: export template to an export domain Issue is still under investigation. On Thursday evening and Friday we seem to have had failures which indicate issues with migration and ipv6 configuration, The tests that failed were 006_migrations.migrate_vm and 004_basic_sanity.hotplug_cpu cpu hotplug failure was reporting these changes: Failed patch: virt: storage: minimal changes to the drive XML - https://gerrit.ovirt.org/#/c/89508/ Reported as failure cause: virt: extract local adjustments to XML - https://gerrit.ovirt.org/#/c/89506/ and migration failure was reporting this patch: *aaa: can't switch user when accessing the engine with an active kerberos ticket - https://gerrit.ovirt.org/#/c/89872/ <https://gerrit.ovirt.org/#/c/89872/>* *This three issues are still on-going and you can see the cases resolved vs the backlog which is going forward to next week: * *Infra/OST tests related issues: * We also had a failure which seems to be related to the ost test and was fixed in this patch: https://gerrit.ovirt.org/#/c/89781/ - Prevent network to be imported by auto_sync Further discussion on this can be found in the google doc opened by Dominik: https://docs.google.com/document/d/1VeKfI7luw-HHTCCmfogLNiYLTtIG_3maENVQlDw… *Below you can see the chart for this week's resolved issues but cause of failure:* *Code* = regression of working components/functionalities *Infra* = infrastructure/OST Infrastructure/Lago related issues/Power outages *OST Tests* - package related issues, failed build artifacts *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * Thanks, Dafna

2 1

[ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 06-04-2018 ] [ 006_migrations.migrate_vm ]
by Dafna Ron 06 Apr '18

06 Apr '18

hi, 006_migrations.migrate_vm failure points to the below patch. There seems to be an issue to migrate vms but I am not sure its related to this patch. can you please check? *Link and headline of suspected patches: aaa: can't switch user when accessing the engine with an active kerberos ticket - https://gerrit.ovirt.org/#/c/89872/ <https://gerrit.ovirt.org/#/c/89872/>Link to Job:* * http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6704/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6704/>Link to all logs:* http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6704/artifact… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6704/> *(Relevant) error snippet from the log: <error>2018-04-06 04:07:56,517-0400 ERROR (qgapoller/2) [Executor] Unhandled exception in <Task discardable <NetworkInterfacesCheck vm=aad106a1-9043-4661-bb9b-9894f66b506b at 0x7fc4681a2590> timeout=10, duration=0.00 at 0x7fc4541193d0> (executor:317)Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task task() File "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__ self._callable() File "/usr/lib/python2.7/site-packages/vdsm/virt/periodic.py", line 314, in __call__ self._execute() File "/usr/lib/python2.7/site-packages/vdsm/virt/qemuguestagent.py", line 388, in _execute for ifname, ifparams in six.iteritems(interfaces): File "/usr/lib/python2.7/site-packages/six.py", line 599, in iteritems return d.iteritems(**kw)AttributeError: 'list' object has no attribute 'iteritems'*2018-04-06 04:23:10,165-0400 ERROR (migsrc/aad106a1) [virt.vm] (vmId='aad106a1-9043-4661-bb9b-9894f66b506b') migration destination error: Virtual machine already exists (migration:290) *</error>*

2 2

[oVirt Jenkins] ovirt-node-ng_master_build-artifacts-el7-x86_64 - Build # 750 - Failure!
by jenkins＠jenkins.phx.ovirt.org 06 Apr '18

06 Apr '18

------=_Part_1135_1836134680.1523012359754 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build: http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6… Build Number: 750 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #750 [Ryan Barry] image-update: fail upgrades if there is a local storage domain on / ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_1135_1836134680.1523012359754--

1 1

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1364
by jenkins＠jenkins.phx.ovirt.org 06 Apr '18

06 Apr '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1364/displ…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 205666e21777874f4d41c930cf14962dd86ce4c4 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 205666e21777874f4d41c930cf14962dd86ce4c4 Commit message: "Workaround for a race in gerrit-trigger plugin" > git rev-list --no-walk 205666e21777874f4d41c930cf14962dd86ce4c4 # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins2728499843727993467.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1474, in _commonLoadRepoXML if self._latestRepoXML(local): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1452, in _latestRepoXML repomd = self.metalink_data.repomd File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 918, in <lambda> metalink_data = property(fget=lambda self: self._getMetalink(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 906, in _getMetalink shutil.move(result, self.metalink_filename) File "/usr/lib64/python2.7/shutil.py", line 301, in move copy2(src, real_dst) File "/usr/lib64/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/usr/lib64/python2.7/shutil.py", line 82, in copyfile with open(src, 'rb') as fsrc: IOError: [Errno 2] No such file or directory: '/home/jenkins/mirrors_cache/fedora-updates-fc26/metalink.xml.tmp' Build step 'Execute shell' marked build as failure

1 1

[oVirt Jenkins] ovirt-system-tests_he-basic-ansible-suite-master - Build # 139 - Failure!
by jenkins＠jenkins.phx.ovirt.org 06 Apr '18

06 Apr '18

------=_Part_1061_1792029887.1522854492337 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mast… Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-mast… Build Number: 139 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #139 [Gal Ben Haim] he: Share control.sh between all the non-ansible suites [Daniel Belenky] Workaround for a race in gerrit-trigger plugin ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_1061_1792029887.1522854492337--

4 9

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.1 - Build # 277 - Failure!
by jenkins＠jenkins.phx.ovirt.org 06 Apr '18

06 Apr '18

------=_Part_1110_834586657.1522937532175 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.1/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.1/277/ Build Number: 277 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #277 [Yaniv Kaul] Disable 006_migrations.prepare_migration_attachments_ipv6 ----------------- Failed Tests: ----------------- No tests ran. ------=_Part_1110_834586657.1522937532175--

1 1