December 2018 - Infra - oVirt List Archives

[JIRA] (OVIRT-2533) multi-stage support in CI slaves
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2533?page=com.atlassian.jira.… ] Eyal Edri updated OVIRT-2533: ----------------------------- Issue Type: Improvement (was: By-EMAIL) > multi-stage support in CI slaves > -------------------------------- > > Key: OVIRT-2533 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2533 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Roy Golan > Assignee: infra > > I would like to get multi-stage support upstream. Our default docker > packages for centos 7.5 is 1.13. For multi-stage support we need > 1.17 or > have buildah working in mock (preferable over docker, won't happen now I > guess?). > Using this method[1] to install docker-ce I was able to get 1.18 running. > Doable? > Regards, > Roy > [1] https://docs.docker.com/install/linux/docker-ce/centos/ -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2538) ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
by Greg Sheremeta (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jira.… ] Greg Sheremeta commented on OVIRT-2538: --------------------------------------- It depends on how often it happens. There are 2 things, generally, that can go wrong in the test. 1 gwt doesn't load at all, and 2 there are timeouts when waiting for gwt to change pages when menu items are clicked. we are pretty resilient when it comes to problem 2, but maybe .05% of time the graceful handling of this could fail. we don't deal with 1 other than silently fail the test at the beginning, and you have to look at the screenshot and take a guess that that's what happened. [I believe that's what happened above.] So, the test could be more graceful. It can be upgraded to perform detection of this condition (gwt doesn't load) and do further inspection (is the gwt script there? is it valid javascript? can we try again? etc.) > ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome > --------------------------------------------------------------------------- > > Key: OVIRT-2538 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: High > Labels: ost_failures, ost_race > > I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes. > Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change > https://gerrit.ovirt.org/#/c/89572/ > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/ > I cannot see anything in the http logs and in engine logs. > I can see a warn on ssl and some issues which seems to be resolved later on domain names: > '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name' > I think the issue is the test since the only errors I can see are in the lago logs. > I saved logs to allow debugging of the test failure. > [~gbenhaim(a)redhat.com] > [~gshereme(a)redhat.com] > Can you please check this issue? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2602) Add a CI mirror for CentOS Storage SIG - Gluster 5
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2602?page=com.atlassian.jira.… ] Eyal Edri reassigned OVIRT-2602: -------------------------------- Assignee: Ehud Yonasi (was: infra) Ehud, I can't see the sync is done, please send relevant patches to add it. > Add a CI mirror for CentOS Storage SIG - Gluster 5 > -------------------------------------------------- > > Key: OVIRT-2602 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2602 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: Ehud Yonasi > > Gluster 5 is going to be officially released in CentOS 7.6 GA. > Repositories already exists on mirror.centos.org > Please add them to our CI mirroring so we can start consume them in our CI > environment. > CCing Sahina and Niels to let them know. > [centos-gluster5] > name=CentOS-$releasever - Gluster 5 > mirrorlist= > http://mirrorlist.centos.org?arch=$basearch&release=$releasever&repo=storag… > #baseurl= > http://mirror.centos.org/$contentdir/$releasever/storage/$basearch/gluster-… > gpgcheck=1 > enabled=1 > gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage > Thanks, > -- > SANDRO BONAZZOLA > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2620) Mirroring error
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2620?page=com.atlassian.jira.… ] Eyal Edri updated OVIRT-2620: ----------------------------- Resolution: Done Status: Done (was: To Do) I think this is by design, the sync job can fail if the target repo isn't finished running or something like that and we rely on the next job to get a working fix. The next sync job works, so no action items here AFAIU. > Mirroring error > --------------- > > Key: OVIRT-2620 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2620 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: infra > > *Hi,* > *looks like in our mirror script there's a bug, we don't clean the > <repo>/gen directory before starting the mirroring* > *04:35:12* yum.Errors.RepoError: Error making cache directory: > /home/jenkins/mirrors_cache/centos-kvm-common-el7/gen error was: > [Errno 17] File exists: > '/home/jenkins/mirrors_cache/centos-kvm-common-el7/gen' > Job: > https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86… > -- > SANDRO BONAZZOLA > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2632) No space left on device
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2632?page=com.atlassian.jira.… ] Eyal Edri reassigned OVIRT-2632: -------------------------------- Assignee: Galit Rosenthal (was: infra) [~ederevea] do we have monitoring/cleanup on the Jenkins VM slaves that can alert us if space is running our on the filesystem? if not, can we add it? maybe to the global setup.sh? [~bkorren(a)redhat.com] [~grosenth(a)redhat.com] I see a newer build is failing on timeout downloading from the fedora mirror, few options to check: 1. increase timeout for the CURL command 2. See if we can mirror/proxy these images on our infra 3. Check with [~ederevea] if we have any known network issues between Fedora to PHX DC. > No space left on device > ----------------------- > > Key: OVIRT-2632 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2632 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: Galit Rosenthal > > Hi, > looks like on some slave there's not enough space for handling the jobs: > https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-fc28-x… > Can you please check if we need larger disks or if it's a case of cleanup > failing? > In both cases, please take action on fixing the issue. > -- > SANDRO BONAZZOLA > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2055) failure in suspend resume vm not related to reported change - need to investigate
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2055?page=com.atlassian.jira.… ] Eyal Edri updated OVIRT-2055: ----------------------------- Resolution: Cannot Reproduce Status: Done (was: To Do) old issue, please reopen if we still see it so developers can debug it. > failure in suspend resume vm not related to reported change - need to investigate > --------------------------------------------------------------------------------- > > Key: OVIRT-2055 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2055 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Labels: ost_failures, ost_race > > Change 91507,1 (ovirt-engine-sdk-java) is probably the reason behind recent > system test failures in the "ovirt-master" change queue and needs to be fixed. > This change had been removed from the testing queue. Artifacts build from this > change will not be released until it is fixed. > For further details about the change see: > https://gerrit.ovirt.org/#/c/91507/1 > For failed test results see: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7721/ > I think the issue is on qemu/libvirt changed queries: > https://pastebin.com/afyeUAcT > https://pastebin.com/va20Y5Hp > checking his with vdsm people. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2202) 006_migrations.prepare_migration_attachments_ipv4 for ovirt-engine-dashboard
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2202?page=com.atlassian.jira.… ] Eyal Edri updated OVIRT-2202: ----------------------------- Resolution: Fixed Status: Done (was: To Do) from the JIRA ticket seems this might need a bz on RHV, closing as we don't see this anymore, if it will keep failing again, we'll reconsult with network team how to handle it. > 006_migrations.prepare_migration_attachments_ipv4 for ovirt-engine-dashboard > ---------------------------------------------------------------------------- > > Key: OVIRT-2202 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2202 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Labels: ost_failures, ost_race > > There was a failed test 006_migrations.prepare_migration_attachments_ipv4 > Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML > Build: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8208/ > patch reported: https://gerrit.ovirt.org/#/c/92238/1 - add resolutions section to force jquery and bootstrap versions > The actual error is: Unable to process messages Broken pipe: java.io.IOException: Broken pipe > this would be the error in vdsm: > '2018-06-14 04:05:34,698-0400 ERROR (jsonrpc/5) [vds] connectivity check failed (API:1537) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1534, in setupNetworks > supervdsm.getProxy().setupNetworks(networks, bondings, options) > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__ > return callMethod() > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda> > **kwargs) > File "<string>", line 2, in setupNetworks > File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod > raise convert_to_error(kind, result) > ConfigNetworkError: (10, 'connectivity check failed') > 2018-06-14 04:05:34,699-0400 INFO (jsonrpc/5) [api.network] FINISH setupNetworks return={'status': {'message': 'connectivity check failed', 'code': 10}} from=::ffff:192.168.201.4,50952 (api:53) > 2018-06-14 04:05:34,699-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.setupNetworks failed (error 10) in 127.20 seconds (__init__:311)' > The patch may be related so I will as Greg to take a look -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2538) ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jira.… ] Eyal Edri commented on OVIRT-2538: ---------------------------------- [~gshereme(a)redhat.com] [~dron] is this still an issue worth investigating? race maybe? > ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome > --------------------------------------------------------------------------- > > Key: OVIRT-2538 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: High > Labels: ost_failures, ost_race > > I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes. > Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change > https://gerrit.ovirt.org/#/c/89572/ > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/ > I cannot see anything in the http logs and in engine logs. > I can see a warn on ssl and some issues which seems to be resolved later on domain names: > '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name' > I think the issue is the test since the only errors I can see are in the lago logs. > I saved logs to allow debugging of the test failure. > [~gbenhaim(a)redhat.com] > [~gshereme(a)redhat.com] > Can you please check this issue? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

[JIRA] (OVIRT-2634) all ovirt projects failing due to 001_upgrade_engine.test_initialize_engine
by Eyal Edri (oVirt JIRA) 25 Dec '18

25 Dec '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2634?page=com.atlassian.jira.… ] Eyal Edri updated OVIRT-2634: ----------------------------- Resolution: Fixed Status: Done (was: To Do) CQ is passing now. > all ovirt projects failing due to 001_upgrade_engine.test_initialize_engine > --------------------------------------------------------------------------- > > Key: OVIRT-2634 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2634 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: Highest > Labels: ost_code_regression, ost_failures > > we fail on updating schema. > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/12052/ > regression caused by https://gerrit.ovirt.org/#/c/96040/9 > Fix introduced: https://gerrit.ovirt.org/#/c/96371/ > We have a long queue of patches and fix is still in untested queue so it may take some time to pass -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100096)

1 0

Build failed in Jenkins: system-sync_mirrors-fedora-base-fc29-x86_64 #142
by jenkins＠jenkins.phx.ovirt.org 25 Dec '18

25 Dec '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc29-x86_64/14…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc29-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision cfe4d8cfee724d0179eb657e00980116f4c34414 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f cfe4d8cfee724d0179eb657e00980116f4c34414 Commit message: "Print out node information" > git rev-list --no-walk cfe4d8cfee724d0179eb657e00980116f4c34414 # timeout=10 [system-sync_mirrors-fedora-base-fc29-x86_64] $ /bin/bash -xe /tmp/jenkins230601372153969823.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror fedora-base-fc29 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-kvm-common-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-kvm-common-el7' Build step 'Execute shell' marked build as failure

1 1