- Infra - oVirt List Archives

oVirt infra weekly meeting notes 22.08.2018
by Evgheni Dereveanchin 22 Aug '18

22 Aug '18

Hi everyone, Please find the topics of this week's infra meeting below: PHX: - Jenkins outage reported on Tuesday OVIRT-2445 <https://ovirt-jira.atlassian.net/browse/OVIRT-2445> - Jenkins updated OVIRT-2438 <https://ovirt-jira.atlassian.net/browse/OVIRT-2438> - Resources updated to ensure newest kernel is in place OVIRT-2338 <https://ovirt-jira.atlassian.net/browse/OVIRT-2338> - HE bug root cause identified BZ#1613278 <https://bugzilla.redhat.com/show_bug.cgi?id=1613278> - Storage01 connected to internal VLANs and rebuilt OVIRT-2420 <https://ovirt-jira.atlassian.net/browse/OVIRT-2420> - OpenShift login fixed, update to 3.10 planned OVIRT-2436 <https://ovirt-jira.atlassian.net/browse/OVIRT-2436> Website - Documentation will be moved under a separate vhost -- Regards, Evgheni Dereveanchin

1 0

[JIRA] (OVIRT-2264) Cannot deactivate Storage while there are running tasks on this Storage
by Eyal Edri (oVirt JIRA) 22 Aug '18

22 Aug '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2264?page=com.atlassian.jira.… ] Eyal Edri commented on OVIRT-2264: ---------------------------------- [~tnisan(a)redhat.com] is leading the oVirt storage team, he should be able to direct you to an engineer who can debug it. > Cannot deactivate Storage while there are running tasks on this Storage > ----------------------------------------------------------------------- > > Key: OVIRT-2264 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2264 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Labels: ost_failures, ost_race > > we failed test: 007_sd_reattach.deactivate_storage_domain in vdsm project with error: Cannot deactivate Storage while there are running tasks on this Storage. -Please wait until tasks will finish and try again.]". HTTP response code is 409. > I am opening this as a race in case something changed in OST tests which would cause this to repeat. > https://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/2506 > https://gerrit.ovirt.org/#/c/92313/2 -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100090)

1 0

[JIRA] (OVIRT-2264) Cannot deactivate Storage while there are running tasks on this Storage
by Galit Rosenthal (oVirt JIRA) 22 Aug '18

22 Aug '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2264?page=com.atlassian.jira.… ] Galit Rosenthal commented on OVIRT-2264: ---------------------------------------- I understand these tests are not related to him, Who else can we contact? > Cannot deactivate Storage while there are running tasks on this Storage > ----------------------------------------------------------------------- > > Key: OVIRT-2264 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2264 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Labels: ost_failures, ost_race > > we failed test: 007_sd_reattach.deactivate_storage_domain in vdsm project with error: Cannot deactivate Storage while there are running tasks on this Storage. -Please wait until tasks will finish and try again.]". HTTP response code is 409. > I am opening this as a race in case something changed in OST tests which would cause this to repeat. > https://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/2506 > https://gerrit.ovirt.org/#/c/92313/2 -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100090)

1 0

[JIRA] (OVIRT-2448) update OpenShift to OKD 3.10
by Evgheni Dereveanchin (oVirt JIRA) 22 Aug '18

22 Aug '18

Evgheni Dereveanchin created OVIRT-2448: ------------------------------------------- Summary: update OpenShift to OKD 3.10 Key: OVIRT-2448 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2448 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: infra As there were issues with stability of the 3.9 deployment (OVIRT-2436) our envs should be updated to OKD 3.10 -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100090)

1 0

[JIRA] (OVIRT-2001) Create FAQ/Knowledge Base/HOWTO pages for STDCI
by Liora Milbaum (oVirt JIRA) 22 Aug '18

22 Aug '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2001?page=com.atlassian.jira.… ] Liora Milbaum commented on OVIRT-2001: -------------------------------------- I can certainly try > Create FAQ/Knowledge Base/HOWTO pages for STDCI > ----------------------------------------------- > > Key: OVIRT-2001 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2001 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Documentation > Reporter: Barak Korren > Assignee: infra > Labels: first_time_task > > There are many common use cases and issues in STDCI that could benefit from case/task-specific documentation. So we should have a FAQ page or a Knowledge Base about STDCI to cover these issues. > Before we can make the page we need a critical mass (at least 3) of issues and solutions, so we will use comments on this ticket to collect them as they arise. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100090)

1 0

[JIRA] (OVIRT-2001) Create FAQ/Knowledge Base/HOWTO pages for STDCI
by Barak Korren (oVirt JIRA) 22 Aug '18

22 Aug '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2001?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2001: ------------------------------------- [~lmilbaum(a)redhat.com] do you think you can take on this ticket? > Create FAQ/Knowledge Base/HOWTO pages for STDCI > ----------------------------------------------- > > Key: OVIRT-2001 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2001 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Documentation > Reporter: Barak Korren > Assignee: infra > Labels: first_time_task > > There are many common use cases and issues in STDCI that could benefit from case/task-specific documentation. So we should have a FAQ page or a Knowledge Base about STDCI to cover these issues. > Before we can make the page we need a critical mass (at least 3) of issues and solutions, so we will use comments on this ticket to collect them as they arise. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100090)

1 0

[JIRA] (OVIRT-2001) Create FAQ/Knowledge Base/HOWTO pages for STDCI
by Barak Korren (oVirt JIRA) 22 Aug '18

22 Aug '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2001?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2001: ------------------------------------- Hers is some text posted elsewhere describing how to re-trigger post-merge events from GitHub: {quote}[~gbenhaim(a)redhat.com] the way to do this is: # Go to the 'standard-webhook-trigger' job build that triggered the stuck job (in this case [#1345|https://jenkins.ovirt.org/job/standard-webhook-trigger/1345/]) # The build description includes something like 'push \[79309ea2-a53f-11e8-92b7-a759f1132bfd]' - this is the event ID from Github. # Go to the [repo on GitHub|https://github.com/ovirt/ovirt-ansible-image-template] and from there go to Settings->Webhooks and click on the webhook with the ' http://jenkins.ovirt.org/generic-webhook-trigger/invoke' URL. You will be required to enter your gitHub password to move beyond this point. # In the "Recnet Deliveries" list belowe click on the '...' button next to the event ID we saw in the 'standard-webhook-trigger' job build # Click on 'Redeliver' In this case I already did the above for the relevant patch. {quote} > Create FAQ/Knowledge Base/HOWTO pages for STDCI > ----------------------------------------------- > > Key: OVIRT-2001 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2001 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Documentation > Reporter: Barak Korren > Assignee: infra > Labels: first_time_task > > There are many common use cases and issues in STDCI that could benefit from case/task-specific documentation. So we should have a FAQ page or a Knowledge Base about STDCI to cover these issues. > Before we can make the page we need a critical mass (at least 3) of issues and solutions, so we will use comments on this ticket to collect them as they arise. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100090)

1 0

[CQ]: 93745,3 (imgbased) failed "ovirt-master" system tests
by oVirt Jenkins 22 Aug '18

22 Aug '18

Change 93745,3 (imgbased) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/93745/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/9190/

1 0

Build failed in Jenkins: system-sync_mirrors-centos-updates-el7-x86_64 #1866
by jenkins＠jenkins.phx.ovirt.org 22 Aug '18

22 Aug '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 89a51f35237a574500539f1299c589067ac5faac (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 89a51f35237a574500539f1299c589067ac5faac Commit message: "Remove Ruby SDK build for s390x" > git rev-list --no-walk 89a51f35237a574500539f1299c589067ac5faac # timeout=10 [system-sync_mirrors-centos-updates-el7-x86_64] $ /bin/bash -xe /tmp/jenkins395495011305010798.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 96, in retrieveAllMD downloading = repo._commonRetrieveDataMD_list(mdtypes) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1618, in _commonRetrieveDataMD_list os.rename(local, local + '.old.tmp') OSError: [Errno 2] No such file or directory Build step 'Execute shell' marked build as failure

1 1

[JIRA] (OVIRT-2202) 006_migrations.prepare_migration_attachments_ipv4 for ovirt-engine-dashboard
by Alona Kaplan (oVirt JIRA) 22 Aug '18

22 Aug '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2202?page=com.atlassian.jira.… ] Alona Kaplan commented on OVIRT-2202: ------------------------------------- Hi, I've already talked about it with Dafna. The last race she found is caused because of an event sent from the vdsm. Due to a recent fix (https://bugzilla.redhat.com/1590109) now the event mechanism actually works in the engine. And if the engine gets event from the vdsm a getCaps is invoked on the host. Since it's unpredictable when an event will arrive (it can happen when setting dhcp on a host nic, when the dhcp lease is renewed, ...). We have an open jira task planned for RHV sprint 4 to handle the issue in the OST - https://projects.engineering.redhat.com/browse/RHV-2165 > 006_migrations.prepare_migration_attachments_ipv4 for ovirt-engine-dashboard > ---------------------------------------------------------------------------- > > Key: OVIRT-2202 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2202 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Labels: ost_failures, ost_race > > There was a failed test 006_migrations.prepare_migration_attachments_ipv4 > Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML > Build: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8208/ > patch reported: https://gerrit.ovirt.org/#/c/92238/1 - add resolutions section to force jquery and bootstrap versions > The actual error is: Unable to process messages Broken pipe: java.io.IOException: Broken pipe > this would be the error in vdsm: > '2018-06-14 04:05:34,698-0400 ERROR (jsonrpc/5) [vds] connectivity check failed (API:1537) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1534, in setupNetworks > supervdsm.getProxy().setupNetworks(networks, bondings, options) > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__ > return callMethod() > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda> > **kwargs) > File "<string>", line 2, in setupNetworks > File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod > raise convert_to_error(kind, result) > ConfigNetworkError: (10, 'connectivity check failed') > 2018-06-14 04:05:34,699-0400 INFO (jsonrpc/5) [api.network] FINISH setupNetworks return={'status': {'message': 'connectivity check failed', 'code': 10}} from=::ffff:192.168.201.4,50952 (api:53) > 2018-06-14 04:05:34,699-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.setupNetworks failed (error 10) in 127.20 seconds (__init__:311)' > The patch may be related so I will as Greg to take a look -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100090)

1 0