- Infra - oVirt List Archives

[JIRA] (OVIRT-2015) CQ marks two different changes in two different projects as cause of failure - both are not the cause
by eyal edri (oVirt JIRA) 13 May '18

13 May '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2015?page=com.atlassian.jira.… ] eyal edri commented on OVIRT-2015: ---------------------------------- CQ can also fail on OST test change, which is not (yet) a CQ object, you have to take that into account when debugging failures. so I'm not sure this is a real issue. [~bkorren(a)redhat.com] > CQ marks two different changes in two different projects as cause of failure - both are not the cause > ----------------------------------------------------------------------------------------------------- > > Key: OVIRT-2015 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2015 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Labels: CQ_false_report > > we have what seems to be a code regression in deploy ovn on upgrade suite: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/artifact… > The failure caused failure in two different projects otopi+imgbased: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/ > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7488/ > CQ marked a change in each of the projects as the cause of failure: > Otopi: > For further details about the change see: > https://gerrit.ovirt.org/#/c/88930/2 > imgbase; > For further details about the change see: > https://gerrit.ovirt.org/#/c/90692/43 > Since the cause of failure is the same for these two projects and the changes themselves do not seem like they can be the cause of failure I am reporting this ticket for further investigation. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100083)

1 0

[JIRA] (OVIRT-2001) Create FAQ/Knowledge Base for STDCI
by Daniel Belenky (oVirt JIRA) 13 May '18

13 May '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2001?page=com.atlassian.jira.… ] Daniel Belenky commented on OVIRT-2001: --------------------------------------- Issue when users try to use Docker in their builds from a Fedora chroot running on an el7 slave: The docker (client) version in Fedora repositories is not backward compatible with the docker (server) version in centos repositories. The solution is to run such jobs on slaves where the distro is same or newer than the chroot's distro. It can be specified in STDCI configuration file: {code:java} --- runtime-requirements: host-distro: newer {code} > Create FAQ/Knowledge Base for STDCI > ----------------------------------- > > Key: OVIRT-2001 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2001 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Documentation > Reporter: Barak Korren > Assignee: infra > Labels: first_time_task > > There are many common use cases and issues in STDCI that could benefit from case/task-specific documentation. So we should have a FAQ page or a Knowledge Base about STDCI to cover these issues. > Before we can make the page we need a critical mass (at least 3) of issues and solutions, so we will use comments on this ticket to collect them as they arise. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100083)

1 0

[JIRA] (OVIRT-1759) Gerrit patches can reach CQ out-of-order
by Barak Korren (oVirt JIRA) 13 May '18

13 May '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1759?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1759: ------------------------------------- [~eedri] this is not a new ticket - and the cost of fixing this properly is high, so it really depends on how likely is this to happen in practice. In the meantime we can try to invest some time in reporting the issue to the upstreams. We need to: 1. Setup a job that runs on gerrit merge events with no parallel processing but finishes quickly. 2. Submit a chain of patches 3. See if/when the job processes patchs out-of-order 4. Check if the events reach us out of order from Gerrit - in that case a issue needs to be reported to Gerrit 5. Check if events reaches in-order from Gerrti but the job was scheduled out-of-order - in this case issue needs to be reported on the Gerrit plugin. > Gerrit patches can reach CQ out-of-order > ---------------------------------------- > > Key: OVIRT-1759 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1759 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Change Queue, Gerrit/git > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: change-queue, gerrit > > Is seems that despite the fact that all the jobs involved were designed to preserve proper patch ordering, there can be cases where patches trigger Jenkins jobs in the wrong order, which in turn end up putting the patches in the wrong order into the CQ. This in turn, causes the CQ to fail accurately detecting the patches that cause issues. > Here is en example of such an occurance. The following patches were merged in the order specified below: > # https://gerrit.ovirt.org/c/83917 > # https://gerrit.ovirt.org/c/83918 > # https://gerrit.ovirt.org/c/83919 > # https://gerrit.ovirt.org/c/83920 > The standard-enqueue runs putting them in the CQ however, seem to have run in a different order: > # http://jenkins.ovirt.org/job/standard-enqueue/6712/ (83917) > # http://jenkins.ovirt.org/job/standard-enqueue/6713/ (83919) > # http://jenkins.ovirt.org/job/standard-enqueue/6714/ (83920) > # http://jenkins.ovirt.org/job/standard-enqueue/6715/ (83918) > This resulted in CQ 'add' command running in the wrong order: > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12200/ (add 83917) > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12201/ (add 83919) > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12202/ (add 83920) > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12203/ (add 83918) > We need to come up with a way to make the system properly preserver order. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100083)

1 0

[JIRA] (OVIRT-2016) Mailman footer does not show proper URL
by eyal edri (oVirt JIRA) 13 May '18

13 May '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2016?page=com.atlassian.jira.… ] eyal edri reassigned OVIRT-2016: -------------------------------- Assignee: Marc Dequènes (Duck) (was: infra) > Mailman footer does not show proper URL > --------------------------------------- > > Key: OVIRT-2016 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2016 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Mailing lists (Mailman) > Reporter: Emil Natan > Assignee: Marc Dequènes (Duck) > Priority: Low > > ovirt-devel mailing list (devel(a)ovirt.org) does not show proper url in the footer: > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100083)

1 0

[JIRA] (OVIRT-2016) Mailman footer does not show proper URL
by Emil Natan (oVirt JIRA) 13 May '18

13 May '18

Emil Natan created OVIRT-2016: --------------------------------- Summary: Mailman footer does not show proper URL Key: OVIRT-2016 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2016 Project: oVirt - virtualization made easy Issue Type: Bug Components: Mailing lists (Mailman) Reporter: Emil Natan Assignee: infra Priority: Low ovirt-devel mailing list (devel(a)ovirt.org) does not show proper url in the footer: Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100083)

1 0

[JIRA] (OVIRT-1759) Gerrit patches can reach CQ out-of-order
by eyal edri (oVirt JIRA) 13 May '18

13 May '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1759?page=com.atlassian.jira.… ] eyal edri commented on OVIRT-1759: ---------------------------------- Thanks for tracking this. please prioritize, we believed that this will be over once we move out of the old nightly jobs, and some maintainers also hoped for it. > Gerrit patches can reach CQ out-of-order > ---------------------------------------- > > Key: OVIRT-1759 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1759 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Change Queue, Gerrit/git > Reporter: Barak Korren > Assignee: infra > Priority: High > Labels: change-queue, gerrit > > Is seems that despite the fact that all the jobs involved were designed to preserve proper patch ordering, there can be cases where patches trigger Jenkins jobs in the wrong order, which in turn end up putting the patches in the wrong order into the CQ. This in turn, causes the CQ to fail accurately detecting the patches that cause issues. > Here is en example of such an occurance. The following patches were merged in the order specified below: > # https://gerrit.ovirt.org/c/83917 > # https://gerrit.ovirt.org/c/83918 > # https://gerrit.ovirt.org/c/83919 > # https://gerrit.ovirt.org/c/83920 > The standard-enqueue runs putting them in the CQ however, seem to have run in a different order: > # http://jenkins.ovirt.org/job/standard-enqueue/6712/ (83917) > # http://jenkins.ovirt.org/job/standard-enqueue/6713/ (83919) > # http://jenkins.ovirt.org/job/standard-enqueue/6714/ (83920) > # http://jenkins.ovirt.org/job/standard-enqueue/6715/ (83918) > This resulted in CQ 'add' command running in the wrong order: > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12200/ (add 83917) > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12201/ (add 83919) > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12202/ (add 83920) > # http://jenkins.ovirt.org/job/ovirt-master_change-queue/12203/ (add 83918) > We need to come up with a way to make the system properly preserver order. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100083)

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1477
by jenkins＠jenkins.phx.ovirt.org 13 May '18

13 May '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1477/displ…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 66094b4d3d86dd9fc473191f5f07798f0b7499b4 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 66094b4d3d86dd9fc473191f5f07798f0b7499b4 Commit message: "Drop obselete gluster 3.7,3.8 repos from mirrors" > git rev-list --no-walk 66094b4d3d86dd9fc473191f5f07798f0b7499b4 # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins5358794788832145685.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-el7' Build step 'Execute shell' marked build as failure

1 1

Build failed in Jenkins: system-sync_mirrors-centos-updates-el7-x86_64 #1552
by jenkins＠jenkins.phx.ovirt.org 12 May '18

12 May '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-updates-el7-x86_64/…> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 66094b4d3d86dd9fc473191f5f07798f0b7499b4 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 66094b4d3d86dd9fc473191f5f07798f0b7499b4 Commit message: "Drop obselete gluster 3.7,3.8 repos from mirrors" > git rev-list --no-walk 66094b4d3d86dd9fc473191f5f07798f0b7499b4 # timeout=10 [system-sync_mirrors-centos-updates-el7-x86_64] $ /bin/bash -xe /tmp/jenkins8953595643583965298.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-updates-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-el7' Build step 'Execute shell' marked build as failure

1 1

[CQ]: 90858, 5 (ovirt-engine-dashboard) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 11 May '18

11 May '18

A system test invoked by the "ovirt-master" change queue including change 90858,5 (ovirt-engine-dashboard) failed. However, this change seems not to be the root cause for this failure. Change 90729,5 (ovirt-engine-dashboard) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 90729,5 (ovirt-engine- dashboard) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/90858/5 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/90729/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7543/

1 0

[ OST Failure Report ] [ oVirt Master (otopi+imgbased) ] [ 11-05-2018 ] [ 001_initialize_engine.test_initialize_engine ]
by Dafna Ron 11 May '18

11 May '18

Hi, We are failing in 001_initialize_engine.test_initialize_engine in the upgrade suite. the issue seems to be related to ovn configuration. The changes reported by CQ are not the cause of this failure and I may be mistaken but I suspect it may be related to one of the below changes. *Link and headline of suspected patches: * *https://gerrit.ovirt.org/#/c/90784/ <https://gerrit.ovirt.org/#/c/90784/> - network: default ovn provider client is returned by fixturehttps://gerrit.ovirt.org/#/c/90327/ <https://gerrit.ovirt.org/#/c/90327/> - backend, packing: Add default MTU for tunnelled networksLink to Job:* *http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/>* http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7488/ *Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7492/artifact…>(Relevant) error snippet from the log: <error>* 2018-05-11 04:14:34,940-0400 DEBUG otopi.context context._executeMethod:143 method exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py", line 779, in _customization self._query_install_ovn() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/network/ovirtproviderovn.py", line 399, in _query_install_ovn default=True File "/usr/lib/python2.7/site-packages/ovirt_setup_lib/dialog.py", line 47, in queryBoolean default=true if default else false, File "/usr/share/otopi/plugins/otopi/dialog/human.py", line 211, in queryString value = self._readline(hidden=hidden) File "/usr/lib/python2.7/site-packages/otopi/dialog.py", line 248, in _readline raise IOError(_('End of file')) IOError: End of file 2018-05-11 04:14:34,942-0400 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Environment customization': End of file 2018-05-11 04:14:34,972-0400 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: id uid local foreign state pid exe 0: 0 0.0.0.0:111 0.0.0.0:0 LISTEN 1829 /usr/sbin/rpcbind 1: 29 0.0.0.0:662 0.0.0.0:0 LISTEN 1868 /usr/sbin/rpc.statd 2: 0 0.0.0.0:22 0.0.0.0:0 LISTEN 970 /usr/sbin/sshd 3: 0 192.168.201.2:3260 0.0.0.0:0 LISTEN UnknownPID UnknownEXE 4: 0 192.168.200.2:3260 0.0.0.0:0 LISTEN UnknownPID UnknownEXE 5: 0 0.0.0.0:892 0.0.0.0:0 LISTEN 1874 /usr/sbin/rpc.mountd 6: 0 0.0.0.0:2049 0.0.0.0:0 LISTEN UnknownPID UnknownEXE 7: 0 0.0.0.0:32803 0.0.0.0:0 LISTEN UnknownPID UnknownEXE 8: 0 192.168.201.2:22 192.168.201.1:33338 ESTABLISHED 5544 /usr/sbin/sshd 2018-05-11 04:14:34,973-0400 DEBUG otopi.context context.dumpEnvironment:859 ENVIRONMENT DUMP - BEGIN 2018-05-11 04:14:34,973-0400 DEBUG otopi.context context.dumpEnvironment:869 ENV BASE/error=bool:'True' 2018-05-11 04:14:34,973-0400 DEBUG otopi.context context.dumpEnvironment:869 ENV BASE/exceptionInfo=list:'[(<type 'exceptions.IOError'>, IOError('End of file',), <traceback object at 0x239ab90>)]' 2018-05-11 04:14:34,974-0400 DEBUG otopi.context context.dumpEnvironment:873 ENVIRONMENT DUMP - END 2018-05-11 04:14:34,975-0400 INFO otopi.context context.runSequence:741 Stage: Clean up 2018-05-11 04:14:34,975-0400 DEBUG otopi.context context.runSequence:745 STAGE cleanup 2018-05-11 04:14:34,976-0400 DEBUG otopi.context context._executeMethod:128 Stage cleanup METHOD otopi.plugins.otopi.dialog.answer_file.Plugin._generate_answer_file 2018-05-11 04:14:34,977-0400 DEBUG otopi.context context.dumpEnvironment:859 ENVIRONMENT DUMP - BEGIN 2018-05-11 04:14:34,977-0400 DEBUG otopi.context context.dumpEnvironment:869 ENV DIALOG/answerFileContent=str:'# OTOPI answer file, generated by human dialog [environment:default] ' *</error>Thanks, Dafna*

3 3