- Infra - oVirt List Archives

[CQ]: 101915, 18 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 26 Aug '19

26 Aug '19

A system test invoked by the "ovirt-master" change queue including change 101915,18 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 101913,10 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 101913,10 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/101915/18 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/101913/10 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15507/

1 0

oVirt infra daily report - unstable production jobs - 928
by jenkins＠jenkins.phx.ovirt.org 26 Aug '19

26 Aug '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/928//artifact/exported-a… Cheers, Jenkins

1 0

[JIRA] (OVIRT-2783) CI fails - tox error
by Nir Soffer (oVirt JIRA) 25 Aug '19

25 Aug '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2783?page=com.atlassian.jira.… ] Nir Soffer commented on OVIRT-2783: ----------------------------------- On Sun, Aug 25, 2019, 15:19 Pavel Bar <pbar(a)redhat.com> wrote: > Hi, > We experience problems with failed CI on VDSM patches. > Does it work locally and in travis? Examples: > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10633/ > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10634/ > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10635/ > > This is what we see in logs: > > out=`tox --version`; \ > if [ $? -ne 0 ]; then \ > echo "Error: cannot run tox, please install tox \ > 2.9.1 or later"; \ > exit 1; \ > fi; \ > version=`echo $out | cut -d' ' -f1`; \ > if python2.7 build-aux/vercmp $version 2.9.1; then \ > echo "Error: tox is too old, please install tox \ > 2.9.1 or later"; \ > exit 1; \ > fi > Traceback (most recent call last): > File "/usr/bin/tox", line 7, in <module> > from tox import cmdline > File "/usr/lib/python2.7/site-packages/tox/__init__.py", line 4, in > <module> > from .hookspecs import hookimpl > File "/usr/lib/python2.7/site-packages/tox/hookspecs.py", line 4, in > <module> > from pluggy import HookimplMarker > File "/usr/lib/python2.7/site-packages/pluggy/__init__.py", line 16, in > <module> > from .manager import PluginManager, PluginValidationError > File "/usr/lib/python2.7/site-packages/pluggy/manager.py", line 6, in > <module> > import importlib_metadata > File "/usr/lib/python2.7/site-packages/importlib_metadata/__init__.py", > line 9, in <module> > import zipp > File "/usr/lib/python2.7/site-packages/zipp.py", line 12, in <module> > import more_itertools > File "/usr/lib/python2.7/site-packages/more_itertools/__init__.py", line > 1, in <module> > from more_itertools.more import * # noqa > File "/usr/lib/python2.7/site-packages/more_itertools/more.py", line 340 > def _collate(*iterables, key=lambda a: a, reverse=False): > ^ > SyntaxError: invalid syntax > Error: cannot run tox, please install tox 2.9.1 or later > make: *** [tox] Error 1 > + teardown > + res=2 > + '[' 2 -ne 0 ']' > + echo '*** err: 2' > *** err: 2 > + collect_logs > + tar --directory /var/log --exclude 'journal/*' -czf > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/mock_varlogs.tar.gz > . > + tar --directory /var/host_log --exclude 'journal/*' -czf > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/host_varlogs.tar.gz > . > + teardown_storage > + python2 tests/storage/userstorage.py teardown > > Can someone look at it? > > Thank you in advance! > > Pavel > > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/VADZM35IRVN4E… > > CI fails - tox error > -------------------- > > Key: OVIRT-2783 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2783 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Pavel Bar > Assignee: infra > > Hi, > We experience problems with failed CI on VDSM patches. > Examples: > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10633/ > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10634/ > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10635/ > This is what we see in logs: > out=`tox --version`; \ > if [ $? -ne 0 ]; then \ > echo "Error: cannot run tox, please install tox \ > 2.9.1 or later"; \ > exit 1; \ > fi; \ > version=`echo $out | cut -d' ' -f1`; \ > if python2.7 build-aux/vercmp $version 2.9.1; then \ > echo "Error: tox is too old, please install tox \ > 2.9.1 or later"; \ > exit 1; \ > fi > Traceback (most recent call last): > File "/usr/bin/tox", line 7, in <module> > from tox import cmdline > File "/usr/lib/python2.7/site-packages/tox/__init__.py", line 4, in > <module> > from .hookspecs import hookimpl > File "/usr/lib/python2.7/site-packages/tox/hookspecs.py", line 4, in > <module> > from pluggy import HookimplMarker > File "/usr/lib/python2.7/site-packages/pluggy/__init__.py", line 16, in > <module> > from .manager import PluginManager, PluginValidationError > File "/usr/lib/python2.7/site-packages/pluggy/manager.py", line 6, in > <module> > import importlib_metadata > File "/usr/lib/python2.7/site-packages/importlib_metadata/__init__.py", > line 9, in <module> > import zipp > File "/usr/lib/python2.7/site-packages/zipp.py", line 12, in <module> > import more_itertools > File "/usr/lib/python2.7/site-packages/more_itertools/__init__.py", line > 1, in <module> > from more_itertools.more import * # noqa > File "/usr/lib/python2.7/site-packages/more_itertools/more.py", line 340 > def _collate(*iterables, key=lambda a: a, reverse=False): > ^ > SyntaxError: invalid syntax > Error: cannot run tox, please install tox 2.9.1 or later > make: *** [tox] Error 1 > + teardown > + res=2 > + '[' 2 -ne 0 ']' > + echo '*** err: 2' > *** err: 2 > + collect_logs > + tar --directory /var/log --exclude 'journal/*' -czf > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/mock_varlogs.tar.gz > . > + tar --directory /var/host_log --exclude 'journal/*' -czf > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/host_varlogs.tar.gz > . > + teardown_storage > + python2 tests/storage/userstorage.py teardown > Can someone look at it? > Thank you in advance! > Pavel -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100107)

1 0

[CQ]: 102129, 13 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 25 Aug '19

25 Aug '19

A system test invoked by the "ovirt-master" change queue including change 102129,13 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 101913,10 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 101913,10 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/102129/13 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/101913/10 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15501/

1 0

[JIRA] (OVIRT-2783) CI fails - tox error
by Eyal Edri (oVirt JIRA) 25 Aug '19

25 Aug '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2783?page=com.atlassian.jira.… ] Eyal Edri commented on OVIRT-2783: ---------------------------------- It looks like a syntax error, before it can’t install tox: File "/usr/lib/python2.7/site-packages/more_itertools/more.py", line 340def _collate(*iterables, key=lambda a: a, reverse=False):^SyntaxError: invalid syntax > CI fails - tox error > -------------------- > > Key: OVIRT-2783 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2783 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Pavel Bar > Assignee: infra > > Hi, > We experience problems with failed CI on VDSM patches. > Examples: > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10633/ > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10634/ > http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10635/ > This is what we see in logs: > out=`tox --version`; \ > if [ $? -ne 0 ]; then \ > echo "Error: cannot run tox, please install tox \ > 2.9.1 or later"; \ > exit 1; \ > fi; \ > version=`echo $out | cut -d' ' -f1`; \ > if python2.7 build-aux/vercmp $version 2.9.1; then \ > echo "Error: tox is too old, please install tox \ > 2.9.1 or later"; \ > exit 1; \ > fi > Traceback (most recent call last): > File "/usr/bin/tox", line 7, in <module> > from tox import cmdline > File "/usr/lib/python2.7/site-packages/tox/__init__.py", line 4, in > <module> > from .hookspecs import hookimpl > File "/usr/lib/python2.7/site-packages/tox/hookspecs.py", line 4, in > <module> > from pluggy import HookimplMarker > File "/usr/lib/python2.7/site-packages/pluggy/__init__.py", line 16, in > <module> > from .manager import PluginManager, PluginValidationError > File "/usr/lib/python2.7/site-packages/pluggy/manager.py", line 6, in > <module> > import importlib_metadata > File "/usr/lib/python2.7/site-packages/importlib_metadata/__init__.py", > line 9, in <module> > import zipp > File "/usr/lib/python2.7/site-packages/zipp.py", line 12, in <module> > import more_itertools > File "/usr/lib/python2.7/site-packages/more_itertools/__init__.py", line > 1, in <module> > from more_itertools.more import * # noqa > File "/usr/lib/python2.7/site-packages/more_itertools/more.py", line 340 > def _collate(*iterables, key=lambda a: a, reverse=False): > ^ > SyntaxError: invalid syntax > Error: cannot run tox, please install tox 2.9.1 or later > make: *** [tox] Error 1 > + teardown > + res=2 > + '[' 2 -ne 0 ']' > + echo '*** err: 2' > *** err: 2 > + collect_logs > + tar --directory /var/log --exclude 'journal/*' -czf > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/mock_varlogs.tar.gz > . > + tar --directory /var/host_log --exclude 'journal/*' -czf > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/host_varlogs.tar.gz > . > + teardown_storage > + python2 tests/storage/userstorage.py teardown > Can someone look at it? > Thank you in advance! > Pavel -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100107)

1 0

[CQ]: 101914, 12 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 25 Aug '19

25 Aug '19

A system test invoked by the "ovirt-master" change queue including change 101914,12 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 101913,10 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 101913,10 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/101914/12 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/101913/10 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15500/

1 0

[JIRA] (OVIRT-2783) CI fails - tox error
by Pavel Bar (oVirt JIRA) 25 Aug '19

25 Aug '19

Pavel Bar created OVIRT-2783: -------------------------------- Summary: CI fails - tox error Key: OVIRT-2783 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2783 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Pavel Bar Assignee: infra Hi, We experience problems with failed CI on VDSM patches. Examples: http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10633/ http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10634/ http://jenkins.ovirt.org/job/vdsm_standard-check-patch/10635/ This is what we see in logs: out=`tox --version`; \ if [ $? -ne 0 ]; then \ echo "Error: cannot run tox, please install tox \ 2.9.1 or later"; \ exit 1; \ fi; \ version=`echo $out | cut -d' ' -f1`; \ if python2.7 build-aux/vercmp $version 2.9.1; then \ echo "Error: tox is too old, please install tox \ 2.9.1 or later"; \ exit 1; \ fi Traceback (most recent call last): File "/usr/bin/tox", line 7, in <module> from tox import cmdline File "/usr/lib/python2.7/site-packages/tox/__init__.py", line 4, in <module> from .hookspecs import hookimpl File "/usr/lib/python2.7/site-packages/tox/hookspecs.py", line 4, in <module> from pluggy import HookimplMarker File "/usr/lib/python2.7/site-packages/pluggy/__init__.py", line 16, in <module> from .manager import PluginManager, PluginValidationError File "/usr/lib/python2.7/site-packages/pluggy/manager.py", line 6, in <module> import importlib_metadata File "/usr/lib/python2.7/site-packages/importlib_metadata/__init__.py", line 9, in <module> import zipp File "/usr/lib/python2.7/site-packages/zipp.py", line 12, in <module> import more_itertools File "/usr/lib/python2.7/site-packages/more_itertools/__init__.py", line 1, in <module> from more_itertools.more import * # noqa File "/usr/lib/python2.7/site-packages/more_itertools/more.py", line 340 def _collate(*iterables, key=lambda a: a, reverse=False): ^ SyntaxError: invalid syntax Error: cannot run tox, please install tox 2.9.1 or later make: *** [tox] Error 1 + teardown + res=2 + '[' 2 -ne 0 ']' + echo '*** err: 2' *** err: 2 + collect_logs + tar --directory /var/log --exclude 'journal/*' -czf /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/mock_varlogs.tar.gz . + tar --directory /var/host_log --exclude 'journal/*' -czf /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/exported-artifacts/host_varlogs.tar.gz . + teardown_storage + python2 tests/storage/userstorage.py teardown Can someone look at it? Thank you in advance! Pavel -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100107)

1 0

[CQ]: 102693, 2 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 25 Aug '19

25 Aug '19

A system test invoked by the "ovirt-master" change queue including change 102693,2 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 101913,10 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 101913,10 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/102693/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/101913/10 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15499/

1 0

[oVirt] [CQ weekly status] [25-08-2019]
by Dusan Fodor 25 Aug '19

25 Aug '19

Hi, This mail is to provide the current status of CQ and allow people to review status before and after the weekend. Please refer to below colour map for further information on the meaning of the colours. *CQ-4.2*: GREEN (#2) Last failure was on 20-08 for ovirt-ansible-hosted-engine-setup. It was caused by random network issue (fetching repo from github), retriggered job for the change passed well. *CQ-4.3*: GREEN (#2) Last failure was on 20-08 for ovirt-ansible-hosted-engine-setup. It was caused by random network issue, retriggered job for the change passed well. *CQ-Master:* RED (#1) Last failure was on 25-08 for ovirt-engine, caused by failed cloning of engine repo (even with lately increased timeout value 20min). Related ticket from last time was reopened. Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found here: [1] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt -4.2_change-queue-tester/ [2] https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt -4.3_change-queue-tester/ [3] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt -master_change-queue-tester/ Have a nice week! Dusan ------------------------------------------------------------------------------------------------------------------- COLOUR MAP Green = job has been passing successfully ** green for more than 3 days may suggest we need a review of our test coverage 1. 1-3 days GREEN (#1) 2. 4-7 days GREEN (#2) 3. Over 7 days GREEN (#3) Yellow = intermittent failures for different projects but no lasting or current regressions ** intermittent would be a healthy project as we expect a number of failures during the week ** I will not report any of the solved failures or regressions. 1. Solved job failures YELLOW (#1) 2. Solved regressions YELLOW (#2) Red = job has been failing ** Active Failures. The colour will change based on the amount of time the project/s has been broken. Only active regressions would be reported. 1. 1-3 days RED (#1) 2. 4-7 days RED (#2) 3. Over 7 days RED (#3) _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt .org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt .org/message/YCNCKRK3G4EJXA3OCYAUS4VMKRDA67F4/

1 0

[JIRA] (OVIRT-2782) Configure Jenkins to wait less and allocate more
by Daniel Belenky (oVirt JIRA) 25 Aug '19

25 Aug '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2782?page=com.atlassian.jira.… ] Daniel Belenky updated OVIRT-2782: ---------------------------------- Description: By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting {code:java} hudson.slaves.NodeProvisioner.initialDelay=0 {code} We can also control the excessive workload threshold and force Jenkins to allocate slaves for jobs in the queue... but, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we set it's margins to higher values and it effectively lowers its threshold so that Jenkins will allocate nodes faster and in advance. The recommended settings by the k8s plugin to spawn a node for every build in the queue: {code:java} -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 {code} was: By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting {code:java} hudson.slaves.NodeProvisioner.initialDelay=0 {code} We can also control the excessive workload threshold and force Jenkins to allocate slaves for jobs in the queue... but, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we set it's margins to higher values and it effectively lowers its threshold so that Jenkins will allocate nodes faster and in advance. The recommended settings by the k8s plugin to spawn a node for every build in the queue: ``` By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance. {code:java} -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 {code} > Configure Jenkins to wait less and allocate more > ------------------------------------------------ > > Key: OVIRT-2782 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2782 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Daniel Belenky > Assignee: infra > > By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting > {code:java} > hudson.slaves.NodeProvisioner.initialDelay=0 > {code} > We can also control the excessive workload threshold and force Jenkins to allocate slaves for jobs in the queue... but, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we set it's margins to higher values and it effectively lowers its threshold so that Jenkins will allocate nodes faster and in advance. > The recommended settings by the k8s plugin to spawn a node for every build in the queue: > {code:java} > -Dhudson.slaves.NodeProvisioner.initialDelay=0 > -Dhudson.slaves.NodeProvisioner.MARGIN=50 > -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 > {code} -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100107)

1 0