- Infra - oVirt List Archives

[CQ]: 103032,8 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 09 Sep '19

09 Sep '19

Change 103032,8 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/103032/8 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15759/

1 0

[JENKINS] Failed to setup proejct nmstate_kubernetes-nmstate_standard-check-pr
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #589 kubernetes-nmstate [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-5l56s.

1 8

[JENKINS] Failed to setup proejct standard-manual-runner
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #695 vdsm [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-4t4xg.

1 0

[JENKINS] Failed to setup proejct standard-manual-runner
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #695 vdsm [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-cvr0r.

1 0

[JENKINS] Failed to setup proejct standard-manual-runner
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #695 vdsm [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-7xjq2.

1 0

[JENKINS] Failed to setup proejct standard-manual-runner
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #695 vdsm [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-63knw.

1 0

[oVirt] [CQ weekly status] [07-09-2019]
by Dusan Fodor 09 Sep '19

09 Sep '19

Hi, This mail is to provide the current status of CQ and allow people to review status before and after the weekend. Please refer to below colour map for further information on the meaning of the colours. *CQ-4.2*: NOT RELEVANT This CQ is not followed anymore, since it's about to be decommissioned. *CQ-4.3*: YELLOW (#1) Last failure was on 06-09 for vdsm in host installation. It was caused by missing package and solved soon after. *CQ-Master:* YELLOW (#1) Last failure was on 05-09 was random one-time failure on adding storage domain. Current running jobs for 4.2 [1], 4.3 [2] and master [3] can be found here: [1] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt -4.2_change-queue-tester/ [2] https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt -4.3_change-queue-tester/ [3] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt -master_change-queue-tester/ Have a nice week! Dusan ------------------------------------------------------------------------------------------------------------------- COLOUR MAP Green = job has been passing successfully ** green for more than 3 days may suggest we need a review of our test coverage 1. 1-3 days GREEN (#1) 2. 4-7 days GREEN (#2) 3. Over 7 days GREEN (#3) Yellow = intermittent failures for different projects but no lasting or current regressions ** intermittent would be a healthy project as we expect a number of failures during the week ** I will not report any of the solved failures or regressions. 1. Solved job failures YELLOW (#1) 2. Solved regressions YELLOW (#2) Red = job has been failing ** Active Failures. The colour will change based on the amount of time the project/s has been broken. Only active regressions would be reported. 1. 1-3 days RED (#1) 2. 4-7 days RED (#2) 3. Over 7 days RED (#3) _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt .org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt .org/message/YCNCKRK3G4EJXA3OCYAUS4VMKRDA67F4/

3 2

[CQ]: 103112,5 (vdsm) failed "ovirt-master" system tests
by oVirt Jenkins 09 Sep '19

09 Sep '19

Change 103112,5 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/103112/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15755/

1 0

[JIRA] (OVIRT-2794) OST is broken since this morning - looks like infra issue
by Barak Korren (oVirt JIRA) 09 Sep '19

09 Sep '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2794?page=com.atlassian.jira.… ] Barak Korren reassigned OVIRT-2794: ----------------------------------- Assignee: Ehud Yonasi (was: infra) > OST is broken since this morning - looks like infra issue > --------------------------------------------------------- > > Key: OVIRT-2794 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2794 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins Slaves > Reporter: Nir Soffer > Assignee: Ehud Yonasi > Labels: docker > > The last successful build was today at 08:10: > Since then all builds fail very early with the error below - which is not > related to oVirt. > {code} > Removing image: > sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, > force=True > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, > in _raise_for_status > response.raise_for_status() > File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in > raise_for_status > raise HTTPError(http_error_msg, response=self) > requests.exceptions.HTTPError: 404 Client Error: Not Found for url: > http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 349, in <module> > main() > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 37, in main > safe_image_cleanup(client, whitelisted_repos) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 107, in safe_image_cleanup > _safe_rm(client, parent) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 329, in _safe_rm > client.images.remove(image_id, force=force) > File "/usr/lib/python3.6/site-packages/docker/models/images.py", line > 288, in remove > self.client.api.remove_image(*args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line > 19, in wrapped > return f(self, resource_id, *args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in > remove_image > return self._result(res, True) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, > in _result > self._raise_for_status(response) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, > in _raise_for_status > raise create_api_error_from_http_exception(e) > File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in > create_api_error_from_http_exception > raise cls(e, response=response, explanation=explanation) > docker.errors.NotFound: 404 Client Error: Not Found ("reference does not > exist") > Aborting. > Build step 'Execute shell' marked build as failure > {code} > x > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> > #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> > #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> > #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > Sep 5, 2019 3:01 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> > #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > Sep 5, 2019 2:13 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> > #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > Sep 5, 2019 1:58 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> > #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > Sep 5, 2019 1:50 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> > #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > Sep 5, 2019 10:21 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > [image: x] > <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> > [image: Success > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> > #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> > Sep 5, 2019 8:10 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100109)

1 0

[JIRA] (OVIRT-2794) OST is broken since this morning - looks like infra issue
by Barak Korren (oVirt JIRA) 09 Sep '19

09 Sep '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2794?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2794: ------------------------------------- [~accountid:5aa0f39f5a4d022884128a0f] had started testing {{docker_cleanup.py}} on CentOS 7, so assigning the ticket to him. > OST is broken since this morning - looks like infra issue > --------------------------------------------------------- > > Key: OVIRT-2794 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2794 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins Slaves > Reporter: Nir Soffer > Assignee: infra > Labels: docker > > The last successful build was today at 08:10: > Since then all builds fail very early with the error below - which is not > related to oVirt. > {code} > Removing image: > sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, > force=True > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, > in _raise_for_status > response.raise_for_status() > File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in > raise_for_status > raise HTTPError(http_error_msg, response=self) > requests.exceptions.HTTPError: 404 Client Error: Not Found for url: > http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 349, in <module> > main() > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 37, in main > safe_image_cleanup(client, whitelisted_repos) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 107, in safe_image_cleanup > _safe_rm(client, parent) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 329, in _safe_rm > client.images.remove(image_id, force=force) > File "/usr/lib/python3.6/site-packages/docker/models/images.py", line > 288, in remove > self.client.api.remove_image(*args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line > 19, in wrapped > return f(self, resource_id, *args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in > remove_image > return self._result(res, True) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, > in _result > self._raise_for_status(response) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, > in _raise_for_status > raise create_api_error_from_http_exception(e) > File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in > create_api_error_from_http_exception > raise cls(e, response=response, explanation=explanation) > docker.errors.NotFound: 404 Client Error: Not Found ("reference does not > exist") > Aborting. > Build step 'Execute shell' marked build as failure > {code} > x > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> > #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> > #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> > #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > Sep 5, 2019 3:01 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> > #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > Sep 5, 2019 2:13 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> > #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > Sep 5, 2019 1:58 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> > #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > Sep 5, 2019 1:50 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> > #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > Sep 5, 2019 10:21 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > [image: x] > <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> > [image: Success > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> > #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> > Sep 5, 2019 8:10 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100109)

1 0

[JIRA] (OVIRT-2794) OST is broken since this morning - looks like infra issue
by Barak Korren (oVirt JIRA) 09 Sep '19

09 Sep '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2794?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2794: -------------------------------- Labels: docker (was: ) > OST is broken since this morning - looks like infra issue > --------------------------------------------------------- > > Key: OVIRT-2794 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2794 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins Slaves > Reporter: Nir Soffer > Assignee: infra > Labels: docker > > The last successful build was today at 08:10: > Since then all builds fail very early with the error below - which is not > related to oVirt. > {code} > Removing image: > sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, > force=True > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, > in _raise_for_status > response.raise_for_status() > File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in > raise_for_status > raise HTTPError(http_error_msg, response=self) > requests.exceptions.HTTPError: 404 Client Error: Not Found for url: > http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 349, in <module> > main() > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 37, in main > safe_image_cleanup(client, whitelisted_repos) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 107, in safe_image_cleanup > _safe_rm(client, parent) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 329, in _safe_rm > client.images.remove(image_id, force=force) > File "/usr/lib/python3.6/site-packages/docker/models/images.py", line > 288, in remove > self.client.api.remove_image(*args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line > 19, in wrapped > return f(self, resource_id, *args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in > remove_image > return self._result(res, True) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, > in _result > self._raise_for_status(response) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, > in _raise_for_status > raise create_api_error_from_http_exception(e) > File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in > create_api_error_from_http_exception > raise cls(e, response=response, explanation=explanation) > docker.errors.NotFound: 404 Client Error: Not Found ("reference does not > exist") > Aborting. > Build step 'Execute shell' marked build as failure > {code} > x > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> > #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> > #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> > #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > Sep 5, 2019 3:01 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> > #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > Sep 5, 2019 2:13 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> > #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > Sep 5, 2019 1:58 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> > #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > Sep 5, 2019 1:50 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> > #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > Sep 5, 2019 10:21 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > [image: x] > <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> > [image: Success > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> > #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> > Sep 5, 2019 8:10 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100109)

1 0

[JIRA] (OVIRT-2794) OST is broken since this morning - looks like infra issue
by Barak Korren (oVirt JIRA) 09 Sep '19

09 Sep '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2794?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2794: -------------------------------- Components: Jenkins Slaves > OST is broken since this morning - looks like infra issue > --------------------------------------------------------- > > Key: OVIRT-2794 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2794 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins Slaves > Reporter: Nir Soffer > Assignee: infra > > The last successful build was today at 08:10: > Since then all builds fail very early with the error below - which is not > related to oVirt. > {code} > Removing image: > sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, > force=True > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, > in _raise_for_status > response.raise_for_status() > File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in > raise_for_status > raise HTTPError(http_error_msg, response=self) > requests.exceptions.HTTPError: 404 Client Error: Not Found for url: > http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 349, in <module> > main() > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 37, in main > safe_image_cleanup(client, whitelisted_repos) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 107, in safe_image_cleanup > _safe_rm(client, parent) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 329, in _safe_rm > client.images.remove(image_id, force=force) > File "/usr/lib/python3.6/site-packages/docker/models/images.py", line > 288, in remove > self.client.api.remove_image(*args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line > 19, in wrapped > return f(self, resource_id, *args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in > remove_image > return self._result(res, True) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, > in _result > self._raise_for_status(response) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, > in _raise_for_status > raise create_api_error_from_http_exception(e) > File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in > create_api_error_from_http_exception > raise cls(e, response=response, explanation=explanation) > docker.errors.NotFound: 404 Client Error: Not Found ("reference does not > exist") > Aborting. > Build step 'Execute shell' marked build as failure > {code} > x > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> > #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> > #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> > #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > Sep 5, 2019 3:01 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> > #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > Sep 5, 2019 2:13 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> > #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > Sep 5, 2019 1:58 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> > #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > Sep 5, 2019 1:50 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> > #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > Sep 5, 2019 10:21 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > [image: x] > <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> > [image: Success > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> > #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> > Sep 5, 2019 8:10 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100109)

1 0

[JIRA] (OVIRT-2794) OST is broken since this morning - looks like infra issue
by Barak Korren (oVirt JIRA) 09 Sep '19

09 Sep '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2794?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2794: ------------------------------------- This was a bit Puzzling, we've seen issues between {{docker_cleanup.py}} and Docker appear sporadically in the past, and therefore have have made the job code generally not fail when {{docker_cleanup.py}} fails, and instead send an email to the infra list. It turn out that was only true for the V2 code, for the V1 code (which is still used in the manual job and the nightly jobs) thos failures could still arise. We did verify that {{docker_cleanup.py}} works on CentOS 7 with the Python 3 docker API client before merging the patch, so its strange we did not see the issue then. [~accountid:557058:5ca52a09-2675-4285-a044-12ad20f6166a] some of your statements above seem to include some wrong assumption about how the system is built. We're not actually exposing the host's Docker deamon to the CI code, instead we we our own docker instance running inside the container that is used to run the CI code. That way we can ensure there can be no cross-talk when running multiple CI containers on the same hosts. [~accountid:557058:cc1e0e66-9881-45e2-b0b7-ccaa3e60f26e] as far as using podman, I think doing that at this point will be quite a challenge for a number of reasons: # We're currently using OpenShift 3.7 to manage our containers, this implies that we must run Docker on our hosts, since AFAIK OpenShift only started supporting CRIO in 4.0 or 4.1. # To allow CI scripts and tests suits to use Docker we run nested Docker instances inside the CI containers. We know that Docker in Docker work well for our use cases. Running Podman in Docker will probably be more challenging. # Since we're still using {{mock}} to encapsulate the CI script inside the CI container, we're bind-mounting the docker socket from the container into mock. We know there are issues when running Podman in mock, so solving those will take some work. # People that write CI scripts and suits tend to expect things to "just work" in CI like it does on their laptops, and hence tend to use Docker commands. Removing docker will force everyone to learn Podman, and we'll need to make changes everywhere. Out current suspicion is that this issue may have to do with the particular version Docker that is installed inside the CI container. While our {{global_setup.sh}} script generally keeps Docker up to date on the CI slaves, we've intentionally skipped that update code when running in a container. I suspect that the version of Docker that is in the CI containers is older then the once running on the CI slaves. That would explain why we did not see this issue when working on the {{docker_cleanup.py}} patch, since that was tested on the the normal slaves and not the containers. Here is what I think we should do now: # Verify again, that {{docker_cleanup.py}} woks well on CentOS with the Python 3 Docker client API . # If so, inspect the version of Docker we have in the containers and finally # Build an updated container image with a newer version of Docker as needed Note that updating the container image will require us to tests it thoroughly and ensure it can properly run both OST and {{kubevirt-ci}}. > OST is broken since this morning - looks like infra issue > --------------------------------------------------------- > > Key: OVIRT-2794 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2794 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > The last successful build was today at 08:10: > Since then all builds fail very early with the error below - which is not > related to oVirt. > {code} > Removing image: > sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, > force=True > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, > in _raise_for_status > response.raise_for_status() > File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in > raise_for_status > raise HTTPError(http_error_msg, response=self) > requests.exceptions.HTTPError: 404 Client Error: Not Found for url: > http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 349, in <module> > main() > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 37, in main > safe_image_cleanup(client, whitelisted_repos) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 107, in safe_image_cleanup > _safe_rm(client, parent) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 329, in _safe_rm > client.images.remove(image_id, force=force) > File "/usr/lib/python3.6/site-packages/docker/models/images.py", line > 288, in remove > self.client.api.remove_image(*args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line > 19, in wrapped > return f(self, resource_id, *args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in > remove_image > return self._result(res, True) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, > in _result > self._raise_for_status(response) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, > in _raise_for_status > raise create_api_error_from_http_exception(e) > File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in > create_api_error_from_http_exception > raise cls(e, response=response, explanation=explanation) > docker.errors.NotFound: 404 Client Error: Not Found ("reference does not > exist") > Aborting. > Build step 'Execute shell' marked build as failure > {code} > x > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> > #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> > #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> > #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > Sep 5, 2019 3:01 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> > #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > Sep 5, 2019 2:13 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> > #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > Sep 5, 2019 1:58 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> > #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > Sep 5, 2019 1:50 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> > #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > Sep 5, 2019 10:21 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > [image: x] > <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> > [image: Success > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> > #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> > Sep 5, 2019 8:10 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100109)

1 0

[JIRA] (OVIRT-2794) OST is broken since this morning - looks like infra issue
by Barak Korren (oVirt JIRA) 09 Sep '19

09 Sep '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2794?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2794: -------------------------------- Description: The last successful build was today at 08:10: Since then all builds fail very early with the error below - which is not related to oVirt. {code} Removing image: sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, force=True Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, in _raise_for_status response.raise_for_status() File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 349, in <module> main() File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 37, in main safe_image_cleanup(client, whitelisted_repos) File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 107, in safe_image_cleanup _safe_rm(client, parent) File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 329, in _safe_rm client.images.remove(image_id, force=force) File "/usr/lib/python3.6/site-packages/docker/models/images.py", line 288, in remove self.client.api.remove_image(*args, **kwargs) File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped return f(self, resource_id, *args, **kwargs) File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in remove_image return self._result(res, True) File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, in _result self._raise_for_status(response) File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, in _raise_for_status raise create_api_error_from_http_exception(e) File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception raise cls(e, response=response, explanation=explanation) docker.errors.NotFound: 404 Client Error: Not Found ("reference does not exist") Aborting. Build step 'Execute shell' marked build as failure {code} x [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> Sep 5, 2019 3:02 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> Sep 5, 2019 3:02 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> Sep 5, 2019 3:01 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> Sep 5, 2019 2:13 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> Sep 5, 2019 1:58 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> Sep 5, 2019 1:50 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> Sep 5, 2019 10:21 AM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> [image: x] <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> [image: Success > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> Sep 5, 2019 8:10 AM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> was: The last successful build was today at 08:10: Since then all builds fail very early with the error below - which is not related to oVirt. Removing image: sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, force=True Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, in _raise_for_status response.raise_for_status() File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 349, in <module> main() File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 37, in main safe_image_cleanup(client, whitelisted_repos) File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 107, in safe_image_cleanup _safe_rm(client, parent) File "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", line 329, in _safe_rm client.images.remove(image_id, force=force) File "/usr/lib/python3.6/site-packages/docker/models/images.py", line 288, in remove self.client.api.remove_image(*args, **kwargs) File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped return f(self, resource_id, *args, **kwargs) File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in remove_image return self._result(res, True) File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, in _result self._raise_for_status(response) File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, in _raise_for_status raise create_api_error_from_http_exception(e) File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception raise cls(e, response=response, explanation=explanation) docker.errors.NotFound: 404 Client Error: Not Found ("reference does not exist") Aborting. Build step 'Execute shell' marked build as failure x [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> Sep 5, 2019 3:02 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> Sep 5, 2019 3:02 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> Sep 5, 2019 3:01 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> Sep 5, 2019 2:13 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> Sep 5, 2019 1:58 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> Sep 5, 2019 1:50 PM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> [image: Failed > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> Sep 5, 2019 10:21 AM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> [image: x] <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> [image: Success > Console Output] <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> Sep 5, 2019 8:10 AM <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> > OST is broken since this morning - looks like infra issue > --------------------------------------------------------- > > Key: OVIRT-2794 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2794 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Nir Soffer > Assignee: infra > > The last successful build was today at 08:10: > Since then all builds fail very early with the error below - which is not > related to oVirt. > {code} > Removing image: > sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa, > force=True > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 222, > in _raise_for_status > response.raise_for_status() > File "/usr/lib/python3.6/site-packages/requests/models.py", line 893, in > raise_for_status > raise HTTPError(http_error_msg, response=self) > requests.exceptions.HTTPError: 404 Client Error: Not Found for url: > http+docker://localunixsocket/v1.30/images/sha256:f8e5aa8e979155e074411bfef9adade6cdcdf3a5a2eb1d5ad2dbf0288d585ffa?force=True&noprune=False > During handling of the above exception, another exception occurred: > Traceback (most recent call last): > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 349, in <module> > main() > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 37, in main > safe_image_cleanup(client, whitelisted_repos) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 107, in safe_image_cleanup > _safe_rm(client, parent) > File > "/home/jenkins/workspace/ovirt-system-tests_manual/jenkins/scripts/docker_cleanup.py", > line 329, in _safe_rm > client.images.remove(image_id, force=force) > File "/usr/lib/python3.6/site-packages/docker/models/images.py", line > 288, in remove > self.client.api.remove_image(*args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/utils/decorators.py", line > 19, in wrapped > return f(self, resource_id, *args, **kwargs) > File "/usr/lib/python3.6/site-packages/docker/api/image.py", line 481, in > remove_image > return self._result(res, True) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 228, > in _result > self._raise_for_status(response) > File "/usr/lib/python3.6/site-packages/docker/api/client.py", line 224, > in _raise_for_status > raise create_api_error_from_http_exception(e) > File "/usr/lib/python3.6/site-packages/docker/errors.py", line 31, in > create_api_error_from_http_exception > raise cls(e, response=response, explanation=explanation) > docker.errors.NotFound: 404 Client Error: Not Found ("reference does not > exist") > Aborting. > Build step 'Execute shell' marked build as failure > {code} > x > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/console> > #5542 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5542/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/console> > #5541 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > Sep 5, 2019 3:02 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5541/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/console> > #5540 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > Sep 5, 2019 3:01 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5540/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/console> > #5539 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > Sep 5, 2019 2:13 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5539/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/console> > #5538 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > Sep 5, 2019 1:58 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5538/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/console> > #5537 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > Sep 5, 2019 1:50 PM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5537/> > [image: Failed > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/console> > #5536 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > Sep 5, 2019 10:21 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5536/> > [image: x] > <http://jenkins.ovirt.org/job/ovirt-system-tests_manual/jobConfigHistory/sho…> > [image: Success > Console Output] > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/console> > #5535 <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> > Sep 5, 2019 8:10 AM > <https://jenkins.ovirt.org/job/ovirt-system-tests_manual/5535/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100109)

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-ggkn9.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-j60m3.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-x61vs.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-m1lfj.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-hkmdk.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-d7b04.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-d12p9.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-zs1l4.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Failed to run project_setup.sh for: #350 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-mvh2g.

1 0

oVirt infra daily report - unstable production jobs - 942
by jenkins＠jenkins.phx.ovirt.org 09 Sep '19

09 Sep '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/942//artifact/exported-a… Cheers, Jenkins

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-6kz3l.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-tzdrf.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-rqsss.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-81wrx.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-cnlff.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-9g30r.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-1rlr3.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #5446 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-00shm.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-f8bq7.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-kqw5d.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-lmfjn.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-gnr8r.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-7hz33.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-gmd5l.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-dks2q.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-cf376.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_master_standard-poll-upstream-sources
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Failed to run project_setup.sh for: #349 ovirt-system-tests [poll-upstream-sources]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-qmmqd.

1 0

oVirt infra daily report - unstable production jobs - 941
by jenkins＠jenkins.phx.ovirt.org 08 Sep '19

08 Sep '19

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/941//artifact/exported-a… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-fedora-base-fc29-x86_64 #917
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc29-x86_64/91…> ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc29-x86_64/ws/> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 04e85715491aefb65d2ff2546d1770f9c0028945 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 04e85715491aefb65d2ff2546d1770f9c0028945 Commit message: "global-setup: Fix Python package names on Fedora" > git rev-list --no-walk 04e85715491aefb65d2ff2546d1770f9c0028945 # timeout=10 [system-sync_mirrors-fedora-base-fc29-x86_64] $ /bin/bash -xe /tmp/jenkins5556327735720705584.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror fedora-base-fc29 x86_64 jenkins/data/mirrors-reposync.conf + MIRRORS_MP_BASE=/var/www/html/repos + MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos + MIRRORS_CACHE=/home/jenkins/mirrors_cache + MAX_LOCK_ATTEMPTS=120 + LOCK_WAIT_INTERVAL=5 + LOCK_BASE=/home/jenkins + OLD_MD_TO_KEEP=100 + HTTP_SELINUX_TYPE=httpd_sys_content_t + HTTP_FILE_MODE=644 + main resync_yum_mirror fedora-base-fc29 x86_64 jenkins/data/mirrors-reposync.conf + local command=resync_yum_mirror + command_args=("${@:2}") + local command_args + cmd_resync_yum_mirror fedora-base-fc29 x86_64 jenkins/data/mirrors-reposync.conf + local repo_name=fedora-base-fc29 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local sync_needed + mkdir -p /home/jenkins/mirrors_cache + verify_repo_fs fedora-base-fc29 yum + local repo_name=fedora-base-fc29 + local repo_type=yum + sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/fedora-base-fc29 /var/www/html/repos/yum/fedora-base-fc29/base + check_yum_sync_needed fedora-base-fc29 x86_64 jenkins/data/mirrors-reposync.conf sync_needed + local repo_name=fedora-base-fc29 + local repo_archs=x86_64 + local reposync_conf=jenkins/data/mirrors-reposync.conf + local p_sync_needed=sync_needed + local reposync_out + echo 'Checking if mirror needs a resync' Checking if mirror needs a resync + rm -rf /home/jenkins/mirrors_cache/fedora-base-fc29 ++ IFS=, ++ echo x86_64 + for arch in '$(IFS=,; echo $repo_archs)' ++ run_reposync fedora-base-fc29 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet ++ local repo_name=fedora-base-fc29 ++ local repo_arch=x86_64 ++ local reposync_conf=jenkins/data/mirrors-reposync.conf ++ extra_args=("${@:4}") ++ local extra_args ++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=fedora-base-fc29 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/fedora-base-fc29/base --norepopath --newest-only --urls --quiet Traceback (most recent call last): File "/usr/bin/reposync", line 373, in <module> main() File "/usr/bin/reposync", line 185, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1482, in _commonLoadRepoXML result = self._getFileRepoXML(local, text) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1259, in _getFileRepoXML size=102400) # setting max size as 100K File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1025, in _getFile result = self.grab.urlgrab(misc.to_utf8(relative), local, File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 703, in <lambda> grab = property(lambda self: self._getgrab()) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 698, in _getgrab self._setupGrab() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 635, in _setupGrab urls = self.urls File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 881, in <lambda> urls = property(fget=lambda self: self._geturls(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 878, in _geturls self._baseurlSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 824, in _baseurlSetup mirrorurls.extend(list(self.metalink_data.urls())) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 921, in <lambda> metalink_data = property(fget=lambda self: self._getMetalink(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 917, in _getMetalink self._metalink = metalink.MetaLinkRepoMD(self.metalink_filename) File "/usr/lib/python2.7/site-packages/yum/metalink.py", line 185, in __init__ raise MetaLinkRepoErrorParseFail, "File %s does not exist" %filename yum.metalink.MetaLinkRepoErrorParseFail: File /home/jenkins/mirrors_cache/fedora-base-fc29/metalink.xml does not exist + reposync_out='Could not parse metalink https://mirrors.fedoraproject.org/metalink?repo=fedora-29&arch=x86_64 error was File /home/jenkins/mirrors_cache/fedora-base-fc29/metalink.xml.tmp is not XML' Build step 'Execute shell' marked build as failure

1 1

Build failed in Jenkins: system-sync_mirrors-centos-kvm-common-el7-x86_64 #2816
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_…> ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_…> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/changes/59/95959/1:test --prune > git rev-parse origin/test^{commit} # timeout=10 > git rev-parse test^{commit} # timeout=10 Checking out Revision 05b40dfb4ec43a82530e8c471395d1858e5c59e1 (test) > git config core.sparsecheckout # timeout=10 > git checkout -f 05b40dfb4ec43a82530e8c471395d1858e5c59e1 Commit message: "mirror-reposync: remove gluster-3.10 mirror" > git rev-list --no-walk 05b40dfb4ec43a82530e8c471395d1858e5c59e1 # timeout=10 [system-sync_mirrors-centos-kvm-common-el7-x86_64] $ /bin/bash -xe /tmp/jenkins8659319486375862751.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-kvm-common-el7 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 373, in <module> main() File "/usr/bin/reposync", line 185, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1482, in _commonLoadRepoXML result = self._getFileRepoXML(local, text) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1259, in _getFileRepoXML size=102400) # setting max size as 100K File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1025, in _getFile result = self.grab.urlgrab(misc.to_utf8(relative), local, File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 703, in <lambda> grab = property(lambda self: self._getgrab()) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 698, in _getgrab self._setupGrab() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 635, in _setupGrab urls = self.urls File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 881, in <lambda> urls = property(fget=lambda self: self._geturls(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 878, in _geturls self._baseurlSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 824, in _baseurlSetup mirrorurls.extend(list(self.metalink_data.urls())) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 921, in <lambda> metalink_data = property(fget=lambda self: self._getMetalink(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 900, in _getMetalink raise Errors.RepoError, msg yum.Errors.RepoError: Cannot retrieve metalink for repository: fedora-base-fc29. Please verify its path and try again Build step 'Execute shell' marked build as failure

1 1

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

Failed to run project_setup.sh for: #5445 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-rr7r8.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

Failed to run project_setup.sh for: #5444 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-jdl3w.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

Failed to run project_setup.sh for: #5442 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-64w73.

1 1

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

Failed to run project_setup.sh for: #5442 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-5h1s8.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

Failed to run project_setup.sh for: #5441 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-vgvw6.

1 0

[JENKINS] Failed to setup proejct ovirt-system-tests_standard-check-patch
by jenkins＠jenkins.phx.ovirt.org 07 Sep '19

07 Sep '19

Failed to run project_setup.sh for: #5440 ovirt-system-tests [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-integ-tests-container-8pstj.

1 0