- Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_compat-4.2-suite-master - Build # 184 - Failure!
by jenkins＠jenkins.phx.ovirt.org 22 Sep '18

22 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/184/ Build Number: 184 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #184 [Sahina Bose] hc: Fixing ansible version for 4.2 as well ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_basic_ui_sanity.initialize_firefox_42 Error Message: Message: Error forwarding the new session new session request for webdriver should contain a location header or an 'application/json;charset=UTF-8' response body with the session ID. Stacktrace: at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117) at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84) at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68) at javax.servlet.http.HttpServlet.service (HttpServlet.java:707) at javax.servlet.http.HttpServlet.service (HttpServlet.java:790) at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860) at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168) at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473) at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166) at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155) at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141) at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132) at org.seleniumhq.jetty9.server.Server.handle (Server.java:530) at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347) at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256) at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279) at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102) at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131) at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626) at java.lang.Thread.run (Thread.java:748) -------------------- >> begin captured stdout << --------------------- _init_browser, connecting to hub at http://172.18.0.2:4444/wd/hub Message: Error forwarding the new session new session request for webdriver should contain a location header or an 'application/json;charset=UTF-8' response body with the session ID. Stacktrace: at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117) at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84) at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68) at javax.servlet.http.HttpServlet.service (HttpServlet.java:707) at javax.servlet.http.HttpServlet.service (HttpServlet.java:790) at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860) at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188) at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168) at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473) at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564) at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166) at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155) at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141) at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132) at org.seleniumhq.jetty9.server.Server.handle (Server.java:530) at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347) at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256) at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279) at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102) at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140) at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131) at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708) at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626) at java.lang.Thread.run (Thread.java:748) WebDriverException in _init_browser connecting to hub --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- selenium.webdriver.remote.remote_connection: DEBUG: POST http://172.18.0.2:4444/wd/hub/session {"capabilities": {"alwaysMatch": {"acceptInsecureCerts": true, "browserName": "firefox", "platformName": "linux", "browserVersion": "58.0.1"}, "firstMatch": [{}]}, "desiredCapabilities": {"acceptInsecureCerts": true, "platform": "LINUX", "browserName": "firefox", "version": "58.0.1", "marionette": true}} urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): 172.18.0.2:4444 urllib3.connectionpool: DEBUG: http://172.18.0.2:4444 "POST /wd/hub/session HTTP/1.1" 500 8055 selenium.webdriver.remote.remote_connection: DEBUG: Finished Request --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/home/jenkins/workspace/ovirt-system-tests_compat-4.2-suite-master/ovirt-system-tests/compat-4.2-suite-master/test-scenarios/008_basic_ui_sanity.py", line 219, in initialize_firefox _init_driver(_get_firefox_capabilities()) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.2-suite-master/ovirt-system-tests/compat-4.2-suite-master/test-scenarios/008_basic_ui_sanity.py", line 239, in _init_driver raise e 'Message: Error forwarding the new session new session request for webdriver should contain a location header or an \'application/json;charset=UTF-8\' response body with the session ID.\nStacktrace:\n at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117)\n at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84)\n at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)\n at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141)\n at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)\n at org.seleniumhq.jetty9.server.Server.handle (Server.java:530)\n at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347)\n at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256)\n at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)\n at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102)\n at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131)\n at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626)\n at java.lang.Thread.run (Thread.java:748)\n-------------------- >> begin captured stdout << ---------------------\n_init_browser, connecting to hub at http://172.18.0.2:4444/wd/hub\nMessage: Error forwarding the new session new session request for webdriver should contain a location header or an \'application/json;charset=UTF-8\' response body with the session ID.\nStacktrace:\n at org.openqa.grid.web.servlet.handler.RequestHandler.process (RequestHandler.java:117)\n at org.openqa.grid.web.servlet.DriverServlet.process (DriverServlet.java:84)\n at org.openqa.grid.web.servlet.DriverServlet.doPost (DriverServlet.java:68)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)\n at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)\n at org.seleniumhq.jetty9.servlet.ServletHolder.handle (ServletHolder.java:860)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle (ServletHandler.java:535)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle (SessionHandler.java:1595)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle (ContextHandler.java:1253)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)\n at org.seleniumhq.jetty9.servlet.ServletHandler.doScope (ServletHandler.java:473)\n at org.seleniumhq.jetty9.server.session.SessionHandler.doScope (SessionHandler.java:1564)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)\n at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope (ContextHandler.java:1155)\n at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle (ScopedHandler.java:141)\n at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)\n at org.seleniumhq.jetty9.server.Server.handle (Server.java:530)\n at org.seleniumhq.jetty9.server.HttpChannel.handle (HttpChannel.java:347)\n at org.seleniumhq.jetty9.server.HttpConnection.onFillable (HttpConnection.java:256)\n at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)\n at org.seleniumhq.jetty9.io.FillInterest.fillable (FillInterest.java:102)\n at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce (EatWhatYouKill.java:247)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.produce (EatWhatYouKill.java:140)\n at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run (EatWhatYouKill.java:131)\n at org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run (ReservedThreadExecutor.java:382)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:708)\n at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:626)\n at java.lang.Thread.run (Thread.java:748)\nWebDriverException in _init_browser connecting to hub\n\n--------------------- >> end captured stdout << ----------------------\n-------------------- >> begin captured logging << --------------------\nselenium.webdriver.remote.remote_connection: DEBUG: POST http://172.18.0.2:4444/wd/hub/session {"capabilities": {"alwaysMatch": {"acceptInsecureCerts": true, "browserName": "firefox", "platformName": "linux", "browserVersion": "58.0.1"}, "firstMatch": [{}]}, "desiredCapabilities": {"acceptInsecureCerts": true, "platform": "LINUX", "browserName": "firefox", "version": "58.0.1", "marionette": true}}\nurllib3.connectionpool: DEBUG: Starting new HTTP connection (1): 172.18.0.2:4444\nurllib3.connectionpool: DEBUG: http://172.18.0.2:4444 "POST /wd/hub/session HTTP/1.1" 500 8055\nselenium.webdriver.remote.remote_connection: DEBUG: Finished Request\n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 590
by jenkins＠jenkins.phx.ovirt.org 22 Sep '18

22 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/590//artifact/exported-a… Cheers, Jenkins

1 0

[oVirt CI] standard-enqueue - Build #16565 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 21 Sep '18

21 Sep '18

Build: http://jenkins.ovirt.org/job/standard-enqueue/16565/ Build Name: #16565 Build Description: Gerrit: 94471 - ovirt-engine-wildfly-overlay (4.1) Build Status: FAILURE Gerrit change: https://gerrit.ovirt.org/94471 - title: build: Add support for oVirt STDCIv2 - project: ovirt-engine-wildfly-overlay - branch: wildfly-10 - author: Martin Peřina <mperina(a)redhat.com>

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 462 - Failure!
by jenkins＠jenkins.phx.ovirt.org 21 Sep '18

21 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/462/ Build Number: 462 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #462 [Ehud Yonasi] ovirt-master.repo: Added new packages to repo file [Sandro Bonazzola] ovirt-iso-uploader: drop 4.1 jobs ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7f19a81840c8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4 host_obj = host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37037, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault body = self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 229, in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 273, in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4 host_obj = host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37037, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault body = self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type raise Error(msg) 'The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n-------------------- >> begin captured logging << --------------------\novirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7f19a81840c8>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4\n host_obj = host_service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37037, in get\n return self._internal_get(headers, query, wait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get\n return future.wait() if wait else future\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait\n return self._code(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback\n self._check_fault(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault\n body = self._internal_read_body(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body\n self._connection.check_xml_content_type(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type\n response.headers\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type\n raise Error(msg)\nError: The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n--------------------- >> end captured logging << ---------------------'

1 33

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-master - Build # 712 - Failure!
by jenkins＠jenkins.phx.ovirt.org 21 Sep '18

21 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/712/ Build Number: 712 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #712 [Dominik Holler] network: Enforce specific version of ansible [Daniel Belenky] stdci_runner: don't fail when project setup fails ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7f3e4a826ed8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 253, in _host_is_up_4 host_obj = host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 36676, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault body = self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type raise Error(msg) Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 228, in add_hosts add_hosts_4(prefix) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 275, in add_hosts_4 testlib.assert_true_within(_host_is_up_4, timeout=15*60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 253, in _host_is_up_4 host_obj = host_service.get() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 36676, in get return self._internal_get(headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault body = self._internal_read_body(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body self._connection.check_xml_content_type(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type response.headers File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type raise Error(msg) 'The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n-------------------- >> begin captured logging << --------------------\novirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7f3e4a826ed8>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 253, in _host_is_up_4\n host_obj = host_service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 36676, in get\n return self._internal_get(headers, query, wait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get\n return future.wait() if wait else future\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait\n return self._code(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback\n self._check_fault(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault\n body = self._internal_read_body(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body\n self._connection.check_xml_content_type(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type\n response.headers\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type\n raise Error(msg)\nError: The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n--------------------- >> end captured logging << ---------------------'

1 3

[CQ]: 94439,1 (ovirt-setup-lib) failed "ovirt-4.2" system tests
by oVirt Jenkins 21 Sep '18

21 Sep '18

Change 94439,1 (ovirt-setup-lib) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94439/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3163/

2 1

[oVirt Jenkins] ovirt-node-ng_4.2_build-artifacts-el7-x86_64 - Build # 204 - Failure!
by jenkins＠jenkins.phx.ovirt.org 21 Sep '18

21 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-node-ng_4.2_build-artifacts-el7-x86_64/ Build: http://jenkins.ovirt.org/job/ovirt-node-ng_4.2_build-artifacts-el7-x86_64/2… Build Number: 204 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #204 [Ryan Barry] kickstart: let VDSM reconfigure itself ----------------- Failed Tests: ----------------- No tests ran.

1 1

oVirt infra daily report - unstable production jobs - 589
by jenkins＠jenkins.phx.ovirt.org 21 Sep '18

21 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/589//artifact/exported-a… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_network-suite-4.2 - Build # 586 - Failure!
by jenkins＠jenkins.phx.ovirt.org 20 Sep '18

20 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.2/586/ Build Number: 586 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #586 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 2 tests failed. FAILED: network-suite-4.2.tests.ovn-provider.test_ovn_provider.test_ovn_provider_create_scenario Error Message: AnsibleExecutionFailure Stack Trace: openstack_client_config = 'ovirt' def test_ovn_provider_create_scenario(openstack_client_config): > _test_ovn_provider('create_scenario.yml') network-suite-4.2/tests/ovn-provider/test_ovn_provider.py:49: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ network-suite-4.2/tests/ovn-provider/test_ovn_provider.py:167: in _test_ovn_provider playbook.run() network-suite-4.2/lib/ansiblelib.py:127: in run self._run_playbook_executor() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <lib.ansiblelib.Playbook object at 0x7fa01b1b1f90> def _run_playbook_executor(self): pbex = PlaybookExecutor(**self._pbex_args) if pbex.run() != 0: > raise AnsibleExecutionFailure E AnsibleExecutionFailure network-suite-4.2/lib/ansiblelib.py:140: AnsibleExecutionFailure FAILED: network-suite-4.2.tests.ovn-provider.test_ovn_provider.test_validate_ovn_provider_connectivity Error Message: PingFailed: Ssh command "ip netns exec net10_port1 ping -4 -c 1 -M do -s 56 10.11.0.11" exited with status code 1. Stderr: Stack Trace: default_ovn_provider_client = <shade.openstackcloud.OpenStackCloud object at 0x7fa01ab9fd90> host_0 = <lib.hostlib.Host object at 0x7fa01969a750> host_1 = <lib.hostlib.Host object at 0x7fa01969a090> def test_validate_ovn_provider_connectivity(default_ovn_provider_client, host_0, host_1): net10_port1 = default_ovn_provider_client.get_port(NETWORK10_PORT1_NAME) net11_port1 = default_ovn_provider_client.get_port(NETWORK11_PORT1_NAME) net14_port1 = default_ovn_provider_client.get_port(NETWORK14_PORT1_NAME) net10_subnet1 = default_ovn_provider_client.get_subnet( NETWORK10_SUBNET1_NAME) net11_subnet1 = default_ovn_provider_client.get_subnet( NETWORK11_SUBNET1_NAME) net14_subnet1 = default_ovn_provider_client.get_subnet( NETWORK14_SUBNET1_NAME) connections = ( (host_0, net10_port1, net10_subnet1,), (host_1, net11_port1, net11_subnet1,), (host_1, net14_port1, net14_subnet1,) ) with _create_namespaces(connections): with _create_ovs_ports(connections): assert_connectivity_between(host=host_0, from_port=net10_port1, > to_port=net11_port1) network-suite-4.2/tests/ovn-provider/test_ovn_provider.py:73: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ network-suite-4.2/tests/ovn-provider/test_ovn_provider.py:96: in assert_connectivity_between _ping(host=host, from_port=from_port, to_port=to_port) network-suite-4.2/tests/ovn-provider/test_ovn_provider.py:109: in _ping netns=from_port.name) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ source = '192.168.201.2', password = '123456', destination = '10.11.0.11' data_size = 56, netns = 'net10_port1' def ssh_ping(source, password, destination, data_size=56, netns=None): """ Ping a given destination from source. :parameter source: the host which executes the ping command :parameter password: the root password for the host source :parameter destination: the destination of the ping command :parameter data_size: the size of the data payload :parameter netns: optional networking namespace in which to execute """ netns_prefix = 'ip netns exec {} '.format(netns) if netns else '' cmd = netns_prefix + 'ping -4 -c 1 -M do -s {} {}'.format( data_size, destination) try: sshlib.exec_command(source, password, cmd) except SshException as err: > raise PingFailed(err) E PingFailed: Ssh command "ip netns exec net10_port1 ping -4 -c 1 -M do -s 56 10.11.0.11" exited with status code 1. Stderr: network-suite-4.2/testlib/ping.py:45: PingFailed

1 9

[CQ]: bec8839 (ovirt-ansible-engine-setup) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 20 Sep '18

20 Sep '18

A system test invoked by the "ovirt-master" change queue including change bec8839 (ovirt-ansible-engine-setup) failed. However, this change seems not to be the root cause for this failure. Change e842386 (ovirt-ansible-engine-setup) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change e842386 (ovirt-ansible-engine- setup) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/ovirt-ansible-engine-setup/commit/bec88397c975664c… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/ovirt-ansible-engine-setup/commit/e8423862f86db4f5… For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10371/

1 0

[oVirt Jenkins] ovirt-system-tests_network-suite-master - Build # 585 - Failure!
by jenkins＠jenkins.phx.ovirt.org 20 Sep '18

20 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-master/585/ Build Number: 585 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #585 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 2 tests failed. FAILED: network-suite-master.tests.ovn-provider.test_ovn_provider.test_ovn_provider_create_scenario Error Message: AnsibleExecutionFailure Stack Trace: openstack_client_config = 'ovirt' def test_ovn_provider_create_scenario(openstack_client_config): > _test_ovn_provider('create_scenario.yml') network-suite-master/tests/ovn-provider/test_ovn_provider.py:49: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ network-suite-master/tests/ovn-provider/test_ovn_provider.py:167: in _test_ovn_provider playbook.run() network-suite-master/lib/ansiblelib.py:127: in run self._run_playbook_executor() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <lib.ansiblelib.Playbook object at 0x7f7977502510> def _run_playbook_executor(self): pbex = PlaybookExecutor(**self._pbex_args) if pbex.run() != 0: > raise AnsibleExecutionFailure E AnsibleExecutionFailure network-suite-master/lib/ansiblelib.py:140: AnsibleExecutionFailure FAILED: network-suite-master.tests.ovn-provider.test_ovn_provider.test_validate_ovn_provider_connectivity Error Message: PingFailed: Ssh command "ip netns exec net10_port1 ping -4 -c 1 -M do -s 56 10.11.0.11" exited with status code 1. Stderr: Stack Trace: default_ovn_provider_client = <shade.openstackcloud.OpenStackCloud object at 0x7f7976cc5310> host_0 = <lib.hostlib.Host object at 0x7f798404ac10> host_1 = <lib.hostlib.Host object at 0x7f798404a2d0> def test_validate_ovn_provider_connectivity(default_ovn_provider_client, host_0, host_1): net10_port1 = default_ovn_provider_client.get_port(NETWORK10_PORT1_NAME) net11_port1 = default_ovn_provider_client.get_port(NETWORK11_PORT1_NAME) net14_port1 = default_ovn_provider_client.get_port(NETWORK14_PORT1_NAME) net10_subnet1 = default_ovn_provider_client.get_subnet( NETWORK10_SUBNET1_NAME) net11_subnet1 = default_ovn_provider_client.get_subnet( NETWORK11_SUBNET1_NAME) net14_subnet1 = default_ovn_provider_client.get_subnet( NETWORK14_SUBNET1_NAME) connections = ( (host_0, net10_port1, net10_subnet1,), (host_1, net11_port1, net11_subnet1,), (host_1, net14_port1, net14_subnet1,) ) with _create_namespaces(connections): with _create_ovs_ports(connections): assert_connectivity_between(host=host_0, from_port=net10_port1, > to_port=net11_port1) network-suite-master/tests/ovn-provider/test_ovn_provider.py:73: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ network-suite-master/tests/ovn-provider/test_ovn_provider.py:96: in assert_connectivity_between _ping(host=host, from_port=from_port, to_port=to_port) network-suite-master/tests/ovn-provider/test_ovn_provider.py:109: in _ping netns=from_port.name) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ source = '192.168.201.3', password = '123456', destination = '10.11.0.11' data_size = 56, netns = 'net10_port1' def ssh_ping(source, password, destination, data_size=56, netns=None): """ Ping a given destination from source. :parameter source: the host which executes the ping command :parameter password: the root password for the host source :parameter destination: the destination of the ping command :parameter data_size: the size of the data payload :parameter netns: optional networking namespace in which to execute """ netns_prefix = 'ip netns exec {} '.format(netns) if netns else '' cmd = netns_prefix + 'ping -4 -c 1 -M do -s {} {}'.format( data_size, destination) try: sshlib.exec_command(source, password, cmd) except SshException as err: > raise PingFailed(err) E PingFailed: Ssh command "ip netns exec net10_port1 ping -4 -c 1 -M do -s 56 10.11.0.11" exited with status code 1. Stderr: network-suite-master/testlib/ping.py:45: PingFailed

1 9

[JIRA] (OVIRT-2506) Accessing logs is sometimes extremely slow even without blueocean
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2506?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2506: ------------------------------------- Its still beeign server by jenkins, just not via any complex code; but if the whole JVM becomes slow because of GC issues, it can be slow too. In any case we've restarted Jenkins a few minutes ago, so everything aught to be faster now. > Accessing logs is sometimes extremely slow even without blueocean > ----------------------------------------------------------------- > > Key: OVIRT-2506 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2506 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins Master > Reporter: Roman Mohr > Assignee: infra > > Hi, > With the switch from blueocean to the text-based build summary in > kubevirt/kubevirt, in general accessing works much better. Still > especially in the morning hours it can take a long time to access the > logs. > Can it be that the build summary and the logs are still served by > jenkins? If yes, would it help to copied out/cache/serve the build > logs via a dedicated webserver? > Best Regards, > Roman -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2506) Accessing logs is sometimes extremely slow even without blueocean
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2506?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2506: -------------------------------- Component/s: Jenkins Master > Accessing logs is sometimes extremely slow even without blueocean > ----------------------------------------------------------------- > > Key: OVIRT-2506 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2506 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins Master > Reporter: Roman Mohr > Assignee: infra > > Hi, > With the switch from blueocean to the text-based build summary in > kubevirt/kubevirt, in general accessing works much better. Still > especially in the morning hours it can take a long time to access the > logs. > Can it be that the build summary and the logs are still served by > jenkins? If yes, would it help to copied out/cache/serve the build > logs via a dedicated webserver? > Best Regards, > Roman -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2506) Accessing logs is sometimes extremely slow even without blueocean
by Roman Mohr (oVirt JIRA) 20 Sep '18

20 Sep '18

Roman Mohr created OVIRT-2506: --------------------------------- Summary: Accessing logs is sometimes extremely slow even without blueocean Key: OVIRT-2506 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2506 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Roman Mohr Assignee: infra Hi, With the switch from blueocean to the text-based build summary in kubevirt/kubevirt, in general accessing works much better. Still especially in the morning hours it can take a long time to access the logs. Can it be that the build summary and the logs are still served by jenkins? If yes, would it help to copied out/cache/serve the build logs via a dedicated webserver? Best Regards, Roman -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2505) ovirt github organization
by Ahmad Khiet (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2505?page=com.atlassian.jira.… ] Ahmad Khiet commented on OVIRT-2505: ------------------------------------ Hi [~bkorren(a)redhat.com] ovirt-engine > ovirt github organization > ------------------------- > > Key: OVIRT-2505 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2505 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Ahmad Khiet > Assignee: infra > > Hi, > can you please add my github user to ovirt org > https://github.com/NaySoftware > Thanks, > Ahmad. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2505) ovirt github organization
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2505?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2505: ------------------------------------- Hi [~akhiet(a)redhat.com] which projects are you going to be working on? Please note that most of the oVirt sources are maintained in [gerrit.ovirt.org|https://gerrit.ovirt.org] and not GitHub. > ovirt github organization > ------------------------- > > Key: OVIRT-2505 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2505 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Ahmad Khiet > Assignee: infra > > Hi, > can you please add my github user to ovirt org > https://github.com/NaySoftware > Thanks, > Ahmad. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[CQ]: 94266,4 (ovirt-engine) failed "ovirt-master" system tests
by oVirt Jenkins 20 Sep '18

20 Sep '18

Change 94266,4 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94266/4 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10367/

2 1

[JIRA] (OVIRT-2505) ovirt github organization
by Ahmad Khiet (oVirt JIRA) 20 Sep '18

20 Sep '18

Ahmad Khiet created OVIRT-2505: ---------------------------------- Summary: ovirt github organization Key: OVIRT-2505 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2505 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Ahmad Khiet Assignee: infra Hi, can you please add my github user to ovirt org https://github.com/NaySoftware Thanks, Ahmad. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1881
by jenkins＠jenkins.phx.ovirt.org 20 Sep '18

20 Sep '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1881/displ…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision f9508e32027645ea57bcfec09dafe628fbb06677 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f f9508e32027645ea57bcfec09dafe628fbb06677 Commit message: "standard_pipelines: add nmstate project" > git rev-list --no-walk f9508e32027645ea57bcfec09dafe628fbb06677 # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins3983678000755324930.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 209, in main my.doSackSetup(arches) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 741, in doSackSetup return self._getSacks(archlist=archlist, thisrepo=thisrepo) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks self.repos.populateSack(which=repos) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 386, in populateSack sack.populate(repo, mdtype, callback, cacheonly) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 256, in populate ret = misc.repo_gen_decompress(xml, gen, cached=repo.cache) File "/usr/lib/python2.7/site-packages/yum/misc.py", line 1165, in repo_gen_decompress return decompress(filename, dest=dest, check_timestamps=True) File "/usr/lib/python2.7/site-packages/yum/misc.py", line 1150, in decompress _decompress_chunked(filename, out, ztype) File "/usr/lib/python2.7/site-packages/yum/misc.py", line 791, in _decompress_chunked raise Errors.MiscError, msg yum.Errors.MiscError: Error reading from file /home/jenkins/mirrors_cache/epel-el6/b4d4d39ae211c497fd1d621791a9b1757c2509be7347291886b9d35d6e1a61ff-primary.xml.gz: Not a gzipped file Build step 'Execute shell' marked build as failure

1 1

[CQ]: 94345, 3 (ovirt-provider-ovn) failed "ovirt-master" system tests
by oVirt Jenkins 20 Sep '18

20 Sep '18

Change 94345,3 (ovirt-provider-ovn) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94345/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10365/

2 1

[JIRA] (OVIRT-2503) Testing automatic Jira ticket logging for CQ monitoring
by Dafna Ron (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2503?page=com.atlassian.jira.… ] Dafna Ron commented on OVIRT-2503: ---------------------------------- [~bkorren(a)redhat.com] yes. but to remind you, when we discussed monitoring you stated that we need to reply to every alert that comes in so that we know nothing has been missed and I have also been looking into the best way to do that (since I do not think that its viable in email). I was discussing this issue with leads of some of the jboss developing team who have been working agile for a while and they have recommended that we stop doing monitoring in email and just configure a jira project for the monitoring which would allow us to easily monitor every alert and assign them forward when needed. Jira is very flexible and we can configure it to do a lot of things that we cannot do now. I was also concerned about having too many opened tickets but then, if you have a way to close/dismiss tickets quickly (which is something that is possible in Jira) then I see it as two birds one stone since we would be able to: 1. make sure all alerts have been viewed by someone in the team. 2. give us an easier way to move forwards in collaboration with Dev on monitoring and perhaps, partially automating it. > Testing automatic Jira ticket logging for CQ monitoring > ------------------------------------------------------- > > Key: OVIRT-2503 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2503 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > > As part of monitoring improvement and shifting partial responsibility for CQ monitoring to the developers I would like to do some experiments on connecting CQ alerts to create Jira tickets automatically. > As we have the ost monitoring project we start by doing some tests on that project so that we do not overflow our jira with alerts. > For now I would like to start with Jira's opening with the following specifications: > 1. Subject: [ CQ ] [$patch number] [ oVirt $VER ($project) ] [ TEST NAME ] > 2. at the beginning, email sent to: dron(a)redhat.com > 3. jira description [cq message] > 4. all jira's has to have label=ost_failure and infra-owner as default. > I would like to change the project type to allow: > 1. easy closing of Jira's (one/two clicks if we can) > 2. view of Jira's like service tickets (rather then bugs) > There is a plugin called zapier that allows to easily connect a jira from an email and also allow to add some rules to the Jira which may make this easier for us. > can you also install it and link it to the ost jira? I have an email account that we can use for that. > cq.ovirt(a)gmail.com > https://zapier.com/apps/jira/integrations > [~bkorren(a)redhat.com] > once I do some tests on my own on this, I wanted to try and collaborate with one of the projects (maybe networking or one of sandro's teams) where CQ failures would automatically open a ticket to their team and they can handle the monitoring and escalate issues to us if needed. > Any advice on configurations we should be thinking of for that? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2503) Testing automatic Jira ticket logging for CQ monitoring
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2503?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2503: ------------------------------------- [~dron] I really don`t think this makes any sense since the relationship between jira ticket and CQ alerts aught to be one-to-many where this would result ina one-to-one relationship... > Testing automatic Jira ticket logging for CQ monitoring > ------------------------------------------------------- > > Key: OVIRT-2503 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2503 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > > As part of monitoring improvement and shifting partial responsibility for CQ monitoring to the developers I would like to do some experiments on connecting CQ alerts to create Jira tickets automatically. > As we have the ost monitoring project we start by doing some tests on that project so that we do not overflow our jira with alerts. > For now I would like to start with Jira's opening with the following specifications: > 1. Subject: [ CQ ] [$patch number] [ oVirt $VER ($project) ] [ TEST NAME ] > 2. at the beginning, email sent to: dron(a)redhat.com > 3. jira description [cq message] > 4. all jira's has to have label=ost_failure and infra-owner as default. > I would like to change the project type to allow: > 1. easy closing of Jira's (one/two clicks if we can) > 2. view of Jira's like service tickets (rather then bugs) > There is a plugin called zapier that allows to easily connect a jira from an email and also allow to add some rules to the Jira which may make this easier for us. > can you also install it and link it to the ost jira? I have an email account that we can use for that. > cq.ovirt(a)gmail.com > https://zapier.com/apps/jira/integrations > [~bkorren(a)redhat.com] > once I do some tests on my own on this, I wanted to try and collaborate with one of the projects (maybe networking or one of sandro's teams) where CQ failures would automatically open a ticket to their team and they can handle the monitoring and escalate issues to us if needed. > Any advice on configurations we should be thinking of for that? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[oVirt Jenkins] ovirt-appliance_master_build-artifacts-el7-x86_64 - Build # 923 - Failure!
by jenkins＠jenkins.phx.ovirt.org 20 Sep '18

20 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el7-x86… Build: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el7-x86… Build Number: 923 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #923 [Yuval Turgeman] automation: remove build-artifacts* from stdci-v2 ----------------- Failed Tests: ----------------- No tests ran.

1 1

[JIRA] (OVIRT-2498) Failing KubeVirt CI
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2498?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2498: -------------------------------- Epic Link: OVIRT-2339 > Failing KubeVirt CI > ------------------- > > Key: OVIRT-2498 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2498 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: CI client projects > Reporter: Petr Kotas > Assignee: infra > > Hi, > I am working on fixing the issues on the KubeVirt e2e test suites. This > task is directly related to unstable CI, due to unknown errors. > The progress is reported in the CNV trello: > https://trello.com/c/HNXcMEQu/161-epic-improve-ci > I am creating this issue since the KubeVirt experience random timeouts on > random tests most of the times when test suites run. > The issue from outside is showing as timeouts on difference part of tests. > Sometimes the tests fails in set up phase, again due to random timeout. > The example in the link bellow timed out for network connection on > localhost. > [check-patch.k8s-1.11.0-dev.el7.x86_64] > requests.exceptions.ReadTimeout: > UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. > (read timeout=60) > Example of failing test suites is here > https://jenkins.ovirt.org/job/kubevirt_kubevirt_standard-check-pr/1916/cons… > The list of errors related to the failing CI can be found in my notes > https://docs.google.com/document/d/1_ll1DOMHgCRHn_Df9i4uvtRFyMK-bDCHEeGfJFT… > I am not sure whether KubeVirt already shared the resource requirements, so > I provide short summary: > *Resources for KubeVirt e2e tests:* > - at least 12GB of RAM - we start 3 nodes (3 docker images) each require > 4GB of RAM > - exposed /dev/kvm to enable native virtualization > - cached images, since these are used to build the test cluster: > - kubevirtci/os-3.10.0-crio:latest > - kubevirtci/os-3.10.0-multus:latest > - kubevirtci/os-3.10.0:latest > - kubevirtci/k8s-1.10.4:latest > - kubevirtci/k8s-multus-1.11.1:latest > - kubevirtci/k8s-1.11.0:latest > How can we overcome this? Can we work together to build a suitable > requirements for running the tests so it passes each time? > Kind regards, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2498) Failing KubeVirt CI
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2498?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2498: -------------------------------- Epic Link: OVIRT-2339 > Failing KubeVirt CI > ------------------- > > Key: OVIRT-2498 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2498 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: CI client projects > Reporter: Petr Kotas > Assignee: infra > > Hi, > I am working on fixing the issues on the KubeVirt e2e test suites. This > task is directly related to unstable CI, due to unknown errors. > The progress is reported in the CNV trello: > https://trello.com/c/HNXcMEQu/161-epic-improve-ci > I am creating this issue since the KubeVirt experience random timeouts on > random tests most of the times when test suites run. > The issue from outside is showing as timeouts on difference part of tests. > Sometimes the tests fails in set up phase, again due to random timeout. > The example in the link bellow timed out for network connection on > localhost. > [check-patch.k8s-1.11.0-dev.el7.x86_64] > requests.exceptions.ReadTimeout: > UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. > (read timeout=60) > Example of failing test suites is here > https://jenkins.ovirt.org/job/kubevirt_kubevirt_standard-check-pr/1916/cons… > The list of errors related to the failing CI can be found in my notes > https://docs.google.com/document/d/1_ll1DOMHgCRHn_Df9i4uvtRFyMK-bDCHEeGfJFT… > I am not sure whether KubeVirt already shared the resource requirements, so > I provide short summary: > *Resources for KubeVirt e2e tests:* > - at least 12GB of RAM - we start 3 nodes (3 docker images) each require > 4GB of RAM > - exposed /dev/kvm to enable native virtualization > - cached images, since these are used to build the test cluster: > - kubevirtci/os-3.10.0-crio:latest > - kubevirtci/os-3.10.0-multus:latest > - kubevirtci/os-3.10.0:latest > - kubevirtci/k8s-1.10.4:latest > - kubevirtci/k8s-multus-1.11.1:latest > - kubevirtci/k8s-1.11.0:latest > How can we overcome this? Can we work together to build a suitable > requirements for running the tests so it passes each time? > Kind regards, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2498) Failing KubeVirt CI
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2498?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2498: -------------------------------- Component/s: CI client projects > Failing KubeVirt CI > ------------------- > > Key: OVIRT-2498 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2498 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: CI client projects > Reporter: Petr Kotas > Assignee: infra > > Hi, > I am working on fixing the issues on the KubeVirt e2e test suites. This > task is directly related to unstable CI, due to unknown errors. > The progress is reported in the CNV trello: > https://trello.com/c/HNXcMEQu/161-epic-improve-ci > I am creating this issue since the KubeVirt experience random timeouts on > random tests most of the times when test suites run. > The issue from outside is showing as timeouts on difference part of tests. > Sometimes the tests fails in set up phase, again due to random timeout. > The example in the link bellow timed out for network connection on > localhost. > [check-patch.k8s-1.11.0-dev.el7.x86_64] > requests.exceptions.ReadTimeout: > UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. > (read timeout=60) > Example of failing test suites is here > https://jenkins.ovirt.org/job/kubevirt_kubevirt_standard-check-pr/1916/cons… > The list of errors related to the failing CI can be found in my notes > https://docs.google.com/document/d/1_ll1DOMHgCRHn_Df9i4uvtRFyMK-bDCHEeGfJFT… > I am not sure whether KubeVirt already shared the resource requirements, so > I provide short summary: > *Resources for KubeVirt e2e tests:* > - at least 12GB of RAM - we start 3 nodes (3 docker images) each require > 4GB of RAM > - exposed /dev/kvm to enable native virtualization > - cached images, since these are used to build the test cluster: > - kubevirtci/os-3.10.0-crio:latest > - kubevirtci/os-3.10.0-multus:latest > - kubevirtci/os-3.10.0:latest > - kubevirtci/k8s-1.10.4:latest > - kubevirtci/k8s-multus-1.11.1:latest > - kubevirtci/k8s-1.11.0:latest > How can we overcome this? Can we work together to build a suitable > requirements for running the tests so it passes each time? > Kind regards, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2498) Failing KubeVirt CI
by Barak Korren (oVirt JIRA) 20 Sep '18

20 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2498?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2498: -------------------------------- Issue Type: Bug (was: By-EMAIL) > Failing KubeVirt CI > ------------------- > > Key: OVIRT-2498 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2498 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Petr Kotas > Assignee: infra > > Hi, > I am working on fixing the issues on the KubeVirt e2e test suites. This > task is directly related to unstable CI, due to unknown errors. > The progress is reported in the CNV trello: > https://trello.com/c/HNXcMEQu/161-epic-improve-ci > I am creating this issue since the KubeVirt experience random timeouts on > random tests most of the times when test suites run. > The issue from outside is showing as timeouts on difference part of tests. > Sometimes the tests fails in set up phase, again due to random timeout. > The example in the link bellow timed out for network connection on > localhost. > [check-patch.k8s-1.11.0-dev.el7.x86_64] > requests.exceptions.ReadTimeout: > UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. > (read timeout=60) > Example of failing test suites is here > https://jenkins.ovirt.org/job/kubevirt_kubevirt_standard-check-pr/1916/cons… > The list of errors related to the failing CI can be found in my notes > https://docs.google.com/document/d/1_ll1DOMHgCRHn_Df9i4uvtRFyMK-bDCHEeGfJFT… > I am not sure whether KubeVirt already shared the resource requirements, so > I provide short summary: > *Resources for KubeVirt e2e tests:* > - at least 12GB of RAM - we start 3 nodes (3 docker images) each require > 4GB of RAM > - exposed /dev/kvm to enable native virtualization > - cached images, since these are used to build the test cluster: > - kubevirtci/os-3.10.0-crio:latest > - kubevirtci/os-3.10.0-multus:latest > - kubevirtci/os-3.10.0:latest > - kubevirtci/k8s-1.10.4:latest > - kubevirtci/k8s-multus-1.11.1:latest > - kubevirtci/k8s-1.11.0:latest > How can we overcome this? Can we work together to build a suitable > requirements for running the tests so it passes each time? > Kind regards, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

oVirt infra daily report - unstable production jobs - 588
by jenkins＠jenkins.phx.ovirt.org 20 Sep '18

20 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/588//artifact/exported-a… Cheers, Jenkins

1 0

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-master - Build # 644 - Failure!
by jenkins＠jenkins.phx.ovirt.org 20 Sep '18

20 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/644/ Build Number: 644 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #644 [Milan Zamazal] ovf_import test [Eyal Edri] adding kubevirt/containerized-data-importer to stdci v2 ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.wait_engine Error Message: None != True after 600 seconds Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 110, in wait_engine testlib.assert_true_within(_engine_is_up, timeout=10 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: None != True after 600 seconds

1 66

[CQ]: 94345,3 (ovirt-provider-ovn) failed "ovirt-4.2" system tests
by oVirt Jenkins 19 Sep '18

19 Sep '18

Change 94345,3 (ovirt-provider-ovn) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/94345/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3150/

1 0

[CQ]: 94375, 2 (ovirt-provider-ovn) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 19 Sep '18

19 Sep '18

A system test invoked by the "ovirt-master" change queue including change 94375,2 (ovirt-provider-ovn) failed. However, this change seems not to be the root cause for this failure. Change 94307,6 (ovirt-provider-ovn) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 94307,6 (ovirt-provider-ovn) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/94375/2 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/94307/6 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10360/

1 0

[CQ]: 6993caa (ovirt-ansible-manageiq) failed "ovirt-4.2" system tests
by oVirt Jenkins 19 Sep '18

19 Sep '18

Change 6993caa (ovirt-ansible-manageiq) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/ovirt-ansible-manageiq/commit/6993caa02dc761e0f874… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3142/

1 0

Build failed in Jenkins: system-sync_mirrors-epel-el6-x86_64 #1877
by jenkins＠jenkins.phx.ovirt.org 19 Sep '18

19 Sep '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/1877/displ…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-epel-el6-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision f9508e32027645ea57bcfec09dafe628fbb06677 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f f9508e32027645ea57bcfec09dafe628fbb06677 Commit message: "standard_pipelines: add nmstate project" > git rev-list --no-walk f9508e32027645ea57bcfec09dafe628fbb06677 # timeout=10 [system-sync_mirrors-epel-el6-x86_64] $ /bin/bash -xe /tmp/jenkins8583691673519336510.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror epel-el6 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-updates-el7/packages error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-updates-el7/packages' Build step 'Execute shell' marked build as failure

1 1

[OST][hc-basic-suite-master] Fails on HE deployment
by Sahina Bose 19 Sep '18

19 Sep '18

xargs -I{} sudo -u vdsm dd if={} | tar -tvf - 772a3dfe-aee8-45d9-9df5-beddcbf92010.ovf\", u'removes': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'stdout_lines': [], u'stderr': u\"vdsm-client: Command Image.prepare with args {'storagepoolID': 'e90984b6-af79-11e8-a2dd-00163e24d363', 'storagedomainID': 'b4c84ead-fb7f-4478-acd2-0309d569b9aa', 'volumeID': 'b18f30b1-5625-4581-b815-94291b226c79', 'imageID': '[u7894c850-665f-47dd-8376-e4da5ae7807f]'} failed:\\n(code=201, message=Volume does not exist: (u'b18f30b1-5625-4581-b815-94291b226c79',))\\ntar: This does not look like a tar archive Is this a known issue? thanks! sahina

2 3

[oVirt Jenkins] ovirt-system-tests_he-basic-ansible-suite-4.2 - Build # 578 - Failure!
by jenkins＠jenkins.phx.ovirt.org 19 Sep '18

19 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-4.2/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-ansible-suite-4.2/… Build Number: 578 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #578 [Milan Zamazal] Update test-vm.ovf file ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 008_restart_he_vm.restart_he_vm Error Message: could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:6469ff97-fd82-4fba-8353-9b736936a12c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:6469ff97-fd82-4fba-8353-9b736936a12c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 54a3b8d8 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 54a3b8d8 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 54a3b8d8 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2561 (Tue Sep 18 12:10:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5bb7aefb", "local_conf_timestamp": 2561, "host-ts": 2561}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2560 (Tue Sep 18 12:10:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d172629f", "local_conf_timestamp": 2561, "host-ts": 2560}, "global_maintenance": true} root: INFO: * Shutting down HE VM on host: lago-he-basic-ansible-suite-4-2-host-0 lago.ssh: DEBUG: start task:a1e2f644-5484-4092-976b-f93b9bff8d63:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:a1e2f644-5484-4092-976b-f93b9bff8d63:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 55273866 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-shutdown lago.ssh: DEBUG: Command 55273866 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 root: INFO: * Command succeeded root: INFO: * Waiting for VM to be down... lago.ssh: DEBUG: start task:19bbae64-260d-4591-8719-ec52c46081a3:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:19bbae64-260d-4591-8719-ec52c46081a3:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 55d4e434 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 55d4e434 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 55d4e434 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2561 (Tue Sep 18 12:10:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5bb7aefb", "local_conf_timestamp": 2561, "host-ts": 2561}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2560 (Tue Sep 18 12:10:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d172629f", "local_conf_timestamp": 2561, "host-ts": 2560}, "global_maintenance": true} lago.ssh: DEBUG: start task:41902ed5-eb63-4017-8b3a-b612b34f6bcc:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:41902ed5-eb63-4017-8b3a-b612b34f6bcc:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 5c8a15ec on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5c8a15ec on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5c8a15ec on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2571 (Tue Sep 18 12:10:20 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2571 (Tue Sep 18 12:10:20 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fcf99ac3", "local_conf_timestamp": 2571, "host-ts": 2571}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2570 (Tue Sep 18 12:10:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2570 (Tue Sep 18 12:10:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "60fe16df", "local_conf_timestamp": 2570, "host-ts": 2570}, "global_maintenance": true} lago.ssh: DEBUG: start task:f9baa42f-d1b9-44e8-9d0a-42f41a495362:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:f9baa42f-d1b9-44e8-9d0a-42f41a495362:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 5eddf14c on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 5eddf14c on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 5eddf14c on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2581 (Tue Sep 18 12:10:30 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2581 (Tue Sep 18 12:10:31 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e5962d00", "local_conf_timestamp": 2581, "host-ts": 2581}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2580 (Tue Sep 18 12:10:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2580 (Tue Sep 18 12:10:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "de32fbcb", "local_conf_timestamp": 2580, "host-ts": 2580}, "global_maintenance": true} root: INFO: * VM is down. root: INFO: * Stopping services... lago.ssh: DEBUG: start task:cb8c09f1-87dd-405d-8904-907a1989cb31:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:cb8c09f1-87dd-405d-8904-907a1989cb31:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 5f714e2e on lago-he-basic-ansible-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent lago.ssh: DEBUG: Command 5f714e2e on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 root: INFO: * Starting services... lago.ssh: DEBUG: start task:7ccaa6ca-f2ea-4e2a-98c6-0c543fd3cb72:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:7ccaa6ca-f2ea-4e2a-98c6-0c543fd3cb72:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 66c72220 on lago-he-basic-ansible-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent lago.ssh: DEBUG: Command 66c72220 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 root: INFO: * Waiting for agent to be ready... lago.ssh: DEBUG: start task:170c54e8-f608-4663-adb8-07875043648b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:170c54e8-f608-4663-adb8-07875043648b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 6938ed86 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 6938ed86 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 6938ed86 on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:181570eb-206a-4ec7-91ea-b0b5169fedfd:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:181570eb-206a-4ec7-91ea-b0b5169fedfd:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 6f91b55a on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 6f91b55a on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 6f91b55a on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:326af4dd-1f16-4935-9874-0d80259c7697:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:326af4dd-1f16-4935-9874-0d80259c7697:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 71b4333a on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 71b4333a on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 71b4333a on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:4290047c-da3a-49e5-a7b2-a9d60c8383cb:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:4290047c-da3a-49e5-a7b2-a9d60c8383cb:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 73da3b46 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 73da3b46 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 73da3b46 on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:e1724b51-9e94-4e12-be43-374a22a56e5c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:e1724b51-9e94-4e12-be43-374a22a56e5c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 76029bf2 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 76029bf2 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 76029bf2 on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:3f9b1f59-ee4f-49a6-ad21-c77d3c403aba:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:3f9b1f59-ee4f-49a6-ad21-c77d3c403aba:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 783cb2ae on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 783cb2ae on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 783cb2ae on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:0a356812-9169-42e4-bb61-f41c500bab0b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:0a356812-9169-42e4-bb61-f41c500bab0b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 7a64f6c2 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 7a64f6c2 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 7a64f6c2 on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:4440aaf7-97cd-4106-9d80-4f3d6265ba1d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:4440aaf7-97cd-4106-9d80-4f3d6265ba1d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 7c8cedf6 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 7c8cedf6 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 7c8cedf6 on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:4a3838f9-2b52-4189-8915-0c39dc71ccb1:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:4a3838f9-2b52-4189-8915-0c39dc71ccb1:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 7eb64906 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 7eb64906 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1 lago.ssh: DEBUG: Command 7eb64906 on lago-he-basic-ansible-suite-4-2-host-0 output: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. lago.ssh: DEBUG: start task:78ae6cc0-4511-4e8f-89c1-d015830bab8c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:78ae6cc0-4511-4e8f-89c1-d015830bab8c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 80ec9144 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status lago.ssh: DEBUG: Command 80ec9144 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 80ec9144 on lago-he-basic-ansible-suite-4-2-host-0 output: !! Cluster is in GLOBAL MAINTENANCE mode !! --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : lago-he-basic-ansible-suite-4-2-host-0 Host ID : 1 Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"} Score : 0 stopped : False Local maintenance : False crc32 : 486f8a4a local_conf_timestamp : 2641 Host timestamp : 2641 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=2641 (Tue Sep 18 12:11:30 2018) host-id=1 score=0 vm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018) conf_on_shared_storage=True maintenance=False state=ReinitializeFSM stopped=False --== Host 2 status ==-- conf_on_shared_storage : True Status up-to-date : False Hostname : lago-he-basic-ansible-suite-4-2-host-1 Host ID : 2 Engine status : unknown stale-data Score : 3400 stopped : False Local maintenance : False crc32 : fa3768d0 local_conf_timestamp : 2631 Host timestamp : 2631 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=2631 (Tue Sep 18 12:11:21 2018) host-id=2 score=3400 vm_conf_refresh_time=2631 (Tue Sep 18 12:11:21 2018) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False !! Cluster is in GLOBAL MAINTENANCE mode !! root: INFO: * Agent is ready. root: INFO: * Starting VM... lago.ssh: DEBUG: start task:a93330e3-27f0-4aa9-8491-3716ebc19284:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:a93330e3-27f0-4aa9-8491-3716ebc19284:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 8179b344 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-start lago.ssh: DEBUG: Command 8179b344 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8179b344 on lago-he-basic-ansible-suite-4-2-host-0 output: VM exists and is down, cleaning up and restarting root: INFO: * Command succeeded root: INFO: * Waiting for VM to be UP... lago.ssh: DEBUG: start task:142d6f14-a10b-4195-93eb-b0e1b92926d8:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:142d6f14-a10b-4195-93eb-b0e1b92926d8:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 83234016 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 83234016 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 83234016 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2641 (Tue Sep 18 12:11:30 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "486f8a4a", "local_conf_timestamp": 2641, "host-ts": 2641}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2631 (Tue Sep 18 12:11:21 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2631 (Tue Sep 18 12:11:21 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa3768d0", "local_conf_timestamp": 2631, "host-ts": 2631}, "global_maintenance": true} lago.ssh: DEBUG: start task:5c32f6b0-8dec-42c6-9ec4-1b96850c35b4:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:5c32f6b0-8dec-42c6-9ec4-1b96850c35b4:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 89b990e2 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 89b990e2 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 89b990e2 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2651 (Tue Sep 18 12:11:40 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2651 (Tue Sep 18 12:11:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "159650eb", "local_conf_timestamp": 2651, "host-ts": 2651}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2641 (Tue Sep 18 12:11:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c614bc31", "local_conf_timestamp": 2641, "host-ts": 2641}, "global_maintenance": true} root: INFO: * VM is UP. root: INFO: * Waiting for engine to start... lago.ssh: DEBUG: start task:f05d40f1-75a1-43bf-b63a-a6209f4a0878:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:f05d40f1-75a1-43bf-b63a-a6209f4a0878:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 8a3fc81a on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 8a3fc81a on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 8a3fc81a on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2651 (Tue Sep 18 12:11:40 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2651 (Tue Sep 18 12:11:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "159650eb", "local_conf_timestamp": 2651, "host-ts": 2651}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2641 (Tue Sep 18 12:11:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c614bc31", "local_conf_timestamp": 2641, "host-ts": 2641}, "global_maintenance": true} lago.ssh: DEBUG: start task:8af4d2ba-7614-4b29-8c96-7957e80f3911:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:8af4d2ba-7614-4b29-8c96-7957e80f3911:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 90d5a046 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 90d5a046 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 90d5a046 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a658da6c", "local_conf_timestamp": 2661, "host-ts": 2661}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2651 (Tue Sep 18 12:11:40 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2651 (Tue Sep 18 12:11:40 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "009ff8e7", "local_conf_timestamp": 2651, "host-ts": 2651}, "global_maintenance": true} lago.ssh: DEBUG: start task:ef2e841b-741f-48ab-ab4e-12b9e761930d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:ef2e841b-741f-48ab-ab4e-12b9e761930d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 931b2fb0 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 931b2fb0 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 931b2fb0 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a658da6c", "local_conf_timestamp": 2661, "host-ts": 2661}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "907333dc", "local_conf_timestamp": 2661, "host-ts": 2661}, "global_maintenance": true} lago.ssh: DEBUG: start task:76207a3b-da9a-4920-a92b-137966b18627:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:76207a3b-da9a-4920-a92b-137966b18627:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 956f9c56 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 956f9c56 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 956f9c56 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e73b0f94", "local_conf_timestamp": 2672, "host-ts": 2671}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "907333dc", "local_conf_timestamp": 2661, "host-ts": 2661}, "global_maintenance": true} lago.ssh: DEBUG: start task:9399a6ca-99fa-47fe-99b2-90964b83d06d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:9399a6ca-99fa-47fe-99b2-90964b83d06d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 97c62998 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 97c62998 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 97c62998 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e73b0f94", "local_conf_timestamp": 2672, "host-ts": 2671}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "907333dc", "local_conf_timestamp": 2661, "host-ts": 2661}, "global_maintenance": true} lago.ssh: DEBUG: start task:0c6ad5e5-1a85-4270-9c6d-b9dfe9237306:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:0c6ad5e5-1a85-4270-9c6d-b9dfe9237306:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 9a47efd0 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9a47efd0 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9a47efd0 on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cff126b0", "local_conf_timestamp": 2672, "host-ts": 2671}, "global_maintenance": true} lago.ssh: DEBUG: start task:e00ae9c4-1936-456d-9965-c5ae52b9bd88:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: end task:e00ae9c4-1936-456d-9965-c5ae52b9bd88:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0: lago.ssh: DEBUG: Running 9c9892bc on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json lago.ssh: DEBUG: Command 9c9892bc on lago-he-basic-ansible-suite-4-2-host-0 returned with 0 lago.ssh: DEBUG: Command 9c9892bc on lago-he-basic-ansible-suite-4-2-host-0 output: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f091b01f050> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 187, in <lambda> for k, v in _get_he_status(host).items() File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 128, in _get_he_status raise RuntimeError('could not parse JSON: %s' % ret.out) RuntimeError: could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 53, in restart_he_vm _wait_for_engine_health(host) File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 185, in _wait_for_engine_health testlib.assert_true_within_long(lambda: any( File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 187, in <lambda> for k, v in _get_he_status(host).items() File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 128, in _get_he_status raise RuntimeError('could not parse JSON: %s' % ret.out) 'could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": \n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:6469ff97-fd82-4fba-8353-9b736936a12c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:6469ff97-fd82-4fba-8353-9b736936a12c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 54a3b8d8 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 54a3b8d8 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 54a3b8d8 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2561 (Tue Sep 18 12:10:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5bb7aefb", "local_conf_timestamp": 2561, "host-ts": 2561}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2560 (Tue Sep 18 12:10:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d172629f", "local_conf_timestamp": 2561, "host-ts": 2560}, "global_maintenance": true}\n\nroot: INFO: * Shutting down HE VM on host: lago-he-basic-ansible-suite-4-2-host-0\nlago.ssh: DEBUG: start task:a1e2f644-5484-4092-976b-f93b9bff8d63:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a1e2f644-5484-4092-976b-f93b9bff8d63:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 55273866 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-shutdown\nlago.ssh: DEBUG: Command 55273866 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be down...\nlago.ssh: DEBUG: start task:19bbae64-260d-4591-8719-ec52c46081a3:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:19bbae64-260d-4591-8719-ec52c46081a3:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 55d4e434 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 55d4e434 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 55d4e434 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2561 (Tue Sep 18 12:10:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5bb7aefb", "local_conf_timestamp": 2561, "host-ts": 2561}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2560 (Tue Sep 18 12:10:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2561 (Tue Sep 18 12:10:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "d172629f", "local_conf_timestamp": 2561, "host-ts": 2560}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:41902ed5-eb63-4017-8b3a-b612b34f6bcc:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:41902ed5-eb63-4017-8b3a-b612b34f6bcc:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5c8a15ec on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5c8a15ec on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5c8a15ec on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2571 (Tue Sep 18 12:10:20 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2571 (Tue Sep 18 12:10:20 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Powering down"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "fcf99ac3", "local_conf_timestamp": 2571, "host-ts": 2571}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2570 (Tue Sep 18 12:10:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2570 (Tue Sep 18 12:10:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "60fe16df", "local_conf_timestamp": 2570, "host-ts": 2570}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:f9baa42f-d1b9-44e8-9d0a-42f41a495362:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f9baa42f-d1b9-44e8-9d0a-42f41a495362:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5eddf14c on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5eddf14c on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 5eddf14c on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2581 (Tue Sep 18 12:10:30 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2581 (Tue Sep 18 12:10:31 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e5962d00", "local_conf_timestamp": 2581, "host-ts": 2581}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2580 (Tue Sep 18 12:10:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2580 (Tue Sep 18 12:10:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "de32fbcb", "local_conf_timestamp": 2580, "host-ts": 2580}, "global_maintenance": true}\n\nroot: INFO: * VM is down.\nroot: INFO: * Stopping services...\nlago.ssh: DEBUG: start task:cb8c09f1-87dd-405d-8904-907a1989cb31:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:cb8c09f1-87dd-405d-8904-907a1989cb31:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5f714e2e on lago-he-basic-ansible-suite-4-2-host-0: systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent\nlago.ssh: DEBUG: Command 5f714e2e on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nroot: INFO: * Starting services...\nlago.ssh: DEBUG: start task:7ccaa6ca-f2ea-4e2a-98c6-0c543fd3cb72:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:7ccaa6ca-f2ea-4e2a-98c6-0c543fd3cb72:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 66c72220 on lago-he-basic-ansible-suite-4-2-host-0: systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent\nlago.ssh: DEBUG: Command 66c72220 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for agent to be ready...\nlago.ssh: DEBUG: start task:170c54e8-f608-4663-adb8-07875043648b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:170c54e8-f608-4663-adb8-07875043648b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6938ed86 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 6938ed86 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 6938ed86 on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:181570eb-206a-4ec7-91ea-b0b5169fedfd:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:181570eb-206a-4ec7-91ea-b0b5169fedfd:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6f91b55a on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 6f91b55a on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 6f91b55a on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:326af4dd-1f16-4935-9874-0d80259c7697:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:326af4dd-1f16-4935-9874-0d80259c7697:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71b4333a on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 71b4333a on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 71b4333a on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:4290047c-da3a-49e5-a7b2-a9d60c8383cb:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4290047c-da3a-49e5-a7b2-a9d60c8383cb:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 73da3b46 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 73da3b46 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 73da3b46 on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:e1724b51-9e94-4e12-be43-374a22a56e5c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e1724b51-9e94-4e12-be43-374a22a56e5c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 76029bf2 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 76029bf2 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 76029bf2 on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:3f9b1f59-ee4f-49a6-ad21-c77d3c403aba:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3f9b1f59-ee4f-49a6-ad21-c77d3c403aba:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 783cb2ae on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 783cb2ae on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 783cb2ae on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:0a356812-9169-42e4-bb61-f41c500bab0b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0a356812-9169-42e4-bb61-f41c500bab0b:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7a64f6c2 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 7a64f6c2 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 7a64f6c2 on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:4440aaf7-97cd-4106-9d80-4f3d6265ba1d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4440aaf7-97cd-4106-9d80-4f3d6265ba1d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7c8cedf6 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 7c8cedf6 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 7c8cedf6 on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:4a3838f9-2b52-4189-8915-0c39dc71ccb1:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:4a3838f9-2b52-4189-8915-0c39dc71ccb1:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7eb64906 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 7eb64906 on lago-he-basic-ansible-suite-4-2-host-0 returned with 1\nlago.ssh: DEBUG: Command 7eb64906 on lago-he-basic-ansible-suite-4-2-host-0 output:\n The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.\n\nlago.ssh: DEBUG: start task:78ae6cc0-4511-4e8f-89c1-d015830bab8c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:78ae6cc0-4511-4e8f-89c1-d015830bab8c:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 80ec9144 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status\nlago.ssh: DEBUG: Command 80ec9144 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 80ec9144 on lago-he-basic-ansible-suite-4-2-host-0 output:\n \n\n!! Cluster is in GLOBAL MAINTENANCE mode !!\n\n\n\n--== Host 1 status ==--\n\nconf_on_shared_storage : True\nStatus up-to-date : True\nHostname : lago-he-basic-ansible-suite-4-2-host-0\nHost ID : 1\nEngine status : {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}\nScore : 0\nstopped : False\nLocal maintenance : False\ncrc32 : 486f8a4a\nlocal_conf_timestamp : 2641\nHost timestamp : 2641\nExtra metadata (valid at timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=2641 (Tue Sep 18 12:11:30 2018)\n\thost-id=1\n\tscore=0\n\tvm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=ReinitializeFSM\n\tstopped=False\n\n\n--== Host 2 status ==--\n\nconf_on_shared_storage : True\nStatus up-to-date : False\nHostname : lago-he-basic-ansible-suite-4-2-host-1\nHost ID : 2\nEngine status : unknown stale-data\nScore : 3400\nstopped : False\nLocal maintenance : False\ncrc32 : fa3768d0\nlocal_conf_timestamp : 2631\nHost timestamp : 2631\nExtra metadata (valid at timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=2631 (Tue Sep 18 12:11:21 2018)\n\thost-id=2\n\tscore=3400\n\tvm_conf_refresh_time=2631 (Tue Sep 18 12:11:21 2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=GlobalMaintenance\n\tstopped=False\n\n\n!! Cluster is in GLOBAL MAINTENANCE mode !!\n\n\nroot: INFO: * Agent is ready.\nroot: INFO: * Starting VM...\nlago.ssh: DEBUG: start task:a93330e3-27f0-4aa9-8491-3716ebc19284:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a93330e3-27f0-4aa9-8491-3716ebc19284:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8179b344 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-start\nlago.ssh: DEBUG: Command 8179b344 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8179b344 on lago-he-basic-ansible-suite-4-2-host-0 output:\n VM exists and is down, cleaning up and restarting\n\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be UP...\nlago.ssh: DEBUG: start task:142d6f14-a10b-4195-93eb-b0e1b92926d8:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:142d6f14-a10b-4195-93eb-b0e1b92926d8:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 83234016 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 83234016 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 83234016 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2641 (Tue Sep 18 12:11:30 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32": "486f8a4a", "local_conf_timestamp": 2641, "host-ts": 2641}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2631 (Tue Sep 18 12:11:21 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2631 (Tue Sep 18 12:11:21 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "fa3768d0", "local_conf_timestamp": 2631, "host-ts": 2631}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:5c32f6b0-8dec-42c6-9ec4-1b96850c35b4:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:5c32f6b0-8dec-42c6-9ec4-1b96850c35b4:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 89b990e2 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 89b990e2 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 89b990e2 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2651 (Tue Sep 18 12:11:40 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2651 (Tue Sep 18 12:11:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "159650eb", "local_conf_timestamp": 2651, "host-ts": 2651}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2641 (Tue Sep 18 12:11:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c614bc31", "local_conf_timestamp": 2641, "host-ts": 2641}, "global_maintenance": true}\n\nroot: INFO: * VM is UP.\nroot: INFO: * Waiting for engine to start...\nlago.ssh: DEBUG: start task:f05d40f1-75a1-43bf-b63a-a6209f4a0878:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:f05d40f1-75a1-43bf-b63a-a6209f4a0878:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 8a3fc81a on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 8a3fc81a on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 8a3fc81a on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2651 (Tue Sep 18 12:11:40 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2651 (Tue Sep 18 12:11:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "159650eb", "local_conf_timestamp": 2651, "host-ts": 2651}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2641 (Tue Sep 18 12:11:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2641 (Tue Sep 18 12:11:30 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c614bc31", "local_conf_timestamp": 2641, "host-ts": 2641}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:8af4d2ba-7614-4b29-8c96-7957e80f3911:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:8af4d2ba-7614-4b29-8c96-7957e80f3911:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 90d5a046 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 90d5a046 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 90d5a046 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a658da6c", "local_conf_timestamp": 2661, "host-ts": 2661}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2651 (Tue Sep 18 12:11:40 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2651 (Tue Sep 18 12:11:40 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "009ff8e7", "local_conf_timestamp": 2651, "host-ts": 2651}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:ef2e841b-741f-48ab-ab4e-12b9e761930d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:ef2e841b-741f-48ab-ab4e-12b9e761930d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 931b2fb0 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 931b2fb0 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 931b2fb0 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "a658da6c", "local_conf_timestamp": 2661, "host-ts": 2661}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "907333dc", "local_conf_timestamp": 2661, "host-ts": 2661}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:76207a3b-da9a-4920-a92b-137966b18627:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:76207a3b-da9a-4920-a92b-137966b18627:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 956f9c56 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 956f9c56 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 956f9c56 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e73b0f94", "local_conf_timestamp": 2672, "host-ts": 2671}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "907333dc", "local_conf_timestamp": 2661, "host-ts": 2661}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:9399a6ca-99fa-47fe-99b2-90964b83d06d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:9399a6ca-99fa-47fe-99b2-90964b83d06d:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 97c62998 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 97c62998 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 97c62998 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "e73b0f94", "local_conf_timestamp": 2672, "host-ts": 2671}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2661 (Tue Sep 18 12:11:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2661 (Tue Sep 18 12:11:51 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "907333dc", "local_conf_timestamp": 2661, "host-ts": 2661}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:0c6ad5e5-1a85-4270-9c6d-b9dfe9237306:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:0c6ad5e5-1a85-4270-9c6d-b9dfe9237306:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9a47efd0 on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9a47efd0 on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9a47efd0 on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "cff126b0", "local_conf_timestamp": 2672, "host-ts": 2671}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start task:e00ae9c4-1936-456d-9965-c5ae52b9bd88:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:e00ae9c4-1936-456d-9965-c5ae52b9bd88:Get ssh client for lago-he-basic-ansible-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 9c9892bc on lago-he-basic-ansible-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 9c9892bc on lago-he-basic-ansible-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 9c9892bc on lago-he-basic-ansible-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": \novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at 0x7f091b01f050>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 187, in <lambda>\n for k, v in _get_he_status(host).items()\n File "/home/jenkins/workspace/ovirt-system-tests_he-basic-ansible-suite-4.2/ovirt-system-tests/he-basic-ansible-suite-4.2/test-scenarios/008_restart_he_vm.py", line 128, in _get_he_status\n raise RuntimeError(\'could not parse JSON: %s\' % ret.out)\nRuntimeError: could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2681 (Tue Sep 18 12:12:10 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2681 (Tue Sep 18 12:12:10 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-0", "host-id": 1, "engine-status": {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "ca5cd2cc", "local_conf_timestamp": 2681, "host-ts": 2681}, "2": {"conf_on_shared_storage": true, "live-data": false, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2671 (Tue Sep 18 12:12:01 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2672 (Tue Sep 18 12:12:01 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n", "hostname": "lago-he-basic-ansible-suite-4-2-host-1", "host-id": \n--------------------- >> end captured logging << ---------------------'

1 1

oVirt infra daily report - unstable production jobs - 587
by jenkins＠jenkins.phx.ovirt.org 19 Sep '18

19 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/587//artifact/exported-a… Cheers, Jenkins

1 0

[CQ]: 6993caa (ovirt-ansible-manageiq) failed "ovirt-master" system tests
by oVirt Jenkins 18 Sep '18

18 Sep '18

Change 6993caa (ovirt-ansible-manageiq) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/ovirt-ansible-manageiq/commit/6993caa02dc761e0f874… For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10346/

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren reassigned OVIRT-2504: ----------------------------------- Assignee: Daniel Belenky (was: infra) > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: docker_cleanup.py > Reporter: Petr Kotas > Assignee: Daniel Belenky > Priority: Highest > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2504: -------------------------------- Epic Link: OVIRT-2339 > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: docker_cleanup.py > Reporter: Petr Kotas > Assignee: infra > Priority: Highest > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2504: -------------------------------- Epic Link: OVIRT-2339 > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: docker_cleanup.py > Reporter: Petr Kotas > Assignee: infra > Priority: Highest > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2504: -------------------------------- Priority: Highest (was: Medium) > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: docker_cleanup.py > Reporter: Petr Kotas > Assignee: infra > Priority: Highest > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2504: -------------------------------- Issue Type: Bug (was: By-EMAIL) > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: docker_cleanup.py > Reporter: Petr Kotas > Assignee: infra > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2504: -------------------------------- Component/s: docker_cleanup.py > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: docker_cleanup.py > Reporter: Petr Kotas > Assignee: infra > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2504: ------------------------------------- [~dbelenky(a)redhat.com] docker-cleanup seems to be timing out talking to Dockerd. Can you have a look? Lets see if we can have a quick and dirty work around in place before we all go on a long PTO > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Petr Kotas > Assignee: infra > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2504) standard CI fails in project_setup phase
by Barak Korren (oVirt JIRA) 18 Sep '18

18 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2504?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2504: ------------------------------------- [~dbelenky(a)redhat.com] docker-cleanup seems to be timing out talking to Dockerd. Can you have a look? Lets see if we can have a quick and dirty work around in place before we all go on a long PTO > standard CI fails in project_setup phase > ---------------------------------------- > > Key: OVIRT-2504 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2504 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Petr Kotas > Assignee: infra > > Hi, > the standard CI is failing in project_setup phase. Can you please debug > that? > The issue is somehow related to docker clean up phase. > Example of failing CI is : > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > - > https://jenkins.ovirt.org/blue/organizations/jenkins/kubevirt_kubevirt_stan… > Unfortunately, the logs are the only thing I can see. From our side, there > is nothing special. > Just normal test run. We did not tune anything. > Thank you, > Best, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[CQ]: 6993caa (ovirt-ansible-manageiq) failed "ovirt-4.2" system tests
by oVirt Jenkins 18 Sep '18

18 Sep '18

Change 6993caa (ovirt-ansible-manageiq) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/ovirt-ansible-manageiq/commit/6993caa02dc761e0f874… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/3136/

1 0

oVirt infra daily report - unstable production jobs - 586
by jenkins＠jenkins.phx.ovirt.org 18 Sep '18

18 Sep '18

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/586//artifact/exported-a… Cheers, Jenkins

1 0

Build failed in Jenkins: system-sync_mirrors-fedora-base-fc28-x86_64 #417
by jenkins＠jenkins.phx.ovirt.org 17 Sep '18

17 Sep '18

See <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc28-x86_64/41…> ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc28-x86_64/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git > git --version # timeout=10 > git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision f9508e32027645ea57bcfec09dafe628fbb06677 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f f9508e32027645ea57bcfec09dafe628fbb06677 Commit message: "standard_pipelines: add nmstate project" > git rev-list --no-walk f9508e32027645ea57bcfec09dafe628fbb06677 # timeout=10 [system-sync_mirrors-fedora-base-fc28-x86_64] $ /bin/bash -xe /tmp/jenkins5577468338365067102.sh + jenkins/scripts/mirror_mgr.sh resync_yum_mirror fedora-base-fc28 x86_64 jenkins/data/mirrors-reposync.conf Checking if mirror needs a resync Traceback (most recent call last): File "/usr/bin/reposync", line 343, in <module> main() File "/usr/bin/reposync", line 175, in main my.doRepoSetup() File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup return self._getRepos(thisrepo, True) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos self._repos.doSetup(thisrepo) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML local = self.cachedir + '/repomd.xml' File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda> cachedir = property(lambda self: self._dirGetAttr('cachedir')) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr self.dirSetup() File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup self._dirSetupMkdir_p(dir) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p raise Errors.RepoError, msg yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/fedora-base-fc28 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/fedora-base-fc28' Build step 'Execute shell' marked build as failure

1 1

[JIRA] (OVIRT-2463) http://192.168.201.1:8585/el7/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
by Dafna Ron (oVirt JIRA) 17 Sep '18

17 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2463?page=com.atlassian.jira.… ] Dafna Ron commented on OVIRT-2463: ---------------------------------- [~eyonasi(a)redhat.com] [~ybardavi] can we close this? > http://192.168.201.1:8585/el7/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found > ---------------------------------------------------------------------------------------- > > Key: OVIRT-2463 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2463 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Change Queue > Reporter: Anton Marchukov > Assignee: infra > Labels: ost_failures > > Project name: > ovirt-ansible-engine-setup > Branch: > master > Failed Suite: > upgrade-from-release-suite.el7.x86_64 > Failed test/failed phase: > 001_upgrade_engine.test_initialize_engine > Link to first reported failed build: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/9255/ > Link to first reported failed patch: > ??? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2467) Failed connect to gerrit.ovirt.org:443; No route to host
by Dafna Ron (oVirt JIRA) 17 Sep '18

17 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2467?page=com.atlassian.jira.… ] Dafna Ron updated OVIRT-2467: ----------------------------- Resolution: Cannot Reproduce Status: Done (was: To Do) > Failed connect to gerrit.ovirt.org:443; No route to host > -------------------------------------------------------- > > Key: OVIRT-2467 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2467 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Anton Marchukov > Assignee: infra > Labels: ost_failures, ost_infra > > Project name: > ovirt-ansible-engine-setup > Branch: > master > Failed Suite: > ovirt-master_change-queue-tester > Link to first reported failed build: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/9260 > the root cause is that > oVirt_ovirt-ansible-engine-setup_standard-on-ghpush (29) failed building > and that is caused by: > https://jenkins.ovirt.org/job/oVirt_ovirt-ansible-engine-setup_standard-on-… > 11:44:39 [build-artifacts.fc27.x86_64] ERROR: Error cloning remote repo 'origin' > 11:44:39 [build-artifacts.fc27.x86_64] hudson.plugins.git.GitException: Command "git fetch --tags --progress https://gerrit.ovirt.org/jenkins +refs/heads/*:refs/remotes/origin/*" returned status code 128: > 11:44:39 [build-artifacts.fc27.x86_64] stdout: > 11:44:39 [build-artifacts.fc27.x86_64] stderr: fatal: unable to access 'https://gerrit.ovirt.org/jenkins/': Failed connect to gerrit.ovirt.org:443; No route to host -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0