[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build #
429 - Failure!
by jenkins@jenkins.phx.ovirt.org
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/429/
Build Number: 429
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #429
[Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 012_local_maintenance_sdk.local_maintenance
Error Message:
143
-------------------- >> begin captured logging << --------------------
root: INFO: * Waiting For System Stability...
root: INFO: * Performing Deactivation...
root: INFO: * Performing Activation...
root: INFO: * Waiting For System Stability...
cli: DEBUG: signal 15 was caught
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper
prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/012_local_maintenance_sdk.py", line 79, in local_maintenance
time.sleep(wait_value)
File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler
sys.exit(128 + signum)
'143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nroot: INFO: * Performing Deactivation...\nroot: INFO: * Performing Activation...\nroot: INFO: * Waiting For System Stability...\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
6 years, 2 months
[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.2 - Build # 590
- Failure!
by jenkins@jenkins.phx.ovirt.org
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/590/
Build Number: 590
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #590
[Miguel Duarte Barroso] Openvswitch 2.10 requires latest dpdk version
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 002_bootstrap.add_hosts
Error Message:
The response content type 'text/html; charset=iso-8859-1' isn't the expected XML
-------------------- >> begin captured logging << --------------------
ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7f2465ad1578>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within
res = func()
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4
host_obj = host_service.get()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37093, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault
body = self._internal_read_body(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body
self._connection.check_xml_content_type(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type
response.headers
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type
raise Error(msg)
Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 229, in add_hosts
add_hosts_4(prefix)
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 273, in add_hosts_4
testlib.assert_true_within(_host_is_up_4, timeout=15*60)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within
res = func()
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4
host_obj = host_service.get()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37093, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault
body = self._internal_read_body(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body
self._connection.check_xml_content_type(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type
response.headers
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type
raise Error(msg)
'The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n-------------------- >> begin captured logging << --------------------\novirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7f2465ad1578>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.2/ovirt-system-tests/hc-basic-suite-4.2/test-scenarios/002_bootstrap.py", line 254, in _host_is_up_4\n host_obj = host_service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 37093, in get\n return self._internal_get(headers, query, wait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get\n return future.wait() if wait else future\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait\n return self._code(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback\n self._check_fault(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault\n body = self._internal_read_body(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body\n self._connection.check_xml_content_type(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 849, in check_xml_content_type\n response.headers\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 889, in _check_content_type\n raise Error(msg)\nError: The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n--------------------- >> end captured logging << ---------------------'
6 years, 2 months
[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-master - Build #
804 - Failure!
by jenkins@jenkins.phx.ovirt.org
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/804/
Build Number: 804
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #804
[Galit] Fix stdci.yaml to be able to run poll upstream sources
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 002_bootstrap.add_hosts
Error Message:
The response content type 'text/html; charset=iso-8859-1' isn't the expected XML
-------------------- >> begin captured logging << --------------------
ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7fde909da230>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within
res = func()
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 253, in _host_is_up_4
host_obj = host_service.get()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 38104, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault
body = self._internal_read_body(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body
self._connection.check_xml_content_type(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 850, in check_xml_content_type
response.headers
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 890, in _check_content_type
raise Error(msg)
Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 228, in add_hosts
add_hosts_4(prefix)
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 275, in add_hosts_4
testlib.assert_true_within(_host_is_up_4, timeout=15*60)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 278, in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within
res = func()
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 253, in _host_is_up_4
host_obj = host_service.get()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 38104, in get
return self._internal_get(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault
body = self._internal_read_body(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body
self._connection.check_xml_content_type(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 850, in check_xml_content_type
response.headers
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 890, in _check_content_type
raise Error(msg)
'The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n-------------------- >> begin captured logging << --------------------\novirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up_4 at 0x7fde909da230>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 234, in assert_equals_within\n res = func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-master/ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 253, in _host_is_up_4\n host_obj = host_service.get()\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 38104, in get\n return self._internal_get(headers, query, wait)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get\n return future.wait() if wait else future\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait\n return self._code(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback\n self._check_fault(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 130, in _check_fault\n body = self._internal_read_body(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 316, in _internal_read_body\n self._connection.check_xml_content_type(response)\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 850, in check_xml_content_type\n response.headers\n File "/usr/lib64/python2.7/site-packages/ovirtsdk4/__init__.py", line 890, in _check_content_type\n raise Error(msg)\nError: The response content type \'text/html; charset=iso-8859-1\' isn\'t the expected XML\n--------------------- >> end captured logging << ---------------------'
6 years, 2 months
[oVirt Jenkins] ovirt-system-tests_he-node-ng-suite-4.2 - Build #
411 - Failure!
by jenkins@jenkins.phx.ovirt.org
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-4.2/411/
Build Number: 411
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #411
[Galit] Update the config for poll u/s sources
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 010_local_maintenance_cli.local_maintenance
Error Message:
143
-------------------- >> begin captured logging << --------------------
root: INFO: * Waiting For System Stability...
lago.ssh: DEBUG: start task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1:
lago.ssh: DEBUG: end task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1:
lago.ssh: DEBUG: Running 5e792150 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 returned with 0
lago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3933 (Thu Oct 18 01:44:27 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3933 (Thu Oct 18 01:44:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7c0dd5c", "local_conf_timestamp": 3933, "host-ts": 3933}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}
lago.ssh: DEBUG: start task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 5f815572 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local
lago.ssh: DEBUG: Command 5f815572 on lago-he-node-ng-suite-4-2-host-0 returned with 0
root: INFO: * Waiting for engine to migrate...
lago.ssh: DEBUG: start task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 60b78506 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3943 (Thu Oct 18 01:44:38 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=3943 (Thu Oct 18 01:44:38 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8f671722", "local_conf_timestamp": 3943, "host-ts": 3943}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}
lago.ssh: DEBUG: start task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 67d368b4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3953 (Thu Oct 18 01:44:47 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3953 (Thu Oct 18 01:44:47 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "706f0031", "local_conf_timestamp": 3953, "host-ts": 3953}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3945 (Thu Oct 18 01:44:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3945 (Thu Oct 18 01:44:39 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}
lago.ssh: DEBUG: start task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 6bf19f06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false}
lago.ssh: DEBUG: start task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 6ef7539e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false}
lago.ssh: DEBUG: start task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 71ed0dfa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}
lago.ssh: DEBUG: start task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 74df0478 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}
lago.ssh: DEBUG: start task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 77d8a918 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}
lago.ssh: DEBUG: start task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 7ac56292 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}
lago.ssh: DEBUG: start task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 7db577e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}
lago.ssh: DEBUG: start task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: end task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0:
lago.ssh: DEBUG: Running 81550626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 returned with 0
lago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineMigratingAway\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}
cli: DEBUG: signal 15 was caught
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 125, in local_maintenance
_wait_for_engine_migration(host, he_index, "bad", "Migration Destination")
File "/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-4.2/ovirt-system-tests/he-node-ng-suite-4.2/test-scenarios/010_local_maintenance_cli.py", line 36, in _wait_for_engine_migration
testlib.assert_true_within_long(lambda: _get_he_status(host)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 286, in assert_true_within_long
assert_equals_within_long(func, True, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 273, in assert_equals_within_long
func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 246, in assert_equals_within
time.sleep(3)
File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 922, in exit_handler
sys.exit(128 + signum)
'143\n-------------------- >> begin captured logging << --------------------\nroot: INFO: * Waiting For System Stability...\nlago.ssh: DEBUG: start task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: end task:461f166d-b3cb-4766-af33-4319bd3b5227:Get ssh client for lago-he-node-ng-suite-4-2-host-1:\nlago.ssh: DEBUG: Running 5e792150 on lago-he-node-ng-suite-4-2-host-1: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 returned with 0\nlago.ssh: DEBUG: Command 5e792150 on lago-he-node-ng-suite-4-2-host-1 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3933 (Thu Oct 18 01:44:27 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3933 (Thu Oct 18 01:44:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "f7c0dd5c", "local_conf_timestamp": 3933, "host-ts": 3933}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a64ae178-4ca1-4b6a-b5a6-e4250fe83398:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 5f815572 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --set-maintenance --mode=local\nlago.ssh: DEBUG: Command 5f815572 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nroot: INFO: * Waiting for engine to migrate...\nlago.ssh: DEBUG: start task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:23a6c556-e234-43d1-8313-4612bd35fdf0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 60b78506 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 60b78506 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3943 (Thu Oct 18 01:44:38 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=3943 (Thu Oct 18 01:44:38 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "8f671722", "local_conf_timestamp": 3943, "host-ts": 3943}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3935 (Thu Oct 18 01:44:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3935 (Thu Oct 18 01:44:29 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "da25a075", "local_conf_timestamp": 3935, "host-ts": 3935}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:080e3a23-a075-43dd-895c-00855470de8a:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 67d368b4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 67d368b4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3953 (Thu Oct 18 01:44:47 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3953 (Thu Oct 18 01:44:47 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "706f0031", "local_conf_timestamp": 3953, "host-ts": 3953}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3945 (Thu Oct 18 01:44:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3945 (Thu Oct 18 01:44:39 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "c7f55d6e", "local_conf_timestamp": 3945, "host-ts": 3945}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:592ee562-ac13-4dee-9eeb-8e72bbe78ced:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6bf19f06 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6bf19f06 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:3aac038b-901e-4bca-8616-7dae3ce335b0:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 6ef7539e on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 6ef7539e on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3963 (Thu Oct 18 01:44:58 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3963 (Thu Oct 18 01:44:58 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "6263fdaa", "local_conf_timestamp": 3963, "host-ts": 3963}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3955 (Thu Oct 18 01:44:50 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3956 (Thu Oct 18 01:44:50 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "5abaf3d1", "local_conf_timestamp": 3956, "host-ts": 3955}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:a8248fe7-4e2c-455d-b5ec-b1954bf41a8e:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 71ed0dfa on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 71ed0dfa on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:65b6000b-8e80-4dee-af83-a6a74c034238:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 74df0478 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 74df0478 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3973 (Thu Oct 18 01:45:08 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3974 (Thu Oct 18 01:45:08 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "523456cb", "local_conf_timestamp": 3974, "host-ts": 3973}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3965 (Thu Oct 18 01:44:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3965 (Thu Oct 18 01:44:59 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "b84596fc", "local_conf_timestamp": 3965, "host-ts": 3965}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:847d9012-2456-4e12-b2f9-c7e4da9c4ff7:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 77d8a918 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 77d8a918 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:dd6b5da4-d22b-4284-85d5-dfe9a51ee00c:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7ac56292 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7ac56292 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3983 (Thu Oct 18 01:45:17 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3983 (Thu Oct 18 01:45:17 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "41e59273", "local_conf_timestamp": 3983, "host-ts": 3983}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3975 (Thu Oct 18 01:45:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3975 (Thu Oct 18 01:45:09 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "bc0369f9", "local_conf_timestamp": 3975, "host-ts": 3975}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:315c897e-fc06-484c-a036-617859dbb2ec:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 7db577e4 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 7db577e4 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\nlago.ssh: DEBUG: start task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: end task:c5f00620-37e4-4d82-b99e-e9ac42e3e145:Get ssh client for lago-he-node-ng-suite-4-2-host-0:\nlago.ssh: DEBUG: Running 81550626 on lago-he-node-ng-suite-4-2-host-0: hosted-engine --vm-status --json\nlago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 returned with 0\nlago.ssh: DEBUG: Command 81550626 on lago-he-node-ng-suite-4-2-host-0 output:\n {"1": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3993 (Thu Oct 18 01:45:28 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=3994 (Thu Oct 18 01:45:28 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineMigratingAway\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-0", "host-id": 1, "engine-status": {"health": "good", "vm": "up", "detail": "Migration Source"}, "score": 3000, "stopped": false, "maintenance": false, "crc32": "71b23912", "local_conf_timestamp": 3994, "host-ts": 3993}, "2": {"conf_on_shared_storage": true, "live-data": true, "extra": "metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=3985 (Thu Oct 18 01:45:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=3985 (Thu Oct 18 01:45:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n", "hostname": "lago-he-node-ng-suite-4-2-host-1", "host-id": 2, "engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down", "detail": "Migration Destination"}, "score": 3400, "stopped": false, "maintenance": false, "crc32": "83e46cca", "local_conf_timestamp": 3985, "host-ts": 3985}, "global_maintenance": false}\n\ncli: DEBUG: signal 15 was caught\n--------------------- >> end captured logging << ---------------------'
6 years, 2 months
[JIRA] (OVIRT-2558) containerize more bare metals
by Evgheni Dereveanchin (oVirt JIRA)
Evgheni Dereveanchin created OVIRT-2558:
-------------------------------------------
Summary: containerize more bare metals
Key: OVIRT-2558
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2558
Project: oVirt - virtualization made easy
Issue Type: Task
Reporter: Evgheni Dereveanchin
Assignee: infra
The build queue on the Jenkins is growing and we still have two larger bare metals under-utilized as they run just a single Lago instance at a time: ovirt-srv07 and ovirt-srv08
I've offlined them so that they can be containerized and run up to 7 CI suites at the same time, same as we have on ovirt-srv04-06 at the moment.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100094)
6 years, 2 months