- Devel - oVirt List Archives

[ OST Failure Report ] [ oVirt master ] [ 28-05-2017 ] [ 004_basic_sanity.hotunplug_disk ]
by Dafna Ron 29 May '17

29 May '17

This is a multi-part message in MIME format. --------------992DE8C067CE91FDF29339AC Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ** *Hi All, * * ** *We are seeing the same failure**sporadically**in *three*different places (hotunplug disk). * *We think this is a race condition which was introduced to OST with the the live storage migration test (lock on the disk is preventing the hotunplug). * Test failed: [ 004_basic_sanity.hotunplug_disk] Link to suspected patches: https://gerrit.ovirt.org/#/c/77083 Link to Job: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881 ** Link to all logs: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881/artif… Error snippet from the log: <error> * Error Message status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied. Stacktrace Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 68, in wrapper return func(prefix.virt_env.engine_vm().get_api(), *args, **kwargs) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 498, in hotunplug_disk disk.deactivate() File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31839, in deactivate headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied. from engine log: 2017-05-28 09:05:32,891-04 INFO [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISKS_LOCKED$diskAliases vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_VM_IS_LOCKED>]'}' 2017-05-28 09:05:32,891-04 WARN [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Validation of action 'HotUnPlugDiskFromVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__HOT_UNPLUG,VAR__TYPE__DISK,ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED,$DiskName vm0_disk0 2017-05-28 09:05:32,895-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-10) [] Operation Failed: [Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied.] 2017-05-28 09:05:33,323-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Ending command 'org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand' successfully. 2017-05-28 09:05:33,324-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Lock freed to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED$DiskName vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>]'}' ** *</error>* * * --------------992DE8C067CE91FDF29339AC Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Hi All, <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">We are seeing the same failure<span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> sporadically<span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> in <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">three<span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> different places (hotunplug disk). <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">We think this is a race condition which was introduced to OST with the the live storage migration test (lock on the disk is preventing the hotunplug). <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Test failed: [ 004_basic_sanity.hotunplug_disk] <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/77083">https://gerrit.ovirt.org/#/c/77083</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs: <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881/artif…">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6881/artif…</a> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Error snippet from the log: <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error> <h3 style="box-sizing: border-box; color: rgb(51, 51, 51); font-family: Helvetica, Arial, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Error Message</h3> <pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied.</pre> <h3 style="box-sizing: border-box; color: rgb(51, 51, 51); font-family: Helvetica, Arial, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Stacktrace</h3> <pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 68, in wrapper return func(prefix.virt_env.engine_vm().get_api(), *args, **kwargs) File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 498, in hotunplug_disk disk.deactivate() File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31839, in deactivate headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 409 reason: Conflict detail: Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied. from engine log: 2017-05-28 09:05:32,891-04 INFO [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISKS_LOCKED$diskAliases vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_VM_IS_LOCKED>]'}' 2017-05-28 09:05:32,891-04 WARN [org.ovirt.engine.core.bll.storage.disk.HotUnPlugDiskFromVmCommand] (default task-10) [6d47ba60-4491-48ab-bb1c-fb0cdbeb76b0] Validation of action 'HotUnPlugDiskFromVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__HOT_UNPLUG,VAR__TYPE__DISK,ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED,$DiskName vm0_disk0 2017-05-28 09:05:32,895-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-10) [] Operation Failed: [Cannot hot unplug Virtual Disk. Disk vm0_disk0 is being moved or copied.] 2017-05-28 09:05:33,323-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Ending command 'org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand' successfully. 2017-05-28 09:05:33,324-04 INFO [org.ovirt.engine.core.bll.storage.lsm.LiveMigrateVmDisksCommand] (DefaultQuartzScheduler8) [99f56ceb-3c13-423f-b015-71222a8fec01] Lock freed to object 'EngineLock:{exclusiveLocks='[9e7d8b37-2afb-4012-9bc9-8cc277ac9cdd=<DISK, ACTION_TYPE_FAILED_DISK_IS_BEING_MIGRATED$DiskName vm0_disk0>]', sharedLocks='[461224f5-a2af-4be2-ba1b-3563a88f398c=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>]'}' </pre> <span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error> </body> </html> --------------992DE8C067CE91FDF29339AC--

3 3

Re: [ovirt-devel] [oVirt Jenkins] ovirt_4.1_hc-system-tests - Build # 76 - Failure!
by Eyal Edri 29 May '17

29 May '17

Adding devel and Piotr. On Thu, May 25, 2017 at 9:31 AM, Sahina Bose <sabose(a)redhat.com> wrote: > Connecting to lago-hc-basic-suite-4-1-host0.lago.local/192.168.200.2 > 2017-05-24 23:31:00,492-04 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-1) [712d05a6] Error: org.ovirt.engine.core.vdsbroker.TransportRunTimeException: Connection issues during send request > 2017-05-24 23:31:00,492-04 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-1) [712d05a6] Exception: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.TransportRunTimeException: Connection issues during send request > Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils from [Module "org.ovirt.vdsm-jsonrpc-java:main" from local module loader @5e91993f (finder: local module finder @1c4af82c (roots: /usr/share/ovirt-engine-wildfly-overlay/modules,/usr/share/ovirt-engine/modules/common,/usr/share/ovirt-engine-extension-aaa-jdbc/modules,/usr/share/ovirt-engine-wildfly/modules,/usr/share/ovirt-engine-wildfly/modules/system/layers/base))] > > > Known issue? > > > On Thu, May 25, 2017 at 9:16 AM, <jenkins(a)jenkins.phx.ovirt.org> wrote: > >> Project: http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/ >> Build: http://jenkins.ovirt.org/job/ovirt_4.1_hc-system-tests/76/ >> Build Number: 76 >> Build Status: Failure >> Triggered By: Started by timer >> >> ------------------------------------- >> Changes Since Last Success: >> ------------------------------------- >> Changes for Build #76 >> [Ondřej Svoboda] Use SDKv4 code in add_labeled_network. >> >> [Barak Korren] Refactor: Use macros in defaults file >> >> [Barak Korren] Default values for 'arch' and 'distro' in STD-CI >> >> >> >> >> ----------------- >> Failed Tests: >> ----------------- >> 1 tests failed. >> FAILED: 002_bootstrap.add_hosts >> >> Error Message: >> >> status: 409 >> reason: Conflict >> detail: Cannot add Host. There is no available server in the cluster to >> probe the new server. >> -------------------- >> begin captured logging << -------------------- >> lago.utils: ERROR: Error while running thread >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in >> _ret_via_queue >> queue.put({'return': func()}) >> File "/home/jenkins/workspace/ovirt_4.1_hc-system-tests/ovirt-sys >> tem-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 141, >> in _add_host >> return api.hosts.add(p) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", >> line 18306, in add >> headers={"Correlation-Id":correlation_id, "Expect":expect} >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 79, in add >> return self.request('POST', url, body, headers, cls=cls) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 122, in request >> persistent_auth=self.__persistent_auth >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 79, in do_request >> persistent_auth) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 162, in __do_request >> raise errors.RequestError(response_code, response_reason, >> response_body) >> RequestError: >> status: 409 >> reason: Conflict >> detail: Cannot add Host. There is no available server in the cluster to >> probe the new server. >> lago.utils: ERROR: Error while running thread >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in >> _ret_via_queue >> queue.put({'return': func()}) >> File "/home/jenkins/workspace/ovirt_4.1_hc-system-tests/ovirt-sys >> tem-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 141, >> in _add_host >> return api.hosts.add(p) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", >> line 18306, in add >> headers={"Correlation-Id":correlation_id, "Expect":expect} >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 79, in add >> return self.request('POST', url, body, headers, cls=cls) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 122, in request >> persistent_auth=self.__persistent_auth >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 79, in do_request >> persistent_auth) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 162, in __do_request >> raise errors.RequestError(response_code, response_reason, >> response_body) >> RequestError: >> status: 409 >> reason: Conflict >> detail: Cannot add Host. There is no available server in the cluster to >> probe the new server. >> --------------------- >> end captured logging << --------------------- >> >> Stack Trace: >> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >> testMethod() >> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in >> runTest >> self.test(*self.arg) >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line >> 129, in wrapped_test >> test() >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, >> in wrapper >> return func(get_test_prefix(), *args, **kwargs) >> File "/home/jenkins/workspace/ovirt_4.1_hc-system-tests/ovirt-sys >> tem-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 160, >> in add_hosts >> nt.assert_true(all(vt.join_all())) >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in >> _ret_via_queue >> queue.put({'return': func()}) >> File "/home/jenkins/workspace/ovirt_4.1_hc-system-tests/ovirt-sys >> tem-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 141, >> in _add_host >> return api.hosts.add(p) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", >> line 18306, in add >> headers={"Correlation-Id":correlation_id, "Expect":expect} >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 79, in add >> return self.request('POST', url, body, headers, cls=cls) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 122, in request >> persistent_auth=self.__persistent_auth >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 79, in do_request >> persistent_auth) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 162, in __do_request >> raise errors.RequestError(response_code, response_reason, >> response_body) >> >> status: 409 >> reason: Conflict >> detail: Cannot add Host. There is no available server in the cluster to >> probe the new server. >> -------------------- >> begin captured logging << -------------------- >> lago.utils: ERROR: Error while running thread >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in >> _ret_via_queue >> queue.put({'return': func()}) >> File "/home/jenkins/workspace/ovirt_4.1_hc-system-tests/ovirt-sys >> tem-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 141, >> in _add_host >> return api.hosts.add(p) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", >> line 18306, in add >> headers={"Correlation-Id":correlation_id, "Expect":expect} >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 79, in add >> return self.request('POST', url, body, headers, cls=cls) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 122, in request >> persistent_auth=self.__persistent_auth >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 79, in do_request >> persistent_auth) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 162, in __do_request >> raise errors.RequestError(response_code, response_reason, >> response_body) >> RequestError: >> status: 409 >> reason: Conflict >> detail: Cannot add Host. There is no available server in the cluster to >> probe the new server. >> lago.utils: ERROR: Error while running thread >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in >> _ret_via_queue >> queue.put({'return': func()}) >> File "/home/jenkins/workspace/ovirt_4.1_hc-system-tests/ovirt-sys >> tem-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 141, >> in _add_host >> return api.hosts.add(p) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", >> line 18306, in add >> headers={"Correlation-Id":correlation_id, "Expect":expect} >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 79, in add >> return self.request('POST', url, body, headers, cls=cls) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", >> line 122, in request >> persistent_auth=self.__persistent_auth >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 79, in do_request >> persistent_auth) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", >> line 162, in __do_request >> raise errors.RequestError(response_code, response_reason, >> response_body) >> RequestError: >> status: 409 >> reason: Conflict >> detail: Cannot add Host. There is no available server in the cluster to >> probe the new server. >> --------------------- >> end captured logging << --------------------- > > > > _______________________________________________ > Infra mailing list > Infra(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -- Eyal edri ASSOCIATE MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

4 7

Making event types backwards compatible?
by Juan Hernández 29 May '17

29 May '17

Hello, It has been recently requested that the API provides event types: [RFE] Expose event types to API https://bugzilla.redhat.com/1453170 Currently the API provides the event code and description, for example: <event href="/ovirt-engine/api/events/8021" id="8021"> <code>19</code> <description>Host myhost failed to recover.</description ... </event> There is no documentation of what is the meaning of codes, except the source code of the engine itself. This forces some applications to add their own code to name mapping. For example, the 'ovirt' Ruby gem used by older versions of ManageIQ to interact with oVirt contains the following: https://github.com/ManageIQ/ovirt/blob/v0.17.0/lib/ovirt/event.rb#L25-L485 We could avoid this by adding to the API a new event attribute that indicates the type: <event href="/ovirt-engine/api/events/8021" id="8021"> <code>19</code> <type>host_recover_failure</type> <description>Host myhost failed to recover.</description> ... </event> Ideally this should be defined as an enum, so that it will be represented as an enum in the SDKs. Alternatively it could just be an string, and we could reuse the 'name' attribute: <event href="/ovirt-engine/api/events/8021" id="8021"> <code>19</code> <name>host_recover_failure</name> <description>Host myhost failed to recover.</description> ... </event> However, the key point to making this useful would be to keep the types (or names) backwards compatible, so that users of the API can rely on their values and meanings. So this is my question to you: can we commit to keep the names and meanings of the backend event types backwards compatible? Regards, Juan Hernandez

2 2

Fedora wiki: oVirt
by Sandro Bonazzola 29 May '17

29 May '17

Hi, looking at https://fedoraproject.org/wiki/Features/oVirt I think we should state oVirt is not shipped within Fedora anymore and our plans about either abandon the feature or re-target it. Thanks, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 1

ovirt-imageio leaves .guestfs-0 folder under /var/tmp during check-patch
by Gil Shinar 28 May '17

28 May '17

Hi, Should it be like that? Is there a way to clean this leftover in the check-patch script? Thanks Gil

6 24

[ENGINE] standardizing query creation
by Allon Mureinik 26 May '17

26 May '17

Hi list, As you may have noticed, there's a bit of an ambiguity on how query objects are created - some have a constructor from a parameters object, some have a constructor from a parameters object and an EngineContext, and some have both. This ambiguity leads to all sorts of annoying bugs where some queries can't be used as internal queries (because they lack the proper constructor), or other queries couldn't be used without a context. I just merged a series of patches to streamline queries creation and eliminate the problem. Looking forwards, queries should have a single constructor receiving a parameters object and a context and pass them on to super. A unit test was added to enforce this constructor's existence in build time. From a caller's perspective, nothing's changed, and all of the Backend#run[Internal]Query methods were left unchanged. The build, CI and OST suite were ran on this patch series and everything seems to be in order, but please let me know if you encounter any issues. -Allon

1 0

upgrade of CL and DC vs running VMs
by Michal Skrivanek 25 May '17

25 May '17

Hi all, I believe that introduction of bug 1413150 (Add warning to change CL to the match the installed engine version) may have an unfortunate consequence of people actually moving forward with the CL and DC without realizing the constraints on running existing VMs. The periodic nagging is likely going to make people run into the following issue even more frequently We have a cluster level override per VM which takes care of compatibility on CL update by setting the VM’s override to the original CL - that is visible in VM properties, but that’s pretty much it, it’s not very prominent at the moment and it can’t be searched on (bug 1454389). When the update cluster change is made there is a dialog informing you, and there’s also the pending config change for those running VMs…until you shut the VM down, from that time on it only has the CL override set. But the real problem is with DC which AFAIK does not have an override capability, and currently does not have any checks for running VMs. With the above mechanism you can easily get a VM with CL override (say. 3.6) and mindlessly updated DC to 4.1…and once you stop such VM you won’t be able to start it anymore as there is a proper check for unsupported 3.6 CL VM in a newer DC (as implemented by bug 1436577 - Solve DC/Cluster upgrade of VMs with now-unsupported custom compatibility level) We either need to warn/block on DC upgrade, or implement some kind of a DC override (I guess this is a storage question?) Thoughts/ideas? Thanks, michal

3 3

OST - possible failure today due to patches in two projects
by Marcin Mirecki 25 May '17

25 May '17

Hi all, Related patches in two different projects were merged today, which might result in the OST's failing for a while until changes from both projects are picked up. The affected test suite: basic-suite-master The affected test scenario: 098_ovirt_provider_ovn The related patches: https://gerrit.ovirt.org/#/c/77028/ https://gerrit.ovirt.org/#/c/77029/ Thanks, Marcin

2 1

ovirt-provider-ovn versioning issue
by Sandro Bonazzola 25 May '17

25 May '17

Hi, noticed you're going to release ovirt-provider-ovn-1.0-8.tar.gz <http://jenkins.ovirt.org/job/ovirt-provider-ovn_4.1_build-artifacts-fc24-x8…> as a new version. Please note that version is usually something like x.y.z and not x.y-r The release suffix is used in general by distribution packagers to address build issues specific to distributions. If tomorrow I'll have to rebuild ovirt-provider-ovn for CentOS or Fedora I'll have to rename the package to: ovirt-provide-ovn-1.0_8-1.fc27 because you're misusing the release suffix in the package naming. Please consider proper versioning for ovirt-provider-ovn. Thanks. -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 1

Jenkins was down
by Gil Shinar 24 May '17

24 May '17

Hi all, Our Jenkins had an issue that caused it to use 100% CPU. We needed to restart it and now it is working fine again. If any of you sent patches that had failed to run check-[patch/merged], build-artifa during that time, please retrigger your patches. Sorry and thanks Gil

1 0