[ovirt-devel] Re: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.3 - Build # 472 - Failure!

On 25 Jun 2020, at 09:56, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: Il giorno lun 15 giu 2020 alle ore 09:15 Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> ha scritto: On Sat, Jun 13, 2020 at 5:01 AM <jenkins(a)jenkins.phx.ovirt.org <mailto:jenkins@jenkins.phx.ovirt.org>> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/ <https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/> > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/472/ <https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/472/> This failed in: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/474/t... <https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/474/t... Fault reason is "Operation Failed". Fault detail is "[Cannot activate Host. Related operation is currently in progress. Please try again later.]". HTTP response code is 409. engine.log has: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/474/a... <https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/474/a... 2020-06-15 00:00:58,330-04 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ForkJoinPool-1-worker-8) [3715b680] Running command: RefreshHostCapabilitiesCommand internal: true. Entities affected : ID: e75bc41d-c044-40f6-a645-9a86f2ef2536 Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2020-06-15 00:00:58,331-04 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ForkJoinPool-1-worker-8) [3715b680] Before acquiring lock in order to prevent monitoring for host 'lago-he-basic-suite-4-3-host-0' from data-center 'Default' 2020-06-15 00:00:58,332-04 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ForkJoinPool-1-worker-8) [3715b680] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[e75bc41d-c044-40f6-a645-9a86f2ef2536=VDS_INIT]', sharedLocks=''}' 2020-06-15 00:00:58,337-04 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-281) [] Clearing cache of pool: 'e186881e-aeb0-11ea-8e5b-5452c0a8c863' for problematic entities of VDS: 'lago-he-basic-suite-4-3-host-0'. 2020-06-15 00:00:58,337-04 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-281) [] Removing vds '[e75bc41d-c044-40f6-a645-9a86f2ef2536]' from the domain in maintenance cache 2020-06-15 00:00:58,338-04 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-281) [] Removing host(s) '[e75bc41d-c044-40f6-a645-9a86f2ef2536]' from hosts unseen domain report cache 2020-06-15 00:00:58,382-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [] START, DisconnectStoragePoolVDSCommand(HostName = lago-he-basic-suite-4-3-host-0, DisconnectStoragePoolVDSCommandParameters:{hostId='e75bc41d-c044-40f6-a645-9a86f2ef2536', storagePoolId='e186881e-aeb0-11ea-8e5b-5452c0a8c863', vds_spm_id='1'}), log id: e9ba207 2020-06-15 00:00:58,498-04 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ForkJoinPool-1-worker-8) [3715b680] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[e75bc41d-c044-40f6-a645-9a86f2ef2536=VDS_INIT]', sharedLocks=''}' 2020-06-15 00:01:01,657-04 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (ForkJoinPool-1-worker-8) [3715b680] Failed to acquire lock and wait lock 'HostEngineLock:{exclusiveLocks='[e75bc41d-c044-40f6-a645-9a86f2ef2536=VDS_INIT]', sharedLocks=''}' 2020-06-15 00:01:03,556-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (default task-1) [e4c7e809-7883-47dc-b7da-3236cc754ebb] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[e75bc41d-c044-40f6-a645-9a86f2ef2536=VDS]', sharedLocks=''}' 2020-06-15 00:01:03,585-04 WARN [org.ovirt.engine.core.bll.ActivateVdsCommand] (default task-1) [e4c7e809-7883-47dc-b7da-3236cc754ebb] Validation of action 'ActivateVds' failed for user admin@internal-authz. Reasons: VAR__ACTION__ACTIVATE,VAR__TYPE__HOST,ACTION_TYPE_FAILED_OBJECT_LOCKED 2020-06-15 00:01:03,864-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-1) [] Operation Failed: [Cannot activate Host. Related operation is currently in progress. Please try again later.] Sandro reported about a similar case in the thread: [ovirt-devel] ovirt-system-tests_he-basic-suite-4.3 fails on storage domain unreachable About: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/427/ <https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/427/> Can't see there any response. Can someone please have a look? Thanks. I haven't seen any reaction on this. Anyone from storage team can have a look here? -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>