Failed to update OVF disks 1798e945-5be9-466e-b52d-f7f0a3bb2043, OVF data isn't updated on those OVF stores (Data Center Default, Storage Domain hosted_storage). I did not see anything with error in SPM host3 in /var/log/vdsm/vdsm.log In /var/log/vdsm/supervdsm.log i see MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:05,845::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call webhookAdd with ('http://ovirt-engine.domain.local:80/ovirt-engine/services/glusterevents', None) {} MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:05,845::commands::153::common.commands::(start) /usr/bin/taskset --cpu-list 0-39 /sbin/gluster-eventsapi webhook-add http://ovirt-engine.domain.local:80/ovirt-engine/services/glusterevents (cwd None) MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:06,599::commands::98::common.commands::(run) FAILED: <err> = b'Webhook already exists\n'; <rc> = 5 MainProcess|jsonrpc/6::ERROR::2020-07-29 19:05:06,600::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) Error in webhookAdd Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/gluster/events.py", line 42, in webhookAdd commands.run(command) File "/usr/lib/python3.6/site-packages/vdsm/common/commands.py", line 101, in run raise cmdutils.Error(args, p.returncode, out, err) vdsm.common.cmdutils.Error: Command ['/sbin/gluster-eventsapi', 'webhook-add', 'http://ovirt-engine.domain.local:80/ovirt-engine/services/glusterevents'] failed with rc=5 out=b'' err=b'Webhook already exists\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 95, in wrapper res = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/gluster/events.py", line 44, in webhookAdd raise ge.GlusterWebhookAddException(rc=e.rc, err=e.err) vdsm.gluster.exception.GlusterWebhookAddException: Failed to add webhook: rc=5 out=() err=b'Webhook already exists\n' MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:10,818::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call tasksList with ([],) {} MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:10,819::commands::153::common.commands::(start) /usr/bin/taskset --cpu-list 0-39 /usr/sbin/gluster --mode=script volume status all tasks --xml (cwd None) SPM message logs : Jul 29 18:58:22 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:58:31 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:58:42 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:58:52 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:59:03 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Hosted-Engine /var/log/ovirt-engine/engine.log 2020-07-29 18:52:40,901+03 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (default task-153) [1ec2d268-8d00-42ad-9660-0a05362a878b] The message key 'UpdateOvfStoreForStorageDomain' is missing from 'bundles/ExecutionMessages' 2020-07-29 18:52:40,933+03 INFO [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (default task-153) [1ec2d268-8d00-42ad-9660-0a05362a878b] Lock Acquired to object 'EngineLock:{exclusiveLocks='[affd38b2-457d-4c9f-9802-1f5fadd7cd34=STORAGE]', sharedLocks=''}' 2020-07-29 18:52:40,989+03 INFO [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (default task-153) [1ec2d268-8d00-42ad-9660-0a05362a878b] Running command: UpdateOvfStoreForStorageDomainCommand internal: false. Entities affected : ID: affd38b2-457d-4c9f-9802-1f5fadd7cd34 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN 2020-07-29 18:52:41,002+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Before acquiring and wait lock 'EngineLock:{exclusiveLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]', sharedLocks=''}' 2020-07-29 18:52:41,003+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Lock-wait acquired to object 'EngineLock:{exclusiveLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]', sharedLocks=''}' 2020-07-29 18:52:41,023+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Running command: ProcessOvfUpdateForStoragePoolCommand internal: true. Entities affected : ID: 15ea58fc-9435-11e9-b093-00163e11a571 Type: StoragePool 2020-07-29 18:52:41,039+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Attempting to update VM OVFs in Data Center 'Default' 2020-07-29 18:52:41,060+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Successfully updated VM OVFs in Data Center 'Default' 2020-07-29 18:52:41,060+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Attempting to update template OVFs in Data Center 'Default' 2020-07-29 18:52:41,062+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Successfully updated templates OVFs in Data Center 'Default' 2020-07-29 18:52:41,062+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Attempting to remove unneeded template/vm OVFs in Data Center 'Default' 2020-07-29 18:52:41,073+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Successfully removed unneeded template/vm OVFs in Data Center 'Default' 2020-07-29 18:52:41,091+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Lock freed to object 'EngineLock:{exclusiveLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]', sharedLocks=''}' 2020-07-29 18:52:41,418+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock Acquired to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]'}' 2020-07-29 18:52:41,440+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: affd38b2-457d-4c9f-9802-1f5fadd7cd34 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN 2020-07-29 18:52:41,465+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] START, SetVolumeDescriptionVDSCommand( SetVolumeDescriptionVDSCommandParameters:{storagePoolId='15ea58fc-9435-11e9-b093-00163e11a571', ignoreFailoverLimit='false', storageDomainId='affd38b2-457d-4c9f-9802-1f5fadd7cd34', imageGroupId='1798e945-5be9-466e-b52d-f7f0a3bb2043', imageId='8c0afaac-6c1a-40e5-ad21-54ce53ee6fdb'}), log id: 1f1cc366 2020-07-29 18:52:41,470+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] -- executeIrsBrokerCommand: calling 'setVolumeDescription', parameters: 2020-07-29 18:52:41,470+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ spUUID=15ea58fc-9435-11e9-b093-00163e11a571 2020-07-29 18:52:41,470+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ sdUUID=affd38b2-457d-4c9f-9802-1f5fadd7cd34 2020-07-29 18:52:41,471+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ imageGroupGUID=1798e945-5be9-466e-b52d-f7f0a3bb2043 2020-07-29 18:52:41,471+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ volUUID=8c0afaac-6c1a-40e5-ad21-54ce53ee6fdb 2020-07-29 18:52:41,471+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ description={"Updated":false,"Last Updated":null,"Storage Domains":[{"uuid":"affd38b2-457d-4c9f-9802-1f5fadd7cd34"}],"Disk Description":"OVF_STORE"} 2020-07-29 18:52:41,545+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] FINISH, SetVolumeDescriptionVDSCommand, return: , log id: 1f1cc366 2020-07-29 18:52:41,570+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock Acquired to object 'EngineLock:{exclusiveLocks='', sharedLocks='[33cde062-b8be-46af-b528-5963eb5d8e0d=VDS_EXECUTION]'}' 2020-07-29 18:52:41,615+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Running command: UploadStreamCommand internal: true. Entities affected : ID: affd38b2-457d-4c9f-9802-1f5fadd7cd34 Type: Storage 2020-07-29 18:52:41,618+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] START, UploadStreamVDSCommand(HostName = host3.domain.local, UploadStreamVDSCommandParameters:{hostId='33cde062-b8be-46af-b528-5963eb5d8e0d'}), log id: 10e8c127 2020-07-29 18:52:41,618+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] -- executeVdsBrokerCommand, parameters: 2020-07-29 18:52:41,618+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ spUUID=15ea58fc-9435-11e9-b093-00163e11a571 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ sdUUID=affd38b2-457d-4c9f-9802-1f5fadd7cd34 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ imageGUID=1798e945-5be9-466e-b52d-f7f0a3bb2043 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ volUUID=8c0afaac-6c1a-40e5-ad21-54ce53ee6fdb 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ size=26112 2020-07-29 18:52:41,664+03 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'UploadStreamVDSCommand(HostName = host3.domain.local, UploadStreamVDSCommandParameters:{hostId='33cde062-b8be-46af-b528-5963eb5d8e0d'})' execution failed: javax.net.ssl.SSLPeerUnverifiedException: Certificate for <host3.domain.local> doesn't match any of the subject alternative names: [host3.domain.local] 2020-07-29 18:52:41,664+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] FINISH, UploadStreamVDSCommand, return: , log id: 10e8c127 2020-07-29 18:52:41,664+03 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-29741) [b4f2874] Clearing domains data for host host3.domain.local 2020-07-29 18:52:41,665+03 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-29741) [b4f2874] Host 'host3.domain.local' is not responding. 2020-07-29 18:52:41,664+03 ERROR [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: javax.net.ssl.SSLPeerUnverifiedException: Certificate for <host3.domain.local> doesn't match any of the subject alternative names: [host3.domain.local] (Failed with error VDS_NETWORK_ERROR and code 5022) 2020-07-29 18:52:41,686+03 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-29741) [b4f2874] EVENT_ID: VDS_HOST_NOT_RESPONDING(9,027), Host host3.domain.local is not responding. Host cannot be fenced automatically because power management for the host is disabled. 2020-07-29 18:52:41,711+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock freed to object 'EngineLock:{exclusiveLocks='', sharedLocks='[33cde062-b8be-46af-b528-5963eb5d8e0d=VDS_EXECUTION]'}' 2020-07-29 18:52:41,740+03 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] EVENT_ID: UPDATE_FOR_OVF_STORES_FAILED(1,016), Failed to update OVF disks 1798e945-5be9-466e-b52d-f7f0a3bb2043, OVF data isn't updated on those OVF stores (Data Center Default, Storage Domain hosted_storage). 2020-07-29 18:52:41,777+03 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] EVENT_ID: UPDATE_OVF_FOR_STORAGE_DOMAIN_FAILED(190), Failed to update VMs/Templates OVF data for Storage Domain hosted_storage in Data Center Default. 2020-07-29 18:52:41,788+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock freed to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]'}' 2020-07-29 18:52:41,797+03 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'UpdateOvfStoreForStorageDomain' id: '9578b72d-bb33-4de6-b177-b851f7c367c8' with children [] failed when attempting to perform the next operation, marking as 'FAILED' 2020-07-29 18:52:41,797+03 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] EngineException: ENGINE (Failed with error ENGINE and code 5001): org.ovirt.engine.core.common.errors.EngineException: EngineException: ENGINE (Failed with error ENGINE and code 5001) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand.executeNextOperation(UpdateOvfStoreForStorageDomainCommand.java:124) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand.performNextOperation(UpdateOvfStoreForStorageDomainCommand.java:112) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360) at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227) 2020-07-29 18:52:41,798+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'UpdateOvfStoreForStorageDomain' id: '9578b72d-bb33-4de6-b177-b851f7c367c8' child commands '[]' executions were completed, status 'FAILED' 2020-07-29 18:52:42,725+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] START, GlusterServersListVDSCommand(HostName = host1.domain.local, VdsIdVDSCommandParametersBase:{hostId='157d2f37-6387-47a1-8bf9-bc20231726e7'}), log id: 42f3f507 2020-07-29 18:52:42,822+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [b4f2874] Command 'ProcessOvfUpdateForStorageDomain' id: 'c83199b3-626d-494d-ab93-6ffadcc4e47a' child commands '[55b09b51-04d7-4257-ba05-8fbdd2f236fb]' executions were completed, status 'FAILED' 2020-07-29 18:52:42,851+03 ERROR [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [1ec2d268-8d00-42ad-9660-0a05362a878b] Ending command 'org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand' with failure. 2020-07-29 18:52:42,860+03 INFO [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [1ec2d268-8d00-42ad-9660-0a05362a878b] Lock freed to object 'EngineLock:{exclusiveLocks='[affd38b2-457d-4c9f-9802-1f5fadd7cd34=STORAGE]', sharedLocks=''}' 2020-07-29 18:52:43,254+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] FINISH, GlusterServersListVDSCommand, return: [192.168.10.20/24:CONNECTED, gluster3.domain.local:CONNECTED, gluster2.domain.local:CONNECTED], log id: 42f3f507 2020-07-29 18:52:43,266+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] START, GlusterVolumesListVDSCommand(HostName = host1.domain.local, GlusterVolumesListVDSParameters:{hostId='157d2f37-6387-47a1-8bf9-bc20231726e7'}), log id: 69d508fc 2020-07-29 18:52:43,605+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] FINISH, GlusterVolumesListVDSCommand, return: {46d80a34-6871-43a4-8b40-e5efe75bf578=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4ff50ace, 4ef84624-8219-4850-96d5-ceeb7ed073ae=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@193d2158, 2169ec3c-b8fb-482e-9cd4-fdb751a037c5=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e0fb06d6}, log id: 69d508fc 2020-07-29 18:52:43,898+03 ERROR [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [b4f2874] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand' with failure. 2020-07-29 18:52:47,327+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-37) [] START, GetHardwareInfoAsyncVDSCommand(HostName = host3.domain.local, VdsIdAndVdsVDSCommandParametersBase:{hostId='33cde062-b8be-46af-b528-5963eb5d8e0d', vds='Host[host3.domain.local,33cde062-b8be-46af-b528-5963eb5d8e0d]'}), log id: 9a0e1c5 2020-07-29 18:52:47,328+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-37) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 9a0e1c5