- Users - oVirt List Archives

Update OVF disks fails on each gluster 7.6 volumes using ovirt 4.4.1.1
by shadow emy 29 Jul '20

29 Jul '20

Failed to update OVF disks 1798e945-5be9-466e-b52d-f7f0a3bb2043, OVF data isn't updated on those OVF stores (Data Center Default, Storage Domain hosted_storage). I did not see anything with error in SPM host3 in /var/log/vdsm/vdsm.log In /var/log/vdsm/supervdsm.log i see MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:05,845::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call webhookAdd with ('http://ovirt-engine.domain.local:80/ovirt-engine/services/glusterevents', None) {} MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:05,845::commands::153::common.commands::(start) /usr/bin/taskset --cpu-list 0-39 /sbin/gluster-eventsapi webhook-add http://ovirt-engine.domain.local:80/ovirt-engine/services/glusterevents (cwd None) MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:06,599::commands::98::common.commands::(run) FAILED: <err> = b'Webhook already exists\n'; <rc> = 5 MainProcess|jsonrpc/6::ERROR::2020-07-29 19:05:06,600::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) Error in webhookAdd Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/gluster/events.py", line 42, in webhookAdd commands.run(command) File "/usr/lib/python3.6/site-packages/vdsm/common/commands.py", line 101, in run raise cmdutils.Error(args, p.returncode, out, err) vdsm.common.cmdutils.Error: Command ['/sbin/gluster-eventsapi', 'webhook-add', 'http://ovirt-engine.domain.local:80/ovirt-engine/services/glusterevents'] failed with rc=5 out=b'' err=b'Webhook already exists\n' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 95, in wrapper res = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/gluster/events.py", line 44, in webhookAdd raise ge.GlusterWebhookAddException(rc=e.rc, err=e.err) vdsm.gluster.exception.GlusterWebhookAddException: Failed to add webhook: rc=5 out=() err=b'Webhook already exists\n' MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:10,818::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call tasksList with ([],) {} MainProcess|jsonrpc/6::DEBUG::2020-07-29 19:05:10,819::commands::153::common.commands::(start) /usr/bin/taskset --cpu-list 0-39 /usr/sbin/gluster --mode=script volume status all tasks --xml (cwd None) SPM message logs : Jul 29 18:58:22 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:58:31 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:58:42 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:58:52 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Jul 29 18:59:03 host3 journal[460535]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Hosted-Engine /var/log/ovirt-engine/engine.log 2020-07-29 18:52:40,901+03 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (default task-153) [1ec2d268-8d00-42ad-9660-0a05362a878b] The message key 'UpdateOvfStoreForStorageDomain' is missing from 'bundles/ExecutionMessages' 2020-07-29 18:52:40,933+03 INFO [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (default task-153) [1ec2d268-8d00-42ad-9660-0a05362a878b] Lock Acquired to object 'EngineLock:{exclusiveLocks='[affd38b2-457d-4c9f-9802-1f5fadd7cd34=STORAGE]', sharedLocks=''}' 2020-07-29 18:52:40,989+03 INFO [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (default task-153) [1ec2d268-8d00-42ad-9660-0a05362a878b] Running command: UpdateOvfStoreForStorageDomainCommand internal: false. Entities affected : ID: affd38b2-457d-4c9f-9802-1f5fadd7cd34 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN 2020-07-29 18:52:41,002+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Before acquiring and wait lock 'EngineLock:{exclusiveLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]', sharedLocks=''}' 2020-07-29 18:52:41,003+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Lock-wait acquired to object 'EngineLock:{exclusiveLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]', sharedLocks=''}' 2020-07-29 18:52:41,023+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Running command: ProcessOvfUpdateForStoragePoolCommand internal: true. Entities affected : ID: 15ea58fc-9435-11e9-b093-00163e11a571 Type: StoragePool 2020-07-29 18:52:41,039+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Attempting to update VM OVFs in Data Center 'Default' 2020-07-29 18:52:41,060+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Successfully updated VM OVFs in Data Center 'Default' 2020-07-29 18:52:41,060+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Attempting to update template OVFs in Data Center 'Default' 2020-07-29 18:52:41,062+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Successfully updated templates OVFs in Data Center 'Default' 2020-07-29 18:52:41,062+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Attempting to remove unneeded template/vm OVFs in Data Center 'Default' 2020-07-29 18:52:41,073+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Successfully removed unneeded template/vm OVFs in Data Center 'Default' 2020-07-29 18:52:41,091+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStoragePoolCommand] (default task-153) [71d7de10] Lock freed to object 'EngineLock:{exclusiveLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]', sharedLocks=''}' 2020-07-29 18:52:41,418+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock Acquired to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]'}' 2020-07-29 18:52:41,440+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: affd38b2-457d-4c9f-9802-1f5fadd7cd34 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN 2020-07-29 18:52:41,465+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] START, SetVolumeDescriptionVDSCommand( SetVolumeDescriptionVDSCommandParameters:{storagePoolId='15ea58fc-9435-11e9-b093-00163e11a571', ignoreFailoverLimit='false', storageDomainId='affd38b2-457d-4c9f-9802-1f5fadd7cd34', imageGroupId='1798e945-5be9-466e-b52d-f7f0a3bb2043', imageId='8c0afaac-6c1a-40e5-ad21-54ce53ee6fdb'}), log id: 1f1cc366 2020-07-29 18:52:41,470+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] -- executeIrsBrokerCommand: calling 'setVolumeDescription', parameters: 2020-07-29 18:52:41,470+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ spUUID=15ea58fc-9435-11e9-b093-00163e11a571 2020-07-29 18:52:41,470+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ sdUUID=affd38b2-457d-4c9f-9802-1f5fadd7cd34 2020-07-29 18:52:41,471+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ imageGroupGUID=1798e945-5be9-466e-b52d-f7f0a3bb2043 2020-07-29 18:52:41,471+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ volUUID=8c0afaac-6c1a-40e5-ad21-54ce53ee6fdb 2020-07-29 18:52:41,471+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ description={"Updated":false,"Last Updated":null,"Storage Domains":[{"uuid":"affd38b2-457d-4c9f-9802-1f5fadd7cd34"}],"Disk Description":"OVF_STORE"} 2020-07-29 18:52:41,545+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] FINISH, SetVolumeDescriptionVDSCommand, return: , log id: 1f1cc366 2020-07-29 18:52:41,570+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock Acquired to object 'EngineLock:{exclusiveLocks='', sharedLocks='[33cde062-b8be-46af-b528-5963eb5d8e0d=VDS_EXECUTION]'}' 2020-07-29 18:52:41,615+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Running command: UploadStreamCommand internal: true. Entities affected : ID: affd38b2-457d-4c9f-9802-1f5fadd7cd34 Type: Storage 2020-07-29 18:52:41,618+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] START, UploadStreamVDSCommand(HostName = host3.domain.local, UploadStreamVDSCommandParameters:{hostId='33cde062-b8be-46af-b528-5963eb5d8e0d'}), log id: 10e8c127 2020-07-29 18:52:41,618+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] -- executeVdsBrokerCommand, parameters: 2020-07-29 18:52:41,618+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ spUUID=15ea58fc-9435-11e9-b093-00163e11a571 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ sdUUID=affd38b2-457d-4c9f-9802-1f5fadd7cd34 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ imageGUID=1798e945-5be9-466e-b52d-f7f0a3bb2043 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ volUUID=8c0afaac-6c1a-40e5-ad21-54ce53ee6fdb 2020-07-29 18:52:41,619+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] ++ size=26112 2020-07-29 18:52:41,664+03 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'UploadStreamVDSCommand(HostName = host3.domain.local, UploadStreamVDSCommandParameters:{hostId='33cde062-b8be-46af-b528-5963eb5d8e0d'})' execution failed: javax.net.ssl.SSLPeerUnverifiedException: Certificate for <host3.domain.local> doesn't match any of the subject alternative names: [host3.domain.local] 2020-07-29 18:52:41,664+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] FINISH, UploadStreamVDSCommand, return: , log id: 10e8c127 2020-07-29 18:52:41,664+03 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-29741) [b4f2874] Clearing domains data for host host3.domain.local 2020-07-29 18:52:41,665+03 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-29741) [b4f2874] Host 'host3.domain.local' is not responding. 2020-07-29 18:52:41,664+03 ERROR [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: javax.net.ssl.SSLPeerUnverifiedException: Certificate for <host3.domain.local> doesn't match any of the subject alternative names: [host3.domain.local] (Failed with error VDS_NETWORK_ERROR and code 5022) 2020-07-29 18:52:41,686+03 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-29741) [b4f2874] EVENT_ID: VDS_HOST_NOT_RESPONDING(9,027), Host host3.domain.local is not responding. Host cannot be fenced automatically because power management for the host is disabled. 2020-07-29 18:52:41,711+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock freed to object 'EngineLock:{exclusiveLocks='', sharedLocks='[33cde062-b8be-46af-b528-5963eb5d8e0d=VDS_EXECUTION]'}' 2020-07-29 18:52:41,740+03 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] EVENT_ID: UPDATE_FOR_OVF_STORES_FAILED(1,016), Failed to update OVF disks 1798e945-5be9-466e-b52d-f7f0a3bb2043, OVF data isn't updated on those OVF stores (Data Center Default, Storage Domain hosted_storage). 2020-07-29 18:52:41,777+03 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] EVENT_ID: UPDATE_OVF_FOR_STORAGE_DOMAIN_FAILED(190), Failed to update VMs/Templates OVF data for Storage Domain hosted_storage in Data Center Default. 2020-07-29 18:52:41,788+03 INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Lock freed to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[15ea58fc-9435-11e9-b093-00163e11a571=OVF_UPDATE]'}' 2020-07-29 18:52:41,797+03 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'UpdateOvfStoreForStorageDomain' id: '9578b72d-bb33-4de6-b177-b851f7c367c8' with children [] failed when attempting to perform the next operation, marking as 'FAILED' 2020-07-29 18:52:41,797+03 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] EngineException: ENGINE (Failed with error ENGINE and code 5001): org.ovirt.engine.core.common.errors.EngineException: EngineException: ENGINE (Failed with error ENGINE and code 5001) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand.executeNextOperation(UpdateOvfStoreForStorageDomainCommand.java:124) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand.performNextOperation(UpdateOvfStoreForStorageDomainCommand.java:112) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360) at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227) 2020-07-29 18:52:41,798+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [b4f2874] Command 'UpdateOvfStoreForStorageDomain' id: '9578b72d-bb33-4de6-b177-b851f7c367c8' child commands '[]' executions were completed, status 'FAILED' 2020-07-29 18:52:42,725+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] START, GlusterServersListVDSCommand(HostName = host1.domain.local, VdsIdVDSCommandParametersBase:{hostId='157d2f37-6387-47a1-8bf9-bc20231726e7'}), log id: 42f3f507 2020-07-29 18:52:42,822+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [b4f2874] Command 'ProcessOvfUpdateForStorageDomain' id: 'c83199b3-626d-494d-ab93-6ffadcc4e47a' child commands '[55b09b51-04d7-4257-ba05-8fbdd2f236fb]' executions were completed, status 'FAILED' 2020-07-29 18:52:42,851+03 ERROR [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [1ec2d268-8d00-42ad-9660-0a05362a878b] Ending command 'org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand' with failure. 2020-07-29 18:52:42,860+03 INFO [org.ovirt.engine.core.bll.storage.domain.UpdateOvfStoreForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [1ec2d268-8d00-42ad-9660-0a05362a878b] Lock freed to object 'EngineLock:{exclusiveLocks='[affd38b2-457d-4c9f-9802-1f5fadd7cd34=STORAGE]', sharedLocks=''}' 2020-07-29 18:52:43,254+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] FINISH, GlusterServersListVDSCommand, return: [192.168.10.20/24:CONNECTED, gluster3.domain.local:CONNECTED, gluster2.domain.local:CONNECTED], log id: 42f3f507 2020-07-29 18:52:43,266+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] START, GlusterVolumesListVDSCommand(HostName = host1.domain.local, GlusterVolumesListVDSParameters:{hostId='157d2f37-6387-47a1-8bf9-bc20231726e7'}), log id: 69d508fc 2020-07-29 18:52:43,605+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] FINISH, GlusterVolumesListVDSCommand, return: {46d80a34-6871-43a4-8b40-e5efe75bf578=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4ff50ace, 4ef84624-8219-4850-96d5-ceeb7ed073ae=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@193d2158, 2169ec3c-b8fb-482e-9cd4-fdb751a037c5=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e0fb06d6}, log id: 69d508fc 2020-07-29 18:52:43,898+03 ERROR [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [b4f2874] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand' with failure. 2020-07-29 18:52:47,327+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-37) [] START, GetHardwareInfoAsyncVDSCommand(HostName = host3.domain.local, VdsIdAndVdsVDSCommandParametersBase:{hostId='33cde062-b8be-46af-b528-5963eb5d8e0d', vds='Host[host3.domain.local,33cde062-b8be-46af-b528-5963eb5d8e0d]'}), log id: 9a0e1c5 2020-07-29 18:52:47,328+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-37) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 9a0e1c5

1 0

ovirt-engine DB datastore full
by Askhat ECC 29 Jul '20

29 Jul '20

Hello! I have ovirt cluster Version 4.2.4.5-1.el7 and hosted-engine with local database. HE-virtio-disk is full directory /var and can not be moved another domain storage. Can you say how migrate or another things to solve this problem? Thanks. Regards [image: Screenshot_1.jpg] [image: Screenshot_13.jpg] [image: image_2020_07_13T06_51_27_733Z.png]

2 1

VM locked and script unlock_entity.sh don't show any problem
by Thomas BURGUIERE 29 Jul '20

29 Jul '20

The WEB interface show that the VM is locked, but when we run unlock_entity.sh there is no problem reported. How we think the VM were locked : There were filesystem issues (shortage disk space on the ovirt engine HOST) during a backup sequence, those consequences were discovered a few days later. The VMs are backed-up with this way: https://github.com/wefixit-AT/oVirtBackup The ovirt-engine logs tell us that Postgres DB has encountered problem like this one during the backup process: Message: PL/pgSQL function updatevdsdynamic(integer,integer,character varying,numeric,character varying,boolean,integer,integer,integer,uuid,integer,integer,integer,integer,integer,integer,integer,integer,character varying,character varying,character varying,character varying,integer,character varying,integer,integer,integer,boolean,character varying,character varying,character varying,character varying,character varying,character varying,character varying,character varying,character varying,character varying,character varying,integer,text,integer,character varying,character varying,character varying,character varying,character varying,character varying,text,text,smallint,integer,smallint,boolean,character varying,text,text,boolean,boolean,text,character varying,boolean,uuid,boolean,jsonb) line 4 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: could not extend file "base/16385/190997": No space left on device Later, we saw that those VMs they got a lock icon in the status column since this episode. This did not appear immediatly, but maybe a few hours after. It happens that those locked VMs are sort-of limited in the way that there are numerous things we cannot do with them, such as: - edit parameters (i.e. memory). - start them again after they get halted by unix command (halt -p) - copy the related disk. The version of ovirt used 4.2.1 Is there an other way to unlock the VMs ? Or to get unlock_entity.sh to work ?

1 1

New ovirt 4.4.0.3-1.el8 leaves disks in illegal state on all snapshot actions
by Henri Aanstoot 28 Jul '20

28 Jul '20

Hi all, I've got 2 two node setup, image based installs. When doing ova exports or generic snapshots, things seem in order. Removing snapshots shows warning 'disk in illegal state' Mouse hover shows .. please do not shutdown before succesfully remove snapshot ovirt-engine log 2020-07-22 16:40:37,549+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedExecutorService-commandCoordinator-Thread-2) [264b0047-5aa6-4380-9d32-eb328fd6bed0] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM node2.lab command MergeVDS failed: Merge failed 2020-07-22 16:40:37,549+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-2) [264b0047-5aa6-4380-9d32-eb328fd6bed0] Command 'MergeVDSCommand(HostName = node2.lab, MergeVDSCommandParameters:{hostId='02df5213-1243-4671-a1c6-6489d7146319', vmId='64c25543-bef7-4fdd-8204-6507046f5a34', storagePoolId='5a4ea80c-b3b2-11ea-a890-00163e3cb866', storageDomainId='9a12f1b2-5378-46cc-964d-3575695e823f', imageGroupId='3f7ac8d8-f1ab-4c7a-91cc-f34d0b8a1cb8', imageId='c757e740-9013-4ae0-901d-316932f4af0e', baseImageId='ebe50730-dec3-4f29-8a38-9ae7c59f2aef', topImageId='c757e740-9013-4ae0-901d-316932f4af0e', bandwidth='0'})' execution failed: VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52 2020-07-22 16:40:37,549+02 ERROR [org.ovirt.engine.core.bll.MergeCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-2) [264b0047-5aa6-4380-9d32-eb328fd6bed0] Engine exception thrown while sending merge command: org.ovirt.engine.core.common.errors.EngineException: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52 (Failed with error mergeErr and code 52) Caused by: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Merge failed, code = 52 <driver name='qemu' error_policy='report'/> <driver name='qemu' type='qcow2' cache='none' error_policy='stop' io='threads'/> 2020-07-22 16:40:39,659+02 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedExecutorService-commandCoordinator-Thread-3) [264b0047-5aa6-4380-9d32-eb328fd6bed0] Failed to live merge. Top volume c757e740-9013-4ae0-901d-316932f4af0e is still in qemu chain [ebe50730-dec3-4f29-8a38-9ae7c59f2aef, c757e740-9013-4ae0-901d-316932f4af0e] 2020-07-22 16:40:41,524+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-58) [264b0047-5aa6-4380-9d32-eb328fd6bed0] Command id: 'e0b2bce7-afe0-4955-ae46-38bcb8719852 failed child command status for step 'MERGE_STATUS' 2020-07-22 16:40:42,597+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-53) [264b0047-5aa6-4380-9d32-eb328fd6bed0] Merging of snapshot 'ef8f7e06-e48c-4a8c-983c-64e3d4ebfcf9' images 'ebe50730-dec3-4f29-8a38-9ae7c59f2aef'..'c757e740-9013-4ae0-901d-316932f4af0e' failed. Images have been marked illegal and can no longer be previewed or reverted to. Please retry Live Merge on the snapshot to complete the operation. 2020-07-22 16:40:42,603+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-53) [264b0047-5aa6-4380-9d32-eb328fd6bed0] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand' with failure. 2020-07-22 16:40:43,679+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-15) [264b0047-5aa6-4380-9d32-eb328fd6bed0] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. 2020-07-22 16:40:43,774+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-15) [264b0047-5aa6-4380-9d32-eb328fd6bed0] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'Auto-generated for Export To OVA' for VM 'Adhoc'. VDSM on hypervisor 2020-07-22 14:14:30,220+0200 ERROR (jsonrpc/5) [virt.vm] (vmId='14283e6d-c3f0-4011-b90f-a1272f0fbc10') Live merge failed (job: e59c54d9-b8d3-44d0-9147-9dd40dff57b9) (vm:5381) if ret == -1: raise libvirtError ('virDomainBlockCommit() failed', dom=self) libvirt.libvirtError: internal error: qemu block name 'json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "/rhev/data-center/mnt/10.12.0.9:_exports_data/9a12f1b2-5378-46cc-964d-3575695e823f/images/3206de41-ccdc-4f2d-a968-5e4da6c2ca3e/bb3aed4b-fc41-456a-9c18-1409a9aa6d14"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "/rhev/data-center/mnt/10.12.0.9:_exports_data/9a12f1b2-5378-46cc-964d-3575695e823f/images/3206de41-ccdc-4f2d-a968-5e4da6c2ca3e/3995b256-2afb-4853-9360-33d0c12e5fd1"}}' doesn't match expected '/rhev/data-center/mnt/10.12.0.9: _exports_data/9a12f1b2-5378-46cc-964d-3575695e823f/images/3206de41-ccdc-4f2d-a968-5e4da6c2ca3e/3995b256-2afb-4853-9360-33d0c12e5fd1' 2020-07-22 14:14:30,234+0200 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call VM.merge failed (error 52) in 0.17 seconds (__init__:312) 2020-07-22 14:17:28,798+0200 INFO (jsonrpc/2) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': '698d486c-edbf-4e28-a199-31a2e27bd808'} (api:129) 2020-07-22 14:17:28,798+0200 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312) Also in log, INFO (jsonrpc/1) [api.virt] FINISH getStats return={'status': {'code': 1, 'message': "Virtual machine does not exist: But is is there and accessible Any advice here? Henri ovirt 4.4.0.3-1.el8 OS Version: RHEL - 8 - 1.1911.0.9.el8 OS Description: oVirt Node 4.4.0 Kernel Version: 4.18.0 - 147.8.1.el8_1.x86_64 KVM Version: 4.1.0 - 23.el8.1 LIBVIRT Version: libvirt-5.6.0-10.el8 VDSM Version: vdsm-4.40.16-1.el8 SPICE Version: 0.14.2 - 1.el8 GlusterFS Version: glusterfs-7.5-1.el8 CEPH Version: librbd1-12.2.7-9.el8 Open vSwitch Version: openvswitch-2.11.1-5.el8 Nmstate Version: nmstate-0.2.10-1.el8

3 4

Re: Issue with ovirt 4.4 after doing some incremental backups.
by Eyal Shenitzky 28 Jul '20

28 Jul '20

Hi Łukasz, Can you please provide vdsm.log and libvirt.log? On Tue, 28 Jul 2020 at 16:05, Łukasz Kołaciński <l.kolacinski(a)storware.eu> wrote: > Hello, > > After doing a few vm backups, something breaks and I am unable to perform > any operations. I cannot do incremental backups and even full backups > doesn't work. I have this issue third time. I don't know how to fix this so > I am currently making new vms for testing purposes > > VDSM ovirt44-h2.storware.local command StartVmBackupVDS failed: Backup > Error: {'vm_id': '116aa6eb-31a1-43db-9b1e-ad6e32fb9260', 'backup': > <vdsm.virt.backup.BackupConfig object at 0x7f42602bba20>, 'reason': "Error > starting backup: internal error: unable to execute QEMU command > 'transaction': Dirty bitmap 'ef0dfe55-c08c-4d9e-ad32-d6b6d5cbdac6' not > found"} > > > Best Regards, > > Łukasz Kołaciński > > Junior Java Developer > > e-mail: l.kolacinski(a)storware.eu > <m.helbert(a)storware.eu> > > > > > *[image: STORWARE]* <http://www.storware.eu/> > > > > *ul. Leszno 8/44 01-192 Warszawa www.storware.eu > <https://www.storware.eu/>* > > *[image: facebook]* <https://www.facebook.com/storware> > > *[image: twitter]* <https://twitter.com/storware> > > *[image: linkedin]* <https://www.linkedin.com/company/storware> > > *[image: Storware_Stopka_09]* > <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> > > > > *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa > 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie > dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne > i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, > przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub > podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub > podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez > pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie > tej wiadomości z wszelkich komputerów. **This message is intended only > for the person or entity to which it is addressed and may contain > confidential and/or privileged material. Any review, retransmission, > dissemination or other use of, or taking of any action in reliance upon, > this information by persons or entities other than the intended recipient > is prohibited. If you have received this message in error, please contact > the sender and remove the material from all of your computer systems.* > > -- Regards, Eyal Shenitzky

1 0

Issue with ovirt 4.4 after doing some incremental backups.
by Łukasz Kołaciński 28 Jul '20

28 Jul '20

Hello, After doing a few vm backups, something breaks and I am unable to perform any operations. I cannot do incremental backups and even full backups doesn't work. I have this issue third time. I don't know how to fix this so I am currently making new vms for testing purposes VDSM ovirt44-h2.storware.local command StartVmBackupVDS failed: Backup Error: {'vm_id': '116aa6eb-31a1-43db-9b1e-ad6e32fb9260', 'backup': <vdsm.virt.backup.BackupConfig object at 0x7f42602bba20>, 'reason': "Error starting backup: internal error: unable to execute QEMU command 'transaction': Dirty bitmap 'ef0dfe55-c08c-4d9e-ad32-d6b6d5cbdac6' not found"} Best Regards, Łukasz Kołaciński Junior Java Developer e-mail: l.kolacinski(a)storware.eu<mailto:l.kolacinski@storware.eu> <mailto:m.helbert@storware.eu> [STORWARE]<http://www.storware.eu/> ul. Leszno 8/44 01-192 Warszawa www.storware.eu <https://www.storware.eu/> [facebook]<https://www.facebook.com/storware> [twitter]<https://twitter.com/storware> [linkedin]<https://www.linkedin.com/company/storware> [Storware_Stopka_09]<https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa 000510131 , NIP 5213672602. Wiadomość ta jest przeznaczona jedynie dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie tej wiadomości z wszelkich komputerów. This message is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you have received this message in error, please contact the sender and remove the material from all of your computer systems.

1 0

Re: affinity/pool/fencing ?
by Nardus Geldenhuys 28 Jul '20

28 Jul '20

Hi Paul Thanks for the response, easy way to add 100 VM's in your second step ? Regards Nar On Tue, 28 Jul 2020 at 13:17, Staniforth, Paul < P.Staniforth(a)leedsbeckett.ac.uk> wrote: > Hello Nar, > you can achieve this with 2 affinity rules, a positive > and negative one. > > In the cluster create the first affinity group VM affinity rule disabled > HOST affinity rule positive set enforcing mode. Then add the VMs and 2 > Hosts to force them to run on these hosts. > > In the cluster create the second affinity group VM affinity rule disabled > HOST affinity rule negative set enforcing mode. Then add the rest of the > VMs and the 2 Hosts to force them not to run on these hosts. > > > https://www.ovirt.org/documentation/vmm-guide/chap-Administrative_Tasks.html > > > Regards, > > Paul S. > > > ------------------------------ > *From:* Nardus Geldenhuys <nardusg(a)gmail.com> > *Sent:* 28 July 2020 11:22 > *To:* users(a)ovirt.org <users(a)ovirt.org> > *Subject:* [ovirt-users] affinity/pool/fencing ? > > > *Caution External Mail:* Do not click any links or open any attachments > unless you trust the sender and know that the content is safe. > Hi oVirt land > > Hope you are well. Don't even know what to call this. But let me describe > what I want to achieve. > > We have a cluster with say 100 vm's. But we want two use two hosts in the > cluster two run only certain VM's. I think you can do that with affinity > rules. But how can I restrict those two hosts to only run the VM's, meaning > that no other VM's will run on them. > > I don't want to go and edit the 100 other VM's to not run on 2 hosts. Is > there an easy way for doing this? > > Oracle VM uses pools, I dont know what vmware uses. > > Any advice will help. > > Regards > > Nar > To view the terms under which this email is distributed, please go to:- > http://leedsbeckett.ac.uk/disclaimer/email/ > >

3 2

affinity/pool/fencing ?
by Nardus Geldenhuys 28 Jul '20

28 Jul '20

Hi oVirt land Hope you are well. Don't even know what to call this. But let me describe what I want to achieve. We have a cluster with say 100 vm's. But we want two use two hosts in the cluster two run only certain VM's. I think you can do that with affinity rules. But how can I restrict those two hosts to only run the VM's, meaning that no other VM's will run on them. I don't want to go and edit the 100 other VM's to not run on 2 hosts. Is there an easy way for doing this? Oracle VM uses pools, I dont know what vmware uses. Any advice will help. Regards Nar

2 1

Management Engine IP change
by Anton Louw 28 Jul '20

28 Jul '20

Hi All, Does somebody perhaps know the process of changing the Hosted Engine IP address? I see that it is possible, I am just not sure if it is a straight forward process using 'nmtui' or editing the network config file. I have also ensured that everything was configured using the FQDN. Thanks Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za

1 0

CentOS 8.2, ovVirt 4.4.1.8, SHE -- kernel: device-mapper: core: ovirt-ha-broker: sending ioctl 5401 to DM device without required privilege
by Dmitry Kharlamov 28 Jul '20

28 Jul '20

Good day! CentOS 8.2.2004 (Core), ovVirt 4.4.1.8-1.el8, storage type FC with multipathing. Please tell me what could be the problem. Messages are constantly poured into the console and / var / log / messages: kernel: device-mapper: core: ovirt-ha-broker: sending ioctl 5401 to DM device without required privilege Messages appeared during the installation process (SHE via CLI), during the LUN partitioning phase, and after that they do not stop. At the same time, the installation was successful and now everything is working fine. A similar problem is described here, but no solution has been proposed: https://bugzilla.redhat.com/show_bug.cgi?id=1838453 Best regards, Dmitry

3 7

Re: Failed to execute stage 'Environment customization': IPAddress() does not support netmasks or subnet prefixes! See documentation for details.
by James Loker-Steele 28 Jul '20

28 Jul '20

Hi Anton, I tried without the /24 it threw an error saying it wasn’t a correct cidr address 10.93.130.50/64 (it added the /64) Interesting though, I have manually created the ovirtmgmt bridge interface and assigned an IP address to it (as per another post) and it has gone past this step now without the /24. The part I am crashing out on is being unable to resolve the address. I can post the error shortly. This is strange behaviour as the host can resolve, but I will be adding dns addresses to the ovirtmgmt interface and trying again(post didn’t show to add them) When I installed <4.2 I didn’t get these issues it went smoothly. The problems seem related to ansible. Best regards James Sent from my iPhone > On 28 Jul 2020, at 07:25, Anton Louw <Anton.Louw(a)voxtelecom.co.za> wrote: > > > > Hi James, > > You can remove the “/24” prefix after the IP address for the management VM. You will get prompts to add the netmask and gateway after you add the IP address. > > > Anton Louw > Cloud Engineer: Storage and Virtualization at Vox > T: 087 805 0000 | D: 087 805 1572 > M: N/A > E: anton.louw(a)voxtelecom.co.za > A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg > www.vox.co.za > > > > > > > > > > > > > > Disclaimer > > The contents of this email are confidential to the sender and the intended recipient. Unless the contents are clearly and entirely of a personal nature, they are subject to copyright in favour of the holding company of the Vox group of companies. Any recipient who receives this email in error should immediately report the error to the sender and permanently delete this email from all storage devices. > > This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here. > > > > From: James Loker-Steele via Users <users(a)ovirt.org> > Sent: 27 July 2020 11:36 > To: users(a)ovirt.org > Subject: [ovirt-users] Failed to execute stage 'Environment customization': IPAddress() does not support netmasks or subnet prefixes! See documentation for details. > > Hi All, > Trying to install a self-hosted engine but cannot get past this step. > > [root@<hostname hidden> /]# hosted-engine --deploy > [ INFO ] Stage: Initializing > [ INFO ] Stage: Environment setup > During customization use CTRL-D to abort. > Continuing will configure this host for serving as hypervisor and will create a local VM with a running engine. > The locally running engine will be used to configure a new storage domain and create a VM there. > At the end the disk of the local VM will be moved to the shared storage. > Are you sure you want to continue? (Yes, No)[Yes]: > Configuration files: [] > Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200724164325-fyyler.log > Version: otopi-1.8.3 (otopi-1.8.3-1.el7) > [ INFO ] Stage: Environment packages setup > [ INFO ] Stage: Programs detection > [ INFO ] Stage: Environment setup (late) > [ INFO ] Stage: Environment customization > > --== STORAGE CONFIGURATION ==-- > > > --== HOST NETWORK CONFIGURATION ==-- > > Please indicate the gateway IP address [10.x.x.x]: > [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Detecting interface on existing management bridge] > [ INFO ] TASK [ovirt.hosted_engine_setup : Get all active network interfaces] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Filter bonds with bad naming] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Generate output list] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Validate selected bridge interface if management bridge does not exists] > [ INFO ] skipping: [localhost] > Please indicate a nic to set ovirtmgmt bridge on: (bond0, bond0.130, bond0.60, em1) [em1]: bond0.130 > Please specify which way the network connectivityshould be checked (ping, dns, tcp, none) [dns]: > > --== VM CONFIGURATION ==-- > > Please enter the name of the datacenter where you want to deploy this hosted-engine host. [Default]: > Please enter the name of the cluster where you want to deploy this hosted-engine host. [Default]: > If you want to deploy with a custom engine appliance image, > please specify the path to the OVA archive you would like to use > (leave it empty to skip, the setup will use ovirt-engine-appliance rpm installing it if missing): /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-4.3-20190926.1.0.5.el7.ova > [ INFO ] Checking OVF archive content (could take a few minutes depending on archive size) > [ INFO ] Checking OVF XML content (could take a few minutes depending on archive size) > Please specify the number of virtual CPUs for the VM (Defaults to appliance OVF value): [4]: > Please specify the memory size of the VM in MB (Defaults to appliance OVF value): [4]: 16384 > [ INFO ] Detecting host timezone. > Please provide the FQDN you would like to use for the engine. > Note: This will be the FQDN of the engine VM you are now going to launch, > it should not point to the base host or to any other existing machine. > Engine VM FQDN: []: <fqdn hidden> > Please provide the domain name you would like to use for the engine appliance. > Engine VM domain: [<tld hidden>] > Enter root password that will be used for the engine appliance: > Confirm appliance root password: > Enter ssh public key for the root user that will be used for the engine appliance (leave it empty to skip): > [WARNING] Skipping appliance root ssh public key > Do you want to enable ssh access for the root user (yes, no, without-password) [yes]: > Do you want to apply a default OpenSCAP security profile (Yes, No) [No]: > You may specify a unicast MAC address for the VM or accept a randomly generated default [00:16:3e:3a:4b:fc]: > > How should the engine VM network be configured (DHCP, Static)[DHCP]? static > Please enter the IP address to be used for the engine VM []: 10.93.130.50/24 > [ ERROR ] Failed to execute stage 'Environment customization': IPAddress() does not support netmasks or subnet prefixes! See documentation for details. > [ INFO ] Stage: Clean up > [ INFO ] Cleaning temporary resources > [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch logs from the engine VM] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Set destination directory path] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Create destination directory] > [ INFO ] changed: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Find the local appliance image] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Set local_vm_disk_path] > [ INFO ] skipping: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Give the vm time to flush dirty buffers] > [ INFO ] ok: [localhost -> localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Copy engine logs] > [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Remove local vm dir] > [ INFO ] ok: [localhost] > [ INFO ] TASK [ovirt.hosted_engine_setup : Remove temporary entry in /etc/hosts for the local VM] > [ INFO ] ok: [localhost] > [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20200724165204.conf' > [ INFO ] Stage: Pre-termination > [ INFO ] Stage: Termination > [ ERROR ] Hosted Engine deployment failed > Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200724164325-fyyler.log > > > Nothing in log is of much help. > > Also tried to install via Cockpit but gets to prepareVM and displays settings, click next and deployment failed. > No log. > I have tried specifying the OVA and not specifying. > > Anyone can help? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/E676EUG7Y3TQF…

1 1

Failed to execute stage 'Environment customization': IPAddress() does not support netmasks or subnet prefixes! See documentation for details.
by James Loker-Steele 28 Jul '20

28 Jul '20

Hi All, Trying to install a self-hosted engine but cannot get past this step. [root@<hostname hidden> /]# hosted-engine --deploy [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. Continuing will configure this host for serving as hypervisor and will create a local VM with a running engine. The locally running engine will be used to configure a new storage domain and create a VM there. At the end the disk of the local VM will be moved to the shared storage. Are you sure you want to continue? (Yes, No)[Yes]: Configuration files: [] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200724164325-fyyler.log Version: otopi-1.8.3 (otopi-1.8.3-1.el7) [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup (late) [ INFO ] Stage: Environment customization --== STORAGE CONFIGURATION ==-- --== HOST NETWORK CONFIGURATION ==-- Please indicate the gateway IP address [10.x.x.x]: [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Detecting interface on existing management bridge] [ INFO ] TASK [ovirt.hosted_engine_setup : Get all active network interfaces] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Filter bonds with bad naming] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Generate output list] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Validate selected bridge interface if management bridge does not exists] [ INFO ] skipping: [localhost] Please indicate a nic to set ovirtmgmt bridge on: (bond0, bond0.130, bond0.60, em1) [em1]: bond0.130 Please specify which way the network connectivityshould be checked (ping, dns, tcp, none) [dns]: --== VM CONFIGURATION ==-- Please enter the name of the datacenter where you want to deploy this hosted-engine host. [Default]: Please enter the name of the cluster where you want to deploy this hosted-engine host. [Default]: If you want to deploy with a custom engine appliance image, please specify the path to the OVA archive you would like to use (leave it empty to skip, the setup will use ovirt-engine-appliance rpm installing it if missing): /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-4.3-20190926.1.0.5.el7.ova [ INFO ] Checking OVF archive content (could take a few minutes depending on archive size) [ INFO ] Checking OVF XML content (could take a few minutes depending on archive size) Please specify the number of virtual CPUs for the VM (Defaults to appliance OVF value): [4]: Please specify the memory size of the VM in MB (Defaults to appliance OVF value): [4]: 16384 [ INFO ] Detecting host timezone. Please provide the FQDN you would like to use for the engine. Note: This will be the FQDN of the engine VM you are now going to launch, it should not point to the base host or to any other existing machine. Engine VM FQDN: []: <fqdn hidden> Please provide the domain name you would like to use for the engine appliance. Engine VM domain: [<tld hidden>] Enter root password that will be used for the engine appliance: Confirm appliance root password: Enter ssh public key for the root user that will be used for the engine appliance (leave it empty to skip): [WARNING] Skipping appliance root ssh public key Do you want to enable ssh access for the root user (yes, no, without-password) [yes]: Do you want to apply a default OpenSCAP security profile (Yes, No) [No]: You may specify a unicast MAC address for the VM or accept a randomly generated default [00:16:3e:3a:4b:fc]: How should the engine VM network be configured (DHCP, Static)[DHCP]? static Please enter the IP address to be used for the engine VM []: 10.93.130.50/24 [ ERROR ] Failed to execute stage 'Environment customization': IPAddress() does not support netmasks or subnet prefixes! See documentation for details. [ INFO ] Stage: Clean up [ INFO ] Cleaning temporary resources [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch logs from the engine VM] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set destination directory path] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Create destination directory] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Find the local appliance image] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set local_vm_disk_path] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Give the vm time to flush dirty buffers] [ INFO ] ok: [localhost -> localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Copy engine logs] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove local vm dir] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove temporary entry in /etc/hosts for the local VM] [ INFO ] ok: [localhost] [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20200724165204.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200724164325-fyyler.log Nothing in log is of much help. Also tried to install via Cockpit but gets to prepareVM and displays settings, click next and deployment failed. No log. I have tried specifying the OVA and not specifying. Anyone can help?

2 1

Adding hypervisor fails, yum links deprecated (Centos 7.7/ oVirt 4.3)
by wjb2130＠cumc.columbia.edu 28 Jul '20

28 Jul '20

Greetings. In a multi-hypervisor environment, we are running Centos 7.7.1908, with ovirt-4.3. There are problems adding a new hypervisor, because all the yum resources have switched in favor of newer versions. Is there a proceedure to add a hypervisor under the oVirt Virtualization manager while retaining an older, known working OS version like Centos 7.7.1908 ? Upgrading everything across the board isn't an immediate option. There are many VMs and hypervisors in production, and can't be disturbed. In sum: is it still possible to install known working 4.30.33-1.el7? All that is presented now is 4.30.46-1.el7 I ran: yum install vdsm-4.30.33-1.el7.x86_64. This completed. However, when I tried to add the resulting hypservisor using the oVirt web UI, that fails. Logs on that management server showed that the install process insists on using 4.30.46-1.el7. Any way to avoid this? 2020-07-22 20:30:58,621-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [45782a5f] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host hypervisor7: Failed to execute stage 'Package installation': [u'vdsm-4.30.46-1.el7.x86_64 requires lvm2 >= 7:2.02.186-7.el7_8.1'].

2 1

change lun for hosted_engine use?
by Philip Brown 28 Jul '20

28 Jul '20

I currently have a self-hosted engine, running in a dedicated ISCSI storage domain. I'd like to change the backend hardware for the storage. i think there's some longwinded and complicated method of shutting down the engine, backup, creating a NEW one, restore, and whoknowswhat. I was wondering though, if maybe there's a way to just add an iscsi lun to the storage domain, and tell ovirt, "Hey, move the stuff over to that other lun" ? Ive added the new lun already :) havent found a reference to move the files' residence though. If it helps, my end goal is to remove that lun completely. -- Philip Brown| Sr. Linux System Administrator | Medata, Inc. 5 Peters Canyon Rd Suite 250 Irvine CA 92606 Office 714.918.1310| Fax 714.918.1325 pbrown(a)medata.com| www.medata.com

2 3

Deploying new Ovirt Node on new 4.4 cluster. ansible code incorrect
by erin.sims＠bookit.com 27 Jul '20

27 Jul '20

Hi Guys we attempted to deploy a new ovirt cluster two weeks ago. 4.4.1 and 4.4.0 Once we got the engine up we tried to add a new node and Ansible began to fail on the deploy scripts. I search the logs and discovered the Ansible script was not evaluating the right versions of python, python3. Also the yum module should of been dnf. Once i change those in many places it worked great. I am just trying to figure out what went wrong with our deploy so this doesn't happen again. we are using a hosted engine on a centos 8 vm. please let me know what logs i can grab or apps to run for more details. details: engine: [root@ovirt ~]# uname -a Linux ovirt.ism.ld 4.18.0-193.6.3.el8_2.x86_64 #1 SMP Wed Jun 10 11:09:32 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux [root@ovirt ~]# cat /etc/centos-release CentOS Linux release 8.2.2004 (Core) [root@ovirt ~]# uname -a Linux ovirt.ism.ld 4.18.0-193.6.3.el8_2.x86_64 #1 SMP Wed Jun 10 11:09:32 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux [root@ovirt ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="8 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" PLATFORM_ID="platform:el8" PRETTY_NAME="CentOS Linux 8 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" node2: CENTOS_MANTISBT_PROJECT="CentOS-8" CENTOS_MANTISBT_PROJECT_VERSION="8" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="8" [root@web2 ~]# cat /etc/centos-release CentOS Linux release 8.2.2004 (Core) [root@web2 ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="8 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" VARIANT="oVirt Node 4.4.1.1" VARIANT_ID="ovirt-node" PRETTY_NAME="oVirt Node 4.4.1.1" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://www.ovirt.org/" BUG_REPORT_URL="https://bugzilla.redhat.com/" PLATFORM_ID="platform:el8" [root@web2 ~]# uname -a Linux web2.ism.ld 4.18.0-193.6.3.el8_2.x86_64 #1 SMP Wed Jun 10 11:09:32 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux [root@web2 ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="8 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" VARIANT="oVirt Node 4.4.1.1" VARIANT_ID="ovirt-node" PRETTY_NAME="oVirt Node 4.4.1.1" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://www.ovirt.org/" BUG_REPORT_URL="https://bugzilla.redhat.com/" PLATFORM_ID="platform:el8" [root@web2 ~]# cat /etc/centos-release CentOS Linux release 8.2.2004 (Core) node1: [root@web1 ~]# uname -a Linux web1.ism.ld 4.18.0-147.8.1.el8_1.x86_64 #1 SMP Thu Apr 9 13:49:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux [root@web1 ~]# dnf update CentOS-8 - Gluster 7 0.0 B/s | 0 B 00:00 Failed to download metadata for repo 'ovirt-4.4-centos-gluster7' Error: Failed to download metadata for repo 'ovirt-4.4-centos-gluster7' [root@web1 ~]# cat /etc/centos-release CentOS Linux release 8.1.1911 (Core) [root@web1 ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="8 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" VARIANT="oVirt Node 4.4.0" VARIANT_ID="ovirt-node" PRETTY_NAME="oVirt Node 4.4.0" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://www.ovirt.org/" BUG_REPORT_URL="https://bugzilla.redhat.com/" PLATFORM_ID="platform:el8"

2 2

Centos based Ovirt 4.3.10.4-1.el7, VMs cannot communicate over VM networks
by Konstantinos B 27 Jul '20

27 Jul '20

Hi all Can anyone point into the right direction on how to troubleshoot VM connectivity between VM networks? The nodes have been double-installed from the engine. The external ovn-provider is reachable through HTTP. ovs-vsctl shows the interfaces once a VM is booted. However, no traffic is passed between VMs. Any help is greatly appreciated.

2 1

problem with custom bond options
by Jiří Sléžka 27 Jul '20

27 Jul '20

Hello, CentOS8, oVirt 4.4.1.10-1.el8 I am trying to setup active-backup (mode=1) bonding mode with custom properties. I have one 10GE switch, the second is just 1G. 10GE link is the primary one. cat /etc/sysconfig/network-scripts/ifcfg-bond0 BONDING_OPTS="active_slave=ens5 downdelay=0 miimon=100 mode=active-backup primary=ens5 updelay=0" TYPE=Bond BONDING_MASTER=yes PROXY_METHOD=none BROWSER_ONLY=no IPV4_FAILURE_FATAL=no IPV6_DISABLED=yes IPV6INIT=no NAME=bond0 UUID=c054364e-47cf-47ee-a7fc-70b37c9977e7 DEVICE=bond0 ONBOOT=yes MTU=9000 When I try to add a custom parameter "fail_over_mac=active" (which I believe could solve my problems with stalled mac addresses in switch's cam table in case of failover) I got... "Error while executing action HostSetupNetworks: Unexpected exception" ...in manager. In the engine.log it looks like 2020-07-22 21:20:35,774+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (default task-8) [da1984f3-f38b-4e0a-ac80-a81e67d73ff0] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': 'MAC address cannot be specified in bond interface along with specified bond options'}] 2020-07-22 21:20:35,774+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (default task-8) [da1984f3-f38b-4e0a-ac80-a81e67d73ff0] Failed in 'HostSetupNetworksVDS' method 2020-07-22 21:20:35,774+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (default task-8) [da1984f3-f38b-4e0a-ac80-a81e67d73ff0] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': 'MAC address cannot be specified in bond interface along with specified bond options'}] 2020-07-22 21:20:35,811+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-8) [da1984f3-f38b-4e0a-ac80-a81e67d73ff0] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt-hci01.mch.local command HostSetupNetworksVDS failed: Internal JSON-RPC error: {'reason': 'MAC address cannot be specified in bond interface along with specified bond options'} Could anybody explain me what 'MAC address cannot be specified in bond interface along with specified bond options' means? I believe a MAC address is not configured in interface configuration. Or does it mean 'fail_over_mac=active' is not supported in oVirt? Thanks in advance, Jiri

3 12

Re: Ovirt SYSTEM user does not allow deletion of VM networks
by Konstantinos B 25 Jul '20

25 Jul '20

I've deleted them from the cluster as well but re-appear. I've removed ovirt-engine through engine-cleanup installed again and they reappear. Removed ovirt-engine deleted all "ovirt*" files and currenty trying to re-install. I've checked the host's ovs-vsctl bridges and only the desired are shown. So i believe it's an issue on the engine itself.

3 3

Shutdown procedure for single host HCI Gluster
by Gianluca Cecchi 25 Jul '20

25 Jul '20

Hello, I'm testing the single node HCI with ovirt-node-ng 4.3.9 iso. Very nice and many improvements over the last time I tried it. Good! I have a doubt related to shutdown procedure of the server. Here below my steps: - Shutdown all VMs (except engine) - Put into maintenance data and vmstore domains - Enable Global HA Maintenance - Shutdown engine - Shutdown hypervisor It seems that the last step doesn't end and I had to brutally power off the hypervisor. Here the screenshot regarding infinite failure in unmounting /gluster_bricks/engine https://drive.google.com/file/d/1ee0HG21XmYVA0t7LYo5hcFx1iLxZdZ-E/view?usp=… What would be the right step to do before the final shutdown of hypervisor? Thanks, Gianluca

3 10

RDP
by eevans＠digitaldatatechs.com 24 Jul '20

24 Jul '20

I am using Ovirt 4.3. I followed the instructions for get RDP to work for a user, but admin@internal is the only user the RDP will launch for. The other test users I created and are added to the Remote Desktop Users group which is the permission I assigned to the VM's as well as the user name. I added users first, then the group to see if it would help. So far, admin@internal, the Ovirt admin user, is the only user RDP will work for. I know I am missing something, I'm just not sure what. Any help would be appreciated. Eric.

3 4

very very bad iscsi performance
by Philip Brown 24 Jul '20

24 Jul '20

I'm trying to get optimal iscsi performance. We're a heavy iscsi shop, with 10g net. I'mm experimenting with SSDs, and the performance in ovirt is way, way less than I would have hoped. More than an order of magnitude slower. here's a datapoint. Im running filebench, with the OLTP workload. First, i run it on one of the hosts, that has an SSD directly attached. create an xfs filesystem (created on a vg "device" on top of the SSD), mount it with noatime, and run the benchmark. 37166: 74.084: IO Summary: 3746362 ops, 62421.629 ops/s, (31053/31049 r/w), 123.6mb/s, 161us cpu/op, 1.1ms latency I then unmount it, and make the exact same device an iscsi target, and create a storage domain with it. I then create a disk for a VM running *on the same host*, and run the benchmark. The same thing: filebench, oltp workload, xfs filesystem, noatime. 13329: 91.728: IO Summary: 153548 ops, 2520.561 ops/s, (1265/1243 r/w), 4.9mb/s, 289us cpu/op, 88.4ms latency 62,000 ops/s vs 2500 ops/s. what???? Someone might be tempted to say, "try making the device directly available, AS a device, to the VM". Unfortunately,this is not an option. My goal is specifically to put together a new, high performing storage domain, that I can use as database devices in VMs. I'm not expecting the same 62,000 ops/second. but I was expecting at *least* 5,000. Ideally more like 10,000. -- Philip Brown| Sr. Linux System Administrator | Medata, Inc. 5 Peters Canyon Rd Suite 250 Irvine CA 92606 Office 714.918.1310| Fax 714.918.1325 pbrown(a)medata.com| www.medata.com

5 13

qemu-guest-agent on Ubuntu doesn't report FQDN
by Florian Schmid 24 Jul '20

24 Jul '20

Hi, I have a problem with Ubuntu 20.04 VM reporting the correct FQDN to the engine. Starting with this release, the ovirt-guest-agent is not available anymore. Therefore, I have installed qemu-geust-agent with package defaults. Now in the Engine, I only see the hostname under FQDN tab, instead the real full name with domain. I'm running an oVirt environment on 4.3.8. The VM is resolveable, forward and reverse DNS entries are working. hostname -f shows the correct FQDN. Even adding IP and FQDN to /etc/hosts file doesn't change anything. qemu-guest-agent version: 4.2-3ubuntu6.3 I manage this VM via ansible 2.9 and ansible is able to get the FQDN of the VM without any issues... What can I do here to debug my issue? Does the engine cache the wrong result? Even after stopping and starting the VM again, engine is only showing the hostname instead of the FQDN. Best regards, Florian

5 10

Problem with paused VMs in ovirt 4.3.10.
by Damien 24 Jul '20

24 Jul '20

Hi! We have a problem with paused VMs in ovirt cluster. Please, help to solve this. In ovirt manager massege "VM rtb-stagedsw02-ovh has been paused." Resume fails with error "Failed to resume VM rtb-stagedsw02-ovh (Host: ovirt-node09-ovh.local, User: admin@internal-authz)." In oVirt Cluster 38 VM, paused VM only ubuntu 20.04 focal with docker swarm. Archived logs in attach. Packeges on ovirt nodes: python2-ovirt-setup-lib-1.2.0-1.el7.noarch ovirt-vmconsole-1.0.7-2.el7.noarch ovirt-provider-ovn-driver-1.2.29-1.el7.noarch ovirt-vmconsole-host-1.0.7-2.el7.noarch python2-ovirt-host-deploy-1.8.5-1.el7.noarch ovirt-imageio-common-1.5.3-0.el7.x86_64 cockpit-machines-ovirt-195.6-1.el7.centos.noarch ovirt-ansible-engine-setup-1.1.9-1.el7.noarch ovirt-host-dependencies-4.3.5-1.el7.x86_64 ovirt-host-4.3.5-1.el7.x86_64 python-ovirt-engine-sdk4-4.3.4-2.el7.x86_64 ovirt-host-deploy-common-1.8.5-1.el7.noarch ovirt-ansible-hosted-engine-setup-1.0.32-1.el7.noarch ovirt-hosted-engine-setup-2.3.13-1.el7.noarch ovirt-ansible-repositories-1.1.5-1.el7.noarch ovirt-imageio-daemon-1.5.3-0.el7.noarch cockpit-ovirt-dashboard-0.13.10-1.el7.noarch ovirt-release43-4.3.10-1.el7.noarch ovirt-hosted-engine-ha-2.3.6-1.el7.noarch Packeges on HostedEngine: ovirt-ansible-infra-1.1.13-1.el7.noarch ovirt-vmconsole-1.0.7-2.el7.noarch ovirt-engine-setup-plugin-websocket-proxy-4.3.10.4-1.el7.noarch ovirt-engine-websocket-proxy-4.3.10.4-1.el7.noarch ovirt-engine-restapi-4.3.10.4-1.el7.noarch ovirt-ansible-engine-setup-1.1.9-1.el7.noarch ovirt-ansible-shutdown-env-1.0.3-1.el7.noarch ovirt-iso-uploader-4.3.2-1.el7.noarch ovirt-provider-ovn-1.2.29-1.el7.noarch ovirt-imageio-proxy-setup-1.5.3-0.el7.noarch ovirt-engine-extension-aaa-ldap-setup-1.3.10-1.el7.noarch ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.3.10.4-1.el7.noarch python-ovirt-engine-sdk4-4.3.4-2.el7.x86_64 python2-ovirt-host-deploy-1.8.5-1.el7.noarch ovirt-ansible-vm-infra-1.1.22-1.el7.noarch ovirt-engine-metrics-1.3.7-1.el7.noarch ovirt-ansible-disaster-recovery-1.2.0-1.el7.noarch ovirt-engine-wildfly-overlay-17.0.1-1.el7.noarch ovirt-ansible-roles-1.1.7-1.el7.noarch ovirt-engine-dwh-setup-4.3.8-1.el7.noarch python2-ovirt-engine-lib-4.3.10.4-1.el7.noarch ovirt-engine-extension-aaa-ldap-1.3.10-1.el7.noarch ovirt-engine-setup-plugin-ovirt-engine-4.3.10.4-1.el7.noarch ovirt-engine-vmconsole-proxy-helper-4.3.10.4-1.el7.noarch ovirt-engine-tools-backup-4.3.10.4-1.el7.noarch ovirt-engine-webadmin-portal-4.3.10.4-1.el7.noarch ovirt-host-deploy-common-1.8.5-1.el7.noarch ovirt-ansible-image-template-1.1.12-1.el7.noarch ovirt-ansible-manageiq-1.1.14-1.el7.noarch ovirt-engine-wildfly-17.0.1-1.el7.x86_64 ovirt-ansible-hosted-engine-setup-1.0.32-1.el7.noarch ovirt-imageio-common-1.5.3-0.el7.x86_64 ovirt-imageio-proxy-1.5.3-0.el7.noarch python2-ovirt-setup-lib-1.2.0-1.el7.noarch ovirt-vmconsole-proxy-1.0.7-2.el7.noarch ovirt-engine-setup-base-4.3.10.4-1.el7.noarch ovirt-engine-setup-plugin-cinderlib-4.3.10.4-1.el7.noarch ovirt-engine-extensions-api-impl-4.3.10.4-1.el7.noarch ovirt-release43-4.3.10-1.el7.noarch ovirt-engine-backend-4.3.10.4-1.el7.noarch ovirt-engine-tools-4.3.10.4-1.el7.noarch ovirt-web-ui-1.6.0-1.el7.noarch ovirt-ansible-cluster-upgrade-1.1.14-1.el7.noarch ovirt-cockpit-sso-0.1.1-1.el7.noarch ovirt-engine-ui-extensions-1.0.10-1.el7.noarch ovirt-engine-setup-plugin-ovirt-engine-common-4.3.10.4-1.el7.noarch ovirt-engine-4.3.10.4-1.el7.noarch ovirt-ansible-repositories-1.1.5-1.el7.noarch ovirt-engine-extension-aaa-jdbc-1.1.10-1.el7.noarch ovirt-host-deploy-java-1.8.5-1.el7.noarch ovirt-engine-dwh-4.3.8-1.el7.noarch ovirt-engine-api-explorer-0.0.5-1.el7.noarch ovirt-guest-agent-common-1.0.16-1.el7.noarch ovirt-engine-setup-4.3.10.4-1.el7.noarch ovirt-engine-dbscripts-4.3.10.4-1.el7.noarch In /var/log/ovirt-engine/engine.log: 2020-07-24 09:38:44,472+03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] VM '18f6bb79-ba9b-4a0e-bcb2-b4ef4904ef99'(rtb-stagedsw02-ovh) move d from 'Up' --> 'Paused' 2020-07-24 09:38:44,493+03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] EVENT_ID: VM_PAUSED(1,025), VM rtb-stagedsw02-ovh has been paused. In /var/log/vdsm/vdsm.log 2020-07-24 09:38:42,771+0300 INFO (libvirt/events) [virt.vm] (vmId='18f6bb79-ba9b-4a0e-bcb2-b4ef4904ef99') CPU stopped: onSuspend (vm:6100) 2020-07-24 09:38:44,328+0300 INFO (jsonrpc/1) [api.host] FINISH getAllVmIoTunePolicies return={'status': {'message': 'Done', 'code': 0}, 'io_tune_policies_dict': {'4d9519f6-1ab9-4032-8fdf-4c6118531544': {'poli cy': [], 'current_values': [{'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path': '/rhev/data-center/mnt/glust erSD/10.0.11.107:_vmstore02/16c5070c-cc5f-4595-965f-66838c7c17a5/images/e1cfb9ec-39d8-416d-9f5f-0b54765301d4/8f95d60d-931b-4764-993c-ba9373efe361', 'name': 'sda'}]}, 'b031a269-6bcd-40b7-9737-e47112a54b3a': {'po licy': [], 'current_values': [{'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path': '/rhev/data-center/mnt/glu sterSD/10.0.11.101:_vmstore01/5e05fed3-448b-4f86-b5ba-004982194c90/images/9c3cc7a0-254e-4756-91b6-fb54e21abf38/71dd8024-8aec-46da-a80f-34260655e929', 'name': 'sda'}, {'ioTune': {'write_bytes_sec': 0L, 'total_io ps_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path': '/rhev/data-center/mnt/glusterSD/10.0.11.101:_vmstore01/5e05fed3-448b-4f86-b5ba-004982194c90/images/ 3e3a5064-5fe1-40c0-81f5-44f1a3a4d503/13549972-82de-4746-aeea-3e1531f9c180', 'name': 'sdb'}]}, 'b5fad17c-fa9d-4a80-99e7-6f86e6e19c9b': {'policy': [], 'current_values': [{'ioTune': {'write_bytes_sec': 0L, 'total_ iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path': '/rhev/data-center/mnt/glusterSD/10.0.11.107:_vmstore02/16c5070c-cc5f-4595-965f-66838c7c17a5/image s/15ce6cb0-6f06-4a31-92d8-b6e1bcabf3bc/613de344-d1ad-49aa-a2d0-d60ca9eb7cd3', 'name': 'sda'}]}, '18f6bb79-ba9b-4a0e-bcb2-b4ef4904ef99': {'policy': [], 'current_values': [{'ioTune': {'write_bytes_sec': 0L, 'tota l_iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path': u'/rhev/data-center/mnt/glusterSD/10.0.11.107:_vmstore02/16c5070c-cc5f-4595-965f-66838c7c17a5/im ages/7978e2db-c560-4315-a775-223f1b13ae31/d927eea8-e588-449e-b07b-c845d15b082e', 'name': 'sda'}, {'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_s ec': 0L, 'total_bytes_sec': 0L}, 'path': u'/rhev/data-center/mnt/glusterSD/10.0.11.107:_vmstore02/16c5070c-cc5f-4595-965f-66838c7c17a5/images/b925dc2e-17ba-470d-a9be-cb96d4ef1f0d/951d9712-7160-4f88-a838-970aec8 2b3ea', 'name': 'sdb'}]}}} from=::1,34598 (api:54) 2020-07-24 09:38:49,747+0300 WARN (qgapoller/1) [virt.periodic.VmDispatcher] could not run <function <lambda> at 0x7fe5c84de6e0> on ['18f6bb79-ba9b-4a0e-bcb2-b4ef4904ef99'] (periodic:289) In /var/log/libvirt/qemu/rtb-stagedsw03-ovh.log KVM: entry failed, hardware error 0x80000021 If you're running a guest on an Intel machine without unrestricted mode support, the failure can be most likely due to the guest entering an invalid state for Intel VT. For example, the guest maybe running in big real mode which is not supported on less recent Intel processors. EAX=00001000 EBX=43117da8 ECX=0000000c EDX=00000121 ESI=00000003 EDI=17921000 EBP=43117cb0 ESP=43117c98 EIP=00008000 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=1 HLT=0 ES =0000 00000000 ffffffff 00809300 CS =9b00 7ff9b000 ffffffff 00809300 SS =0000 00000000 ffffffff 00809300 DS =0000 00000000 ffffffff 00809300 FS =0000 00000000 ffffffff 00809300 GS =0000 00000000 ffffffff 00809300 LDT=0000 00000000 000fffff 00000000 TR =0040 001ce000 0000206f 00008b00 GDT= 001cc000 0000007f IDT= 00000000 00000000 CR0=00050032 CR2=17921000 CR3=2b92a003 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000fffe0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff <ff> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

1 0

iSCSI multipath with separate subnets... still not possible in 4.4.x?
by Mark R 24 Jul '20

24 Jul '20

I'm looking through quite a few bug reports and mailing list threads, but want to make sure I'm not missing some recent development. It appears that doing iSCSI with two separate, non-routed subnets is still not possible with 4.4.x. I have the dead-standard iSCSI setup with two separate switches, separate interfaces on hosts and storage, and separate subnets that have no gateway and are completely unreachable except from directly attached interfaces. The hosted-engine comes up with multiple paths and everything is perfect, but that's because Ansible/hosted-engine deploy script have configured things correctly. Once you need to import or add new storage domains, it's not possible to do so in a way that gets both paths connected *and* persists across host reboots. Following the docs to create 'iSCSI Multipath" bonds (plural, you can _not_ create a single bond that includes both interfaces and hope things route correctly... oVirt will try to connect from the interface for storage network A to the target on storage network B, which can't happen since they are not routed (and should not be). So, there's nothing in the docs about how you can accomplish multipathing, but there are a few mailing list messages that say "just create two separate "iSCSI Multipath" bonds in the datacenter, one for each of your two interfaces. You can do this, and you'll get hopeful that things might work now. You can do discovery and it succeeds, because no more trying to connect to unreachable targets. However, and big caveat, there's no way to tell this new/imported domain, "Oh, use this other interface as well, so you have redundant paths". Once the domain is attached and activated, you have a single path. You can then manage the domain, do a discovery, see a path that isn't connected yet, and log into it as well. Now you have two paths, is everything right with the world?!? Nope, it's impossible to persist that connection, it will be gone on next reboot and you'll always have to manually visit each host, do discovery, and login. Nothing in the UI allows you to "Save" that second connection in a way that it will be used again. Clicking "OK" does not, and going back to the "iSCSI Multipath" area of the Data Center you can't edit each of the bonds and make sure each logical network has every possible target checked, because these targets you've manually logged into are never listed in that area of the UI. So I really, really hope I'm wrong because I'd like to move past this snag and onto the next one (which is that bond interfaces in 4.4.x will not allow you to attach additional networks... works great in 4.3, appears broken in 4.4.x). But, no sense chasing that yet if iSCSI multipath isn't possible, which is looking likely. Has anyone had success, running iSCSI in by far the most common setup out there, but also in a way oVirt really doesn't want to let you? This is driving me nuts, I've paved and rebuilt these hosts dozens of times now, trying different methods in the hopes of getting multipath that persists.

6 7

Hosted Engine 4.4.1
by Vijay Sachdeva 24 Jul '20

24 Jul '20

Hello Everyone, Waiting for host to be up task is stuck for hours and when checked engine log found this below: 2020-07-22 16:50:35,717+02 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-1) [] OAuthException access_denied: Cannot authenticate user 'None@N/A': No valid profile found in credentials.. Has anyone faced such issue then please help me out..!! Thanks Vijay Sachdeva

2 4