- Users - oVirt List Archives

live merge of snapshots failed
by g.vasilopoulos＠uoc.gr 10 Aug '21

10 Aug '21

Hello I have a situation with a vm in which I cannot delete the snapshot. The whole thing is quite strange because I can delete the snapshot when I create and delete it from the web interface but when I do it with a python script through the API it failes. The script does create snapshot-> download snapshot-> delete snapshot and I used the examples from ovirt python sdk on githab to create it ,in general it works prety well. But on a specific machine (so far) it cannot delete the live snapshot Ovirt is 4.3.10 and the guest is a windows 10 pc. Windows 10 guest has 2 disks attached both on different fc domains one on an ssd emc and the other on an hdd emc. Both disks are prealocated. I cannot figure out what the problem is so far the related engine log: 2021-08-03 15:51:00,385+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Comma nd 'RemoveSnapshotSingleDiskLive' (id: '80dc4609-b91f-4e93-bc12-7b2083933e5a') waiting on child command id: '74c83880-581b-4774-ae51-8c4af0c92c53' type:'Merge' to complete 2021-08-03 15:51:00,385+03 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Waiting on merge command to complete ( jobId = 62bf8c83-cd78-42a5-b57d-d67ddfdee8ee) 2021-08-03 15:51:00,387+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' (id: '87bc90c7-2aa5-4a1b-b58c-54296518658a') waiting on child command id: 'ec806ac6-929f-42d9-a86e-98d6a39a4718' type:'Merge' to complete 2021-08-03 15:51:01,388+03 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Waiting on merge command to complete (jobId = c57fb3e5-da20-4838-8db3-31655ba76c1f) 2021-08-03 15:51:07,491+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [b929fd4a-8ce7-408f-927d-ab0169879c4e] Command 'MoveImageGroup' (id: '1de1b800-873f-405f-805b-f44397740909') waiting on child command id: 'd1136344-2888-4d63-8fe1-b506426bc8aa' type:'CopyImageGroupWithData' to complete 2021-08-03 15:51:11,513+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshot' (id: '04e9d61e-28a2-4ab0-9bb7-5c805ee871e9') waiting on child command id: '87bc90c7-2aa5-4a1b-b58c-54296518658a' type:'RemoveSnapshotSingleDiskLive' to complete 2021-08-03 15:51:12,522+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' (id: '80dc4609-b91f-4e93-bc12-7b2083933e5a') waiting on child command id: '74c83880-581b-4774-ae51-8c4af0c92c53' type:'Merge' to complete 2021-08-03 15:51:12,523+03 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Waiting on merge command to complete (jobId = 62bf8c83-cd78-42a5-b57d-d67ddfdee8ee) 2021-08-03 15:51:12,527+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' (id: '87bc90c7-2aa5-4a1b-b58c-54296518658a') waiting on child command id: 'ec806ac6-929f-42d9-a86e-98d6a39a4718' type:'Merge' to complete 2021-08-03 15:51:13,528+03 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Waiting on merge command to complete (jobId = c57fb3e5-da20-4838-8db3-31655ba76c1f) 2021-08-03 15:51:21,635+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshot' (id: '04e9d61e-28a2-4ab0-9bb7-5c805ee871e9') waiting on child command id: '87bc90c7-2aa5-4a1b-b58c-54296518658a' type:'RemoveSnapshotSingleDiskLive' to complete 2021-08-03 15:51:22,655+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' (id: '80dc4609-b91f-4e93-bc12-7b2083933e5a') waiting on child command id: '74c83880-581b-4774-ae51-8c4af0c92c53' type:'Merge' to complete 2021-08-03 15:51:22,661+03 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Merge command (jobId = 62bf8c83-cd78-42a5-b57d-d67ddfdee8ee) has completed for images '7611ebcf-5323-45ca-b16c-9302d0bdedc6'..'17618ba1-4ab8-49eb-a991-fc3d602ced14' 2021-08-03 15:51:22,664+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' (id: '87bc90c7-2aa5-4a1b-b58c-54296518658a') waiting on child command id: 'ec806ac6-929f-42d9-a86e-98d6a39a4718' type:'Merge' to complete 2021-08-03 15:51:23,664+03 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Waiting on merge command to complete (jobId = c57fb3e5-da20-4838-8db3-31655ba76c1f) 2021-08-03 15:51:24,672+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Executing Live Merge command step 'MERGE_STATUS' 2021-08-03 15:51:24,699+03 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Running command: MergeStatusCommand internal: true. Entities affected : ID: 96000ec9-e181-44eb-893f-e0a36e3a6775 Type: Storage 8-03 15:51:24,749+03 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Successfully removed volume 17618ba1-4 ab8-49eb-a991-fc3d602ced14 from the chain 2021-08-03 15:51:24,749+03 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Volume merge type 'COMMIT' 2021-08-03 15:51:25,691+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [b929fd4a-8ce7-408f-927d-ab0169879c4e] Command 'CopyImageGroup WithData' (id: 'd1136344-2888-4d63-8fe1-b506426bc8aa') waiting on child command id: 'b40aeec9-b7cf-4ee5-9683-54d98f4307d5' type:'CopyImageGroupVolumesData' to complete 2021-08-03 15:51:26,692+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [b929fd4a-8ce7-408f-927d-ab0169879c4e] Command 'CopyImageGrou pVolumesData' (id: 'b40aeec9-b7cf-4ee5-9683-54d98f4307d5') waiting on child command id: 'f46d65cb-32d9-4269-982e-6e19331b8a27' type:'CopyData' to complete 2021-08-03 15:51:26,711+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Executing Liv e Merge command step 'DESTROY_IMAGE' 2021-08-03 15:51:26,726+03 INFO [org.ovirt.engine.core.bll.storage.disk.image.DestroyImageCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Running command: DestroyImageCommand internal: true. Entities affected : ID: 96000ec9-e181-44eb-893f-e0a36e3a6775 Type: Storage 2021-08-03 15:51:26,747+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DestroyImageVDSCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] START, DestroyIma geVDSCommand( DestroyImageVDSCommandParameters:{storagePoolId='5da76866-7b7d-11eb-9913-00163e1f2643', ignoreFailoverLimit='false', storageDomainId='96000ec9-e181-44eb-893f-e0a36e3a6775', imageGroupId='205a30a3-f c06-4ceb-8ef2-018f16d4ccbb', imageId='00000000-0000-0000-0000-000000000000', imageList='[17618ba1-4ab8-49eb-a991-fc3d602ced14]', postZero='false', force='false'}), log id: 307a3fb1 2021-08-03 15:51:26,834+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DestroyImageVDSCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] FINISH, DestroyIm ageVDSCommand, return: , log id: 307a3fb1 2021-08-03 15:51:26,846+03 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] CommandAsyncTask::Adding CommandM ultiAsyncTasks object for command 'baaa5254-261c-452a-84b4-0a8b397cdb62' 2021-08-03 15:51:26,847+03 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] CommandMultiAsyncTasks::attachTas k: Attaching task 'ae2e80a4-d224-4fc3-a84b-859042811525' to command 'baaa5254-261c-452a-84b4-0a8b397cdb62'. 2021-08-03 15:51:26,857+03 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Adding task 'ae2e80a4-d224-4fc3-a 84b-859042811525' (Parent Command 'DestroyImage', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2021-08-03 15:51:26,858+03 INFO [org.ovirt.engine.core.bll.storage.disk.image.DestroyImageCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Successfully star ted task to remove orphaned volumes 2021-08-03 15:51:26,863+03 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] BaseAsyncTask::startPollingTask: Star ting to poll task 'ae2e80a4-d224-4fc3-a84b-859042811525'. 2021-08-03 15:51:26,863+03 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [3bf9345d-fab2-490f-ba44-6aa014bbb743] BaseAsyncTask::startPollingTask: Star ting to poll task 'ae2e80a4-d224-4fc3-a84b-859042811525'. 2021-08-03 15:51:28,799+03 INFO [org.ovirt.engine.core.bll.storage.disk.image.DestroyImageCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Waiting on destroy image command to complete the task (taskId = ae2e80a4-d224-4fc3-a84b-859042811525) 2021-08-03 15:51:30,805+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' (id: '80dc4609-b91f-4e93-bc12-7b2083933e5a') waiting on child command id: 'baaa5254-261c-452a-84b4-0a8b397cdb62' type:'DestroyImage' to complete 2021-08-03 15:51:31,824+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshot' (id: '04e9d61e-28a2-4ab0-9bb7-5c805ee871e9') waiting on child command id: '87bc90c7-2aa5-4a1b-b58c-54296518658a' type:'RemoveSnapshotSingleDiskLive' to complete 2021-08-03 15:51:32,825+03 INFO [org.ovirt.engine.core.bll.storage.disk.image.DestroyImageCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Waiting on destroy image command to complete the task (taskId = ae2e80a4-d224-4fc3-a84b-859042811525) 2021-08-03 15:51:32,831+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' (id: '87bc90c7-2aa5-4a1b-b58c-54296518658a') waiting on child command id: 'ec806ac6-929f-42d9-a86e-98d6a39a4718' type:'Merge' to complete 2021-08-03 15:51:33,594+03 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'DestroyImage' completed, handling the result. 2021-08-03 15:51:33,594+03 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'DestroyImage' succeeded, clearing tasks. 2021-08-03 15:51:33,594+03 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'ae2e80a4-d224-4fc3-a84b-859042811525' 2021-08-03 15:51:33,595+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='5da76866-7b7d-11eb-9913-00163e1f2643', ignoreFailoverLimit='false', taskId='ae2e80a4-d224-4fc3-a84b-859042811525'}), log id: 6225ae22 2021-08-03 15:51:33,596+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] START, HSMClearTaskVDSCommand(HostName = ovirt2-7.vmmgmt-int.uoc.gr, HSMTaskGuidBaseVDSCommandParameters:{hostId='10599b78-5f45-48d2-bfe0-028f3dae69eb', taskId='ae2e80a4-d224-4fc3-a84b-859042811525'}), log id: 45b18a8a 2021-08-03 15:51:33,611+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] FINISH, HSMClearTaskVDSCommand, return: , log id: 45b18a8a 2021-08-03 15:51:33,611+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] FINISH, SPMClearTaskVDSCommand, return: , log id: 6225ae22 2021-08-03 15:51:33,614+03 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] BaseAsyncTask::removeTaskFromDB: Removed task 'ae2e80a4-d224-4fc3-a84b-859042811525' from DataBase 2021-08-03 15:51:33,614+03 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-1481337) [3bf9345d-fab2-490f-ba44-6aa014bbb743] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'baaa5254-261c-452a-84b4-0a8b397cdb62' 2021-08-03 15:51:33,839+03 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Merge command (jobId = c57fb3e5-da20-4838-8db3-31655ba76c1f) has completed for images '84c005da-cbec-4ace-8619-5a8e2ae5ea75'..'b43b7c33-5b53-4332-a2e0-f950debb919b' 2021-08-03 15:51:34,847+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Executing Live Merge command step 'MERGE_STATUS' 2021-08-03 15:51:34,917+03 ERROR [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Failed to live merge. Top volume b43b7c33-5b53-4332-a2e0-f950debb919b is still in qemu chain [b43b7c33-5b53-4332-a2e0-f950debb919b, 84c005da-cbec-4ace-8619-5a8e2ae5ea75] 2021-08-03 15:51:35,866+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [b929fd4a-8ce7-408f-927d-ab0169879c4e] Command 'CopyImageGroupWithData' (id: 'd1136344-2888-4d63-8fe1-b506426bc8aa') waiting on child command id: 'b40aeec9-b7cf-4ee5-9683-54d98f4307d5' type:'CopyImageGroupVolumesData' to complete 2021-08-03 15:51:36,867+03 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [b929fd4a-8ce7-408f-927d-ab0169879c4e] Command 'CopyImageGroupVolumesData' (id: 'b40aeec9-b7cf-4ee5-9683-54d98f4307d5') waiting on child command id: 'f46d65cb-32d9-4269-982e-6e19331b8a27' type:'CopyData' to complete 2021-08-03 15:51:36,873+03 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command id: '87bc90c7-2aa5-4a1b-b58c-54296518658a failed child command status for step 'MERGE_STATUS' 2021-08-03 15:51:36,873+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' id: '87bc90c7-2aa5-4a1b-b58c-54296518658a' child commands '[a7de75fe-e94c-4795-9310-2c8fc3d6d3fc, ec806ac6-929f-42d9-a86e-98d6a39a4718, 6443529c-d753-48f6-8a9e-af1f9f09dfb5]' executions were completed, status 'FAILED' 2021-08-03 15:51:37,982+03 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Merging of snapshot '1cd4985b-b0c0-40d6-bbde-54451e43bef6' images '84c005da-cbec-4ace-8619-5a8e2ae5ea75'..'b43b7c33-5b53-4332-a2e0-f950debb919b' failed. Images have been marked illegal and can no longer be previewed or reverted to. Please retry Live Merge on the snapshot to complete the operation. 2021-08-03 15:51:37,985+03 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand' with failure. 2021-08-03 15:51:39,014+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Executing Live Merge command step 'REDUCE_IMAGE' 2021-08-03 15:51:39,029+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [3bf9345d-fab2-490f-ba44-6aa014bbb743] No need to execute reduce image command, skipping its execution. Storage Type: 'FCP', Disk: 'anova.admin.uoc.gr_Disk2' Snapshot: 'anova.admin.uoc.gr-2021-08-03' 2021-08-03 15:51:39,034+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshotSingleDiskLive' id: '80dc4609-b91f-4e93-bc12-7b2083933e5a' child commands '[18653ea6-166c-41a3-b335-84525871b9e6, 74c83880-581b-4774-ae51-8c4af0c92c53, 92e92d8e-b099-47dd-ba8c-e3db907f9a62, baaa5254-261c-452a-84b4-0a8b397cdb62]' executions were completed, status 'SUCCEEDED' 2021-08-03 15:51:39,040+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshot' (id: '04e9d61e-28a2-4ab0-9bb7-5c805ee871e9') waiting on child command id: '80dc4609-b91f-4e93-bc12-7b2083933e5a' type:'RemoveSnapshotSingleDiskLive' to complete 2021-08-03 15:51:40,046+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [3bf9345d-fab2-490f-ba44-6aa014bbb743] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{storagePoolId='5da76866-7b7d-11eb-9913-00163e1f2643', ignoreFailoverLimit='false', storageDomainId='96000ec9-e181-44eb-893f-e0a36e3a6775', imageGroupId='205a30a3-fc06-4ceb-8ef2-018f16d4ccbb', imageId='7611ebcf-5323-45ca-b16c-9302d0bdedc6'}), log id: 4c98e4db 2021-08-03 15:51:40,047+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [3bf9345d-fab2-490f-ba44-6aa014bbb743] START, GetVolumeInfoVDSCommand(HostName = ovirt2-7.vmmgmt-int.uoc.gr, GetVolumeInfoVDSCommandParameters:{hostId='10599b78-5f45-48d2-bfe0-028f3dae69eb', storagePoolId='5da76866-7b7d-11eb-9913-00163e1f2643', storageDomainId='96000ec9-e181-44eb-893f-e0a36e3a6775', imageGroupId='205a30a3-fc06-4ceb-8ef2-018f16d4ccbb', imageId='7611ebcf-5323-45ca-b16c-9302d0bdedc6'}), log id: 4aca5f83 2021-08-03 15:51:40,084+03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [3bf9345d-fab2-490f-ba44-6aa014bbb743] FINISH, GetVolumeInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@70097e6, log id: 4aca5f83 2021-08-03 15:51:40,084+03 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [3bf9345d-fab2-490f-ba44-6aa014bbb743] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@70097e6, log id: 4c98e4db 2021-08-03 15:51:40,283+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Successfully merged snapshot '1cd4985b-b0c0-40d6-bbde-54451e43bef6' images '17618ba1-4ab8-49eb-a991-fc3d602ced14'..'7611ebcf-5323-45ca-b16c-9302d0bdedc6' 2021-08-03 15:51:40,287+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand' successfully. 2021-08-03 15:51:40,296+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Command 'RemoveSnapshot' id: '04e9d61e-28a2-4ab0-9bb7-5c805ee871e9' child commands '[87bc90c7-2aa5-4a1b-b58c-54296518658a, 80dc4609-b91f-4e93-bc12-7b2083933e5a]' executions were completed, status 'FAILED' 2021-08-03 15:51:41,322+03 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [3bf9345d-fab2-490f-ba44-6aa014bbb743] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. 2021-08-03 15:51:41,353+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [3bf9345d-fab2-490f-ba44-6aa014bbb743] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'anova.admin.uoc.gr-2021-08-03' for VM 'anova.admin.uoc.gr'. Any help would be highly appreciated

3 8

Q: Get Host Capabilities Failed after restart
by Andrei Verovski 10 Aug '21

10 Aug '21

Hi, I have oVirt 4.4.7.6-1.el8 and one problematic node (HP ProLiant with CentOS 8 stream). After replacing server rack router switch and restart got this error I can’t recover from: VDSM node14 command Get Host Capabilities failed: Message timeout which can be caused by communication issues vdsm-network running fine, but vdsmd can’t start on node14 for whatever reason. All other nodes running fine. Aug 09 10:24:12 node14.mydomain.lv vdsmd_init_common.sh[4825]: vdsm: Running dummybr Aug 09 10:24:13 node14.mydomain.lv vdsmd_init_common.sh[4825]: vdsm: Running tune_system Aug 09 10:24:13 node14.mydomain.lv vdsmd_init_common.sh[4825]: vdsm: Running test_space Aug 09 10:24:13 node14.mydomain.lv vdsmd_init_common.sh[4825]: vdsm: Running test_lo Aug 09 10:24:13 node14.mydomain.lv systemd[1]: Started Virtual Desktop Server Manager. Aug 09 10:24:16 node14.mydomain.lv sudo[7721]: pam_systemd(sudo:session): Failed to create session: Start job for unit user-0.slice failed with 'canceled' Aug 09 10:24:16 node14.mydomain.lv sudo[7721]: pam_unix(sudo:session): session opened for user root by (uid=0) Aug 09 10:24:16 node14.mydomain.lv sudo[7721]: pam_unix(sudo:session): session closed for user root Aug 09 10:24:17 node14.mydomain.lv vdsm[6754]: WARN MOM not available. Error: [Errno 2] No such file or directory Aug 09 10:24:17 node14.mydomain.lv vdsm[6754]: WARN MOM not available, KSM stats will be missing. Error: In web gui -> Management I can’t do anything with the host except restart. Stop aborts with error, all other commands are gray-ed out. Status is “Unassigned”. Host is answering to pings as usual. vdsm.log (from node14) attached. Thanks in advance for any help.

2 11

Storage Domains for a VM from Template
by Shantur Rathore 09 Aug '21

09 Aug '21

Hi all, I have multiple storage domains in my DC. One is backed up that has templates and another one which isn't backed up and I want to use it for VM instances of the template. I cannot find an option to create the VM from a template and use a storage domain other than where the template is stored. Is it even possible? Thanks, Shantur

2 3

any configuration when clients can't directly access oVirt cluster?
by yam yam 09 Aug '21

09 Aug '21

Hello, Under my environment, clients must go through a proxy to reach the oVirt cluster(physical nodes), so that I applied DNAT rules for engine and VM IPs to clients. but there were still some issues like VNC console(+browser) don't work, and I'm not sure other features work well. so, I'm wondering whether there is any oVirt configuration or some guides for such case using the proxy. Thanks,

2 1

ISO Upload in in Paused by System Status
by louisb＠ameritech.net 09 Aug '21

09 Aug '21

I'm attempting to upload an ISO that is approximately 9GB in size. I've succssfullly started th upload process via the oVirt Management Console/Disk. The upload started however, it now has a status of "Paused by System". My storage type is set to NFS Data. Is something happing the back ground that contributing to the "Paused by System"? I have mor the 84TB of space available so I don't think it is a space issue. Is there something that I need to do? At this time I'm going to wait and see if it moves forward on its own. Please provide me with any help or direction. Thanks

5 10

adding new host failed with error "CPU does not match the Cluster CPU Type"
by jimod4＠yahoo.com 09 Aug '21

09 Aug '21

i get the below error when i try to add a new kvm host to the cluster. I know that the cpu is a haswell, Intel E5-2640 v3. if i run "virsh capabilities" it returns <model>Haswell-noTSX-IBRS</model>. My KVM server is redhat 8.4 fully patched today on an DL-360 Gen9. oVirt is at 4.4.7.7-1.el8. i have tried many diffent CPU settings for the cluster and none of them work. this is new oVirt install. ERROR: "The host CPU does not match the Cluster CPU Type and is running in a degraded mode. It is missing the following CPU flags: vmx, model_Haswell-noTSX, nx. Please update the host CPU microcode or change the Cluster CPU Type." MIRCOCODE: root@kvm09 ~]# dmesg | grep 'microcode' [ 0.000000] microcode: microcode updated early to revision 0x46, date = 2021-01-27 [ 1.625108] microcode: sig=0x306f2, pf=0x1, revision=0x46 [ 1.625753] microcode: Microcode Update Driver: v2.2.

3 3

Adding new host to a cluster and stuck at "Detect if host is a prebuilt image".
by fauzuwan.nazri93＠gmail.com 08 Aug '21

08 Aug '21

Hello everyone, I try to add a host to a new cluster and during provisioning the event showed "Detect if host is a prebuilt image." and it just stuck there forever. Version: ovirt-node-ng-image-update-placeholder-4.4.7.1-1.el8.noarch Actual results: Host provisioning stuck at "Detect if host is a prebuilt image." Expected results: Provisioning successful. By checking the ovirt host deployment playbook logs, I can't found any useful output. Below is the last output form the logs. 2021-08-06 19:25:12 MYT - TASK [ovirt-host-deploy-facts : Detect if host is a prebuilt image] ************ 2021-08-06 19:25:12 MYT - ok: [ovirth02.zyzyx.virtnet] 2021-08-06 19:25:12 MYT - { "status" : "OK", "msg" : "", "data" : { "uuid" : "a7b74be9-30ab-4ca4-b2b6-e7af6b86bb6c", "counter" : 24, "stdout" : "ok: [ovirth02.zyzyx.virtnet]", "start_line" : 22, "end_line" : 23, "runner_ident" : "f155ccc8-f6a8-11eb-94c2-00e04cf8ff45", "event" : "runner_on_ok", "pid" : 1549383, "created" : "2021-08-06T11:25:10.333884", "parent_uuid" : "00e04cf8-ff45-b99d-863c-000000000182", "event_data" : { "playbook" : "ovirt-host-deploy.yml", "playbook_uuid" : "c1e368bf-e183-47c0-b6f5-377114c20eab", "play" : "all", "play_uuid" : "00e04cf8-ff45-b99d-863c-000000000007", "play_pattern" : "all", "task" : "Detect if host is a prebuilt image", "task_uuid" : "00e04cf8-ff45-b99d-863c-000000000182", "task_action" : "set_fact", "task_args" : "", "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-deploy-facts/tasks/host-os.yml:26", "role" : "ovirt-host-deploy-facts", "host" : "ovirth02.zyzyx.virtnet", "remote_addr" : "ovirth02.zyzyx.virtnet", "res" : { "changed" : false, "ansible_facts" : { "node_host" : true }, "_ansible_no_log" : false }, "start" : "2021-08-06T11:25:10.254355", "end" : "2021-08-06T11:25:10.333550", "duration" : 0.079195, "event_loop" : null, "uuid" : "a7b74be9-30ab-4ca4-b2b6-e7af6b86bb6c" } } } 2021-08-06 19:25:12 MYT - TASK [ovirt-host-deploy-facts : Reset configuration of advanced virtualization module] *** 2021-08-06 19:25:12 MYT - TASK [ovirt-host-deploy-facts : Find relevant advanced virtualization module version] *** 2021-08-06 19:25:12 MYT - TASK [ovirt-host-deploy-facts : Enable advanced virtualization module] ********* 2021-08-06 19:25:12 MYT - TASK [ovirt-host-deploy-facts : Ensure Python3 is installed for CentOS/RHEL8 hosts] *** Thank You.

2 1

fencing and (virtual) power button pression reaction
by Gianluca Cecchi 07 Aug '21

07 Aug '21

Hello, in RHCS we have the fencing concept for targets very similar to the oVirt ones: avoid data corruption and also fast react to problematic hosts situations. The implementation is quite similar to the oVirt one, with several fencing agents sometimes common, like fence_ipmilan. In RHCS documentation there is a chapter describing how to configure hosts so that they don't react to power button pressure. This guarantees that failover is as fast as possible, and also that the same host to be fenced, could create more damages if it reacts and begins the shutdown procedure instead of simply powering off. See: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/ht… It seems to me that oVirt / RHV lack this feature. Inside oVirt NGN and also RHVH-H, the /etc/systemd/logind.conf file is not configured with an entry of type HandlePowerKey=ignore So in some cases of tests I'm doing, I see in its virtual console that the to-be-fenced host begins its OS shutdown flow when it detects the power button pressure. Typically after 2-3 seconds the system powers off and then on again, but I also saw about 10 seconds delays in one case. I have opened a case (number 03002278) for my RHV products, but I would also like to get your comments here if I'm wrong with my considerations. Gianluca

2 1

Architecture design docs
by Jesse Hu 07 Aug '21

07 Aug '21

Hi there, is there any any architecture design docs for oVirt (Engine & Node & VDSM) for neebee to learn? https://www.ovirt.org/Architecture gives 404. Thanks.

4 5

Re: Install of RHV 4.4 failing - "Host is not up, please check logs, perhaps also on the engine machine"
by Brock Collins 07 Aug '21

07 Aug '21

Needed to add appropriate DNS entry / MAC for the VM It is a 192.168.2.122 typically* Sent from my iPhone

1 0