Cannot clone snapshots

Hi, I tried this script https://github.com/wefixit-AT/oVirtBackup which basically does: Snapshot a vm clone the snapshot delete the snapshot export the clone to export domain delete the clone It worked fine for 4 vms then exited with error: Message: VDSM kvm02 command failed: low level Image copy failed VDSM command failed: Image does not exist in domain: u'image=a651c514-a81b-4cfc-a9bc-0bc4caf51568, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' Now I'm not able to clone snapshots anymore, not even manually everytime I get the same error: This is engine.log when I snapshot the vm: 2016-01-26 14:58:32,984 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] There is no host with less than 8 running guests 2016-01-26 14:58:32,985 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 14:58:38,685 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (default task-1) [5b009390] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:58:39,007 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-12) [5b009390] Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER 2016-01-26 14:58:39,028 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:39,074 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageSizeInBytes='37580963840', volumeFormat='COW', newImageId='5d8f7880-9125-4c91-ba04-532901d1d805', newImageDescription='', imageInitialSizeInBytes='0', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', sourceImageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c'}), log id: 1113aa14 2016-01-26 14:58:39,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:40,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] FINISH, CreateSnapshotVDSCommand, return: 5d8f7880-9125-4c91-ba04-532901d1d805, log id: 1113aa14 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandMultiAsyncTasks::AttachTask: Attaching task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:40,382 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [217c647e] Adding task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:40,420 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:40,434 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0', newImageDescription='', imageInitialSizeInBytes='0', imageId='0856943c-c714-43a5-8f26-087143131d74', sourceImageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437'}), log id: 5b8489d6 2016-01-26 14:58:40,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:41,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] FINISH, CreateSnapshotVDSCommand, return: 88a9ab18-7d4c-42f0-9067-0241b64cb7d0, log id: 5b8489d6 2016-01-26 14:58:41,288 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] CommandMultiAsyncTasks::AttachTask: Attaching task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:41,425 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Adding task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:41,499 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:41,613 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93', newImageDescription='', imageInitialSizeInBytes='0', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', sourceImageGroupId='d98b8313-de2f-477a-94f8-78026497612b'}), log id: 7e1deb44 2016-01-26 14:58:41,620 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:43,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] FINISH, CreateSnapshotVDSCommand, return: dbbae52f-a408-41a4-8e6c-0c1919df5c93, log id: 7e1deb44 2016-01-26 14:58:43,852 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [a3e8512] CommandMultiAsyncTasks::AttachTask: Attaching task '9f0ac94f-deb9-4662-824b-f7144c869af0' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:43,871 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Adding task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:43,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' was initiated by rfocesq@omme.net@omme.net. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '010eec77-3b20-483c-bdd3-e9a8912f7ba5'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '9f0ac94f-deb9-4662-824b-f7144c869af0'. 2016-01-26 14:58:46,992 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 14:58:48,951 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,960 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,971 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 14:59:00,359 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:59:00,361 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:59:00,368 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endActionIfNecessary: All tasks of command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' has ended -> executing 'endAction' 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'): calling endAction '. 2016-01-26 14:59:00,390 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'CreateAllSnapshotsFromVm', executionIndex: '0' 2016-01-26 14:59:00,541 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Ending command 'org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand' successfully. 2016-01-26 14:59:00,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SnapshotVDSCommand(HostName = kvm01, SnapshotVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}), log id: 459fcd47 2016-01-26 14:59:05,465 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SnapshotVDSCommand, log id: 459fcd47 2016-01-26 14:59:05,483 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:05,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0'}), log id: 17bd1fc 2016-01-26 14:59:06,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@ce4f3490, log id: 17bd1fc 2016-01-26 14:59:06,607 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:06,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93'}), log id: 6e1489cb 2016-01-26 14:59:08,023 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@3294df55, log id: 6e1489cb 2016-01-26 14:59:08,077 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:08,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='5d8f7880-9125-4c91-ba04-532901d1d805'}), log id: 1dae9b25 2016-01-26 14:59:08,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@4bd18557, log id: 1dae9b25 2016-01-26 14:59:08,419 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:59:08,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' has been completed. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' completed, handling the result. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' succeeded, clearing tasks. 2016-01-26 14:59:08,510 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' 2016-01-26 14:59:08,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 14a34aa3 2016-01-26 14:59:08,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 3c0fbdc7 2016-01-26 14:59:08,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0fbdc7 2016-01-26 14:59:08,960 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 14a34aa3 2016-01-26 14:59:08,973 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' from DataBase 2016-01-26 14:59:08,974 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '9f0ac94f-deb9-4662-824b-f7144c869af0' 2016-01-26 14:59:08,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: b2a607c 2016-01-26 14:59:08,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: 3c0d79bc 2016-01-26 14:59:10,001 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0d79bc 2016-01-26 14:59:10,002 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: b2a607c 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '9f0ac94f-deb9-4662-824b-f7144c869af0' from DataBase 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' 2016-01-26 14:59:10,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 63c6b6df 2016-01-26 14:59:12,174 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 593cc157 2016-01-26 14:59:12,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] START, FullListVDSCommand(HostName = , FullListVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vds='Host[,f18a051d-b893-41f7-9e97-0037abc7180d]', vmIds='[4e683012-8895-43da-b3e1-6d6271f4d829]'}), log id: 3672e426 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 593cc157 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 63c6b6df 2016-01-26 14:59:12,585 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' from DataBase 2016-01-26 14:59:12,586 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:59:13,652 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel7.2.0, afterMigrationStatus=, tabletEnable=true, pid=32651, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, guestNumaNodes=[Ljava.lang.Object;@3c99c1b9, hypervEnable=true, numaTune={nodeset=0,1, mode=interleave}, custom={device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40device_af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47=VmDevice:{id='VmDeviceId:{deviceId='af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394=VmDevice:{id='VmDeviceId:{deviceId='88ae2b1a-bcd6-4252-abc8-5dd81e315394', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40=VmDevice:{id='VmDeviceId:{deviceId='9b763617-eb2b-44ec-b227-397ca7a14d40', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}}, vmType=kvm, memSize=2048, smpCoresPerSocket=1, vmName=hpserver, nice=0, status=Up, maxMemSize=20480, bootMenuEnable=false, guestDiskMapping={}, vmId=4e683012-8895-43da-b3e1-6d6271f4d829, displayIp=0, displayPort=5914, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, maxMemSlots=16, nicModel=rtl8139,pv, keyboardLayout=it, kvmEnable=true, devices=[Ljava.lang.Object;@214fa7db, timeOffset=14401, maxVCpus=16, clientIp=, statusTime=5328205390, display=vnc}], log id: 3672e426 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a memballoon Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {target=2097152, alias=balloon0, specParams={model=none}, deviceType=balloon, device=memballoon, type=balloon} 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a vnc Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {specParams={displayIp=0}, deviceType=graphics, deviceId=333c2dc5-837e-4fd3-b311-e0a7eb5ed45f, device=vnc, type=graphics, port=5914} 2016-01-26 14:59:13,659 ERROR [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] VM '4e683012-8895-43da-b3e1-6d6271f4d829' managed non pluggable device was removed unexpectedly from libvirt: 'VmDevice:{id='VmDeviceId:{deviceId='333c2dc5-837e-4fd3-b311-e0a7eb5ed45f', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='vnc', type='GRAPHICS', bootOrder='0', specParams='[]', address='', managed='true', plugged='false', readOnly='false', deviceAlias='', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}' And this when I clone the snapshot: 2016-01-26 15:00:14,932 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [30049735] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,511 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000002-0002-0002-0002-0000000000ca Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group CREATE_VM with role type USER 2016-01-26 15:00:15,607 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Locking VM(id = 'fa0f7dca-19d3-4f55-82c9-b81b235d913a') with compensation. 2016-01-26 15:00:15,610 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync='true', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', status='ImageLocked', exitStatus='Normal'}), log id: 6a218716 2016-01-26 15:00:15,614 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] FINISH, SetVmStatusVDSCommand, log id: 6a218716 2016-01-26 15:00:15,617 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,661 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [1a40e7c1] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:15,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='0856943c-c714-43a5-8f26-087143131d74', dstImageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='58da4129-eb0f-476e-b060-94cb2740ab8d', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 54b72148 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcImageGUID=dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcVolUUID=0856943c-c714-43a5-8f26-087143131d74 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstImageGUID=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstVolUUID=58da4129-eb0f-476e-b060-94cb2740ab8d 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ descr= 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] FINISH, CopyImageVDSCommand, return: bf3ef9aa-5099-47b6-b954-9a6902eb17c9, log id: 54b72148 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (default task-20) [1a40e7c1] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [1a40e7c1] CommandMultiAsyncTasks::AttachTask: Attaching task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:17,668 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [1a40e7c1] Adding task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:17,872 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [388cd805] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:17,928 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', dstImageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='31f56d0e-cf73-464d-ad9a-8a6b75c68563', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 3e52cd55 2016-01-26 15:00:17,930 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcImageGUID=d98b8313-de2f-477a-94f8-78026497612b 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcVolUUID=dd30f129-e8d0-4155-9c63-6ac1ee76fa8e 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstImageGUID=3f83427e-d3a9-4d0a-a176-aa713fbab255 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstVolUUID=31f56d0e-cf73-464d-ad9a-8a6b75c68563 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ descr= 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:18,975 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) [50bfaa54] Setting new tasks map. The map contains now 1 tasks 2016-01-26 15:00:19,707 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] FINISH, CopyImageVDSCommand, return: 8347d2c2-1907-431c-baeb-b885ba6f200d, log id: 3e52cd55 2016-01-26 15:00:19,729 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [388cd805] CommandMultiAsyncTasks::AttachTask: Attaching task '8347d2c2-1907-431c-baeb-b885ba6f200d' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:19,768 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [388cd805] Adding task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:19,886 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [66941705] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:19,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', dstImageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='136257df-c62b-421b-a12a-dea765b16721', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 70cc5e38 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcImageGUID=42e212c1-f669-43e1-8f90-20baf0c3f58c 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcVolUUID=449e8b88-ab34-4cf8-bc91-00128c35dc65 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstImageGUID=b06a9ad5-62ca-478e-9f0c-278feca4eda3 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstVolUUID=136257df-c62b-421b-a12a-dea765b16721 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ descr= 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:23,635 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] FINISH, CopyImageVDSCommand, return: b910d97d-a091-4648-8840-e870651745c7, log id: 70cc5e38 2016-01-26 15:00:23,654 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [66941705] CommandMultiAsyncTasks::AttachTask: Attaching task 'b910d97d-a091-4648-8840-e870651745c7' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:23,676 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [66941705] Adding task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:23,780 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-20) [2cab3adb] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: fa0f7dca-19d3-4f55-82c9-b81b235d913a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [2cab3adb] Correlation ID: 30049735, Job ID: fe7f7d93-4a22-4bfe-baae-bc2357dc01e9, Call Stack: null, Custom Event ID: -1, Message: VM test_clone creation was initiated by rfocesq@omme.net@omme.net. 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task '8347d2c2-1907-431c-baeb-b885ba6f200d'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'b910d97d-a091-4648-8840-e870651745c7'. 2016-01-26 15:00:30,393 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 15:00:31,835 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-95) [76980f69] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:31,846 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) [76980f69] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'cleaning'. 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:31,855 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-95) [76980f69] Task with DB Task ID 'a773241b-1473-4d42-97b3-bc077d2973fd' and VDSM Task ID '8347d2c2-1907-431c-baeb-b885ba6f200d' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 15:00:33,309 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] There is no host with less than 8 running guests 2016-01-26 15:00:33,310 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 15:00:42,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,888 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,905 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,916 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,916 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,930 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endActionIfNecessary: All tasks of command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' has ended -> executing 'endAction' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: 'aaac9162-4be3-473f-91f4-491c8eedb4ad'): calling endAction '. 2016-01-26 15:00:42,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [2c031c31] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'AddVmFromSnapshot', executionIndex: '0' 2016-01-26 15:00:43,014 ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-46) [30049735] Ending command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' with failure. 2016-01-26 15:00:43,036 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [388cd805] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:43,049 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:43,093 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'}), log id: 5be417ac 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] FINISH, DeleteImageGroupVDSCommand, log id: 5be417ac 2016-01-26 15:00:44,593 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Disk '3f83427e-d3a9-4d0a-a176-aa713fbab255' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:44,790 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Removed task '0797d4de-2df6-4283-8952-c9d186d2b010' from DataBase 2016-01-26 15:00:44,808 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [1a40e7c1] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:44,821 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:44,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'}), log id: 475a2763 2016-01-26 15:00:46,268 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] FINISH, DeleteImageGroupVDSCommand, log id: 475a2763 2016-01-26 15:00:46,270 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Disk '9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:46,476 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Removed task 'b4d1eaf2-6955-41cc-a813-39da5faeca41' from DataBase 2016-01-26 15:00:46,492 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [66941705] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:46,502 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:46,540 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'}), log id: 5ec9b85c 2016-01-26 15:00:47,792 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] FINISH, DeleteImageGroupVDSCommand, log id: 5ec9b85c 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Disk 'b06a9ad5-62ca-478e-9f0c-278feca4eda3' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:47,953 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Removed task 'fb467301-4397-49db-a200-d623916c8f37' from DataBase 2016-01-26 15:00:48,037 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [] Correlation ID: 30049735, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM test_clone creation. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' completed, handling the result. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' succeeded, clearing tasks. 2016-01-26 15:00:48,043 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '8347d2c2-1907-431c-baeb-b885ba6f200d' 2016-01-26 15:00:48,045 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 29d50ec8 2016-01-26 15:00:48,742 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 415f863f 2016-01-26 15:00:49,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 415f863f 2016-01-26 15:00:49,763 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 29d50ec8 2016-01-26 15:00:49,769 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task '8347d2c2-1907-431c-baeb-b885ba6f200d' from DataBase 2016-01-26 15:00:49,770 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' 2016-01-26 15:00:49,771 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 543d8586 2016-01-26 15:00:49,772 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 35bd0ad7 2016-01-26 15:00:50,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 35bd0ad7 2016-01-26 15:00:50,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 543d8586 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' from DataBase 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'b910d97d-a091-4648-8840-e870651745c7' 2016-01-26 15:00:50,802 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 1ded2669 2016-01-26 15:00:50,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 1ded2669 2016-01-26 15:00:51,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'b910d97d-a091-4648-8840-e870651745c7' from DataBase 2016-01-26 15:00:51,848 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'aaac9162-4be3-473f-91f4-491c8eedb4ad'

Hi Cristian, We already seen this before and have a bug for it: https://bugzilla.redhat.com/show_bug.cgi?id=1201268 Thanks, Raz Tamir On Jan 26, 2016 16:02, "Cristian Mammoli" <c.mammoli@apra.it> wrote:
Hi, I tried this script https://github.com/wefixit-AT/oVirtBackup which basically does: Snapshot a vm clone the snapshot delete the snapshot export the clone to export domain delete the clone
It worked fine for 4 vms then exited with error: Message: VDSM kvm02 command failed: low level Image copy failed VDSM command failed: Image does not exist in domain: u'image=a651c514-a81b-4cfc-a9bc-0bc4caf51568, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'
Now I'm not able to clone snapshots anymore, not even manually everytime I get the same error: This is engine.log when I snapshot the vm:
2016-01-26 14:58:32,984 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] There is no host with less than 8 running guests 2016-01-26 14:58:32,985 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 14:58:38,685 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (default task-1) [5b009390] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:58:39,007 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-12) [5b009390] Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER 2016-01-26 14:58:39,028 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:39,074 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageSizeInBytes='37580963840', volumeFormat='COW', newImageId='5d8f7880-9125-4c91-ba04-532901d1d805', newImageDescription='', imageInitialSizeInBytes='0', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', sourceImageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c'}), log id: 1113aa14 2016-01-26 14:58:39,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:40,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] FINISH, CreateSnapshotVDSCommand, return: 5d8f7880-9125-4c91-ba04-532901d1d805, log id: 1113aa14 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandMultiAsyncTasks::AttachTask: Attaching task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:40,382 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [217c647e] Adding task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:40,420 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:40,434 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0', newImageDescription='', imageInitialSizeInBytes='0', imageId='0856943c-c714-43a5-8f26-087143131d74', sourceImageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437'}), log id: 5b8489d6 2016-01-26 14:58:40,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:41,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] FINISH, CreateSnapshotVDSCommand, return: 88a9ab18-7d4c-42f0-9067-0241b64cb7d0, log id: 5b8489d6 2016-01-26 14:58:41,288 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] CommandMultiAsyncTasks::AttachTask: Attaching task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:41,425 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Adding task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:41,499 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:41,613 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93', newImageDescription='', imageInitialSizeInBytes='0', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', sourceImageGroupId='d98b8313-de2f-477a-94f8-78026497612b'}), log id: 7e1deb44 2016-01-26 14:58:41,620 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:43,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] FINISH, CreateSnapshotVDSCommand, return: dbbae52f-a408-41a4-8e6c-0c1919df5c93, log id: 7e1deb44 2016-01-26 14:58:43,852 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [a3e8512] CommandMultiAsyncTasks::AttachTask: Attaching task '9f0ac94f-deb9-4662-824b-f7144c869af0' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:43,871 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Adding task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:43,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' was initiated by rfocesq@omme.net@omme.net. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '010eec77-3b20-483c-bdd3-e9a8912f7ba5'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '9f0ac94f-deb9-4662-824b-f7144c869af0'. 2016-01-26 14:58:46,992 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 14:58:48,951 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,960 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,971 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 14:59:00,359 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:59:00,361 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:59:00,368 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endActionIfNecessary: All tasks of command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' has ended -> executing 'endAction' 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'): calling endAction '. 2016-01-26 14:59:00,390 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'CreateAllSnapshotsFromVm', executionIndex: '0' 2016-01-26 14:59:00,541 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Ending command 'org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand' successfully. 2016-01-26 14:59:00,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SnapshotVDSCommand(HostName = kvm01, SnapshotVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}), log id: 459fcd47 2016-01-26 14:59:05,465 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SnapshotVDSCommand, log id: 459fcd47 2016-01-26 14:59:05,483 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:05,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0'}), log id: 17bd1fc 2016-01-26 14:59:06,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@ce4f3490, log id: 17bd1fc 2016-01-26 14:59:06,607 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:06,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93'}), log id: 6e1489cb 2016-01-26 14:59:08,023 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@3294df55, log id: 6e1489cb 2016-01-26 14:59:08,077 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:08,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='5d8f7880-9125-4c91-ba04-532901d1d805'}), log id: 1dae9b25 2016-01-26 14:59:08,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@4bd18557, log id: 1dae9b25 2016-01-26 14:59:08,419 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:59:08,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' has been completed. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' completed, handling the result. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' succeeded, clearing tasks. 2016-01-26 14:59:08,510 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' 2016-01-26 14:59:08,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 14a34aa3 2016-01-26 14:59:08,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 3c0fbdc7 2016-01-26 14:59:08,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0fbdc7 2016-01-26 14:59:08,960 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 14a34aa3 2016-01-26 14:59:08,973 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' from DataBase 2016-01-26 14:59:08,974 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '9f0ac94f-deb9-4662-824b-f7144c869af0' 2016-01-26 14:59:08,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: b2a607c 2016-01-26 14:59:08,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: 3c0d79bc 2016-01-26 14:59:10,001 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0d79bc 2016-01-26 14:59:10,002 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: b2a607c 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '9f0ac94f-deb9-4662-824b-f7144c869af0' from DataBase 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' 2016-01-26 14:59:10,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 63c6b6df 2016-01-26 14:59:12,174 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 593cc157 2016-01-26 14:59:12,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] START, FullListVDSCommand(HostName = , FullListVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vds='Host[,f18a051d-b893-41f7-9e97-0037abc7180d]', vmIds='[4e683012-8895-43da-b3e1-6d6271f4d829]'}), log id: 3672e426 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 593cc157 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 63c6b6df 2016-01-26 14:59:12,585 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' from DataBase 2016-01-26 14:59:12,586 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:59:13,652 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel7.2.0, afterMigrationStatus=, tabletEnable=true, pid=32651, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, guestNumaNodes=[Ljava.lang.Object;@3c99c1b9, hypervEnable=true, numaTune={nodeset=0,1, mode=interleave}, custom={device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40device_af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47=VmDevice:{id='VmDeviceId:{deviceId='af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394=VmDevice:{id='VmDeviceId:{deviceId='88ae2b1a-bcd6-4252-abc8-5dd81e315394', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40=VmDevice:{id='VmDeviceId:{deviceId='9b763617-eb2b-44ec-b227-397ca7a14d40', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}}, vmType=kvm, memSize=2048, smpCoresPerSocket=1, vmName=hpserver, nice=0, status=Up, maxMemSize=20480, bootMenuEnable=false, guestDiskMapping={}, vmId=4e683012-8895-43da-b3e1-6d6271f4d829, displayIp=0, displayPort=5914, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, maxMemSlots=16, nicModel=rtl8139,pv, keyboardLayout=it, kvmEnable=true, devices=[Ljava.lang.Object;@214fa7db, timeOffset=14401, maxVCpus=16, clientIp=, statusTime=5328205390, display=vnc}], log id: 3672e426 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a memballoon Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {target=2097152, alias=balloon0, specParams={model=none}, deviceType=balloon, device=memballoon, type=balloon} 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a vnc Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {specParams={displayIp=0}, deviceType=graphics, deviceId=333c2dc5-837e-4fd3-b311-e0a7eb5ed45f, device=vnc, type=graphics, port=5914} 2016-01-26 14:59:13,659 ERROR [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] VM '4e683012-8895-43da-b3e1-6d6271f4d829' managed non pluggable device was removed unexpectedly from libvirt: 'VmDevice:{id='VmDeviceId:{deviceId='333c2dc5-837e-4fd3-b311-e0a7eb5ed45f', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='vnc', type='GRAPHICS', bootOrder='0', specParams='[]', address='', managed='true', plugged='false', readOnly='false', deviceAlias='', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}'
And this when I clone the snapshot: 2016-01-26 15:00:14,932 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [30049735] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,511 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000002-0002-0002-0002-0000000000ca Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group CREATE_VM with role type USER 2016-01-26 15:00:15,607 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Locking VM(id = 'fa0f7dca-19d3-4f55-82c9-b81b235d913a') with compensation. 2016-01-26 15:00:15,610 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync='true', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', status='ImageLocked', exitStatus='Normal'}), log id: 6a218716 2016-01-26 15:00:15,614 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] FINISH, SetVmStatusVDSCommand, log id: 6a218716 2016-01-26 15:00:15,617 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,661 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [1a40e7c1] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:15,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='0856943c-c714-43a5-8f26-087143131d74', dstImageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='58da4129-eb0f-476e-b060-94cb2740ab8d', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 54b72148 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcImageGUID=dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcVolUUID=0856943c-c714-43a5-8f26-087143131d74 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstImageGUID=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstVolUUID=58da4129-eb0f-476e-b060-94cb2740ab8d 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ descr= 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] FINISH, CopyImageVDSCommand, return: bf3ef9aa-5099-47b6-b954-9a6902eb17c9, log id: 54b72148 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (default task-20) [1a40e7c1] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [1a40e7c1] CommandMultiAsyncTasks::AttachTask: Attaching task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:17,668 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [1a40e7c1] Adding task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:17,872 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [388cd805] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:17,928 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', dstImageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='31f56d0e-cf73-464d-ad9a-8a6b75c68563', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 3e52cd55 2016-01-26 15:00:17,930 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcImageGUID=d98b8313-de2f-477a-94f8-78026497612b 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcVolUUID=dd30f129-e8d0-4155-9c63-6ac1ee76fa8e 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstImageGUID=3f83427e-d3a9-4d0a-a176-aa713fbab255 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstVolUUID=31f56d0e-cf73-464d-ad9a-8a6b75c68563 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ descr= 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:18,975 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) [50bfaa54] Setting new tasks map. The map contains now 1 tasks 2016-01-26 15:00:19,707 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] FINISH, CopyImageVDSCommand, return: 8347d2c2-1907-431c-baeb-b885ba6f200d, log id: 3e52cd55 2016-01-26 15:00:19,729 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [388cd805] CommandMultiAsyncTasks::AttachTask: Attaching task '8347d2c2-1907-431c-baeb-b885ba6f200d' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:19,768 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [388cd805] Adding task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:19,886 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [66941705] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:19,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', dstImageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='136257df-c62b-421b-a12a-dea765b16721', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 70cc5e38 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcImageGUID=42e212c1-f669-43e1-8f90-20baf0c3f58c 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcVolUUID=449e8b88-ab34-4cf8-bc91-00128c35dc65 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstImageGUID=b06a9ad5-62ca-478e-9f0c-278feca4eda3 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstVolUUID=136257df-c62b-421b-a12a-dea765b16721 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ descr= 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:23,635 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] FINISH, CopyImageVDSCommand, return: b910d97d-a091-4648-8840-e870651745c7, log id: 70cc5e38 2016-01-26 15:00:23,654 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [66941705] CommandMultiAsyncTasks::AttachTask: Attaching task 'b910d97d-a091-4648-8840-e870651745c7' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:23,676 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [66941705] Adding task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:23,780 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-20) [2cab3adb] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: fa0f7dca-19d3-4f55-82c9-b81b235d913a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [2cab3adb] Correlation ID: 30049735, Job ID: fe7f7d93-4a22-4bfe-baae-bc2357dc01e9, Call Stack: null, Custom Event ID: -1, Message: VM test_clone creation was initiated by rfocesq@omme.net@ omme.net. 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task '8347d2c2-1907-431c-baeb-b885ba6f200d'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'b910d97d-a091-4648-8840-e870651745c7'. 2016-01-26 15:00:30,393 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 15:00:31,835 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-95) [76980f69] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:31,846 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) [76980f69] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'cleaning'. 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:31,855 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-95) [76980f69] Task with DB Task ID 'a773241b-1473-4d42-97b3-bc077d2973fd' and VDSM Task ID '8347d2c2-1907-431c-baeb-b885ba6f200d' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 15:00:33,309 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] There is no host with less than 8 running guests 2016-01-26 15:00:33,310 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 15:00:42,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,888 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,905 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,916 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,916 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,930 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endActionIfNecessary: All tasks of command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' has ended -> executing 'endAction' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: 'aaac9162-4be3-473f-91f4-491c8eedb4ad'): calling endAction '. 2016-01-26 15:00:42,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [2c031c31] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'AddVmFromSnapshot', executionIndex: '0' 2016-01-26 15:00:43,014 ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-46) [30049735] Ending command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' with failure. 2016-01-26 15:00:43,036 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [388cd805] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:43,049 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:43,093 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'}), log id: 5be417ac 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] FINISH, DeleteImageGroupVDSCommand, log id: 5be417ac 2016-01-26 15:00:44,593 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Disk '3f83427e-d3a9-4d0a-a176-aa713fbab255' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:44,790 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Removed task '0797d4de-2df6-4283-8952-c9d186d2b010' from DataBase 2016-01-26 15:00:44,808 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [1a40e7c1] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:44,821 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:44,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'}), log id: 475a2763 2016-01-26 15:00:46,268 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] FINISH, DeleteImageGroupVDSCommand, log id: 475a2763 2016-01-26 15:00:46,270 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Disk '9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:46,476 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Removed task 'b4d1eaf2-6955-41cc-a813-39da5faeca41' from DataBase 2016-01-26 15:00:46,492 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [66941705] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:46,502 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:46,540 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'}), log id: 5ec9b85c 2016-01-26 15:00:47,792 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] FINISH, DeleteImageGroupVDSCommand, log id: 5ec9b85c 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Disk 'b06a9ad5-62ca-478e-9f0c-278feca4eda3' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:47,953 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Removed task 'fb467301-4397-49db-a200-d623916c8f37' from DataBase 2016-01-26 15:00:48,037 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [] Correlation ID: 30049735, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM test_clone creation. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' completed, handling the result. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' succeeded, clearing tasks. 2016-01-26 15:00:48,043 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '8347d2c2-1907-431c-baeb-b885ba6f200d' 2016-01-26 15:00:48,045 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 29d50ec8 2016-01-26 15:00:48,742 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 415f863f 2016-01-26 15:00:49,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 415f863f 2016-01-26 15:00:49,763 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 29d50ec8 2016-01-26 15:00:49,769 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task '8347d2c2-1907-431c-baeb-b885ba6f200d' from DataBase 2016-01-26 15:00:49,770 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' 2016-01-26 15:00:49,771 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 543d8586 2016-01-26 15:00:49,772 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 35bd0ad7 2016-01-26 15:00:50,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 35bd0ad7 2016-01-26 15:00:50,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 543d8586 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' from DataBase 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'b910d97d-a091-4648-8840-e870651745c7' 2016-01-26 15:00:50,802 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 1ded2669 2016-01-26 15:00:50,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 1ded2669 2016-01-26 15:00:51,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'b910d97d-a091-4648-8840-e870651745c7' from DataBase 2016-01-26 15:00:51,848 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'aaac9162-4be3-473f-91f4-491c8eedb4ad'
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users

This is a multi-part message in MIME format. --------------090400010105060408010501 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit I read the but but in my case it's not sporadic, it happens every time with all the vms Il 26/01/2016 22:01, Raz Tamir ha scritto:
Hi Cristian, We already seen this before and have a bug for it: https://bugzilla.redhat.com/show_bug.cgi?id=1201268
Thanks, Raz Tamir
On Jan 26, 2016 16:02, "Cristian Mammoli" <c.mammoli@apra.it <mailto:c.mammoli@apra.it>> wrote:
Hi, I tried this script https://github.com/wefixit-AT/oVirtBackup which basically does: Snapshot a vm clone the snapshot delete the snapshot export the clone to export domain delete the clone
It worked fine for 4 vms then exited with error: Message: VDSM kvm02 command failed: low level Image copy failed VDSM command failed: Image does not exist in domain: u'image=a651c514-a81b-4cfc-a9bc-0bc4caf51568, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'
Now I'm not able to clone snapshots anymore, not even manually everytime I get the same error: This is engine.log when I snapshot the vm:
2016-01-26 14:58:32,984 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] There is no host with less than 8 running guests 2016-01-26 14:58:32,985 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 14:58:38,685 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (default task-1) [5b009390] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:58:39,007 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-12) [5b009390] Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER 2016-01-26 14:58:39,028 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:39,074 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageSizeInBytes='37580963840', volumeFormat='COW', newImageId='5d8f7880-9125-4c91-ba04-532901d1d805', newImageDescription='', imageInitialSizeInBytes='0', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', sourceImageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c'}), log id: 1113aa14 2016-01-26 14:58:39,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:40,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] FINISH, CreateSnapshotVDSCommand, return: 5d8f7880-9125-4c91-ba04-532901d1d805, log id: 1113aa14 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandMultiAsyncTasks::AttachTask: Attaching task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:40,382 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [217c647e] Adding task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:40,420 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:40,434 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0', newImageDescription='', imageInitialSizeInBytes='0', imageId='0856943c-c714-43a5-8f26-087143131d74', sourceImageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437'}), log id: 5b8489d6 2016-01-26 14:58:40,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:41,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] FINISH, CreateSnapshotVDSCommand, return: 88a9ab18-7d4c-42f0-9067-0241b64cb7d0, log id: 5b8489d6 2016-01-26 14:58:41,288 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] CommandMultiAsyncTasks::AttachTask: Attaching task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:41,425 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Adding task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:41,499 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:41,613 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93', newImageDescription='', imageInitialSizeInBytes='0', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', sourceImageGroupId='d98b8313-de2f-477a-94f8-78026497612b'}), log id: 7e1deb44 2016-01-26 14:58:41,620 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:43,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] FINISH, CreateSnapshotVDSCommand, return: dbbae52f-a408-41a4-8e6c-0c1919df5c93, log id: 7e1deb44 2016-01-26 14:58:43,852 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [a3e8512] CommandMultiAsyncTasks::AttachTask: Attaching task '9f0ac94f-deb9-4662-824b-f7144c869af0' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:43,871 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Adding task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:43,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' was initiated by rfocesq@omme.net@omme.net <http://omme.net>. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '010eec77-3b20-483c-bdd3-e9a8912f7ba5'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '9f0ac94f-deb9-4662-824b-f7144c869af0'. 2016-01-26 14:58:46,992 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 14:58:48,951 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,960 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,971 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 14:59:00,359 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:59:00,361 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:59:00,368 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endActionIfNecessary: All tasks of command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' has ended -> executing 'endAction' 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'): calling endAction '. 2016-01-26 14:59:00,390 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'CreateAllSnapshotsFromVm', executionIndex: '0' 2016-01-26 14:59:00,541 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Ending command 'org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand' successfully. 2016-01-26 14:59:00,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SnapshotVDSCommand(HostName = kvm01, SnapshotVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}), log id: 459fcd47 2016-01-26 14:59:05,465 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SnapshotVDSCommand, log id: 459fcd47 2016-01-26 14:59:05,483 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:05,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0'}), log id: 17bd1fc 2016-01-26 14:59:06,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@ce4f3490, log id: 17bd1fc 2016-01-26 14:59:06,607 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:06,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93'}), log id: 6e1489cb 2016-01-26 14:59:08,023 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@3294df55, log id: 6e1489cb 2016-01-26 14:59:08,077 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:08,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='5d8f7880-9125-4c91-ba04-532901d1d805'}), log id: 1dae9b25 2016-01-26 14:59:08,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@4bd18557, log id: 1dae9b25 2016-01-26 14:59:08,419 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:59:08,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' has been completed. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' completed, handling the result. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' succeeded, clearing tasks. 2016-01-26 14:59:08,510 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' 2016-01-26 14:59:08,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 14a34aa3 2016-01-26 14:59:08,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 3c0fbdc7 2016-01-26 14:59:08,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0fbdc7 2016-01-26 14:59:08,960 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 14a34aa3 2016-01-26 14:59:08,973 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' from DataBase 2016-01-26 14:59:08,974 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '9f0ac94f-deb9-4662-824b-f7144c869af0' 2016-01-26 14:59:08,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: b2a607c 2016-01-26 14:59:08,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: 3c0d79bc 2016-01-26 14:59:10,001 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0d79bc 2016-01-26 14:59:10,002 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: b2a607c 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '9f0ac94f-deb9-4662-824b-f7144c869af0' from DataBase 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' 2016-01-26 14:59:10,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 63c6b6df 2016-01-26 14:59:12,174 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 593cc157 2016-01-26 14:59:12,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] START, FullListVDSCommand(HostName = , FullListVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vds='Host[,f18a051d-b893-41f7-9e97-0037abc7180d]', vmIds='[4e683012-8895-43da-b3e1-6d6271f4d829]'}), log id: 3672e426 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 593cc157 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 63c6b6df 2016-01-26 14:59:12,585 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' from DataBase 2016-01-26 14:59:12,586 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:59:13,652 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel7.2.0, afterMigrationStatus=, tabletEnable=true, pid=32651, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, guestNumaNodes=[Ljava.lang.Object;@3c99c1b9, hypervEnable=true, numaTune={nodeset=0,1, mode=interleave}, custom={device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40device_af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47=VmDevice:{id='VmDeviceId:{deviceId='af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394=VmDevice:{id='VmDeviceId:{deviceId='88ae2b1a-bcd6-4252-abc8-5dd81e315394', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40=VmDevice:{id='VmDeviceId:{deviceId='9b763617-eb2b-44ec-b227-397ca7a14d40', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}}, vmType=kvm, memSize=2048, smpCoresPerSocket=1, vmName=hpserver, nice=0, status=Up, maxMemSize=20480, bootMenuEnable=false, guestDiskMapping={}, vmId=4e683012-8895-43da-b3e1-6d6271f4d829, displayIp=0, displayPort=5914, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, maxMemSlots=16, nicModel=rtl8139,pv, keyboardLayout=it, kvmEnable=true, devices=[Ljava.lang.Object;@214fa7db, timeOffset=14401, maxVCpus=16, clientIp=, statusTime=5328205390, display=vnc}], log id: 3672e426 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a memballoon Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {target=2097152, alias=balloon0, specParams={model=none}, deviceType=balloon, device=memballoon, type=balloon} 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a vnc Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {specParams={displayIp=0}, deviceType=graphics, deviceId=333c2dc5-837e-4fd3-b311-e0a7eb5ed45f, device=vnc, type=graphics, port=5914} 2016-01-26 14:59:13,659 ERROR [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] VM '4e683012-8895-43da-b3e1-6d6271f4d829' managed non pluggable device was removed unexpectedly from libvirt: 'VmDevice:{id='VmDeviceId:{deviceId='333c2dc5-837e-4fd3-b311-e0a7eb5ed45f', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='vnc', type='GRAPHICS', bootOrder='0', specParams='[]', address='', managed='true', plugged='false', readOnly='false', deviceAlias='', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}'
And this when I clone the snapshot: 2016-01-26 15:00:14,932 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [30049735] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,511 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000002-0002-0002-0002-0000000000ca Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group CREATE_VM with role type USER 2016-01-26 15:00:15,607 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Locking VM(id = 'fa0f7dca-19d3-4f55-82c9-b81b235d913a') with compensation. 2016-01-26 15:00:15,610 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync='true', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', status='ImageLocked', exitStatus='Normal'}), log id: 6a218716 2016-01-26 15:00:15,614 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] FINISH, SetVmStatusVDSCommand, log id: 6a218716 2016-01-26 15:00:15,617 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,661 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [1a40e7c1] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:15,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='0856943c-c714-43a5-8f26-087143131d74', dstImageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='58da4129-eb0f-476e-b060-94cb2740ab8d', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 54b72148 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcImageGUID=dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcVolUUID=0856943c-c714-43a5-8f26-087143131d74 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstImageGUID=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstVolUUID=58da4129-eb0f-476e-b060-94cb2740ab8d 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ descr= 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] FINISH, CopyImageVDSCommand, return: bf3ef9aa-5099-47b6-b954-9a6902eb17c9, log id: 54b72148 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (default task-20) [1a40e7c1] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [1a40e7c1] CommandMultiAsyncTasks::AttachTask: Attaching task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:17,668 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [1a40e7c1] Adding task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:17,872 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [388cd805] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:17,928 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', dstImageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='31f56d0e-cf73-464d-ad9a-8a6b75c68563', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 3e52cd55 2016-01-26 15:00:17,930 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcImageGUID=d98b8313-de2f-477a-94f8-78026497612b 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcVolUUID=dd30f129-e8d0-4155-9c63-6ac1ee76fa8e 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstImageGUID=3f83427e-d3a9-4d0a-a176-aa713fbab255 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstVolUUID=31f56d0e-cf73-464d-ad9a-8a6b75c68563 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ descr= 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:18,975 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) [50bfaa54] Setting new tasks map. The map contains now 1 tasks 2016-01-26 15:00:19,707 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] FINISH, CopyImageVDSCommand, return: 8347d2c2-1907-431c-baeb-b885ba6f200d, log id: 3e52cd55 2016-01-26 15:00:19,729 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [388cd805] CommandMultiAsyncTasks::AttachTask: Attaching task '8347d2c2-1907-431c-baeb-b885ba6f200d' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:19,768 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [388cd805] Adding task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:19,886 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [66941705] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:19,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', dstImageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='136257df-c62b-421b-a12a-dea765b16721', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 70cc5e38 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcImageGUID=42e212c1-f669-43e1-8f90-20baf0c3f58c 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcVolUUID=449e8b88-ab34-4cf8-bc91-00128c35dc65 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstImageGUID=b06a9ad5-62ca-478e-9f0c-278feca4eda3 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstVolUUID=136257df-c62b-421b-a12a-dea765b16721 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ descr= 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:23,635 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] FINISH, CopyImageVDSCommand, return: b910d97d-a091-4648-8840-e870651745c7, log id: 70cc5e38 2016-01-26 15:00:23,654 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [66941705] CommandMultiAsyncTasks::AttachTask: Attaching task 'b910d97d-a091-4648-8840-e870651745c7' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:23,676 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [66941705] Adding task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:23,780 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-20) [2cab3adb] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: fa0f7dca-19d3-4f55-82c9-b81b235d913a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [2cab3adb] Correlation ID: 30049735, Job ID: fe7f7d93-4a22-4bfe-baae-bc2357dc01e9, Call Stack: null, Custom Event ID: -1, Message: VM test_clone creation was initiated by rfocesq@omme.net@omme.net <http://omme.net>. 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task '8347d2c2-1907-431c-baeb-b885ba6f200d'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'b910d97d-a091-4648-8840-e870651745c7'. 2016-01-26 15:00:30,393 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 15:00:31,835 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-95) [76980f69] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:31,846 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) [76980f69] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'cleaning'. 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:31,855 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-95) [76980f69] Task with DB Task ID 'a773241b-1473-4d42-97b3-bc077d2973fd' and VDSM Task ID '8347d2c2-1907-431c-baeb-b885ba6f200d' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 15:00:33,309 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] There is no host with less than 8 running guests 2016-01-26 15:00:33,310 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 15:00:42,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,888 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,905 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,916 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,916 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,930 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endActionIfNecessary: All tasks of command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' has ended -> executing 'endAction' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: 'aaac9162-4be3-473f-91f4-491c8eedb4ad'): calling endAction '. 2016-01-26 15:00:42,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [2c031c31] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'AddVmFromSnapshot', executionIndex: '0' 2016-01-26 15:00:43,014 ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-46) [30049735] Ending command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' with failure. 2016-01-26 15:00:43,036 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [388cd805] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:43,049 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:43,093 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'}), log id: 5be417ac 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] FINISH, DeleteImageGroupVDSCommand, log id: 5be417ac 2016-01-26 15:00:44,593 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Disk '3f83427e-d3a9-4d0a-a176-aa713fbab255' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:44,790 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Removed task '0797d4de-2df6-4283-8952-c9d186d2b010' from DataBase 2016-01-26 15:00:44,808 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [1a40e7c1] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:44,821 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:44,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'}), log id: 475a2763 2016-01-26 15:00:46,268 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] FINISH, DeleteImageGroupVDSCommand, log id: 475a2763 2016-01-26 15:00:46,270 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Disk '9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:46,476 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Removed task 'b4d1eaf2-6955-41cc-a813-39da5faeca41' from DataBase 2016-01-26 15:00:46,492 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [66941705] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:46,502 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:46,540 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'}), log id: 5ec9b85c 2016-01-26 15:00:47,792 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] FINISH, DeleteImageGroupVDSCommand, log id: 5ec9b85c 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Disk 'b06a9ad5-62ca-478e-9f0c-278feca4eda3' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:47,953 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Removed task 'fb467301-4397-49db-a200-d623916c8f37' from DataBase 2016-01-26 15:00:48,037 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [] Correlation ID: 30049735, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM test_clone creation. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' completed, handling the result. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' succeeded, clearing tasks. 2016-01-26 15:00:48,043 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '8347d2c2-1907-431c-baeb-b885ba6f200d' 2016-01-26 15:00:48,045 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 29d50ec8 2016-01-26 15:00:48,742 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 415f863f 2016-01-26 15:00:49,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 415f863f 2016-01-26 15:00:49,763 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 29d50ec8 2016-01-26 15:00:49,769 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task '8347d2c2-1907-431c-baeb-b885ba6f200d' from DataBase 2016-01-26 15:00:49,770 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' 2016-01-26 15:00:49,771 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 543d8586 2016-01-26 15:00:49,772 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 35bd0ad7 2016-01-26 15:00:50,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 35bd0ad7 2016-01-26 15:00:50,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 543d8586 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' from DataBase 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'b910d97d-a091-4648-8840-e870651745c7' 2016-01-26 15:00:50,802 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 1ded2669 2016-01-26 15:00:50,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 1ded2669 2016-01-26 15:00:51,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'b910d97d-a091-4648-8840-e870651745c7' from DataBase 2016-01-26 15:00:51,848 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'aaac9162-4be3-473f-91f4-491c8eedb4ad'
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
-- Mammoli Cristian System administrator T. +39 0731 22911 Via Brodolini 6 | 60035 Jesi (an) --------------090400010105060408010501 Content-Type: multipart/related; boundary="------------040001080806090303030900" --------------040001080806090303030900 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> I read the but but in my case it's not sporadic, it happens every time with all the vms<br> <br> <div class="moz-cite-prefix">Il 26/01/2016 22:01, Raz Tamir ha scritto:<br> </div> <blockquote cite="mid:CAJ9Ngn9uU2-2ZU0u9w1yKGJVVc3h7=OWSXx8BgouJbR3z+dCSg@mail.gmail.com" type="cite"> <p dir="ltr">Hi Cristian,<br> We already seen this before and have a bug for it:<br> <a moz-do-not-send="true" href="https://bugzilla.redhat.com/show_bug.cgi?id=1201268">https://bugzilla.redhat.com/show_bug.cgi?id=1201268</a><br> </p> <p dir="ltr">Thanks,<br> Raz Tamir</p> <div class="gmail_quote">On Jan 26, 2016 16:02, "Cristian Mammoli" <<a moz-do-not-send="true" href="mailto:c.mammoli@apra.it">c.mammoli@apra.it</a>> wrote:<br type="attribution"> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi, I tried this script <a moz-do-not-send="true" href="https://github.com/wefixit-AT/oVirtBackup" rel="noreferrer" target="_blank">https://github.com/wefixit-AT/oVirtBackup</a><br> which basically does:<br> Snapshot a vm<br> clone the snapshot<br> delete the snapshot<br> export the clone to export domain<br> delete the clone<br> <br> It worked fine for 4 vms then exited with error:<br> Message: VDSM kvm02 command failed: low level Image copy failed<br> VDSM command failed: Image does not exist in domain: u'image=a651c514-a81b-4cfc-a9bc-0bc4caf51568, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'<br> <br> Now I'm not able to clone snapshots anymore, not even manually everytime I get the same error:<br> This is engine.log when I snapshot the vm:<br> <br> 2016-01-26 14:58:32,984 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] There is no host with less than 8 running guests<br> 2016-01-26 14:58:32,985 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method<br> 2016-01-26 14:58:38,685 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (default task-1) [5b009390] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}'<br> 2016-01-26 14:58:39,007 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-12) [5b009390] Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER<br> 2016-01-26 14:58:39,028 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage<br> 2016-01-26 14:58:39,074 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageSizeInBytes='37580963840', volumeFormat='COW', newImageId='5d8f7880-9125-4c91-ba04-532901d1d805', newImageDescription='', imageInitialSizeInBytes='0', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', sourceImageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c'}), log id: 1113aa14<br> 2016-01-26 14:58:39,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID<br> 2016-01-26 14:58:40,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] FINISH, CreateSnapshotVDSCommand, return: 5d8f7880-9125-4c91-ba04-532901d1d805, log id: 1113aa14<br> 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'<br> 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandMultiAsyncTasks::AttachTask: Attaching task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'.<br> 2016-01-26 14:58:40,382 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [217c647e] Adding task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet..<br> 2016-01-26 14:58:40,420 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage<br> 2016-01-26 14:58:40,434 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0', newImageDescription='', imageInitialSizeInBytes='0', imageId='0856943c-c714-43a5-8f26-087143131d74', sourceImageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437'}), log id: 5b8489d6<br> 2016-01-26 14:58:40,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID<br> 2016-01-26 14:58:41,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] FINISH, CreateSnapshotVDSCommand, return: 88a9ab18-7d4c-42f0-9067-0241b64cb7d0, log id: 5b8489d6<br> 2016-01-26 14:58:41,288 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] CommandMultiAsyncTasks::AttachTask: Attaching task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'.<br> 2016-01-26 14:58:41,425 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Adding task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet..<br> 2016-01-26 14:58:41,499 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage<br> 2016-01-26 14:58:41,613 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93', newImageDescription='', imageInitialSizeInBytes='0', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', sourceImageGroupId='d98b8313-de2f-477a-94f8-78026497612b'}), log id: 7e1deb44<br> 2016-01-26 14:58:41,620 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID<br> 2016-01-26 14:58:43,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] FINISH, CreateSnapshotVDSCommand, return: dbbae52f-a408-41a4-8e6c-0c1919df5c93, log id: 7e1deb44<br> 2016-01-26 14:58:43,852 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [a3e8512] CommandMultiAsyncTasks::AttachTask: Attaching task '9f0ac94f-deb9-4662-824b-f7144c869af0' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'.<br> 2016-01-26 14:58:43,871 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Adding task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet..<br> 2016-01-26 14:58:43,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' was initiated by <a class="moz-txt-link-abbreviated" href="mailto:rfocesq@omme.net@">rfocesq@omme.net@</a><a moz-do-not-send="true" href="http://omme.net" rel="noreferrer" target="_blank">omme.net</a>.<br> 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'.<br> 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '010eec77-3b20-483c-bdd3-e9a8912f7ba5'.<br> 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '9f0ac94f-deb9-4662-824b-f7144c869af0'.<br> 2016-01-26 14:58:46,992 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now<br> 2016-01-26 14:58:48,951 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'.<br> 2016-01-26 14:58:48,960 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully.<br> 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed.<br> 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'.<br> 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'.<br> 2016-01-26 14:58:48,971 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully.<br> 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed.<br> 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Finished polling Tasks, will poll again in 10 seconds.<br> 2016-01-26 14:59:00,359 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully.<br> 2016-01-26 14:59:00,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed.<br> 2016-01-26 14:59:00,361 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'.<br> 2016-01-26 14:59:00,368 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully.<br> 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endActionIfNecessary: All tasks of command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' has ended -> executing 'endAction'<br> 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'): calling endAction '.<br> 2016-01-26 14:59:00,390 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'CreateAllSnapshotsFromVm', executionIndex: '0'<br> 2016-01-26 14:59:00,541 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Ending command 'org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand' successfully.<br> 2016-01-26 14:59:00,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SnapshotVDSCommand(HostName = kvm01, SnapshotVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}), log id: 459fcd47<br> 2016-01-26 14:59:05,465 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SnapshotVDSCommand, log id: 459fcd47<br> 2016-01-26 14:59:05,483 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully.<br> 2016-01-26 14:59:05,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0'}), log id: 17bd1fc<br> 2016-01-26 14:59:06,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@ce4f3490, log id: 17bd1fc<br> 2016-01-26 14:59:06,607 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully.<br> 2016-01-26 14:59:06,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93'}), log id: 6e1489cb<br> 2016-01-26 14:59:08,023 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@3294df55, log id: 6e1489cb<br> 2016-01-26 14:59:08,077 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully.<br> 2016-01-26 14:59:08,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='5d8f7880-9125-4c91-ba04-532901d1d805'}), log id: 1dae9b25<br> 2016-01-26 14:59:08,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@4bd18557, log id: 1dae9b25<br> 2016-01-26 14:59:08,419 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}'<br> 2016-01-26 14:59:08,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' has been completed.<br> 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' completed, handling the result.<br> 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' succeeded, clearing tasks.<br> 2016-01-26 14:59:08,510 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '010eec77-3b20-483c-bdd3-e9a8912f7ba5'<br> 2016-01-26 14:59:08,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 14a34aa3<br> 2016-01-26 14:59:08,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 3c0fbdc7<br> 2016-01-26 14:59:08,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0fbdc7<br> 2016-01-26 14:59:08,960 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 14a34aa3<br> 2016-01-26 14:59:08,973 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' from DataBase<br> 2016-01-26 14:59:08,974 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '9f0ac94f-deb9-4662-824b-f7144c869af0'<br> 2016-01-26 14:59:08,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: b2a607c<br> 2016-01-26 14:59:08,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: 3c0d79bc<br> 2016-01-26 14:59:10,001 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0d79bc<br> 2016-01-26 14:59:10,002 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: b2a607c<br> 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '9f0ac94f-deb9-4662-824b-f7144c869af0' from DataBase<br> 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'<br> 2016-01-26 14:59:10,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 63c6b6df<br> 2016-01-26 14:59:12,174 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 593cc157<br> 2016-01-26 14:59:12,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] START, FullListVDSCommand(HostName = , FullListVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vds='Host[,f18a051d-b893-41f7-9e97-0037abc7180d]', vmIds='[4e683012-8895-43da-b3e1-6d6271f4d829]'}), log id: 3672e426<br> 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 593cc157<br> 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 63c6b6df<br> 2016-01-26 14:59:12,585 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' from DataBase<br> 2016-01-26 14:59:12,586 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'<br> 2016-01-26 14:59:13,652 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel7.2.0, afterMigrationStatus=, tabletEnable=true, pid=32651, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, guestNumaNodes=[Ljava.lang.Object;@3c99c1b9, hypervEnable=true, numaTune={nodeset=0,1, mode=interleave}, custom={device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40device_af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47=VmDevice:{id='VmDeviceId:{deviceId='af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394=VmDevice:{id='VmDeviceId:{deviceId='88ae2b1a-bcd6-4252-abc8-5dd81e315394', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40=VmDevice:{id='VmDeviceId:{deviceId='9b763617-eb2b-44ec-b227-397ca7a14d40', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}}, vmType=kvm, memSize=2048, smpCoresPerSocket=1, vmName=hpserver, nice=0, status=Up, maxMemSize=20480, bootMenuEnable=false, guestDiskMapping={}, vmId=4e683012-8895-43da-b3e1-6d6271f4d829, displayIp=0, displayPort=5914, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, maxMemSlots=16, nicModel=rtl8139,pv, keyboardLayout=it, kvmEnable=true, devices=[Ljava.lang.Object;@214fa7db, timeOffset=14401, maxVCpus=16, clientIp=, statusTime=5328205390, display=vnc}], log id: 3672e426<br> 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a memballoon Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {target=2097152, alias=balloon0, specParams={model=none}, deviceType=balloon, device=memballoon, type=balloon}<br> 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a vnc Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {specParams={displayIp=0}, deviceType=graphics, deviceId=333c2dc5-837e-4fd3-b311-e0a7eb5ed45f, device=vnc, type=graphics, port=5914}<br> 2016-01-26 14:59:13,659 ERROR [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] VM '4e683012-8895-43da-b3e1-6d6271f4d829' managed non pluggable device was removed unexpectedly from libvirt: 'VmDevice:{id='VmDeviceId:{deviceId='333c2dc5-837e-4fd3-b311-e0a7eb5ed45f', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='vnc', type='GRAPHICS', bootOrder='0', specParams='[]', address='', managed='true', plugged='false', readOnly='false', deviceAlias='', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}'<br> <br> <br> And this when I clone the snapshot:<br> 2016-01-26 15:00:14,932 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [30049735] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}'<br> 2016-01-26 15:00:15,511 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000002-0002-0002-0002-0000000000ca Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group CREATE_VM with role type USER<br> 2016-01-26 15:00:15,607 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Locking VM(id = 'fa0f7dca-19d3-4f55-82c9-b81b235d913a') with compensation.<br> 2016-01-26 15:00:15,610 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync='true', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', status='ImageLocked', exitStatus='Normal'}), log id: 6a218716<br> 2016-01-26 15:00:15,614 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] FINISH, SetVmStatusVDSCommand, log id: 6a218716<br> 2016-01-26 15:00:15,617 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}'<br> 2016-01-26 15:00:15,661 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [1a40e7c1] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage<br> 2016-01-26 15:00:15,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='0856943c-c714-43a5-8f26-087143131d74', dstImageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='58da4129-eb0f-476e-b060-94cb2740ab8d', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 54b72148<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters:<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ spUUID=00000001-0001-0001-0001-000000000296<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcImageGUID=dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcVolUUID=0856943c-c714-43a5-8f26-087143131d74<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstImageGUID=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstVolUUID=58da4129-eb0f-476e-b060-94cb2740ab8d<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ descr=<br> 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71<br> 2016-01-26 15:00:17,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] FINISH, CopyImageVDSCommand, return: bf3ef9aa-5099-47b6-b954-9a6902eb17c9, log id: 54b72148<br> 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (default task-20) [1a40e7c1] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'<br> 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [1a40e7c1] CommandMultiAsyncTasks::AttachTask: Attaching task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'.<br> 2016-01-26 15:00:17,668 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [1a40e7c1] Adding task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet..<br> 2016-01-26 15:00:17,872 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [388cd805] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage<br> 2016-01-26 15:00:17,928 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', dstImageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='31f56d0e-cf73-464d-ad9a-8a6b75c68563', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 3e52cd55<br> 2016-01-26 15:00:17,930 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters:<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ spUUID=00000001-0001-0001-0001-000000000296<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcImageGUID=d98b8313-de2f-477a-94f8-78026497612b<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcVolUUID=dd30f129-e8d0-4155-9c63-6ac1ee76fa8e<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstImageGUID=3f83427e-d3a9-4d0a-a176-aa713fbab255<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstVolUUID=31f56d0e-cf73-464d-ad9a-8a6b75c68563<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ descr=<br> 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71<br> 2016-01-26 15:00:18,975 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) [50bfaa54] Setting new tasks map. The map contains now 1 tasks<br> 2016-01-26 15:00:19,707 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] FINISH, CopyImageVDSCommand, return: 8347d2c2-1907-431c-baeb-b885ba6f200d, log id: 3e52cd55<br> 2016-01-26 15:00:19,729 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [388cd805] CommandMultiAsyncTasks::AttachTask: Attaching task '8347d2c2-1907-431c-baeb-b885ba6f200d' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'.<br> 2016-01-26 15:00:19,768 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [388cd805] Adding task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet..<br> 2016-01-26 15:00:19,886 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [66941705] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage<br> 2016-01-26 15:00:19,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', dstImageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='136257df-c62b-421b-a12a-dea765b16721', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 70cc5e38<br> 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters:<br> 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71<br> 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ spUUID=00000001-0001-0001-0001-000000000296<br> 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a<br> 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcImageGUID=42e212c1-f669-43e1-8f90-20baf0c3f58c<br> 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcVolUUID=449e8b88-ab34-4cf8-bc91-00128c35dc65<br> 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstImageGUID=b06a9ad5-62ca-478e-9f0c-278feca4eda3<br> 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstVolUUID=136257df-c62b-421b-a12a-dea765b16721<br> 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ descr=<br> 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71<br> 2016-01-26 15:00:23,635 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] FINISH, CopyImageVDSCommand, return: b910d97d-a091-4648-8840-e870651745c7, log id: 70cc5e38<br> 2016-01-26 15:00:23,654 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [66941705] CommandMultiAsyncTasks::AttachTask: Attaching task 'b910d97d-a091-4648-8840-e870651745c7' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'.<br> 2016-01-26 15:00:23,676 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [66941705] Adding task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet..<br> 2016-01-26 15:00:23,780 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-20) [2cab3adb] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: fa0f7dca-19d3-4f55-82c9-b81b235d913a Type: VMAction group EDIT_VM_PROPERTIES with role type USER<br> 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [2cab3adb] Correlation ID: 30049735, Job ID: fe7f7d93-4a22-4bfe-baae-bc2357dc01e9, Call Stack: null, Custom Event ID: -1, Message: VM test_clone creation was initiated by <a class="moz-txt-link-abbreviated" href="mailto:rfocesq@omme.net@">rfocesq@omme.net@</a><a moz-do-not-send="true" href="http://omme.net" rel="noreferrer" target="_blank">omme.net</a>.<br> 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9'.<br> 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task '8347d2c2-1907-431c-baeb-b885ba6f200d'.<br> 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'b910d97d-a091-4648-8840-e870651745c7'.<br> 2016-01-26 15:00:30,393 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now<br> 2016-01-26 15:00:31,835 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-95) [76980f69] Failed in 'HSMGetAllTasksStatusesVDS' method<br> 2016-01-26 15:00:31,846 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) [76980f69] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed<br> 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'cleaning'.<br> 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'.<br> 2016-01-26 15:00:31,855 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:<br> -- Result: 'cleanSuccess'<br> -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261',<br> -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261'<br> 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-95) [76980f69] Task with DB Task ID 'a773241b-1473-4d42-97b3-bc077d2973fd' and VDSM Task ID '8347d2c2-1907-431c-baeb-b885ba6f200d' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed.<br> 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'.<br> 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Finished polling Tasks, will poll again in 10 seconds.<br> 2016-01-26 15:00:33,309 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] There is no host with less than 8 running guests<br> 2016-01-26 15:00:33,310 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method<br> 2016-01-26 15:00:42,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method<br> 2016-01-26 15:00:42,888 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed<br> 2016-01-26 15:00:42,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method<br> 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed<br> 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method<br> 2016-01-26 15:00:42,905 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed<br> 2016-01-26 15:00:42,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'.<br> 2016-01-26 15:00:42,916 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:<br> -- Result: 'cleanSuccess'<br> -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261',<br> -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261'<br> 2016-01-26 15:00:42,916 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed.<br> 2016-01-26 15:00:42,922 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:<br> -- Result: 'cleanSuccess'<br> -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261',<br> -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261'<br> 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed.<br> 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'.<br> 2016-01-26 15:00:42,930 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:<br> -- Result: 'cleanSuccess'<br> -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261',<br> -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261'<br> 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endActionIfNecessary: All tasks of command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' has ended -> executing 'endAction'<br> 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: 'aaac9162-4be3-473f-91f4-491c8eedb4ad'): calling endAction '.<br> 2016-01-26 15:00:42,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [2c031c31] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'AddVmFromSnapshot', executionIndex: '0'<br> 2016-01-26 15:00:43,014 ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-46) [30049735] Ending command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' with failure.<br> 2016-01-26 15:00:43,036 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [388cd805] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure.<br> 2016-01-26 15:00:43,049 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage<br> 2016-01-26 15:00:43,093 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'}), log id: 5be417ac<br> 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'<br> 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'<br> 2016-01-26 15:00:44,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] FINISH, DeleteImageGroupVDSCommand, log id: 5be417ac<br> 2016-01-26 15:00:44,593 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Disk '3f83427e-d3a9-4d0a-a176-aa713fbab255' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward<br> 2016-01-26 15:00:44,790 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Removed task '0797d4de-2df6-4283-8952-c9d186d2b010' from DataBase<br> 2016-01-26 15:00:44,808 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [1a40e7c1] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure.<br> 2016-01-26 15:00:44,821 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage<br> 2016-01-26 15:00:44,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'}), log id: 475a2763<br> 2016-01-26 15:00:46,268 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'<br> 2016-01-26 15:00:46,269 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'<br> 2016-01-26 15:00:46,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] FINISH, DeleteImageGroupVDSCommand, log id: 475a2763<br> 2016-01-26 15:00:46,270 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Disk '9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward<br> 2016-01-26 15:00:46,476 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Removed task 'b4d1eaf2-6955-41cc-a813-39da5faeca41' from DataBase<br> 2016-01-26 15:00:46,492 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [66941705] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure.<br> 2016-01-26 15:00:46,502 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage<br> 2016-01-26 15:00:46,540 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'}), log id: 5ec9b85c<br> 2016-01-26 15:00:47,792 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'<br> 2016-01-26 15:00:47,793 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'<br> 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] FINISH, DeleteImageGroupVDSCommand, log id: 5ec9b85c<br> 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Disk 'b06a9ad5-62ca-478e-9f0c-278feca4eda3' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward<br> 2016-01-26 15:00:47,953 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Removed task 'fb467301-4397-49db-a200-d623916c8f37' from DataBase<br> 2016-01-26 15:00:48,037 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [] Correlation ID: 30049735, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM test_clone creation.<br> 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' completed, handling the result.<br> 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' succeeded, clearing tasks.<br> 2016-01-26 15:00:48,043 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '8347d2c2-1907-431c-baeb-b885ba6f200d'<br> 2016-01-26 15:00:48,045 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 29d50ec8<br> 2016-01-26 15:00:48,742 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 415f863f<br> 2016-01-26 15:00:49,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 415f863f<br> 2016-01-26 15:00:49,763 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 29d50ec8<br> 2016-01-26 15:00:49,769 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task '8347d2c2-1907-431c-baeb-b885ba6f200d' from DataBase<br> 2016-01-26 15:00:49,770 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9'<br> 2016-01-26 15:00:49,771 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 543d8586<br> 2016-01-26 15:00:49,772 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 35bd0ad7<br> 2016-01-26 15:00:50,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 35bd0ad7<br> 2016-01-26 15:00:50,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 543d8586<br> 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' from DataBase<br> 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'b910d97d-a091-4648-8840-e870651745c7'<br> 2016-01-26 15:00:50,802 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 1ded2669<br> 2016-01-26 15:00:50,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 60352414<br> 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 60352414<br> 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 1ded2669<br> 2016-01-26 15:00:51,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'b910d97d-a091-4648-8840-e870651745c7' from DataBase<br> 2016-01-26 15:00:51,848 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'aaac9162-4be3-473f-91f4-491c8eedb4ad'<br> <br> <br> _______________________________________________<br> Users mailing list<br> <a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br> <a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br> </blockquote> </div> </blockquote> <br> <div class="moz-signature">-- <br> <span style=" font-family:'Helvetica Neue', Helvetica, Arial, sans-serif; font-size:16px; font-weight:bold; color:#000; margin-left:10px;">Mammoli Cristian</span><br> <span style=" font-family:'Helvetica Neue', Helvetica, Arial, sans-serif; font-size:12px;color:#000;margin-left:10px;">System administrator</span><br> <span style=" font-family:'Helvetica Neue', Helvetica, Arial, sans-serif; font-size:12px;color:#000;margin-left:10px;">T. +39 0731 22911</span><br> <span style=" font-family:'Helvetica Neue', Helvetica, Arial, sans-serif; font-size:12px;color:#000;margin-left:10px;">Via Brodolini 6 | 60035 Jesi (an)</span><br> <br> <img src="cid:part8.08050007.04020603@apra.it" alt="" usemap="#Map"> <map name="Map"> <area shape="rect" coords="9,4,27,24" href="https://www.facebook.com/aprainformatica" style=" cursor:pointer" title="Facebook"> <area shape="rect" coords="40,4,60,22" href="https://twitter.com/ApraInformatica" style=" cursor:pointer" title="Twitter"> <area shape="rect" coords="75,3,94,25" href="https://plus.google.com/u/0/+ApraIt/posts" style=" cursor:pointer" title="Google Plus"> <area shape="rect" coords="106,4,126,22" href="http://www.linkedin.com/company/apra_2" style=" cursor:pointer" title="Linkedin"> <area shape="rect" coords="7,60,182,117" href="http://www.apra.it/" style=" cursor:pointer" title="Apra"> </map> </div> </body> </html> --------------040001080806090303030900 Content-Type: image/jpeg; name="Firma2.jpg" Content-Transfer-Encoding: base64 Content-ID: <part8.08050007.04020603@apra.it> Content-Disposition: inline; filename="Firma2.jpg" /9j/4QAYRXhpZgAASUkqAAgAAAAAAAAAAAAAAP/sABFEdWNreQABAAQAAABQAAD/4QP4aHR0 cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLwA8P3hwYWNrZXQgYmVnaW49Iu+7vyIgaWQ9Ilc1 TTBNcENlaGlIenJlU3pOVGN6a2M5ZCI/PiA8eDp4bXBtZXRhIHhtbG5zOng9ImFkb2JlOm5z Om1ldGEvIiB4OnhtcHRrPSJBZG9iZSBYTVAgQ29yZSA1LjUtYzAyMSA3OS4xNTQ5MTEsIDIw MTMvMTAvMjktMTE6NDc6MTYgICAgICAgICI+IDxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDov L3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+IDxyZGY6RGVzY3JpcHRp b24gcmRmOmFib3V0PSIiIHhtbG5zOnhtcE1NPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8x LjAvbW0vIiB4bWxuczpzdFJlZj0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL3NUeXBl L1Jlc291cmNlUmVmIyIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAv IiB4bWxuczpkYz0iaHR0cDovL3B1cmwub3JnL2RjL2VsZW1lbnRzLzEuMS8iIHhtcE1NOk9y aWdpbmFsRG9jdW1lbnRJRD0ieG1wLmRpZDphYzgzNzU2NS00ZDU3LTQ2MzktODU1OC1kYTI4 MjczNDhlMDciIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6RURGMjgyREQ5M0QwMTFFMzhE Mjg4RDRGNzEzQzA1Q0YiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6RURGMjgyREM5M0Qw MTFFMzhEMjg4RDRGNzEzQzA1Q0YiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9w IENDIChNYWNpbnRvc2gpIj4gPHhtcE1NOkRlcml2ZWRGcm9tIHN0UmVmOmluc3RhbmNlSUQ9 InhtcC5paWQ6RDc5NzEzQkU4NEU5MTFFMzhCMTZDOUZENkZCN0M4NjciIHN0UmVmOmRvY3Vt ZW50SUQ9InhtcC5kaWQ6RDc5NzEzQkY4NEU5MTFFMzhCMTZDOUZENkZCN0M4NjciLz4gPGRj OnRpdGxlPiA8cmRmOkFsdD4gPHJkZjpsaSB4bWw6bGFuZz0ieC1kZWZhdWx0Ij5GaXJtYTwv cmRmOmxpPiA8L3JkZjpBbHQ+IDwvZGM6dGl0bGU+IDwvcmRmOkRlc2NyaXB0aW9uPiA8L3Jk ZjpSREY+IDwveDp4bXBtZXRhPiA8P3hwYWNrZXQgZW5kPSJyIj8+/+0ASFBob3Rvc2hvcCAz LjAAOEJJTQQEAAAAAAAPHAFaAAMbJUccAgAAAgACADhCSU0EJQAAAAAAEPzhH4nIt8l4LzRi NAdYd+v/7gAOQWRvYmUAZMAAAAAB/9sAhAACAgICAgICAgICAwICAgMEAwICAwQFBAQEBAQF BgUFBQUFBQYGBwcIBwcGCQkKCgkJDAwMDAwMDAwMDAwMDAwMAQMDAwUEBQkGBgkNCwkLDQ8O Dg4ODw8MDAwMDA8PDAwMDAwMDwwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCAB/ AV4DAREAAhEBAxEB/8QApQABAAEEAwEBAAAAAAAAAAAAAAkFBwgKAwQGAQIBAQEBAAMBAQEA AAAAAAAAAAABAgMEBQYHCBAAAQMDAwIEBQMDAwQDAQAAAgEDBAAFBhESByEIMRMUCUEilVYX USMVYTIWQlIkcZEzU4GzdDURAAIBAgIIBQMEAQIHAAAAAAABAhEDIRIxQZJTBAUVFlEik1QG YTITcYGRQqFScrHB0fFDFAf/2gAMAwEAAhEDEQA/AJ758r0MGbNUFd9Gw4+rSLopeWKlpr8N dKA12E95zm9/c8xw/g7cdwiKO25JuRGIKq7RIkcRCVE6KqImvjonhVoZzF6uHPcN74Of5V1j cRdsOJZg1YfLS93JuZLiQ4puopA05KmSmGvMJE1QBJS066aUoE6kiHZ1z5nncBgGb3fk3C7f gecce53eMGyPH7a+b7ASrQMdXepkehCbxNloZCu3ci6LokKjLqhSAXlT3dOYMK5S5JwqycTY g/ZsNyq749bJdwkXEpDwWyU5GQ3TaMG95o3v2iiaIunXTVbQzmKPaPdo7psht15vOO9uVhyK zY42juR3i1Rb9Ni28FTchS3mFMGk06/MqdOvhShamfvYP3k533YpyxFz3BLXhVy43kWgGQth yk84bo3KIgeZl6mBNrGRddeu7Tam3VTCZIrUKQ7c9+4ZzvgvcxyDwJxbxDjGTwsDitTp19vM uUz5EBq1s3O4XOe+LjMeLFjA6u4zXREROqkSDVoSp4Dkj3IO5vi/DsHzu8YLw1fcZ5IWQuEz 8evtxuSTxhKgTCDyi0H0zhI26JqJCfy7ei6KBuhensT9wPkHuq5WyjjnM8Bx7G41lxVzIodz sr8sjVxmbGiqy4EkjRUJJG5FRU02/Hd8poJ1JX6hSL7ux70uduI+4PGOA+DeF7dypfr/AIq1 kSR3yluTHScflg4LTUYmxBtluLuIjLru+GnzCVMFp/vEdwdrnzrVdOFMLtl2tUl2FdbZKcur UiNJjmrbzDzZOIQG2YqJCvVFSrQmYyD7Svc25Q5+7gcI4fyzjXF7LaMwaum+72iRN9THOBBe mASDIIwISVnYqaIvza69NFUClUmnqGhQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAU AoBQCgFAUjIP/wCDe/8A8En/AOoqA0WIpq3AFwQUyBsiEE8SVNV0/wDmtnEbFnBfC+K2D28s jxTGe7ay4HEzi/w8ivnNVrfahNY9OnfxRyLLIfauLSo8PlDHNSfbJd6CoInyrg2tBdT2rIDd p4a5ntTGVJnjFr5qymIznYmTg3wGWIADc0MjcIklInm6qRKu7xXxqsqJPqhTX97P7JxFy53E 98vbly/ZG8pt2dZxecislilRxFoXLRdpzD8qLcG3G5MaU2MoEHyl1IFJdURDQqyIl74z4v4j 7PeHL5aMdkP2HjrFFuuU326XZ8X3mxJFkSXXn9gK5sAEEd2pbREdV0qFIzvaIzO88jZn3h57 kEl2VeMwvuP3ea+8gI4qy1uzooSAiCiiJImiJ8KrIibioU1secMwv+L+4l3U2i18cTeVce5J xD/D+ScWtzvpJjePXCyWspM5i4G24zDOM402SOPp5f8ApXRVFU2Z1nheUeOMYwrCu1rAbxkV xuvanPv0y/TOVLclvvEsstvSsjOsFyajTgjwPRR2vLcQX13L57w71HywwDIr26bRxlYe/fnu 0cQ3iZecEgYhfWLQ9KjtstMttZJFbbYgutypfqooNCHlSCNCcH5lEarKtJP9UKQy874eeee6 vxJjLeXZJgxS+J3XkyPE5yW+5tenK8ObAeJt4dh7dpiQKip/XRaGdZijcO2/tNxPhG59x3PV 15QvLzvMOS4xelstyYlzLurF5uUVk5JSAb0Mm4/nvvI4hmQkg9XESrUtC4vH3bbi/bL7n3Ae MYJeZt3wbMMWu2UY2NzNHZsQJNrukd2M48gh5goUfeBKKFtLaWqjuWEpibAdDQoBQCgFAKAU AoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgKbd47su1XOKwiK/JiPNMiq6Ipm2Qiir8O q0Bp8tdkfd5EH0rvbvmJOxiJtw24rTgKQkqLsMXlEk18FRdF8U6Vqpx5SRrDe3PnKH7YHMHE k3iXIIvJN75GZudqwpyMHrpMNJlmdWQ20hrqCCy51Vf9C/DRayapgZr+1vxPyPw9275Bj3J2 HTsHvdzzq6XWBZrkgBIWE7Egsg6TYEWxFNk0RC0Xpr4KirWVEk9QprSrxL3dcNdzPMHI2Kdn 8DlYp3IF6veJXvI7OdyaYB64vSYky2yos6OrakBiXVC08FRCToJQ/fcRcfc77mrWmM51whkm P4MrwSH8Fxi1+kgSnGiQ2lmuPSnn5KNkKEIGexCRC2bkRUEdTM32m+DeYOHGeepXKvH11wJv KJOOjYW7uINOylhNT/UKDYmRbQ84PmXouuiaqhaViKoTFVDRBD3B8X80n3Dd3WOXLtvzTkvg zuRaxEpeX4Y9HZusRzG4sZ6I/Cdf81hwRkgYvRnwTdtEkIdEUgMM8r7bO4+HxxK4Z4w7Z+T2 8Cu2Tx8wyK6ZMzEO4TLhBiOQoTLUWGYxYrLTb7hGo73HTUd5ILYjQy0Ze+1n24c68T89Zrln JXFd8wTHJGBvWmLcbu22wLkxy5Q3m2WgQyIv22TVVRNE066ajrWypUJ7ahSHDu2x/uVwTvj4 /wC4rhfgq4cv22wcefwYDHL/AIvqn3ri1IbeJo/MbJtuSBjqOha9PBdBNZhJyRjXe7yXwPI4 Dndod+s9jk57c8+O/wARtxyUku5zZs44qNOu7PLbKcQCWuqiIqqaqtCGVnEFu7sOZe+HgbmL lntwn8R45xpitzx+dP3EsNI/orgjDhHINTUzeloCAKKunXwQlQXWTfUKKAUAoBQCgFAKAUAo BQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFA KAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUBQr7frJjNufvGQ3SNZ7XFJsZFwluC002rpo2 CEZKiJqRIiVzcNwt3iZq3ai5SehLF4YnU43jrHBWndvzUIKlW3RYui/yVVp1t9tt1pwXWnRQ mnRVFEhVNUVFToqKlcLVMHpOzGSkk06pnYoaFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAK AUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoDrPOK20bgtk6oCpI0GikWia7U1VE1X4daqx dDMnlTdKkZ3cQ7yhlFghZznEN3EMWO+fxeLcfO6k+2nlvKU+cors8wkbUQTVei/LonzH+w/E 48Bw158NwzVy5kzTuatK8kNdMcfrpxwX84//AEWXNuN4aPGcXWzZ/Jkt2cc2iTdyeqvlotdG 8qUcZZS8FXDLMVZTiDPIBpcsbiI/iWRxgcct9ytSKiCIPqKIjkdSQVEtC27enRVX4r5Na4fi n/7/AAsvLN0nF0UoT+q8JaaqqrXE/T/hV/jOBj0rjo+e3Gtuaq4XLfgpU0w0UdHlphhV5K18 kfoJxGG8CDco7kVNwroqa/FF+C0TxI1VUPAJx6yiaf5fln1d6vV6q91a2EeB29H3F/1ZH38f M/d2WfV3qvVXurWwh29H3F/1ZD8fM/d2WfV3qdVe6tbCHb0fcX/VkPx8z93ZZ9Xep1V7q1sI dvR9xf8AVkPx8z93ZZ9Xep1V7q1sIdvR9xf9WQ/HzP3dln1d6nVXurWwh29H3F/1ZD8fM/d2 WfV3qdVe6tbCHb0fcX/VkPx8z93ZZ9Xep1V7q1sIdvR9xf8AVkPx8z93ZZ9Xep1V7q1sIdvR 9xf9WQ/HzP3dln1d6nVXurWwh29H3F/1ZD8fM/d2WfV3qdVe6tbCHb0fcX/VkPx8z93ZZ9Xe p1V7q1sIdvR9xf8AVkPx8z93ZZ9Xep1V7q1sIdvR9xf9WQ/HzP3dln1d6nVXurWwh29H3F/1 ZD8fM/d2WfV3qdVe6tbCHb0fcX/VkPx8z93ZZ9Xep1V7q1sIdvR9xf8AVkPx8z93ZZ9Xep1V 7q1sIdvR9xf9WQ/HzP3dln1d6nVXurWwh29H3F/1ZD8fM/d2WfV3qdVe6tbCHb0fcX/VkPx8 z93ZZ9Xep1V7q1sIdvR9xf8AVkPx8z93ZZ9Xep1V7q1sIdvR9xf9WQ/HzP3dln1d6nVXurWw h29H3F/1ZD8fM/d2WfV3qdVe6tbCHb0fcX/VkPx8z93ZZ9Xep1V7q1sIdvR9xf8AVkPx8z93 ZZ9Xep1V7q1sIdvR9xf9WQ/HzP3dln1d6nVXurWwh29H3F/1ZD8fM/d2WfV3qdVe6tbCHb0f cX/VkPx8z93ZZ9Xep1V7q1sIdvR9xf8AVkPx8z93ZZ9Xep1V7q1sIdvR9xf9WQ/HzP3dln1d 6nVXurWwh29H3F/1ZD8fM/d2WfV3qdVe6tbCHb0fcX/VkPx8z93ZZ9Xep1V7q1sIdvR9xf8A VkPx8z93ZZ9Xep1V7q1sIdvR9xf9WQ/HzP3dln1d6nVXurWwh29H3F/1ZD8fM/d2WfV3qdVe 6tbCHb0fcX/VkPx8z93ZZ9Xep1V7q1sIdvR9xf8AVkPx8z93ZZ9Xep1V7q1sIdvR9xf9WQ/H zP3dln1d6nVXurWwh29H3F/1ZD8fM/d2WfV3qdVe6tbCHb0fcX/VkPx8z93ZZ9Xep1V7q1sI dvR9xf8AVkeeyLhXE8vgha8rn3/ILc08MhuDNuskmxeESEXEQCFdUQiROvxrn4X5DxPCTz2I 24SpSqgq08Dpcd8N4Pj4K3xU7tyKdUpXJNVxVdWNG/5LtMMNxmGY7SbWmQRtsNVXQRTRE1Xr 4JXhyk5Nt6WfVQgoRUVoSodqobFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFA KAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQHSlSo8KNIly324sSK2T0mQ6SA22AIpGZ kXREFE1VVqpOTUYqrehGJzjCLlJ0Sxb8EeD495WwLlS3yrlg2QsXlmA8TE5lEJp9kkJUFXGX EExE9NRLTQk8Frv8y5RxXLpqPEQcaqq8H+jWGGvwPP5ZzjhOZQcuHmpU0+K/YuTXnnpigFAK AUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUBTbj cINpgy7ncpTUC3QGTfmzXiQG2mm03EZEuiIiIlatW5XZqEE3JuiS1s4b96FmDuXGoxiqtvQk iKzuB5zzLmlnI8e48YlQOKcbaCRk14QSbdntk4LQlI6psYVwkQG+hH/cSaJon7B8b+OWOVOF zimnxE3SK/04Vw8ZU0vQtR+FfLfml7m0LtvgVJ8PbVZySpXFJVepVeC0vX9LMcC5zK4W5Atm ZGy9Kxq5iVmyMUEwFYzhAZuNkKKJuR1RHNnio6omm5Fr3vknK1zXhJWU0px80f1x/hS0V/6H yvw75LLlPGRvXIv8UvLJ40xp/lYP/uTeQZ8O6QYlyt8luZAnshIhS2l3A424KEBiqeKKi61+ A3IStycJKjTo19T+oLV2N2CnB1i1VNa0yoVk5RQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQ CgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFARzc6d5XCcfPB4UyW0O5Zx+8ZQOTcyhvmjFq liYqyDIs/PIRhwdZBAX7fTRDJCFO1wXF3eDuxvWnScdD0/8AE6XMeXWOYWJWL8c0JaVVr/KL u8n4XguN9reW2zABZdxgrczdLbcYj3q/WIUlp8X1kApechp4FqqbdPglfTci5pf43nVq9elW TdPBJZWqJaj4r5NyPheX/Hb/AA/DxyxST8W3mTq3r/5Itb2j4xjnIHE/IGI5Pbm7paXL8hKB /wBzRuxGkR1ktdW3B26oQ6LXu/NuMvcDzCzfsyyyyfzSTwfivofLf/NOXcPzLlHEcNxEM0HP XqrFYrwappRY+5d6/HnalnVr4QstwuPLmEY/eZTXIGVh5aHjjbqIKQIDbKKk1yM9uckCm1AR VAEVz5E+K51zTqd787tqEmkpUr5mteOjDA/Tfj3JHyjhv/X/ACyuQT8uZYxj/prrxx1foSu4 xk+P5lj9oyrFbxFv+O36K3Ns15hOI6xIYdTUTAx8f6p4ovReteOe8ehoDxvIGZ2rjvB8szm9 OI3a8Ttcq6S9V03JHbI0bHX/AFGqIIp+qpQGv493191jpuPryU1DWS4bqQwtFtUWUMlJGhUm FVUbRduqrr01rNQSjdhvcNk3OeAZTB5Au7d4z/CburU+cLDMZZFunCr0J1WmBAE2qjjSqidd iKvVaqBVe/bljkHhvhKBlnG1/THchfyq121y4LGYl6xZAvK635cgDD5tiddNaMET+Ld4nfPn 16TG8Dy+bl+QlHclDZLVYLU9JVhlRRx3arAptBSTVdfjVB3bn3x98vFWRN2nkGakC6MoEp3F 8uxhiGrzCroiiTAxnFAtNN7ZqmtAZeZd355JyF2c5nypxuY8cct4LkFjsuV21G2bizHWfLaF H43qWyE2JLSlsUh3CqEK9R1UDBDGO9rvszq/RMWwfOJeWZNcAdcgWC14/aXpTwsArjqgCxxR UAEUl6+CUBVcr7z/AHCuJrnCi8jXK4YnLuAE5At+UYlBYalA2qb/ACTBltHNNfm8tzVP6UBJ v2N97z/c4uQ4Rm9hhY7ydikJu5G5a1c/jrrbzcRkpDDbxE4y404oo42pEnzCQloqoIGdGdZh Z+PsLyvOsgeSNZMPtMy8XV5V00Yhsk8aJ/VUHRP60BrFzPcj7x5sqXPj8oMWmPcJDsiJaRsd qMYrTpkbccSOOpF5YKg6qqqunWgJGO07u05q544H5FtVwvZXzmHBL8x6y8Q4bLDzthujLhw3 /IiMrtFuS0TLzjTRGLepCKkiLQEl3DE3Mp+DQZGapK/klcUYz05vypTjKAGpODtAlHzd6NEY iZtIBmiES0BdqgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoCLvvA7uXY4XXibh+6/ 81UOJm2dQHUX0viLkCA4Cr+94o66n/j6iP7mqhUiojG4+hcXsZZb3eXivQYLbkR6bZrDGR6X cSEk2xCM3WfJaLqrhoqkqfKOiluTRGTG433udv4cW5rkVgxy+2XD+Ko9qtqY6VrjxkfK5q6x AgwGG3yb6+QSKhbREU1VdNazRp1RJRTVHoKFx53x8BSeOeSuTYOD3fD3MNm2xc5xuHAiHPeS 6PpEhTAVh0GngU1USVSQh0XVPDXmvcRevKKuTcsuCq26fodfh+CscO5O1CMXLF0SVf1oRWd4 PIfbLzJfPyNxBasmw7kCe6KZfZ59qYjWq8CvT1m9mS55MsE03Eg6Op/d86IS8aqdkpvZ93j5 J2wZEliviSsi4Vv0rzMjxprV2RaH3V+e52sFXx+LzCdHE+YdHP7jVQbJ+KZXjmc47ZsuxC9R Mixi/wAVuZZr3BcR1iQw4mokBJ/2VF6ouqKiKlYBGd7pfOltwPjjEeLAkuldeR563C6QY3Vx bTaCBxUPVUQRdkk0PXx2kn61GCO7t44svHNvAndjyNAx9kbjgdkiMcdK9vfM7nAJLxO2abUU jjNgx0/9ipUQPPe3r3ET8E7mcKi3mUw1ivKrK4ndCEEbEX5ao9bHSVV06SRRvX9HFogSs+6r c27T2z2mW6yT7f8AndjbIAVEL5hk9U16VWCJ3s157wThfmqPyPlAXKfZ2MfuNscgWtgHpyPS yYJtfJccaRQTyl1VC6VED3fe93VY13L3zCX8axeZjOO4C1PRu7XpWW5s124eUhIoNG4LbTaM oqIpqqkvgmnXQKJiPGOWY72T8/8AJ98tki1WDkS/YXbsQGU2TRzWLZdN705sD0LyiN7Y2Wnz bSVNR0VQLQds/L1n4D5uxLli/wBonX61Y0xcmpFqtqtDJcKdDcjAoK+QBoJHquq+HhQF/O9H vgt3dNjeKYbjnH8rE7Bjt3/nJN2u77D05+SDDsdtlkI6mDTe14lNVNVJdqaIidQMnPan7f8A ILddck7ib49EYsl4s543hVvYlMyJDyHIB+ZJktsmfp9qsgANuaOLqRKIjt3AXc92Pl7/ABPh XH+JbZJ8u78u3REuzYloSWS0EEmTuTx0dfVlv+qKSUBF52sdtMjnHijuvyxIXqZuFYg3DwJx RQl/ngL+VcJpV/1+RFFldPg8v60BQ/b65nHiPuc4+uEmV6bGOSR/w3JEItAEboQFAcPX/wBU wG01+CEX60Btc0AoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAhM9wD2/np53vnzt +tDoXv8Acncj8bWxCD13ibtztbLemkjxJ5kf/N1Mf3dUclARJcK5RhX+a2qBzDleW2njq6f8 abkGPyGnJFrdMkQJbzEtiSjzAdUdARQxT5k127VJsE6OYdtfbrwt2vcwZddsxy7OeNsmt9jv 0q/2h23y5jZW+Qv8fOtaNhHjuLrM1JDLYQapWqgtN2m8WdsXPnHvM/F+BZFn8p+/fwEvMc1u se3W+SkeJLORDhw2GTlA2IuskTm8dxa+PRNFQYh92PE3b7wvfl4+4xy3KM8z6A6K5bMmy4bl qtAIiqsQ0YjNk7LPVFUULRpP7/mVBrSYOl2jdnWRdzuTLdLssrHuG8dlI3lOTt6tv3J5tUU7 ZbDVP718Hnk6NJ0TVxURNN0BsmYnieN4Jjlmw/DrLExzGMeihDs1kgto2xHZbTRBEU/7qq9V XVVVVVVriBqpd9nMicz9zfJGRR5nn4vhr3+IYo5u3NejsxGMh4P6OyyePX4ppWWD3XGnD3uM 43hUGLxNYeRMYwHJG/5mHbbPcrfDiShuDYEskmXH0PV5vbruRF00TRK0DDTLMRzTjDLLjimT 2iXhOeYjKjuyLa/tGRBloLcyK6itEQr0JtwVElRUWsAmw74eXInOnt08R8pRVAZGUZNjTl7j Aqf8e5sDJYns6fDZIbNE1+Gi1sGAXt7cWYBzJ3KxcJ5NxqPluLLil3uX8PJN5sPVRnIosuoT BtmiijhafN8aygbDOM9nfbBiFwZuli4Rxhq4R1QmJUuKs8gIV1Qh9YTyIqfr41oFk/cwKNC7 R8q3KEaMze8dAURNBFEuTCCiIidE+FAQ1di9hwvkDun46xLLLRbctsFziXs5tguTAyYrysW1 51sjacFRJQIUJNfBaAzD90HgTh7ijCeNM747wK2YNcblkjlivf8ABxxisS47sJ+Q0jrDeje8 DY1E0RF0VUVVRegHjPaFy+Y3zLyzhsd55uyXrEo97kwVLRlZsGaEcX9idN6tyFFS8VRERfBK iYMVPcD5rXl7ufzyVBli9i/HKJhmOOIqEG22kRXB4VToqOSzcTX/AGgP6UbBVeKeJfcTx/Db fK4YsvIeN4LloN3+C1Y7jAhRpyTWW1CWTTr6Hq60If3Ii7dOlUGHuc4Rm3GuVXbDc4sszDs3 shsvz7XIUEkRXXwGVHeQmSIF1QhcFRJUrANtrtW5ja544B405LUxK63i1NxsoZFU/au8FViz wVE8P3myJE/2qlbBkNQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoCJjum9vviWd m917jYlivU/GrTEm5ByfwjisdFkZVOjijjIwiQh9N6g9fVoCKrg6kG01JSlARa3/AL7uUM74 55q4sy23W0+POU7SzCwfGLOy1Di4csN1hYkaAIgPmRUZYRtwC+bf+4OmpCsqC23BPcvmnbti /L0Djkf4/MuU2LVAiZgWw/4eLBKQUh1hkxJDkOecgtkXyh1JUVdqUqCQzgThW1e4BbrVyxk9 vncaZFYLuNp5svVohI1aM5FlpS/kLW4hIkW4Eoi1NURIfm3D8+mm1IE5WK4rjuD45ZsRxGzR bBjWPxW4dms8MEbYYZbTRBFE8VXxVV6kqqqqqqq1AdjIbbMvNgvdot92esU+6wJMOFfI4C49 DdfaJsJDYH8pE0pISIXTVOtARDRvZr4uB2N6zmjMLhFbdA5kRyJb09QCGhOARo3uTzE1RV11 661KAmKhxY0CJGgw2Rjw4TQMRY4JoLbbYoICKfoiIiJVBHn3Me3Nx93J8ovcqXDPb9hN4nWq HbbtBtUeG8zJKErgtSCWQBEh+WaAui6aCNSgKSx7beLh29XHtzkcw5PMxSRmjGaW26ORIPqo UltogdjtIgbFbdNfMXVNULX9Vqgq3bN7dWGds/KTfKdk5MyHLLg3Z5lnG1XONDaY8uaTREe5 gBLUfKTSpQEi9UFhO5Lge09yXFN34pveQzsYt93mQJrl4tzbTkhsoEkJICIvIoKhKGi6pQGJ nb37auEdvnLeM8uWflLJMmuOMsz2Y9muEWE1HdSfFcikpky2JptRzcmnxSpQGTXdJ20Yr3T8 dw+P8ovdxxpLZeI18tF9taNG+xJjg40qK28JAYm28Yqip8dfhVBjDwN7b1r7eOQWuRMF52yx LkVtm2i4wJMG3LHlxJjaogOoLaL+08jbwqiou4ETwVUqUBZsvZs40fMym84ZnNSS6TtwRyJb tz6uHve3H5e7VxVXVddetKAmFtdsh2W2W6zW1gY1utMVmHAjimgtssAjbYIn6IIolUGAndB7 eOAdznJMbk2551fMIvKWaPZ7kxaWIjzUsYjjhsvOeoAlQxF1Q6fBEqUBeHtR7XoPapiOR4TZ c+vGbWK+3b+ZisXdiO0sKQ40LUjyljiKKLuwCVFToqap4rVBlXQCgFAKAUAoBQCgFAKAUAoB QCgFAKAUAoBQCgFAKAUAoBQEIfuCe32c877z7wHYt1yNXLhyZxtAb6zPE3rpa2QT/wA/iTzI p+7/AHgnmaocaBgp2Vdl2R91OSDfbwsrHuE8fkoGTZQ0itvXR4NCK2WwlT+9U0R53waRdE1c VESJA2gcTxPGsExqyYbh1li49jGORG4VkssJtG2I7DaaIIinxXxVV6kqqqqqqq1oHpqAp82Q 9FhyH2IbtweaBSbhsqAuOqn+kVcIRRV/qqVq3FSkk3ReL1fwcV6bhByjFya1KlX/ADRFsF5U jhCt8t3F7nEW6SZMe3syXoDKuek3I+Sm5JQBQSHaiKWq/BNK9fozzyirkXlSbopP7tH9anzz +RxyRm7M1mbSTcFXL92LlTSqaasrH+fNElykM45dJFptIyEn3sEY8gXorROOttoTqG5tIFb3 iKjv6a6da4OmvypzipSpSONaN0T0UXjR40O31pPNJWpuEa1lhlrFVaWNXR+WqVM2vWcJcm2R yJbZFvh3C4v3hhpy2W9llBfcdcdNlYxC4Qo240TZ+YhqiBtLVelaXJ7qlJScUot1beFEk82G lOqy001Rh/ILDjFwjKTmllSWLbbjlxapKLTzV+2jqVK0ZrEutzSyPWydaby2shJkCUAfs+nF ktVcbMwIXBfFQUVVF666KmlcN/l8rUPyKSlHCjWutdTxwyutfodjhebQvXfwuEozxqmlhlyv Sm001JUarr8CjflC1tx2brNs91t+OTReW15C6yCsSfJAnB2tgZOh5wgvlbwTf0ROqoi9hcnu N5Iyi5qlY1xVcNNKOn9qPA6ncVpRV2cJxtOuWbSpKib0J5lmp5KpZv3RU4GbDIeZYuFhnWaR ImR4LUeQcZ098ltx0FcFh5xW9EbXchaL1Txrhu8vcU3Gakkm8KrQ0sKpV06js2ebqTUZ25Qb kopPK35k3jlk6aManNBzywzrLfr8KyGIGPPSG5qutKjhgwm4Xmmx1IweFUJtdPmRazc5bdhd hbwcppUx8dTepr+3gW1znh7lm5exUbbdcMaLGqWlqSxj4nAznTDJwEyC0TcTG6SPIgO3Q4wN kvpzkakYPGgKgtqiiXXdolaly5yr+KSuZVV5a+KXhjp/gxHnMYuKvwlazOiz5UvtctKbpopR 41Oi1ybaZt0g2u3Qn5qTS0Scj8RppB9W9DQw818SdRSYIk8tC6afFdK5ZcouQtuc2lTVSTf2 qWpYYSWmhww+Q2bl2Nu3FyrrrFL7pQqqyTljFvy1wprOebyHAgXaZb37TPWHBuDFqk3kPJVk ZcloHGwRvzUeVF8wR3IGmv8ARFWs2+VTnbU1JVcXJRxrlTaeNKavE3e57btXpW3CWWMlBywp mkk0qVza0q0/wdeNybZpUmUAw5TcC3Qxn3W4unHH0zJxBmbiY83zyHYYpuAFHcu1FXRas+U3 IxWKq3RLHF5summXTqbrTEzb+QWZydIvLGOaUnl8qyqf21zNUaVVFquB+m+R4zRxQu+O3iyF cUZctSSW2S9Q29Iaj6/tOnsIFeAiE9CQV1TXRaPlUpVyTjKla0rhRN60q1yujWFQufRjRXbU 4ZqONUvMnJR1N0azJtOjoVaZnFohLPGQ1JGTbroNrdiI2iuE4bPqBcBNeoKzqev6CXxSuGHL rk8tGqOOaurTlp+tcP3R2bvOLNvMpJ1jPJSmNWs1V9Mvmr4JnJiOXsZfCOfGgPQWEFpxnzX4 rxELwbxVUjPO7FRPFC0Wpx3AvhJZXKrx1NaP9yVf2N8r5nHj4OcYuKw0uL0qv9XKn70Pa10j 0xQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUB81RPjQDcP60BTLZa7VZIbdus9uiW i3tG44EGEyDDIm84TrpI22giimZkRLp1VVVeq0BU9w/rQH3VP1oBQFtZvHcCVDtERie8wVkf mvxjeZjS0JZxk46htyGjDopfKqIion/Va9O3zWcJSk4p5lFOjcftwWKdf1PCvcjtzhCMZNZH JqqjL7226qSa/Q/SYGqNXKAOSXBuxXhJCzrKIR/L82W2oPK04rSm2BESubBXRCXpoPy1FzLG MvxxzxpSWOiLwqq0b1V8PriXouE4K7JW51rHy0rJUlR0qk35sqwzaMMDhk8aWc7k/d4Fwn2u 5ujDNl6O4Cg1Kh7kSSLRgQ73QPy3UX5THTVN3zVqHNrigoSjGUcdNcVL+ta6E8Y60/pgZucg su47sJSjPy4qmEoV81GqVknlnqktVcT63gL7M4Ly1lVw/wAgM3yn3Q2Yp+cD4Mt+X5JNKAAA sDtQURf7tVVVWo+Zpx/G7ccmFFV4Uq61rVt1da/QR5JKM/yq9L8uNZUi61SVMtKJLKqU+ta1 OsfGMJ+HHss6/wBynYzAF0bXYTJoRj+YBgC+eLaOueQhr5W4vl6Ku5RSt9XlGTuRhFXHSssc f2rRZqeamn6VMP49CUFZncm7Ua5YYeWqaXmpmeWvkq8MNNEfseNYzsqRcbhfJk27vCDY3UGo sd1AbB0BUvJZFDNfN1UyReojpomqKfNpKKhGCUfCsmtKet4LDQvFlXIYym7k7knN0WakYuiT WpYvzaX4KlDsweMcatsO626GU9u3Xq2NWufDOW66nlsIQtONm4pEBgJqiKK6eHTpWLnN79yU Zyy5oyck6JadKdNKZuz8e4azCduGbLOCg1mbwVaNN4pqur+DttYUMh23u5HepOU/xT6vwW5z MVG0/YcY+cG2RQiVHFVSXrqiaaJ0rEuYZVJWoKGZUdG/FPW8NGg5Y8pzuLv3HdyOqzKNNDji ksdNa+J0I/G9ugXSNcrdMKKMctfSHEiPgg+qfloLROMqTKIUgh/bVOmnxTWuSfNZzg4TVa66 yX9VHGj832rScMOQ27V1XLcqU1ZYv+0p4VVY4ya8tMKazuSOPLC/encjUXBvZz1mpctrSuoh xRhmwhK2q+UTY66KuqF1RUrjjzS7G3+L+mWlMafdmrp01/wck+R8PK87/wD5M2bNhX7cjjo+ 1rV44opsrjG2XD+LjT7lJmWi0MtsQ4BtRxdEGmFYQElA0LyAqKpEKF1Vf9vy1zW+bzt5nGKU pNturpi6/bXLX60/zicFz49au5Izm3CCSSpGuCy0zpZqa2q6fpgfJHHJzmmkueW3a4PwGmmb HJMYwlGRp9l/eoiygumasAJEaLqOuiJqq1I81UH5LcUm25LHGqa8cEszolrE+RO6l+S9OTik oPy+WjUq6KSbypNy1eFWVEsChyLol5ud1mT562163ySTymWnCe3gMlW2wREebacNoSTptJdU VetcfUpRh+OEUo5lLW2qf1q39raUmvFHM+TQnd/LcnKUsji9CTrVZ6JfdGLcU/BvAqOJYo1i cFYDM31rAgy20XpY8ckFkNib1jNt71VPFSri47jHxUszVHjrb0/7m6HY5Zy5cDDIpVWC+2Md CpjlSr+rPZ10z0xQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKA6z3maLt8f6UBQ5BTt V2CS/pQFEfO+Iv7QKv8A1RaA4m3MhVfnbVEoCpslc1/vEkoCsxlkrpvRUoCrhrp1oD9UAoBQ CgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKA//Z --------------040001080806090303030900-- --------------090400010105060408010501--

Ok, I'll try to reproduce this with the flow you provided and update the bug if needed Thanks, Raz Tamir On Jan 28, 2016 10:30, "Cristian Mammoli" <c.mammoli@apra.it> wrote:
I read the but but in my case it's not sporadic, it happens every time with all the vms
Il 26/01/2016 22:01, Raz Tamir ha scritto:
Hi Cristian, We already seen this before and have a bug for it: https://bugzilla.redhat.com/show_bug.cgi?id=1201268
Thanks, Raz Tamir On Jan 26, 2016 16:02, "Cristian Mammoli" <c.mammoli@apra.it> wrote:
Hi, I tried this script https://github.com/wefixit-AT/oVirtBackup which basically does: Snapshot a vm clone the snapshot delete the snapshot export the clone to export domain delete the clone
It worked fine for 4 vms then exited with error: Message: VDSM kvm02 command failed: low level Image copy failed VDSM command failed: Image does not exist in domain: u'image=a651c514-a81b-4cfc-a9bc-0bc4caf51568, domain=239c7309-7e4c-4545-8137-6bf1d43cad71'
Now I'm not able to clone snapshots anymore, not even manually everytime I get the same error: This is engine.log when I snapshot the vm:
2016-01-26 14:58:32,984 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] There is no host with less than 8 running guests 2016-01-26 14:58:32,985 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-46) [5de5361] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 14:58:38,685 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (default task-1) [5b009390] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:58:39,007 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-12) [5b009390] Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER 2016-01-26 14:58:39,028 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:39,074 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageSizeInBytes='37580963840', volumeFormat='COW', newImageId='5d8f7880-9125-4c91-ba04-532901d1d805', newImageDescription='', imageInitialSizeInBytes='0', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', sourceImageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c'}), log id: 1113aa14 2016-01-26 14:58:39,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:40,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [217c647e] FINISH, CreateSnapshotVDSCommand, return: 5d8f7880-9125-4c91-ba04-532901d1d805, log id: 1113aa14 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:58:40,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [217c647e] CommandMultiAsyncTasks::AttachTask: Attaching task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:40,382 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [217c647e] Adding task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:40,420 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:40,434 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0', newImageDescription='', imageInitialSizeInBytes='0', imageId='0856943c-c714-43a5-8f26-087143131d74', sourceImageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437'}), log id: 5b8489d6 2016-01-26 14:58:40,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:41,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] FINISH, CreateSnapshotVDSCommand, return: 88a9ab18-7d4c-42f0-9067-0241b64cb7d0, log id: 5b8489d6 2016-01-26 14:58:41,288 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] CommandMultiAsyncTasks::AttachTask: Attaching task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:41,425 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [2adcf7fc] Adding task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:41,499 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 14:58:41,613 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageSizeInBytes='108447924224', volumeFormat='COW', newImageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93', newImageDescription='', imageInitialSizeInBytes='0', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', sourceImageGroupId='d98b8313-de2f-477a-94f8-78026497612b'}), log id: 7e1deb44 2016-01-26 14:58:41,620 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2016-01-26 14:58:43,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-12) [a3e8512] FINISH, CreateSnapshotVDSCommand, return: dbbae52f-a408-41a4-8e6c-0c1919df5c93, log id: 7e1deb44 2016-01-26 14:58:43,852 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-12) [a3e8512] CommandMultiAsyncTasks::AttachTask: Attaching task '9f0ac94f-deb9-4662-824b-f7144c869af0' to command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'. 2016-01-26 14:58:43,871 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Adding task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 14:58:43,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-12) [a3e8512] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' was initiated by rfocesq@omme.net@omme.net. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '010eec77-3b20-483c-bdd3-e9a8912f7ba5'. 2016-01-26 14:58:43,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) [a3e8512] BaseAsyncTask::startPollingTask: Starting to poll task '9f0ac94f-deb9-4662-824b-f7144c869af0'. 2016-01-26 14:58:46,992 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 14:58:48,951 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,960 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 14:58:48,965 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] SPMAsyncTask::PollTask: Polling task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:58:48,971 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) [4c17fe49] BaseAsyncTask::onTaskEndSuccess: Task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:58:48,972 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) [4c17fe49] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 14:59:00,359 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [] Task with DB Task ID '05694254-0fcc-4e77-95ac-ec4f4482c6f8' and VDSM Task ID '9f0ac94f-deb9-4662-824b-f7144c869af0' is in state Polling. End action for command 8f3ccf29-3056-4ab0-bc11-b987814fe5fd will proceed when all the entity's tasks are completed. 2016-01-26 14:59:00,361 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] SPMAsyncTask::PollTask: Polling task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2016-01-26 14:59:00,368 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [] BaseAsyncTask::onTaskEndSuccess: Task '9f0ac94f-deb9-4662-824b-f7144c869af0' (Parent Command 'CreateAllSnapshotsFromVm', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endActionIfNecessary: All tasks of command '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' has ended -> executing 'endAction' 2016-01-26 14:59:00,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: '8f3ccf29-3056-4ab0-bc11-b987814fe5fd'): calling endAction '. 2016-01-26 14:59:00,390 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'CreateAllSnapshotsFromVm', executionIndex: '0' 2016-01-26 14:59:00,541 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Ending command 'org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand' successfully. 2016-01-26 14:59:00,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SnapshotVDSCommand(HostName = kvm01, SnapshotVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}), log id: 459fcd47 2016-01-26 14:59:05,465 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SnapshotVDSCommand, log id: 459fcd47 2016-01-26 14:59:05,483 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:05,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='88a9ab18-7d4c-42f0-9067-0241b64cb7d0'}), log id: 17bd1fc 2016-01-26 14:59:06,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [2adcf7fc] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@ce4f3490, log id: 17bd1fc 2016-01-26 14:59:06,607 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:06,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dbbae52f-a408-41a4-8e6c-0c1919df5c93'}), log id: 6e1489cb 2016-01-26 14:59:08,023 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [a3e8512] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@3294df55, log id: 6e1489cb 2016-01-26 14:59:08,077 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] Ending command 'org.ovirt.engine.core.bll.CreateSnapshotCommand' successfully. 2016-01-26 14:59:08,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='5d8f7880-9125-4c91-ba04-532901d1d805'}), log id: 1dae9b25 2016-01-26 14:59:08,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-25) [217c647e] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@4bd18557, log id: 1dae9b25 2016-01-26 14:59:08,419 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM$VmName hpserver>]', sharedLocks='null'}' 2016-01-26 14:59:08,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [] Correlation ID: 5b009390, Job ID: b76122a8-a780-47e4-9c63-eac6b7f8b6e4, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test_snap' creation for VM 'hpserver' has been completed. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' completed, handling the result. 2016-01-26 14:59:08,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateAllSnapshotsFromVm' succeeded, clearing tasks. 2016-01-26 14:59:08,510 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' 2016-01-26 14:59:08,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 14a34aa3 2016-01-26 14:59:08,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='010eec77-3b20-483c-bdd3-e9a8912f7ba5'}), log id: 3c0fbdc7 2016-01-26 14:59:08,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0fbdc7 2016-01-26 14:59:08,960 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 14a34aa3 2016-01-26 14:59:08,973 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '010eec77-3b20-483c-bdd3-e9a8912f7ba5' from DataBase 2016-01-26 14:59:08,974 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '9f0ac94f-deb9-4662-824b-f7144c869af0' 2016-01-26 14:59:08,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: b2a607c 2016-01-26 14:59:08,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='9f0ac94f-deb9-4662-824b-f7144c869af0'}), log id: 3c0d79bc 2016-01-26 14:59:10,001 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 3c0d79bc 2016-01-26 14:59:10,002 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: b2a607c 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '9f0ac94f-deb9-4662-824b-f7144c869af0' from DataBase 2016-01-26 14:59:10,012 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' 2016-01-26 14:59:10,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 63c6b6df 2016-01-26 14:59:12,174 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='293a6312-8c4f-4c89-a3f7-2f9f0557a9b8'}), log id: 593cc157 2016-01-26 14:59:12,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] START, FullListVDSCommand(HostName = , FullListVDSCommandParameters:{runAsync='true', hostId='f18a051d-b893-41f7-9e97-0037abc7180d', vds='Host[,f18a051d-b893-41f7-9e97-0037abc7180d]', vmIds='[4e683012-8895-43da-b3e1-6d6271f4d829]'}), log id: 3672e426 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, HSMClearTaskVDSCommand, log id: 593cc157 2016-01-26 14:59:12,575 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-25) [] FINISH, SPMClearTaskVDSCommand, log id: 63c6b6df 2016-01-26 14:59:12,585 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] BaseAsyncTask::removeTaskFromDB: Removed task '293a6312-8c4f-4c89-a3f7-2f9f0557a9b8' from DataBase 2016-01-26 14:59:12,586 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '8f3ccf29-3056-4ab0-bc11-b987814fe5fd' 2016-01-26 14:59:13,652 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-6) [] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel7.2.0, afterMigrationStatus=, tabletEnable=true, pid=32651, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, guestNumaNodes=[Ljava.lang.Object;@3c99c1b9, hypervEnable=true, numaTune={nodeset=0,1, mode=interleave}, custom={device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40device_af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47=VmDevice:{id='VmDeviceId:{deviceId='af2dd0fa-cdaf-42f5-8483-a2f6a2ad4a47', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394=VmDevice:{id='VmDeviceId:{deviceId='88ae2b1a-bcd6-4252-abc8-5dd81e315394', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}, device_88ae2b1a-bcd6-4252-abc8-5dd81e315394device_9b763617-eb2b-44ec-b227-397ca7a14d40=VmDevice:{id='VmDeviceId:{deviceId='9b763617-eb2b-44ec-b227-397ca7a14d40', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide0', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}}, vmType=kvm, memSize=2048, smpCoresPerSocket=1, vmName=hpserver, nice=0, status=Up, maxMemSize=20480, bootMenuEnable=false, guestDiskMapping={}, vmId=4e683012-8895-43da-b3e1-6d6271f4d829, displayIp=0, displayPort=5914, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, maxMemSlots=16, nicModel=rtl8139,pv, keyboardLayout=it, kvmEnable=true, devices=[Ljava.lang.Object;@214fa7db, timeOffset=14401, maxVCpus=16, clientIp=, statusTime=5328205390, display=vnc}], log id: 3672e426 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a memballoon Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {target=2097152, alias=balloon0, specParams={model=none}, deviceType=balloon, device=memballoon, type=balloon} 2016-01-26 14:59:13,658 INFO [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] Received a vnc Device without an address when processing VM 4e683012-8895-43da-b3e1-6d6271f4d829 devices, skipping device: {specParams={displayIp=0}, deviceType=graphics, deviceId=333c2dc5-837e-4fd3-b311-e0a7eb5ed45f, device=vnc, type=graphics, port=5914} 2016-01-26 14:59:13,659 ERROR [org.ovirt.engine.core.vdsbroker.VmsMonitoring] (DefaultQuartzScheduler_Worker-6) [] VM '4e683012-8895-43da-b3e1-6d6271f4d829' managed non pluggable device was removed unexpectedly from libvirt: 'VmDevice:{id='VmDeviceId:{deviceId='333c2dc5-837e-4fd3-b311-e0a7eb5ed45f', vmId='4e683012-8895-43da-b3e1-6d6271f4d829'}', device='vnc', type='GRAPHICS', bootOrder='0', specParams='[]', address='', managed='true', plugged='false', readOnly='false', deviceAlias='', customProperties='[]', snapshotId='null', logicalName='null', usingScsiReservation='false'}'
And this when I clone the snapshot: 2016-01-26 15:00:14,932 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [30049735] Lock Acquired to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,511 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000002-0002-0002-0002-0000000000ca Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: StorageAction group CREATE_DISK with role type USER, ID: 4e683012-8895-43da-b3e1-6d6271f4d829 Type: VMAction group CREATE_VM with role type USER 2016-01-26 15:00:15,607 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Locking VM(id = 'fa0f7dca-19d3-4f55-82c9-b81b235d913a') with compensation. 2016-01-26 15:00:15,610 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync='true', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', status='ImageLocked', exitStatus='Normal'}), log id: 6a218716 2016-01-26 15:00:15,614 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-20) [] FINISH, SetVmStatusVDSCommand, log id: 6a218716 2016-01-26 15:00:15,617 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-20) [] Lock freed to object 'EngineLock:{exclusiveLocks='[4e683012-8895-43da-b3e1-6d6271f4d829=<VM, ACTION_TYPE_FAILED_OBJECT_LOCKED>, test_clone=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='null'}' 2016-01-26 15:00:15,661 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [1a40e7c1] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:15,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437', imageId='0856943c-c714-43a5-8f26-087143131d74', dstImageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='58da4129-eb0f-476e-b060-94cb2740ab8d', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 54b72148 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcImageGUID=dd1f13d8-4fb1-4f48-b7b6-cb912c0c1437 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ srcVolUUID=0856943c-c714-43a5-8f26-087143131d74 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstImageGUID=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstVolUUID=58da4129-eb0f-476e-b060-94cb2740ab8d 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ descr= 2016-01-26 15:00:15,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [1a40e7c1] FINISH, CopyImageVDSCommand, return: bf3ef9aa-5099-47b6-b954-9a6902eb17c9, log id: 54b72148 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (default task-20) [1a40e7c1] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' 2016-01-26 15:00:17,632 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [1a40e7c1] CommandMultiAsyncTasks::AttachTask: Attaching task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:17,668 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [1a40e7c1] Adding task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:17,872 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [388cd805] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:17,928 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='d98b8313-de2f-477a-94f8-78026497612b', imageId='dd30f129-e8d0-4155-9c63-6ac1ee76fa8e', dstImageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='31f56d0e-cf73-464d-ad9a-8a6b75c68563', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 3e52cd55 2016-01-26 15:00:17,930 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcImageGUID=d98b8313-de2f-477a-94f8-78026497612b 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ srcVolUUID=dd30f129-e8d0-4155-9c63-6ac1ee76fa8e 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstImageGUID=3f83427e-d3a9-4d0a-a176-aa713fbab255 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstVolUUID=31f56d0e-cf73-464d-ad9a-8a6b75c68563 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ descr= 2016-01-26 15:00:17,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:18,975 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) [50bfaa54] Setting new tasks map. The map contains now 1 tasks 2016-01-26 15:00:19,707 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [388cd805] FINISH, CopyImageVDSCommand, return: 8347d2c2-1907-431c-baeb-b885ba6f200d, log id: 3e52cd55 2016-01-26 15:00:19,729 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [388cd805] CommandMultiAsyncTasks::AttachTask: Attaching task '8347d2c2-1907-431c-baeb-b885ba6f200d' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:19,768 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [388cd805] Adding task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:19,886 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (default task-20) [66941705] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 239c7309-7e4c-4545-8137-6bf1d43cad71 Type: Storage 2016-01-26 15:00:19,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='42e212c1-f669-43e1-8f90-20baf0c3f58c', imageId='449e8b88-ab34-4cf8-bc91-00128c35dc65', dstImageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', vmId='fa0f7dca-19d3-4f55-82c9-b81b235d913a', dstImageId='136257df-c62b-421b-a12a-dea765b16721', imageDescription='', dstStorageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='false'}), log id: 70cc5e38 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID. Parameters: 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ sdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:19,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ spUUID=00000001-0001-0001-0001-000000000296 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ vmGUID=fa0f7dca-19d3-4f55-82c9-b81b235d913a 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcImageGUID=42e212c1-f669-43e1-8f90-20baf0c3f58c 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ srcVolUUID=449e8b88-ab34-4cf8-bc91-00128c35dc65 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstImageGUID=b06a9ad5-62ca-478e-9f0c-278feca4eda3 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstVolUUID=136257df-c62b-421b-a12a-dea765b16721 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ descr= 2016-01-26 15:00:19,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] ++ dstSdUUID=239c7309-7e4c-4545-8137-6bf1d43cad71 2016-01-26 15:00:23,635 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-20) [66941705] FINISH, CopyImageVDSCommand, return: b910d97d-a091-4648-8840-e870651745c7, log id: 70cc5e38 2016-01-26 15:00:23,654 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-20) [66941705] CommandMultiAsyncTasks::AttachTask: Attaching task 'b910d97d-a091-4648-8840-e870651745c7' to command 'aaac9162-4be3-473f-91f4-491c8eedb4ad'. 2016-01-26 15:00:23,676 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-20) [66941705] Adding task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2016-01-26 15:00:23,780 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-20) [2cab3adb] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: fa0f7dca-19d3-4f55-82c9-b81b235d913a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [2cab3adb] Correlation ID: 30049735, Job ID: fe7f7d93-4a22-4bfe-baae-bc2357dc01e9, Call Stack: null, Custom Event ID: -1, Message: VM test_clone creation was initiated by rfocesq@omme.net@ omme.net. 2016-01-26 15:00:23,871 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task '8347d2c2-1907-431c-baeb-b885ba6f200d'. 2016-01-26 15:00:23,872 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-20) [2cab3adb] BaseAsyncTask::startPollingTask: Starting to poll task 'b910d97d-a091-4648-8840-e870651745c7'. 2016-01-26 15:00:30,393 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now 2016-01-26 15:00:31,835 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-95) [76980f69] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:31,846 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) [76980f69] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'cleaning'. 2016-01-26 15:00:31,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:31,855 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-95) [76980f69] Task with DB Task ID 'a773241b-1473-4d42-97b3-bc077d2973fd' and VDSM Task ID '8347d2c2-1907-431c-baeb-b885ba6f200d' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) [76980f69] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'running'. 2016-01-26 15:00:31,857 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [76980f69] Finished polling Tasks, will poll again in 10 seconds. 2016-01-26 15:00:33,309 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] There is no host with less than 8 running guests 2016-01-26 15:00:33,310 WARN [org.ovirt.engine.core.bll.scheduling.policyunits.CpuAndMemoryBalancingPolicyUnit] (DefaultQuartzScheduler_Worker-65) [50bfaa54] All hosts are over-utilized, can't balance the cluster 'Default' using the primary method 2016-01-26 15:00:42,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,888 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) [2c031c31] Failed in 'HSMGetAllTasksStatusesVDS' method 2016-01-26 15:00:42,905 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [2c031c31] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM kvm02 command failed: low level Image copy failed 2016-01-26 15:00:42,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,916 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task '8347d2c2-1907-431c-baeb-b885ba6f200d' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,916 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) [2c031c31] Task with DB Task ID '8c4d9c72-995a-47b6-b8b4-08ce70dbca2e' and VDSM Task ID 'b910d97d-a091-4648-8840-e870651745c7' is in state Polling. End action for command aaac9162-4be3-473f-91f4-491c8eedb4ad will proceed when all the entity's tasks are completed. 2016-01-26 15:00:42,922 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] SPMAsyncTask::PollTask: Polling task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2016-01-26 15:00:42,930 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] BaseAsyncTask::logEndTaskFailure: Task 'b910d97d-a091-4648-8840-e870651745c7' (Parent Command 'AddVmFromSnapshot', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = low level Image copy failed, code = 261' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endActionIfNecessary: All tasks of command 'aaac9162-4be3-473f-91f4-491c8eedb4ad' has ended -> executing 'endAction' 2016-01-26 15:00:42,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) [2c031c31] CommandAsyncTask::endAction: Ending action for '3' tasks (command ID: 'aaac9162-4be3-473f-91f4-491c8eedb4ad'): calling endAction '. 2016-01-26 15:00:42,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [2c031c31] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'AddVmFromSnapshot', executionIndex: '0' 2016-01-26 15:00:43,014 ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-46) [30049735] Ending command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' with failure. 2016-01-26 15:00:43,036 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [388cd805] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:43,049 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:43,093 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'}), log id: 5be417ac 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='3f83427e-d3a9-4d0a-a176-aa713fbab255', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=3f83427e-d3a9-4d0a-a176-aa713fbab255, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:44,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] FINISH, DeleteImageGroupVDSCommand, log id: 5be417ac 2016-01-26 15:00:44,593 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Disk '3f83427e-d3a9-4d0a-a176-aa713fbab255' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:44,790 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [7c5a150a] Removed task '0797d4de-2df6-4283-8952-c9d186d2b010' from DataBase 2016-01-26 15:00:44,808 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [1a40e7c1] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:44,821 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:44,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'}), log id: 475a2763 2016-01-26 15:00:46,268 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:46,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] FINISH, DeleteImageGroupVDSCommand, log id: 475a2763 2016-01-26 15:00:46,270 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Disk '9a1bd54c-a55a-4edc-b1a4-0d6fcfed8bdc' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:46,476 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [1a177c73] Removed task 'b4d1eaf2-6955-41cc-a813-39da5faeca41' from DataBase 2016-01-26 15:00:46,492 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-46) [66941705] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupCommand' with failure. 2016-01-26 15:00:46,502 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage 2016-01-26 15:00:46,540 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'}), log id: 5ec9b85c 2016-01-26 15:00:47,792 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', storageDomainId='239c7309-7e4c-4545-8137-6bf1d43cad71', imageGroupId='b06a9ad5-62ca-478e-9f0c-278feca4eda3', postZeros='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=b06a9ad5-62ca-478e-9f0c-278feca4eda3, domain=239c7309-7e4c-4545-8137-6bf1d43cad71' 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] FINISH, DeleteImageGroupVDSCommand, log id: 5ec9b85c 2016-01-26 15:00:47,793 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Disk 'b06a9ad5-62ca-478e-9f0c-278feca4eda3' doesn't exist on storage domain '239c7309-7e4c-4545-8137-6bf1d43cad71', rolling forward 2016-01-26 15:00:47,953 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [4a3987be] Removed task 'fb467301-4397-49db-a200-d623916c8f37' from DataBase 2016-01-26 15:00:48,037 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [] Correlation ID: 30049735, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM test_clone creation. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' completed, handling the result. 2016-01-26 15:00:48,037 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddVmFromSnapshot' succeeded, clearing tasks. 2016-01-26 15:00:48,043 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '8347d2c2-1907-431c-baeb-b885ba6f200d' 2016-01-26 15:00:48,045 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 29d50ec8 2016-01-26 15:00:48,742 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='8347d2c2-1907-431c-baeb-b885ba6f200d'}), log id: 415f863f 2016-01-26 15:00:49,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 415f863f 2016-01-26 15:00:49,763 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 29d50ec8 2016-01-26 15:00:49,769 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task '8347d2c2-1907-431c-baeb-b885ba6f200d' from DataBase 2016-01-26 15:00:49,770 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' 2016-01-26 15:00:49,771 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 543d8586 2016-01-26 15:00:49,772 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='bf3ef9aa-5099-47b6-b954-9a6902eb17c9'}), log id: 35bd0ad7 2016-01-26 15:00:50,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 35bd0ad7 2016-01-26 15:00:50,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 543d8586 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'bf3ef9aa-5099-47b6-b954-9a6902eb17c9' from DataBase 2016-01-26 15:00:50,800 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'b910d97d-a091-4648-8840-e870651745c7' 2016-01-26 15:00:50,802 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000296', ignoreFailoverLimit='false', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 1ded2669 2016-01-26 15:00:50,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] START, HSMClearTaskVDSCommand(HostName = kvm02, HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', hostId='572aa833-37fb-4c4b-9576-9d367def2d04', taskId='b910d97d-a091-4648-8840-e870651745c7'}), log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, HSMClearTaskVDSCommand, log id: 60352414 2016-01-26 15:00:51,824 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [] FINISH, SPMClearTaskVDSCommand, log id: 1ded2669 2016-01-26 15:00:51,847 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] BaseAsyncTask::removeTaskFromDB: Removed task 'b910d97d-a091-4648-8840-e870651745c7' from DataBase 2016-01-26 15:00:51,848 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'aaac9162-4be3-473f-91f4-491c8eedb4ad'
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Mammoli Cristian System administrator T. +39 0731 22911 Via Brodolini 6 | 60035 Jesi (an)
participants (2)
-
Cristian Mammoli
-
Raz Tamir