<div dir="ltr"><div><br></div><div>I&#39;m running a cluster of oVIRT on 4.0.6 and we are using purely NFS storage for ISO/EXPORT/DATA domain. We are using a Tintri NFS appliance. What I can&#39;t seem to figure out is when I go to create a Pool from a template it fails. I have tried to export the template to export domain, then delete the template and re-import the template, however it fails with the logs below from the engine.log. It appears to give error message trying to create snapshot from Template? We use to be on a gluster volumes be we migrated everything off gluster. I don&#39;t see any mention of gluster errors below so hoping someone might have an idea?</div><div><br></div><br clear="all"><div><div>2017-01-25 16:55:30,200 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler3) [6e36f82d] Setting new tasks map. The map contains now 0 tasks</div><div>2017-01-25 16:55:30,200 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler3) [6e36f82d] Cleared all tasks of pool &#39;5824bb37-0258-026d-0107-00000000004d&#39;.</div><div>2017-01-25 16:55:30,470 INFO  [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (default task-40) [12f23e68] Lock Acquired to object &#39;EngineLock:{exclusiveLocks=&#39;[devins-pool=&lt;VM_POOL_NAME, ACTION_TYPE_FAILED_VM_POOL_IS_BEING_CREATED$VmPoolName devins-pool&gt;]&#39;, sharedLocks=&#39;null&#39;}&#39;</div><div>2017-01-25 16:55:30,551 INFO  [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (org.ovirt.thread.pool-6-thread-18) [12f23e68] Running command: AddVmPoolWithVmsCommand internal: false. Entities affected :  ID: 5824bb37-00b7-031b-0035-0000000002b7 Type: ClusterAction group CREATE_VM_POOL with role type USER,  ID: bca9dcd9-f00f-494d-8d5a-f2903eb8632a Type: VmTemplateAction group CREATE_VM with role type USER</div><div>2017-01-25 16:55:30,552 INFO  [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (org.ovirt.thread.pool-6-thread-18) [12f23e68] Lock freed to object &#39;EngineLock:{exclusiveLocks=&#39;[devins-pool=&lt;VM_POOL_NAME, ACTION_TYPE_FAILED_VM_POOL_IS_BEING_CREATED$VmPoolName devins-pool&gt;]&#39;, sharedLocks=&#39;null&#39;}&#39;</div><div>2017-01-25 16:55:30,679 INFO  [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-6-thread-18) [424815a6] Lock Acquired to object &#39;EngineLock:{exclusiveLocks=&#39;[devins-pool-1=&lt;VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]&#39;, sharedLocks=&#39;[bca9dcd9-f00f-494d-8d5a-f2903eb8632a=&lt;TEMPLATE, ACTION_TYPE_FAILED_TEMPLATE_IS_USED_FOR_CREATE_VM$VmName devins-pool-1&gt;, 612217c0-87e0-41f6-b06d-edb99f177da8=&lt;VM_POOL, ACTION_TYPE_FAILED_VM_POOL_IS_USED_FOR_CREATE_VM$VmName devins-pool-1&gt;, 1cd672e8-9f78-403e-9d8c-c61c2047b672=&lt;DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName devins-pool-1&gt;]&#39;}&#39;</div><div>2017-01-25 16:55:30,726 INFO  [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-6-thread-18) [] Running command: AddVmCommand internal: true. Entities affected :  ID: 5824bb37-00b7-031b-0035-0000000002b7 Type: ClusterAction group CREATE_VM with role type USER,  ID: bca9dcd9-f00f-494d-8d5a-f2903eb8632a Type: VmTemplateAction group CREATE_VM with role type USER,  ID: f8d5f826-bd95-4229-9e87-9005b79f2448 Type: StorageAction group CREATE_DISK with role type USER</div><div>2017-01-25 16:55:30,808 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-6-thread-18) [] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync=&#39;true&#39;, vmId=&#39;0d203249-b120-4d04-9b8e-21d874957592&#39;, status=&#39;ImageLocked&#39;, exitStatus=&#39;Normal&#39;}), log id: 7a39ac9f</div><div>2017-01-25 16:55:30,813 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-6-thread-18) [] FINISH, SetVmStatusVDSCommand, log id: 7a39ac9f</div><div>2017-01-25 16:55:30,819 INFO  [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-6-thread-18) [6a0aa969] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected :  ID: f8d5f826-bd95-4229-9e87-9005b79f2448 Type: Storage</div><div>2017-01-25 16:55:30,834 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-6-thread-18) [6a0aa969] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync=&#39;true&#39;, storagePoolId=&#39;5824bb37-0258-026d-0107-00000000004d&#39;, ignoreFailoverLimit=&#39;false&#39;, storageDomainId=&#39;f8d5f826-bd95-4229-9e87-9005b79f2448&#39;, imageGroupId=&#39;3e53272f-180e-4517-9dac-75813951a56b&#39;, imageSizeInBytes=&#39;75161927680&#39;, volumeFormat=&#39;COW&#39;, newImageId=&#39;a9f169f8-6005-42f7-8437-4c6271db016d&#39;, newImageDescription=&#39;&#39;, imageInitialSizeInBytes=&#39;0&#39;, imageId=&#39;150f3171-e992-473e-a96d-aac599c9e556&#39;, sourceImageGroupId=&#39;1cd672e8-9f78-403e-9d8c-c61c2047b672&#39;}), log id: 2f0c9075</div><div>2017-01-25 16:55:30,835 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-6-thread-18) [6a0aa969] -- executeIrsBrokerCommand: calling &#39;createVolume&#39; with two new parameters: description and UUID</div><div>2017-01-25 16:55:31,885 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-6-thread-18) [6a0aa969] FINISH, CreateSnapshotVDSCommand, return: a9f169f8-6005-42f7-8437-4c6271db016d, log id: 2f0c9075</div><div>2017-01-25 16:55:31,889 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-18) [6a0aa969] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command &#39;f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260&#39;</div><div>2017-01-25 16:55:31,889 INFO  [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-6-thread-18) [6a0aa969] CommandMultiAsyncTasks::attachTask: Attaching task &#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39; to command &#39;f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260&#39;.</div><div>2017-01-25 16:55:31,901 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-18) [6a0aa969] Adding task &#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39; (Parent Command &#39;CreateSnapshotFromTemplate&#39;, Parameters Type &#39;org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters&#39;), polling hasn&#39;t started yet..</div><div>2017-01-25 16:55:32,178 INFO  [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (org.ovirt.thread.pool-6-thread-18) [25fb544d] Running command: AddGraphicsDeviceCommand internal: true. Entities affected :  ID: 0d203249-b120-4d04-9b8e-21d874957592 Type: VMAction group EDIT_VM_PROPERTIES with role type USER</div><div>2017-01-25 16:55:32,237 INFO  [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-6-thread-18) [f2d0c64] Running command: AddVmToPoolCommand internal: true. Entities affected :  ID: 612217c0-87e0-41f6-b06d-edb99f177da8 Type: VmPool</div><div>2017-01-25 16:55:32,249 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-18) [f2d0c64] Correlation ID: 424815a6, Job ID: c6b9ef03-c542-45be-b659-8946450b79c3, Call Stack: null, Custom Event ID: -1, Message: VM devins-pool-1 creation was initiated by devin.acosta@lxi.domain.com-authz.</div><div>2017-01-25 16:55:32,249 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-18) [f2d0c64] BaseAsyncTask::startPollingTask: Starting to poll task &#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39;.</div><div>2017-01-25 16:55:32,368 INFO  [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-6-thread-18) [6504b822] Lock Acquired to object &#39;EngineLock:{exclusiveLocks=&#39;[devins-pool-2=&lt;VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]&#39;, sharedLocks=&#39;[bca9dcd9-f00f-494d-8d5a-f2903eb8632a=&lt;TEMPLATE, ACTION_TYPE_FAILED_TEMPLATE_IS_USED_FOR_CREATE_VM$VmName devins-pool-2&gt;, 612217c0-87e0-41f6-b06d-edb99f177da8=&lt;VM_POOL, ACTION_TYPE_FAILED_VM_POOL_IS_USED_FOR_CREATE_VM$VmName devins-pool-2&gt;, 1cd672e8-9f78-403e-9d8c-c61c2047b672=&lt;DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName devins-pool-2&gt;]&#39;}&#39;</div><div>2017-01-25 16:55:32,415 INFO  [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-6-thread-18) [] Running command: AddVmCommand internal: true. Entities affected :  ID: 5824bb37-00b7-031b-0035-0000000002b7 Type: ClusterAction group CREATE_VM with role type USER,  ID: bca9dcd9-f00f-494d-8d5a-f2903eb8632a Type: VmTemplateAction group CREATE_VM with role type USER,  ID: f8d5f826-bd95-4229-9e87-9005b79f2448 Type: StorageAction group CREATE_DISK with role type USER</div><div>2017-01-25 16:55:32,459 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-6-thread-18) [] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync=&#39;true&#39;, vmId=&#39;4631557a-4aa7-48da-bd0b-e3ba8191cf61&#39;, status=&#39;ImageLocked&#39;, exitStatus=&#39;Normal&#39;}), log id: 68fb2cd8</div><div>2017-01-25 16:55:32,463 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-6-thread-18) [] FINISH, SetVmStatusVDSCommand, log id: 68fb2cd8</div><div>2017-01-25 16:55:32,470 INFO  [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-6-thread-18) [57bd66ee] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected :  ID: f8d5f826-bd95-4229-9e87-9005b79f2448 Type: Storage</div><div>2017-01-25 16:55:32,482 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-6-thread-18) [57bd66ee] START, CreateSnapshotVDSCommand( CreateSnapshotVDSCommandParameters:{runAsync=&#39;true&#39;, storagePoolId=&#39;5824bb37-0258-026d-0107-00000000004d&#39;, ignoreFailoverLimit=&#39;false&#39;, storageDomainId=&#39;f8d5f826-bd95-4229-9e87-9005b79f2448&#39;, imageGroupId=&#39;b604b900-83f9-47f8-bc7f-d2267095586f&#39;, imageSizeInBytes=&#39;75161927680&#39;, volumeFormat=&#39;COW&#39;, newImageId=&#39;c1f0485d-b2b9-4931-8eb1-6fc5b5f0a106&#39;, newImageDescription=&#39;&#39;, imageInitialSizeInBytes=&#39;0&#39;, imageId=&#39;150f3171-e992-473e-a96d-aac599c9e556&#39;, sourceImageGroupId=&#39;1cd672e8-9f78-403e-9d8c-c61c2047b672&#39;}), log id: 46e84547</div><div>2017-01-25 16:55:32,482 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-6-thread-18) [57bd66ee] -- executeIrsBrokerCommand: calling &#39;createVolume&#39; with two new parameters: description and UUID</div><div>2017-01-25 16:55:32,591 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-6-thread-18) [57bd66ee] FINISH, CreateSnapshotVDSCommand, return: c1f0485d-b2b9-4931-8eb1-6fc5b5f0a106, log id: 46e84547</div><div>2017-01-25 16:55:32,595 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-18) [57bd66ee] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command &#39;74f8e242-92fe-4f3a-bf4a-a63b482f89c4&#39;</div><div>2017-01-25 16:55:32,595 INFO  [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-6-thread-18) [57bd66ee] CommandMultiAsyncTasks::attachTask: Attaching task &#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39; to command &#39;74f8e242-92fe-4f3a-bf4a-a63b482f89c4&#39;.</div><div>2017-01-25 16:55:32,605 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-18) [57bd66ee] Adding task &#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39; (Parent Command &#39;CreateSnapshotFromTemplate&#39;, Parameters Type &#39;org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters&#39;), polling hasn&#39;t started yet..</div><div>2017-01-25 16:55:32,694 INFO  [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (org.ovirt.thread.pool-6-thread-18) [5266def0] Running command: AddGraphicsDeviceCommand internal: true. Entities affected :  ID: 4631557a-4aa7-48da-bd0b-e3ba8191cf61 Type: VMAction group EDIT_VM_PROPERTIES with role type USER</div><div>2017-01-25 16:55:32,703 INFO  [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-6-thread-18) [15776f98] Running command: AddVmToPoolCommand internal: true. Entities affected :  ID: 612217c0-87e0-41f6-b06d-edb99f177da8 Type: VmPool</div><div>2017-01-25 16:55:32,716 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-18) [15776f98] Correlation ID: 6504b822, Job ID: c6b9ef03-c542-45be-b659-8946450b79c3, Call Stack: null, Custom Event ID: -1, Message: VM devins-pool-2 creation was initiated by devin.acosta@lxi.domain.com-authz.</div><div>2017-01-25 16:55:32,716 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-18) [15776f98] BaseAsyncTask::startPollingTask: Starting to poll task &#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39;.</div><div>2017-01-25 16:55:32,731 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-18) [15776f98] Correlation ID: 12f23e68, Job ID: c6b9ef03-c542-45be-b659-8946450b79c3, Call Stack: null, Custom Event ID: -1, Message: VM Pool devins-pool (containing 2 VMs) was created by devin.acosta@lxi.domain.com-authz.</div><div>2017-01-25 16:55:32,896 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler7) [424815a6] Command &#39;AddVmPoolWithVms&#39; (id: &#39;36264463-029e-4457-85c4-1c73f6cda00d&#39;) waiting on child command id: &#39;fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3&#39; type:&#39;AddVm&#39; to complete</div><div>2017-01-25 16:55:32,921 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler7) [57bd66ee] Command &#39;AddVm&#39; (id: &#39;10a5e5ba-6024-4e5b-a2be-34f107829e80&#39;) waiting on child command id: &#39;74f8e242-92fe-4f3a-bf4a-a63b482f89c4&#39; type:&#39;CreateSnapshotFromTemplate&#39; to complete</div><div>2017-01-25 16:55:32,941 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler7) [6a0aa969] Command &#39;AddVm&#39; (id: &#39;fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3&#39;) waiting on child command id: &#39;f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260&#39; type:&#39;CreateSnapshotFromTemplate&#39; to complete</div><div>2017-01-25 16:55:34,982 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler6) [57bd66ee] Command &#39;AddVm&#39; (id: &#39;10a5e5ba-6024-4e5b-a2be-34f107829e80&#39;) waiting on child command id: &#39;74f8e242-92fe-4f3a-bf4a-a63b482f89c4&#39; type:&#39;CreateSnapshotFromTemplate&#39; to complete</div><div>2017-01-25 16:55:37,058 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler10) [424815a6] Command &#39;AddVmPoolWithVms&#39; (id: &#39;36264463-029e-4457-85c4-1c73f6cda00d&#39;) waiting on child command id: &#39;fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3&#39; type:&#39;AddVm&#39; to complete</div><div>2017-01-25 16:55:37,086 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler10) [6a0aa969] Command &#39;AddVm&#39; (id: &#39;fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3&#39;) waiting on child command id: &#39;f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260&#39; type:&#39;CreateSnapshotFromTemplate&#39; to complete</div><div>2017-01-25 16:55:37,644 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler5) [457a14b4] Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now</div><div>2017-01-25 16:55:38,584 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5) [457a14b4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM dev01-002-001 command failed: Cannot get parent volume</div><div>2017-01-25 16:55:38,588 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5) [457a14b4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM dev01-002-001 command failed: Cannot get parent volume</div><div>2017-01-25 16:55:38,588 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler5) [457a14b4] SPMAsyncTask::PollTask: Polling task &#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39; (Parent Command &#39;CreateSnapshotFromTemplate&#39;, Parameters Type &#39;org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters&#39;) returned status &#39;finished&#39;, result &#39;cleanSuccess&#39;.</div><div>2017-01-25 16:55:38,590 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler5) [457a14b4] BaseAsyncTask::logEndTaskFailure: Task &#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39; (Parent Command &#39;CreateSnapshotFromTemplate&#39;, Parameters Type &#39;org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters&#39;) ended with failure:</div><div>-- Result: &#39;cleanSuccess&#39;</div><div>-- Message: &#39;VDSGenericException: VDSErrorException: Failed in vdscommand to HSMGetAllTasksStatusesVDS, error = Cannot get parent volume&#39;,</div><div>-- Exception: &#39;VDSGenericException: VDSErrorException: Failed in vdscommand to HSMGetAllTasksStatusesVDS, error = Cannot get parent volume&#39;</div><div>2017-01-25 16:55:38,592 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler5) [457a14b4] CommandAsyncTask::endActionIfNecessary: All tasks of command &#39;f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260&#39; has ended -&gt; executing &#39;endAction&#39;</div><div>2017-01-25 16:55:38,592 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler5) [457a14b4] CommandAsyncTask::endAction: Ending action for &#39;1&#39; tasks (command ID: &#39;f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260&#39;): calling endAction &#39;.</div><div>2017-01-25 16:55:38,592 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler5) [457a14b4] SPMAsyncTask::PollTask: Polling task &#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39; (Parent Command &#39;CreateSnapshotFromTemplate&#39;, Parameters Type &#39;org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters&#39;) returned status &#39;finished&#39;, result &#39;cleanSuccess&#39;.</div><div>2017-01-25 16:55:38,592 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) [457a14b4] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction &#39;CreateSnapshotFromTemplate&#39;, executionIndex: &#39;0&#39;</div><div>2017-01-25 16:55:38,594 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler5) [457a14b4] BaseAsyncTask::logEndTaskFailure: Task &#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39; (Parent Command &#39;CreateSnapshotFromTemplate&#39;, Parameters Type &#39;org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters&#39;) ended with failure:</div><div>-- Result: &#39;cleanSuccess&#39;</div><div>-- Message: &#39;VDSGenericException: VDSErrorException: Failed in vdscommand to HSMGetAllTasksStatusesVDS, error = Cannot get parent volume&#39;,</div><div>-- Exception: &#39;VDSGenericException: VDSErrorException: Failed in vdscommand to HSMGetAllTasksStatusesVDS, error = Cannot get parent volume&#39;</div><div>2017-01-25 16:55:38,597 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler5) [457a14b4] CommandAsyncTask::endActionIfNecessary: All tasks of command &#39;74f8e242-92fe-4f3a-bf4a-a63b482f89c4&#39; has ended -&gt; executing &#39;endAction&#39;</div><div>2017-01-25 16:55:38,597 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler5) [457a14b4] CommandAsyncTask::endAction: Ending action for &#39;1&#39; tasks (command ID: &#39;74f8e242-92fe-4f3a-bf4a-a63b482f89c4&#39;): calling endAction &#39;.</div><div>2017-01-25 16:55:38,597 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-39) [457a14b4] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction &#39;CreateSnapshotFromTemplate&#39;, executionIndex: &#39;0&#39;</div><div>2017-01-25 16:55:38,600 INFO  [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] Command [id=f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260]: Updating status to &#39;FAILED&#39;, The command end method logic will be executed by one of its parent commands.</div><div>2017-01-25 16:55:38,600 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type &#39;CreateSnapshotFromTemplate&#39; completed, handling the result.</div><div>2017-01-25 16:55:38,601 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type &#39;CreateSnapshotFromTemplate&#39; succeeded, clearing tasks.</div><div>2017-01-25 16:55:38,601 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] SPMAsyncTask::ClearAsyncTask: Attempting to clear task &#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39;</div><div>2017-01-25 16:55:38,601 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync=&#39;true&#39;, storagePoolId=&#39;5824bb37-0258-026d-0107-00000000004d&#39;, ignoreFailoverLimit=&#39;false&#39;, taskId=&#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39;}), log id: 4cd9d13e</div><div>2017-01-25 16:55:38,602 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] START, HSMClearTaskVDSCommand(HostName = dev01-002-001, HSMTaskGuidBaseVDSCommandParameters:{runAsync=&#39;true&#39;, hostId=&#39;fb1dbd7b-c059-44f6-8da8-e1db4540d91c&#39;, taskId=&#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39;}), log id: 61f4ae2a</div><div>2017-01-25 16:55:38,604 INFO  [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] Command [id=74f8e242-92fe-4f3a-bf4a-a63b482f89c4]: Updating status to &#39;FAILED&#39;, The command end method logic will be executed by one of its parent commands.</div><div>2017-01-25 16:55:38,604 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type &#39;CreateSnapshotFromTemplate&#39; completed, handling the result.</div><div>2017-01-25 16:55:38,604 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type &#39;CreateSnapshotFromTemplate&#39; succeeded, clearing tasks.</div><div>2017-01-25 16:55:38,604 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] SPMAsyncTask::ClearAsyncTask: Attempting to clear task &#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39;</div><div>2017-01-25 16:55:38,605 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{runAsync=&#39;true&#39;, storagePoolId=&#39;5824bb37-0258-026d-0107-00000000004d&#39;, ignoreFailoverLimit=&#39;false&#39;, taskId=&#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39;}), log id: 2d04146e</div><div>2017-01-25 16:55:38,619 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] FINISH, HSMClearTaskVDSCommand, log id: 61f4ae2a</div><div>2017-01-25 16:55:38,619 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] FINISH, SPMClearTaskVDSCommand, log id: 4cd9d13e</div><div>2017-01-25 16:55:38,620 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] START, HSMClearTaskVDSCommand(HostName = dev01-002-001, HSMTaskGuidBaseVDSCommandParameters:{runAsync=&#39;true&#39;, hostId=&#39;fb1dbd7b-c059-44f6-8da8-e1db4540d91c&#39;, taskId=&#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39;}), log id: 71c703c8</div><div>2017-01-25 16:55:38,622 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] BaseAsyncTask::removeTaskFromDB: Removed task &#39;a22c4e91-dac8-40e4-ba47-42807173ea91&#39; from DataBase</div><div>2017-01-25 16:55:38,622 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) [6a0aa969] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity &#39;f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260&#39;</div><div>2017-01-25 16:55:39,130 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler4) [57bd66ee] Command &#39;AddVm&#39; id: &#39;10a5e5ba-6024-4e5b-a2be-34f107829e80&#39; child commands &#39;[74f8e242-92fe-4f3a-bf4a-a63b482f89c4]&#39; executions were completed, status &#39;FAILED&#39;</div><div>2017-01-25 16:55:39,438 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] FINISH, HSMClearTaskVDSCommand, log id: 71c703c8</div><div>2017-01-25 16:55:39,438 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] FINISH, SPMClearTaskVDSCommand, log id: 2d04146e</div><div>2017-01-25 16:55:39,441 INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] BaseAsyncTask::removeTaskFromDB: Removed task &#39;dbad59d1-711b-4807-af30-250ff7cfd1cd&#39; from DataBase</div><div>2017-01-25 16:55:39,442 INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-39) [57bd66ee] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity &#39;74f8e242-92fe-4f3a-bf4a-a63b482f89c4&#39;</div><div>2017-01-25 16:55:40,179 ERROR [org.ovirt.engine.core.bll.AddVmCommand] (DefaultQuartzScheduler9) [57bd66ee] Ending command &#39;org.ovirt.engine.core.bll.AddVmCommand&#39; with failure.</div><div>2017-01-25 16:55:40,185 ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (DefaultQuartzScheduler9) [57bd66ee] Ending command &#39;org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand&#39; with failure.</div><div>2017-01-25 16:55:40,308 INFO  [org.ovirt.engine.core.bll.AddVmCommand] (DefaultQuartzScheduler9) [] Lock freed to object &#39;EngineLock:{exclusiveLocks=&#39;[devins-pool-2=&lt;VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]&#39;, sharedLocks=&#39;[bca9dcd9-f00f-494d-8d5a-f2903eb8632a=&lt;TEMPLATE, ACTION_TYPE_FAILED_TEMPLATE_IS_USED_FOR_CREATE_VM$VmName devins-pool-2&gt;, 612217c0-87e0-41f6-b06d-edb99f177da8=&lt;VM_POOL, ACTION_TYPE_FAILED_VM_POOL_IS_USED_FOR_CREATE_VM$VmName devins-pool-2&gt;, 1cd672e8-9f78-403e-9d8c-c61c2047b672=&lt;DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName devins-pool-2&gt;]&#39;}&#39;</div><div>2017-01-25 16:55:40,319 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler9) [] Correlation ID: 6504b822, Job ID: c6b9ef03-c542-45be-b659-8946450b79c3, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM devins-pool-2 creation.</div><div>2017-01-25 16:55:41,390 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler10) [424815a6] Command &#39;AddVmPoolWithVms&#39; (id: &#39;36264463-029e-4457-85c4-1c73f6cda00d&#39;) waiting on child command id: &#39;fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3&#39; type:&#39;AddVm&#39; to complete</div><div>2017-01-25 16:55:43,461 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler6) [424815a6] Command &#39;AddVmPoolWithVms&#39; (id: &#39;36264463-029e-4457-85c4-1c73f6cda00d&#39;) waiting on child command id: &#39;fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3&#39; type:&#39;AddVm&#39; to complete</div><div>2017-01-25 16:55:45,503 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler2) [6a0aa969] Command &#39;AddVm&#39; id: &#39;fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3&#39; child commands &#39;[f4a67fb3-e3ca-4fe1-ab8e-8240bd85f260]&#39; executions were completed, status &#39;FAILED&#39;</div><div>2017-01-25 16:55:46,551 ERROR [org.ovirt.engine.core.bll.AddVmCommand] (DefaultQuartzScheduler9) [6a0aa969] Ending command &#39;org.ovirt.engine.core.bll.AddVmCommand&#39; with failure.</div><div>2017-01-25 16:55:46,554 ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (DefaultQuartzScheduler9) [6a0aa969] Ending command &#39;org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand&#39; with failure.</div><div>2017-01-25 16:55:46,643 INFO  [org.ovirt.engine.core.bll.AddVmCommand] (DefaultQuartzScheduler9) [] Lock freed to object &#39;EngineLock:{exclusiveLocks=&#39;[devins-pool-1=&lt;VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]&#39;, sharedLocks=&#39;[bca9dcd9-f00f-494d-8d5a-f2903eb8632a=&lt;TEMPLATE, ACTION_TYPE_FAILED_TEMPLATE_IS_USED_FOR_CREATE_VM$VmName devins-pool-1&gt;, 612217c0-87e0-41f6-b06d-edb99f177da8=&lt;VM_POOL, ACTION_TYPE_FAILED_VM_POOL_IS_USED_FOR_CREATE_VM$VmName devins-pool-1&gt;, 1cd672e8-9f78-403e-9d8c-c61c2047b672=&lt;DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName devins-pool-1&gt;]&#39;}&#39;</div><div>2017-01-25 16:55:46,654 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler9) [] Correlation ID: 424815a6, Job ID: c6b9ef03-c542-45be-b659-8946450b79c3, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM devins-pool-1 creation.</div><div>2017-01-25 16:55:47,781 INFO  [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (DefaultQuartzScheduler2) [6504b822] Command &#39;AddVmPoolWithVms&#39; id: &#39;36264463-029e-4457-85c4-1c73f6cda00d&#39; child commands &#39;[fbe6e9e5-aabf-48d8-a9c0-266b2f420eb3, 10a5e5ba-6024-4e5b-a2be-34f107829e80]&#39; executions were completed, status &#39;FAILED&#39;</div><div>2017-01-25 16:55:48,881 ERROR [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (DefaultQuartzScheduler4) [6504b822] Ending command &#39;org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand&#39; with failure.</div><div>2017-01-25 16:55:48,888 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler4) [6504b822] Correlation ID: 12f23e68, Job ID: c6b9ef03-c542-45be-b659-8946450b79c3, Call Stack: null, Custom Event ID: -1, Message: VM Pool devins-pool (containing 2 VMs) was created by devin.acosta@lxi.domain.com-authz.</div><div>2017-01-25 16:56:02,640 INFO  [org.ovirt.engine.core.sso.servlets.OAuthRevokeServlet] (default task-58) [] User <a href="mailto:devin.acosta@lxi.domain.com">devin.acosta@lxi.domain.com</a> successfully logged out</div><div>2017-01-25 16:56:02,653 INFO  [org.ovirt.engine.core.bll.aaa.TerminateSessionsForTokenCommand] (default task-61) [343a5bd4] Running command: TerminateSessionsForTokenCommand internal: true.</div><div>2017-01-25 16:56:42,271 INFO  [org.ovirt.engine.core.bll.RemoveVmPoolCommand] (default task-25) [2d22ba57] Lock Acquired to object &#39;EngineLock:{exclusiveLocks=&#39;[612217c0-87e0-41f6-b06d-edb99f177da8=&lt;VM_POOL, ACTION_TYPE_FAILED_VM_POOL_IS_BEING_REMOVED$VmPoolName devins-pool&gt;]&#39;, sharedLocks=&#39;null&#39;}&#39;</div><div>2017-01-25 16:56:42,296 INFO  [org.ovirt.engine.core.bll.RemoveVmPoolCommand] (org.ovirt.thread.pool-6-thread-5) [2d22ba57] Running command: RemoveVmPoolCommand internal: false. Entities affected :  ID: 612217c0-87e0-41f6-b06d-edb99f177da8 Type: VmPoolAction group DELETE_VM_POOL with role type USER</div><div>2017-01-25 16:56:42,301 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-5) [2d22ba57] Correlation ID: 2d22ba57, Job ID: d1b36387-a69d-425a-88ab-fc302ac20208, Call Stack: null, Custom Event ID: -1, Message: VM Pool devins-pool removal was initiated by devin.acosta@lxi.domain.com-authz.</div><div>2017-01-25 16:56:42,307 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-5) [2d22ba57] Correlation ID: 2d22ba57, Job ID: d1b36387-a69d-425a-88ab-fc302ac20208, Call Stack: null, Custom Event ID: -1, Message: VM Pool devins-pool removal was initiated by devin.acosta@lxi.domain.com-authz.</div><div>2017-01-25 16:56:42,313 INFO  [org.ovirt.engine.core.bll.RemoveVmPoolCommand] (org.ovirt.thread.pool-6-thread-5) [2d22ba57] Lock freed to object &#39;EngineLock:{exclusiveLocks=&#39;[612217c0-87e0-41f6-b06d-edb99f177da8=&lt;VM_POOL, ACTION_TYPE_FAILED_VM_POOL_IS_BEING_REMOVED$VmPoolName devins-pool&gt;]&#39;, sharedLocks=&#39;null&#39;}&#39;</div><div>2017-01-25 16:56:42,918 INFO  [org.ovirt.engine.core.bll.RemoveVmPoolCommandCallback] (DefaultQuartzScheduler9) [] Command &#39;RemoveVmPool&#39; id: &#39;00ba7c5d-a30d-4ab9-977e-14c139305461&#39; child commands &#39;[]&#39; executions were completed, status &#39;SUCCEEDED&#39;</div></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><br></div><div>Devin Acosta</div><div>Red Hat Certified Architect, LinuxStack </div><div><a href="mailto:devin@linuxguru.co" target="_blank">devin@linuxguru.co</a></div></div></div>
</div>