Cloning a VM with Ceph/Cinder based disk leaves disk in locked state
Dear oVirt Users, we're using cinder (based on the Kolla setup) to provide storage for ovirt. Everything works fine except the clone process of a VM. Cloning a VM with NFS based storage works as expected, thus I think it's the cinder integration that causes the problem here. When cloning a VM with cinder/ceph-based storage we see, that the VM clone is created, the attached image is cloned as well, but the disk/image remains in locked state. We then need to issue a "update images set imagestatus=1 where imagestatus=2;" on the engine to make the VM clone work. Is this a bug in the cinder integration? Thanks and best regards, Thomas engine.log: 2016-11-21 10:00:20,216 INFO [org.ovirt.engine.core.bll.CloneVmCommand] (default task-19) [2dd83801] Lock Acquired to object 'EngineLock:{exclusiveLocks='[vm-vertr-kp-klon=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='[897d96c8-0ea9-4d06-b815-66a42b63c49b=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_CLONED>, e5e06033-e099-4efe-a5cd-9de2ebc0238b=<DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName vm-vertr-kp-klon>]'}' 2016-11-21 10:00:21,290 INFO [org.ovirt.engine.core.bll.CloneVmCommand] (default task-19) [] Running command: CloneVmCommand internal: false. Entities affected : ID: 897d96c8-0ea9-4d06-b815-66a42b63c49b Type: VMAction group CREATE_VM with role type USER 2016-11-21 10:00:21,846 INFO [org.ovirt.engine.core.bll.storage.disk.cinder.CloneSingleCinderDiskCommand] (default task-19) [4c8e58b8] Running command: CloneSingleCinderDiskCommand internal: true. Entities affected : ID: 1f342ea3-49f8-4f65-bf15-ce48514e9bd3 Type: StorageAction group CONFIGURE_VM_STORAGE with role type USER 2016-11-21 10:00:23,143 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-19) [420ca5bf] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: b9f78fd2-9a55-42f0-9ae9-7bca4ae93d9a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-11-21 10:00:23,151 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-19) [7f72e8e4] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: b9f78fd2-9a55-42f0-9ae9-7bca4ae93d9a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-11-21 10:00:23,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-19) [7f72e8e4] Correlation ID: 2dd83801, Job ID: a02e5069-a9ef-4b1b-8ec1-b1922d2e3135, Call Stack: null, Custom Event ID: -1, Message: VM vm-vertr-kp-klon was created by admin@internal-authz. 2016-11-21 10:00:23,240 INFO [org.ovirt.engine.core.bll.CloneVmCommand] (default task-19) [7f72e8e4] Lock freed to object 'EngineLock:{exclusiveLocks='[vm-vertr-kp-klon=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='[897d96c8-0ea9-4d06-b815-66a42b63c49b=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_CLONED>, e5e06033-e099-4efe-a5cd-9de2ebc0238b=<DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName vm-vertr-kp-klon>]'}' 2016-11-21 10:00:29,283 INFO [org.ovirt.engine.core.bll.storage.disk.cinder.CloneSingleCinderDiskCommandCallback] (DefaultQuartzScheduler9) [4c8e58b8] Command 'CloneSingleCinderDisk' id: 'a76696d7-f698-4591-9a26-888f47462888' child commands '[]' executions were completed, status 'SUCCEEDED' 2016-11-21 10:00:29,284 INFO [org.ovirt.engine.core.bll.storage.disk.cinder.CloneSingleCinderDiskCommandCallback] (DefaultQuartzScheduler9) [4c8e58b8] Command 'CloneSingleCinderDisk' id: 'a76696d7-f698-4591-9a26-888f47462888' Updating status to 'SUCCEEDED', The command end method logic will be executed by one of its parent commands. Packets: ovirt-engine-jboss-as-7.1.1-1.el7.centos.x86_64 ovirt-vmconsole-proxy-1.0.4-1.el7.centos.noarch ovirt-engine-wildfly-overlay-10.0.0-1.el7.noarch ovirt-engine-setup-base-4.0.5.5-1.el7.centos.noarch ovirt-guest-agent-common-1.0.12-3.el7.noarch ovirt-engine-setup-plugin-ovirt-engine-4.0.5.5-1.el7.centos.noarch ovirt-host-deploy-1.5.3-1.el7.centos.noarch ovirt-engine-websocket-proxy-4.0.5.5-1.el7.centos.noarch ovirt-engine-extensions-api-impl-4.0.5.5-1.el7.centos.noarch ovirt-engine-wildfly-10.1.0-1.el7.x86_64 ovirt-engine-dbscripts-4.0.5.5-1.el7.centos.noarch ovirt-engine-restapi-4.0.5.5-1.el7.centos.noarch ovirt-vmconsole-1.0.4-1.el7.centos.noarch ovirt-release36-3.6.6-1.noarch ovirt-engine-lib-4.0.5.5-1.el7.centos.noarch ovirt-setup-lib-1.0.2-1.el7.centos.noarch ovirt-engine-setup-plugin-ovirt-engine-common-4.0.5.5-1.el7.centos.noarch ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.0.5.5-1.el7.centos.noarch ovirt-engine-setup-plugin-websocket-proxy-4.0.5.5-1.el7.centos.noarch ovirt-host-deploy-java-1.5.3-1.el7.centos.noarch ovirt-iso-uploader-4.0.2-1.el7.centos.noarch ovirt-engine-setup-4.0.5.5-1.el7.centos.noarch ovirt-engine-extension-aaa-jdbc-1.1.1-1.el7.noarch ovirt-imageio-common-0.4.0-1.el7.noarch ovirt-imageio-proxy-setup-0.4.0-0.201608310602.gita9b573b.el7.centos.noarch ovirt-image-uploader-4.0.1-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch ovirt-release40-4.0.5-2.noarch ovirt-engine-tools-4.0.5.5-1.el7.centos.noarch ovirt-engine-dashboard-1.0.5-1.el7.centos.noarch ovirt-engine-backend-4.0.5.5-1.el7.centos.noarch ovirt-engine-4.0.5.5-1.el7.centos.noarch ovirt-engine-dwh-4.0.5-1.el7.centos.noarch ovirt-engine-cli-3.6.8.1-1.el7.centos.noarch ovirt-imageio-proxy-0.4.0-0.201608310602.gita9b573b.el7.centos.noarch ovirt-guest-tools-iso-4.0-1.fc23.noarch ovirt-engine-dwh-setup-4.0.5-1.el7.centos.noarch python-ovirt-engine-sdk4-4.0.2-1.el7.centos.x86_64 ovirt-engine-vmconsole-proxy-helper-4.0.5.5-1.el7.centos.noarch ovirt-engine-tools-backup-4.0.5.5-1.el7.centos.noarch ovirt-engine-webadmin-portal-4.0.5.5-1.el7.centos.noarch ovirt-engine-userportal-4.0.5.5-1.el7.centos.noarch Sollten noch Fragen offen sein, stehen wir Ihnen natürlich jederzeit gerne zur Verfügung. Mit Grüßen aus Dortmund, Thomas Klute -- ________________________________________________________________________ Dipl.-Inform. Thomas Klute klute@ingenit.com Geschäftsführer / CEO ---------------------------------------------------------------------- ingenit GmbH & Co. KG Tel. +49 (0)231 58 698-120 Emil-Figge-Strasse 76-80 Fax. +49 (0)231 58 698-121 D-44227 Dortmund www.ingenit.com Registergericht: Amtsgericht Dortmund, HRA 13 914 Gesellschafter : Thomas Klute, Marc-Christian Schröer ________________________________________________________________________
Hi Thomas, That does looks like a bug, can u please open a new bug in Bugzilla at https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-engine Please attach it also the engine logs. Thanks, Maor On Mon, Nov 21, 2016 at 12:00 PM, Thomas Klute <klute@ingenit.com> wrote:
Dear oVirt Users,
we're using cinder (based on the Kolla setup) to provide storage for ovirt. Everything works fine except the clone process of a VM. Cloning a VM with NFS based storage works as expected, thus I think it's the cinder integration that causes the problem here.
When cloning a VM with cinder/ceph-based storage we see, that the VM clone is created, the attached image is cloned as well, but the disk/image remains in locked state. We then need to issue a
"update images set imagestatus=1 where imagestatus=2;"
on the engine to make the VM clone work.
Is this a bug in the cinder integration?
Thanks and best regards, Thomas
engine.log: 2016-11-21 10:00:20,216 INFO [org.ovirt.engine.core.bll.CloneVmCommand] (default task-19) [2dd83801] Lock Acquired to object 'EngineLock:{exclusiveLocks='[vm-vertr-kp-klon=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='[897d96c8-0ea9-4d06-b815-66a42b63c49b=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_CLONED>, e5e06033-e099-4efe-a5cd-9de2ebc0238b=<DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName vm-vertr-kp-klon>]'}' 2016-11-21 10:00:21,290 INFO [org.ovirt.engine.core.bll.CloneVmCommand] (default task-19) [] Running command: CloneVmCommand internal: false. Entities affected : ID: 897d96c8-0ea9-4d06-b815-66a42b63c49b Type: VMAction group CREATE_VM with role type USER 2016-11-21 10:00:21,846 INFO [org.ovirt.engine.core.bll.storage.disk.cinder. CloneSingleCinderDiskCommand] (default task-19) [4c8e58b8] Running command: CloneSingleCinderDiskCommand internal: true. Entities affected : ID: 1f342ea3-49f8-4f65-bf15-ce48514e9bd3 Type: StorageAction group CONFIGURE_VM_STORAGE with role type USER 2016-11-21 10:00:23,143 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-19) [420ca5bf] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: b9f78fd2-9a55-42f0-9ae9-7bca4ae93d9a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-11-21 10:00:23,151 INFO [org.ovirt.engine.core.bll.AddGraphicsDeviceCommand] (default task-19) [7f72e8e4] Running command: AddGraphicsDeviceCommand internal: true. Entities affected : ID: b9f78fd2-9a55-42f0-9ae9-7bca4ae93d9a Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2016-11-21 10:00:23,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-19) [7f72e8e4] Correlation ID: 2dd83801, Job ID: a02e5069-a9ef-4b1b-8ec1-b1922d2e3135, Call Stack: null, Custom Event ID: -1, Message: VM vm-vertr-kp-klon was created by admin@internal-authz. 2016-11-21 10:00:23,240 INFO [org.ovirt.engine.core.bll.CloneVmCommand] (default task-19) [7f72e8e4] Lock freed to object 'EngineLock:{exclusiveLocks='[vm-vertr-kp-klon=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', sharedLocks='[897d96c8-0ea9-4d06-b815-66a42b63c49b=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_CLONED>, e5e06033-e099-4efe-a5cd-9de2ebc0238b=<DISK, ACTION_TYPE_FAILED_DISK_IS_USED_FOR_CREATE_VM$VmName vm-vertr-kp-klon>]'}' 2016-11-21 10:00:29,283 INFO [org.ovirt.engine.core.bll.storage.disk.cinder. CloneSingleCinderDiskCommandCallback] (DefaultQuartzScheduler9) [4c8e58b8] Command 'CloneSingleCinderDisk' id: 'a76696d7-f698-4591-9a26-888f47462888' child commands '[]' executions were completed, status 'SUCCEEDED' 2016-11-21 10:00:29,284 INFO [org.ovirt.engine.core.bll.storage.disk.cinder. CloneSingleCinderDiskCommandCallback] (DefaultQuartzScheduler9) [4c8e58b8] Command 'CloneSingleCinderDisk' id: 'a76696d7-f698-4591-9a26-888f47462888' Updating status to 'SUCCEEDED', The command end method logic will be executed by one of its parent commands.
Packets: ovirt-engine-jboss-as-7.1.1-1.el7.centos.x86_64 ovirt-vmconsole-proxy-1.0.4-1.el7.centos.noarch ovirt-engine-wildfly-overlay-10.0.0-1.el7.noarch ovirt-engine-setup-base-4.0.5.5-1.el7.centos.noarch ovirt-guest-agent-common-1.0.12-3.el7.noarch ovirt-engine-setup-plugin-ovirt-engine-4.0.5.5-1.el7.centos.noarch ovirt-host-deploy-1.5.3-1.el7.centos.noarch ovirt-engine-websocket-proxy-4.0.5.5-1.el7.centos.noarch ovirt-engine-extensions-api-impl-4.0.5.5-1.el7.centos.noarch ovirt-engine-wildfly-10.1.0-1.el7.x86_64 ovirt-engine-dbscripts-4.0.5.5-1.el7.centos.noarch ovirt-engine-restapi-4.0.5.5-1.el7.centos.noarch ovirt-vmconsole-1.0.4-1.el7.centos.noarch ovirt-release36-3.6.6-1.noarch ovirt-engine-lib-4.0.5.5-1.el7.centos.noarch ovirt-setup-lib-1.0.2-1.el7.centos.noarch ovirt-engine-setup-plugin-ovirt-engine-common-4.0.5.5-1.el7.centos.noarch ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.0.5. 5-1.el7.centos.noarch ovirt-engine-setup-plugin-websocket-proxy-4.0.5.5-1.el7.centos.noarch ovirt-host-deploy-java-1.5.3-1.el7.centos.noarch ovirt-iso-uploader-4.0.2-1.el7.centos.noarch ovirt-engine-setup-4.0.5.5-1.el7.centos.noarch ovirt-engine-extension-aaa-jdbc-1.1.1-1.el7.noarch ovirt-imageio-common-0.4.0-1.el7.noarch ovirt-imageio-proxy-setup-0.4.0-0.201608310602.gita9b573b. el7.centos.noarch ovirt-image-uploader-4.0.1-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch ovirt-release40-4.0.5-2.noarch ovirt-engine-tools-4.0.5.5-1.el7.centos.noarch ovirt-engine-dashboard-1.0.5-1.el7.centos.noarch ovirt-engine-backend-4.0.5.5-1.el7.centos.noarch ovirt-engine-4.0.5.5-1.el7.centos.noarch ovirt-engine-dwh-4.0.5-1.el7.centos.noarch ovirt-engine-cli-3.6.8.1-1.el7.centos.noarch ovirt-imageio-proxy-0.4.0-0.201608310602.gita9b573b.el7.centos.noarch ovirt-guest-tools-iso-4.0-1.fc23.noarch ovirt-engine-dwh-setup-4.0.5-1.el7.centos.noarch python-ovirt-engine-sdk4-4.0.2-1.el7.centos.x86_64 ovirt-engine-vmconsole-proxy-helper-4.0.5.5-1.el7.centos.noarch ovirt-engine-tools-backup-4.0.5.5-1.el7.centos.noarch ovirt-engine-webadmin-portal-4.0.5.5-1.el7.centos.noarch ovirt-engine-userportal-4.0.5.5-1.el7.centos.noarch
Sollten noch Fragen offen sein, stehen wir Ihnen natürlich jederzeit gerne zur Verfügung.
Mit Grüßen aus Dortmund, Thomas Klute
-- ________________________________________________________________________
Dipl.-Inform. Thomas Klute klute@ingenit.com Geschäftsführer / CEO ---------------------------------------------------------------------- ingenit GmbH & Co. KG Tel. +49 (0)231 58 698-120 Emil-Figge-Strasse 76-80 Fax. +49 (0)231 58 698-121 D-44227 Dortmund www.ingenit.com
Registergericht: Amtsgericht Dortmund, HRA 13 914 Gesellschafter : Thomas Klute, Marc-Christian Schröer ________________________________________________________________________
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (2)
-
Maor Lipchuk -
Thomas Klute