[ovirt-users] Clones from snapshots on Cinder RBD problem

[Andrew Richards]

Hello all, 

I already opened bug <https://bugzilla.redhat.com/show_bug.cgi?id=1377442> on this, but I thought I'd post to the list to see if anyone has seen this. I've got oVirt 4.0.3 using a Cinder external provider for Ceph (Cinder version is 7.0.1 from RDO Liberty, Ceph is Jewel 10.2.2) to get access to RBD images. I am able to boot VMs from these RBD images, create snapshots of them, both live and when the VM is offline, but when I try to clone a new VM from those snapshots, the task never completes and I see errors like these <http://pastebin.com/ZLqRDSNW> in /var/log/ovirt-engine/engine.log:
 
> ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-93) [50af631a] Command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' failed: null
> 
> ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (default task-93) [50af631a] Exception: java.lang.NullPointerException

...which is then followed repeatedly by:

> ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (DefaultQuartzScheduler8) [50af631a] Ending command 'org.ovirt.engine.core.bll.AddVmFromSnapshotCommand' with failure.
> 
> ERROR [org.ovirt.engine.core.bll.storage.disk.cinder.CloneSingleCinderDiskCommand] (DefaultQuartzScheduler8) [50af631a] Ending command 'org.ovirt.engine.core.bll.storage.disk.cinder.CloneSingleCinderDiskCommand' with failure.
> 
> INFO  [org.ovirt.engine.core.utils.transaction.TransactionSupport] (DefaultQuartzScheduler8) [] transaction rolled back
> 
> ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler8) [] Correlation ID: 284e62be, Job ID: 1899e73a-e5f8-4cda-b166-fbd5d444dc7c, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM testRBDclone1 creation.
> 
> ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (DefaultQuartzScheduler8) [] Failed invoking callback end method 'onFailed' for command '81f6cbd0-6298-4033-99a2-6a740e10c61f' with exception 'null', the callback is marked for end method retries

The only way I've been able to back out of this state is to use engine-cleanup and then restore from a backup. No amount of stopping and restarting ovirt-engine.service seemed to get the task to fail completely so that it could be cleaned up.

Is anyone successfully using Cinder snapshots? Or is this a known issue for which I have not yet found a bug report? My review of the trace under that java.lang.NullPointerException did not lead me anywhere, but I might not know what I'm looking for there. 

Thanks,
Andrew Richards
Senior Systems Engineer
keepertechnology

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160919/3a618de0/attachment.html>