[ovirt-users] The bumpy road to Ceph

[Bond, Darryl]

We intend to migrate off our small Openstack/Ceph cluster to oVirt as it is a better fit for our organisation.


We wanted to run oVirt backed on to Ceph storage for quite some time. We waited for the 3.6 development to take it's course and started testing last year.

The kolla cinder container implementation changed fundamentally in the middle of the oVirt release process which broke the oVirt 'official' cinder implementation.


We had tested oVirt using our Openstack cinder so we knew it basically worked but we didn't want to mix the oVirt and Openstack volumes, ie put oVirt into it's own ceph pool.

We waited and waited for the Kolla container ...


There didn't seem to be any end in sight, so we chose the following configuration:

- Hosted engine on Centos 7.2 with gluster storage

- Cinder/Keystone VM on Centos 7.2 using RDO and cinder 'liberty' kept in the same gluster store as the engine and managed by the engine.

- Used the oVirt engine Postgres database as the Cinder/Keystone store so we didn't introduce more dependencies with MySQL that Kolla required.

        (New user and database for the cinder store. Backups exported separate to the engine backups)


It's early days yet but here are some observations:

- It works very well integrated with oVirt from the point of view of building and managing new VM's

- Snapshots and cloning use ceph's features

- No simple means to move Vm disks to and from other storage domains.

- Not impossible though, it means that you have to delete and import ceph RBD's behind both oVirt and Cinder. It is fairly dangerous as I haven't found a simple way to get the rbd volume name with vdsClient other than 'vdsClient list' running VM's. Openstack at least publishes the volume name (UUID) in the gui, oVirt does not.

- Successfully imported VM images from both Openstack and VMWare.

- There is a hard dependency between oVirt and Cinder when starting VM's but does not require Cinder to run VMs. Strange really, when the volume name could be stored in oVirt when the volume is created rather than query cinder for each start.


Where to from here.

Hosting Cinder/Keystone in containers on the engine VM makes sense.

We are going to roll our own container(s), given the data will be in the engine Postgres database it should be possible to transition in a clean fassion.




Darryl




________________________________

The contents of this electronic message and any attachments are intended only for the addressee and may contain legally privileged, personal, sensitive or confidential information. If you are not the intended addressee, and have received this email, any transmission, distribution, downloading, printing or photocopying of the contents of this message or attachments is strictly prohibited. Any legal privilege or confidentiality attached to this message and attachments is not waived, lost or destroyed by reason of delivery to any person other than intended addressee. If you have received this message and are not the intended addressee you should notify the sender by return email and destroy all copies of the message and any attachments. Unless expressly attributed, the views expressed in this email do not necessarily represent the views of the company.