Has there been any consideration of allowing the hosted engine to be installed on a Ceph rbd. I'm not suggesting using cinder but addressing the rbd directly in the hosted engine install process. This would allow ceph only hosting of oVirt without another replicated storage for the engine. Darryl ________________________________ The contents of this electronic message and any attachments are intended only for the addressee and may contain legally privileged, personal, sensitive or confidential information. If you are not the intended addressee, and have received this email, any transmission, distribution, downloading, printing or photocopying of the contents of this message or attachments is strictly prohibited. Any legal privilege or confidentiality attached to this message and attachments is not waived, lost or destroyed by reason of delivery to any person other than intended addressee. If you have received this message and are not the intended addressee you should notify the sender by return email and destroy all copies of the message and any attachments. Unless expressly attributed, the views expressed in this email do not necessarily represent the views of the company.
+1 On 16/06/2016 23:14, Bond, Darryl wrote:
Has there been any consideration of allowing the hosted engine to be installed on a Ceph rbd.
I'm not suggesting using cinder but addressing the rbd directly in the hosted engine install process.
This would allow ceph only hosting of oVirt without another replicated storage for the engine.
Darryl
________________________________
The contents of this electronic message and any attachments are intended only for the addressee and may contain legally privileged, personal, sensitive or confidential information. If you are not the intended addressee, and have received this email, any transmission, distribution, downloading, printing or photocopying of the contents of this message or attachments is strictly prohibited. Any legal privilege or confidentiality attached to this message and attachments is not waived, lost or destroyed by reason of delivery to any person other than intended addressee. If you have received this message and are not the intended addressee you should notify the sender by return email and destroy all copies of the message and any attachments. Unless expressly attributed, the views expressed in this email do not necessarily represent the views of the company. _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Fri, Jun 17, 2016 at 5:30 AM, Fernando Frediani <fernando.frediani@upx.com.br> wrote:
+1
On 16/06/2016 23:14, Bond, Darryl wrote:
Has there been any consideration of allowing the hosted engine to be installed on a Ceph rbd.
I'm not suggesting using cinder but addressing the rbd directly in the hosted engine install process.
This would allow ceph only hosting of oVirt without another replicated storage for the engine.
I agree that this would be nice, but unfortunately working with ceph storage requires cinder. Engine is managing disks via cinder, and also manage deployment of ceph secrets on the hosts that need access to the ceph. For long term we want to support not only ceph but all storage supported by cinder, leveraging cinder echo system. Nir
Nir, I absolutely understand why you would want to use Cinder for oVirt disk management. My question as only about the hosted engine storage which is a 'special case'. The engine-setup process could have the path to the RBD and secrets file. At the moment, you need shared storage just for the hosted engine,. Even when it is hosted on gluster it is recommended to be on its own volume. Darryl ________________________________________ From: users-bounces@ovirt.org <users-bounces@ovirt.org> on behalf of Nir Soffer <nsoffer@redhat.com> Sent: Friday, 17 June 2016 9:07 PM To: Fernando Frediani Cc: users Subject: Re: [ovirt-users] Hosted engine on Ceph RBD On Fri, Jun 17, 2016 at 5:30 AM, Fernando Frediani <fernando.frediani@upx.com.br> wrote:
+1
On 16/06/2016 23:14, Bond, Darryl wrote:
Has there been any consideration of allowing the hosted engine to be installed on a Ceph rbd.
I'm not suggesting using cinder but addressing the rbd directly in the hosted engine install process.
This would allow ceph only hosting of oVirt without another replicated storage for the engine.
I agree that this would be nice, but unfortunately working with ceph storage requires cinder. Engine is managing disks via cinder, and also manage deployment of ceph secrets on the hosts that need access to the ceph. For long term we want to support not only ceph but all storage supported by cinder, leveraging cinder echo system. Nir _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ________________________________ The contents of this electronic message and any attachments are intended only for the addressee and may contain legally privileged, personal, sensitive or confidential information. If you are not the intended addressee, and have received this email, any transmission, distribution, downloading, printing or photocopying of the contents of this message or attachments is strictly prohibited. Any legal privilege or confidentiality attached to this message and attachments is not waived, lost or destroyed by reason of delivery to any person other than intended addressee. If you have received this message and are not the intended addressee you should notify the sender by return email and destroy all copies of the message and any attachments. Unless expressly attributed, the views expressed in this email do not necessarily represent the views of the company.
On Mon, Jun 20, 2016 at 2:23 AM, Bond, Darryl <dbond@nrggos.com.au> wrote:
Nir, I absolutely understand why you would want to use Cinder for oVirt disk management. My question as only about the hosted engine storage which is a 'special case'. The engine-setup process could have the path to the RBD and secrets file.
We are not keeping ceph key in a file. We use libvirt ephemeral private secrets: https://libvirt.org/formatsecret.html This means that only libvirt and the vm it starts can access the ceph key, and once libvirt is killed or the host reboot, the key is not accessible on the host. Once you add a cinder/ceph storage domain and add a key to ovirt-engine, we register the secrets on the hosts that need access to the key, and unregister when a host does not need access. Hosted engine setup can bootstrap the system by adding ceph key to the first host, but from this point, ovirt-engine must know about the cinder/ceph storage domain and the key, so it can register the key on other hosted engine hosts. But even if we solve this issue, we cannot run yet hosted engine on ceph, since hosted engine depend on sanlock, and we don't support sanlock on ceph storage. hosted engine agent also communicate via shared storage, but it does not support ceph storage. We use ceph volumes as network disks - they do not appear on the host as local devices. qemu is accessing these volumes directly using librbd and the ceph key provided by libvirt. This is the most secure way and it give the best performance. The issue with sanlock and hosted engine agents may be solved by exposing certain ceph volumes as local devices, we are considering this for host storage monitoring, which is not implemented yet for ceph. Nir
participants (3)
-
Bond, Darryl -
Fernando Frediani -
Nir Soffer