GlusterFS Hyperconvergence
Hello, I'm in the planning stage of setting up a new oVirt cluster, where I would prefer to use the local disks of the hypervisors for a GlusterFS storage domain. What is the current recommendation (for version 3.5.x)? I found this presentation which seems to suggest that it's possbile with some caveats: http://www.ovirt.org/images/6/6c/2015-ovirt-glusterfs-hyperconvergence.pdf And a few open bugs: https://bugzilla.redhat.com/show_bug.cgi?id=1177791 https://bugzilla.redhat.com/show_bug.cgi?id=1177775 https://bugzilla.redhat.com/show_bug.cgi?id=1177773 What I can't find is current information on if this setup is already possible with 3.5.x or it's better to wait for 3.6. Could someone enlighten me? -- Tiemen Ruiten Systems Engineer R&D Media
Please note I will -not- be using Hosted Engine. On 6 July 2015 at 13:54, Tiemen Ruiten <t.ruiten@rdmedia.com> wrote:
Hello,
I'm in the planning stage of setting up a new oVirt cluster, where I would prefer to use the local disks of the hypervisors for a GlusterFS storage domain. What is the current recommendation (for version 3.5.x)?
I found this presentation which seems to suggest that it's possbile with some caveats: http://www.ovirt.org/images/6/6c/2015-ovirt-glusterfs-hyperconvergence.pdf
And a few open bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1177791 https://bugzilla.redhat.com/show_bug.cgi?id=1177775 https://bugzilla.redhat.com/show_bug.cgi?id=1177773
What I can't find is current information on if this setup is already possible with 3.5.x or it's better to wait for 3.6. Could someone enlighten me?
-- Tiemen Ruiten Systems Engineer R&D Media
-- Tiemen Ruiten Systems Engineer R&D Media
On 06/07/15 15:03, Tiemen Ruiten wrote:
Please note I will -not- be using Hosted Engine.
On 6 July 2015 at 13:54, Tiemen Ruiten <t.ruiten@rdmedia.com <mailto:t.ruiten@rdmedia.com>> wrote:
Hello,
I'm in the planning stage of setting up a new oVirt cluster, where I would prefer to use the local disks of the hypervisors for a GlusterFS storage domain. What is the current recommendation (for version 3.5.x)?
I found this presentation which seems to suggest that it's possbile with some caveats: http://www.ovirt.org/images/6/6c/2015-ovirt-glusterfs-hyperconvergence.pdf
And a few open bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1177791 https://bugzilla.redhat.com/show_bug.cgi?id=1177775 https://bugzilla.redhat.com/show_bug.cgi?id=1177773
What I can't find is current information on if this setup is already possible with 3.5.x or it's better to wait for 3.6. Could someone enlighten me?
-- Tiemen Ruiten Systems Engineer R&D Media
-- Tiemen Ruiten Systems Engineer R&D Media
Hi, most of the current work is focused in hyperconverged using hosted engine and this includes the installation and a few other features such as not fencing hypervisors to avoid killing running bricks. If you do not want to use hosted engine, then you should be able to create bricks and manage them as a gluster volume which your setup should be able to work with. If you do not use HA and/or fencing you should be mostly ok with this use case. Anything specific you're looking for?
Thanks Doron, I would initially start with a 3 host cluster, possibly adding more hosts later on. Is this enough to ensure integrity of the Gluster storage domain, eg. in case of host failure? What are recommendations (if they exist) for the type of Gluster volume for a 3-5 host cluster (replicated or replicated distributed, number of replica's)? Would this setup already leverage libgfapi instead of FUSE? On 6 July 2015 at 16:33, Doron Fediuck <dfediuck@redhat.com> wrote:
On 06/07/15 15:03, Tiemen Ruiten wrote:
Please note I will -not- be using Hosted Engine.
On 6 July 2015 at 13:54, Tiemen Ruiten <t.ruiten@rdmedia.com <mailto:t.ruiten@rdmedia.com>> wrote:
Hello,
I'm in the planning stage of setting up a new oVirt cluster, where I would prefer to use the local disks of the hypervisors for a GlusterFS storage domain. What is the current recommendation (for version 3.5.x)?
I found this presentation which seems to suggest that it's possbile with some caveats: http://www.ovirt.org/images/6/6c/2015-ovirt-glusterfs-hyperconvergence.pdf
And a few open bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1177791 https://bugzilla.redhat.com/show_bug.cgi?id=1177775 https://bugzilla.redhat.com/show_bug.cgi?id=1177773
What I can't find is current information on if this setup is already possible with 3.5.x or it's better to wait for 3.6. Could someone enlighten me?
-- Tiemen Ruiten Systems Engineer R&D Media
-- Tiemen Ruiten Systems Engineer R&D Media
Hi, most of the current work is focused in hyperconverged using hosted engine and this includes the installation and a few other features such as not fencing hypervisors to avoid killing running bricks. If you do not want to use hosted engine, then you should be able to create bricks and manage them as a gluster volume which your setup should be able to work with. If you do not use HA and/or fencing you should be mostly ok with this use case.
Anything specific you're looking for?
-- Tiemen Ruiten Systems Engineer R&D Media
Also, while I can imagine why fencing might be a problem, what would be the issue with HA? On 6 July 2015 at 16:52, Tiemen Ruiten <t.ruiten@rdmedia.com> wrote:
Thanks Doron,
I would initially start with a 3 host cluster, possibly adding more hosts later on. Is this enough to ensure integrity of the Gluster storage domain, eg. in case of host failure?
What are recommendations (if they exist) for the type of Gluster volume for a 3-5 host cluster (replicated or replicated distributed, number of replica's)?
Would this setup already leverage libgfapi instead of FUSE?
On 6 July 2015 at 16:33, Doron Fediuck <dfediuck@redhat.com> wrote:
On 06/07/15 15:03, Tiemen Ruiten wrote:
Please note I will -not- be using Hosted Engine.
On 6 July 2015 at 13:54, Tiemen Ruiten <t.ruiten@rdmedia.com <mailto:t.ruiten@rdmedia.com>> wrote:
Hello,
I'm in the planning stage of setting up a new oVirt cluster, where I would prefer to use the local disks of the hypervisors for a GlusterFS storage domain. What is the current recommendation (for version 3.5.x)?
I found this presentation which seems to suggest that it's possbile with some caveats: http://www.ovirt.org/images/6/6c/2015-ovirt-glusterfs-hyperconvergence.pdf
And a few open bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1177791 https://bugzilla.redhat.com/show_bug.cgi?id=1177775 https://bugzilla.redhat.com/show_bug.cgi?id=1177773
What I can't find is current information on if this setup is already possible with 3.5.x or it's better to wait for 3.6. Could someone enlighten me?
-- Tiemen Ruiten Systems Engineer R&D Media
-- Tiemen Ruiten Systems Engineer R&D Media
Hi, most of the current work is focused in hyperconverged using hosted engine and this includes the installation and a few other features such as not fencing hypervisors to avoid killing running bricks. If you do not want to use hosted engine, then you should be able to create bricks and manage them as a gluster volume which your setup should be able to work with. If you do not use HA and/or fencing you should be mostly ok with this use case.
Anything specific you're looking for?
-- Tiemen Ruiten Systems Engineer R&D Media
-- Tiemen Ruiten Systems Engineer R&D Media
On 06/07/15 16:01, Tiemen Ruiten wrote:
Also, while I can imagine why fencing might be a problem, what would be the issue with HA? /users
Hi, Fencing is required for HA. If a box hosting HA vms seem to have gone away, it *has* to be guaranteed those VMs are not running before they are restarted elsewhere. Otherwise there could be more than 1 VM accessing the same storage, which will corrupt the VM's disk and leave you in a far worse situation. Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. "Transact" is operated by Integrated Financial Arrangements plc. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
Hey, As for libgfapi, there are several uses for it in oVirt, not all fully implemented at his point. However, you need to remember that this is an optimization so whenever it's ready it will be release (as a part of the relevant version) but not having it is still not lacking any functionality. Per your question, we usually recommend working with replica-3 in hosted engine scenario. I'd stick to this mode even without HE to ensure you have replicas as well as quorum in case of a failure. In this mode you can loose the first host and keep running with replica. Loosing the 2nd host will cause the FS to become read only. This is double failure which is pretty impressive for a community project. Finally, HA is leveraging fencing to ensure we do not start an HA VM on a host while it's still running on the old host and cause a split brain. If we do not wish to fence bricks yet still use HA you need to manually approve each time something happens that host has been rebooted which kind of make this a manual mode and less highly available. Doron On 06/07/15 18:01, Tiemen Ruiten wrote:
Also, while I can imagine why fencing might be a problem, what would be the issue with HA?
On 6 July 2015 at 16:52, Tiemen Ruiten <t.ruiten@rdmedia.com <mailto:t.ruiten@rdmedia.com>> wrote:
Thanks Doron,
I would initially start with a 3 host cluster, possibly adding more hosts later on. Is this enough to ensure integrity of the Gluster storage domain, eg. in case of host failure?
What are recommendations (if they exist) for the type of Gluster volume for a 3-5 host cluster (replicated or replicated distributed, number of replica's)?
Would this setup already leverage libgfapi instead of FUSE?
On 6 July 2015 at 16:33, Doron Fediuck <dfediuck@redhat.com <mailto:dfediuck@redhat.com>> wrote:
On 06/07/15 15:03, Tiemen Ruiten wrote: > Please note I will -not- be using Hosted Engine. > > On 6 July 2015 at 13:54, Tiemen Ruiten <t.ruiten@rdmedia.com <mailto:t.ruiten@rdmedia.com> > <mailto:t.ruiten@rdmedia.com <mailto:t.ruiten@rdmedia.com>>> wrote: > > Hello, > > I'm in the planning stage of setting up a new oVirt cluster, where I > would prefer to use the local disks of the hypervisors for a > GlusterFS storage domain. What is the current recommendation (for > version 3.5.x)? > > I found this presentation which seems to suggest that it's possbile > with some > caveats: http://www.ovirt.org/images/6/6c/2015-ovirt-glusterfs-hyperconvergence.pdf > > And a few open bugs: > > https://bugzilla.redhat.com/show_bug.cgi?id=1177791 > https://bugzilla.redhat.com/show_bug.cgi?id=1177775 > https://bugzilla.redhat.com/show_bug.cgi?id=1177773 > > What I can't find is current information on if this setup is already > possible with 3.5.x or it's better to wait for 3.6. Could someone > enlighten me? > > -- > Tiemen Ruiten > Systems Engineer > R&D Media > > > > > -- > Tiemen Ruiten > Systems Engineer > R&D Media > >
Hi, most of the current work is focused in hyperconverged using hosted engine and this includes the installation and a few other features such as not fencing hypervisors to avoid killing running bricks. If you do not want to use hosted engine, then you should be able to create bricks and manage them as a gluster volume which your setup should be able to work with. If you do not use HA and/or fencing you should be mostly ok with this use case.
Anything specific you're looking for?
-- Tiemen Ruiten Systems Engineer R&D Media
-- Tiemen Ruiten Systems Engineer R&D Media
participants (3)
-
Alex Crow -
Doron Fediuck -
Tiemen Ruiten