[ovirt-users] Gluster storage question

[Doug Ingham]

On 11 February 2017 at 13:32, Bartosiak-Jentys, Chris <
chris.bartosiak-jentys at certico.co.uk> wrote:

> Hello list,
>
> Just wanted to get your opinion on my ovirt home lab setup. While this is
> not a production setup I would like it to run relatively reliably so please
> tell me if the following storage configuration is likely to result in
> corruption or just bat s**t insane.
>
> I have a 3 node hosted engine setup, VM data store and engine data store
> are both replica 3 gluster volumes (one brick on each host).
> I do not want to run all 3 hosts 24/7 due to electricity costs, I only
> power up the larger hosts (2 Dell R710's) when I need additional resources
> for VM's.
>
> I read about using CTDB and floating/virtual IP's to allow the storage
> mount point to transition between available hosts but after some thought
> decided to go about this another, simpler, way:
>
> I created a common hostname for the storage mount points: gfs-data and
> gfs-engine
>
> On each host I edited /etc/hosts file to have these hostnames resolve to
> each hosts IP i.e. on host1 gfs-data & gfs-engine --> host1 IP
> on host2 gfs-data & gfs-engine --> host2 IP
> etc.
>
> In ovirt engine each storage domain is mounted as gfs-data:/data and
> gfs-engine:/engine
> My thinking is that this way no matter which host is up and acting as SPM
> it will be able to mount the storage as its only dependent on that host
> being up.
>
> I changed gluster options for server-quorum-ratio so that the volumes
> remain up even if quorum is not met, I know this is risky but its just a
> lab setup after all.
>
> So, any thoughts on the /etc/hosts method to ensure the storage mount
> point is always available? Is data corruption more or less inevitable with
> this setup? Am I insane ;) ?
>

Why not just use localhost? And no need for CTDB with a floating IP, oVirt
uses libgfapi for Gluster which deals with that all natively.

As for the quorum issue, I would most definitely *not* run with quorum
disabled when you're running more than one node. As you say you
specifically plan for when the other 2 nodes of the replica 3 set will be
active or not, I'd do something along the lines of the following...

Going from 3 nodes to 1 node:
 - Put nodes 2 & 3 in maintenance to offload their virtual load;
 - Once the 2 nodes are free of load, disable quorum on the Gluster volumes;
 - Power down the 2 nodes.

Going from 1 node to 3 nodes:
 - Power on *only* 1 of the pair of nodes (if you power on both & self-heal
is enabled, Gluster will "heal" the files on the main node with the older
files on the 2 nodes which were powered down);
 - Allow Gluster some time to detect that the files are in split-brain;
 - Tell Gluster to heal the files in split-brain based on modification time;
 - Once the 2 nodes are in sync, re-enable quorum & power on the last node,
which will be resynchronised automatically;
 - Take the 2 hosts out of maintenance mode.

If you want to power on the 2nd two nodes at the same time, make absolutely
sure self-heal is disabled first! If you don't, Gluster will see the 2nd
two nodes as in quorum & heal the data on your 1st node with the
out-of-date data.


-- 
Doug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170211/9e88b021/attachment.html>