[ovirt-users] Dedicated NICs for gluster network

[Nicolas Ecarnot]

Hi Ivan, Nathanaël and everyone,

Today, I finally upgraded this oVirt 3.5.2 into 3.6.1, and I saw 
appearing the gluster network in the networks settings.
I'm not using it yet because I still have questions.

I think I perfectly understood your answer, but there are points I'd 
like to know better :

- I'm not sure that using DNS or /etc/hosts is making any difference. 
Once every dedicated NIC is using a dedicated ip address, with the 
dedicated DNS-record + PTR, I see no benefit to use /etc/hosts... ?

- The oVirt setup I'm working on in this discussion is an 
storage+compute setup (oVirt+gluster on the same hosts). That means that 
my oVirt nodes are both clients and servers of themselves - data storage 
speaking.
I'm worrying about not specifying any gateway for the gluster network :
* How will all my hosts - as gluster clients - be able to find a route 
towards the gluster network (gluster servers)?
* How will my guest - my VMs as gluster clients - be able to find a 
route towards the gluster network (indeed, sometimes also provided that 
way!) ?

- Obviously, all this wouldn't be that fun if is wasn't already in 
production (well OK, QA), and thus I have to change the ip of my gluster 
nodes 
(https://www.gluster.org/pipermail/gluster-users/2014-May/017328.html).

-- 
Nicolas ECARNOT

Le 27/11/2015 20:00, Ivan Bulatovic a écrit :
> Hi Nicolas,
>
> what works for me in 3.6 is creating a new network for gluster within
> oVirt, marking it for gluster use only, optionally setting bonded
> interface upon NIC's that are dedicated for gluster traffic and
> providing it with an IP address without configuring a gateway, and then
> modifying /etc/hosts so that hostnames are resolvable between nodes.
> Every node should have two hostnames, one for ovirtmgmt network that is
> resolvable via DNS (or via /etc/hosts), and the other for gluster
> network that is resolvable purely via /etc/hosts (every node should
> contain entries for themselves and for each gluster node).
>
> Peers should be probed via their gluster hostnames, while ensuring that
> gluster peer status contains only addresses and hostnames that are
> dedicated for gluster on each node. Same goes for adding bricks,
> creating a volume etc.
>
> This way, no communication (except gluster one) should be allowed
> through gluster dedicated vlan. To be on the safe side, we can also
> force gluster to listen only on dedicated interfaces via
> transport.socket.bind-address option (haven't tried this one, will do).
>
> Separation of gluster (or in the future any storage network), live
> migration network, vm and management network is always a good thing.
> Perhaps, we could manage failover of those networks within oVirt, ie. in
> case lm network is down - use gluster network for lm and vice versa.
> Cool candidate for an RFE, but first we need this supported within
> gluster itself. This may prove useful when there is not enough NIC's
> available to do a bond beneath every defined network. But we can still
> separate traffic and provide failover by selecting multiple networks
> without actually doing any load balancing between the two.
>
> As Nathanaël mentioned, marking network for gluster use is only
> available in 3.6. I'm also interested if there is a better way around
> this procedure, or perharps enhancing it.
>
> Kind regards,
>
> Ivan
>
> On 11/27/2015 05:47 PM, Nathanaël Blanchet wrote:
>> Hello Nicolas,
>>
>> Did you have a look to this :
>> http://www.ovirt.org/Features/Select_Network_For_Gluster ?
>> But it is only available from >=3.6...
>>
>> Le 27/11/2015 17:02, Nicolas Ecarnot a écrit :
>>> Hello,
>>>
>>> [Here : oVirt 3.5.3, 3 x CentOS 7.0 hosts with replica-3 gluster SD
>>> on the hosts].
>>>
>>> On the switchs, I have created a dedicated VLAN to isolate the
>>> glusterFS traffic, but I'm not using it yet.
>>> I was thinking of creating a dedicated IP for each node's gluster
>>> NIC, and a DNS record by the way ("my_nodes_name_GL"), but I fear
>>> using this hostname or this ip in oVirt GUI host network interface
>>> tab, leading oVirt think this is a different host.
>>>
>>> Not being sure this fear is clearly described, let's say :
>>> - On each node, I create a second ip+(dns record in the soa) used by
>>> gluster, plugged on the correct VLAN
>>> - in oVirt gui, in the host network setting tab, the interface will
>>> be seen, with its ip, but reverse-dns-related to a different hostname.
>>> Here, I fear oVirt might check this reverse DNS and declare this NIC
>>> belongs to another host.
>>>
>>> I would also prefer not use a reverse pointing to the name of the
>>> host management ip, as this is evil and I'm a good guy.
>>>
>>> On your side, how do you cope with a dedicated storage network in
>>> case of storage+compute mixed hosts?
>>>
>>
>


-- 
Nicolas ECARNOT