[ovirt-users] Dedicated NICs for gluster network

[Ivan Bulatovic]

Hi Nicolas,

what works for me in 3.6 is creating a new network for gluster within 
oVirt, marking it for gluster use only, optionally setting bonded 
interface upon NIC's that are dedicated for gluster traffic and 
providing it with an IP address without configuring a gateway, and then 
modifying /etc/hosts so that hostnames are resolvable between nodes. 
Every node should have two hostnames, one for ovirtmgmt network that is 
resolvable via DNS (or via /etc/hosts), and the other for gluster 
network that is resolvable purely via /etc/hosts (every node should 
contain entries for themselves and for each gluster node).

Peers should be probed via their gluster hostnames, while ensuring that 
gluster peer status contains only addresses and hostnames that are 
dedicated for gluster on each node. Same goes for adding bricks, 
creating a volume etc.

This way, no communication (except gluster one) should be allowed 
through gluster dedicated vlan. To be on the safe side, we can also 
force gluster to listen only on dedicated interfaces via 
transport.socket.bind-address option (haven't tried this one, will do).

Separation of gluster (or in the future any storage network), live 
migration network, vm and management network is always a good thing. 
Perhaps, we could manage failover of those networks within oVirt, ie. in 
case lm network is down - use gluster network for lm and vice versa. 
Cool candidate for an RFE, but first we need this supported within 
gluster itself. This may prove useful when there is not enough NIC's 
available to do a bond beneath every defined network. But we can still 
separate traffic and provide failover by selecting multiple networks 
without actually doing any load balancing between the two.

As Nathanaël mentioned, marking network for gluster use is only 
available in 3.6. I'm also interested if there is a better way around 
this procedure, or perharps enhancing it.

Kind regards,

Ivan

On 11/27/2015 05:47 PM, Nathanaël Blanchet wrote:
> Hello Nicolas,
>
> Did you have a look to this : 
> http://www.ovirt.org/Features/Select_Network_For_Gluster ?
> But it is only available from >=3.6...
>
> Le 27/11/2015 17:02, Nicolas Ecarnot a écrit :
>> Hello,
>>
>> [Here : oVirt 3.5.3, 3 x CentOS 7.0 hosts with replica-3 gluster SD 
>> on the hosts].
>>
>> On the switchs, I have created a dedicated VLAN to isolate the 
>> glusterFS traffic, but I'm not using it yet.
>> I was thinking of creating a dedicated IP for each node's gluster 
>> NIC, and a DNS record by the way ("my_nodes_name_GL"), but I fear 
>> using this hostname or this ip in oVirt GUI host network interface 
>> tab, leading oVirt think this is a different host.
>>
>> Not being sure this fear is clearly described, let's say :
>> - On each node, I create a second ip+(dns record in the soa) used by 
>> gluster, plugged on the correct VLAN
>> - in oVirt gui, in the host network setting tab, the interface will 
>> be seen, with its ip, but reverse-dns-related to a different hostname.
>> Here, I fear oVirt might check this reverse DNS and declare this NIC 
>> belongs to another host.
>>
>> I would also prefer not use a reverse pointing to the name of the 
>> host management ip, as this is evil and I'm a good guy.
>>
>> On your side, how do you cope with a dedicated storage network in 
>> case of storage+compute mixed hosts?
>>
>