[ovirt-users] Dedicated NICs for gluster network

[Nicolas Ecarnot]

Le 19/08/2016 à 09:55, Sahina Bose a écrit :
>
>
> On Fri, Aug 19, 2016 at 12:29 PM, Nicolas Ecarnot <nicolas at ecarnot.net 
> <mailto:nicolas at ecarnot.net>> wrote:
>
>     Hello,
>
>     I'm digging out this thread because I now had the time to work on
>     this subject, and I'm stuck.
>
>     This oVirt setup has a standalone engine, and 3 hosts.
>     These 3 hosts are hypervisors and gluster nodes, each using one
>     NIC for all the traffic, that is a very bad idea. (Well, it's
>     working, but not recommended).
>
>     I added 3 OTHER nodes, and so far, I only created the gluster
>     setup and created a replica-3 volume.
>     Each of these new nodes now have one NIC for management, one NIC
>     for gluster, and other NICs for other things.
>     Each NIC has an IP + DNS name in its dedicated VLAN : one for mgmt
>     and one for gluster.
>     The mgmt subnet is routed, though the gluster subnet is not.
>     Every node can ping each other, either using the mgmt or the
>     gluster subnets.
>
>     The creation of the gluster subnet and volume went very well and
>     seems to be perfect.
>
>     Now, in the oVirt web gui, I'm trying to add these nodes as oVirt
>     hosts.
>     I'm using their mgmt DNS names, and I'm getting :
>     "Error while executing action: Server xxxxxxxx is already part of
>     another cluster."
>
>
> Did you peer probe the gluster cluster prior to adding the nodes to oVirt?

Yes, and using their "gluster subnet" names.
It went fine.

> What's the output of "gluster peer status"

[root at serv-vm-al04 log]# gluster peer status

Number of Peers: 2

Hostname: serv-vm-al05-data.sdis.isere.fr

Uuid: eddb3c6d-2e98-45ca-bd1f-6d2153bbb60e

State: Peer in Cluster (Connected)

Hostname: serv-vm-al06-data.sdis.isere.fr

Uuid: cafefdf3-ffc3-4589-abf6-6ca76905593b

State: Peer in Cluster (Connected)


On the two other nodes, the same command output is OK.

>
> If I understand correctly:
> node1 - mgmt.ip.1 & gluster.ip.1
> node2 - mgmt.ip.2 & gluster.ip.2
> node3 - mgmt.ip.3 & gluster.ip.3
Right
>
> Did you create a network and assign "gluster" role to it in the cluster?
I created a gluster network, but did not assign the gluster role so far, 
as my former 3 hosts had no dedicated NIC not ip for that.
I planned to assign this role once my 3 new hosts were part of the game.
> Were you able to add the first node to cluster
No
> , and got this error on second node addition ?
I had the error when trying to add the first node.
> From the error, it looks like oVirt does not understand the peer list 
> returned from gluster is a match with node being added.
Sounds correct
> Please provide the log snippet of the failure (from engine.log as well 
> as vdsm.log on node)
See attached file
>
>
>
>     I found no idea when googling, except something related to gluster
>     (you bet!), telling this may be related to the fact that there is
>     already a volume, managed with a different name.
>
>     Obviously, using a different name and IP is what I needed!
>     I used "transport.socket.bind-address" to make sure the gluster
>     traffic will only use the dedicated NICs.
>
>
>     Well, I also tried to created a storage domain relying on the
>     freshly created gluster volume, but as this subnet is not routed,
>     it is not reachable from the manager nor the existing SPM.
>
>
> The existing SPM - isn't it one of the the 3 new nodes being added?
No, the SPM is one of the 3 former and still existing hosts.
> Or are you adding the 3 nodes to your existing cluster? If so, I 
> suggest you try adding this to a new cluster
OK, I tried and succeed to create a new cluster.
In this new cluster, I was ABLE to add the first new host, using its 
mgmt DNS name.
This first host still has to have its NICs configured, and (using Chrome 
or FF) the access to the network settings window is stalling the browser 
(I tried to restart even the engine, to no avail). Thus, I can not setup 
this first node NICs.

Thus, I can not add any further host because oVirt relies on a first 
host to validate the further ones.

-- 
Nicolas ECARNOT
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160819/387c0ae5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dedicatedNICissue.tar.gz
Type: application/gzip
Size: 1737684 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160819/387c0ae5/attachment-0001.bin>