On Fri, Aug 19, 2016 at 2:33 PM, Nicolas Ecarnot <nicolas@ecarnot.net> wrote:
Le 19/08/2016 à 09:55, Sahina Bose a écrit :

On Fri, Aug 19, 2016 at 12:29 PM, Nicolas Ecarnot <nicolas@ecarnot.net> wrote:

I'm digging out this thread because I now had the time to work on this subject, and I'm stuck.

This oVirt setup has a standalone engine, and 3 hosts.
These 3 hosts are hypervisors and gluster nodes, each using one NIC for all the traffic, that is a very bad idea. (Well, it's working, but not recommended).

I added 3 OTHER nodes, and so far, I only created the gluster setup and created a replica-3 volume.
Each of these new nodes now have one NIC for management, one NIC for gluster, and other NICs for other things.
Each NIC has an IP + DNS name in its dedicated VLAN : one for mgmt and one for gluster.
The mgmt subnet is routed, though the gluster subnet is not.
Every node can ping each other, either using the mgmt or the gluster subnets.

The creation of the gluster subnet and volume went very well and seems to be perfect.

Now, in the oVirt web gui, I'm trying to add these nodes as oVirt hosts.
I'm using their mgmt DNS names, and I'm getting :
"Error while executing action: Server xxxxxxxx is already part of another cluster."

Did you peer probe the gluster cluster prior to adding the nodes to oVirt?

Yes, and using their "gluster subnet" names.
It went fine.

What's the output of "gluster peer status"

[root@serv-vm-al04 log]# gluster peer status
Number of Peers: 2
Hostname: serv-vm-al05-data.sdis.isere.fr
Uuid: eddb3c6d-2e98-45ca-bd1f-6d2153bbb60e
State: Peer in Cluster (Connected)
Hostname: serv-vm-al06-data.sdis.isere.fr
Uuid: cafefdf3-ffc3-4589-abf6-6ca76905593b
State: Peer in Cluster (Connected)

On the two other nodes, the same command output is OK.

If I understand correctly:
node1 - mgmt.ip.1 & gluster.ip.1
node2 - mgmt.ip.2 & gluster.ip.2
node3 - mgmt.ip.3 & gluster.ip.3

Did you create a network and assign "gluster" role to it in the cluster?
I created a gluster network, but did not assign the gluster role so far, as my former 3 hosts had no dedicated NIC not ip for that.
I planned to assign this role once my 3 new hosts were part of the game.
Were you able to add the first node to cluster
, and got this error on second node addition ?
I had the error when trying to add the first node.
From the error, it looks like oVirt does not understand the peer list returned from gluster is a match with node being added.
Sounds correct
Please provide the log snippet of the failure (from engine.log as well as vdsm.log on node)
See attached file

I couldn't view the attached log files for some reason, but the issue is that you're adding the node which is part of a cluster to an existing cluster. That will not work, even from gluster CLI

I found no idea when googling, except something related to gluster (you bet!), telling this may be related to the fact that there is already a volume, managed with a different name.

Obviously, using a different name and IP is what I needed!
I used "transport.socket.bind-address" to make sure the gluster traffic will only use the dedicated NICs.

Well, I also tried to created a storage domain relying on the freshly created gluster volume, but as this subnet is not routed, it is not reachable from the manager nor the existing SPM.

The existing SPM - isn't it one of the the 3 new nodes being added?
No, the SPM is one of the 3 former and still existing hosts.
Or are you adding the 3 nodes to your existing cluster? If so, I suggest you try adding this to a new cluster
OK, I tried and succeed to create a new cluster.
In this new cluster, I was ABLE to add the first new host, using its mgmt DNS name.
This first host still has to have its NICs configured, and (using Chrome or FF) the access to the network settings window is stalling the browser (I tried to restart even the engine, to no avail). Thus, I can not setup this first node NICs.

Thus, I can not add any further host because oVirt relies on a first host to validate the further ones.

Network team should be able to help you here.