Re: [ovirt-users] Dedicated NICs for gluster network

Le 19/08/2016 à 09:55, Sahina Bose a écrit : On Fri, Aug 19, 2016 at 12:29 PM, Nicolas Ecarnot <nicolas(a)ecarnot.net&gt; wrote: > Hello, > > I'm digging out this thread because I now had the time to work on this > subject, and I'm stuck. > > This oVirt setup has a standalone engine, and 3 hosts. > These 3 hosts are hypervisors and gluster nodes, each using one NIC for > all the traffic, that is a very bad idea. (Well, it's working, but not > recommended). > > I added 3 OTHER nodes, and so far, I only created the gluster setup and > created a replica-3 volume. > Each of these new nodes now have one NIC for management, one NIC for > gluster, and other NICs for other things. > Each NIC has an IP + DNS name in its dedicated VLAN : one for mgmt and > one for gluster. > The mgmt subnet is routed, though the gluster subnet is not. > Every node can ping each other, either using the mgmt or the gluster > subnets. > > The creation of the gluster subnet and volume went very well and seems to > be perfect. > > Now, in the oVirt web gui, I'm trying to add these nodes as oVirt hosts. > I'm using their mgmt DNS names, and I'm getting : > "Error while executing action: Server xxxxxxxx is already part of another > cluster." > Did you peer probe the gluster cluster prior to adding the nodes to oVirt? Yes, and using their "gluster subnet" names. It went fine. What's the output of "gluster peer status" [root@serv-vm-al04 log]# gluster peer status Number of Peers: 2 Hostname: serv-vm-al05-data.sdis.isere.fr Uuid: eddb3c6d-2e98-45ca-bd1f-6d2153bbb60e State: Peer in Cluster (Connected) Hostname: serv-vm-al06-data.sdis.isere.fr Uuid: cafefdf3-ffc3-4589-abf6-6ca76905593b State: Peer in Cluster (Connected) On the two other nodes, the same command output is OK.

If I understand correctly: node1 - mgmt.ip.1 & gluster.ip.1 node2 - mgmt.ip.2 & gluster.ip.2 node3 - mgmt.ip.3 & gluster.ip.3 Right Did you create a network and assign "gluster" role to it in the cluster? I created a gluster network, but did not assign the gluster role so far, as my former 3 hosts had no dedicated NIC not ip for that. I planned to assign this role once my 3 new hosts were part of the game. Were you able to add the first node to cluster No , and got this error on second node addition ? I had the error when trying to add the first node. From the error, it looks like oVirt does not understand the peer list returned from gluster is a match with node being added. Sounds correct Please provide the log snippet of the failure (from engine.log as well as vdsm.log on node) See attached file

> I found no idea when googling, except something related to gluster (you > bet!), telling this may be related to the fact that there is already a > volume, managed with a different name. > > Obviously, using a different name and IP is what I needed! > I used "transport.socket.bind-address" to make sure the gluster traffic > will only use the dedicated NICs. > > > Well, I also tried to created a storage domain relying on the freshly > created gluster volume, but as this subnet is not routed, it is not > reachable from the manager nor the existing SPM. > The existing SPM - isn't it one of the the 3 new nodes being added? No, the SPM is one of the 3 former and still existing hosts. Or are you adding the 3 nodes to your existing cluster? If so, I suggest you try adding this to a new cluster OK, I tried and succeed to create a new cluster. In this new cluster, I was ABLE to add the first new host, using its mgmt DNS name. This first host still has to have its NICs configured, and (using Chrome or FF) the access to the network settings window is stalling the browser (I tried to restart even the engine, to no avail). Thus, I can not setup this first node NICs. Thus, I can not add any further host because oVirt relies on a first host to validate the further ones.

-- Nicolas ECARNOT