[ovirt-users] Dedicated NICs for gluster network

[Sahina Bose]

On Fri, Aug 19, 2016 at 2:33 PM, Nicolas Ecarnot <nicolas at ecarnot.net>
wrote:

> Le 19/08/2016 à 09:55, Sahina Bose a écrit :
>
>
>
> On Fri, Aug 19, 2016 at 12:29 PM, Nicolas Ecarnot <nicolas at ecarnot.net>
> wrote:
>
>> Hello,
>>
>> I'm digging out this thread because I now had the time to work on this
>> subject, and I'm stuck.
>>
>> This oVirt setup has a standalone engine, and 3 hosts.
>> These 3 hosts are hypervisors and gluster nodes, each using one NIC for
>> all the traffic, that is a very bad idea. (Well, it's working, but not
>> recommended).
>>
>> I added 3 OTHER nodes, and so far, I only created the gluster setup and
>> created a replica-3 volume.
>> Each of these new nodes now have one NIC for management, one NIC for
>> gluster, and other NICs for other things.
>> Each NIC has an IP + DNS name in its dedicated VLAN : one for mgmt and
>> one for gluster.
>> The mgmt subnet is routed, though the gluster subnet is not.
>> Every node can ping each other, either using the mgmt or the gluster
>> subnets.
>>
>> The creation of the gluster subnet and volume went very well and seems to
>> be perfect.
>>
>> Now, in the oVirt web gui, I'm trying to add these nodes as oVirt hosts.
>> I'm using their mgmt DNS names, and I'm getting :
>> "Error while executing action: Server xxxxxxxx is already part of another
>> cluster."
>>
>
> Did you peer probe the gluster cluster prior to adding the nodes to oVirt?
>
>
> Yes, and using their "gluster subnet" names.
> It went fine.
>
> What's the output of "gluster peer status"
>
>
> [root at serv-vm-al04 log]# gluster peer status
>
> Number of Peers: 2
>
> Hostname: serv-vm-al05-data.sdis.isere.fr
>
> Uuid: eddb3c6d-2e98-45ca-bd1f-6d2153bbb60e
>
> State: Peer in Cluster (Connected)
>
> Hostname: serv-vm-al06-data.sdis.isere.fr
>
> Uuid: cafefdf3-ffc3-4589-abf6-6ca76905593b
>
> State: Peer in Cluster (Connected)
>
>
> On the two other nodes, the same command output is OK.
>


>
>
> If I understand correctly:
> node1 - mgmt.ip.1 & gluster.ip.1
> node2 - mgmt.ip.2 & gluster.ip.2
> node3 - mgmt.ip.3 & gluster.ip.3
>
> Right
>
>
> Did you create a network and assign "gluster" role to it in the cluster?
>
> I created a gluster network, but did not assign the gluster role so far,
> as my former 3 hosts had no dedicated NIC not ip for that.
> I planned to assign this role once my 3 new hosts were part of the game.
>
> Were you able to add the first node to cluster
>
> No
>
> , and got this error on second node addition ?
>
> I had the error when trying to add the first node.
>
> From the error, it looks like oVirt does not understand the peer list
> returned from gluster is a match with node being added.
>
> Sounds correct
>
> Please provide the log snippet of the failure (from engine.log as well as
> vdsm.log on node)
>
> See attached file
>



I couldn't view the attached log files for some reason, but the issue is
that you're adding the node which is part of a cluster to an existing
cluster. That will not work, even from gluster CLI


>
>
>> I found no idea when googling, except something related to gluster (you
>> bet!), telling this may be related to the fact that there is already a
>> volume, managed with a different name.
>>
>> Obviously, using a different name and IP is what I needed!
>> I used "transport.socket.bind-address" to make sure the gluster traffic
>> will only use the dedicated NICs.
>>
>>
>> Well, I also tried to created a storage domain relying on the freshly
>> created gluster volume, but as this subnet is not routed, it is not
>> reachable from the manager nor the existing SPM.
>>
>
> The existing SPM - isn't it one of the the 3 new nodes being added?
>
> No, the SPM is one of the 3 former and still existing hosts.
>
> Or are you adding the 3 nodes to your existing cluster? If so, I suggest
> you try adding this to a new cluster
>
> OK, I tried and succeed to create a new cluster.
> In this new cluster, I was ABLE to add the first new host, using its mgmt
> DNS name.
> This first host still has to have its NICs configured, and (using Chrome
> or FF) the access to the network settings window is stalling the browser (I
> tried to restart even the engine, to no avail). Thus, I can not setup this
> first node NICs.
>
> Thus, I can not add any further host because oVirt relies on a first host
> to validate the further ones.
>


Network team should be able to help you here.


>
> --
> Nicolas ECARNOT
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160819/93d0d349/attachment.html>