<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 19, 2016 at 2:33 PM, Nicolas Ecarnot <span dir="ltr"><<a href="mailto:nicolas@ecarnot.net" target="_blank">nicolas@ecarnot.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span class="">
<div>Le 19/08/2016 à 09:55, Sahina Bose a
écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Aug 19, 2016 at 12:29 PM,
Nicolas Ecarnot <span dir="ltr"><<a href="mailto:nicolas@ecarnot.net" target="_blank">nicolas@ecarnot.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello,<br>
<br>
I'm digging out this thread because I now had the time to
work on this subject, and I'm stuck.<br>
<br>
This oVirt setup has a standalone engine, and 3 hosts.<br>
These 3 hosts are hypervisors and gluster nodes, each
using one NIC for all the traffic, that is a very bad
idea. (Well, it's working, but not recommended).<br>
<br>
I added 3 OTHER nodes, and so far, I only created the
gluster setup and created a replica-3 volume.<br>
Each of these new nodes now have one NIC for management,
one NIC for gluster, and other NICs for other things.<br>
Each NIC has an IP + DNS name in its dedicated VLAN : one
for mgmt and one for gluster.<br>
The mgmt subnet is routed, though the gluster subnet is
not.<br>
Every node can ping each other, either using the mgmt or
the gluster subnets.<br>
<br>
The creation of the gluster subnet and volume went very
well and seems to be perfect.<br>
<br>
Now, in the oVirt web gui, I'm trying to add these nodes
as oVirt hosts.<br>
I'm using their mgmt DNS names, and I'm getting :<br>
"Error while executing action: Server xxxxxxxx is already
part of another cluster."<br>
</blockquote>
<div><br>
</div>
<div>Did you peer probe the gluster cluster prior to adding
the nodes to oVirt?</div>
</div>
</div>
</div>
</blockquote>
<br></span>
Yes, and using their "gluster subnet" names.<br>
It went fine.<span class=""><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> What's the output of "gluster peer status"<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</span><pre>[root@serv-vm-al04 log]# gluster peer status</pre>
<pre>Number of Peers: 2</pre>
<pre>Hostname: <a href="http://serv-vm-al05-data.sdis.isere.fr" target="_blank">serv-vm-al05-data.sdis.isere.<wbr>fr</a></pre>
<pre>Uuid: eddb3c6d-2e98-45ca-bd1f-<wbr>6d2153bbb60e</pre>
<pre>State: Peer in Cluster (Connected)</pre>
<pre>Hostname: <a href="http://serv-vm-al06-data.sdis.isere.fr" target="_blank">serv-vm-al06-data.sdis.isere.<wbr>fr</a></pre>
<pre>Uuid: cafefdf3-ffc3-4589-abf6-<wbr>6ca76905593b</pre>
<pre>State: Peer in Cluster (Connected)</pre>
<br>
On the two other nodes, the same command output is OK.<span class=""><br></span></div></blockquote> <br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><span class="">
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>If I understand correctly:<br>
</div>
<div>node1 - mgmt.ip.1 & gluster.ip.1<br>
</div>
<div>node2 - mgmt.ip.2 & gluster.ip.2<br>
</div>
<div>node3 - mgmt.ip.3 & gluster.ip.3<br>
</div>
</div>
</div>
</div>
</blockquote></span>
Right<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>Did you create a network and assign "gluster" role to
it in the cluster?</div>
</div>
</div>
</div>
</blockquote></span>
I created a gluster network, but did not assign the gluster role so
far, as my former 3 hosts had no dedicated NIC not ip for that.<br>
I planned to assign this role once my 3 new hosts were part of the
game.<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> Were you able to add the first node to cluster</div>
</div>
</div>
</div>
</blockquote></span>
No<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>, and got this error on second node addition ?<br>
</div>
</div>
</div>
</div>
</blockquote></span>
I had the error when trying to add the first node.<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>From the error, it looks like oVirt does not understand
the peer list returned from gluster is a match with node
being added. <br>
</div>
</div>
</div>
</div>
</blockquote></span>
Sounds correct<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>Please provide the log snippet of the failure (from
engine.log as well as vdsm.log on node)<br>
</div>
</div>
</div>
</div>
</blockquote></span>
See attached file<span class=""><br></span></div></blockquote><div><br><br><div><br></div>I couldn't view the attached log files for some reason, but the issue is that you're adding the node which is part of a cluster to an existing cluster. That will not work, even from gluster CLI<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><span class="">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
I found no idea when googling, except something related to
gluster (you bet!), telling this may be related to the
fact that there is already a volume, managed with a
different name.<br>
<br>
Obviously, using a different name and IP is what I needed!<br>
I used "transport.socket.bind-address<wbr>" to make sure
the gluster traffic will only use the dedicated NICs.<br>
<br>
<br>
Well, I also tried to created a storage domain relying on
the freshly created gluster volume, but as this subnet is
not routed, it is not reachable from the manager nor the
existing SPM.<br>
</blockquote>
<div><br>
</div>
<div>The existing SPM - isn't it one of the the 3 new nodes
being added?</div>
</div>
</div>
</div>
</blockquote></span>
No, the SPM is one of the 3 former and still existing hosts.<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> Or are you adding the 3 nodes to your existing
cluster? If so, I suggest you try adding this to a new
cluster<br>
</div>
</div>
</div>
</div>
</blockquote></span>
OK, I tried and succeed to create a new cluster.<br>
In this new cluster, I was ABLE to add the first new host, using its
mgmt DNS name.<br>
This first host still has to have its NICs configured, and (using
Chrome or FF) the access to the network settings window is stalling
the browser (I tried to restart even the engine, to no avail). Thus,
I can not setup this first node NICs.<br>
<br>
Thus, I can not add any further host because oVirt relies on a first
host to validate the further ones.<span class=""><font color="#888888"><br></font></span></div></blockquote><div><br><br></div><div>Network team should be able to help you here.<br> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><span class=""><font color="#888888">
<br>
-- <br>
Nicolas ECARNOT<br>
</font></span></div>
</blockquote></div><br></div></div>