This is a multi-part message in MIME format.
--------------D8CDA99B33546FA7B54F7704
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable
Le 16/12/2016 =C3=A0 16:34, Sahina Bose a =C3=A9crit :
Failed to find host=20
'Host[guadalupe1,7a30c899-a317-479a-b07b-244bc2374485]' in gluster=20
peer list from 'Host[guadalupe1,7a30c899-a317-479a-b07b-244bc2374485]'=20
on attempt 2
It looks the gluster uuid saved in the ovirt engine db does not match=20
the one returned from CLI
Was this host reinstalled?
You may need to remove host from engine and add it again. If that=20
doesn't work you may need to manually change the uuid value in the=20
database (gluster_server table)
Removing host did nothing, indeed I had to go to
the gluster_server=20
table to remove any disconnected host uuid, but it was not enough. Then=20
I had then to remove the host and reinstall it as a new host.
Thank you, I've been spending a lot of time to solve this issue.
On Fri, Dec 16, 2016 at 7:00 PM, Nathana=C3=ABl Blanchet <blanchet@abes=
.fr=20
<mailto:blanchet@abes.fr>> wrote:
extract of the last engine logs, thank you
Le 16/12/2016 =C3=A0 14:02, Sahina Bose a =C3=A9crit :
> Could you attach the engine log with this error?
> On Fri, Dec 16, 2016 at 4:29 PM, Nathana=C3=ABl
Blanchet
> <blanchet(a)abes.fr <mailto:blanchet@abes.fr>> wrote:
> Hi,
> I used to successfully run a replica 3 gluster
volume, but
> since the last 4.0.5 update, they can't connect each other
> with the message : gluster [gluster peer status
> guadalupe1.v100.abes.fr <
http://guadalupe1.v100.abes.fr>]
> command failed on server guadalupe2.v100.abes.fr
> <
http://guadalupe2.v100.abes.fr>.
> So host guadalupe1 can't never be up.
> When doing gluster peer probe, they are connected
as
> expected. I reinstalled vdsm and gluster, but it is still the
> same.
> I found this on guadalupe2 supervdsm.log
> MainProcess|jsonrpc.Executor/6::DEBUG::2016-12-16
> 11:53:21,429::supervdsmServer::99::SuperVdsm.ServerCallback::(=
wrapper)
> return peerStatus with [{'status':
'CONNECTED', 'hostname':
> '10.34.101.56/24 <
http://10.34.101.56/24>';, 'uuid':
> 'c259c09b-8d7c-4b12-8745-677199877583'}, {'status':
> 'CONNECTED', 'hostname': 'guadalupe3.v100.abes.fr
> <
http://guadalupe3.v100.abes.fr>';, 'uuid':
> '6af67cd3-7931-446d-aaa2-ffea51325adc'}, {'status':
> 'CONNECTED', 'hostname': 'guadalupe1.v100.abes.fr
> <
http://guadalupe1.v100.abes.fr>';, 'uuid':
> '8eb485cd-31c4-4c3a-a315-3dc6d3ddc0c9'}]
> MainProcess|jsonrpc.Executor/7::DEBUG::2016-12-16
> 11:53:21,490::supervdsmServer::92::SuperVdsm.ServerCallback::(=
wrapper)
> call peerProbe with () {}
> MainProcess|jsonrpc.Executor/7::DEBUG::2016-12-16
> 11:53:21,491::commands::68::root::(execCmd) /usr/bin/taskset
> --cpu-list 0-63 /usr/sbin/gluster --mode=3Dscript peer probe
> guadalupe1.v100.abes.fr <
http://guadalupe1.v100.abes.fr
> --xml (cwd None)
> MainProcess|jsonrpc.Executor/7::DEBUG::2016-12-16
> 11:53:21,570::commands::86::root::(execCmd) SUCCESS: <err> =3D
> ''; <rc> =3D 0
> MainProcess|jsonrpc.Executor/7::DEBUG::2016-12-16
> 11:53:21,570::supervdsmServer::99::SuperVdsm.ServerCallback::(=
wrapper)
> return peerProbe with True
> We can see guadalupe2 can see guadalupe1 but
taskset still
> executes peer probe to guadalupe1 with message "Host
> guadalupe1.v100.abes.fr <
http://guadalupe1.v100.abes.fr> port
> 24007 already in peer list"
> How can I say to guadalupe2 stop trying to probe
guadalupe1?
> --=20
> Nathana=C3=ABl Blanchet
> Supervision r=C3=A9seau
> P=C3=B4le Infrastrutures Informatiques
> 227 avenue Professeur-Jean-Louis-Viala
> 34193 MONTPELLIER CEDEX 5
> T=C3=A9l. 33 (0)4 67 54 84 55
> Fax 33 (0)4 67 54 84 14
> blanchet(a)abes.fr <mailto:blanchet@abes.fr
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org <mailto:Users@ovirt.org
>
http://lists.ovirt.org/mailman/listinfo/users
> <
http://lists.ovirt.org/mailman/listinfo/users
--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr <mailto:blanchet@abes.fr> =20
--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
--------------D8CDA99B33546FA7B54F7704
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html
<head
<meta content=3D"text/html; charset=3Dutf-8"
http-equiv=3D"Content-Ty=
pe"
</head
<body text=3D"#000000"
bgcolor=3D"#FFFFFF"
<p><br
</p
<br
<div class=3D"moz-cite-prefix">Le
16/12/2016 =C3=A0 16:34, Sahina Bos=
e a
=C3=A9crit=C2=A0:<br
</div
<blockquote
cite=3D"mid:CACjzOvd1bZgvaYnpK-yO+yYeFoD0B_G4rHP9e2diQy+4Wgq07g@mail.gmai=
l.com"
type=3D"cite"
<div
dir=3D"ltr"
<div
<div>Failed to find host
'Host[guadalupe1,7a30c899-a317-479a-b07b-244bc2374485]' in
gluster peer list from
'Host[guadalupe1,7a30c899-a317-479a-b07b-244bc2374485]' on
attempt 2<br
It looks the gluster
uuid=C2=A0 saved in the ovirt engine db =
does
not match the one returned from CLI<br
<br
</div
Was this host reinstalled? <br
</div
<div>You may need to remove host from engine and add it again.
If that doesn't work you may need to manually change the uuid
value in the database (gluster_server table)<br
</div
</div
</blockquote
Removing host did nothing, indeed I had to go to the gluster_server
table to remove any disconnected host uuid, but it was not enough.
Then I had then to remove the host and reinstall it as a new host.<br=
Thank you, I've been spending a lot of time to solve
this issue.<br
<blockquote
cite=3D"mid:CACjzOvd1bZgvaYnpK-yO+yYeFoD0B_G4rHP9e2diQy+4Wgq07g@mail.gmai=
l.com"
type=3D"cite"
<div
class=3D"gmail_extra"><br
<div class=3D"gmail_quote">On Fri, Dec 16, 2016 at 7:00 PM,
Nathana=C3=ABl Blanchet <span dir=3D"ltr"><<a
moz-do-not-send=3D"true"
href=3D"mailto:blanchet@abes.fr"
target=3D"_blank"><a
class=3D"moz-txt-link-abbreviated" hre=
f=3D"mailto:blanchet@abes.fr">blanchet@abes.fr</a></a>></span>
wrote:<=
br
<blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"
<div text=3D"#000000"
bgcolor=3D"#FFFFFF"> extract of the las=
t
engine logs, thank you
<div
<div
class=3D"h5"><br
<br
<div
class=3D"m_8928738385687730066moz-cite-prefix">Le
16/12/2016 =C3=A0 14:02, Sahina Bose a =C3=A9crit=C2=A0=
:<br
</div
<blockquote type=3D"cite"
<div dir=3D"ltr">Could you attach the engine log with
this error?<br
</div
<div
class=3D"gmail_extra"><br
<div class=3D"gmail_quote">On Fri, Dec 16, 2016 at
4:29 PM, Nathana=C3=ABl Blanchet <span dir=3D"ltr=
"><<a
moz-do-not-send=3D"true"
class=3D"m_8928738385687730066moz-txt-link-ab=
breviated"
href=3D"mailto:blanchet@abes.fr"
target=3D"_blank"><a
class=3D"moz-txt-link-ab=
breviated"
href=3D"mailto:blanchet@abes.fr">blanchet@abes.fr</a></a>><=
/span
wrote:<br
<blockquote
class=3D"gmail_quote" style=3D"margin=
:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">Hi,<br
<br
I used to successfully run a replica 3 gluster
volume, but since the last 4.0.5 update, they
can't connect each other with the message :
gluster [gluster peer status <a
moz-do-not-send=3D"true"
href=3D"http://guadalupe1.v100.abes.fr"
rel=3D"noreferrer"
target=3D"_blank">guadalup=
e1.v100.abes.fr</a>]
command failed on server <a
moz-do-not-send=3D"true"
href=3D"http://guadalupe2.v100.abes.fr"
rel=3D"noreferrer"
target=3D"_blank">guadalup=
e2.v100.abes.fr</a>.<br
<br
So host guadalupe1 can't never be
up.<br
<br
When doing gluster peer probe, they
are
connected as expected. I reinstalled vdsm and
gluster, but it is still the same.<br
<br
I found this on guadalupe2 supervdsm.log<br
<br
MainProcess|jsonrpc.Executor/6<wbr>::DEBUG::201=
6-12-16
11:53:21,429::supervdsmServer:<wbr>:99::SuperVd=
sm.ServerCallback:<wbr>:(wrapper)
return peerStatus with [{'status':
'CONNECTED', 'hostname': '<a
moz-do-not-send=3D"true"
href=3D"http://10.34.101.56/24"
rel=3D"noreferrer"
target=3D"_blank">10.34.10=
1.56/24</a>',
'uuid':
'c259c09b-8d7c-4b12-8745-67719<wbr>9877=
583'},
{'status': 'CONNECTED', 'hostname':
'<a
moz-do-not-send=3D"true"
href=3D"http://guadalupe3.v100.abes.fr"
rel=3D"noreferrer"
target=3D"_blank">guadalup=
e3.v100.abes.fr</a>',
'uuid':
'6af67cd3-7931-446d-aaa2-ffea5<wbr>1325=
adc'},
{'status': 'CONNECTED', 'hostname':
'<a
moz-do-not-send=3D"true"
href=3D"http://guadalupe1.v100.abes.fr"
rel=3D"noreferrer"
target=3D"_blank">guadalup=
e1.v100.abes.fr</a>',
'uuid':
'8eb485cd-31c4-4c3a-a315-3dc6d<wbr>3ddc=
0c9'}]<br
MainProcess|jsonrpc.Executor/7<wbr>::DEBUG::201=
6-12-16
11:53:21,490::supervdsmServer:<wbr>:92::SuperVd=
sm.ServerCallback:<wbr>:(wrapper)
call peerProbe with () {}<br
MainProcess|jsonrpc.Executor/7<wbr>::DEBUG::201=
6-12-16
11:53:21,491::commands::68::ro<wbr>ot::(execCmd=
)
/usr/bin/taskset --cpu-list 0-63
/usr/sbin/gluster --mode=3Dscript peer probe <a
moz-do-not-send=3D"true"
href=3D"http://guadalupe1.v100.abes.fr"
rel=3D"noreferrer"
target=3D"_blank">guadalup=
e1.v100.abes.fr</a
--xml (cwd
None)<br
MainProcess|jsonrpc.Executor/7<wbr>::DEBUG::201=
6-12-16
11:53:21,570::commands::86::ro<wbr>ot::(execCmd=
)
SUCCESS: <err> =3D ''; <rc>
=3D 0<b=
r
MainProcess|jsonrpc.Executor/7<wbr>::DEBUG::201=
6-12-16
11:53:21,570::supervdsmServer:<wbr>:99::SuperVd=
sm.ServerCallback:<wbr>:(wrapper)
return peerProbe with True<br
<br
We can see
guadalupe2 can see guadalupe1 but
taskset still executes peer probe to
guadalupe1 with message "Host <a
moz-do-not-send=3D"true"
href=3D"http://guadalupe1.v100.abes.fr"
rel=3D"noreferrer"
target=3D"_blank">guadalup=
e1.v100.abes.fr</a
port 24007
already in peer list"<br
<br
How can I say to guadalupe2 stop
trying to
probe guadalupe1?<br
<br
<br
-- <br
Nathana=C3=ABl Blanchet<br
<br
Supervision r=C3=A9seau<br
P=C3=B4le Infrastrutures Informatiques<br
227 avenue Professeur-Jean-Louis-Viala<br
34193 MONTPELLIER CEDEX 5=C2=A0 =C2=A0 =C2=A0 =C2=
=A0<br
T=C3=A9l.
33 (0)4 67 54 84 55<br
Fax=C2=A0
33 (0)4 67 54 84 14<br
<a
moz-do-not-send=3D"true"
href=3D"mailto:blanchet@abes.fr"
target=3D"_blank">blanchet(a)abes.fr</a><br
<br
______________________________<wbr>____________=
_____<br
Users
mailing list<br
<a
moz-do-not-send=3D"true"
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br
<a
moz-do-not-send=3D"true"
href=3D"http://lists.ovirt.org/mailman/listin=
fo/users"
rel=3D"noreferrer"
target=3D"_blank">http://l=
ists.ovirt.org/mailman<wbr>/listinfo/users</a><br
</blockquote
</div
<br
</div
</blockquote
<br
<pre
class=3D"m_8928738385687730066moz-signature" cols=3D=
"72">--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
<a moz-do-not-send=3D"true"
class=3D"m_8928738385687730066moz-txt-link-ab=
breviated" href=3D"mailto:blanchet@abes.fr"
target=3D"_blank">blanchet@ab=
es.fr</a> </pre
</div
</div
</div
</blockquote
</div
<br
</div
</blockquote
<br
<pre class=3D"moz-signature"
cols=3D"72">--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
<a class=3D"moz-txt-link-abbreviated"
href=3D"mailto:blanchet@abes.fr">bl=
anchet(a)abes.fr</a> </pre
</body
</html
--------------D8CDA99B33546FA7B54F7704--