4:52 a.m.
This is a multi-part message in MIME format.
--------------90504E1C27EE95060580B78B
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hello,
I have set up 3 hosts for engine, 2 of them are working correct. There
is no other host even having broker/agent installed. Is it possible that
the error occurs because the hosts are multihomed (Management IP, IP for
storage) and can communicate with different IP's ?
hosted-engine --vm-status on both working hosts seems correct: (3 is out
of order...)
[root@microcloud21 ~]# hosted-engine --vm-status
--=3D=3D Host 1 status =3D=3D--
conf_on_shared_storage : True
Status up-to-date : True
Hostname : microcloud21.sub.mydomain.de
Host ID : 1
Engine status : {"health": "good",
"vm": "up",
"detail": "up"}
Score : 3400
stopped : False
Local maintenance : False
crc32 : 5941227d
local_conf_timestamp : 152316
Host timestamp : 152302
Extra metadata (valid at timestamp):
metadata_parse_version=3D1
metadata_feature_version=3D1
timestamp=3D152302 (Sat Feb 4 11:49:29 2017)
host-id=3D1
score=3D3400
vm_conf_refresh_time=3D152316 (Sat Feb 4 11:49:43 2017)
conf_on_shared_storage=3DTrue
maintenance=3DFalse
state=3DEngineUp
stopped=3DFalse
--=3D=3D Host 2 status =3D=3D--
conf_on_shared_storage : True
Status up-to-date : True
Hostname : microcloud24.sub.mydomain.de
Host ID : 2
Engine status : {"reason": "vm not running on this
host", "health": "bad", "
vm": "down",
"detail": "unknown"}
Score : 3400
stopped : False
Local maintenance : False
crc32 : 77e25433
local_conf_timestamp : 157637
Host timestamp : 157623
Extra metadata (valid at timestamp):
metadata_parse_version=3D1
metadata_feature_version=3D1
timestamp=3D157623 (Sat Feb 4 11:49:34 2017)
host-id=3D2
score=3D3400
vm_conf_refresh_time=3D157637 (Sat Feb 4 11:49:48 2017)
conf_on_shared_storage=3DTrue
maintenance=3DFalse
state=3DEngineDown
stopped=3DFalse
--=3D=3D Host 3 status =3D=3D--
conf_on_shared_storage : True
Status up-to-date : False
Hostname : microcloud27.sub.mydomain.de
Host ID : 3
Engine status : unknown stale-data
Score : 0
stopped : True
Local maintenance : False
crc32 : 74798986
local_conf_timestamp : 77946
Host timestamp : 77932
Extra metadata (valid at timestamp):
metadata_parse_version=3D1
metadata_feature_version=3D1
timestamp=3D77932 (Fri Feb 3 15:19:25 2017)
host-id=3D3
score=3D0
vm_conf_refresh_time=3D77946 (Fri Feb 3 15:19:39 2017)
conf_on_shared_storage=3DTrue
maintenance=3DFalse
state=3DAgentStopped
stopped=3DTrue
Am 03.02.2017 um 19:20 schrieb Simone Tiraboschi:
On Fri, Feb 3, 2017 at 5:22 PM, Ralf Schenk <rs(a)databay.de
<mailto:rs@databay.de>> wrote:
Hello,
of course:
[root@microcloud27 mnt]# sanlock client status
daemon 8a93c9ea-e242-408c-a63d-a9356bb22df5.microcloud
p -1 helper
p -1 listener
p -1 status
sanlock.log attached. (Beginning 2017-01-27 where everything was fi=
ne)
Thanks, the issue is here:
2017-02-02 19:01:22+0100 4848 [1048]: s36 lockspace 7c8deaa8-be02-4aaf-=
b9b4-ddc8da99ad96:3:/rhev/data-center/mnt/glusterSD/glusterfs.sub.mydomai=
n.de:_engine/7c8deaa8-be02-4aaf-b9b4-ddc8da99ad96/dom_md/ids:0
2017-02-02 19:03:42+0100 4988 [12983]: s36 delta_acquire host_id 3
busy=
1 3 15 13129 7ad427b1-fbb6-4cee-b9ee-01f596fddfbb.microcloud
2017-02-02 19:03:43+0100 4989 [1048]: s36 add_lockspace fail result
-26=
2
Could you please check if you have other hosts contending for the
same
ID (id=3D3 in this case).
=20
--=20
*Ralf Schenk*
fon +49 (0) 24 05 / 40 83 70
fax +49 (0) 24 05 / 40 83 759
mail *rs(a)databay.de* <mailto:rs@databay.de>
=09
*Databay AG*
Jens-Otto-Krag-Stra=C3=9Fe 11
D-52146 W=C3=BCrselen
*www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen =E2=80=A2 HRB:8437 =E2=80=A2 USt-IdNr.: DE 210844=
202
Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm.
Philipp Hermanns
Aufsichtsratsvorsitzender: Klaus Scholzen (RA)
------------------------------------------------------------------------
--------------90504E1C27EE95060580B78B
Content-Type: multipart/related;
boundary="------------AADA2BBC8C36F1CE2A14BB50"
--------------AADA2BBC8C36F1CE2A14BB50
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hello,</p>
<p>I have set up 3 hosts for engine, 2 of them are working correct.
There is no other host even having broker/agent installed. Is it
possible that the error occurs because the hosts are multihomed
(Management IP, IP for storage) and can communicate with different
IP's ?</p>
<p>hosted-engine --vm-status on both working hosts seems correct: (3
is out of order...)<br>
</p>
<p><tt>[root@microcloud21 ~]# hosted-engine
--vm-status</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>--== Host 1 status ==--</tt><tt><br>
</tt><tt><br>
</tt><tt>conf_on_shared_storage :
True</tt><tt><br>
</tt><tt>Status up-to-date :
True</tt><tt><br>
</tt><tt>Hostname :
microcloud21.sub.mydomain.de</tt><tt><br>
</tt><tt>Host ID :
1</tt><tt><br>
</tt><tt>Engine status : {"health":
"good",
"vm": "up", "detail":
"up"}</tt><tt><br>
</tt><tt>Score :
3400</tt><tt><br>
</tt><tt>stopped :
False</tt><tt><br>
</tt><tt>Local maintenance :
False</tt><tt><br>
</tt><tt>crc32 :
5941227d</tt><tt><br>
</tt><tt>local_conf_timestamp :
152316</tt><tt><br>
</tt><tt>Host timestamp :
152302</tt><tt><br>
</tt><tt>Extra metadata (valid at
timestamp):</tt><tt><br>
</tt><tt>
metadata_parse_version=1</tt><tt><br>
</tt><tt>
metadata_feature_version=1</tt><tt><br>
</tt><tt> timestamp=152302 (Sat Feb 4 11:49:29
2017)</tt><tt><br>
</tt><tt> host-id=1</tt><tt><br>
</tt><tt> score=3400</tt><tt><br>
</tt><tt> vm_conf_refresh_time=152316 (Sat Feb 4 11:49:43
2017)</tt><tt><br>
</tt><tt>
conf_on_shared_storage=True</tt><tt><br>
</tt><tt> maintenance=False</tt><tt><br>
</tt><tt> state=EngineUp</tt><tt><br>
</tt><tt> stopped=False</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>--== Host 2 status ==--</tt><tt><br>
</tt><tt><br>
</tt><tt>conf_on_shared_storage :
True</tt><tt><br>
</tt><tt>Status up-to-date :
True</tt><tt><br>
</tt><tt>Hostname :
microcloud24.sub.mydomain.de</tt><tt><br>
</tt><tt>Host ID :
2</tt><tt><br>
</tt><tt>Engine status : {"reason":
"vm not
running on this host", "health": "bad",
" vm": "down",
"detail":
"unknown"}</tt><tt><br>
</tt><tt>Score :
3400</tt><tt><br>
</tt><tt>stopped :
False</tt><tt><br>
</tt><tt>Local maintenance :
False</tt><tt><br>
</tt><tt>crc32 :
77e25433</tt><tt><br>
</tt><tt>local_conf_timestamp :
157637</tt><tt><br>
</tt><tt>Host timestamp :
157623</tt><tt><br>
</tt><tt>Extra metadata (valid at
timestamp):</tt><tt><br>
</tt><tt>
metadata_parse_version=1</tt><tt><br>
</tt><tt>
metadata_feature_version=1</tt><tt><br>
</tt><tt> timestamp=157623 (Sat Feb 4 11:49:34
2017)</tt><tt><br>
</tt><tt> host-id=2</tt><tt><br>
</tt><tt> score=3400</tt><tt><br>
</tt><tt> vm_conf_refresh_time=157637 (Sat Feb 4 11:49:48
2017)</tt><tt><br>
</tt><tt>
conf_on_shared_storage=True</tt><tt><br>
</tt><tt> maintenance=False</tt><tt><br>
</tt><tt> state=EngineDown</tt><tt><br>
</tt><tt> stopped=False</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>--== Host 3 status ==--</tt><tt><br>
</tt><tt><br>
</tt><tt>conf_on_shared_storage :
True</tt><tt><br>
</tt><tt>Status up-to-date :
False</tt><tt><br>
</tt><tt>Hostname :
microcloud27.sub.mydomain.de</tt><tt><br>
</tt><tt>Host ID :
3</tt><tt><br>
</tt><tt>Engine status : unknown
stale-data</tt><tt><br>
</tt><tt>Score :
0</tt><tt><br>
</tt><tt>stopped :
True</tt><tt><br>
</tt><tt>Local maintenance :
False</tt><tt><br>
</tt><tt>crc32 :
74798986</tt><tt><br>
</tt><tt>local_conf_timestamp :
77946</tt><tt><br>
</tt><tt>Host timestamp :
77932</tt><tt><br>
</tt><tt>Extra metadata (valid at
timestamp):</tt><tt><br>
</tt><tt>
metadata_parse_version=1</tt><tt><br>
</tt><tt>
metadata_feature_version=1</tt><tt><br>
</tt><tt> timestamp=77932 (Fri Feb 3 15:19:25
2017)</tt><tt><br>
</tt><tt> host-id=3</tt><tt><br>
</tt><tt> score=0</tt><tt><br>
</tt><tt> vm_conf_refresh_time=77946 (Fri Feb 3 15:19:39
2017)</tt><tt><br>
</tt><tt>
conf_on_shared_storage=True</tt><tt><br>
</tt><tt> maintenance=False</tt><tt><br>
</tt><tt> state=AgentStopped</tt><tt><br>
</tt><tt> stopped=True</tt><tt><br>
</tt><br>
</p>
<p>Am 03.02.2017 um 19:20 schrieb Simone Tiraboschi:<br>
</p>
<blockquote
cite="mid:CAN8-ONo3+L+R3iPr1DoS0s8Fn3kZNkekKrP4ipyKPs_e_2jhiQ@mail.gmail.com"
type="cite"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Feb 3, 2017 at 5:22 PM, Ralf
Schenk <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:rs@databay.de"
target="_blank">rs(a)databay.de</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hello,</p>
<p>of course:</p>
<p>[root@microcloud27 mnt]# sanlock client status<br>
daemon
8a93c9ea-e242-408c-a63d-<wbr>a9356bb22df5.microcloud<br>
p -1 helper<br>
p -1 listener<br>
p -1 status<br>
</p>
<p>sanlock.log attached. (Beginning 2017-01-27 where
everything was fine)</p>
</div>
</blockquote>
Thanks, the issue is here:
<div>
<pre class="gmail-aLF-aPX-K0-aPE
gmail-aLF-aPX-aLK-ayr-auR">2017-02-02 19:01:22+0100 4848 [1048]: s36 lockspace
7c8deaa8-be02-4aaf-b9b4-ddc8da99ad96:3:/rhev/data-center/mnt/glusterSD/glusterfs.sub.mydomain.de:_engine/7c8deaa8-be02-4aaf-b9b4-ddc8da99ad96/dom_md/ids:0
2017-02-02 19:03:42+0100 4988 [12983]: s36 delta_acquire host_id 3 busy1 3 15 13129
7ad427b1-fbb6-4cee-b9ee-01f596fddfbb.microcloud
2017-02-02 19:03:43+0100 4989 [1048]: s36 add_lockspace fail result -262</pre>
</div>
<div>Could you please check if you have other hosts contending
for the same ID (id=3 in this case).</div>
<div> </div>
</div>
</div>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<p>
</p>
<table border="0" cellpadding="0"
cellspacing="0">
<tbody>
<tr>
<td colspan="3"> <img title="" alt=""
src="cid:part2.0624039A.C0EFA82C@databay.de"
height="30"
border="0" width="151"> </td>
</tr>
<tr>
<td valign="top"> <font face="Verdana, Arial,
sans-serif"
size="-1"><br>
<b>Ralf Schenk</b><br>
fon +49 (0) 24 05 / 40 83 70<br>
fax +49 (0) 24 05 / 40 83 759<br>
mail <a href="mailto:rs@databay.de"><font
color="#FF0000"><b>rs(a)databay.de</b></font></a><br>
</font> </td>
<td width="30"> </td>
<td valign="top"> <font face="Verdana, Arial,
sans-serif"
size="-1"><br>
<b>Databay AG</b><br>
Jens-Otto-Krag-Straße 11<br>
D-52146 Würselen<br>
<a href="http://www.databay.de"><font
color="#FF0000"><b>www.databay.de</b></font></a>
</font> </td>
</tr>
<tr>
<td colspan="3" valign="top"> <font
face="Verdana, Arial,
sans-serif" size="1"><br>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE
210844202<br>
Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch
Yavari, Dipl.-Kfm. Philipp Hermanns<br>
Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font>
</td>
</tr>
</tbody>
</table>
<hr color="#000000" noshade="noshade" size="1"
width="100%">
</div>
</body>
</html>
--------------AADA2BBC8C36F1CE2A14BB50
Content-Type: image/gif;
name="mgphgcckibnbbilh.gif"
Content-Transfer-Encoding: base64
Content-ID: <part2.0624039A.C0EFA82C(a)databay.de>
Content-Disposition: inline;
filename="mgphgcckibnbbilh.gif"
R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L
EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA///
/yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K
QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF
bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8
rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf
FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4
4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS
cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD
x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl
0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1
kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt
eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi
wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb
ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg
HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA
ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D
0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc
CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4
oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX
OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U
V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C
BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw
cPfs+xACADs=
--------------AADA2BBC8C36F1CE2A14BB50--
--------------90504E1C27EE95060580B78B--