This is a multi-part message in MIME format. --------------90504E1C27EE95060580B78B Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello, I have set up 3 hosts for engine, 2 of them are working correct. There is no other host even having broker/agent installed. Is it possible that the error occurs because the hosts are multihomed (Management IP, IP for storage) and can communicate with different IP's ? hosted-engine --vm-status on both working hosts seems correct: (3 is out of order...) [root@microcloud21 ~]# hosted-engine --vm-status --=3D=3D Host 1 status =3D=3D-- conf_on_shared_storage : True Status up-to-date : True Hostname : microcloud21.sub.mydomain.de Host ID : 1 Engine status : {"health": "good", "vm": "up", "detail": "up"} Score : 3400 stopped : False Local maintenance : False crc32 : 5941227d local_conf_timestamp : 152316 Host timestamp : 152302 Extra metadata (valid at timestamp): metadata_parse_version=3D1 metadata_feature_version=3D1 timestamp=3D152302 (Sat Feb 4 11:49:29 2017) host-id=3D1 score=3D3400 vm_conf_refresh_time=3D152316 (Sat Feb 4 11:49:43 2017) conf_on_shared_storage=3DTrue maintenance=3DFalse state=3DEngineUp stopped=3DFalse --=3D=3D Host 2 status =3D=3D-- conf_on_shared_storage : True Status up-to-date : True Hostname : microcloud24.sub.mydomain.de Host ID : 2 Engine status : {"reason": "vm not running on this host", "health": "bad", " vm": "down", "detail": "unknown"} Score : 3400 stopped : False Local maintenance : False crc32 : 77e25433 local_conf_timestamp : 157637 Host timestamp : 157623 Extra metadata (valid at timestamp): metadata_parse_version=3D1 metadata_feature_version=3D1 timestamp=3D157623 (Sat Feb 4 11:49:34 2017) host-id=3D2 score=3D3400 vm_conf_refresh_time=3D157637 (Sat Feb 4 11:49:48 2017) conf_on_shared_storage=3DTrue maintenance=3DFalse state=3DEngineDown stopped=3DFalse --=3D=3D Host 3 status =3D=3D-- conf_on_shared_storage : True Status up-to-date : False Hostname : microcloud27.sub.mydomain.de Host ID : 3 Engine status : unknown stale-data Score : 0 stopped : True Local maintenance : False crc32 : 74798986 local_conf_timestamp : 77946 Host timestamp : 77932 Extra metadata (valid at timestamp): metadata_parse_version=3D1 metadata_feature_version=3D1 timestamp=3D77932 (Fri Feb 3 15:19:25 2017) host-id=3D3 score=3D0 vm_conf_refresh_time=3D77946 (Fri Feb 3 15:19:39 2017) conf_on_shared_storage=3DTrue maintenance=3DFalse state=3DAgentStopped stopped=3DTrue Am 03.02.2017 um 19:20 schrieb Simone Tiraboschi:
On Fri, Feb 3, 2017 at 5:22 PM, Ralf Schenk <rs@databay.de <mailto:rs@databay.de>> wrote:
Hello,
of course:
[root@microcloud27 mnt]# sanlock client status daemon 8a93c9ea-e242-408c-a63d-a9356bb22df5.microcloud p -1 helper p -1 listener p -1 status
sanlock.log attached. (Beginning 2017-01-27 where everything was fi=
ne)
Thanks, the issue is here: 2017-02-02 19:01:22+0100 4848 [1048]: s36 lockspace 7c8deaa8-be02-4aaf-=
b9b4-ddc8da99ad96:3:/rhev/data-center/mnt/glusterSD/glusterfs.sub.mydomai= n.de:_engine/7c8deaa8-be02-4aaf-b9b4-ddc8da99ad96/dom_md/ids:0
2017-02-02 19:03:42+0100 4988 [12983]: s36 delta_acquire host_id 3 busy= 1 3 15 13129 7ad427b1-fbb6-4cee-b9ee-01f596fddfbb.microcloud 2017-02-02 19:03:43+0100 4989 [1048]: s36 add_lockspace fail result -26= 2 Could you please check if you have other hosts contending for the same ID (id=3D3 in this case). =20
--=20 *Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <mailto:rs@databay.de> =09 *Databay AG* Jens-Otto-Krag-Stra=C3=9Fe 11 D-52146 W=C3=BCrselen *www.databay.de* <http://www.databay.de> Sitz/Amtsgericht Aachen =E2=80=A2 HRB:8437 =E2=80=A2 USt-IdNr.: DE 210844= 202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------------------------------------------------ --------------90504E1C27EE95060580B78B Content-Type: multipart/related; boundary="------------AADA2BBC8C36F1CE2A14BB50" --------------AADA2BBC8C36F1CE2A14BB50 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p>Hello,</p> <p>I have set up 3 hosts for engine, 2 of them are working correct. There is no other host even having broker/agent installed. Is it possible that the error occurs because the hosts are multihomed (Management IP, IP for storage) and can communicate with different IP's ?</p> <p>hosted-engine --vm-status on both working hosts seems correct: (3 is out of order...)<br> </p> <p><tt>[root@microcloud21 ~]# hosted-engine --vm-status</tt><tt><br> </tt><tt><br> </tt><tt><br> </tt><tt>--== Host 1 status ==--</tt><tt><br> </tt><tt><br> </tt><tt>conf_on_shared_storage : True</tt><tt><br> </tt><tt>Status up-to-date : True</tt><tt><br> </tt><tt>Hostname : microcloud21.sub.mydomain.de</tt><tt><br> </tt><tt>Host ID : 1</tt><tt><br> </tt><tt>Engine status : {"health": "good", "vm": "up", "detail": "up"}</tt><tt><br> </tt><tt>Score : 3400</tt><tt><br> </tt><tt>stopped : False</tt><tt><br> </tt><tt>Local maintenance : False</tt><tt><br> </tt><tt>crc32 : 5941227d</tt><tt><br> </tt><tt>local_conf_timestamp : 152316</tt><tt><br> </tt><tt>Host timestamp : 152302</tt><tt><br> </tt><tt>Extra metadata (valid at timestamp):</tt><tt><br> </tt><tt> metadata_parse_version=1</tt><tt><br> </tt><tt> metadata_feature_version=1</tt><tt><br> </tt><tt> timestamp=152302 (Sat Feb 4 11:49:29 2017)</tt><tt><br> </tt><tt> host-id=1</tt><tt><br> </tt><tt> score=3400</tt><tt><br> </tt><tt> vm_conf_refresh_time=152316 (Sat Feb 4 11:49:43 2017)</tt><tt><br> </tt><tt> conf_on_shared_storage=True</tt><tt><br> </tt><tt> maintenance=False</tt><tt><br> </tt><tt> state=EngineUp</tt><tt><br> </tt><tt> stopped=False</tt><tt><br> </tt><tt><br> </tt><tt><br> </tt><tt>--== Host 2 status ==--</tt><tt><br> </tt><tt><br> </tt><tt>conf_on_shared_storage : True</tt><tt><br> </tt><tt>Status up-to-date : True</tt><tt><br> </tt><tt>Hostname : microcloud24.sub.mydomain.de</tt><tt><br> </tt><tt>Host ID : 2</tt><tt><br> </tt><tt>Engine status : {"reason": "vm not running on this host", "health": "bad", " vm": "down", "detail": "unknown"}</tt><tt><br> </tt><tt>Score : 3400</tt><tt><br> </tt><tt>stopped : False</tt><tt><br> </tt><tt>Local maintenance : False</tt><tt><br> </tt><tt>crc32 : 77e25433</tt><tt><br> </tt><tt>local_conf_timestamp : 157637</tt><tt><br> </tt><tt>Host timestamp : 157623</tt><tt><br> </tt><tt>Extra metadata (valid at timestamp):</tt><tt><br> </tt><tt> metadata_parse_version=1</tt><tt><br> </tt><tt> metadata_feature_version=1</tt><tt><br> </tt><tt> timestamp=157623 (Sat Feb 4 11:49:34 2017)</tt><tt><br> </tt><tt> host-id=2</tt><tt><br> </tt><tt> score=3400</tt><tt><br> </tt><tt> vm_conf_refresh_time=157637 (Sat Feb 4 11:49:48 2017)</tt><tt><br> </tt><tt> conf_on_shared_storage=True</tt><tt><br> </tt><tt> maintenance=False</tt><tt><br> </tt><tt> state=EngineDown</tt><tt><br> </tt><tt> stopped=False</tt><tt><br> </tt><tt><br> </tt><tt><br> </tt><tt>--== Host 3 status ==--</tt><tt><br> </tt><tt><br> </tt><tt>conf_on_shared_storage : True</tt><tt><br> </tt><tt>Status up-to-date : False</tt><tt><br> </tt><tt>Hostname : microcloud27.sub.mydomain.de</tt><tt><br> </tt><tt>Host ID : 3</tt><tt><br> </tt><tt>Engine status : unknown stale-data</tt><tt><br> </tt><tt>Score : 0</tt><tt><br> </tt><tt>stopped : True</tt><tt><br> </tt><tt>Local maintenance : False</tt><tt><br> </tt><tt>crc32 : 74798986</tt><tt><br> </tt><tt>local_conf_timestamp : 77946</tt><tt><br> </tt><tt>Host timestamp : 77932</tt><tt><br> </tt><tt>Extra metadata (valid at timestamp):</tt><tt><br> </tt><tt> metadata_parse_version=1</tt><tt><br> </tt><tt> metadata_feature_version=1</tt><tt><br> </tt><tt> timestamp=77932 (Fri Feb 3 15:19:25 2017)</tt><tt><br> </tt><tt> host-id=3</tt><tt><br> </tt><tt> score=0</tt><tt><br> </tt><tt> vm_conf_refresh_time=77946 (Fri Feb 3 15:19:39 2017)</tt><tt><br> </tt><tt> conf_on_shared_storage=True</tt><tt><br> </tt><tt> maintenance=False</tt><tt><br> </tt><tt> state=AgentStopped</tt><tt><br> </tt><tt> stopped=True</tt><tt><br> </tt><br> </p> <p>Am 03.02.2017 um 19:20 schrieb Simone Tiraboschi:<br> </p> <blockquote cite="mid:CAN8-ONo3+L+R3iPr1DoS0s8Fn3kZNkekKrP4ipyKPs_e_2jhiQ@mail.gmail.com" type="cite"><br> <div class="gmail_extra"><br> <div class="gmail_quote">On Fri, Feb 3, 2017 at 5:22 PM, Ralf Schenk <span dir="ltr"><<a moz-do-not-send="true" href="mailto:rs@databay.de" target="_blank">rs@databay.de</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div bgcolor="#FFFFFF"> <p>Hello,</p> <p>of course:</p> <p>[root@microcloud27 mnt]# sanlock client status<br> daemon 8a93c9ea-e242-408c-a63d-<wbr>a9356bb22df5.microcloud<br> p -1 helper<br> p -1 listener<br> p -1 status<br> </p> <p>sanlock.log attached. (Beginning 2017-01-27 where everything was fine)</p> </div> </blockquote> Thanks, the issue is here: <div> <pre class="gmail-aLF-aPX-K0-aPE gmail-aLF-aPX-aLK-ayr-auR">2017-02-02 19:01:22+0100 4848 [1048]: s36 lockspace 7c8deaa8-be02-4aaf-b9b4-ddc8da99ad96:3:/rhev/data-center/mnt/glusterSD/glusterfs.sub.mydomain.de:_engine/7c8deaa8-be02-4aaf-b9b4-ddc8da99ad96/dom_md/ids:0 2017-02-02 19:03:42+0100 4988 [12983]: s36 delta_acquire host_id 3 busy1 3 15 13129 7ad427b1-fbb6-4cee-b9ee-01f596fddfbb.microcloud 2017-02-02 19:03:43+0100 4989 [1048]: s36 add_lockspace fail result -262</pre> </div> <div>Could you please check if you have other hosts contending for the same ID (id=3 in this case).</div> <div> </div> </div> </div> </blockquote> <br> <div class="moz-signature">-- <br> <p> </p> <table border="0" cellpadding="0" cellspacing="0"> <tbody> <tr> <td colspan="3"> <img title="" alt="" src="cid:part2.0624039A.C0EFA82C@databay.de" height="30" border="0" width="151"> </td> </tr> <tr> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Ralf Schenk</b><br> fon +49 (0) 24 05 / 40 83 70<br> fax +49 (0) 24 05 / 40 83 759<br> mail <a href="mailto:rs@databay.de"><font color="#FF0000"><b>rs@databay.de</b></font></a><br> </font> </td> <td width="30"> </td> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Databay AG</b><br> Jens-Otto-Krag-Straße 11<br> D-52146 Würselen<br> <a href="http://www.databay.de"><font color="#FF0000"><b>www.databay.de</b></font></a> </font> </td> </tr> <tr> <td colspan="3" valign="top"> <font face="Verdana, Arial, sans-serif" size="1"><br> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202<br> Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br> Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font> </td> </tr> </tbody> </table> <hr color="#000000" noshade="noshade" size="1" width="100%"> </div> </body> </html> --------------AADA2BBC8C36F1CE2A14BB50 Content-Type: image/gif; name="mgphgcckibnbbilh.gif" Content-Transfer-Encoding: base64 Content-ID: <part2.0624039A.C0EFA82C@databay.de> Content-Disposition: inline; filename="mgphgcckibnbbilh.gif" R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA/// /yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8 rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4 4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl 0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1 kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D 0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4 oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw cPfs+xACADs= --------------AADA2BBC8C36F1CE2A14BB50-- --------------90504E1C27EE95060580B78B--