<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Feb 4, 2017 at 11:52 AM, Ralf Schenk <span dir="ltr"><<a href="mailto:rs@databay.de" target="_blank">rs@databay.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hello,</p>
<p>I have set up 3 hosts for engine, 2 of them are working correct.
There is no other host even having broker/agent installed. Is it
possible that the error occurs because the hosts are multihomed
(Management IP, IP for storage) and can communicate with different
IP's ?</p></div></blockquote><div>Having multiple logical networks for storage, management and so on is a good practice and it's advised so I tend to exclude any issue there.</div><div>The point is why your <a href="http://microcloud27.sub.mydomain.de">microcloud27.sub.mydomain.de</a> fails acquiring a lock as host 3.</div><div>Probably the simplest fix is just setting it in maintenance mode from the engine, removing it and deploying it from the engine as an hosted engine host again. </div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<p>hosted-engine --vm-status on both working hosts seems correct: (3
is out of order...)<br>
</p>
<p><tt>[root@microcloud21 ~]# hosted-engine --vm-status</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>--== Host 1 status ==--</tt><tt><br>
</tt><tt><br>
</tt><tt>conf_on_shared_storage <wbr> : True</tt><tt><br>
</tt><tt>Status up-to-date : True</tt><tt><br>
</tt><tt>Hostname <wbr> :
<a href="http://microcloud21.sub.mydomain.de" target="_blank">microcloud21.sub.mydomain.de</a></tt><tt><br>
</tt><tt>Host ID : 1</tt><tt><br>
</tt><tt>Engine status : {"health": "good",
"vm": "up", "detail": "up"}</tt><tt><br>
</tt><tt>Score <wbr> : 3400</tt><tt><br>
</tt><tt>stopped <wbr> : False</tt><tt><br>
</tt><tt>Local maintenance : False</tt><tt><br>
</tt><tt>crc32 <wbr> : 5941227d</tt><tt><br>
</tt><tt>local_conf_timestamp <wbr> : 152316</tt><tt><br>
</tt><tt>Host timestamp : 152302</tt><tt><br>
</tt><tt>Extra metadata (valid at timestamp):</tt><tt><br>
</tt><tt> metadata_parse_version=1</tt><tt><br>
</tt><tt> metadata_feature_version=1</tt><tt><br>
</tt><tt> timestamp=152302 (Sat Feb 4 11:49:29 2017)</tt><tt><br>
</tt><tt> host-id=1</tt><tt><br>
</tt><tt> score=3400</tt><tt><br>
</tt><tt> vm_conf_refresh_time=152316 (Sat Feb 4 11:49:43
2017)</tt><tt><br>
</tt><tt> conf_on_shared_storage=True</tt><tt><br>
</tt><tt> maintenance=False</tt><tt><br>
</tt><tt> state=EngineUp</tt><tt><br>
</tt><tt> stopped=False</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>--== Host 2 status ==--</tt><tt><br>
</tt><tt><br>
</tt><tt>conf_on_shared_storage <wbr> : True</tt><tt><br>
</tt><tt>Status up-to-date : True</tt><tt><br>
</tt><tt>Hostname <wbr> :
<a href="http://microcloud24.sub.mydomain.de" target="_blank">microcloud24.sub.mydomain.de</a></tt><tt><br>
</tt><tt>Host ID : 2</tt><tt><br>
</tt><tt>Engine status : {"reason": "vm not
running on this host", "health": "bad",
" <wbr> vm": "down", "detail":
"unknown"}</tt><tt><br>
</tt><tt>Score <wbr> : 3400</tt><tt><br>
</tt><tt>stopped <wbr> : False</tt><tt><br>
</tt><tt>Local maintenance : False</tt><tt><br>
</tt><tt>crc32 <wbr> : 77e25433</tt><tt><br>
</tt><tt>local_conf_timestamp <wbr> : 157637</tt><tt><br>
</tt><tt>Host timestamp : 157623</tt><tt><br>
</tt><tt>Extra metadata (valid at timestamp):</tt><tt><br>
</tt><tt> metadata_parse_version=1</tt><tt><br>
</tt><tt> metadata_feature_version=1</tt><tt><br>
</tt><tt> timestamp=157623 (Sat Feb 4 11:49:34 2017)</tt><tt><br>
</tt><tt> host-id=2</tt><tt><br>
</tt><tt> score=3400</tt><tt><br>
</tt><tt> vm_conf_refresh_time=157637 (Sat Feb 4 11:49:48
2017)</tt><tt><br>
</tt><tt> conf_on_shared_storage=True</tt><tt><br>
</tt><tt> maintenance=False</tt><tt><br>
</tt><tt> state=EngineDown</tt><tt><br>
</tt><tt> stopped=False</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>--== Host 3 status ==--</tt><tt><br>
</tt><tt><br>
</tt><tt>conf_on_shared_storage <wbr> : True</tt><tt><br>
</tt><tt>Status up-to-date : False</tt><tt><br>
</tt><tt>Hostname <wbr> :
<a href="http://microcloud27.sub.mydomain.de" target="_blank">microcloud27.sub.mydomain.de</a></tt><tt><br>
</tt><tt>Host ID : 3</tt><tt><br>
</tt><tt>Engine status : unknown stale-data</tt><tt><br>
</tt><tt>Score <wbr> : 0</tt><tt><br>
</tt><tt>stopped <wbr> : True</tt><tt><br>
</tt><tt>Local maintenance : False</tt><tt><br>
</tt><tt>crc32 <wbr> : 74798986</tt><tt><br>
</tt><tt>local_conf_timestamp <wbr> : 77946</tt><tt><br>
</tt><tt>Host timestamp : 77932</tt><tt><br>
</tt><tt>Extra metadata (valid at timestamp):</tt><tt><br>
</tt><tt> metadata_parse_version=1</tt><tt><br>
</tt><tt> metadata_feature_version=1</tt><tt><br>
</tt><tt> timestamp=77932 (Fri Feb 3 15:19:25 2017)</tt><tt><br>
</tt><tt> host-id=3</tt><tt><br>
</tt><tt> score=0</tt><tt><br>
</tt><tt> vm_conf_refresh_time=77946 (Fri Feb 3 15:19:39
2017)</tt><tt><br>
</tt><tt> conf_on_shared_storage=True</tt><tt><br>
</tt><tt> maintenance=False</tt><tt><br>
</tt><tt> state=AgentStopped</tt><tt><br>
</tt><tt> stopped=True</tt><tt><br>
</tt><br>
</p><span class="gmail-">
<p>Am 03.02.2017 um 19:20 schrieb Simone Tiraboschi:<br>
</p>
</span><blockquote type="cite"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote"><span class="gmail-">On Fri, Feb 3, 2017 at 5:22 PM, Ralf
Schenk <span dir="ltr"><<a href="mailto:rs@databay.de" target="_blank">rs@databay.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hello,</p>
<p>of course:</p>
<p>[root@microcloud27 mnt]# sanlock client status<br>
daemon 8a93c9ea-e242-408c-a63d-a9356b<wbr>b22df5.microcloud<br>
p -1 helper<br>
p -1 listener<br>
p -1 status<br>
</p>
<p>sanlock.log attached. (Beginning 2017-01-27 where
everything was fine)</p>
</div>
</blockquote>
Thanks, the issue is here:
</span><div>
<pre class="gmail-m_-395185777292731078gmail-aLF-aPX-K0-aPE gmail-m_-395185777292731078gmail-aLF-aPX-aLK-ayr-auR">2017-02-02 19:01:22+0100 4848 [1048]: s36 lockspace 7c8deaa8-be02-4aaf-b9b4-<wbr>ddc8da99ad96:3:/rhev/data-<wbr>center/mnt/glusterSD/<wbr>glusterfs.sub.mydomain.de:_<wbr>engine/7c8deaa8-be02-4aaf-<wbr>b9b4-ddc8da99ad96/dom_md/ids:0
2017-02-02 19:03:42+0100 4988 [12983]: s36 delta_acquire host_id 3 busy1 3 15 13129 7ad427b1-fbb6-4cee-b9ee-<wbr>01f596fddfbb.microcloud
2017-02-02 19:03:43+0100 4989 [1048]: s36 add_lockspace fail result -262</pre>
</div><span class="gmail-">
<div>Could you please check if you have other hosts contending
for the same ID (id=3 in this case).</div>
<div> </div>
</span></div>
</div>
</blockquote>
<br>
<div class="gmail-m_-395185777292731078moz-signature">-- <br>
<p>
</p>
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td colspan="3"> <img title="" alt="" src="cid:part2.0624039A.C0EFA82C@databay.de" height="30" border="0" width="151"> </td>
</tr>
<tr>
<td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br>
<b>Ralf Schenk</b><br>
fon <a href="tel:+49%202405%20408370" value="+492405408370" target="_blank">+49 (0) 24 05 / 40 83 70</a><br>
fax <a href="tel:+49%202405%204083759" value="+4924054083759" target="_blank">+49 (0) 24 05 / 40 83 759</a><br>
mail <a href="mailto:rs@databay.de" target="_blank"><font color="#FF0000"><b>rs@databay.de</b></font></a><br>
</font> </td>
<td width="30"> </td>
<td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br>
<b>Databay AG</b><br>
Jens-Otto-Krag-Straße 11<br>
D-52146 Würselen<br>
<a href="http://www.databay.de" target="_blank"><font color="#FF0000"><b>www.databay.de</b></font></a>
</font> </td>
</tr>
<tr>
<td colspan="3" valign="top"> <font face="Verdana, Arial,
sans-serif" size="1"><span class="gmail-"><br>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE
210844202<br>
Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch
Yavari, Dipl.-Kfm. Philipp Hermanns<br></span>
Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font>
</td>
</tr>
</tbody>
</table>
<hr color="#000000" noshade size="1" width="100%">
</div>
</div>
</blockquote></div><br></div></div>