Hosted engine HA does not migrate on maintenance
This is a multi-part message in MIME format. --------------D0B4F99ADAD93F52BD8FD982 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello, I've set up 3 Hosts of my 8 to be able to run hosted-engine (on glusterfs). When I set the host running the HostedEngine to maintenance the hosted-engine VM doesn't migrate to one of the other hosts. I think a state as shown below (Host 1 in Local maintenance but engine running on it) should't be possible. When setting maintenance and observing Engine VM state in Management Interface I can see Status "migrating from.." for a short time in the interface but VM is not migrated and Status switches back to "Up" on the host which should be prepared for maintenance. [root@microcloud21 ~]# hosted-engine --vm-status --=3D=3D Host 1 status =3D=3D-- Status up-to-date : True Hostname : microcloud21.mydomain Host ID : 1 Engine status : {"health": "good", "vm": "up", "detail": "up"} Score : 0 stopped : False Local maintenance : True crc32 : 922bd955 Host timestamp : 426722 --=3D=3D Host 2 status =3D=3D-- Status up-to-date : True Hostname : microcloud24.mydomain Host ID : 2 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"} Score : 3400 stopped : False Local maintenance : False crc32 : a35888e4 Host timestamp : 312349 --=3D=3D Host 3 status =3D=3D-- Status up-to-date : True Hostname : microcloud27.mydomain Host ID : 3 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"} Score : 3400 stopped : False Local maintenance : False crc32 : 06e20597 Host timestamp : 312061 --=20 *Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <mailto:rs@databay.de> =09 *Databay AG* Jens-Otto-Krag-Stra=C3=9Fe 11 D-52146 W=C3=BCrselen *www.databay.de* <http://www.databay.de> Sitz/Amtsgericht Aachen =E2=80=A2 HRB:8437 =E2=80=A2 USt-IdNr.: DE 210844= 202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------------------------------------------------ --------------D0B4F99ADAD93F52BD8FD982 Content-Type: multipart/related; boundary="------------0B844ABBDFD3CA533C26489C" --------------0B844ABBDFD3CA533C26489C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p>Hello,</p> <p>I've set up 3 Hosts of my 8 to be able to run hosted-engine (on glusterfs). When I set the host running the HostedEngine to maintenance the hosted-engine VM doesn't migrate to one of the other hosts. <br> </p> <p>I think a state as shown below (Host 1 in Local maintenance but engine running on it) should't be possible. When setting maintenance and observing Engine VM state in Management Interface I can see Status "migrating from.." for a short time in the interface but VM is not migrated and Status switches back to "Up" on the host which should be prepared for maintenance.<br> </p> <p><tt>[root@microcloud21 ~]# hosted-engine --vm-status</tt><br> <tt><br> </tt><tt>--== Host 1 status ==--</tt><tt><br> </tt><tt><br> </tt><tt>Status up-to-date : True</tt><tt><br> </tt><tt>Hostname : microcloud21.mydomain</tt><tt><br> </tt><tt>Host ID : 1</tt><tt><br> </tt><tt>Engine status : {"health": "good", "vm": "up", "detail": "up"}</tt><tt><br> </tt><tt>Score : 0</tt><tt><br> </tt><tt>stopped : False</tt><tt><br> </tt><tt>Local maintenance : True</tt><tt><br> </tt><tt>crc32 : 922bd955</tt><tt><br> </tt><tt>Host timestamp : 426722</tt><tt><br> </tt><tt><br> </tt><tt><br> </tt><tt>--== Host 2 status ==--</tt><tt><br> </tt><tt><br> </tt><tt>Status up-to-date : True</tt><tt><br> </tt><tt>Hostname : microcloud24.mydomain</tt><tt><br> </tt><tt>Host ID : 2</tt><tt><br> </tt><tt>Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}</tt><tt><br> </tt><tt>Score : 3400</tt><tt><br> </tt><tt>stopped : False</tt><tt><br> </tt><tt>Local maintenance : False</tt><tt><br> </tt><tt>crc32 : a35888e4</tt><tt><br> </tt><tt>Host timestamp : 312349</tt><tt><br> </tt><tt><br> </tt><tt><br> </tt><tt>--== Host 3 status ==--</tt><tt><br> </tt><tt><br> </tt><tt>Status up-to-date : True</tt><tt><br> </tt><tt>Hostname : microcloud27.mydomain</tt><tt><br> </tt><tt>Host ID : 3</tt><tt><br> </tt><tt>Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}</tt><tt><br> </tt><tt>Score : 3400</tt><tt><br> </tt><tt>stopped : False</tt><tt><br> </tt><tt>Local maintenance : False</tt><tt><br> </tt><tt>crc32 : 06e20597</tt><tt><br> </tt><tt>Host timestamp : 312061</tt><tt><br> </tt><br> </p> <br> <div class="moz-signature">-- <br> <p> </p> <table border="0" cellpadding="0" cellspacing="0"> <tbody> <tr> <td colspan="3"><img src="cid:part1.64D6B1FA.4D834312@databay.de" height="30" border="0" width="151"></td> </tr> <tr> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Ralf Schenk</b><br> fon +49 (0) 24 05 / 40 83 70<br> fax +49 (0) 24 05 / 40 83 759<br> mail <a href="mailto:rs@databay.de"><font color="#FF0000"><b>rs@databay.de</b></font></a><br> </font> </td> <td width="30"> </td> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Databay AG</b><br> Jens-Otto-Krag-Straße 11<br> D-52146 Würselen<br> <a href="http://www.databay.de"><font color="#FF0000"><b>www.databay.de</b></font></a> </font> </td> </tr> <tr> <td colspan="3" valign="top"> <font face="Verdana, Arial, sans-serif" size="1"><br> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202<br> Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br> Aufsichtsratsvorsitzender: Klaus Scholzen (RA) </font> </td> </tr> </tbody> </table> <hr color="#000000" noshade="noshade" size="1" width="100%"> </div> </body> </html> --------------0B844ABBDFD3CA533C26489C Content-Type: image/gif; name="logo_databay_email.gif" Content-Transfer-Encoding: base64 Content-ID: <part1.64D6B1FA.4D834312@databay.de> Content-Disposition: inline; filename="logo_databay_email.gif" R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA/// /yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8 rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4 4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl 0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1 kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D 0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4 oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw cPfs+xACADs= --------------0B844ABBDFD3CA533C26489C-- --------------D0B4F99ADAD93F52BD8FD982--
Hi Ralf, can you please attach the following files? - The engine log - The hosted engine agent log from all hosts - The vdsm log from all hosts It will greatly help us diagnosing this. We even have a bug with similar symptoms and this might give us some insight to what happened there as well. Thanks -- Martin Sivak SLA / oVirt On Mon, May 23, 2016 at 11:13 AM, Ralf Schenk <rs@databay.de> wrote:
Hello,
I've set up 3 Hosts of my 8 to be able to run hosted-engine (on glusterfs). When I set the host running the HostedEngine to maintenance the hosted-engine VM doesn't migrate to one of the other hosts.
I think a state as shown below (Host 1 in Local maintenance but engine running on it) should't be possible. When setting maintenance and observing Engine VM state in Management Interface I can see Status "migrating from.." for a short time in the interface but VM is not migrated and Status switches back to "Up" on the host which should be prepared for maintenance.
[root@microcloud21 ~]# hosted-engine --vm-status
--== Host 1 status ==--
Status up-to-date : True Hostname : microcloud21.mydomain Host ID : 1 Engine status : {"health": "good", "vm": "up", "detail": "up"} Score : 0 stopped : False Local maintenance : True crc32 : 922bd955 Host timestamp : 426722
--== Host 2 status ==--
Status up-to-date : True Hostname : microcloud24.mydomain Host ID : 2 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"} Score : 3400 stopped : False Local maintenance : False crc32 : a35888e4 Host timestamp : 312349
--== Host 3 status ==--
Status up-to-date : True Hostname : microcloud27.mydomain Host ID : 3 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"} Score : 3400 stopped : False Local maintenance : False crc32 : 06e20597 Host timestamp : 312061
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Klaus Scholzen (RA) ------------------------------
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (2)
-
Martin Sivak -
Ralf Schenk