<HTML><BODY>Hi All,<br><br>Our oVirt cluster is with 3 nodes with shared fibre channel storage, the engine virtual machine is self hosted.<br> <br>Hypervisors OS: CentOS Linux release 7.3 / x86_64, oVirt version is <span class="st"><span class="st">4.1.2.2. <span id="result_box" lang="en"><span>The environment has been working for about a year without any problems</span></span>.<br><br>After</span></span><span id="result_box" class="short_text" lang="en"><span> shutdown of the hosted engine virtual machine, it doesn't start. <br><br><span id="result_box" class="short_text" lang="en"><span>Тhese commands that were executed:<br><br>hosted-engine --set-maintenance --mode=global<br>hosted-engine --vm-shutdown<br> </span></span><br>after the status of engine vm was down, we executed start.<br><br>[root@alpha] hosted-engine --vm-start<br>VM exists and is down, destroying it<br>Exception in thread Client localhost:54321 (most likely raised during interpreter shutdown):</span></span><span class="st"><br><br>we noticied that at vdsm.log <br><br>017-10-30 13:11:04,863+0200 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StorageDomain.getStats succeeded in 0.26 seconds (__init__:533)<br>2017-10-30 13:11:05,802+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:533)<br>2017-10-30 13:11:05,825+0200 WARN (jsonrpc/2) [virt.vm] (vmId='da98112d-b9fb-4098-93fa-1f1374b41e46') Failed to get metadata, domain not connected. (vm:2765)<br>2017-10-30 13:11:05,825+0200 ERROR (jsonrpc/2) [jsonrpc.JsonRpcServer] Internal server error (__init__:570)<br>Traceback (most recent call last):<br> File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 565, in _handle_request<br> res = method(**params)<br> File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 202, in _dynamicMethod<br> result = fn(*methodArgs)<br> File "/usr/share/vdsm/API.py", line 1454, in getAllVmIoTunePolicies<br> io_tune_policies_dict = self._cif.getAllVmIoTunePolicies()<br> File "/usr/share/vdsm/clientIF.py", line 448, in getAllVmIoTunePolicies<br> 'current_values': v.getIoTune()}<br> File "/usr/share/vdsm/virt/vm.py", line 2803, in getIoTune<br> result = self.getIoTuneResponse()<br> File "/usr/share/vdsm/virt/vm.py", line 2816, in getIoTuneResponse<br> res = self._dom.blockIoTune(<br> File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 47, in __getattr__<br> % self.vmid)<br>NotConnectedError: VM u'da98112d-b9fb-4098-93fa-1f1374b41e46' was not started yet or was shut down<br><br><br>The storage of self hosted engine multipath, pvs, lvs, seems ok... <br><br><span id="result_box" lang="en"><span>At the moment of the three nodes there is a working about 100 virtual machines and we can't manage them.<br><br><span id="result_box" class="short_text" lang="en"><span>Does anyone have any ideas, what can be done</span></span> то recover self hosted engine virtual machine?<br><br>Thahk You!<br>Have a nice day!<br></span></span><br><br><br><br><br><br><br><span></span></span><br><br><br><br></BODY></HTML>