Hi,

Our self hosted engine has been accidentally shut down by a teammate and now I'm trying hard to get it back up without success.

I've tried the --vm-start command but it says the VM is in WaitForLaunch status. 

I've set the global maintenance mode but it does nothing.

root@milhouse-main ~]# hosted-engine --vm-start
VM exists and is down, cleaning up and restarting
VM in WaitForLaunch

[root@milhouse-main ~]# hosted-engine --set-maintenance --mode=global
[root@milhouse-main ~]# hosted-engine --vm-status


!! Cluster is in GLOBAL MAINTENANCE mode !!



--== Host milhouse-main.envrmnt.local (id: 1) status ==--

conf_on_shared_storage             : True
Status up-to-date                  : False
Hostname                           : milhouse-main.envrmnt.local
Host ID                            : 1
Engine status                      : unknown stale-data
Score                              : 3400
stopped                            : False
Local maintenance                  : False
crc32                              : 931b2db9
local_conf_timestamp               : 1642052
Host timestamp                     : 1642052
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=1642052 (Tue May 18 19:52:59 2021)
host-id=1
score=3400
vm_conf_refresh_time=1642052 (Tue May 18 19:53:00 2021)
conf_on_shared_storage=True
maintenance=False
state=EngineDown
stopped=False


!! Cluster is in GLOBAL MAINTENANCE mode !!

You have new mail in /var/spool/mail/root
[root@milhouse-main ~]# hosted-engine --vm-start
VM exists and is down, cleaning up and restarting
VM in WaitForLaunch
[root@milhouse-main ~]# 
And when I list all vms, I can see the hosted engine is in the Shut Off status and the managed vms are all paused

[root@milhouse-main ~]# virsh -r list --all
setlocale: No such file or directory
 Id    Name                           State
----------------------------------------------------
 2     hp_gpu-node11                  paused
 3     fp_gpu-node5                   paused
 4     hp_gpu-node10                  paused
 5     hp_gpu-node7                   paused
 6     cpu-node3                      paused
 7     hp_gpu-node5                   paused
 8     fp_gpu-node1                   paused
 9     fp_gpu-node0                   paused
 10    cpu-node1                      paused
 11    fp_gpu-node6                   paused
 12    hp_gpu-node8                   paused
 13    fp_gpu-node10                  paused
 14    fp_gpu-node4                   paused
 15    fp_gpu-node9                   paused
 16    hp_gpu-node4                   paused
 17    fp_gpu-node15                  paused
 18    fp_gpu-node8                   paused
 19    hp_gpu-node0                   paused
 20    fp_gpu-node14                  paused
 21    fp_gpu-node2                   paused
 22    fp_gpu-node11                  paused
 23    hp_gpu-node9                   paused
 24    cpu-node2                      paused
 25    hp_gpu-node1                   paused
 26    hp_gpu-node2                   paused
 27    fp_gpu-node12                  paused
 28    hp_gpu-node3                   paused
 29    hp_gpu-node6                   paused
 30    infra-vm                       paused
 31    cpu-node0                      paused
 32    fp_gpu-node3                   paused
 33    fp_gpu-node7                   paused
 34    fp_gpu-node13                  paused
 35    bigip-16.1x-milhouse           paused
 -     HostedEngine                   shut off

[root@milhouse-main ~]#

I don't want to reboot the host server, cause I could loose all my VMs.

Can someone help here please?

Thanks.

Regards,
Eugène NG
--
LesCDN
------------------------------------------------------------
Aux hommes il faut un chef, et au chef il faut des hommes!
L'habit ne fait pas le moine, mais lorsqu'on te voit on te juge!