Ok, So removing one downed node cleared all the non syncing issues. In the mean time, when that one node was coming back, it seems to have corrupted the hosted-engine vm. Remote-Viewer nodeip:5900, the console shows: Probing EDD (edd=off to disable)... ok Doesn't matter which of the three remaining nodes try to launch the engine, the engine comes up the same. Had to set cluster to global maintenance, as the engine will keep trying to start off different nodes. I do have backups run nightly so I can restore engine vm, however, I don't see a straight forward method of restoring the engine vm in a hosted-engine gluster setup. Can any of the redhat boys help? Here's the hosted-engine --vm-status --== Host 1 status ==-- conf_on_shared_storage             : True Status up-to-date                  : True Hostname                           : ovirtnode1.abcxyzdomains.net Host ID                            : 1 Engine status                      : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} Score                              : 3400 stopped                            : False Local maintenance                  : False crc32                              : 92254a68 local_conf_timestamp               : 115910 Host timestamp                     : 115910 Extra metadata (valid at timestamp):     metadata_parse_version=1     metadata_feature_version=1     timestamp=115910 (Mon Jun 18 09:43:20 2018)     host-id=1     score=3400     vm_conf_refresh_time=115910 (Mon Jun 18 09:43:20 2018)     conf_on_shared_storage=True     maintenance=False     state=GlobalMaintenance     stopped=False ---clipped--- On 06/16/2018 02:23 PM, Hanson Turner wrote:

Hi Guys,

I've got 60 some odd files for each of the nodes in the cluster, they don't seem to be syncing.

Running a volume heal engine full, reports successful. Running volume heal engine info reports the same files, and doesn't seem to be syncing.

Running a volume heal engine info split-brain, there's nothing listed in split-brain.

Peers show as connected. Gluster volumes are started/up.

Hosted-engine --vm-status reports : The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable.

This is leaving the cluster in an engine down with all vm's down state...

Thanks, _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/YPNWM222K2U7NX...