Hosted engine VM not coming up after Storage rebuild.

Hi, We are running Ovirt 4.3 on IBM x3650, x3550 servers with our SAN on IBM Midrange storage DS5300. All the storage domains are separate LUN's in the storage with a dedicated LUN for the HostedEngine VM. The storage had a failure due to a major power issue. One of the storage array's that have the storage domains had to be re-constructed using IBM's StorageManager tool. After booting the hosts, i can see all the storage domains as LUN's with the multipath -ll from all the hosts, but on starting the hosted engine with #hosted-engine --vm-start, and checking with --vm-status i get "Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"}'. The global maintenance mode is enabled and i tried "hosted-engine --console" - after logging in i checked the boot.log, i find [FAILED] Failed to mount /var/log. See 'systemctl status var-log.mount' for details. [DEPEND] Dependency failed for Update UTMP about System Boot/Shutdown. [DEPEND] Dependency failed for Update UTMP about System Runlevel Changes. [DEPEND] Dependency failed for Flush Journal to Persistent Storage. [DEPEND] Dependency failed for /var/log/audit. [DEPEND] Dependency failed for Local File Systems. [DEPEND] Dependency failed for Mark the need to relabel after reboot. [DEPEND] Dependency failed for Migrate local... structure to the new structure. [DEPEND] Dependency failed for Relabel all filesystems, if necessary. the "systemctl status var-log.mount" shows ● var-log.mount - /var/log Loaded: loaded (/etc/fstab; bad; vendor preset: disabled) Active: failed (Result: exit-code) since Fri 2023-10-06 14:38:58 IST; 10min ago Where: /var/log What: /dev/mapper/ovirt-log Docs: man:fstab(5) man:systemd-fstab-generator(8) Process: 691 ExecMount=/bin/mount /dev/mapper/ovirt-log /var/log -t xfs -o nodev (code=exited, status=32) Oct 06 14:38:57 ovman systemd[1]: Mounting /var/log... Oct 06 14:38:58 ovman mount[691]: mount: mount /dev/mapper/ovirt-...g Oct 06 14:38:58 ovman systemd[1]: var-log.mount mount process exi...2 Oct 06 14:38:58 ovman systemd[1]: Failed to mount /var/log. Oct 06 14:38:58 ovman systemd[1]: Unit var-log.mount entered fail.... Hint: Some lines were ellipsized, use -l to show in full. I need to recover from this quick. Sumit Basu
participants (1)
-
Sumit Basu