Ovirt version 4.4.9.5-1.el8 vm doesn't start after power outage
by florianvanoudgaarden@gmail.com
After we had a power outrage 2 of my 7 vm's doesn't start anymore.
When i look at the logfile while starting i get the following errors in the engine.log
2022-09-14 15:18:29,078+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-138161) [40d05c63] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command GetVmLeaseInfoVDS failed: Command ['/usr/bin/dd', 'if=/rhev/data-center/mnt/_data_images/d036345b-be92-453d-a822-c08977fe3982/dom_md/xleases', 'iflag=direct,skip_bytes', 'skip=1048576', 'bs=253952', 'count=1'] failed with rc=1 out='[suppressed]' err=b"/usr/bin/dd: failed to open '/rhev/data-center/mnt/_data_images/d036345b-be92-453d-a822-c08977fe3982/dom_md/xleases': No such file or directory\n"
2022-09-14 15:18:29,079+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engine-Thread-138161) [40d05c63] IrsBroker::Failed::GetVmLeaseInfoVDS: IRSGenericException: IRSErrorException: Failed to GetVmLeaseInfoVDS, error = Command ['/usr/bin/dd', 'if=/rhev/data-center/mnt/_data_images/d036345b-be92-453d-a822-c08977fe3982/dom_md/xleases', 'iflag=direct,skip_bytes', 'skip=1048576', 'bs=253952', 'count=1'] failed with rc=1 out='[suppressed]' err=b"/usr/bin/dd: failed to open '/rhev/data-center/mnt/_data_images/d036345b-be92-453d-a822-c08977fe3982/dom_md/xleases': No such file or directory\n", code = 100
2022-09-14 15:18:29,085+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmLeaseInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-138161) [40d05c63] FINISH, GetVmLeaseInfoVDSCommand, return: , log id: 7d38bb40
2022-09-14 15:18:29,085+02 ERROR [org.ovirt.engine.core.bll.storage.lease.GetVmLeaseInfoCommand] (EE-ManagedThreadFactory-engine-Thread-138161) [40d05c63] Failure in getting lease info for VM '831e1a9d-2f87-48b5-af69-93d13f24d95d' from storage domains 'd036345b-be92-453d-a822-c08977fe3982', message: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to GetVmLeaseInfoVDS, error = Command ['/usr/bin/dd', 'if=/rhev/data-center/mnt/_data_images/d036345b-be92-453d-a822-c08977fe3982/dom_md/xleases', 'iflag=direct,skip_bytes', 'skip=1048576', 'bs=253952', 'count=1'] failed with rc=1 out='[suppressed]' err=b"/usr/bin/dd: failed to open '/rhev/data-center/mnt/_data_images/d036345b-be92-453d-a822-c08977fe3982/dom_md/xleases': No such file or directory\n", code = 100 (Failed with error GeneralException and code 100)
What noticed is the "No such file or directory" message so did a manual look at the "dom_md" directory and i notice the file "xleases" doesn't exist.
Output from commandline listing
[root@ovirt dom_md]# ll
totaal 4
-rw-rw----. 1 vdsm kvm 0 14 jan 2022 ids
-rw-rw----. 1 vdsm kvm 16777216 14 jan 2022 inbox
-rw-rw----. 1 vdsm kvm 0 14 jan 2022 leases
-rw-r--r--. 1 vdsm kvm 481 14 jan 2022 metadata
-rw-rw----. 1 vdsm kvm 16777216 14 jan 2022 outbox
[root@ovirt dom_md]# pwd
/rhev/data-center/mnt/_data_images/d036345b-be92-453d-a822-c08977fe3982/dom_md
What can I do to get the 2 vm's started again?