On Wed, Jan 2, 2019 at 10:51 AM maoz zadok <maozza(a)gmail.com&gt; wrote: > > Hello, > vm stuck (on two hosts) > I have MySQL VM guest that was stuck in a way that "kill -9" to the MySQL server pid did nothing. > from the shell I execute "init 0", this did not shut down the VM. > > from the ovirt-engine I'm trying to power off the machine, and it is not doing anything. > the event shows: >> >> VDSM kvm5 command DestroyVDS failed: Virtual machine destroy error > > and: >> >> Failed to power off VM mysql-production (Host: kvm5, User: admin@internal-authz). > > > Note, when I list the VM s on the hosts view, I see that that same VM is on two hosts (kvm7 and kvm5). > It looks like the VM is stuck on two hosts, in the status "Powering Down" > > any ideas? No idea about this specific problem and how to solve it. Is this on the same system you restored from backup? Is this VM marked as HA (highly-available)? Did it move to a different host since you took the backup you now restored? If all are 'yes', then it's likely that the engine tried to start it (because it's HA) before it noticed that it's already running on another host (a fact that it could not know, if it was different from what's in the restored database). Sadly, this very likely leads to corruption. See also: https://bugzilla.redhat.com/1441322 Since I do not see 'Resetting HA VM status' in the restore logs you posted, it might be a different issue, but perhaps similar. If indeed you see two qemu processes on the hosts, I'd try to kill -9 both of them. Adding Arik. Good luck, > Thank you > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/E2HKUJZYUNC... -- Didi

Host restart resolve the issue! On Wed, Jan 2, 2019 at 11:50 AM maoz zadok <maozza(a)gmail.com&gt; wrote: > > Thank you Didi for the fast reply! > The problem was before the restore from backup (that was the reason that I reboot the engine and the engine didn't come up, I dug a bigger hole :-( ), > HA is not marked. but The VM was trying to migrate from kvm7 to kvm5 before the engine crashed. > I don't see the qemu process on any of the hosts. > "vdsm-client Host getVMList fullStatus=True" show the machine uuid only on one (kvm5) > > I'll try restarting the host (kvm5), let's see if it helps. > > > > > > On Wed, Jan 2, 2019 at 11:07 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >> >> On Wed, Jan 2, 2019 at 10:51 AM maoz zadok <maozza(a)gmail.com&gt; wrote: >> > >> > Hello, >> > vm stuck (on two hosts) >> > I have MySQL VM guest that was stuck in a way that "kill -9" to the MySQL server pid did nothing. >> > from the shell I execute "init 0", this did not shut down the VM. >> > >> > from the ovirt-engine I'm trying to power off the machine, and it is not doing anything. >> > the event shows: >> >> >> >> VDSM kvm5 command DestroyVDS failed: Virtual machine destroy error >> > >> > and: >> >> >> >> Failed to power off VM mysql-production (Host: kvm5, User: admin@internal-authz). >> > >> > >> > Note, when I list the VM s on the hosts view, I see that that same VM is on two hosts (kvm7 and kvm5). >> > It looks like the VM is stuck on two hosts, in the status "Powering Down" >> > >> > any ideas? >> >> No idea about this specific problem and how to solve it. >> >> Is this on the same system you restored from backup? Is this VM marked >> as HA (highly-available)? Did it move to a different host since you >> took the backup you now restored? >> >> If all are 'yes', then it's likely that the engine tried to start it >> (because it's HA) before it noticed that it's already running on >> another host (a fact that it could not know, if it was different from >> what's in the restored database). Sadly, this very likely leads to >> corruption. See also: >> >> https://bugzilla.redhat.com/1441322 >> >> Since I do not see 'Resetting HA VM status' in the restore logs you posted, >> it might be a different issue, but perhaps similar. >> >> If indeed you see two qemu processes on the hosts, I'd try to kill -9 >> both of them. >> >> Adding Arik. >> >> Good luck, >> >> >> > Thank you >> > _______________________________________________ >> > Users mailing list -- users(a)ovirt.org >> > To unsubscribe send an email to users-leave(a)ovirt.org >> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/E2HKUJZYUNC... >> >> >> >> -- >> Didi