5:01 a.m.
Hi,
You can try with the vdsm-client tool:
https://www.ovirt.org/develop/developer-guide/vdsm/vdsm-client.html
Stopping a VM:
1) Get the vmId:
# vdsm-client Host getVMList fullStatus=True
2) Destroy the VM
# vdsm-client VM destroy vmID=<vmID>
*Regards,*
*Shani Leviim*
On Sat, Apr 3, 2021 at 7:50 AM <davidk(a)riavera.com> wrote:
Hello,
I've somehow gotten one of my VMs stuck in a state that ovirt seems to be
rather confused about its
existence of now. I'm running oVirt 4.3.10 and using oVirt Node on all
the hosts.
My engine and host event logs are now filling up very rapidly with this
error:
VDSM node217 command DestroyVDS failed: General Exception:
("'1048576'",)
I was playing with hugetable support, and that error number or string
looks suspiciously
like the "hugetable size" custom property I set on the VM.
This VM was migrated to another host at one point as well, and now that
host is also
generating the same error as well.
When I try to move these hosts to maintenance mode, they get stuck in
"Preparing for
Maintenance" while it tries to migrate/deal with the VM that's not there
any more.
Forcibly rebooting the hosts does not change anything. The VM state/host
seems to be
captured somewhere persistent in this case.
The VM in question is not running, and I can start it up on another host
successfully,
but ovirt still thinks it exists on the other 2 hosts no matter what I do.
Is there perhaps some way to delete it from the engine database directly
to straighten
things out?
Here's a dump of the vdsm log on one of the hosts. I haven't been able to
pinpoint what
the exact issue is or how to fix it, but hopefully someone here will have
seen this before?
2021-04-03 04:40:35,515+0000 INFO (jsonrpc/1) [api.virt] START
destroy(gracefulAttempts=1) from=::ffff:10.100.0.210,58150,
vmId=58abf0cf-d7b9-4067-a86a-e619928368e7 (api:48)
2021-04-03 04:40:35,516+0000 INFO (jsonrpc/1) [virt.vm]
(vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Release VM resources (vm:5186)
2021-04-03 04:40:35,516+0000 WARN (jsonrpc/1) [virt.vm]
(vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') trying to set state to
Powering down when already Down (vm:626)
2021-04-03 04:40:35,516+0000 INFO (jsonrpc/1) [virt.vm]
(vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Stopping connection
(guestagent:455)
2021-04-03 04:40:35,517+0000 INFO (jsonrpc/1) [vdsm.api] START
teardownImage(sdUUID='a08af6be-3802-4bb1-9fa5-4b6a10227290',
spUUID='78dc095a-5238-11e8-b8bf-00163e6a7af9',
imgUUID='9c896907-59b0-4983-9478-b36b2c2eb01e', volUUID=None)
from=::ffff:10.100.0.210,58150, task_id=fc946d20-126a-4fd0-9078-91
4b4a64b1d9 (api:48)
2021-04-03 04:40:35,518+0000 INFO (jsonrpc/1) [storage.StorageDomain]
Removing image rundir link
u'/var/run/vdsm/storage/a08af6be-3802-4bb1-9fa5-4b6a10227290/9c896907-59b0-4983-9478-b36b2c2eb01e'
(fileSD:592)
2021-04-03 04:40:35,518+0000 INFO (jsonrpc/1) [vdsm.api] FINISH
teardownImage return=None from=::ffff:10.100.0.210,58150,
task_id=fc946d20-126a-4fd0-9078-914b4a64b1d9 (api:54)
2021-04-03 04:40:35,519+0000 INFO (jsonrpc/1) [vdsm.api] START
teardownImage(sdUUID='b891448d-dd92-4a7b-a51a-22abc3d7da67',
spUUID='78dc095a-5238-11e8-b8bf-00163e6a7af9',
imgUUID='c0e95483-35f1-4a61-958e-4e308b70d3f8', volUUID=None)
from=::ffff:10.100.0.210,58150, task_id=77c0fdca-e13a-44b5-9a00-29
0522b194b2 (api:48)
2021-04-03 04:40:35,520+0000 INFO (jsonrpc/1) [storage.StorageDomain]
Removing image rundir link
u'/var/run/vdsm/storage/b891448d-dd92-4a7b-a51a-22abc3d7da67/c0e95483-35f1-4a61-958e-4e308b70d3f8'
(fileSD:592)
2021-04-03 04:40:35,520+0000 INFO (jsonrpc/1) [vdsm.api] FINISH
teardownImage return=None from=::ffff:10.100.0.210,58150,
task_id=77c0fdca-e13a-44b5-9a00-290522b194b2 (api:54)
2021-04-03 04:40:35,521+0000 INFO (jsonrpc/1) [virt.vm]
(vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Stopping connection
(guestagent:455)
2021-04-03 04:40:35,521+0000 WARN (jsonrpc/1) [root] File:
/var/lib/libvirt/qemu/channels/58abf0cf-d7b9-4067-a86a-e619928368e7.ovirt-guest-agent.0
already removed (fileutils:54)
2021-04-03 04:40:35,521+0000 WARN (jsonrpc/1) [root] Attempting to remove
a non existing network: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7
(libvirtnetwork:198)
2021-04-03 04:40:35,522+0000 WARN (jsonrpc/1) [root] Attempting to remove
a non existing net user: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7
(libvirtnetwork:205)
2021-04-03 04:40:35,526+0000 WARN (jsonrpc/1) [root] Attempting to remove
a non existing network: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7
(libvirtnetwork:198)
2021-04-03 04:40:35,526+0000 WARN (jsonrpc/1) [root] Attempting to remove
a non existing net user: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7
(libvirtnetwork:205)
2021-04-03 04:40:35,527+0000 WARN (jsonrpc/1) [root] File:
/var/lib/libvirt/qemu/channels/58abf0cf-d7b9-4067-a86a-e619928368e7.org.qemu.guest_agent.0
already removed (fileutils:54)
2021-04-03 04:40:35,528+0000 WARN (jsonrpc/1) [virt.vm]
(vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') timestamp already removed
from stats cache (vm:2445)
2021-04-03 04:40:35,531+0000 ERROR (jsonrpc/1) [api] FINISH destroy
error='1048576' (api:134)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in
method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 333, in destroy
res = self.vm.destroy(gracefulAttempts)
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5283, in
destroy
result = self.doDestroy(gracefulAttempts, reason)
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5302, in
doDestroy
return self.releaseVm(gracefulAttempts)
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5207, in
releaseVm
self._cleanup()
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2382, in
_cleanup
self._cleanup_hugepages()
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2393, in
_cleanup_hugepages
self.nr_hugepages, self.hugepagesz
File "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 243, in
calculate_required_deallocation
_preallocated_hugepages(vm_hugepagesz))
File "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 262, in
_preallocated_hugepages
kernel_args['hugepagesz']
File "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 291, in
_cmdline_hugepagesz_to_kb
}[cmdline]
KeyError: '1048576'
2021-04-03 04:40:35,531+0000 INFO (jsonrpc/1) [api.virt] FINISH destroy
return={'status': {'message': 'General Exception:
("\'1048576\'",)',
'code': 100}} from=::ffff:10.100.0.210,58150,
vmId=58abf0cf-d7b9-4067-a86a-e619928368e7 (api:54)
2021-04-03 04:40:35,532+0000 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
call VM.destroy failed (error 100) in 0.02 seconds (__init__:312)
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KBABFNZIJ6T...