HPE Oneview KVM appliance 8.8.0 / 8.7.0
by Angus Clarke
Hello folks
I realise this probably isn't the place for this but someone might be interested or have some knowledge.
I deployed the KVM version of HPE Oneview 8.8 to oVirt 4.5 (OLVM 4.5) It came as a single QCOW2 disk image.
The VM I created needed a couple of tweaks to get it to boot (Chipset: i440FX w/Bios for the IDE disk type, OS: RHEL7 x86_64.) The VM boots and the graphical environment starts but sticks not long after (left over weekend, no change.) I did boot into the rescue environment to look at logs, I can see all the logical volumes mounted ok so no issue there, I see apps repeatedly trying to start with no obvious reason. Very similar outcome when I tried the previous 8.7 Oneview version.
We also run a VMWare estate so the sad thing is that I've fallen back to deploying the ESXi version of Oneview 8.8 - it works fine, which is a shame as I am trying to persuade our internal teams to start using oVirt/OLVM instead of VMWare!
Any feedback gladly received.
Thanks
Angus
11 months
Migration problems
by markeczzz＠gmail.com
I am trying to migrate vm from one host to another.
I have already migrated 10 vm-s from that host to another, but 2 of them are having problems.
In the dashboard event logs i get this:
Migration failed due to an Error: Migration canceled (VM: Virtual-NS, Source: node2.ovirt.cluster.com, Destination: node3.ovirt.cluster.com).
Engine log:
2024-04-11 11:11:21,490+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-74) [] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' is migrating to VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) ignoring it in the refresh until migration is done
2024-04-11 11:11:35,446+02 WARN [org.ovirt.engine.core.utils.virtiowin.VirtioWinReader] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-75) [] Directory '/usr/share/virtio-win' doesn't exist.
2024-04-11 11:11:36,521+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' is migrating to VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) ignoring it in the refresh until migration is done
2024-04-11 11:11:51,563+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-44) [] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' is migrating to VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) ignoring it in the refresh until migration is done
2024-04-11 11:12:06,592+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-45) [] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' is migrating to VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) ignoring it in the refresh until migration is done
2024-04-11 11:12:21,625+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-8) [] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' is migrating to VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) ignoring it in the refresh until migration is done
2024-04-11 11:12:36,657+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) [] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' is migrating to VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) ignoring it in the refresh until migration is done
2024-04-11 11:12:43,536+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-5) [681c3996] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' was reported as Down on VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com)
2024-04-11 11:12:43,536+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-5) [681c3996] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d'(Virtual-NS) was unexpectedly detected as 'Down' on VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) (expected on 'c1f1069d-ed61-4ade-afc2-e6f649039386')
2024-04-11 11:12:43,536+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-5) [681c3996] START, DestroyVDSCommand(HostName = node3.ovirt.cluster.com, DestroyVmVDSCommandParameters:{hostId='d90dced4-6715-41b6-953c-119c4133f9db', vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 381766c1
2024-04-11 11:12:43,836+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-5) [681c3996] Failed to destroy VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' because VM does not exist, ignoring
2024-04-11 11:12:43,836+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-5) [681c3996] FINISH, DestroyVDSCommand, return: , log id: 381766c1
2024-04-11 11:12:43,836+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-5) [681c3996] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d'(Virtual-NS) was unexpectedly detected as 'Down' on VDS 'd90dced4-6715-41b6-953c-119c4133f9db'(node3.ovirt.cluster.com) (expected on 'c1f1069d-ed61-4ade-afc2-e6f649039386')
2024-04-11 11:12:43,836+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-5) [681c3996] Migration of VM 'Virtual-NS' to host 'node3.ovirt.cluster.com' failed: VM destroyed during the startup.
2024-04-11 11:12:43,842+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-1) [681c3996] VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d'(Virtual-NS) moved from 'MigratingFrom' --> 'Up'
2024-04-11 11:12:43,842+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-1) [681c3996] Adding VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d'(Virtual-NS) to re-run list
2024-04-11 11:12:43,843+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-1) [681c3996] Rerun VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d'. Called from VDS 'node2.ovirt.cluster.com'
2024-04-11 11:12:43,889+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-9915782) [681c3996] START, MigrateStatusVDSCommand(HostName = node2.ovirt.cluster.com, MigrateStatusVDSCommandParameters:{hostId='c1f1069d-ed61-4ade-afc2-e6f649039386', vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d'}), log id: 41a9e7a3
2024-04-11 11:12:43,892+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-9915782) [681c3996] FINISH, MigrateStatusVDSCommand, return: org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusReturn@4b4bced2, log id: 41a9e7a3
2024-04-11 11:12:43,920+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-9915782) [681c3996] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed due to an Error: Migration canceled (VM: Virtual-NS, Source: node2.ovirt.cluster.com, Destination: node3.ovirt.cluster.com).
2024-04-11 11:12:43,922+02 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (EE-ManagedThreadFactory-engine-Thread-9915782) [681c3996] Lock freed to object 'EngineLock:{exclusiveLocks='[c8e6aafe-1463-4db6-9d3b-76b234f9487d=VM]', sharedLocks=''}'
2024-04-11 11:12:51,685+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-55) [] Fetched 13 VMs from VDS 'd90dced4-6715-41b6-953c-119c4133f9db'
vdsm logs node3:
2024-04-11 11:12:42,360+0200 INFO (jsonrpc/3) [api.host] START getAllVmIoTunePolicies() from=::1,44562 (api:48)
2024-04-11 11:12:42,360+0200 INFO (jsonrpc/3) [api.host] FINISH getAllVmIoTunePolicies return={'status': {'code': 0, 'message': 'Done'}, 'io_tune_policies_dict': {'1a2808ce-5ba4-4f19-9809-baffecb51d83': {'policy': [], 'current_values': [{'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/8ab48aab-a522-4331-ba46-904e13f64d38/images/8766b16f-f587-4475-8e72-fc83486516b9/3616942e-88f1-4327-aad8-2d8d4550f8e7', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, 'f72cc0b9-0bdb-4d23-b50b-99e775d87f1c': {'policy': [], 'current_values': [{'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0/images/f056a516-2257-4f0f-a972-031cf67dc4df/433fd71e-d4bb-4341-ad45-34fa4fd401a1', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD
/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/d5780d31-929b-46a6-b66e-38bcf57695d2/6b477529-daca-4480-9266-27d46ea6063e', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '148118a7-6b05-4d74-9d68-672d8c40f431': {'policy': [], 'current_values': [{'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/c0b21e2c-6b4c-4769-8244-aa69a32c27f2/images/735770ac-7e7f-44c9-a768-3eeeac1ff0a8/aa387d94-ef62-45e9-aa11-e1ac6fcb138f', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, 'db853022-cee8-4f71-ad64-e70bb314686c': {'policy': [], 'current_values': [{'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/1294d8a3-ad40-4b06-9188-d59f527b921d/f944a7af-cda0-4d75-9c39-7b5063dae0cb', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec':
0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/c0b21e2c-6b4c-4769-8244-aa69a32c27f2/images/eed13b69-80cd-4742-b706-bce2f1370fa3/2163d081-bbe0-4d62-9d96-6bf0afd3260c', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '4490de5d-1a95-4906-a197-a471d95ab6b4': {'policy': [], 'current_values': [{'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/454c4196-d772-43fd-99bb-4ac90150cd3c/9779ba55-160d-431a-b408-857e106e5cbe', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/8ab48aab-a522-4331-ba46-904e13f64d38/images/8d782dca-e45b-440c-b0a8-808d2f528dcd/70e4a01f-619e-4764-928f-1cc1207767e8', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec
': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '896840ab-2996-4c5b-afb2-c6a359ab6eda': {'policy': [], 'current_values': [{'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/454c4196-d772-43fd-99bb-4ac90150cd3c/9779ba55-160d-431a-b408-857e106e5cbe', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/8ab48aab-a522-4331-ba46-904e13f64d38/images/a0defce2-7a1a-4872-8141-11df3e1bcba8/5f1745e6-3e0b-4997-915b-f13314fe9092', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '4eeb0711-a8e8-4c0d-aa29-0c9361e15554': {'policy': [], 'current_values': [{'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/260444be-9bd3-43be-bd08-9c494d2cc269/f5658dde-
9853-4d02-9b41-93501633e9e2', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/c0b21e2c-6b4c-4769-8244-aa69a32c27f2/images/f8cb52ef-0dee-440c-96c7-9355522c8674/927d5b15-f46e-41a8-bf6e-053a6cc7cac8', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '0b528685-134b-48ff-9eb0-a8da6d1f3e70': {'policy': [], 'current_values': [{'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/c0b21e2c-6b4c-4769-8244-aa69a32c27f2/images/ae598215-87ed-4ff3-9c3e-66a64e2f55da/320b1540-59c2-495e-96c3-3f523834662f', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '4e621664-b22b-430d-9e90-4f65ac4e99b7': {'policy': [], 'current_values': [{'name': 'sda', 'path': '/rhev/data-center/mn
t/blockSD/c0b21e2c-6b4c-4769-8244-aa69a32c27f2/images/9e37bac4-10f3-4b6d-97db-fb2339b7282d/5595abba-8a8e-4dfe-a4c1-792d61317436', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '88516219-60bf-4d2f-8d50-96414327fe19': {'policy': [], 'current_values': [{'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/454c4196-d772-43fd-99bb-4ac90150cd3c/9779ba55-160d-431a-b408-857e106e5cbe', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/8ab48aab-a522-4331-ba46-904e13f64d38/images/bb87b088-9c6f-4c6e-a2c5-4e10152ae10f/066bcd99-d1d6-411a-a0a2-b62a590112cb', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '281b503c-a76b-4755
-89f1-8a4b3eedb2a8': {'policy': [], 'current_values': [{'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0/images/183bdbcc-b2af-4639-8866-c7066a211378/1ca39c83-59e7-4ed7-b5b9-11e5f3b42cb7', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sdb', 'path': '/rhev/data-center/mnt/blockSD/2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0/images/7c15963f-ebc0-4c87-bc80-f8a64ef8f6db/6e3a4985-4cc9-4fa0-aa67-04297d31a107', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, '5d0fabc9-8671-4f81-9449-32e36091f4f1': {'policy': [], 'current_values': [{'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0/images/460e3282-969f-48fc-9028-8e33e3bbd2dc/396a7e79-2cd5-4815-904a-cfa220a81b14', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_
bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sdb', 'path': '/rhev/data-center/mnt/blockSD/2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0/images/e901a7d6-b029-4d83-93b2-522d1e061cd7/64749e34-56be-49fd-890b-496b19bdf45a', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, 'cc7931ff-8124-4724-9242-abea2ab5bf42': {'policy': [], 'current_values': [{'name': 'vda', 'path': '/run/vdsm/storage/73e03870-1bad-42d2-bf7b-a55041405c71/737c72eb-4e5b-46b6-ba7d-36f25880ceae/e0f8577d-a4bb-4a28-9780-a4cd1e6a2bb1', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}}} from=::1,44562 (api:54)
2024-04-11 11:12:42,752+0200 INFO (jsonrpc/2) [api.host] START getStats() from=::1,44558 (api:48)
2024-04-11 11:12:42,773+0200 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=()) from=::1,44558, task_id=dce62f2b-b9e4-4641-969a-a34c4dccc5b7 (api:48)
2024-04-11 11:12:42,773+0200 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '0.4', 'delay': '0.000354071', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'code': 0, 'lastCheck': '1.2', 'delay': '0.000259322', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '7f655239-96f9-48f9-99e0-b947220c289e': {'code': 0, 'lastCheck': '1.2', 'delay': '0.000269234', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'code': 0, 'lastCheck': '1.2', 'delay': '0.000352778', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'code': 0, 'lastCheck': '1.3', 'delay': '0.000313396', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8ab48aab-a522-4331-ba46-904e13f64d38': {'code': 0, 'lastCheck': '1.2', 'delay': '0.000272123', 'valid': True, 'ver
sion': 5, 'acquired': True, 'actual': True}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'code': 0, 'lastCheck': '1.8', 'delay': '0.000403821', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,44558, task_id=dce62f2b-b9e4-4641-969a-a34c4dccc5b7 (api:54)
2024-04-11 11:12:42,774+0200 INFO (jsonrpc/2) [vdsm.api] START multipath_health() from=::1,44558, task_id=845b5ad7-5f99-40ac-b5b1-dd9700e7eca7 (api:48)
2024-04-11 11:12:42,774+0200 INFO (jsonrpc/2) [vdsm.api] FINISH multipath_health return={} from=::1,44558, task_id=845b5ad7-5f99-40ac-b5b1-dd9700e7eca7 (api:54)
2024-04-11 11:12:42,788+0200 INFO (jsonrpc/2) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,44558 (api:54)
2024-04-11 11:12:43,517+0200 INFO (libvirt/events) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') underlying process disconnected (vm:1135)
2024-04-11 11:12:43,517+0200 INFO (libvirt/events) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Release VM resources (vm:5357)
2024-04-11 11:12:43,517+0200 INFO (libvirt/events) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Stopping connection (guestagent:438)
2024-04-11 11:12:43,517+0200 INFO (libvirt/events) [vdsm.api] START teardownImage(sdUUID='8ab48aab-a522-4331-ba46-904e13f64d38', spUUID='07e52ac2-ab5f-11ec-b531-00163e400079', imgUUID='83f144ac-dfaa-4ede-a0f5-50338f6e1335', volUUID=None) from=internal, task_id=1ac73c98-9313-4e33-aee8-beaea54cb3a5 (api:48)
2024-04-11 11:12:43,518+0200 INFO (libvirt/events) [storage.StorageDomain] Removing image run directory '/run/vdsm/storage/8ab48aab-a522-4331-ba46-904e13f64d38/83f144ac-dfaa-4ede-a0f5-50338f6e1335' (blockSD:1387)
2024-04-11 11:12:43,518+0200 INFO (libvirt/events) [storage.fileUtils] Removing directory: /run/vdsm/storage/8ab48aab-a522-4331-ba46-904e13f64d38/83f144ac-dfaa-4ede-a0f5-50338f6e1335 (fileUtils:182)
2024-04-11 11:12:43,527+0200 INFO (jsonrpc/0) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:10.3.8.12,42396, vmId=c8e6aafe-1463-4db6-9d3b-76b234f9487d (api:48)
2024-04-11 11:12:43,532+0200 WARN (vm/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Couldn't destroy incoming VM: Domain not found: no domain with matching uuid 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' (vm:4100)
2024-04-11 11:12:43,532+0200 INFO (vm/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Changed state to Down: VM destroyed during the startup (code=10) (vm:1921)
2024-04-11 11:12:43,533+0200 INFO (vm/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Stopping connection (guestagent:438)
2024-04-11 11:12:43,537+0200 INFO (jsonrpc/6) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:10.3.8.30,56070, flow_id=681c3996, vmId=c8e6aafe-1463-4db6-9d3b-76b234f9487d (api:48)
2024-04-11 11:12:43,615+0200 INFO (libvirt/events) [storage.LVM] Deactivating lvs: vg=8ab48aab-a522-4331-ba46-904e13f64d38 lvs=['da4b0e32-9649-4f7d-834e-e0676a00916b'] (lvm:1802)
2024-04-11 11:12:43,728+0200 INFO (libvirt/events) [vdsm.api] FINISH teardownImage return=None from=internal, task_id=1ac73c98-9313-4e33-aee8-beaea54cb3a5 (api:54)
2024-04-11 11:12:43,729+0200 INFO (libvirt/events) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Stopping connection (guestagent:438)
2024-04-11 11:12:43,729+0200 WARN (libvirt/events) [root] Attempting to remove a non existing net user: ovirtmgmt/c8e6aafe-1463-4db6-9d3b-76b234f9487d (libvirtnetwork:207)
2024-04-11 11:12:43,729+0200 INFO (libvirt/events) [vdsm.api] START inappropriateDevices(thiefId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') from=internal, task_id=834a0e43-879e-4d32-b719-50f4214b7d38 (api:48)
2024-04-11 11:12:43,730+0200 INFO (libvirt/events) [vdsm.api] FINISH inappropriateDevices return=None from=internal, task_id=834a0e43-879e-4d32-b719-50f4214b7d38 (api:54)
2024-04-11 11:12:43,832+0200 INFO (libvirt/events) [root] /usr/libexec/vdsm/hooks/after_vm_destroy/delete_vhostuserclient_hook: rc=0 err=b'' (hooks:122)
2024-04-11 11:12:43,833+0200 WARN (libvirt/events) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') trying to set state to Down when already Down (vm:702)
2024-04-11 11:12:43,833+0200 INFO (libvirt/events) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Stopping connection (guestagent:438)
2024-04-11 11:12:43,834+0200 INFO (jsonrpc/0) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Can't undefine disconnected VM 'c8e6aafe-1463-4db6-9d3b-76b234f9487d' (vm:2564)
2024-04-11 11:12:43,834+0200 INFO (jsonrpc/0) [api.virt] FINISH destroy return={'status': {'code': 0, 'message': 'Machine destroyed'}} from=::ffff:10.3.8.12,42396, vmId=c8e6aafe-1463-4db6-9d3b-76b234f9487d (api:54)
2024-04-11 11:12:43,835+0200 INFO (jsonrpc/6) [api] FINISH destroy error=Virtual machine does not exist: {'vmId': 'c8e6aafe-1463-4db6-9d3b-76b234f9487d'} (api:129)
2024-04-11 11:12:43,835+0200 INFO (jsonrpc/6) [api.virt] FINISH destroy return={'status': {'code': 1, 'message': "Virtual machine does not exist: {'vmId': 'c8e6aafe-1463-4db6-9d3b-76b234f9487d'}"}} from=::ffff:10.3.8.30,56070, flow_id=681c3996, vmId=c8e6aafe-1463-4db6-9d3b-76b234f9487d (api:54)
2024-04-11 11:12:43,835+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call VM.destroy failed (error 1) in 0.30 seconds (__init__:312)
2024-04-11 11:12:45,602+0200 INFO (jsonrpc/1) [api.host] START getStats() from=::ffff:10.3.8.30,56070 (api:48)
vdsm node2 log:
2024-04-11 11:12:27,504+0200 INFO (jsonrpc/1) [api.host] START getStats() from=::1,33628 (api:48)
2024-04-11 11:12:27,516+0200 INFO (jsonrpc/1) [vdsm.api] START repoStats(domains=()) from=::1,33628, task_id=782990af-56f0-413f-b511-64e60e606537 (api:48)
2024-04-11 11:12:27,516+0200 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '6.9', 'delay': '0.000313373', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8ab48aab-a522-4331-ba46-904e13f64d38': {'code': 0, 'lastCheck': '2.4', 'delay': '0.00024877', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'code': 0, 'lastCheck': '3.1', 'delay': '0.000317606', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '7f655239-96f9-48f9-99e0-b947220c289e': {'code': 0, 'lastCheck': '2.4', 'delay': '0.000527974', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'code': 0, 'lastCheck': '2.4', 'delay': '0.000303832', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'code': 0, 'lastCheck': '2.3', 'delay': '0.00288366', 'valid': True, 'versi
on': 5, 'acquired': True, 'actual': True}, '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'code': 0, 'lastCheck': '2.3', 'delay': '0.000218979', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=782990af-56f0-413f-b511-64e60e606537 (api:54)
2024-04-11 11:12:27,516+0200 INFO (jsonrpc/1) [vdsm.api] START multipath_health() from=::1,33628, task_id=3427f545-0183-49fd-98de-54a52823db1f (api:48)
2024-04-11 11:12:27,516+0200 INFO (jsonrpc/1) [vdsm.api] FINISH multipath_health return={} from=::1,33628, task_id=3427f545-0183-49fd-98de-54a52823db1f (api:54)
2024-04-11 11:12:27,522+0200 INFO (jsonrpc/1) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,33628 (api:54)
2024-04-11 11:12:30,597+0200 INFO (jsonrpc/4) [api.host] START getStats() from=::ffff:10.3.8.30,52426 (api:48)
2024-04-11 11:12:30,610+0200 INFO (jsonrpc/4) [vdsm.api] START repoStats(domains=()) from=::ffff:10.3.8.30,52426, task_id=eb4be2c4-919e-46d9-80c5-fb4bd111dc1f (api:48)
2024-04-11 11:12:30,610+0200 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '0.0', 'delay': '0.000296656', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8ab48aab-a522-4331-ba46-904e13f64d38': {'code': 0, 'lastCheck': '5.5', 'delay': '0.00024877', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'code': 0, 'lastCheck': '6.2', 'delay': '0.000317606', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '7f655239-96f9-48f9-99e0-b947220c289e': {'code': 0, 'lastCheck': '5.5', 'delay': '0.000527974', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'code': 0, 'lastCheck': '5.5', 'delay': '0.000303832', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'code': 0, 'lastCheck': '5.4', 'delay': '0.00288366', 'valid': True, 'versi
on': 5, 'acquired': True, 'actual': True}, '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'code': 0, 'lastCheck': '5.4', 'delay': '0.000218979', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::ffff:10.3.8.30,52426, task_id=eb4be2c4-919e-46d9-80c5-fb4bd111dc1f (api:54)
2024-04-11 11:12:30,611+0200 INFO (jsonrpc/4) [vdsm.api] START multipath_health() from=::ffff:10.3.8.30,52426, task_id=33c103f8-5221-448c-8ef3-900af4a0ff1d (api:48)
2024-04-11 11:12:30,611+0200 INFO (jsonrpc/4) [vdsm.api] FINISH multipath_health return={} from=::ffff:10.3.8.30,52426, task_id=33c103f8-5221-448c-8ef3-900af4a0ff1d (api:54)
2024-04-11 11:12:30,618+0200 INFO (jsonrpc/4) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::ffff:10.3.8.30,52426 (api:54)
2024-04-11 11:12:32,056+0200 INFO (migmon/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') new computed progress 94 < than old value 99, discarded (migration:255)
2024-04-11 11:12:32,057+0200 WARN (migmon/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Migration stalling: remaining (479MiB) > lowmark (185MiB). (migration:873)
2024-04-11 11:12:32,057+0200 INFO (migmon/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Migration Progress: 470.127 seconds elapsed, 98% of data processed, total data: 16484MB, processed data: 28862MB, remaining data: 479MB, transfer speed 496Mbps, zero pages: 515507MB, compressed: 9MB, dirty rate: 13508, memory iteration: 9 (migration:886)
2024-04-11 11:12:33,501+0200 INFO (jsonrpc/6) [api.virt] START getStats() from=::1,33628, vmId=cc7931ff-8124-4724-9242-abea2ab5bf42 (api:48)
2024-04-11 11:12:33,501+0200 INFO (jsonrpc/6) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': 'cc7931ff-8124-4724-9242-abea2ab5bf42'} (api:129)
2024-04-11 11:12:33,501+0200 INFO (jsonrpc/6) [api.virt] FINISH getStats return={'status': {'code': 1, 'message': "Virtual machine does not exist: {'vmId': 'cc7931ff-8124-4724-9242-abea2ab5bf42'}"}} from=::1,33628, vmId=cc7931ff-8124-4724-9242-abea2ab5bf42 (api:54)
2024-04-11 11:12:33,501+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312)
2024-04-11 11:12:33,754+0200 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=['73e03870-1bad-42d2-bf7b-a55041405c71']) from=::1,33628, task_id=16d2af05-faac-4854-89e6-fb3f39452318 (api:48)
2024-04-11 11:12:33,754+0200 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '3.1', 'delay': '0.000296656', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=16d2af05-faac-4854-89e6-fb3f39452318 (api:54)
2024-04-11 11:12:34,124+0200 INFO (jsonrpc/1) [vdsm.api] START repoStats(domains=['73e03870-1bad-42d2-bf7b-a55041405c71']) from=::1,33628, task_id=bdb6aa5e-b9c4-4832-bb8f-fec782cbbde0 (api:48)
2024-04-11 11:12:34,124+0200 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '3.5', 'delay': '0.000296656', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=bdb6aa5e-b9c4-4832-bb8f-fec782cbbde0 (api:54)
2024-04-11 11:12:34,928+0200 INFO (jsonrpc/4) [vdsm.api] START getSpmStatus(spUUID='07e52ac2-ab5f-11ec-b531-00163e400079') from=::ffff:10.3.8.30,52426, task_id=5e808bd5-fefd-454c-a5f6-4785bfad013e (api:48)
2024-04-11 11:12:34,931+0200 INFO (jsonrpc/6) [api.host] START getStats() from=::1,33628 (api:48)
2024-04-11 11:12:34,944+0200 INFO (jsonrpc/6) [vdsm.api] START repoStats(domains=()) from=::1,33628, task_id=2e5ceb8f-f518-46eb-88b7-e9caf55de687 (api:48)
2024-04-11 11:12:34,944+0200 INFO (jsonrpc/6) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '4.3', 'delay': '0.000296656', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8ab48aab-a522-4331-ba46-904e13f64d38': {'code': 0, 'lastCheck': '4.3', 'delay': '0.00024877', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'code': 0, 'lastCheck': '0.5', 'delay': '0.000262637', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '7f655239-96f9-48f9-99e0-b947220c289e': {'code': 0, 'lastCheck': '4.3', 'delay': '0.000527974', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'code': 0, 'lastCheck': '4.3', 'delay': '0.000303832', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'code': 0, 'lastCheck': '4.3', 'delay': '0.00288366', 'valid': True, 'versi
on': 5, 'acquired': True, 'actual': True}, '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'code': 0, 'lastCheck': '4.3', 'delay': '0.000218979', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=2e5ceb8f-f518-46eb-88b7-e9caf55de687 (api:54)
2024-04-11 11:12:34,945+0200 INFO (jsonrpc/6) [vdsm.api] START multipath_health() from=::1,33628, task_id=23391530-d739-47ee-8799-d3db76e805ca (api:48)
2024-04-11 11:12:34,945+0200 INFO (jsonrpc/6) [vdsm.api] FINISH multipath_health return={} from=::1,33628, task_id=23391530-d739-47ee-8799-d3db76e805ca (api:54)
2024-04-11 11:12:34,946+0200 INFO (jsonrpc/4) [vdsm.api] FINISH getSpmStatus return={'spm_st': {'spmStatus': 'SPM', 'spmLver': 13, 'spmId': 2}} from=::ffff:10.3.8.30,52426, task_id=5e808bd5-fefd-454c-a5f6-4785bfad013e (api:54)
2024-04-11 11:12:34,952+0200 INFO (jsonrpc/6) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,33628 (api:54)
2024-04-11 11:12:34,989+0200 INFO (jsonrpc/3) [vdsm.api] START getStoragePoolInfo(spUUID='07e52ac2-ab5f-11ec-b531-00163e400079') from=::ffff:10.3.8.30,52436, task_id=61162b84-6df1-4916-a959-20214aa7bc2b (api:48)
2024-04-11 11:12:34,992+0200 INFO (jsonrpc/3) [vdsm.api] FINISH getStoragePoolInfo return={'info': {'domains': '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0:Active,848dac7f-a73d-476f-9371-b54ac8b647de:Active,8ab48aab-a522-4331-ba46-904e13f64d38:Active,c0b21e2c-6b4c-4769-8244-aa69a32c27f2:Active,aa2fa425-8bf3-44b6-b353-81de4e82764a:Active,7f655239-96f9-48f9-99e0-b947220c289e:Active,73e03870-1bad-42d2-bf7b-a55041405c71:Active', 'isoprefix': '', 'lver': 13, 'master_uuid': 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2', 'master_ver': 2, 'name': 'No Description', 'pool_status': 'connected', 'spm_id': 2, 'type': 'FCP', 'version': '5'}, 'dominfo': {'2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'status': 'Active', 'alerts': [], 'isoprefix': '', 'version': 5, 'disktotal': '3298132230144', 'diskfree': '1233997791232'}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'status': 'Active', 'alerts': [], 'isoprefix': '', 'version': 5, 'disktotal': '321719894016', 'diskfree': '316619620352'}, '8ab48aab-a522-4331-ba46-904e13f6
4d38': {'status': 'Active', 'alerts': [], 'isoprefix': '', 'version': 5, 'disktotal': '2198620602368', 'diskfree': '1906965479424'}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'status': 'Active', 'alerts': [], 'isoprefix': '', 'version': 5, 'disktotal': '3298132230144', 'diskfree': '1311709855744'}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'status': 'Active', 'alerts': [], 'isoprefix': '', 'version': 5, 'disktotal': '106971529216', 'diskfree': '63350767616'}, '7f655239-96f9-48f9-99e0-b947220c289e': {'status': 'Active', 'alerts': [], 'isoprefix': '', 'version': 5, 'disktotal': '1099108974592', 'diskfree': '1071460122624'}, '73e03870-1bad-42d2-bf7b-a55041405c71': {'status': 'Active', 'alerts': [], 'isoprefix': '', 'version': 5, 'disktotal': '106971529216', 'diskfree': '11945377792'}}} from=::ffff:10.3.8.30,52436, task_id=61162b84-6df1-4916-a959-20214aa7bc2b (api:54)
2024-04-11 11:12:35,536+0200 INFO (periodic/2) [vdsm.api] START getVolumeSize(sdUUID='8ab48aab-a522-4331-ba46-904e13f64d38', spUUID='07e52ac2-ab5f-11ec-b531-00163e400079', imgUUID='83f144ac-dfaa-4ede-a0f5-50338f6e1335', volUUID='da4b0e32-9649-4f7d-834e-e0676a00916b') from=internal, task_id=d6717cfc-a75c-4f1d-baa9-887711f4732f (api:48)
2024-04-11 11:12:35,536+0200 INFO (periodic/2) [vdsm.api] FINISH getVolumeSize return={'apparentsize': '110595407872', 'truesize': '110595407872'} from=internal, task_id=d6717cfc-a75c-4f1d-baa9-887711f4732f (api:54)
2024-04-11 11:12:35,537+0200 INFO (periodic/1) [vdsm.api] START getVolumeSize(sdUUID='c0b21e2c-6b4c-4769-8244-aa69a32c27f2', spUUID='07e52ac2-ab5f-11ec-b531-00163e400079', imgUUID='f194f2e8-0191-4073-90f6-21e6ddc3e350', volUUID='f66d26c5-7b48-487a-92d2-dba2d588abff') from=internal, task_id=f65d5fdc-0b1a-4cc0-9a82-8b0a4c28556b (api:48)
2024-04-11 11:12:35,537+0200 INFO (periodic/1) [vdsm.api] FINISH getVolumeSize return={'apparentsize': '340376158208', 'truesize': '340376158208'} from=internal, task_id=f65d5fdc-0b1a-4cc0-9a82-8b0a4c28556b (api:54)
2024-04-11 11:12:37,549+0200 INFO (jsonrpc/7) [api.host] START getStats() from=::1,33628 (api:48)
2024-04-11 11:12:37,562+0200 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=()) from=::1,33628, task_id=bd232d01-4552-4937-b2a3-c4b8ec6d9910 (api:48)
2024-04-11 11:12:37,562+0200 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '6.9', 'delay': '0.000296656', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8ab48aab-a522-4331-ba46-904e13f64d38': {'code': 0, 'lastCheck': '2.4', 'delay': '0.000202745', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'code': 0, 'lastCheck': '3.1', 'delay': '0.000262637', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '7f655239-96f9-48f9-99e0-b947220c289e': {'code': 0, 'lastCheck': '2.4', 'delay': '0.000343811', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'code': 0, 'lastCheck': '2.4', 'delay': '0.000232965', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'code': 0, 'lastCheck': '2.4', 'delay': '0.00022068', 'valid': True, 'vers
ion': 5, 'acquired': True, 'actual': True}, '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'code': 0, 'lastCheck': '2.4', 'delay': '0.000188189', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=bd232d01-4552-4937-b2a3-c4b8ec6d9910 (api:54)
2024-04-11 11:12:37,562+0200 INFO (jsonrpc/7) [vdsm.api] START multipath_health() from=::1,33628, task_id=a6903cbf-c31e-45d4-b662-5167cd499faf (api:48)
2024-04-11 11:12:37,562+0200 INFO (jsonrpc/7) [vdsm.api] FINISH multipath_health return={} from=::1,33628, task_id=a6903cbf-c31e-45d4-b662-5167cd499faf (api:54)
2024-04-11 11:12:37,569+0200 INFO (jsonrpc/7) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,33628 (api:54)
2024-04-11 11:12:40,651+0200 INFO (jsonrpc/0) [api.host] START getAllVmStats() from=::ffff:10.3.8.30,52426 (api:48)
2024-04-11 11:12:40,651+0200 INFO (jsonrpc/0) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') new computed progress 98 < than old value 99, discarded (migration:255)
2024-04-11 11:12:40,653+0200 INFO (jsonrpc/0) [api.host] FINISH getAllVmStats return={'status': {'code': 0, 'message': 'Done'}, 'statsList': (suppressed)} from=::ffff:10.3.8.30,52426 (api:54)
2024-04-11 11:12:40,867+0200 INFO (periodic/2) [vdsm.api] START repoStats(domains=()) from=internal, task_id=f679c2f0-24e4-454c-b511-653b39c23998 (api:48)
2024-04-11 11:12:40,867+0200 INFO (periodic/2) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '0.2', 'delay': '0.000529618', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8ab48aab-a522-4331-ba46-904e13f64d38': {'code': 0, 'lastCheck': '0.2', 'delay': '0.000202745', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'code': 0, 'lastCheck': '0.2', 'delay': '0.000262637', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '7f655239-96f9-48f9-99e0-b947220c289e': {'code': 0, 'lastCheck': '0.2', 'delay': '0.000343811', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'code': 0, 'lastCheck': '0.2', 'delay': '0.000232965', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'code': 0, 'lastCheck': '0.2', 'delay': '0.00022068', 'valid': True, 'ver
sion': 5, 'acquired': True, 'actual': True}, '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'code': 0, 'lastCheck': '0.2', 'delay': '0.000188189', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=internal, task_id=f679c2f0-24e4-454c-b511-653b39c23998 (api:54)
2024-04-11 11:12:42,059+0200 INFO (migmon/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') new computed progress 98 < than old value 99, discarded (migration:255)
2024-04-11 11:12:42,059+0200 WARN (migmon/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Migration stalling: remaining (709MiB) > lowmark (185MiB). (migration:873)
2024-04-11 11:12:42,059+0200 WARN (migmon/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Aborting migration (migration:922)
2024-04-11 11:12:42,455+0200 INFO (jsonrpc/1) [api.host] START getAllVmStats() from=::1,33630 (api:48)
2024-04-11 11:12:42,455+0200 INFO (jsonrpc/1) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') new computed progress 98 < than old value 99, discarded (migration:255)
2024-04-11 11:12:42,456+0200 INFO (jsonrpc/1) [api.host] FINISH getAllVmStats return={'status': {'code': 0, 'message': 'Done'}, 'statsList': (suppressed)} from=::1,33630 (api:54)
2024-04-11 11:12:42,462+0200 INFO (jsonrpc/5) [api.host] START getAllVmIoTunePolicies() from=::1,33630 (api:48)
2024-04-11 11:12:42,462+0200 INFO (jsonrpc/5) [api.host] FINISH getAllVmIoTunePolicies return={'status': {'code': 0, 'message': 'Done'}, 'io_tune_policies_dict': {'c8e6aafe-1463-4db6-9d3b-76b234f9487d': {'policy': [], 'current_values': [{'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/a57a0a15-1b0a-4f7d-83a0-d59815080caf/d0ba288d-23cf-451f-bd76-d4a7bde6032f', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/8ab48aab-a522-4331-ba46-904e13f64d38/images/83f144ac-dfaa-4ede-a0f5-50338f6e1335/da4b0e32-9649-4f7d-834e-e0676a00916b', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}, 'b77300a2-2422-4bfb-a6f6-eea9f4bd4e89': {'policy': [], 'current_values': [{'name': 'sdc', 'path': '/rhev/data-center/mnt/blockSD
/aa2fa425-8bf3-44b6-b353-81de4e82764a/images/1294d8a3-ad40-4b06-9188-d59f527b921d/f944a7af-cda0-4d75-9c39-7b5063dae0cb', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}, {'name': 'sda', 'path': '/rhev/data-center/mnt/blockSD/c0b21e2c-6b4c-4769-8244-aa69a32c27f2/images/f194f2e8-0191-4073-90f6-21e6ddc3e350/f66d26c5-7b48-487a-92d2-dba2d588abff', 'ioTune': {'total_bytes_sec': 0, 'read_bytes_sec': 0, 'write_bytes_sec': 0, 'total_iops_sec': 0, 'write_iops_sec': 0, 'read_iops_sec': 0}}]}}} from=::1,33630 (api:54)
2024-04-11 11:12:43,518+0200 INFO (jsonrpc/6) [api.virt] START getStats() from=::1,33628, vmId=cc7931ff-8124-4724-9242-abea2ab5bf42 (api:48)
2024-04-11 11:12:43,519+0200 INFO (jsonrpc/6) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': 'cc7931ff-8124-4724-9242-abea2ab5bf42'} (api:129)
2024-04-11 11:12:43,519+0200 INFO (jsonrpc/6) [api.virt] FINISH getStats return={'status': {'code': 1, 'message': "Virtual machine does not exist: {'vmId': 'cc7931ff-8124-4724-9242-abea2ab5bf42'}"}} from=::1,33628, vmId=cc7931ff-8124-4724-9242-abea2ab5bf42 (api:54)
2024-04-11 11:12:43,519+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312)
2024-04-11 11:12:43,523+0200 ERROR (migsrc/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') operation aborted: migration out job: canceled by client (migration:331)
2024-04-11 11:12:43,771+0200 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=['73e03870-1bad-42d2-bf7b-a55041405c71']) from=::1,33628, task_id=678887b6-dc37-4ad3-b1e5-b6db22c9f0e4 (api:48)
2024-04-11 11:12:43,771+0200 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '3.1', 'delay': '0.000529618', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=678887b6-dc37-4ad3-b1e5-b6db22c9f0e4 (api:54)
2024-04-11 11:12:43,839+0200 INFO (migsrc/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Switching from State.STARTED to State.FAILED (migration:234)
2024-04-11 11:12:43,839+0200 ERROR (migsrc/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Failed to migrate (migration:503)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 477, in _regular_run
time.time(), machineParams
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 578, in _startUnderlyingMigration
self._perform_with_conv_schedule(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 667, in _perform_with_conv_schedule
self._perform_migration(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 596, in _perform_migration
self._migration_flags)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 159, in call
return getattr(self._vm._dom, name)(*a, **kw)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 101, in f
ret = attr(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/function.py", line 94, in wrapper
return func(inst, *args, **kwargs)
File "/usr/lib64/python3.6/site-packages/libvirt.py", line 2126, in migrateToURI3
raise libvirtError('virDomainMigrateToURI3() failed')
libvirt.libvirtError: operation aborted: migration out job: canceled by client
2024-04-11 11:12:43,839+0200 INFO (migsrc/c8e6aafe) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') Enabling drive monitoring (drivemonitor:52)
2024-04-11 11:12:43,890+0200 INFO (jsonrpc/7) [api.virt] START getMigrationStatus() from=::ffff:10.3.8.30,52426, flow_id=681c3996, vmId=c8e6aafe-1463-4db6-9d3b-76b234f9487d (api:48)
2024-04-11 11:12:43,890+0200 INFO (jsonrpc/7) [virt.vm] (vmId='c8e6aafe-1463-4db6-9d3b-76b234f9487d') new computed progress 98 < than old value 99, discarded (migration:255)
2024-04-11 11:12:43,890+0200 INFO (jsonrpc/7) [api.virt] FINISH getMigrationStatus return={'status': {'code': 0, 'message': 'Done'}, 'migrationStats': {'status': {'code': 47, 'message': 'Migration canceled'}, 'progress': 99}} from=::ffff:10.3.8.30,52426, flow_id=681c3996, vmId=c8e6aafe-1463-4db6-9d3b-76b234f9487d (api:54)
2024-04-11 11:12:44,215+0200 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=['73e03870-1bad-42d2-bf7b-a55041405c71']) from=::1,33628, task_id=f98771e0-235b-4c8e-a445-1f72ab61c1d8 (api:48)
2024-04-11 11:12:44,215+0200 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '3.6', 'delay': '0.000529618', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=f98771e0-235b-4c8e-a445-1f72ab61c1d8 (api:54)
2024-04-11 11:12:44,976+0200 INFO (jsonrpc/6) [api.host] START getStats() from=::1,33628 (api:48)
2024-04-11 11:12:44,989+0200 INFO (jsonrpc/6) [vdsm.api] START repoStats(domains=()) from=::1,33628, task_id=f7db90cd-d4ed-4670-83f1-d4336f07cc2e (api:48)
2024-04-11 11:12:44,989+0200 INFO (jsonrpc/6) [vdsm.api] FINISH repoStats return={'73e03870-1bad-42d2-bf7b-a55041405c71': {'code': 0, 'lastCheck': '4.4', 'delay': '0.000529618', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8ab48aab-a522-4331-ba46-904e13f64d38': {'code': 0, 'lastCheck': '4.4', 'delay': '0.000202745', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'c0b21e2c-6b4c-4769-8244-aa69a32c27f2': {'code': 0, 'lastCheck': '0.5', 'delay': '0.000303342', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '7f655239-96f9-48f9-99e0-b947220c289e': {'code': 0, 'lastCheck': '4.3', 'delay': '0.000343811', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '848dac7f-a73d-476f-9371-b54ac8b647de': {'code': 0, 'lastCheck': '4.3', 'delay': '0.000232965', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'aa2fa425-8bf3-44b6-b353-81de4e82764a': {'code': 0, 'lastCheck': '4.4', 'delay': '0.00022068', 'valid': True, 'vers
ion': 5, 'acquired': True, 'actual': True}, '2a6e7a60-ae0b-4aa2-9b44-1fe9ca8e30b0': {'code': 0, 'lastCheck': '4.4', 'delay': '0.000188189', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,33628, task_id=f7db90cd-d4ed-4670-83f1-d4336f07cc2e (api:54)
2024-04-11 11:12:44,990+0200 INFO (jsonrpc/6) [vdsm.api] START multipath_health() from=::1,33628, task_id=6f2364c5-df82-4104-aed4-e0b71d58ac95 (api:48)
2024-04-11 11:12:44,990+0200 INFO (jsonrpc/6) [vdsm.api] FINISH multipath_health return={} from=::1,33628, task_id=6f2364c5-df82-4104-aed4-e0b71d58ac95 (api:54)
11 months
Singe host installation
by Jakov Sosic
Hi guys,
I want to run oVirt on a single host, is that possible?
Running external engine is a possibility.
Thanks,
Jakov
11 months, 1 week
Can't create shared disk after upgrade to 4.5.4
by Igor Filipovic
Hi,
In my lab I'm using OLVM ( I know ... it's not original oVirt but it is based on it ). I was using 4.4.10, and after upgrade to 4.5.4 I'm no longer able to create shared disks, on any storage domain (nfs or fc). When I was on previous release I was able to create and use shared disks (required for cluster systems), but after upgrade when I try to create shared disk (preallocated) I'm getting an error saying : Error while executing action: Cannot add Virtual Disk. Disk's volume format is not supported for shareable disk.
In engine log I can see only this :
2024-03-31 20:04:28,997+02 WARN [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (default task-37) [b8bc1831-260a-4253-b6fe-e0c81a6f81b6] Validation of action 'AddDisk' failed for user admin@internal-authz. Reasons: VAR__ACTION__ADD,VAR__TYPE__DISK,SHAREABLE_DISK_IS_NOT_SUPPORTED_BY_VOLUME_FORMAT
Does anyone have idea what could be the reason or how to solve this?
Thanks in advance and best regards
Igor
11 months, 1 week
Re: Recreate engine without backup
by Igor Filipovic
Hi, I managed to resolve this issue.
Long story short :
- take configuration details from hosts about disks (especially if direct disks are being used, commands like multipath -ll, lsblk, blkid...)
- take a xml dump from all running VM's
- install new standalone engine
- add existing hosts to new image (running machines were detected with prefix "external"
- configure logical networks after checking hosts network adapters
- if direct disks exists in your configuration, create virtual disks for them by using xml dump files and info from hosts
- shutdown running vm's (all with prefix "external")
- shutdown all kvm hosts and left only one kvm host running (to prevent sanlock)
- import existing storage domain from running host
- use VM import tab in Storage domain to view vm's available for import
- delete machines with prefix "external" because import detect machines with same VM ID
- run import VM's from storage domain, one by one ... and that's it, after import run the machines
***If there are no VM's listed in VM import tab after adding storage domain, than you need to import disks from Disk import tab in storage domain, and then manually create virtual machines with information from xml dump files (to be able to configure disks,networks,cpu,ram ... like it was originally configured). After creating VM's you should be able to run them.***
Maybe it can help someone.
Best regards, Igor
11 months, 1 week
FQDN error while creating ovirt engine
by Marouane
Hi All,
I'm using a fresh 4.5.5 install, when I try to create the ovirt engine, I get the below error :
The task includes an option with an undefined variable. The error was: list object has no element 0. list object has no element 0\nThe error appears to be in '/usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/hosted_engine_setup/tasks/fetch_host_ip.yml': line 32, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n when: hostname_resolution_output.rc != 0\n - name: Parse host address resolution\n ^ here\n
I tried adding entries to my DNS or setting the local /etc/hosts as follows:
192.168.1.2 ovirt-engine-01 ovirt-engine-01.localdomain
in both cases I get the same error...Any ideas how to fix this?
11 months, 1 week
Abnormal behavior on the reinstalled host
by ricardoot＠gmail.com
Hello everyone,
I need help again with a host that I have reinstalled l and I don't understand why it doesn't work well.
I have managed to get the host status to UP. But I can't migrate MV to this host. Yes, I can start a VM on this host and I can migrate it to another but it can't go back if you want.
These are the errors I see:
ID: 128
2024-03-19 20:34:56,809+01 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-77603) [6ea57120-4e64-4b1d-a31f- 1072f69c478b] EVENT_ID: VM_MIGRATION_TRYING_RERUN(128), Failed to migrate VM srvlinux to Host ovirt51.mydomain due to an Error: Fatal error during migration. Trying to migrate to another Host.
It also happens that I cannot access the noVNC console on this Host but the VM is running. It gives me this error:
"Something went wrong, connection is closed"
If I put the host in maintenance mode it does not activate again and gives an error:
Operation Canceled
Error while executing action:
ovirt51.mydomain:
Cannot activate Host. Host has no unique id.
Resolved by reinstalling the host and checking to activate it. But it doesn't seem normal to me.
Please does anyone know how to resolve these issues?
Thank you.
OS Version:
RHEL - 8.9 - 1.8.el8
OS Description:
Rocky Linux 8.9 (Green Obsidian)
Kernel Version:
4.18.0 - 513.18.1.el8_9.x86_64
KVM Version:
6.2.0 - 40.module+el8.9.0+1654+f4df84c4.2
LIBVIRT Version:
libvirt-8.0.0-22.module+el8.9.0+1405+b6048078
VDSM Version:
vdsm-4.50.6-3.git5d82b9e88.el8
SPICE Version:
0.14.3 - 4.el8
GlusterFS Version:
glusterfs-10.5-1.el8s
CEPH Version:
librbd1-16.2.15-1.el8s
Open vSwitch Version:
openvswitch-2.15-4.el8
Nmstate Version:
nmstate-1.4.5-2.el8_9
Kernel Features:
MDS: (Not affected), L1TF: (Not affected), SRBDS: (Not affected), MELTDOWN: (Not affected), RETBLEED: (Not affected), SPECTRE_V1: (Mitigation: usercopy/swapgs barriers and __user pointer sanitization), SPECTRE_V2: (Mitigation: Enhanced / Automatic IBRS, IBPB: conditional, RSB filling, PBRSB-eIBRS: SW sequence), ITLB_MULTIHIT: (Not affected), MMIO_STALE_DATA: (Mitigation: Clear CPU buffers; SMT vulnerable), TSX_ASYNC_ABORT: (Not affected), SPEC_STORE_BYPASS: (Mitigation: Speculative Store Bypass disabled via prctl), GATHER_DATA_SAMPLING: (Mitigation: Microcode), SPEC_RSTACK_OVERFLOW: (Not affected)
VNC Encryption:
Enabled
OVN configured:
No
11 months, 1 week
Hosted engine installation oVirt 4.5 initialize lockspace volume
by lbayerlein＠lbayerlein.de
Hello,
hope, you are doing well. I try to install a new hosted engine on my environment to switch the storage this engine. I use following command to deploy the instance:
hosted-engine --deploy --4 --ansible-extra-vars=he_offline_deployment=true
Everything works fine, except the initialization of lockspace volume. Here I get:
```
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Remove temporary entry in /etc/hosts for the local VM]
[ INFO ] changed: [localhost]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Set the name for add_host]
[ INFO ] ok: [localhost]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch the value of HOST_KEY_CHECKING]
[ INFO ] ok: [localhost]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Get the username running the deploy]
[ INFO ] ok: [localhost]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Register the engine FQDN as a host]
[ INFO ] changed: [localhost]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Start ovirt-ha-broker service on the host]
[ INFO ] changed: [localhost]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Initialize lockspace volume]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Workaround for ovirt-ha-broker start failures]
[ INFO ] changed: [localhost]
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Initialize lockspace volume]
[ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 5, "changed": true, "cmd": ["hosted-engine", "--reinitialize-lock
space", "--force"], "delta": "0:00:00.269920", "end": "2024-03-20 14:43:00.366557", "msg": "non-zero return code", "rc"
: 1, "start": "2024-03-20 14:43:00.096637", "stderr": "Traceback (most recent call last):\n File \"/usr/lib64/python3.
6/runpy.py\", line 193, in _run_module_as_main\n \"__main__\", mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\",
line 85, in _run_code\n exec(code, run_globals)\n File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup
/reinitialize_lockspace.py\", line 30, in <module>\n ha_cli.reset_lockspace(force)\n File \"/usr/lib/python3.6/site
-packages/ovirt_hosted_engine_ha/client/client.py\", line 286, in reset_lockspace\n stats = broker.get_stats_from_st
orage()\n File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py\", line 148, in get_stats_f
rom_storage\n result = self._proxy.get_stats()\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1112, in __ca
ll__\n return self.__send(self.__name, args)\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1452, in __requ
est\n verbose=self.__verbose\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1154, in request\n return se
lf.single_request(host, handler, request_body, verbose)\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1166, i
n single_request\n http_conn = self.send_request(host, handler, request_body, verbose)\n File \"/usr/lib64/python3.
6/xmlrpc/client.py\", line 1279, in send_request\n self.send_content(connection, request_body)\n File \"/usr/lib64/
python3.6/xmlrpc/client.py\", line 1309, in send_content\n connection.endheaders(request_body)\n File \"/usr/lib64/
python3.6/http/client.py\", line 1268, in endheaders\n self._send_output(message_body, encode_chunked=encode_chunked
)\n File \"/usr/lib64/python3.6/http/client.py\", line 1044, in _send_output\n self.send(msg)\n File \"/usr/lib64/
python3.6/http/client.py\", line 982, in send\n self.connect()\n File \"/usr/lib/python3.6/site-packages/ovirt_host
ed_engine_ha/lib/unixrpc.py\", line 76, in connect\n self.sock.connect(base64.b16decode(self.host))\nFileNotFoundErr
or: [Errno 2] No such file or directory", "stderr_lines": ["Traceback (most recent call last):", " File \"/usr/lib64/p
ython3.6/runpy.py\", line 193, in _run_module_as_main", " \"__main__\", mod_spec)", " File \"/usr/lib64/python3.6/r
unpy.py\", line 85, in _run_code", " exec(code, run_globals)", " File \"/usr/lib/python3.6/site-packages/ovirt_host
ed_engine_setup/reinitialize_lockspace.py\", line 30, in <module>", " ha_cli.reset_lockspace(force)", " File \"/usr
/lib/python3.6/site-packages/ovirt_hosted_engine_ha/client/client.py\", line 286, in reset_lockspace", " stats = bro
ker.get_stats_from_storage()", " File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py\", l
ine 148, in get_stats_from_storage", " result = self._proxy.get_stats()", " File \"/usr/lib64/python3.6/xmlrpc/clie
nt.py\", line 1112, in __call__", " return self.__send(self.__name, args)", " File \"/usr/lib64/python3.6/xmlrpc/cl
ient.py\", line 1452, in __request", " verbose=self.__verbose", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", l
ine 1154, in request", " return self.single_request(host, handler, request_body, verbose)", " File \"/usr/lib64/pyt
hon3.6/xmlrpc/client.py\", line 1166, in single_request", " http_conn = self.send_request(host, handler, request_bod
y, verbose)", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1279, in send_request", " self.send_content(co
nnection, request_body)", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1309, in send_content", " connecti
on.endheaders(request_body)", " File \"/usr/lib64/python3.6/http/client.py\", line 1268, in endheaders", " self._se
nd_output(message_body, encode_chunked=encode_chunked)", " File \"/usr/lib64/python3.6/http/client.py\", line 1044, in
_send_output", " self.send(msg)", " File \"/usr/lib64/python3.6/http/client.py\", line 982, in send", " self.co
nnect()", " File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py\", line 76, in connect", "
self.sock.connect(base64.b16decode(self.host))", "FileNotFoundError: [Errno 2] No such file or directory"], "stdout":
"", "stdout_lines": []}
```
And I dont know, what is the reason. I zeroed the iscsi target, rebooted the hypervisor and updated all software packages.
```
[root@hypervisor01 ~]# rpm -qa|grep ovirt-host
ovirt-hosted-engine-setup-2.6.5-1.el8.noarch
ovirt-host-dependencies-4.5.0-3.el8.x86_64
ovirt-hosted-engine-ha-2.5.0-1.el8.noarch
ovirt-host-4.5.0-3.el8.x86_64
```
I tried the deployment without the ansible arguments. Unfortunately, the same error comes up. What can I do to solve this?
Thanks very much
balu
11 months, 2 weeks
Recreate engine without backup
by Igor Filipovic
Hi all,
I have a situation in which customer has deleted standalone engine completely (it was running on vmware), and all backups were saved on the same machine - so there is no backup available.
Can someone suggest how to preserve virtual machines which are still running on kvm hosts, and recreate standalone engine to be able to manage existing kvm hosts and machines? What could be scenario for this?
Thanks in advance and best regards!
Igor
11 months, 2 weeks
Server 2008R2 UEFI ISO Boot Problerm
by Clint Boggio
Greetings oVirt team
I'm using "Version 4.5.7-0.master.20240327074810.git14bca0502d.el8 on CentOS Stream 9"
I have occasion to restore a windows backup image for the purpose of data extraction. To accomplish this I need to boot off a Server 2008R2 ISO. The machine to be restored was UEFI based originally and so it is required to be UEFI for the restoration process.
I seem to be having a problem with booting off of the ISO image in the required UEFI-based system. In pursuit of a solution I've read on some of the other forums about this same problem on other KVM-based platforms.
Booting that same ISO after changing the VM to BIOS works flawlessly. In the articles and threads I've read there is mention of hyper-v services that load during the guest boot process that are causing the ISO boot to hang.
Is it possible to disable (or is it even necessary to disable) "hv_time,hv_relaxed,hv_spinlocks=0x2000" in the "Custom Properties" section of the VM config ?
If any of you have some insight into this problem and a possible solution I'd appreciate the feedback.
11 months, 2 weeks
