May 2022 - Users - oVirt List Archives

OVIRT Engine Restore from old Backup
by Abe E 11 May '22

11 May '22

Hey All, I really dont like spamming the forums but the issues I am having keep changing entirely. I am at the point where I can get much closer to rebuilding my engine although my hurtle currently is restoring. When I restore using my most recent backup it is problematic Currently I am restoring by: clean engine install engine-cleanup engine-backup --mode=restore --file=ovirt-engine-backup-20220504011012.backup --log=restore5.log engine-setup Comes up with alot of annoyances as the ID of the old hosted_storage and the new one are mismatched in the GUI. On top of that it is not allowing me to deploy the engine to the 2nd Gluster (3rd one keeps deleting its glusters on reboot since 4.3 Upgrade). Host 1 which runs the hosted engine is not showing as active or unlike, rather it is unresponsive, when I tried to enroll certificates it finally accepted but failed because it said its missing networks, once added it crashed the engine and ha agent wouldnt come up. Now that I think of it, I could have removed the extra networks from being required although it still doesnt benefit me here as I am unable to deploy or reinstall the 2nd Gluster node and the first one will not show its correct state as the host. Should I just be deploying the DB, or is the full file backup fine? My issue here is it takes long to redeploy so the trial an error of failing takes me hours as my backup is from 4.5 engine even though my nodes are on 4.4 as 4.5 keeps causing the catastrophic failure of my whole cluster.

2 6

Async release for ovirt-engine (4.5.0.7) is now available
by Lev Veyde 10 May '22

10 May '22

The oVirt Team has just released a new version of ovirt-engine package (4.5.0.7) that fixes a few important bugs: - Bug 2066084 <https://bugzilla.redhat.com/2066084> - vmconsole-proxy-user certificate expired - cannot access serial console - Bug 2079799 <https://bugzilla.redhat.com/2079799> - issue the internal Certificate Authority for 20 years - Bug 2079835 <https://bugzilla.redhat.com/2079835> - Separate validity length of Apache and internal certificates - Bug 2079890 <https://bugzilla.redhat.com/2079890> - renew certificates sooner before they expire - Bug 2079901 <https://bugzilla.redhat.com/2079901> - allow Enroll Certificate action when host is Non Responsive The update is already available on resources.ovirt.org and should land on oVirt mirrors within 24 hours. Thanks in advance, -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 1

FC Storage recover
by csabany＠freemail.hu 10 May '22

10 May '22

Hi, We have a 4 node cluster (4.4.10) and a standalone manager. We attached to this cluster 4 volumes on FC. Last week we lost the storage canisters (both), but recovered one of them from the storage disks. Unfortunatly only 3 storages came back from offline. I see the 4th FC ID, but it's on "Can't Extend" status. If i check the storages on nodes (lsblk), i can see only the ID not more on this 4th storage. Do you know any recovery tool or any idea for this situation? Thanks csabany

3 14

Re: Ovirt engine Isuue
by Jonathan Grégoire 09 May '22

09 May '22

Hi Strahil, Do you have any hints for me? Jonathan

2 5

OVA Export from Snapshot?
by Alan G 09 May '22

09 May '22

Hi, Is there a way to export an OVA from a snapshot? Or only from the "Active VM" image? Thanks, Alan

2 2

CLI Offline Engine Deployment failing to activate storage domain to gluster error 400
by Abe E 09 May '22

09 May '22

Due to some complications in my 4.4 to 4.5 upgrade that have taken down ovirt for a few days I must go back to 4.4 to salvage what I can and rethink how I will upgrade. Due to that im having to rebuild my engine although I took backups prior. My issue currently is with 4.4 the engine deployment has an error regarding mirrors so I was able to use the ansible script to deploy offline and I am at the activate storage domain portion where it fails I am setting it to glusterfs with the host:/path as gluster-1.d.com:/engine The host machine i am building the engine on has a mount: gluster-1.d.com:/engine 104804356 13827696 90976660 14% /rhev/data-center/mnt/glusterSD/gluster-1.d.com:_engine within the mapping : drwxr-xr-x. 2 vdsm kvm 6 Apr 5 22:15 gluster-1.d.com:_data drwxr-xr-x. 4 vdsm kvm 38 May 5 15:00 gluster-1.d.com:_engine I am able to get into the temp engines ssh and ping the domain although I cannot seem to find where my steps are incorrect. [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Activate storage domain] [ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400."} Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs)[nfs]: VDSM Logs are showing: 2022-05-05 15:30:32,719-0600 INFO (jsonrpc/6) [storage.StorageDomain] sdUUID=7f7cf7a4-f8fa-41a4-8e6a-275230655a8c (fileSD:535) 2022-05-05 15:30:32,734-0600 INFO (jsonrpc/6) [vdsm.api] FINISH getStorageDomainInfo error=Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) from=::ffff:192.168.222.116,53112, flow_id=661d6831, task_id=480acb07-0a53-4bad-aa30-4cd18ce9e5f2 (api:52) 2022-05-05 15:30:32,734-0600 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='480acb07-0a53-4bad-aa30-4cd18ce9e5f2') Unexpected error (task:877) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 884, in _run return fn(*args, **kargs) File "<decorator-gen-131>", line 2, in getStorageDomainInfo File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2717, in getStorageDomainInfo dom = self.validateSdUUID(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 313, in validateSdUUID sdDom.validate() File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 538, in validate raise se.StorageDomainAccessError(self.sdUUID) vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) 2022-05-05 15:30:32,734-0600 INFO (jsonrpc/6) [storage.TaskManager.Task] (Task='480acb07-0a53-4bad-aa30-4cd18ce9e5f2') aborting: Task is aborted: "value=Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) abortedcode=379" (task:1182) 2022-05-05 15:30:32,735-0600 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH getStorageDomainInfo error=Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) (dispatcher:83) 2022-05-05 15:30:32,735-0600 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call StorageDomain.getInfo failed (error 379) in 0.02 seconds (__init__:312) 2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [vdsm.api] START createStoragePool(spUUID='fd385045-8355-4ef8-9970-ade84698ad89', poolName='LabNet', masterDom='7f7cf7a4-f8fa-41a4-8e6a-275230655a8c', domList=['7f7cf7a4-f8fa-41a4-8e6a-275230655a8c'], masterVersion=13, lockRenewalIntervalSec=5, leaseTimeSec=60, ioOpTimeoutSec=10, leaseRetries=3) from=::ffff:192.168.222.116,53112, flow_id=661d6831, task_id=b58a0503-98c3-41a9-b5d7-3330e2985133 (api:48) 2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [storage.StoragePool] updating pool fd385045-8355-4ef8-9970-ade84698ad89 backend from type NoneType instance 0x7f767080b9d0 to type StoragePoolDiskBackend instance 0x7f760c4dd308 (sp:168) 2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [storage.StoragePool] spUUID=fd385045-8355-4ef8-9970-ade84698ad89 poolName=LabNet master_sd=7f7cf7a4-f8fa-41a4-8e6a-275230655a8c domList=['7f7cf7a4-f8fa-41a4-8e6a-275230655a8c'] masterVersion=13 {'LEASERETRIES': 3, 'LEASETIMESEC': 60, 'LOCKRENEWALINTERVALSEC': 5, 'IOOPTIMEOUTSEC': 10} (sp:624) 2022-05-05 15:30:32,743-0600 INFO (jsonrpc/0) [storage.StorageDomain] sdUUID=7f7cf7a4-f8fa-41a4-8e6a-275230655a8c (fileSD:535) 2022-05-05 15:30:32,759-0600 ERROR (jsonrpc/0) [storage.StoragePool] Unexpected error (sp:640) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/sp.py", line 635, in create domain.validate() File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 538, in validate raise se.StorageDomainAccessError(self.sdUUID) vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) 2022-05-05 15:30:32,759-0600 INFO (jsonrpc/0) [vdsm.api] FINISH createStoragePool error=Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) from=::ffff:192.168.222.116,53112, flow_id=661d6831, task_id=b58a0503-98c3-41a9-b5d7-3330e2985133 (api:52) 2022-05-05 15:30:32,759-0600 ERROR (jsonrpc/0) [storage.TaskManager.Task] (Task='b58a0503-98c3-41a9-b5d7-3330e2985133') Unexpected error (task:877) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/sp.py", line 635, in create domain.validate() File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 538, in validate raise se.StorageDomainAccessError(self.sdUUID) vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 884, in _run return fn(*args, **kargs) File "<decorator-gen-31>", line 2, in createStoragePool File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 1006, in createStoragePool leaseParams) File "/usr/lib/python3.6/site-packages/vdsm/storage/sp.py", line 641, in create raise se.StorageDomainAccessError(sdUUID) vdsm.storage.exception.StorageDomainAccessError: Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) 2022-05-05 15:30:32,759-0600 INFO (jsonrpc/0) [storage.TaskManager.Task] (Task='b58a0503-98c3-41a9-b5d7-3330e2985133') aborting: Task is aborted: "value=Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) abortedcode=379" (task:1182) 2022-05-05 15:30:32,759-0600 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH createStoragePool error=Domain is either partially accessible or entirely inaccessible: ('7f7cf7a4-f8fa-41a4-8e6a-275230655a8c',) (dispatcher:83) 2022-05-05 15:30:32,759-0600 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call StoragePool.create failed (error 379) in 0.02 seconds (__init__:312) 2022-05-05 15:30:34,709-0600 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=1b759c4f-ab60-4f3a-8ede-2606f916bf4a (api:48) 2022-05-05 15:30:34,710-0600 INFO (periodic/0) [vdsm.api] FINISH repoStats return={} from=internal, task_id=1b759c4f-ab60-4f3a-8ede-2606f916bf4a (api:54) 2022-05-05 15:30:36,324-0600 INFO (jsonrpc/7) [api.host] START getStats() from=::ffff:192.168.222.116,53112 (api:48) 2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=()) from=::ffff:192.168.222.116,53112, task_id=3b40703b-83dc-4aff-ae92-41158e5ffefc (api:48) 2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={} from=::ffff:192.168.222.116,53112, task_id=3b40703b-83dc-4aff-ae92-41158e5ffefc (api:54) 2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] START multipath_health() from=::ffff:192.168.222.116,53112, task_id=00953608-b566-4d3d-be62-f20ea77ff7d9 (api:48) 2022-05-05 15:30:36,344-0600 INFO (jsonrpc/7) [vdsm.api] FINISH multipath_health return={} from=::ffff:192.168.222.116,53112, task_id=00953608-b566-4d3d-be62-f20ea77ff7d9 (api:54) 2022-05-05 15:30:36,348-0600 INFO (jsonrpc/7) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::ffff:192.168.222.116,53112 (api:54) 2022-05-05 15:30:36,889-0600 INFO (vmrecovery) [vdsm.api] START getConnectedStoragePoolsList() from=internal, task_id=a5db87b7-5130-49da-a5c4-5622dbe9d74b (api:48) 2022-05-05 15:30:36,889-0600 INFO (vmrecovery) [vdsm.api] FINISH getConnectedStoragePoolsList return={'poollist': []} from=internal, task_id=a5db87b7-5130-49da-a5c4-5622dbe9d74b (api:54) 2022-05-05 15:30:36,889-0600 INFO (vmrecovery) [vds] recovery: waiting for storage pool to go up (clientIF:735) 2022-05-05 15:30:37,444-0600 INFO (jsonrpc/2) [api.host] START getAllVmStats() from=::1,33062 (api:48) 2022-05-05 15:30:37,446-0600 INFO (jsonrpc/2) [api.host] FINISH getAllVmStats return={'status': {'code': 0, 'message': 'Done'}, 'statsList': (suppressed)} from=::1,33062 (api:54) 2022-05-05 15:30:37,451-0600 INFO (jsonrpc/3) [api.host] START getAllVmIoTunePolicies() from=::1,33062 (api:48) 2022-05-05 15:30:37,452-0600 INFO (jsonrpc/3) [api.host] FINISH getAllVmIoTunePolicies return={'status': {'code': 0, 'message': 'Done'}, 'io_tune_policies_dict': {'da57699e-e76a-4125-81ce-50821bb09eef': {'policy': [], 'current_values': []}}} from=::1,33062 (api:54)

2 4

vnc certificate renew
by csabany＠freemail.hu 09 May '22

09 May '22

Hi, LAst month a renewed our hosts certificates by the "Enroll certificates" method. The "/etc/pki/vdsm/libvirt-vnc/server-cert.pem" certificate wasn't renewed on my nodes (other certificates were). How can i renew this certificate too? thanks csabany

6 8

Backing up VMs with qemu-img
by David White 08 May '22

08 May '22

I've recently been working with the qemu-img commands for some work that has nothing to do with oVirt or anything inside an oVirt environment. But learning and using these commands have given me an idea for automating backups. I believe that the following is true, but to confirm, would the qemu-img commands be available on the oVirt hosts to take VM snapshots and disk images? Sent with ProtonMail secure email.

3 3

OVIRT Engine Restore from old Backup
by Abe E 07 May '22

07 May '22

Hey All, I really dont like spamming the forums but the issues I am having keep changing entirely. I am at the point where I can get much closer to rebuilding my engine although my hurtle currently is restoring. When I restore using my most recent backup it is problematic Currently I am restoring by: clean engine install engine-cleanup engine-backup --mode=restore --file=ovirt-engine-backup-20220504011012.backup --log=restore5.log engine-setup Comes up with alot of annoyances as the ID of the old hosted_storage and the new one are mismatched in the GUI. On top of that it is not allowing me to deploy the engine to the 2nd Gluster (3rd one keeps deleting its glusters on reboot since 4.3 Upgrade). Host 1 which runs the hosted engine is not showing as active or unlike, rather it is unresponsive, when I tried to enroll certificates it finally accepted but failed because it said its missing networks, once added it crashed the engine and ha agent wouldnt come up. Now that I think of it, I could have removed the extra networks from being required although it still doesnt benefit me here as I am unable to deploy or reinstall the 2nd Gluster node and the first one will not show its correct state as the host. Should I just be deploying the DB, or is the full file backup fine? My issue here is it takes long to redeploy so the trial an error of failing takes me hours as my backup is from 4.5 engine even though my nodes are on 4.4 as 4.5 keeps causing the catastrophic failure of my whole cluster.

2 1

ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[XML error]". HTTP response code is 400.
by yp414＠163.com 07 May '22

07 May '22

[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Wait for the storage interface to be up] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Check local VM dir stat] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Enforce local VM dir existence] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch host facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch cluster ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch cluster facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch Datacenter facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch Datacenter ID] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch Datacenter name] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch cluster name] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fetch cluster version] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Enforce cluster major version] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Enforce cluster minor version] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Set storage_format] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Add NFS storage domain] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Add glusterfs storage domain] [ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[XML error]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[XML error]\". HTTP response code is 400."}

2 1