February 2022 - Users - oVirt List Archives

Upgrade CentOS 7 oVirt host to CentOS 8 Stream from backup file - Python error
by Matyi Szabolcs 14 Feb '22

14 Feb '22

Hi Support, The following is the problem. I am trying to upgrade a CentOS 7 oVirt host to CentOS 8 Stream with oVirt-engine backup. https://www.ovirt.org/documentation/upgrade_guide/#SHE_Upgrading_from_4-3 I perform the reset as follows: hosted-engine --deploy --restore-from-file=/path/to/backup When ansible-playbook is almost finished it gives the following error: [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Initialize lockspace volume] [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 5, "changed": true, "cmd": ["hosted-engine", "--reinitialize-lockspace", "--force"], "delta": "0:00:00.227640", "end": "2022-02-08 08:22:54.615 461", "msg": "non-zero return code", "rc": 1, "start": "2022-02-08 08:22:54.387821", "stderr": "Traceback (most recent call last):\n File \"/usr/lib64/python3.6/runpy.py\", line 193, in _run_modul e_as_main\n \"__main__\", mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_s etup/reinitialize_lockspace.py\", line 30, in <module>\n ha_cli.reset_lockspace(force)\n File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/client/client.py\", line 286, in reset_lo ckspace\n stats = broker.get_stats_from_storage()\n File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py\", line 148, in get_stats_from_storage\n result = self._p roxy.get_stats()\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1112, in __call__\n return self.__send(self.__name, args)\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1452, i n __request\n verbose=self.__verbose\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1154, in request\n return self.single_request(host, handler, request_body, verbose)\n File \"/usr /lib64/python3.6/xmlrpc/client.py\", line 1166, in single_request\n http_conn = self.send_request(host, handler, request_body, verbose)\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 12 79, in send_request\n self.send_content(connection, request_body)\n File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1309, in send_content\n connection.endheaders(request_body)\n File \ "/usr/lib64/python3.6/http/client.py\", line 1268, in endheaders\n self._send_output(message_body, encode_chunked=encode_chunked)\n File \"/usr/lib64/python3.6/http/client.py\", line 1044, in _ send_output\n self.send(msg)\n File \"/usr/lib64/python3.6/http/client.py\", line 982, in send\n self.connect()\n File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc. py\", line 76, in connect\n self.sock.connect(base64.b16decode(self.host))\nFileNotFoundError: [Errno 2] No such file or directory", "stderr_lines": ["Traceback (most recent call last):", " Fil e \"/usr/lib64/python3.6/runpy.py\", line 193, in _run_module_as_main", " \"__main__\", mod_spec)", " File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code", " exec(code, run_globals )", " File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/reinitialize_lockspace.py\", line 30, in <module>", " ha_cli.reset_lockspace(force)", " File \"/usr/lib/python3.6/site-p ackages/ovirt_hosted_engine_ha/client/client.py\", line 286, in reset_lockspace", " stats = broker.get_stats_from_storage()", " File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/li b/brokerlink.py\", line 148, in get_stats_from_storage", " result = self._proxy.get_stats()", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1112, in __call__", " return self.__send( self.__name, args)", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1452, in __request", " verbose=self.__verbose", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1154, in requ est", " return self.single_request(host, handler, request_body, verbose)", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1166, in single_request", " http_conn = self.send_request(ho st, handler, request_body, verbose)", " File \"/usr/lib64/python3.6/xmlrpc/client.py\", line 1279, in send_request", " self.send_content(connection, request_body)", " File \"/usr/lib64/python3 .6/xmlrpc/client.py\", line 1309, in send_content", " connection.endheaders(request_body)", " File \"/usr/lib64/python3.6/http/client.py\", line 1268, in endheaders", " self._send_output(mes sage_body, encode_chunked=encode_chunked)", " File \"/usr/lib64/python3.6/http/client.py\", line 1044, in _send_output", " self.send(msg)", " File \"/usr/lib64/python3.6/http/client.py\", line 982, in send", " self.connect()", " File \"/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py\", line 76, in connect", " self.sock.connect(base64.b16decode(self.host))", "FileNotFoundError: [Errno 2] No such file or directory"], "stdout": "", "stdout_lines": []} [ ERROR ] Failed to execute stage 'Closing up': Failed executing ansible-playbook [ INFO ] Stage: Clean up What is the problem? Thanks!

3 8

what are the steps to swap/upgrade/reboot my main storage domain with minimum downtime
by Pascal D 14 Feb '22

14 Feb '22

I need to upgrade my main domain storage (NFS) which hosts 100s of VMs and templates including the hosted engine. Can I put the data center in global maintenance, do a quick sync and swap of the domain storage and bring it back up without missing a beat?

3 3

oVirt installation in Ubuntu 20.04
by Kuralamudhan Ramakrishnan 11 Feb '22

11 Feb '22

Is there any documentation, we can refer to install oVirt in Ubuntu 20.04 ? Current documentations refer centos - https://www.ovirt.org/documentation/installing_ovirt_as_a_self-hosted_engin… . Is there any documentation for ubuntu 20.04.?

2 1

Deploy ovirt-csi in the kubernetes cluster
by ssarang520＠gmail.com 10 Feb '22

10 Feb '22

Hi, I want to deploy ovirt-csi in the kubernetes cluster. But the guide only has how to deploy to openshift. How can I deploy the ovirt-csi in the kubernetes cluster? Is there any way to do that?

5 5

Fibre Channel storage issue
by Timothy J. Wielgos 10 Feb '22

10 Feb '22

Thanks everyone for helping me through the last issue I had. Once I changed bonding mode to 2, everything worked. However, now I have another issue. I'm trying to install this using fibre channel storage. I set up a new lun from my storage array, assigned it to my host, and configured multipathing. All looks good. Then, when I ran the install script, I got the following error: [ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Physical device initialization failed. Please check that the device is empty and accessible by the host.]". HTTP response code is 400.[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Physical device initialization failed. Please check that the device is empty and accessible by the host.]\". HTTP response code is 400."} Google says that's because the lun is dirty - maybe reused. I tried doing the suggested - I used dd to write all zeroes to the lun. I used wipefs on the lun. I deleted the lun from the array and created a new one. No matter what I did with that lun, it's still showing up with this error. I then created a second LUN. I mapped it to the host, and attempted to use that lun for storage - and it worked! There must be a dirty config somewhere on this thing that I need to clean up from that first lun. Anybody know what I might have to clean up on this host to clear out that old config?

2 1

OVIRT-Node Engine Deployment Failure due to no URLs in mirrorlist
by aellahib＠gmail.com 09 Feb '22

09 Feb '22

I am running OVIRT-Node and although I updated my mirror list and the hosts are fine. It seems my engine deployment (deploying hyperconverged gluster) is trying to use a different set of mirrors during Engine VM Prep. SO this is the error I see, should I be running a newer RPM on these nodes or something, I am not aware if there is a newer one I should be using for this. [ INFO ] TASK [ovirt.ovirt.engine_setup : Install oVirt Engine package] [ ERROR ] fatal: [localhost -> 192.168.222.193]: FAILED! => {"changed": false, "msg": "Failed to download metadata for repo 'ovirt-4.4-centos-gluster8': Cannot prepare internal mirrorlist: No URLs in mirrorlist", "rc": 1, "results": []} [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Sync on engine machine] [ INFO ] changed: [localhost -> 192.168.222.193] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Set destination directory path] [ INFO ] ok: [localhost -> localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Create destination directory] [ INFO ] changed: [localhost -> localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Find the local appliance image] [ INFO ] ok: [localhost -> localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Set local_vm_disk_path] [ INFO ] ok: [localhost -> localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Give the vm time to flush dirty buffers] [ INFO ] ok: [localhost -> localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Copy engine logs] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "There was a failure deploying the engine on the local engine VM. The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"} It does let me click PrepareVM despite Deployment Failed -- but at that point /var/tmp is filled to more than 67% so it fails due to no space for engine install despite the good minimum requirements set lol

9 13

What replaces ISO domains how to re-use ISO files on multiple Data Centers?
by Brian Levinsen 08 Feb '22

08 Feb '22

Hello. I have been searching but not been able to find a solution. As far as I could find the Storage Domain Type ISO is deprecated. We have used this for hosting one central location for ISO images. Everywhere I found they say to use a Data Domain. But a Data Domain can not be attached to multiple Data Centers? So how do we in newer versions of ovirt share ISO files with multiple Data Centers? /Brian

1 0

Alternative Perspective - Re: oVirt alternatives
by David White 07 Feb '22

07 Feb '22

At the risk of sounding like a Red Hat or IBM fanboy, I have decided to give Red Hat the benefit of the doubt here, and to not make any decisions about switching off of oVirt until and unless an official announcement is made. In the meantime, I know that I need to move off of Gluster (and I made that decision before the Gluster announcement), and I would need storage with any other solution anyway, so that's where I'm going to focus my own efforts. In the meantime, while I realize that the optics of a company like IBM / Red Hat shutting a project like oVirt down looks bad to the FOSS community, I'm going to push back a little bit. We have had access to a FOSS application that obviously works for a lot of people. No company is required to provide their services for free, and likewise, I'm of the opinion that one needs to be willing to pay (or contribute in some way) for a quality product service. It reminds me of the mantra: "Fast, Cheap, Free - pick two". So here's an alternative perspective: What can the community contribute and do in order to keep the project going? Anyone could fork it, rebrand it, and run with it. I claim to be a software developer, and the uplink in my datacenter is only 100mbps right now (of course I can increase it when needed), so I doubt I could provide much value in terms of hosting or coding. But I do know security. I'm a Linux systems engineer with over 10 years of experience. I know website content management systems. And people have told me that I'm good at documentation. So I think I have a lot of skill sets that I could "offer" (albeit I don't have much time, and as we all know, time is money. I've been dealing with a serious personal matter since beginning of December, and I'm effectively an acting single parent at the moment). I'll end this the way I started: I'm going to wait to see what happens before I personally make any decisions to change my entire underlying virtualization infrastructure. In the meantime, I'll continue to work on what I can control - the underlying storage. And if oVirt does shutdown in the future, I'd love to have a conversation with anyone interested in helping out to fork the project and keep it running. Sent with ProtonMail Secure Email.

5 8

Installing oVirt on RHEL vs CentOS Stream, Alma or Rocky Linux
by Richard W.M. Jones 07 Feb '22

07 Feb '22

I'm still struggling to get oVirt installed. It absolutely will not install on RHEL 8, the documentation here is flat out wrong: https://www.ovirt.org/documentation/installing_ovirt_as_a_standalone_manage… javapackages-tools is not a module that exists in RHEL 8. Apparently it's a CentOS PowerTools module. Since original CentOS no longer really exists either, does anyone have any opinions or experience with installing oVirt on any of: - CentOS Stream 8 - Alma Linux 8.5 - Rocky Linux Alma in particular looks like a straight replacement for old CentOS. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top

7 9

Importing export storage domain after redeploy failing, stating export domain still connected.
by Gilboa Davara 07 Feb '22

07 Feb '22

Hello all, I'm rebuilding one of my gluster clusters after it blew up following an unfortunate expired certificate issue. After I finally remembered to downgrade qemu (grrrr...) and started importing the hold gluster storage domains, one of the export domains failed to import due to "connected to another domain" issue. How can I force detach it from the previous cluster? (I remember something about deleting the lease file - but I'm too brain dead to find it in DDG...). Engine errors: $ cat engine.log | grep c3abcfe6-1062-48e2-8ca4-924b96b8c497 2022-02-07 13:31:24,729+02 INFO [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] Lock Acquired to object 'EngineLock:{exclusiveLocks='[22eec6d4-f0be-47db-b5d5-678bd84f47c6=STORAGE]', sharedLocks=''}' 2022-02-07 13:31:24,747+02 INFO [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: 22eec6d4-f0be-47db-b5d5-678bd84f47c6 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN, ID: 257a27cc-87ec-11ec-bc62-00163e3fe79d Type: StoragePoolAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN 2022-02-07 13:31:24,782+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] START, AttachStorageDomainVDSCommand( AttachStorageDomainVDSCommandParameters:{storagePoolId='257a27cc-87ec-11ec-bc62-00163e3fe79d', ignoreFailoverLimit='false', storageDomainId='22eec6d4-f0be-47db-b5d5-678bd84f47c6'}), log id: 777adb39 2022-02-07 13:31:48,281+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command AttachStorageDomainVDS failed: Storage domain already attached to pool: 'domain=22eec6d4-f0be-47db-b5d5-678bd84f47c6, pool=2cb812a0-4a95-11eb-b3bc-00163e6a0a7c' 2022-02-07 13:31:48,281+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] Command 'AttachStorageDomainVDSCommand( AttachStorageDomainVDSCommandParameters:{storagePoolId='257a27cc-87ec-11ec-bc62-00163e3fe79d', ignoreFailoverLimit='false', storageDomainId='22eec6d4-f0be-47db-b5d5-678bd84f47c6'})' execution failed: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=22eec6d4-f0be-47db-b5d5-678bd84f47c6, pool=2cb812a0-4a95-11eb-b3bc-00163e6a0a7c' 2022-02-07 13:31:48,281+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] FINISH, AttachStorageDomainVDSCommand, return: , log id: 777adb39 2022-02-07 13:31:48,281+02 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] Command 'org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=22eec6d4-f0be-47db-b5d5-678bd84f47c6, pool=2cb812a0-4a95-11eb-b3bc-00163e6a0a7c' (Failed with error StorageDomainAlreadyAttached and code 380) 2022-02-07 13:31:48,283+02 INFO [org.ovirt.engine.core.bll.CommandCompensator] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] Command [id=ec43c2c1-3441-4d91-9bd2-98280e9c0eaa]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: StoragePoolIsoMapId:{storagePoolId='257a27cc-87ec-11ec-bc62-00163e3fe79d', storageId='22eec6d4-f0be-47db-b5d5-678bd84f47c6'}. 2022-02-07 13:31:48,290+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] EVENT_ID: USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage Domain local_export_storage_1 to Data Center Default. (User: gilboa@internal-authz) 2022-02-07 13:31:48,294+02 INFO [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-27) [c3abcfe6-1062-48e2-8ca4-924b96b8c497] Lock freed to object 'EngineLock:{exclusiveLocks='[22eec6d4-f0be-47db-b5d5-678bd84f47c6=STORAGE]', sharedLocks=''}' Thanks, Gilboa

1 0