- Users - oVirt List Archives

How to re-enroll (or renew) host certificates for a single-host hosted-engine deployment?
by Derek Atkins 28 Jun '24

28 Jun '24

Hi, I've got a single-host hosted-engine deployment that I originally installed with 4.0 and have upgraded over the years to 4.3.10. I and some of my users have upgraded remote-viewer and now I get an error when I try to view the console of my VMs: (remote-viewer:8252): Spice-WARNING **: 11:30:41.806: ../subprojects/spice-common/common/ssl_verify.c:477:openssl_verify: Error in server certificate verification: CA signature digest algorithm too weak (num=68:depth0:/O=<My Org Name>/CN=<Host's Name>) I am 99.99% sure this is because the old certs use SHA1. I reran engine-setup on the engine and it asked me if I wanted to renew the PKI, and I answered yes. This replaced many[1] of the certificates in /etc/pki/ovirt-engine/certs on the engine, but it did not update the Host's certificate. All the documentation I've seen says that to refresh this certificate I need to put the host into maintenance mode and then re-enroll.. However I cannot do that, because this is a single-host system so I cannot put the host in local mode -- there is no place to migrate the VMs (let alone the Engine VM). So.... Is there a command-line way to re-enroll manually and update the host certs? Or some other way to get all the leftover certs renewed? Thanks, -derek [1] Not only did it not update the Host's cert, it did not update any of the vmconsole-proxy certs, nor the certs in /etc/pki/ovirt-vmconsole/, and obviously nothing in /etc/pki/ on the host itself. -- Derek Atkins 617-623-3745 derek(a)ihtfp.com www.ihtfp.com Computer and Internet Security Consultant

8 16

python-nmstate 2.2.27 and ovirt network setup
by Bill James 28 Jun '24

28 Jun '24

We have a ovirt-host-4.5.0-3.el9.x86_64 cluster. I just recently ran yum update and it upgraded python-nmstate from python3-libnmstate-2.0.0-2.el9.noarch to python3-libnmstate- 2.2.27-2.el9_4.x86_64. This broken ovirt's ability to update the network config: 2024-06-27 13:22:39,716-07 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (EE-ManagedThreadFa ctory-engine-Thread-24) [ea3bc347-156d-4631-a196-c93a9f79c6c1] Command 'HostSetupNetworksVDSCommand(HostName = ovirt-b2n.j2n oc.com, HostSetupNetworksVdsCommandParameters:{hostId='12cd7cbc-0f02-4474-aaa5-b29cde8b56a4', vds='Host[ovirt-b2n.j2noc.com, 12cd7cbc-0f02-4474-aaa5-b29cde8b56a4]', rollbackOnFailure='true', commitOnSuccess='true', connectivityTimeout='120', network s='[HostNetwork:{defaultRoute='true', bonding='false', networkName='ovirtmgmt', vdsmName='ovirtmgmt', nicName='eno12429', vl an='110', vmNetwork='false', stp='false', properties='[]', ipv4BootProtocol='DHCP', ipv4Address='null', ipv4Netmask='null', ipv4Gateway='null', ipv6BootProtocol='NONE', ipv6Address='null', ipv6Prefix='null', ipv6Gateway='null', nameServers='null'}] ', removedNetworks='[]', bonds='[]', removedBonds='[]', clusterSwitchType='LEGACY', managementNetworkChanged='true'})' execu tion failed: VDSGenericException: VDSErrorException: Failed to HostSetupNetworksVDS, error = Internal JSON-RPC error: {'reas on': "[Errno 2] No such file or directory: ' */usr/lib/python3.9/site-packages/libnmstate/VERSION*'"}, code = -32603 that VERSION file, along with many other files, are not in 2.2.27 version of python-nmstate rpm. My fix was to downgrade the rpm. dnf install python3-libnmstate-2.0.0-2.el9.noarch I'm sure that's not the best solution as I can no longer run dnf update. Ideas?

1 0

Re: 4.5.4 Hosted-Engine: change hosted-engine storage
by Novak, Sam 27 Jun '24

27 Jun '24

I'm in the midst of deploying a new EL9 hosted-engine for an existing cluster and it has been an absolute nightmare, and I haven't found any documentation or guidance to account for this type of scenario, which is odd, considering that the EOL for Cent 8 was announced a while ago. I know this is all best effort type work, and I'm very appreciative of that effort, but it's really making me re-think my use case unfortunately. * Sam

1 0

Re: Failed to read or parse '/etc/pki/ovirt-engine/keys/engine.p12'
by Novak, Sam 27 Jun '24

27 Jun '24

Yeah I has to convert all of my P12's to use AES cd /etc/pki/ovirt-engine/keys/ for E in $(ls *.p12); do openssl pkcs12 -in $E -out $E.key -nocerts -nodes -passin file:pass.txt openssl pkcs12 -in $E -out $E.crt -nokeys -nodes -passin file:pass.txt openssl pkcs12 -in ./$E.crt -inkey ./$E.key -export -out $E.test.p12 -keypbe aes-256-cbc -certpbe aes-256-cbc -passin file:pass.txt done You'll also need to make sure you set the file owners and permissions like the original files.

1 0

Re: [ovirt-announce] Re: Reminder: CentOS Stream 8 is now EOL
by Sandro Bonazzola 27 Jun '24

27 Jun '24

Sadly I have no procedure ready for it and no time for working on it. My best advice would be to migrate from a hosted engine on el8 to a stand alone engine on el9 by installing a fresh centos stream 9 engine and re-importing the backup there. Il giorno gio 27 giu 2024 alle ore 12:01 <trockey(a)uw.edu> ha scritto: > Hi Sanbro, > I have 3 production oVirt clusters running and I missed upgrading HE > before May 31, 2024. If I am reading everything that you stated correctly I > will need to point HE CentOS Stream release 8 to vault.centos.org and > then upgrade to CentOS Stream 9. > Do you have the procedures for taking a HE CentOS Stream release 8 pointed > at vault.contos.org and upgrading it to CentOS Stream 9? > _______________________________________________ > Announce mailing list -- announce(a)ovirt.org > To unsubscribe send an email to announce-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/announce@ovirt.org/message/M5HSQFLZLF… > -- Sandro Bonazzola

1 0

Problem importing storage domain from oVirt 4.3.10 to 4.5.5
by Sumit Basu 26 Jun '24

26 Jun '24

Hi, I want to import a FC storage domain from an oVirt 4.3.10 setup to a fresh installed oVirt 4.5.5. I could move domain to maintenance, detach, remove from 4.3.10 and import with activate option to 4.5.5 for one FC LUN fine. When i tried for the second FC lun - during import,activate i get "Error while executing action Attach Storage Domain: Internal Engine Error" and in engine.log i get " Query 'GetUnregisteredDisksQuery' failed: null', and on the standalone engine, i get :- 2024-06-26 07:15:07,277+05 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-102) [] An error occurred while fetching unregistered disks from Storage Domain id 'a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429' 2024-06-26 07:15:07,277+05 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-102) [] Command 'org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand' failed: Failed to retrieve unregistered disks 2024-06-26 07:15:07,277+05 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-102) [] Exception: java.lang.RuntimeException: Failed to retrieve unregistered disks I have checked using "Scan Disk" on the FC lun before exporting from 4.3.10 and there were no floating disks. I have checked if any of the VM's on the domain have any of their disks on any other storage domain - there are none. On the host used for importing the domain, i get :- 2024-06-26 07:15:04,829+0530 INFO (jsonrpc/5) [storage.hba] Scanning FC devices (hba:42) 2024-06-26 07:15:04,938+0530 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=9f03d412-56a7-4f3f-85b6-c3fd72a68817 (api:31) 2024-06-26 07:15:04,938+0530 INFO (periodic/0) [vdsm.api] FINISH repoStats return={'7c9e748c-442b-48d0-9b35-a769d349c73b': {'code': 0, 'lastCheck': '3.7', 'delay': '0.000974057', 'valid': True, 'version': 0, 'acquired': True, 'actual': True}, 'a589fe14-f40a-4d67-afcd-ec74b07009cf': {'code': 0, 'lastCheck': '3.7', 'delay': '0.000501601', 'valid': True, 'version': 0, 'acquired': True, 'actual': True}, '6b625ecb-1cd1-43c5-b533-32000cd050ec': {'code': 0, 'lastCheck': '3.7', 'delay': '0.000570869', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=internal, task_id=9f03d412-56a7-4f3f-85b6-c3fd72a68817 (api:37) 2024-06-26 07:15:04,968+0530 INFO (jsonrpc/5) [storage.hba] Scanning FC devices: 0.14 seconds (utils:373) 2024-06-26 07:15:04,968+0530 INFO (jsonrpc/5) [storage.multipath] Waiting until multipathd is ready (multipath:95) 2024-06-26 07:15:07,003+0530 INFO (jsonrpc/5) [storage.multipath] Waited 2.04 seconds for multipathd (tries=2, ready=2) (multipath:120) 2024-06-26 07:15:07,003+0530 INFO (jsonrpc/5) [storage.multipath] Resizing multipath devices (multipath:223) 2024-06-26 07:15:07,014+0530 INFO (jsonrpc/5) [storage.multipath] Resizing multipath devices: 0.01 seconds (utils:373) 2024-06-26 07:15:07,014+0530 INFO (jsonrpc/5) [storage.storagedomaincache] Refreshing storage domain cache: 2.60 seconds (utils:373) 2024-06-26 07:15:07,263+0530 INFO (jsonrpc/5) [storage.volumemanifest] Creating image directory '/rhev/data-center/mnt/blockSD/a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429/images/f054c394-2b49-4087-9e9d-0490b23e34e4' (blockVolume:106) 2024-06-26 07:15:07,263+0530 ERROR (jsonrpc/5) [storage.volumemanifest] Unexpected error (blockVolume:110) Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/vdsm/storage/blockVolume.py", line 108, in validateImagePath os.mkdir(imageDir, 0o755) FileNotFoundError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/blockSD/a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429/images/f054c394-2b49-4087-9e9d-0490b23e34e4' 2024-06-26 07:15:07,263+0530 INFO (jsonrpc/5) [vdsm.api] FINISH prepareImage error=Image path does not exist or cannot be accessed/created: ('/rhev/data-center/mnt/blockSD/a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429/images/f054c394-2b49-4087-9e9d-0490b23e34e4',) from=::ffff:10.10.99.100,51116, flow_id=872d7930-e487-4fe4-8e0a-599074105d7b, task_id=523690f8-24b3-4f1f-801e-8391ebb087a2 (api:35) 2024-06-26 07:15:07,263+0530 ERROR (jsonrpc/5) [storage.taskmanager.task] (Task='523690f8-24b3-4f1f-801e-8391ebb087a2') Unexpected error (task:860) Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/vdsm/storage/blockVolume.py", line 108, in validateImagePath os.mkdir(imageDir, 0o755) FileNotFoundError: [Errno 2] No such file or directory: '/rhev/data-center/mnt/blockSD/a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429/images/f054c394-2b49-4087-9e9d-0490b23e34e4' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/vdsm/storage/task.py", line 867, in _run return fn(*args, **kargs) File "<decorator-gen-169>", line 2, in prepareImage File "/usr/lib/python3.9/site-packages/vdsm/common/api.py", line 33, in method ret = func(*args, **kwargs) File "/usr/lib/python3.9/site-packages/vdsm/storage/hsm.py", line 2958, in prepareImage legality = dom.produceVolume(imgUUID, volUUID).getLegality() File "/usr/lib/python3.9/site-packages/vdsm/storage/sd.py", line 1188, in produceVolume return self.getVolumeClass()(self.mountpoint, self.sdUUID, imgUUID, File "/usr/lib/python3.9/site-packages/vdsm/storage/volume.py", line 881, in __init__ self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID) File "/usr/lib/python3.9/site-packages/vdsm/storage/blockVolume.py", line 40, in __init__ volume.VolumeManifest.__init__(self, repoPath, sdUUID, imgUUID, File "/usr/lib/python3.9/site-packages/vdsm/storage/volume.py", line 67, in __init__ self.validate() File "/usr/lib/python3.9/site-packages/vdsm/storage/blockVolume.py", line 135, in validate volume.VolumeManifest.validate(self) File "/usr/lib/python3.9/site-packages/vdsm/storage/volume.py", line 99, in validate self.validateImagePath() File "/usr/lib/python3.9/site-packages/vdsm/storage/blockVolume.py", line 111, in validateImagePath raise se.ImagePathError(imageDir) vdsm.storage.exception.ImagePathError: Image path does not exist or cannot be accessed/created: ('/rhev/data-center/mnt/blockSD/a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429/images/f054c394-2b49-4087-9e9d-0490b23e34e4',) 2024-06-26 07:15:07,263+0530 INFO (jsonrpc/5) [storage.taskmanager.task] (Task='523690f8-24b3-4f1f-801e-8391ebb087a2') aborting: Task is aborted: "value=Image path does not exist or cannot be accessed/created: ('/rhev/data-center/mnt/blockSD/a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429/images/f054c394-2b49-4087-9e9d-0490b23e34e4',) abortedcode=254" (task:1165) 2024-06-26 07:15:07,264+0530 ERROR (jsonrpc/5) [storage.dispatcher] FINISH prepareImage error=Image path does not exist or cannot be accessed/created: ('/rhev/data-center/mnt/blockSD/a5a72b44-f9bc-4e5b-ab43-ca2b1d97b429/images/f054c394-2b49-4087-9e9d-0490b23e34e4',) (dispatcher:66) 2024-06-26 07:15:07,264+0530 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Image.prepare failed (error 254) in 2.85 seconds (__init__:300) Any help will be useful, we are moving from 4.3.10 on IBM servers and storage to 4.5.5 on DELL servers and storage Regards Sumit Basu

1 0

4.5.4 with Ceph only storage
by Maurice Burrows 25 Jun '24

25 Jun '24

Hey ... A long story short ... I have an existing Red Hat Virt / Gluster hyperconverged solution that I am moving away from. I have an existing Ceph cluster that I primarily use for OpenStack and a small requirement for S3 via RGW. I'm planning to build a new oVirt 4.5.4 cluster on RHEL9 using Ceph for all storage requirements. I've read many online articles on oVirt and Ceph, and they all seem to use the Ceph iSCSI gateway, which is now in maintenance, so I'm not real keen to commit to iSCSI. So my question is, IS there any reason I cannot use CephFS for both hosted-engine and as a data storage domain? I'm currently running Ceph Pacific FWIW. Cheers

5 10

After restoring the failed host and synchronizing the data, it prompts that there are unsynchronized items
by ziyi Liu 25 Jun '24

25 Jun '24

gluster volume info Volume Name: data Type: Replicate Number of Bricks: 1 x (2 + 1) = 3 Bricks: Brick1: node-gfs1:/gluster_bricks/data/data1 Brick2: node-gfs2:/gluster_bricks/data/data1 Brick3: node-gfs3:/gluster_bricks/data/data1 (arbiter) gluster volume heal data info Number of entries: 39 /var/log/glusterfs/glustershd.log The log appears client-rpc-fops_v2.c:785:client4_0_fsync_cbk] 0-data-client-0: remote operation failed. [{errno=5}, {error=Input/output error}] How should I solve these unrepaired entries

2 2

Error importing OVA on Ovirt-4.5.4 - no value for $TERM is defined
by raulmevi2＠hotmail.com 21 Jun '24

21 Jun '24

Hello, I'm facing an issue where, in a new fresh installed Oracle OLVM - Ovirt 4.5.4, self-hosted engine with NFS shared storage. That an .OVA file cannot be imported. Steps to reproduce: - Created test virtual machine in an NFS storage domain, installed it, allright working fine. - Shutdown test VM - Export test VM to a purposedly created directory /home/vdsm, whose owner is vdsm:kvm with full permissions. - Export .OVA file concludes successfully and the file is created at that location Then, in the exact same server, same datacenter. - Rename source VM so it doesn't conflict because the same name (same result if I delete the source VM) - try to import the .OVA. using the GUI. It finds successfully the file at the specified location and begins the procedure -# up to a given point, it fails without more indications at the web GUI. No more info at the "events" than "Failed to import vm Test to Datacenter". Fiddling into the logs, I was able to find in the hosted-engine VM, at /var/log/ovirt-engine/ova the log that goes by "ovirt-ova-external-data-ansible-DATE-HostFQDN-uuid.log" an error that specifiies stdout_lines" : [ "tput: No value for $TERM and no -T specified", "tput: No value for $TERM and no -T specified", "tput: No value for $TERM and no -T specified" I tinkered with the ansible role that calls a python script called "get_ova_data.py" (located at /usr/share/ovirt-engine/ansible-runner-service-project/project/roles-/ovirt-ova-external-data/) I tried to define the variable TERM into null or into xterm but then it fails in another manner complaining about that variable. By surfing the webs, it seems something calls "tput" like if it was in console mode but then it breaks. Can I have some help with this, please? Thanks

1 0

HE deployment on FC (fibre-channel) disk fails at 99% completed at final "hosted-engine --reinitialize-lockspace --force"
by Jeffrey Slapp 21 Jun '24

21 Jun '24

This identical to many others who have encountered this issue, yet nothing definitive has been suggested. The entire HE deployment nearly finishes, but after the copy of the HE VM to the shared disk, shortly after that I reach the "initialize lockspace" section and the following error occurs: 20083 2024-06-19 13:24:01,777-0400 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 {'changed': True, 'stdout': '', 'stderr': 'Traceback (most recent call last):\n File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main\n return _run_ code(code, main_globals, None,\n File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code\n exec(code, run_globals)\n File "/usr/lib/p ython3.9/site-packages/ovirt_hosted_engine_setup/reinitialize_lockspace.py", line 30, in <module>\n ha_cli.reset_lockspace(force)\n File " /usr/lib/python3.9/site-packages/ovirt_hosted_engine_ha/client/client.py", line 286, in reset_lockspace\n stats = broker.get_stats_from_sto rage()\n File "/usr/lib/python3.9/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 148, in get_stats_from_storage\n result = self._proxy.get_stats()\n File "/usr/lib64/python3.9/xmlrpc/client.py", line 1122, in __call__\n ret urn self.__send(self.__name, args)\n File "/usr/lib64/python3.9/xmlrpc/client.py", line 1464, in __request\n response = self.__transport.request(\n File "/usr/lib64/python3.9/ xmlrpc/client.py", line 1166, in request\n return self.single_request(host, handler, request_body, verbose)\n File "/usr/lib64/python3.9/x mlrpc/client.py", line 1178, in single_request\n http_conn = self.send_request(host, handler, request_body, verbose)\n File "/usr/lib64/py thon3.9/xmlrpc/client.py", line 1291, in send_request\n self.send_content(connection, request_body)\n File "/usr/lib64/python3.9/xmlrpc/cl ient.py", line 1321, in send_content\n connection.endheaders(request_body)\n File "/usr/lib64/python3.9/http/client.py", line 1280, in end headers\n self._send_output(message_body, encode_chunked=encode_chunked)\n File "/usr/lib64/python3.9/http/client.py", line 1040, in _send _output\n self.send(msg)\n File "/usr/lib64/python3.9/http/cl ient.py", line 980, in send\n self.connect()\n File "/usr/lib/python3.9/s ite-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 76, in connect\n self.sock.connect(base64.b16decode(self.host))\nFileNotFoundErro r: [Errno 2] No such file or directory', 'rc': 1, 'cmd': ['hosted-engine', '--reinitialize-lockspace', '--force'], 'start': '2024-06-19 13:24: 01.438386', 'end': '2024-06-19 13:24:01.618227', 'delta': '0:00:00.179841', 'msg': 'non-zero return code', 'invocation': {'module_args': {'_ra w_params': 'hosted-engine --reinitialize-lockspace --force', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv' : None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Traceback (most recent call last):', ' File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main', ' return _run_code(code, main_global s, None,', ' File "/usr/l ib64/python3.9/runpy.py", line 87, in _run_code', ' exec(code, run_globals)', ' File "/usr/lib/python3.9/site-pa ckages/ovirt_hosted_engine_setup/reinitialize_lockspace.py", line 30, in <module>', ' ha_cli.reset_lockspace(force)', ' File "/usr/lib/pyt hon3.9/site-packages/ovirt_hosted_engine_ha/client/client.py", line 286, in reset_lockspace', ' stats = broker.get_stats_from_storage()', ' File "/usr/lib/python3.9/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 148, in get_stats_from_storage', ' result = self._ proxy.get_stats()', ' File "/usr/lib64/python3.9/xmlrpc/client.py", line 1122, in __call__', ' return self.__send(self.__name, args)', ' File "/usr/lib64/python3.9/xmlrpc/client.py", line 1464, in __request', ' response = self.__transport.request(', ' File "/usr/lib64/python 3.9/xmlrpc/client.py", line 1166, in request', ' return self.single_request(host, handler, request_body, verbose)', ' File "/usr/li b64/pyt hon3.9/xmlrpc/client.py", line 1178, in single_request', ' http_conn = self.send_request(host, handler, request_body, verbose)', ' File "/ usr/lib64/python3.9/xmlrpc/client.py", line 1291, in send_request', ' self.send_content(connection, request_body)', ' File "/usr/lib64/pyt hon3.9/xmlrpc/client.py", line 1321, in send_content', ' connection.endheaders(request_body)', ' File "/usr/lib64/python3.9/http/client.py ", line 1280, in endheaders', ' self._send_output(message_body, encode_chunked=encode_chunked)', ' File "/usr/lib64/python3.9/http/client. py", line 1040, in _send_output', ' self.send(msg)', ' File "/usr/lib64/python3.9/http/client.py", line 980, in send', ' self.connect() ', ' File "/usr/lib/python3.9/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 76, in connect', ' self.sock.connect(base64.b16de code(self.host))', 'FileNotFoundError: [Errno 2] No such file or directory'], '_ansible_no_log': False, 'attempts': 5} No errors in /var/log/messages or /var/log/sanlock.log. I have this working with iSCSI on another storage system, but can't seem to get this to work on FC. I have read that the sector sizes could possible cause this. On my iSCSI system I have [PHY-SEC:LOG-SEC] as [512:512] but on my FC system I have [4096:512]. Hoping that someone can confirm whether this is the issue or not. Interestingly all previous "initialize lockspace" phases of the install are fine, it just appears to be this final one.

2 1