Hosted-engine restore failing
by Devin A. Bougie
Hi, All. We are attempting to migrate to a new storage domain for our oVirt 4.5.4 self-hosted engine setup, and are failing with "cannot import name 'Callable' from 'collections'"
Please see below for the errors on the console.
Many thanks,
Devin
------
hosted-engine --deploy --restore-from-file=backup.bck --4
...
[ INFO ] Checking available network interfaces:
[ ERROR ] b'[WARNING]: Skipping plugin (/usr/share/ovirt-hosted-engine-\n'
[ ERROR ] b'setup/he_ansible/callback_plugins/2_ovirt_logger.py), cannot load: cannot\n'
[ ERROR ] b"import name 'Callable' from 'collections'\n"
[ ERROR ] b'(/usr/lib64/python3.11/collections/__init__.py)\n'
[ ERROR ] b"ERROR! Unexpected Exception, this is probably a bug: cannot import name 'Callable' from 'collections' (/usr/lib64/python3.11/collections/__init__.py)\n"
[ ERROR ] Failed to execute stage 'Environment customization': Failed executing ansible-playbook
[ INFO ] Stage: Clean up
[ INFO ] Cleaning temporary resources
[ ERROR ] b'[WARNING]: Skipping plugin (/usr/share/ovirt-hosted-engine-\n'
[ ERROR ] b'setup/he_ansible/callback_plugins/2_ovirt_logger.py), cannot load: cannot\n'
[ ERROR ] b"import name 'Callable' from 'collections'\n"
[ ERROR ] b'(/usr/lib64/python3.11/collections/__init__.py)\n'
[ ERROR ] b"ERROR! Unexpected Exception, this is probably a bug: cannot import name 'Callable' from 'collections' (/usr/lib64/python3.11/collections/__init__.py)\n"
[ ERROR ] Failed to execute stage 'Clean up': Failed executing ansible-playbook
[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20231011110358.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Hosted Engine deployment failed
Log file is located at
/var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20231011110352-raupj9.log
8 months, 1 week
hosted-engine --deploy fails with "The task includes an option with an undefined variable. The error was: 'local_vm_ip' is undefined"
by drew.tonnesen＠emc.com
As basic as can be install/deploy. Used ovirt-node-ng-installer-latest-el8.iso. Followed steps exactly. Took all defaults possible. After step 23 (Specify the hostname of the deployment host:) the deployment starts. It goes for quite a long time until it fails here, never getting to step 25 and asking me for storage:
2023-11-19 22:19:45,568+0000 DEBUG var changed: host "localhost" var "module_setup" type "<class 'bool'>" value: "true"
1888 2023-11-19 22:19:45,568+0000 DEBUG var changed: host "localhost" var "discovered_interpreter_python" type "<class 'ansible.utils.unsafe_proxy.AnsibleUnsafeText'>" value: ""/usr/libexec/platform-python""
1889 2023-11-19 22:19:45,568+0000 INFO ansible ok {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/he_ansible/trigger_role.yml', 'ansible_host': 'localhost', 'ansible_task': 'For ce facts gathering', 'task_duration': 1}
1890 2023-11-19 22:19:45,568+0000 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f2028102220> kwargs
1891 2023-11-19 22:19:45,664+0000 INFO ansible task start {'status': 'OK', 'ansible_type': 'task', 'ansible_playbook': '/usr/share/ovirt-hosted-engine-setup/he_ansible/trigger_role.yml', 'ansible_task': 'ovirt.ovirt.hosted_engin e_setup : Set the name for add_host'}
1892 2023-11-19 22:19:45,664+0000 DEBUG ansible on_any args TASK: ovirt.ovirt.hosted_engine_setup : Set the name for add_host kwargs is_conditional:False
1893 2023-11-19 22:19:45,664+0000 DEBUG ansible on_any args localhost TASK: ovirt.ovirt.hosted_engine_setup : Set the name for add_host kwargs
1894 2023-11-19 22:19:45,759+0000 ERROR ansible failed {
1895 "ansible_host": "localhost",
1896 "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/he_ansible/trigger_role.yml",
1897 "ansible_result": {
1898 "_ansible_no_log": false,
1899 "msg": "The task includes an option with an undefined variable. The error was: 'local_vm_ip' is undefined\n\nThe error appears to be in '/usr/share/ansible/collections/ansible_collections/ovirt/ovirt/roles/hosted_en gine_setup/tasks/sync_on_engine_machine.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: Set the name for add_host\n ^ he re\n"
1900 },
1901 "ansible_task": "Set the name for add_host",
1902 "ansible_type": "task",
1903 "status": "FAILED",
1904 "task_duration": 0
1905 }
Summary is:
2197 2023-11-19 22:20:02,141+0000 INFO SUMMARY:
2198 Duration Task Name
2199 -------- --------
2200 [ < 1 sec ] Execute just a specific set of steps
2201 [ 00:01 ] Force facts gathering
2202 [ FAILED ] Set the name for add_host
2203 [ < 1 sec ] Fetch the value of HOST_KEY_CHECKING
2204 [ < 1 sec ] Get the username running the deploy
2205 [ FAILED ] Register the engine FQDN as a host
2206 [ FAILED ] Sync on engine machine
2207 [ < 1 sec ] Fetch logs from the engine VM
2208 [ < 1 sec ] Set destination directory path
2209 [ < 1 sec ] Create destination directory
2210 [ < 1 sec ] Find the local appliance image
2211 [ 00:10 ] Give the vm time to flush dirty buffers
2212 [ < 1 sec ] Change ownership of copied engine logs
2213 [ < 1 sec ] Remove local vm dir
2214 [ < 1 sec ] Remove temporary entry in /etc/hosts for the local VM
2215 [ < 1 sec ] Clean local storage pools
2216 [ FAILED ] Destroy local storage-pool localvmk0rv3u2g
2217 [ FAILED ] Undefine local storage-pool localvmk0rv3u2g
2218 [ FAILED ] Destroy local storage-pool {{ local_vm_disk_path.split('/')[5] }}
2219 [ FAILED ] Undefine local storage-pool {{ local_vm_disk_path.split('/')[5] }}
I am using a viable FQDN and IP for my engine which is on the same network as the host. I've been at this for more than a week with different options/OSs. Figured this was the most basic possible configuration and yet it still doesn't work. Incredibly frustrating product.
8 months, 1 week
Weirdness with attempting NFS mount
by Michaal R
I'm attempting to mount a share from a TrueNAS server in oVirt through the Storage > Storage Domains UI as a New Domain. Whenever I try to add the domain (as server:share), it kicks back a Permission Denied error. Looking at the vdsm.log file, I see the below snippet:
2023-11-17 10:05:32,059-0600 INFO (jsonrpc/7) [vdsm.api] START connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'protocol_version': 'auto', 'port': '', 'iqn': '', 'connection': '192.168.1.11:/mnt/POOL0', 'ipv6_enabled': 'false', 'id': '00000000-0000-0000-0000-000000000000', 'user': '', 'tpgt': '1'}]) from=::ffff:192.168.1.16,50468, flow_id=31d18c09-2e12-480f-8d6d-722a9fe78029, task_id=05818e9d-562f-4ca7-a19b-5c3b5a02bae8 (api:31)
2023-11-17 10:05:32,060-0600 INFO (jsonrpc/7) [storage.storageServer] Creating directory '/rhev/data-center/mnt/192.168.1.11:_mnt_POOL0' (storageServer:217)
2023-11-17 10:05:32,060-0600 INFO (jsonrpc/7) [storage.fileutils] Creating directory: /rhev/data-center/mnt/192.168.1.11:_mnt_POOL0 mode: None (fileUtils:213)
2023-11-17 10:05:32,060-0600 INFO (jsonrpc/7) [storage.mount] mounting 192.168.1.11:/mnt/POOL0 at /rhev/data-center/mnt/192.168.1.11:_mnt_POOL0 (mount:190)
2023-11-17 10:05:32,150-0600 WARN (jsonrpc/7) [storage.oop] Permission denied for directory: /rhev/data-center/mnt/192.168.1.11:_mnt_POOL0 with permissions:7 (outOfProcess:176)
2023-11-17 10:05:32,150-0600 INFO (jsonrpc/7) [storage.mount] unmounting /rhev/data-center/mnt/192.168.1.11:_mnt_POOL0 (mount:198)
2023-11-17 10:05:32,200-0600 ERROR (jsonrpc/7) [storage.storageServer] Could not connect to storage server (storageServer:75)
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/vdsm/storage/fileSD.py", line 64, in validateDirAccess
getProcPool().fileUtils.validateAccess(dirPath)
File "/usr/lib/python3.9/site-packages/vdsm/storage/outOfProcess.py", line 178, in validateAccess
raise OSError(errno.EACCES, os.strerror(errno.EACCES))
PermissionError: [Errno 13] Permission denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/vdsm/storage/storageServer.py", line 73, in connect_all
con.connect()
File "/usr/lib/python3.9/site-packages/vdsm/storage/storageServer.py", line 507, in connect
return self._mountCon.connect()
File "/usr/lib/python3.9/site-packages/vdsm/storage/storageServer.py", line 241, in connect
six.reraise(t, v, tb)
File "/usr/lib/python3.9/site-packages/six.py", line 709, in reraise
raise value
File "/usr/lib/python3.9/site-packages/vdsm/storage/storageServer.py", line 233, in connect
fileSD.validateDirAccess(
File "/usr/lib/python3.9/site-packages/vdsm/storage/fileSD.py", line 75, in validateDirAccess
raise se.StorageServerAccessPermissionError(dirPath)
vdsm.storage.exception.StorageServerAccessPermissionError: Permission settings on the specified path do not allow access to the storage. Verify permission settings on the specified storage path.: 'path = /rhev/data-center/mnt/192.168.1.11:_mnt_POOL0'
Stepping it through, am I correct that I'm seeing vdsm not able to create the local directory for the NFS mount? It previously was able to mount an NFS share from the host to itself when I initially setup the server for testing.
When I do an ls -l on the directory it's trying to create the mount point in, I see ownership looks correct.
[root@ovirt ~]# cd /rhev/data-center/
[root@ovirt data-center]# ls
c7d57de0-6ba8-11ee-9a48-00163e1989b9 mnt
[root@ovirt data-center]# ls -l
total 0
drwxr-xr-x. 2 vdsm kvm 66 Nov 15 23:32 c7d57de0-6ba8-11ee-9a48-00163e1989b9
drwxr-xr-x. 3 vdsm kvm 59 Nov 17 10:05 mnt
[root@ovirt data-center]# cd mnt
[root@ovirt mnt]# ls -l
total 0
drwxr-xr-x. 3 vdsm kvm 50 Nov 17 10:14 ovirt.theclocktower.home:_home_ovirt__storage
[root@ovirt mnt]#
Is there something missing? Something that got misconfigured, maybe? Or something I can configure to fix this?
8 months, 1 week
I need help to understand...
by martind＠towebs.com
Hi my name is Martin and im currently working in a Hosting Company, we want to Migrate our servers to Ovirt VMS. We want to know if Ovirt is capable of Hosting Loads and we want to know what type of Environment we need to create for this. Thank you
8 months, 1 week
oVirt host locked in installing status
by Michele la Torre
Hi all,
I have an oVirt installation (version 4.5.4-1.el8) with the engine installed in standalone mode. One of the hosts, after encountering some issues with the certificate renewal (which have now been resolved, and there are no current errors related to this), is stuck in the "Installing" state.
I've checked the logs, but there are no errors reported. Do you have any ideas on what I can try to resolve this?
Best regards,
Michele
8 months, 1 week
obtain direct LUN wwid in the guest
by LS CHENG
Hi all
I am working with Oracle Linux Virtualization Manager 4 (OLVM) which is
based on ovirt.
When I worked with OLVM 3 and when assigning direct lun to the guests in
the guest I could obtain LUN's wwid with this command
/usr/lib/udev/scsi_id -g -u -d /dev/sdc1
36000c29f1382ae964d67392dfdfaaf0f
similar to VMWare's disk.EnableUUID
But now in OLVM 4 I am getting this in the guest which is not LUN's wwid
/usr/lib/udev/scsi_id -g -u -d /dev/sdc
0QEMU_QEMU_HARDDISK_671b475e-98ab-4f3b-a23d-2f8eed964e92
I wonder if is it possible to get LUN's wwid in the guest in OLVM 4?
Thanks1
8 months, 2 weeks
/etc/pki/ovirt-engine/ca.pem is not on a local filesystem
by nicolas＠devels.es
Hi,
We're running oVirt 4.5.4, recently we got this alert:
Engine's certification is about to expire at 2023-11-19. Please renew
the engine's certification.
So I'm trying to run:
engine-setup --offline
However, it fails with the following error:
[ INFO ] Upgrading CA
[ INFO ] Renewing engine certificate
[ ERROR ] Failed to execute stage 'Misc configuration': Command
'/usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh' failed to execute
Digging into the logs I can see this:
2023-11-14 08:36:22,848+0000 DEBUG
otopi.plugins.ovirt_engine_setup.ovirt_engine.pki.ca plugin.execute:926
execute-output: ('/usr/share/ovirt-engine/bin/pki-enroll- pkcs12.sh',
'--name=engine', '--password=**FILTERED**',
'--subject=/C=US/O=stic.ull.es/CN=fqdn.es', '--san=DNS:fqdn.es',
'--keep-key') stderr:
Ignoring -days; not generating a certificate
/etc/pki/ovirt-engine/ca.pem is not on a local filesystem
Cannot sign request
2023-11-14 08:36:22,849+0000 DEBUG otopi.context
context._executeMethod:145 method exception
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132,
in _executeMethod
method['method']()
File
"/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/pki/ca.py",
line 753, in _miscUpgrade
self._enrollCertificates(True, uninstall_files)
File
"/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/pki/ca.py",
line 360, in _enrollCertificates
shortLife=entry['shortLife'],
File
"/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/pki/ca.py",
line 250, in _enrollCertificate
+ (('--days=398',) if shortLife else ())
File "/usr/lib/python3.6/site-packages/otopi/plugin.py", line 931,
in execute
command=args[0],
RuntimeError: Command
'/usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh' failed to execute
2023-11-14 08:36:22,852+0000 ERROR otopi.context
context._executeMethod:154 Failed to execute stage 'Misc configuration':
Command '/usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh' failed to
execute
However, the file exists and is on a local filesystem:
# ll /etc/pki/ovirt-engine/ca.pem
-rw-r--r--. 1 root root 4516 jun 24 2015 /etc/pki/ovirt-engine/ca.pem
Can someone shed some light about why is this failing and how to solve
it, please?
Thanks.
8 months, 2 weeks
Ovirt 4.3 console.vv
by eevans＠digitaldatatechs.com
I have CentOS Stream 9 on my laptop with KDE Plasma Workspaces.
I installedd virt-viewer and tried through gui, console and it tells me the virtual machine can't be found.
It works fine with the Windows viewer. It seems to be a Centos issue. Something is not reading or interuppreting the file properly.
I'd like to get this resolved.
I hope someone has an answer.
Thanks in advance.
E
8 months, 2 weeks
Certificate replacing through engine-setup failed
by dominic.gwerder＠bits.ch
Hi
I'm trying to replace the old certificate of the engine, so tha I can again access the management console. I've been reading thorugh a lot of articels and threads and the easiest solution, should be by running the engine-setup. I always get the following error:
[ INFO ] Creating CA: /etc/pki/ovirt-engine/ca.pem
[ ERROR ] Failed to execute stage 'Misc configuration': Command '/usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh' failed to execute
How can I fix this?
Thanks in advance for any help!
Kind regards,
Dominic
8 months, 2 weeks
Gluster: Ideas for migration
by jonas＠rabe.ch
Hello
I have to migrate the Gluster volumes from an old oVirt cluster to a newly built one. I looked into migration strategies, but everything that Red Hat recommends is related to replacing old bricks. In a testing environment I created two clusters and wanted to migrate one volume after the other. Unfortunately that fails because a node cannot be part of two clusters at the same time.
The next thing I see, is to recreate the volumes on the new cluster, then constantly rsync the files from the old cluster to the new one and at a specified point in time make the cut over where I stop the applicaiton, do a final rsync and remount the new volume under the old path.
Is there any other, nicer way I could accomplish migrating a volume from one Gluster cluster to another?
8 months, 2 weeks
