Cluster stuck in maintenance - Hosts not upgrading as expected
by David White
I have a 3-node hyperconverged cluster with Gluster filesystem running on RHEL 8.3 hosts.
It's been stable on oVirt 4.5.
Today, I just upgraded the Engine to v4.6.
[Screenshot from 2021-05-22 20-29-23.png]
I then logged into the oVirt manager, navigated to Compute -> Clusters, and clicked on Upgrade.
The Status of the cluster does have an exclamation mark that says "Upgrade Cluster Compatibility Level", but my understanding is, I shouldn't do that until all 3 of my hosts have been upgraded.
Looking at active tasks, I see that the first host in my cluster is stuck in "Upgrading".
It's now been like this for 3 hours, without any visible progress.
If I try to launch a VM that is not configured for high availability, the VM fails to launch, and I get a message about the cluster being in "Maintenance Mode" and that none of the hosts satisfies current scheduling restraints.
How do I....
a) Cancel / clear the upgrade and take the cluster out of "maintenance mode"
b) Properly upgrade the cluster, since clicking the "Upgrade" button clearly didn't work?
Thanks,
David
Sent with ProtonMail Secure Email.
2 years, 11 months
cleanup does not cleanup
by lejeczek
Hi guys.
-> $ ovirt-hosted-engine-cleanup -q
says it ran its stuff, done & dusted yet I see 'ovirtmgmt'
bridge remains.
Would that be a bug and how to clean it?
many thanks, L.
2 years, 11 months
Problems provisioning 4.4.6 hosted engine
by Sketch
This is a new system is CentOS 8.3, with the oVirt-4.4 repo and all
updates applied. When I try to install the hosted engine with my engine
backup from 4.3.10, the installation fails with a too many open files
error. My 8.3 hosts already had 1M system max files, which is more than
any of my CentOS 7/oVirt 4.3 hosts have. I tried increasing it to 2M with
no luck, so my suspicion is that the error is on the engine itself?
I tried provisioning a new engine just to test, and I get SSH key errors
instead of this one.
Any suggestions?
2021-05-12 23:09:44,731-0700 ERROR ansible failed {
"ansible_host": "localhost",
"ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
"ansible_result": {
"_ansible_no_log": false,
"exception": "Traceback (most recent call last):\n File \"/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py\", line 665, in _execute\n result = self._handler.run(task_vars=variables)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/action/wait_for_connection.py\", line 122, in run\n self._remove_tmp_path(self._connection._shell.tmpdir)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line 417, in _remove_tmp_path\n tmp_rm_res = self._low_level_execute_command(cmd, sudoable=False)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line 1085, in _low_level_execute_command\n rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py\", line 1191, in exec_command\n cmd = self._build_command(*args)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/connection/s
sh.py\", line 562, in _build_command\n self.sshpass_pipe = os.pipe()\nOSError: [Errno 24] Too many open files\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py\", line 147, in run\n res = self._execute()\n File \"/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py\", line 673, in _execute\n self._handler.cleanup()\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line 128, in cleanup\n self._remove_tmp_path(self._connection._shell.tmpdir)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line 417, in _remove_tmp_path\n tmp_rm_res = self._low_level_execute_command(cmd, sudoable=False)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line 1085, in _low_level_execute_command\n rc, stdout, stderr = self._connection.exec_command
(cmd, in_data=in_data, sudoable=sudoable)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py\", line 1191, in exec_command\n cmd = self._build_command(*args)\n File \"/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py\", line 562, in _build_command\n self.sshpass_pipe = os.pipe()\nOSError: [Errno 24] Too many open files\n",
"msg": "Unexpected failure during module execution.",
"stdout": ""
},
"ansible_task": "Wait for the local VM",
"ansible_type": "task",
"status": "FAILED",
"task_duration": 3605
}
2 years, 11 months
Import to VMware
by Darin Schmidt
Hello,
Ive exported an OVA of a server we had made on OVIRT and Im trying to import it to VMware. Its complaining that the ovf:format is incorrect.
ovf:format="http://www.gnome.org/~markmc/qcow-image-format.html"
That link doesnt resolve to anywhere because it appears its now http://people.gnome.org/~markmc/qcow-image-format.html
But I dont know whats supposed to go there nor have I been able to locate anything to help solve this issue. All I really need is access to the files in the, what I assume is, vdmk file that comes along with the ovf file, which mine is called: 154b3d77-d340-4d24-a448-66265c5fb613
So if anything, is there a way to open this vm disk so I can gain access to the files?
Thanks
2 years, 11 months
Error Adding host to cluster
by David Johnson
When I attempt to install a new host to my cluster from the GUI I get this
set of errors/warnings, and the install fails.
- Host has no default route
- The Host CPU does not match the Cluster CPU Tye and is running in a
degraded mode. It is missing the following CPU flags: vmx,
model_Cascadelake-Server-noTSX.
The new host has a brand new bare centos 8 installation on it. It
successfully installed from the internet.
The *new host* hardware is 48x Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
The *pre-existing host* is 64x Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
If I am reading the ansible logs correctly, ansible is failing to find the
*ovirt-hosted-engine-setup* package, which is confusing since I have a
separate physical machine for the engine (which I am using to install
from), and don't want a self hosted engine.
Any advice would be welcome.
---
Ansible logs from the most recent failed installation attempt:
2021-05-14 20:18:04,949 - runner_service.services.playbook - DEBUG -
cb_event_handler event_data={'uuid':
'fa961926-9a3a-4d9e-a6bf-d53f98645af2', 'counter': 52, 'stdout': 'fatal: [
ovirt-host-04.maxisinc.net]: FAILED! => {"changed": false, "failures": ["No
package ovirt-hosted-engine-setup available."], "msg": "Failed to install
some of the specified packages", "rc": 1, "results": []}', 'start_line':
44, 'end_line': 45, 'runner_ident': '6060ba26-b51b-11eb-ba6e-d8cb8a87894e',
'event': 'runner_on_failed', 'pid': 1742807, 'created':
'2021-05-15T01:18:04.947870', 'parent_uuid':
'd8cb8a87-894e-e341-1e1c-0000000001a1', 'event_data': {'playbook':
'ovirt-host-deploy.yml', 'playbook_uuid':
'e5595457-2c74-4525-8d98-e34bf2e0e342', 'play': 'all', 'play_uuid':
'd8cb8a87-894e-e341-1e1c-000000000007', 'play_pattern': 'all', 'task':
'Install ovirt-hosted-engine-setup package', 'task_uuid':
'd8cb8a87-894e-e341-1e1c-0000000001a1', 'task_action': 'yum', 'task_args':
'', 'task_path':
'/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-deploy-vdsm/tasks/packages.yml:6',
'role': 'ovirt-host-deploy-vdsm', 'host': 'ovirt-host-04.maxisinc.net',
'remote_addr': 'ovirt-host-04.maxisinc.net', 'res': {'msg': 'Failed to
install some of the specified packages', 'failures': ['No package
ovirt-hosted-engine-setup available.'], 'results': [], 'rc': 1,
'invocation': {'module_args': {'name': ['ovirt-hosted-engine-setup'],
'state': 'present', 'allow_downgrade': False, 'autoremove': False,
'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [],
'disablerepo': [], 'download_only': False, 'enable_plugin': [],
'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery':
True, 'install_weak_deps': True, 'security': False, 'skip_broken': False,
'update_cache': False, 'update_only': False, 'validate_certs': True,
'lock_timeout': 30, 'conf_file': None, 'disable_excludes': None,
'download_dir': None, 'list': None, 'releasever': None}},
'_ansible_no_log': False, 'changed': False}, 'start':
'2021-05-15T01:18:03.800468', 'end': '2021-05-15T01:18:04.947696',
'duration': 1.147228, 'ignore_errors': None, 'event_loop': None, 'uuid':
'fa961926-9a3a-4d9e-a6bf-d53f98645af2'}}
2021-05-14 20:18:04,951 - runner_service.services.playbook - DEBUG -
cb_event_handler event_data={'uuid':
'1007f755-dee8-407f-999c-b08c8b813ea7', 'counter': 53, 'stdout': '\r\nPLAY
RECAP
*********************************************************************\r\
novirt-host-04.maxisinc.net : ok=14 changed=1 unreachable=0
failed=1 skipped=2 rescued=0 ignored=0 \r\n', 'start_line': 45,
'end_line': 49, 'runner_ident': '6060ba26-b51b-11eb-ba6e-d8cb8a87894e',
'event': 'playbook_on_stats', 'pid': 1742807, 'created':
'2021-05-15T01:18:04.949918', 'parent_uuid':
'e5595457-2c74-4525-8d98-e34bf2e0e342', 'event_data': {'playbook':
'ovirt-host-deploy.yml', 'playbook_uuid':
'e5595457-2c74-4525-8d98-e34bf2e0e342', 'changed': {'
ovirt-host-04.maxisinc.net': 1}, 'dark': {}, 'failures': {'
ovirt-host-04.maxisinc.net': 1}, 'ignored': {}, 'ok': {'
ovirt-host-04.maxisinc.net': 14}, 'processed': {'ovirt-host-04.maxisinc.net':
1}, 'rescued': {}, 'skipped': {'ovirt-host-04.maxisinc.net': 2},
'artifact_data': {}, 'uuid': '1007f755-dee8-407f-999c-b08c8b813ea7'}}
2021-05-14 20:18:05,147 - runner_service.services.playbook - INFO -
Playbook ovirt-host-deploy.yml, UUID=6060ba26-b51b-11eb-ba6e-d8cb8a87894e
ended, status=failed
2021-05-14 20:18:05,154 - runner_service.services.playbook - INFO -
Playbook ovirt-host-deploy.yml Stats: {'skipped': {'
ovirt-host-04.maxisinc.net': 2}, 'ok': {'ovirt-host-04.maxisinc.net': 14},
'dark': {}, 'failures': {'ovirt-host-04.maxisinc.net': 1}, 'processed': {'
ovirt-host-04.maxisinc.net': 1}, 'changed': {'ovirt-host-04.maxisinc.net':
1}}
*David Johnson*
2 years, 11 months
I am trying to setup ovirt hosted engine.
by Ken.Germann＠L3Harris.com
I select eth2 as the default interface for the management network for the VM network.
I get this error:
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The selected network interface is not valid"}
[ ERROR ] Failed to execute stage 'Closing up': Failed executing ansible-playbook
It appears to be expecting a bridge interface to be setup...
Thank You,
Ken Germann
UNAP IRAD System Administrator Lead
Aviation Systems / l3HARRIS Technologies
Office: +1-954-732-0391 / Mobile: +1-954-732-0391
l3harris.com<mailto:l3harris.com> / kgermann(a)l3harris.com<mailto:kgermann@l3harris.com>
MS F-11A / 1025 W. Nasa Dr. / Melbourne, FL 32919 / USA
[rsz_1harriswebpage_rotator]
CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient and may contain material that is proprietary, confidential, privileged or otherwise legally protected or restricted under applicable government laws. Any review, disclosure, distributing or other use without expressed permission of the sender is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies without reading, printing, or saving.
2 years, 11 months
error: Engine VM IP address is ... what?
by lejeczek
Hi guys.
Would anybody be able to decipher this below?
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Fail if
Engine IP is different from engine's he_fqdn resolved IP]
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false,
"msg": "Engine VM IP address is while the engine's he_fqdn
buster.private.road resolves to 10.3.1.113. If you are using
DHCP, check your DHCP reservation configuration"}
[ ERROR ] Failed to execute stage 'Closing up': Failed
executing ansible-playbook
many thanks, L.
2 years, 11 months
vmfex in 4.4.7
by Michal Skrivanek
Hi there,
just a heads up that in 4.4.7 we’ll be removing vdsm’s hard dependency on VM-FEX hook. The functionality has been more or less obsoleted by SR-IOV and its usage is minimal these days. With the upgrade the vdsm-hook-vmfex-dev rpm will get automatically uninstalled.
If you happen to use it and it works well for you then please "dnf install" it after the upgrade. There is no change in functionality and the hook is not going away, it’s just that in order to not make it mandatory from now on, it gets removed during "dnf upgrade”.
Thanks,
michal
2 years, 11 months
Bridge Setup for Ovirt
by ken＠everheartpartners.com
I have 3 network interfaces
eth0 for storage
eth1 for public network
eth2 for the management network
Does someone have the example of how to setup the bridge interface network for the hosted engine setup for Ovir 4.3.10 in my case?
2 years, 11 months
