6:32 a.m.
On Thu, May 13, 2021 at 3:34 PM Sketch <ovirt(a)rednsx.org> wrote:
This is a new system is CentOS 8.3, with the oVirt-4.4 repo and all
updates applied. When I try to install the hosted engine with my engine
backup from 4.3.10, the installation fails with a too many open files
error. My 8.3 hosts already had 1M system max files, which is more than
any of my CentOS 7/oVirt 4.3 hosts have. I tried increasing it to 2M with
no luck, so my suspicion is that the error is on the engine itself?
I tried provisioning a new engine just to test, and I get SSH key errors
instead of this one.
Any suggestions?
2021-05-12 23:09:44,731-0700 ERROR ansible failed {
"ansible_host": "localhost",
"ansible_playbook":
"/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml",
"ansible_result": {
"_ansible_no_log": false,
"exception": "Traceback (most recent call last):\n File
\"/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py\", line
665, in _execute\n result = self._handler.run(task_vars=variables)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/action/wait_for_connection.py\",
line 122, in run\n self._remove_tmp_path(self._connection._shell.tmpdir)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line
417, in _remove_tmp_path\n tmp_rm_res = self._low_level_execute_command(cmd,
sudoable=False)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line
1085, in _low_level_execute_command\n rc, stdout, stderr =
self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py\", line
1191, in exec_command\n cmd = self._build_command(*args)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/connection/s
sh.py\", line 562, in _build_command\n self.sshpass_pipe = os.pipe()\nOSError:
[Errno 24] Too many open files\n\nDuring handling of the above exception, another
exception occurred:\n\nTraceback (most recent call last):\n File
\"/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py\", line
147, in run\n res = self._execute()\n File
\"/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py\", line
673, in _execute\n self._handler.cleanup()\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line
128, in cleanup\n self._remove_tmp_path(self._connection._shell.tmpdir)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line
417, in _remove_tmp_path\n tmp_rm_res = self._low_level_execute_command(cmd,
sudoable=False)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py\", line
1085, in _low_level_execute_command\n rc, stdout, stderr =
self._connection.exec_command
(cmd, in_data=in_data, sudoable=sudoable)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py\", line
1191, in exec_command\n cmd = self._build_command(*args)\n File
\"/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py\", line
562, in _build_command\n self.sshpass_pipe = os.pipe()\nOSError: [Errno 24] Too many
open files\n",
"msg": "Unexpected failure during module execution.",
"stdout": ""
},
"ansible_task": "Wait for the local VM",
"ansible_type": "task",
"status": "FAILED",
"task_duration": 3605
So I suppose it failed after 3605 seconds, or 721 attempts (of 5 seconds each).
Do you see the VM in 'virsh list'?
Can you see the VM running (e.g. 'ps auxww | grep qemu')?
Can you try to ssh to it from the host (search the logs for
local_vm_ip for its local/private temporary address)?
Perhaps open its console (Perhaps 'virsh console HostedEngineLocal')?
That said, I'd personally also consider it a bug in ansible, unless
you made some relevant custom changes - the bug is that it seems to
leak open files.
Thanks and best regards,
--
Didi