hosted-engine-setup --deploy fail on Centos Stream 8
Hy, I am trying to install ovirt 4.5 on a 4-host cluster running Centos Stream 8, but the engine does not start and the whole process fails. Here is my procedure dnf install centos-release-ovirt45 dnf module reset virt dnf module enable virt:rhel dnf install ovirt-engine-appliance dnf install ovirt-hosted-engine-setup The latest version of ansible [ansible-core 2.13] uses python3.9 and the installation fails because some python3.9 modules are missing [python39-netaddr, python39-jmespath] and cannot be installed [conflict python3-jmespath]. So I downgraded ansible to ansible-core 2.12 dnf downgrade ansible-core Now hosted-engine-setup --deploy --4 goes proceed further but stops because it cannot start the engine [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Wait for the host to be up] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Host is not up, please check logs, perhaps also on the engine machine"} I looked into the log file /var/log//ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-ansible-bootstrap_local_vm-20221007132728-yp7cd1.log and I found the following error: 2022-10-07 13:28:30,881+0200 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/he_ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "changed": false, "cmd": [ "virsh", "net-undefine", "default" ], "delta": "0:00:00.039258", "end": "2022-10-07 13:28:30.710401", "invocation": { "module_args": { "_raw_params": "virsh net-undefine default", "_uses_shell": false, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "stdin_add_newline": true, "strip_empty_ends": true, "warn": false } }, "msg": "non-zero return code", "rc": 1, "start": "2022-10-07 13:28:30.671143", "stderr": "error: failed to get network 'default'\nerror: Network not found: no network with matching name 'default'", "stderr_lines": [ "error: failed to get network 'default'", "error: Network not found: no network with matching name 'default'" ], "stdout": "", "stdout_lines": [] }, "ansible_task": "Update libvirt default network configuration, undefine", "ansible_type": "task", "status": "FAILED", "task_duration": 0 } Needless to say firewalld and libvirtd are both up and virsh net-list gives: Name State Autostart Persistent ------------------------------------------------ ;vdsmdummy; active no no default active no yes I googled around without success. Has anyone had similar problems? End of past July I installed Ovirt on another cluster running Centos Stream 8 following the procedure I just described with no problem. If needed I can post all log files. Thanks for the help. Best Andrea
On Tue, Oct 11, 2022 at 4:54 AM andrea.crisanti--- via Users <users@ovirt.org> wrote:
Hy,
I am trying to install ovirt 4.5 on a 4-host cluster running Centos Stream 8, but the engine does not start and the whole process fails.
Here is my procedure
dnf install centos-release-ovirt45 dnf module reset virt dnf module enable virt:rhel dnf install ovirt-engine-appliance dnf install ovirt-hosted-engine-setup
The latest version of ansible [ansible-core 2.13] uses python3.9 and the installation fails because some python3.9 modules are missing [python39-netaddr, python39-jmespath] and cannot be installed [conflict python3-jmespath]. So I downgraded ansible to ansible-core 2.12
dnf downgrade ansible-core
+1
Now
hosted-engine-setup --deploy --4
goes proceed further but stops because it cannot start the engine
[ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Wait for the host to be up] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Host is not up, please check logs, perhaps also on the engine machine"}
Please note the error message - "Host is not up", not "engine is not up". This means that the deploy process did setup and start an engine, then tried to add the host (the one you deploy on) to the engine, and this failed somewhere.
I looked into the log file /var/log//ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-ansible-bootstrap_local_vm-20221007132728-yp7cd1.log and I found the following error:
2022-10-07 13:28:30,881+0200 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/he_ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "changed": false, "cmd": [ "virsh", "net-undefine", "default" ], "delta": "0:00:00.039258", "end": "2022-10-07 13:28:30.710401", "invocation": { "module_args": { "_raw_params": "virsh net-undefine default", "_uses_shell": false, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "stdin_add_newline": true, "strip_empty_ends": true, "warn": false } }, "msg": "non-zero return code", "rc": 1, "start": "2022-10-07 13:28:30.671143", "stderr": "error: failed to get network 'default'\nerror: Network not found: no network with matching name 'default'", "stderr_lines": [ "error: failed to get network 'default'", "error: Network not found: no network with matching name 'default'" ], "stdout": "", "stdout_lines": [] }, "ansible_task": "Update libvirt default network configuration, undefine",
This isn't your issue. If you check the code, you see that this task has "ignore_errors: true", and actually failure there is almost always expected.
"ansible_type": "task", "status": "FAILED", "task_duration": 0 }
Needless to say firewalld and libvirtd are both up and virsh net-list gives:
Name State Autostart Persistent ------------------------------------------------ ;vdsmdummy; active no no default active no yes
I googled around without success.
Has anyone had similar problems?
Please check/share /var/log/ovirt-engine on the engine machine trying to understand why adding the host failed. The engine machine, at this point, is still local to your host, having a private IP address - you can find that one by searching /var/log/ovirt-hosted-engine-setup/* for 'local_vm_ip'. I usually use this: # ssh -o StrictHostKeyChecking=No -o UserKnownHostsFile=/dev/null -l root $(grep -A20 local_vm_ip $(ls -t /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-ansible-bootstrap_local_vm* | head -1) | sed -n 's/^ "stdout": "\([^"]*\)",/\1/p' | sed 's/\\n.*//') Best regards, -- Didi
Dear Didi, thanks for the answer. it seems that the problem is again the latest release di ansible-core and python3.9. At the end of the file I /var/log/ovirt-engine/host-deploy/ovirt-host-deploy-ansible* on the engine machine I found 2022-10-17 09:33:58 CEST - TASK [ovirt-provider-ovn-driver : Install ovs] ********************************* 2022-10-17 09:33:58 CEST - { "uuid" : "b6692a46-8e8d-47a7-9911-0c97ec17de25", "counter" : 391, "stdout" : "fatal: [odip2.phys.uniroma1.it]: FAILED! => {\"msg\": \"The conditional check 'cluster_switch == \\\"ovs\\\" or (ovn_central is defined and ovn_central | ipaddr)' faile d. The error was: The ipaddr filter requires python's netaddr be installed on the ansible controller\\n\\nThe error appears to be in '/usr/share/ovirt-engine/ansible-runner-service-p roject/project/roles/ovirt-provider-ovn-driver/tasks/configure.yml': line 3, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending li ne appears to be:\\n\\n- block:\\n - name: Install ovs\\n ^ here\\n\"}", "start_line" : 393, "end_line" : 394, "runner_ident" : "e1c34977-ad90-4072-8f37-ac9634b540f8", "event" : "runner_on_failed", "pid" : 101191, "created" : "2022-10-17T07:33:57.718868", "parent_uuid" : "00163e51-5206-c673-1166-00000000003d", "event_data" : { "playbook" : "ovirt-host-deploy.yml", "playbook_uuid" : "44a98a56-0d38-4b58-8d7d-b875c545c544", "play" : "all", "play_uuid" : "00163e51-5206-c673-1166-000000000006", "play_pattern" : "all", "task" : "Install ovs", "task_uuid" : "00163e51-5206-c673-1166-00000000003d", "task_action" : "package", "task_args" : "", "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-provider-ovn-driver/tasks/configure.yml:3", "role" : "ovirt-provider-ovn-driver", "host" : "odip2.phys.uniroma1.it", "remote_addr" : "odip2.phys.uniroma1.it", "res" : { "msg" : "The conditional check 'cluster_switch == \"ovs\" or (ovn_central is defined and ovn_central | ipaddr)' failed. The error was: The ipaddr filter requires python's netaddr be installed on the ansible controller\n\nThe error appears to be in '/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-provider-ovn-driver/tasks/configure.ym l': line 3, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n - name: Install ovs\n ^ here\n", "_ansible_no_log" : false }, "start" : "2022-10-17T07:33:57.651853", "end" : "2022-10-17T07:33:57.718602", "duration" : 0.066749, "ignore_errors" : null, "event_loop" : null, "uuid" : "b6692a46-8e8d-47a7-9911-0c97ec17de25" } } It complains about the module netaddr of python. And this is the reason while I downgraded ansible-core on the host machine I use to deploy the engine. On the host machine I have: [root@odip2 ~]# dnf list installed | grep ansible ansible-collection-ansible-netcommon.noarch 2.2.0-3.2.el8 @centos-ovirt45 ansible-collection-ansible-posix.noarch 1.3.0-1.2.el8 @centos-ovirt45 ansible-collection-ansible-utils.noarch 2.3.0-2.2.el8 @centos-ovirt45 ansible-core.x86_64 2.12.7-1.el8 @appstream ovirt-ansible-collection.noarch 2.2.2-1.el8 @centos-ovirt45 [root@odip2 ~]# ansible-playbook --version ansible-playbook [core 2.12.7] config file = /etc/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3.8/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /usr/bin/ansible-playbook python version = 3.8.13 (default, Jun 24 2022, 15:27:57) [GCC 8.5.0 20210514 (Red Hat 8.5.0-13)] jinja version = 2.11.3 libyaml = True [root@odip2 ~]# dnf list installed | grep netad python3-netaddr.noarch 0.7.19-8.1.1.el8 @centos-ovirt45 python38-netaddr.noarch 0.7.19-8.1.1.el8 @centos-ovirt45 while on the engine machine [root@ovirt-dip ovirt-engine]# dnf list installed | grep ansible ansible-collection-ansible-netcommon.noarch 2.2.0-3.2.el8 @centos-ovirt45 ansible-collection-ansible-posix.noarch 1.3.0-1.2.el8 @centos-ovirt45 ansible-collection-ansible-utils.noarch 2.3.0-2.2.el8 @centos-ovirt45 ansible-core.x86_64 2.13.3-1.el8 @appstream ansible-runner.noarch 2.1.3-1.el8 @centos-ovirt45 ovirt-ansible-collection.noarch 2.2.2-1.el8 @centos-ovirt45 python38-ansible-runner.noarch 2.1.3-1.el8 @centos-ovirt45 [root@ovirt-dip ovirt-engine]# ansible-playbook --version ansible-playbook [core 2.13.3] config file = /etc/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3.9/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /usr/bin/ansible-playbook python version = 3.9.13 (main, Jun 24 2022, 15:32:51) [GCC 8.5.0 20210514 (Red Hat 8.5.0-13)] jinja version = 3.1.2 libyaml = True [root@ovirt-dip ovirt-engine]# dnf list installed | grep netad python3-netaddr.noarch 0.7.19-8.1.1.el8 @centos-ovirt45 python38-netaddr.noarch 0.7.19-8.1.1.el8 @centos-ovirt45 No module netaddr for python3.9 Best regards Andrea
Dear Didi, thanks for the answer. I do not know I can can attach file on the online reply, here is the tar of /var/log/ovirt-engine/ Best Andrea -- "Trust, but verify." "Essere libero costa soltanto qualche rimpianto" Russian Proverb Vasco Rossi ========================================================================== Prof. Andrea Crisanti Tel. +39-06 4991 3431 Dipartimento di Fisica Fax. +39-06 4463158 Universita' di Roma "La Sapienza" andrea.crisanti@uniroma1.it P.le A. Moro 2 andrea.crisanti@roma1.infn.it I-00185 Roma, Italy
Solved upgrading ovirt-engine-appliance to the latest version 4.5-20221018071047.1.el8 solve the problem. Still ansible-core must be downgraded before running hosted-engine --deploy Best Andrea
participants (3)
-
Andrea Crisanti -
andrea.crisanti＠uniroma1.it -
Yedidyah Bar David