*Hi all,* *[root@bric-ovirt-1 ~]# cat /etc/*release** CentOS Linux release 7.7.1908 (Core) *[root@bric-ovirt-1 ~]# yum info ovirt-engine-appliance* Installed Packages Name : ovirt-engine-appliance Arch : x86_64 Version : 4.3 Release : 20191121.1.el7 Size : 1.0 G Repo : installed From repo : ovirt-4.3 *Same situation as https://bugzilla.redhat.com/show_bug.cgi?id=1787267 <https://bugzilla.redhat.com/show_bug.cgi?id=1787267>;. The error message almost everywhere is some red herring message about ansible* [ INFO ] TASK [ovirt.hosted_engine_setup : Wait for the host to be up] [ ERROR ] fatal: [localhost]: FAILED! => {"ansible_facts": {"ovirt_hosts": []}, "attempts": 120, "changed": false, "deprecations": [{"msg": "The 'ovirt_host_facts' module has been renamed to 'ovirt_host_info', and the renamed one no longer returns ansible_facts", "version": "2.13"}]} [ INFO ] TASK [ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"} [ ERROR ] Failed to execute stage 'Closing up': Failed executing ansible-playbook [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed: please check the logs for the issue, fix accordingly or re-deploy from scratch. Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200126170315-req4qb.log *But the "real" problem seems to be SSH related, as you can see below* *[root@bric-ovirt-1 ovirt-engine]# pwd* /var/log/ovirt-hosted-engine-setup/engine-logs-2020-01-26T17:19:28Z/ovirt-engine *[root@bric-ovirt-1 ovirt-engine]# grep -i error engine.log* 2020-01-26 17:26:50,178Z ERROR [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1) [2341fd23-f0c7-4f1c-ad48-88af20c2d04b] Failed to establish session with host 'bric-ovirt-1.corp.modmed.com': SSH session closed during connection ' root(a)bric-ovirt-1.corp.modmed.com&#39; 2020-01-26 17:26:50,205Z ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-1) [] Operation Failed: [Cannot add Host. Connecting to host via SSH has failed, verify that the host is reachable (IP address, routable address etc.) You may refer to the engine.log file for further details.] *The funny thing is that the engine can indeed ssh to bric-ovirt-1 (physical host). See below* *[root@bric-ovirt-1 ovirt-hosted-engine-setup]# cat /etc/hosts* 192.168.1.52 bric-ovirt-engine.corp.modmed.com # temporary entry added by hosted-engine-setup for the bootstrap VM 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 #::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 10.130.0.50 bric-ovirt-engine bric-ovirt-engine.corp.modmed.com 10.130.0.51 bric-ovirt-1 bric-ovirt-1.corp.modmed.com 10.130.0.52 bric-ovirt-2 bric-ovirt-2.corp.modmed.com 10.130.0.53 bric-ovirt-3 bric-ovirt-3.corp.modmed.com 192.168.0.1 bric-ovirt-1gluster bric-ovirt-1gluster.corp.modmed.com 192.168.0.2 bric-ovirt-2gluster bric-ovirt-2gluster.corp.modmed.com 192.168.0.3 bric-ovirt-3gluster bric-ovirt-3gluster.corp.modmed.com [root@bric-ovirt-1 ovirt-hosted-engine-setup]# *[root@bric-ovirt-1 ~]# ssh 192.168.1.52* Last login: Sun Jan 26 17:55:20 2020 from 192.168.1.1 [root@bric-ovirt-engine ~]# [root@bric-ovirt-engine ~]# *[root@bric-ovirt-engine ~]# ssh bric-ovirt-1* Password: Password: Last failed login: Sun Jan 26 18:17:16 UTC 2020 from 192.168.1.52 on ssh:notty There was 1 failed login attempt since the last successful login. Last login: Sun Jan 26 18:16:46 2020 ################################################################### # UNAUTHORIZED ACCESS TO THIS SYSTEM IS PROHIBITED # # # # This system is the property of Modernizing Medicine, Inc. # # It is for authorized Company business purposes only. # # All connections are monitored and recorded. # # Disconnect IMMEDIATELY if you are not an authorized user! # ################################################################### [root@bric-ovirt-1 ~]# [root@bric-ovirt-1 ~]# [root@bric-ovirt-1 ~]# exit logout Connection to bric-ovirt-1 closed. [root@bric-ovirt-engine ~]# [root@bric-ovirt-engine ~]# *[root@bric-ovirt-engine ~]# ssh bric-ovirt-1.corp.modmed.com <http://bric-ovirt-1.corp.modmed.com>* Password: Last login: Sun Jan 26 18:17:22 2020 from 192.168.1.52 ################################################################### # UNAUTHORIZED ACCESS TO THIS SYSTEM IS PROHIBITED # # # # This system is the property of Modernizing Medicine, Inc. # # It is for authorized Company business purposes only. # # All connections are monitored and recorded. # # Disconnect IMMEDIATELY if you are not an authorized user! # ################################################################### [root@bric-ovirt-1 ~]# exit logout Connection to bric-ovirt-1.corp.modmed.com closed. [root@bric-ovirt-engine ~]# [root@bric-ovirt-engine ~]# [root@bric-ovirt-engine ~]# exit logout Connection to 192.168.1.52 closed. [root@bric-ovirt-1 ~]# *So, what gives? I already disabled all ssh security in the physical host, and whitelisted all potential IPs from the engine using firewalld. Regardless, the engine can ssh to the host as root :-(. Is there maybe another user that's used for the "Wait for the host to be up" SSH test? Yes, I tried both passwords and certificates.* *Maybe what's really happening is that engine is not getting the right IP? bric-ovirt-engine is supposed to get 10.130.0.50, instead it never gets there, getting 192.168.1.52 from virbr0 in bric-ovirt-1. See below.* --== HOST NETWORK CONFIGURATION ==-- Please indicate the gateway IP address [10.130.0.1] Please indicate a nic to set ovirtmgmt bridge on: (p4p1, p5p1) [p4p1]: --== VM CONFIGURATION ==-- You may specify a unicast MAC address for the VM or accept a randomly generated default [00:16:3e:17:1d:f8]: How should the engine VM network be configured (DHCP, Static)[DHCP]? static Please enter the IP address to be used for the engine VM []: 10.130.0.50 [ INFO ] The engine VM will be configured to use 10.130.0.50/25 Please provide a comma-separated list (max 3) of IP addresses of domain name servers for the engine VM Engine VM DNS (leave it empty to skip) [10.130.0.2,10.130.0.3]: Add lines for the appliance itself and for this host to /etc/hosts on the engine VM? Note: ensuring that this host could resolve the engine VM hostname is still up to you (Yes, No)[No] Yes *[root@bric-ovirt-1 ~]# ip addr* 3: p4p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0a:f7:f1:c6:80 brd ff:ff:ff:ff:ff:ff inet 10.130.0.51/25 brd 10.130.0.127 scope global noprefixroute p4p1 valid_lft forever preferred_lft forever 28: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 52:54:00:25:7b:6f brd ff:ff:ff:ff:ff:ff inet 192.168.1.1/24 brd 192.168.1.255 scope global virbr0 valid_lft forever preferred_lft forever 29: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000 link/ether 52:54:00:25:7b:6f brd ff:ff:ff:ff:ff:ff 30: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UNKNOWN group default qlen 1000 link/ether fe:16:3e:17:1d:f8 brd ff:ff:ff:ff:ff:ff *The newly created engine VM does remain up even after hosted-engine --deploy errors out; just at the wrong IP. I haven't been able to make it get its real IP. At any rate, thank you very much for taking a look at my very long email. Any and all help would be really appreciated.* Cheers, -- Fredy -- *CONFIDENTIALITY NOTICE:* This e-mail message may contain material protected by the Health Insurance Portability and Accountability Act of 1996 and its implementing regulations and other state and federal laws and legal privileges. This message is only for the personal and confidential use of the individuals or organization to whom the message is addressed. If you are an unintended recipient, you have received this message in error, and any reading, distributing, copying or disclosure is unauthorized and strictly prohibited.  All recipients are hereby notified that any unauthorized receipt does not waive any confidentiality obligations or privileges. If you have received this message in error, please notify the sender immediately at the above email address and confirm that you have deleted or destroyed the message.