gdeploy hosted engine failed but host says it is up what now?

Hi, Ovirt 4.2.3 I gdeplyed deployed glusterfs which seems to have gone well and then on to the hosted engine... which seemed to be going well, but then the ovirt engine did not come online... [ INFO ] TASK [Wait for the engine to come up on the target VM] [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": true, "cmd": ["hosted-engine", "--vm-status", "--json"], "delta": "0:00:00.225200", "end": "2018-06-09 15:15:24.434531", "rc": 0, "start": "2018-06-09 15:15:24.209331", "stderr": "", "stderr_lines": [], "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=17346 (Sat Jun 9 15:15:19 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=17346 (Sat Jun 9 15:15:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", \"hostname\": \"usbou-rhev01.pbi.global.pvt\", \"host-id\": 1, \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": false, \"maintenance\": false, \"crc32\": \"e3c31ac3\", \"local_conf_timestamp\": 17346, \"host-ts\": 17346}, \"global_maintenance\": false}", "stdout_lines": ["{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=17346 (Sat Jun 9 15:15:19 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=17346 (Sat Jun 9 15:15:19 2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", \"hostname\": \"usbou-rhev01.pbi.global.pvt\", \"host-id\": 1, \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}, \"score\": 3400, \"stopped\": false, \"maintenance\": false, \"crc32\": \"e3c31ac3\", \"local_conf_timestamp\": 17346, \"host-ts\": 17346}, \"global_maintenance\": false}"]} When I look at the host now it says Hosted Engine is running on the host, Hosted engine is up, but I can not ping it, ssh to it or open the console on the VM. I am pretty new to this setup so would appreciate any steps I can take to move this forward. Thanks Bill

Bill, 3 things that have given me trouble in the past...ymmv! 1) Static IP for the hosted engine, multiple times, I've tried dhcp with no luck 2) Size of thick brick for hosted engine must be 60GB or more, I've tried 50gb mulitple times & it always fails. 3) Hosted engine & admin portal passwords, make them very simple, complex passwords with special characters ...always failed

Thanks for the input Femi... I used static for everything, my brick was 100GB and my password has no special chars. How do I get back to the point that I can try again though? I t seems to think my hosted engine is up on the hosted engine page, so it isn't giving me the opportunity to try to deploy it again. I'm probably missing something stupid, but can't see to figure this out. Thanks again, appreciate any advice. Bill -----Original Message----- From: femi adegoke <ovirt@fateknollogee.com> Sent: Sunday, June 10, 2018 1:07 AM To: users@ovirt.org Subject: [ovirt-users] Re: gdeploy hosted engine failed but host says it is up what now? Bill, 3 things that have given me trouble in the past...ymmv! 1) Static IP for the hosted engine, multiple times, I've tried dhcp with no luck 2) Size of thick brick for hosted engine must be 60GB or more, I've tried 50gb mulitple times & it always fails. 3) Hosted engine & admin portal passwords, make them very simple, complex passwords with special characters ...always failed _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KGE453SG5CEOJ2...

Not sure if it helps, but I just noticed I have about 300 Hosted engine host: usbou-rhev01 changed state: EngineStarting-EngineStop. Then Hosted engine host: usbou-rhev01 changed state: EngineStop-EngineDown. Then Hosted engine host: usbou-rhev01 changed state: EngineDown-EngineStart. Then Hosted engine host: usbou-rhev01 changed state: EngineStart-EngineStarting. And then it starts over... so there is obviously something wrong with the VM... Under the virtualization page, It says I have 1 running virtual machine but I can't see it to delete it or get any visibility into it. Thanks Bill -----Original Message----- From: femi adegoke <ovirt@fateknollogee.com> Sent: Sunday, June 10, 2018 1:07 AM To: users@ovirt.org Subject: [ovirt-users] Re: gdeploy hosted engine failed but host says it is up what now? Bill, 3 things that have given me trouble in the past...ymmv! 1) Static IP for the hosted engine, multiple times, I've tried dhcp with no luck 2) Size of thick brick for hosted engine must be 60GB or more, I've tried 50gb mulitple times & it always fails. 3) Hosted engine & admin portal passwords, make them very simple, complex passwords with special characters ...always failed _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KGE453SG5CEOJ2...

Hi, \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": \"Up\"} means that the engine VM is up at virt side although a liveliness check over http is failing. This could happen if your engine VM got a different network configuration or something like that. I'd suggest to connect to the engine VM over vnc and check the network status there. ovirt-ha-agent will try to keep the engine up so if the http health liveliness check is failing, ovirt-ha-agent will try to restart the engine VM. On Sun, Jun 10, 2018 at 2:05 PM, <william.dossett@gmail.com> wrote:
Not sure if it helps, but I just noticed I have about 300
Hosted engine host: usbou-rhev01 changed state: EngineStarting-EngineStop.
Then
Hosted engine host: usbou-rhev01 changed state: EngineStop-EngineDown.
Then
Hosted engine host: usbou-rhev01 changed state: EngineDown-EngineStart.
Then
Hosted engine host: usbou-rhev01 changed state: EngineStart-EngineStarting.
And then it starts over... so there is obviously something wrong with the VM...
Under the virtualization page, It says I have 1 running virtual machine but I can't see it to delete it or get any visibility into it.
Thanks Bill
-----Original Message----- From: femi adegoke <ovirt@fateknollogee.com> Sent: Sunday, June 10, 2018 1:07 AM To: users@ovirt.org Subject: [ovirt-users] Re: gdeploy hosted engine failed but host says it is up what now?
Bill,
3 things that have given me trouble in the past...ymmv!
1) Static IP for the hosted engine, multiple times, I've tried dhcp with no luck 2) Size of thick brick for hosted engine must be 60GB or more, I've tried 50gb mulitple times & it always fails. 3) Hosted engine & admin portal passwords, make them very simple, complex passwords with special characters ...always failed _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community- guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/ message/KGE453SG5CEOJ2IQFQGN3JEU52UZXSD3/ _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community- guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/ message/D7XP52YGAD3CZ4AC2VT6ON2DFP4YGNDV/

Ok, yes I am starting to think this is a network issue. I have two networks. When I setup hosted engine and gluster I used the private network 172.17.70.x as it was setting up gluster and I assumed that I should use the private network for this. Its been a while and I can’t remember exactly what steps I took setting up the management instance, but I did give it a static address on our normal management VLAN. but now wondering if it is on that private network. Its been a while since I worked with ovirt and rhev, but, I normally could see a VM. I cannot see this VM at all anywhere of I am looking in the wrong place, but I have tried pretty much everything. I don’t know how I would VNC into it as I don’t see the VM or anyway to access the console which I would guess is what I need to do, or access its settings to verify the networking. Thanks for the advice, but I seem to be missing something basic here. Bill From: Simone Tiraboschi <stirabos@redhat.com> Sent: Monday, June 11, 2018 2:18 AM To: william.dossett@gmail.com Cc: femi adegoke <ovirt@fateknollogee.com>; users <users@ovirt.org> Subject: Re: [ovirt-users] Re: gdeploy hosted engine failed but host says it is up what now? Hi, \"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": \"Up\"} means that the engine VM is up at virt side although a liveliness check over http is failing. This could happen if your engine VM got a different network configuration or something like that. I'd suggest to connect to the engine VM over vnc and check the network status there. ovirt-ha-agent will try to keep the engine up so if the http health liveliness check is failing, ovirt-ha-agent will try to restart the engine VM. On Sun, Jun 10, 2018 at 2:05 PM, <william.dossett@gmail.com <mailto:william.dossett@gmail.com> > wrote: Not sure if it helps, but I just noticed I have about 300 Hosted engine host: usbou-rhev01 changed state: EngineStarting-EngineStop. Then Hosted engine host: usbou-rhev01 changed state: EngineStop-EngineDown. Then Hosted engine host: usbou-rhev01 changed state: EngineDown-EngineStart. Then Hosted engine host: usbou-rhev01 changed state: EngineStart-EngineStarting. And then it starts over... so there is obviously something wrong with the VM... Under the virtualization page, It says I have 1 running virtual machine but I can't see it to delete it or get any visibility into it. Thanks Bill -----Original Message----- From: femi adegoke <ovirt@fateknollogee.com> Sent: Sunday, June 10, 2018 1:07 AM To: users@ovirt.org <mailto:users@ovirt.org> Subject: [ovirt-users] Re: gdeploy hosted engine failed but host says it is up what now? Bill, 3 things that have given me trouble in the past...ymmv! 1) Static IP for the hosted engine, multiple times, I've tried dhcp with no luck 2) Size of thick brick for hosted engine must be 60GB or more, I've tried 50gb mulitple times & it always fails. 3) Hosted engine & admin portal passwords, make them very simple, complex passwords with special characters ...always failed _______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KGE453SG5CEOJ2... _______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/D7XP52YGAD3CZ4...

On Mon, Jun 11, 2018 at 2:37 PM, <william.dossett@gmail.com> wrote:
Ok, yes I am starting to think this is a network issue. I have two networks. When I setup hosted engine and gluster I used the private network 172.17.70.x as it was setting up gluster and I assumed that I should use the private network for this. Its been a while and I can’t remember exactly what steps I took setting up the management instance, but I did give it a static address on our normal management VLAN. but now wondering if it is on that private network.
Its been a while since I worked with ovirt and rhev, but, I normally could see a VM. I cannot see this VM at all anywhere of I am looking in the wrong place, but I have tried pretty much everything. I don’t know how I would VNC into it as I don’t see the VM or anyway to access the console which I would guess is what I need to do, or access its settings to verify the networking.
Thanks for the advice, but I seem to be missing something basic here.
Run on your first host hosted-engine --add-console-password to set a temporary VNC password and then connect to it over VNC with something like remote-viewer vnc://<host>:<port>
Bill
*From:* Simone Tiraboschi <stirabos@redhat.com> *Sent:* Monday, June 11, 2018 2:18 AM *To:* william.dossett@gmail.com *Cc:* femi adegoke <ovirt@fateknollogee.com>; users <users@ovirt.org> *Subject:* Re: [ovirt-users] Re: gdeploy hosted engine failed but host says it is up what now?
Hi,
\"engine-status\": {\"reason\": \"failed liveliness check\", \"health\": \"bad\", \"vm\": \"up\", \"detail\": \"Up\"}
means that the engine VM is up at virt side although a liveliness check over http is failing.
This could happen if your engine VM got a different network configuration or something like that.
I'd suggest to connect to the engine VM over vnc and check the network status there.
ovirt-ha-agent will try to keep the engine up so if the http health liveliness check is failing, ovirt-ha-agent will try to restart the engine VM.
On Sun, Jun 10, 2018 at 2:05 PM, <william.dossett@gmail.com> wrote:
Not sure if it helps, but I just noticed I have about 300
Hosted engine host: usbou-rhev01 changed state: EngineStarting-EngineStop.
Then
Hosted engine host: usbou-rhev01 changed state: EngineStop-EngineDown.
Then
Hosted engine host: usbou-rhev01 changed state: EngineDown-EngineStart.
Then
Hosted engine host: usbou-rhev01 changed state: EngineStart-EngineStarting.
And then it starts over... so there is obviously something wrong with the VM...
Under the virtualization page, It says I have 1 running virtual machine but I can't see it to delete it or get any visibility into it.
Thanks Bill
-----Original Message----- From: femi adegoke <ovirt@fateknollogee.com> Sent: Sunday, June 10, 2018 1:07 AM To: users@ovirt.org Subject: [ovirt-users] Re: gdeploy hosted engine failed but host says it is up what now?
Bill,
3 things that have given me trouble in the past...ymmv!
1) Static IP for the hosted engine, multiple times, I've tried dhcp with no luck 2) Size of thick brick for hosted engine must be 60GB or more, I've tried 50gb mulitple times & it always fails. 3) Hosted engine & admin portal passwords, make them very simple, complex passwords with special characters ...always failed _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community- guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/ message/KGE453SG5CEOJ2IQFQGN3JEU52UZXSD3/ _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community- guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/ message/D7XP52YGAD3CZ4AC2VT6ON2DFP4YGNDV/
participants (3)
-
femi adegoke
-
Simone Tiraboschi
-
william.dossett@gmail.com