hosted-engine deploy error terminal state
Hello, I try to deploy a hosted-engine (4.0.2) on my cluster from a existing engine, Everything works fine until i start the last step, my engine is up, cluster state ok, hosts state up ect ... When it come to the last step "Continue setup - oVirt-Engine installation is ready and ovirt-engine service is up" It get stuck on "Connecting to the Engine" and finally crash with [ ERROR ] Failed to execute stage 'Closing up': [-32605] No response for JSON-RPC Host.getCapabilities request. [ INFO ] Stage: Clean up [ ERROR ] Failed to execute stage 'Clean up': [-32605] No response for JSON-RPC Host.stopMonitoringDomain request. [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine- setup/answers/answers-20160816155211.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue,fix and redeploy Log file is located at /var/log/ovirt-hosted-engine- setup/ovirt-hosted-engine-setup-20160816144914-xykpcl.log Exception in thread Client hypervisor1:54321: Traceback (most recent call last): File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner self.run() File "/usr/lib64/python2.7/threading.py", line 764, in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py", line 216, in process_requests count=1, File "/usr/lib64/python2.7/asyncore.py", line 220, in loop poll_fun(timeout, map) File "/usr/lib64/python2.7/asyncore.py", line 201, in poll2 readwrite(obj, flags) TypeError: 'NoneType' object is not callable hosted-engine-deploy and engine log are in attachements
On Tue, Aug 16, 2016 at 5:34 PM, Maxence Sartiaux <contact@makz.me> wrote:
Hello,
I try to deploy a hosted-engine (4.0.2) on my cluster from a existing engine,
Please provide more details about what you are trying to do. It seems like you already have an engine. Can't tell if it's a hosted engine or not. Is it? It seems like the host you are trying to deploy is/was already managed by your engine. Is it? Generally speaking, we do not allow "converting" an engine to a hosted engine nor doing 'hosted-engine --deploy' on a host already managed by the/an engine. For the former, please check [1]. For the latter, you should first remove the host from the engine. [1] is very old, and some details are different today, but the principles remain. I recommend trying this first on an isolated test env before doing on production. BTW, the above does not explain your failure. Your specific failure is: 2016-08-16 15:22:10 INFO otopi.plugins.gr_he_setup.engine.add_host engineapi.get_engine_api:51 Connecting to the Engine 2016-08-16 15:22:10 DEBUG otopi.plugins.gr_he_setup.engine.add_host add_host._closeup:513 Getting the list of available clusters via engine's APIs 2016-08-16 15:37:10 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-setup/engine/add_host.py", line 554, in _closeup net_info = CachingNetInfo(vds_info.capabilities(conn)) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_setup/vds_info.py", line 31, in capabilities result = conn.getVdsCapabilities() File "/usr/lib/python2.7/site-packages/vdsm/jsonrpcvdscli.py", line 153, in _callMethod raise JsonRpcNoResponseError(method) JsonRpcNoResponseError: [-32605] No response for JSON-RPC Host.getCapabilities request. To me it seems like 'hosted-engine --deploy' managed to connect to the engine (not sure which, existing or one created in new hosted-engine vm) but fails to get the list of clusters, probably because the engine fails to connect to one or more hosts. Can't see that in the attached engine.log btw - either it's from the wrong machine, or wrong time/timezone, something like that. [1] http://www.ovirt.org/develop/developer-guide/engine/migrate-to-hosted-engine... Best,
Everything works fine until i start the last step, my engine is up, cluster state ok, hosts state up ect ...
When it come to the last step "Continue setup - oVirt-Engine installation is ready and ovirt-engine service is up"
It get stuck on "Connecting to the Engine" and finally crash with
[ ERROR ] Failed to execute stage 'Closing up': [-32605] No response for JSON-RPC Host.getCapabilities request. [ INFO ] Stage: Clean up [ ERROR ] Failed to execute stage 'Clean up': [-32605] No response for JSON-RPC Host.stopMonitoringDomain request. [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine- setup/answers/answers-20160816155211.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue,fix and redeploy Log file is located at /var/log/ovirt-hosted-engine- setup/ovirt-hosted-engine-setup-20160816144914-xykpcl.log Exception in thread Client hypervisor1:54321: Traceback (most recent call last): File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner self.run() File "/usr/lib64/python2.7/threading.py", line 764, in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py", line 216, in process_requests count=1, File "/usr/lib64/python2.7/asyncore.py", line 220, in loop poll_fun(timeout, map) File "/usr/lib64/python2.7/asyncore.py", line 201, in poll2 readwrite(obj, flags) TypeError: 'NoneType' object is not callable
hosted-engine-deploy and engine log are in attachements _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Didi
- Yes i try to migrate a existing engine 4.0.2 (vm in a proxmox cluster) to a hosted engine - I try to deploy on a host who is already managed by the engine (i only have two host on this cluster) - my hosts are uptodate - When i start the last step my cluster is ok, all hosts are up. - I ran the deploy multiple times, sometime it add the hosted engine but fail later other time it fail at the "connecting engine" - Even with correct timezone, same problem. New logs in attachments That's how i manage to fail the deploy of the hosted engine : - My old engine is down and i have my backup file - hosted-engine --deploy (on hypervisor1) - deploy on glusterfs (on another storage cluster) - create vm from the engine appliance ova - do not run the engine-setup at firstboot - when i need to setup the engine on the vm, i restore my backup (engine-backup --mode=restore --file=migrationfile --restore- permissions --provision-db --provision-dwh-db --provision-reports-db -- log=restore.log) - engine-setup (don't install ioproxy or it get stuck later,as i saw you are assigned to this bug https://bugzilla.redhat.com/show_bug.cgi?i d=1365451, is it fixed on the "stable" branch ?) - in the engine, i reinstall the hypervisor1 if i don't do that it never go UP - back in hypervisor1 engine deploy screen, i continue the deploy process and it get stuck on connecting to engine [ INFO ] Connecting to the Engine [ ERROR ] Failed to execute stage 'Closing up': [-32605] No response for JSON-RPC Host.getCapabilities request. [ INFO ] Stage: Clean up [ ERROR ] Failed to execute stage 'Clean up': [-32605] No response for JSON-RPC Host.stopMonitoringDomain request. [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine- setup/answers/answers-20160823095524.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue,fix and redeploy Log file is located at /var/log/ovirt-hosted-engine- setup/ovirt-hosted-engine-setup-20160823084709-aqgt7r.log Exception in thread Client hypervisor1:54321: Traceback (most recent call last): File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner self.run() File "/usr/lib64/python2.7/threading.py", line 764, in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py", line 216, in process_requests count=1, File "/usr/lib64/python2.7/asyncore.py", line 220, in loop poll_fun(timeout, map) File "/usr/lib64/python2.7/asyncore.py", line 201, in poll2 readwrite(obj, flags) TypeError: 'NoneType' object is not callable On Wed, 2016-08-17 at 09:14 +0300, Yedidyah Bar David wrote:
On Tue, Aug 16, 2016 at 5:34 PM, Maxence Sartiaux <contact@makz.me> wrote:
Hello,
I try to deploy a hosted-engine (4.0.2) on my cluster from a existing engine,
Please provide more details about what you are trying to do.
It seems like you already have an engine. Can't tell if it's a hosted engine or not. Is it?
It seems like the host you are trying to deploy is/was already managed by your engine. Is it?
Generally speaking, we do not allow "converting" an engine to a hosted engine nor doing 'hosted-engine --deploy' on a host already managed by the/an engine. For the former, please check [1]. For the latter, you should first remove the host from the engine. [1] is very old, and some details are different today, but the principles remain. I recommend trying this first on an isolated test env before doing on production.
BTW, the above does not explain your failure. Your specific failure is:
2016-08-16 15:22:10 INFO otopi.plugins.gr_he_setup.engine.add_host engineapi.get_engine_api:51 Connecting to the Engine 2016-08-16 15:22:10 DEBUG otopi.plugins.gr_he_setup.engine.add_host add_host._closeup:513 Getting the list of available clusters via engine's APIs 2016-08-16 15:37:10 DEBUG otopi.context context._executeMethod:142 method exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr- he-setup/engine/add_host.py", line 554, in _closeup net_info = CachingNetInfo(vds_info.capabilities(conn)) File "/usr/lib/python2.7/site- packages/ovirt_hosted_engine_setup/vds_info.py", line 31, in capabilities result = conn.getVdsCapabilities() File "/usr/lib/python2.7/site-packages/vdsm/jsonrpcvdscli.py", line 153, in _callMethod raise JsonRpcNoResponseError(method) JsonRpcNoResponseError: [-32605] No response for JSON-RPC Host.getCapabilities request.
To me it seems like 'hosted-engine --deploy' managed to connect to the engine (not sure which, existing or one created in new hosted-engine vm) but fails to get the list of clusters, probably because the engine fails to connect to one or more hosts. Can't see that in the attached engine.log btw - either it's from the wrong machine, or wrong time/timezone, something like that.
[1] http://www.ovirt.org/develop/developer-guide/engine/migrate-to-ho sted-engine/
Best,
Everything works fine until i start the last step, my engine is up, cluster state ok, hosts state up ect ...
When it come to the last step "Continue setup - oVirt-Engine installation is ready and ovirt-engine service is up"
It get stuck on "Connecting to the Engine" and finally crash with
[ ERROR ] Failed to execute stage 'Closing up': [-32605] No response for JSON-RPC Host.getCapabilities request. [ INFO ] Stage: Clean up [ ERROR ] Failed to execute stage 'Clean up': [-32605] No response for JSON-RPC Host.stopMonitoringDomain request. [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine- setup/answers/answers-20160816155211.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue,fix and redeploy Log file is located at /var/log/ovirt-hosted-engine- setup/ovirt-hosted-engine-setup-20160816144914-xykpcl.log Exception in thread Client hypervisor1:54321: Traceback (most recent call last): File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner self.run() File "/usr/lib64/python2.7/threading.py", line 764, in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib/python2.7/site- packages/yajsonrpc/betterAsyncore.py", line 216, in process_requests count=1, File "/usr/lib64/python2.7/asyncore.py", line 220, in loop poll_fun(timeout, map) File "/usr/lib64/python2.7/asyncore.py", line 201, in poll2 readwrite(obj, flags) TypeError: 'NoneType' object is not callable
hosted-engine-deploy and engine log are in attachements _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (2)
-
Maxence Sartiaux -
Yedidyah Bar David