[ovirt-users] Failed to start self hosted engine after upgrading oVirt to 4.0

Wed Jun 29 07:15:47 UTC 2016

HI to All!!

Il 28/06/2016 17.24, Stefano Danzi ha scritto:
> [CUT]
> We have two issues here. First is that
>> https://gerrit.ovirt.org/gitweb?p=ovirt-hosted-engine-ha.git;a=blob;f=ovirt_hosted_engine_ha/lib/storage_backends.py;h=f2fbdc43d0e4afd7539a3a1de75de0cb07bdca9d;hb=HEAD#l271 
>>
>> is still using vdscli to contact vdsm, instead of the preferred
>> jsonrpccli.
>>
>> The second is that vdscli.connect's heuristic ends up reading the local
>> server address from vdsm config, where it finds the default ipv6-local
>> address of "::".
>>
>> Please try setting
>>
>> [addresses]
>> management_ip='0.0.0.0'
>>
>> in your /etc/vdsm/vdsm.conf instead of the crontab hacks.
>
> this solve the issue, but I still to haven't default gateway on 
> ovirtmgmt inteface.
>

Using this configuration all work, but every 50 minutes I receive 3 
email from broker:

- ovirt-hosted-engine state transition StartState-ReinitializeFSM
- ovirt-hosted-engine state transition ReinitializeFSM-EngineStarting
- ovirt-hosted-engine state transition EngineStarting-EngineUp

the uptime of engine is about 15 hours, so really the engine VM is not 
rebooted.
Agent log:

MainThread::INFO::2016-06-29 
08:46:10,453::hosted_engine::461::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) 
Current sta
te ReinitializeFSM (score: 0)
MainThread::INFO::2016-06-29 
08:46:20,547::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) 
Trying: notify time=1467182780.
55 type=state_transition detail=ReinitializeFSM-EngineStarting 
hostname='ovirt01.hawai.lan'
MainThread::INFO::2016-06-29 
08:46:20,732::brokerlink::120::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) 
Success, was notification of st
ate_transition (ReinitializeFSM-EngineStarting) sent? sent
MainThread::INFO::2016-06-29 
08:46:20,733::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) 
Initializin
g VDSM
MainThread::INFO::2016-06-29 
08:46:24,430::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
C
onnecting the storage
MainThread::INFO::2016-06-29 
08:46:24,431::storage_server::218::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Conn
ecting storage server
MainThread::INFO::2016-06-29 
08:46:31,764::storage_server::225::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Conn
ecting storage server
MainThread::INFO::2016-06-29 
08:46:31,780::storage_server::232::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Refr
eshing the storage domain
MainThread::INFO::2016-06-29 
08:46:31,945::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
P
reparing images
MainThread::INFO::2016-06-29 
08:46:31,946::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) 
Preparing images
MainThread::INFO::2016-06-29 
08:46:35,895::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
R
eloading vm.conf from the shared storage domain
MainThread::INFO::2016-06-29 
08:46:35,896::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::INFO::2016-06-29 
08:46:39,621::ovf_store::102::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) 
Found OVF_STORE: imgUUID:8d07965c-a5c4-4057-912d-901f80cf246c, 
volUUID:ce3aa63e-e1c4-498e-bdca-9d2e9f47f0f9
MainThread::INFO::2016-06-29 
08:46:39,667::ovf_store::102::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) 
Found OVF_STORE: imgUUID:bd9aaf0b-8435-4d78-9871-8c7a7f7fa02a, 
volUUID:3c477b06-063e-4f01-bd05-84c7d467742b
MainThread::INFO::2016-06-29 
08:46:39,760::ovf_store::111::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) 
Extracting Engine VM OVF from the OVF_STORE
MainThread::INFO::2016-06-29 
08:46:39,761::ovf_store::118::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) 
OVF_STORE volume path: 
/rhev/data-center/mnt/ovirt01.hawai.lan:_engine/46f55a31-f35f-465c-b3e2-df45c05e06a7/images/bd9aaf0b-8435-4d78-9871-8c7a7f7fa02a/3c477b06-063e-4f01-bd05-84c7d467742b
MainThread::INFO::2016-06-29 
08:46:39,772::config::226::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Found an OVF for HE VM, trying to convert
MainThread::INFO::2016-06-29 
08:46:39,774::config::231::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Got vm.conf from OVF_STORE
MainThread::INFO::2016-06-29 
08:46:43,489::hosted_engine::461::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) 
Current state EngineStarting (score: 3400)
MainThread::INFO::2016-06-29 
08:46:53,605::state_decorators::88::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) 
Timeout cleared while transitioning <class 
'ovirt_hosted_engine_ha.agent.states.EngineStarting'> -> <class 
'ovirt_hosted_engine_ha.agent.states.EngineUp'>
MainThread::INFO::2016-06-29 
08:46:53,606::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) 
Trying: notify time=1467182813.

Many times at day I have the log:

MainThread::INFO::2016-06-29 
08:47:37,833::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) 
Preparing images
MainThread::INFO::2016-06-29 
08:47:41,721::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Reloading vm.conf from the shared storage domain
MainThread::INFO::2016-06-29 
08:47:41,722::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::INFO::2016-06-29 
08:47:45,444::ovf_store::102::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) 
Found OVF_STORE: imgUUID:8d07965c-a5c4-4057-912d-901f80cf246c, 
volUUID:ce3aa63e-e1c4-498e-bdca-9d2e9f47f0f9
MainThread::INFO::2016-06-29 
08:47:45,490::ovf_store::102::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) 
Found OVF_STORE: imgUUID:bd9aaf0b-8435-4d78-9871-8c7a7f7fa02a, 
volUUID:3c477b06-063e-4f01-bd05-84c7d467742b
MainThread::INFO::2016-06-29 
08:47:45,583::ovf_store::111::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) 
Extracting Engine VM OVF from the OVF_STORE
MainThread::INFO::2016-06-29 
08:47:45,583::ovf_store::118::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) 
OVF_STORE volume path: 
/rhev/data-center/mnt/ovirt01.hawai.lan:_engine/46f55a31-f35f-465c-b3e2-df45c05e06a7/images/bd9aaf0b-8435-4d78-9871-8c7a7f7fa02a/3c477b06-063e-4f01-bd05-84c7d467742b
MainThread::INFO::2016-06-29 
08:47:45,597::config::226::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Found an OVF for HE VM, trying to convert
MainThread::INFO::2016-06-29 
08:47:45,599::config::231::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Got vm.conf from OVF_STORE
MainThread::INFO::2016-06-29 
08:47:49,291::hosted_engine::461::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) 
Current state EngineUp (score: 3400)