[ovirt-users] hosted engine won't come up

Mon Apr 25 22:33:14 UTC 2016

I've just done a clean install of the 3.6 hosted engine (decided to wipe 
out my previous system)

The install went in just fine, no errors I saw, but I'm getting 
interesting errors in the ovirt-hosted-engine-ha agent.log

I have no idea what to do about these errors....

-----
MainThread::INFO::2016-04-25 
22:28:33,842::hosted_engine::462::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) 
Current state EngineUp (score: 3400)
MainThread::INFO::2016-04-25 
22:28:43,896::hosted_engine::613::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) 
Initializing VDSM
MainThread::INFO::2016-04-25 
22:28:43,953::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Connecting the storage
MainThread::INFO::2016-04-25 
22:28:43,954::storage_server::207::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Connecting storage server
MainThread::INFO::2016-04-25 
22:28:43,954::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Refreshing the storage domain
MainThread::INFO::2016-04-25 
22:28:44,064::hosted_engine::685::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Preparing images
MainThread::INFO::2016-04-25 
22:28:44,064::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) 
Preparing images
MainThread::INFO::2016-04-25 
22:28:45,070::hosted_engine::688::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Reloading vm.conf from the shared storage domain
MainThread::INFO::2016-04-25 
22:28:45,071::config::205::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::WARNING::2016-04-25 
22:28:45,658::ovf_store::104::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) 
Unable to find OVF_STORE
MainThread::ERROR::2016-04-25 
22:28:45,658::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::INFO::2016-04-25 
22:28:45,744::hosted_engine::462::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) 
Current state EngineUp (score: 3400)
MainThread::INFO::2016-04-25 
22:28:55,796::hosted_engine::613::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) 
Initializing VDSM
MainThread::INFO::2016-04-25 
22:28:55,855::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Connecting the storage
MainThread::INFO::2016-04-25 
22:28:55,856::storage_server::207::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Connecting storage server
MainThread::INFO::2016-04-25 
22:28:55,856::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Refreshing the storage domain
MainThread::INFO::2016-04-25 
22:28:55,967::hosted_engine::685::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Preparing images
MainThread::INFO::2016-04-25 
22:28:55,967::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) 
Preparing images
MainThread::INFO::2016-04-25 
22:28:56,987::hosted_engine::688::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Reloading vm.conf from the shared storage domain
MainThread::INFO::2016-04-25 
22:28:56,988::config::205::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::WARNING::2016-04-25 
22:28:57,572::ovf_store::104::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) 
Unable to find OVF_STORE
MainThread::ERROR::2016-04-25 
22:28:57,573::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::INFO::2016-04-25 
22:28:57,661::hosted_engine::462::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) 
Current state EngineUp (score: 3400)
MainThread::INFO::2016-04-25 
22:29:07,711::states::421::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) 
Engine vm running on localhost
MainThread::INFO::2016-04-25 
22:29:07,716::hosted_engine::613::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) 
Initializing VDSM
MainThread::INFO::2016-04-25 
22:29:07,774::hosted_engine::658::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Connecting the storage
MainThread::INFO::2016-04-25 
22:29:07,774::storage_server::207::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Connecting storage server
MainThread::INFO::2016-04-25 
22:29:07,775::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) 
Refreshing the storage domain
MainThread::INFO::2016-04-25 
22:29:07,883::hosted_engine::685::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Preparing images
MainThread::INFO::2016-04-25 
22:29:07,883::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) 
Preparing images
MainThread::INFO::2016-04-25 
22:29:08,893::hosted_engine::688::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) 
Reloading vm.conf from the shared storage domain
MainThread::INFO::2016-04-25 
22:29:08,894::config::205::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::WARNING::2016-04-25 
22:29:09,486::ovf_store::104::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) 
Unable to find OVF_STORE
MainThread::ERROR::2016-04-25 
22:29:09,486::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) 
Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
-----

# vdsClient -s 0 getStorageDomainsList
81f19871-4d91-4698-a97d-36452bfae281
# vgdisplay
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdh
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdh
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdh
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdg
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdg
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdg
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdd
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdd
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdd
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdc
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdc
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdc
   --- Volume group ---
   VG Name               81f19871-4d91-4698-a97d-36452bfae281
   System ID
   Format                lvm2
   Metadata Areas        2
   Metadata Sequence No  23
   VG Access             read/write
   VG Status             resizable
   MAX LV                0
   Cur LV                10
   Open LV               3
   Max PV                0
   Cur PV                1
   Act PV                1
   VG Size               60.00 GiB
   PE Size               128.00 MiB
   Total PE              480
   Alloc PE / Size       441 / 55.12 GiB
   Free  PE / Size       39 / 4.88 GiB
   VG UUID               3M7kuh-FjyR-WMPL-ztl0-h1cK-EwQb-32ASCD
# lvs
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdh
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdh
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdh
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdg
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdg
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdg
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdd
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdd
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdd
   WARNING: duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy is being used 
from both devices /dev/mapper/36000402001f45bbe7eecc5f800000000 and /dev/sdc
   Found duplicate PV xpHUtx05gHRcq1U33Mu1NMGLXVOtqnmy: using 
/dev/mapper/36000402001f45bbe7eecc5f800000000 not /dev/sdc
   Using duplicate PV /dev/mapper/36000402001f45bbe7eecc5f800000000 from 
subsystem DM, ignoring /dev/sdc
   LV VG                                   Attr       LSize   Pool 
Origin Data%  Meta%  Move Log Cpy%Sync Convert
   0593a3b8-1d75-4be3-b65b-ce9a164d0309 
81f19871-4d91-4698-a97d-36452bfae281 -wi-ao---- 128.00m
   07c72aa5-7fd0-4159-b83e-4d078ae9c351 
81f19871-4d91-4698-a97d-36452bfae281 -wi-a----- 1.00g
   93cf12c5-d5e6-4ea6-bee7-de64ee52d7a5 
81f19871-4d91-4698-a97d-36452bfae281 -wi-a----- 128.00m
   c4244bdd-80c5-4f68-83c2-9494d9d05723 
81f19871-4d91-4698-a97d-36452bfae281 -wi-ao---- 50.00g
   ids 81f19871-4d91-4698-a97d-36452bfae281 -wi-ao---- 128.00m
   inbox 81f19871-4d91-4698-a97d-36452bfae281 -wi-a----- 128.00m
   leases 81f19871-4d91-4698-a97d-36452bfae281 -wi-a----- 2.00g
   master 81f19871-4d91-4698-a97d-36452bfae281 -wi-a----- 1.00g
   metadata 81f19871-4d91-4698-a97d-36452bfae281 -wi-a----- 512.00m
   outbox 81f19871-4d91-4698-a97d-36452bfae281 -wi-a----- 128.00m
# rpm -qa |grep ovirt
ovirt-engine-sdk-python-3.6.3.0-1.el7
ovirt-vmconsole-1.0.0-1.el7
ovirt-vmconsole-host-1.0.0-1.el7
libgovirt-0.3.3-1.el7_2.1
ovirt-setup-lib-1.0.1-1.el7
ovirt-hosted-engine-setup-1.3.4.0-1.el7
ovirt-host-deploy-1.4.1-1.el7
ovirt-hosted-engine-ha-1.3.5.1-1.el7
# rpm -qa |grep vdsm
vdsm-jsonrpc-4.17.23.2-0.el7
vdsm-4.17.23.2-0.el7
vdsm-python-4.17.23.2-0.el7
vdsm-xmlrpc-4.17.23.2-0.el7
vdsm-yajsonrpc-4.17.23.2-0.el7
vdsm-hook-vmfex-dev-4.17.23.2-0.el7
vdsm-cli-4.17.23.2-0.el7
vdsm-infra-4.17.23.2-0.el7

-- 
Pat Riehecky
Scientific Linux developer

Fermi National Accelerator Laboratory
www.fnal.gov
www.scientificlinux.org