[ovirt-users] ovirt-hosted-engine state transition messages

Tue Jul 18 16:22:19 UTC 2017

I had some of this going on recently under 4.1.2, started with one or two warning messages, then a flood of them. Did the upgrade to 4.1.3 and haven’t seen it yet, but it’s only been a few days so far. A java process was consuming much CPU, and the DataWarehouse appears to not be collecting data (evidenced by a blank dashboard). My DWH has since recovered as well.

I forgot to check, but suspect I was low/out of memory on my engine VM, it’s an old one with only 6G allocated currently. Watching for this to happen again, and will confirm RAM utilization and bump up appropriately if it looks like it’s starved for RAM.

> On Jul 18, 2017, at 5:45 AM, Christophe TREFOIS <christophe.trefois at uni.lu> wrote:
> 
> I have the same as you on 4.1.0
> 
> EngineBadHealth-EngineUp 1 minute later. Sometimes 20 times per day, mostly on weekends.
> 
> Cheers,
> -- 
> 
> Dr Christophe Trefois, Dipl.-Ing.  
> Technical Specialist / Post-Doc
> 
> UNIVERSITÉ DU LUXEMBOURG
> 
> LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE
> Campus Belval | House of Biomedicine  
> 6, avenue du Swing 
> L-4367 Belvaux  
> T: +352 46 66 44 6124 
> F: +352 46 66 44 6949  
> http://www.uni.lu/lcsb <http://www.uni.lu/lcsb>
>  <https://www.facebook.com/trefex>   <https://twitter.com/Trefex>   <https://plus.google.com/+ChristopheTrefois/>   <https://www.linkedin.com/in/trefoischristophe>   <http://skype:Trefex?call>
> 
> ----
> This message is confidential and may contain privileged information. 
> It is intended for the named recipient only. 
> If you receive it in error please notify me and permanently delete the original message and any copies. 
> ----
>   
> 
>> On 17 Jul 2017, at 17:35, Jim Kusznir <jim at palousetech.com <mailto:jim at palousetech.com>> wrote:
>> 
>> Ok, I've been ignoring this for a long time as the logs were so verbose and didn't show anything I could identify as usable debug info.  Recently one of my ovirt hosts (currently NOT running the main engine, but a candidate) was cycling as much as 40 times a day between "EngineUpBadHealth and EngineUp".  Here's the log snippit.  I included some time before and after if that's helpful.  In this case, I got an email about bad health at 8:15 and a restore (engine up) at 8:16.  I see where the messages are sent, but I don't see any explanation as to why / what the problem is.
>> 
>> BTW: 192.168.8.11 is this computer's physical IP; 192.168.8.12 is the computer currently running the engine.  Both are also hosting the gluster store (eg, I have 3 hosts, all are participating in the gluster replica 2+arbitrator).
>> 
>> I'd appreciate it if someone could shed some light on why this keeps happening!
>> 
>> --Jim
>> ----------------
>> 
>> MainThread::INFO::2017-07-17 08:12:06,230::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:12:06,230::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:08,877::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:12:09,432::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:12:09,925::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:10,324::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:12:10,696::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:12:10,704::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:10,705::states::426::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
>> MainThread::INFO::2017-07-17 08:12:10,714::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM
>> MainThread::INFO::2017-07-17 08:12:14,426::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage
>> MainThread::INFO::2017-07-17 08:12:14,470::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:12:19,648::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:12:19,900::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain
>> MainThread::INFO::2017-07-17 08:12:20,298::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images
>> MainThread::INFO::2017-07-17 08:12:20,298::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images
>> MainThread::INFO::2017-07-17 08:12:24,051::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf
>> MainThread::INFO::2017-07-17 08:12:24,051::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:12:24,052::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:26,895::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:12:27,429::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:12:27,976::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:28,240::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:12:28,564::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:12:28,572::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:34,574::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
>> MainThread::INFO::2017-07-17 08:12:34,575::hosted_engine::458::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host ovirt2.nwfiber.com <http://ovirt2.nwfiber.com/> (id: 2, score: 3400)
>> MainThread::INFO::2017-07-17 08:12:44,692::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:12:44,692::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:47,493::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:12:48,127::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:12:48,758::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:49,128::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:12:49,552::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:12:49,559::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:12:49,567::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM
>> MainThread::INFO::2017-07-17 08:12:52,152::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage
>> MainThread::INFO::2017-07-17 08:12:52,192::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:12:58,227::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:12:58,406::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain
>> MainThread::INFO::2017-07-17 08:12:58,805::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images
>> MainThread::INFO::2017-07-17 08:12:58,806::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images
>> MainThread::INFO::2017-07-17 08:13:02,662::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf
>> MainThread::INFO::2017-07-17 08:13:02,662::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:13:02,663::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:05,462::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:13:06,057::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:13:06,646::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:07,100::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:13:07,595::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:13:07,602::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:13,643::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
>> MainThread::INFO::2017-07-17 08:13:13,644::hosted_engine::458::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host ovirt2.nwfiber.com <http://ovirt2.nwfiber.com/> (id: 2, score: 3400)
>> MainThread::INFO::2017-07-17 08:13:23,760::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:13:23,760::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:26,550::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:13:27,129::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:13:27,770::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:28,154::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:13:28,603::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:13:28,612::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:28,613::states::426::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
>> MainThread::INFO::2017-07-17 08:13:28,622::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM
>> MainThread::INFO::2017-07-17 08:13:31,357::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage
>> MainThread::INFO::2017-07-17 08:13:31,420::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:13:37,594::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:13:37,848::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain
>> MainThread::INFO::2017-07-17 08:13:38,300::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images
>> MainThread::INFO::2017-07-17 08:13:38,301::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images
>> MainThread::INFO::2017-07-17 08:13:41,859::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf
>> MainThread::INFO::2017-07-17 08:13:41,860::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:13:41,860::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:44,478::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:13:44,916::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:13:45,437::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:45,703::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:13:46,010::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:13:46,014::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:13:51,170::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
>> MainThread::INFO::2017-07-17 08:13:51,170::hosted_engine::458::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host ovirt2.nwfiber.com <http://ovirt2.nwfiber.com/> (id: 2, score: 3400)
>> MainThread::INFO::2017-07-17 08:14:01,282::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:14:01,282::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:04,906::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:14:05,394::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:14:05,989::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:06,277::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:14:06,627::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:14:06,632::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:06,637::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM
>> MainThread::INFO::2017-07-17 08:14:09,448::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage
>> MainThread::INFO::2017-07-17 08:14:09,494::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:14:14,997::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:14:15,267::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain
>> MainThread::INFO::2017-07-17 08:14:15,670::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images
>> MainThread::INFO::2017-07-17 08:14:15,671::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images
>> MainThread::INFO::2017-07-17 08:14:20,547::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf
>> MainThread::INFO::2017-07-17 08:14:20,548::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:14:20,548::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:23,443::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:14:24,001::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:14:24,604::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:24,988::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:14:25,391::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:14:25,398::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:30,596::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
>> MainThread::INFO::2017-07-17 08:14:30,597::hosted_engine::458::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host ovirt2.nwfiber.com <http://ovirt2.nwfiber.com/> (id: 2, score: 3400)
>> MainThread::INFO::2017-07-17 08:14:40,749::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:14:40,749::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:43,642::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:14:44,310::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:14:44,868::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:45,253::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:14:45,670::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:14:45,676::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:14:45,677::states::426::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
>> MainThread::INFO::2017-07-17 08:14:45,684::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM
>> MainThread::INFO::2017-07-17 08:14:49,208::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage
>> MainThread::INFO::2017-07-17 08:14:49,246::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:14:54,226::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:14:54,453::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain
>> MainThread::INFO::2017-07-17 08:14:54,835::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images
>> MainThread::INFO::2017-07-17 08:14:54,835::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images
>> MainThread::INFO::2017-07-17 08:14:58,377::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf
>> MainThread::INFO::2017-07-17 08:14:58,377::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:14:58,377::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:01,136::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:15:01,714::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:15:02,386::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:02,803::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:15:03,292::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:15:03,299::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:09,285::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
>> MainThread::INFO::2017-07-17 08:15:09,286::hosted_engine::458::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host ovirt2.nwfiber.com <http://ovirt2.nwfiber.com/> (id: 2, score: 3400)
>> MainThread::INFO::2017-07-17 08:15:19,412::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:15:19,413::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:22,266::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:15:22,913::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:15:23,506::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:23,881::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:15:24,303::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:15:24,309::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:24,318::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Trying: notify time=1500304524.32 type=state_transition detail=EngineUp-EngineUpBadHealth hostname='ovirt1.nwfiber.com <http://ovirt1.nwfiber.com/>'
>> MainThread::INFO::2017-07-17 08:15:24,450::brokerlink::121::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineUp-EngineUpBadHealth) sent? sent
>> MainThread::INFO::2017-07-17 08:15:24,451::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM
>> MainThread::INFO::2017-07-17 08:15:27,085::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage
>> MainThread::INFO::2017-07-17 08:15:27,126::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:15:33,220::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:15:33,514::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain
>> MainThread::INFO::2017-07-17 08:15:34,134::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images
>> MainThread::INFO::2017-07-17 08:15:34,134::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images
>> MainThread::INFO::2017-07-17 08:15:37,991::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf
>> MainThread::INFO::2017-07-17 08:15:37,991::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:15:37,991::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:40,745::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:15:41,372::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:15:42,054::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:42,563::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:15:43,062::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:15:43,069::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::INFO::2017-07-17 08:15:49,373::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUpBadHealth (score: 3400)
>> MainThread::INFO::2017-07-17 08:15:49,374::hosted_engine::458::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host ovirt2.nwfiber.com <http://ovirt2.nwfiber.com/> (id: 2, score: 3400)
>> MainThread::INFO::2017-07-17 08:15:59,492::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-07-17 08:15:59,492::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:16:02,358::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:e10c90a5-4d9c-4e18-b6f7-ae8f0cdf4f57, volUUID:a9754d40-eda1-44d7-ac92-76a228f9f1ac
>> MainThread::INFO::2017-07-17 08:16:03,114::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Found OVF_STORE: imgUUID:f22829ab-9fd5-415a-9a8f-809d3f7887d4, volUUID:9f4760ee-119c-412a-a1e8-49e73e6ba929
>> MainThread::INFO::2017-07-17 08:16:03,801::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE
>> MainThread::INFO::2017-07-17 08:16:04,206::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/glusterSD/192.168.8.11:_engine/c0acdefb-7d16-48ec-9d76-659b8fe33e2a/images/f22829ab-9fd5-415a-9a8f-809d3f7887d4/9f4760ee-119c-412a-a1e8-49e73e6ba929 
>> MainThread::INFO::2017-07-17 08:16:04,680::config::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
>> MainThread::INFO::2017-07-17 08:16:04,688::config::436::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE
>> MainThread::ERROR::2017-07-17 08:16:04,690::states::602::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine VM has bad health status, timeout in 300 seconds
>> MainThread::INFO::2017-07-17 08:16:04,691::states::426::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost
>> MainThread::INFO::2017-07-17 08:16:04,700::state_decorators::88::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) Timeout cleared while transitioning <class 'ovirt_hosted_engine_ha.agent.states.EngineUpBadHealth'> -> <class 'ovirt_hosted_engine_ha.agent.states.EngineUp'>
>> MainThread::INFO::2017-07-17 08:16:04,710::brokerlink::111::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Trying: notify time=1500304564.71 type=state_transition detail=EngineUpBadHealth-EngineUp hostname='ovirt1.nwfiber.com <http://ovirt1.nwfiber.com/>'
>> MainThread::INFO::2017-07-17 08:16:04,798::brokerlink::121::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineUpBadHealth-EngineUp) sent? sent
>> MainThread::INFO::2017-07-17 08:16:04,799::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM
>> MainThread::INFO::2017-07-17 08:16:07,435::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage
>> MainThread::INFO::2017-07-17 08:16:07,491::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:16:13,906::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server
>> MainThread::INFO::2017-07-17 08:16:14,131::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain
>> MainThread::INFO::2017-07-17 08:16:14,437::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images
>> MainThread::INFO::2017-07-17 08:16:14,438::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images
>> 
>> 
>> On Thu, Mar 30, 2017 at 5:58 AM, Simone Tiraboschi <stirabos at redhat.com <mailto:stirabos at redhat.com>> wrote:
>> Could you please check your /var/log/ovirt-hosted-engine-ha/agent.log ?
>> 
>> On Thu, Mar 30, 2017 at 3:10 AM, Jim Kusznir <jim at palousetech.com <mailto:jim at palousetech.com>> wrote:
>> Hello:
>> 
>> I find that I often get random-seeming messages.  A lot of them mention "ReintializeFSM", but I also get engine down, engine start, etc.  messages.  All the time, nothing appears to be happening on the cluster, and I rarely can find anything wrong or any trigger/cause.  Is this normal?  What causes this (beyond obvious hardware issues / hosts rebooting)?  Most of the time when I get these, my cluster is going along smoothly, and nothing (not even administrative access) is interrupted.
>> 
>> Could ISP issues cause these messages to be generated?
>> 
>> Thanks!
>> --Jim
>> 
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org <mailto:Users at ovirt.org>
>> http://lists.ovirt.org/mailman/listinfo/users <http://lists.ovirt.org/mailman/listinfo/users>
>> 
>> 
>> 
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org <mailto:Users at ovirt.org>
>> http://lists.ovirt.org/mailman/listinfo/users
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170718/981f7a3e/attachment-0001.html>