This is a multi-part message in MIME format. --------------05F3D0780062C6D37C1619A7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, so i was running a 3.4 hosted engine two node setup on centos 6, had some disk issues so i tried to upgrade to centos 7 and follow the path 3.4 > 3.5 > 3.6 > 4.0. i screwed up dig time somewhere between 3.6 and 4.0, so i wiped the drives, installed a fresh 4.0.3, then created the database and restored the 3.6 engine backup before running engine-setup as per the docs. things seemed to work, but i have the the following issues / symptoms: - ovirt-ha-agent running 100% CPU on both nodes - messages in the UI that the Hosted Engine storage Domain isn't active and Failed to import the Hosted Engine Storage Domain - hosted engine is not visible in the UI and the following repeating in the agent.log: MainThread::INFO::2016-10-03 12:38:27,718::hosted_engine::461::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400) MainThread::INFO::2016-10-03 12:38:27,720::hosted_engine::466::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host vmhost1.oracool.net (id: 1, score: 3400) MainThread::INFO::2016-10-03 12:38:37,979::states::421::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost MainThread::INFO::2016-10-03 12:38:37,985::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2016-10-03 12:38:45,645::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2016-10-03 12:38:45,647::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2016-10-03 12:39:00,543::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2016-10-03 12:39:00,562::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain MainThread::INFO::2016-10-03 12:39:01,235::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images MainThread::INFO::2016-10-03 12:39:01,236::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images MainThread::INFO::2016-10-03 12:39:09,295::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Reloading vm.conf from the shared storage domain MainThread::INFO::2016-10-03 12:39:09,296::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::WARNING::2016-10-03 12:39:16,928::ovf_store::107::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE MainThread::ERROR::2016-10-03 12:39:16,934::config::235::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf I have searched a bit and not really found a solution, and have come to the conclusion that i have made a mess of things, and am wondering if the best solution is to export the VMs, and reinstall everything then import them back? i am using remote NFS storage. if i try and add the hosted engine storage domain it says it is already registered. i have also upgraded and am now running oVirt Engine Version: 4.0.4.4-1.el7.centos hosts were installed using ovirt-node. currently at 3.10.0-327.28.3.el7.x86_64 if a fresh install is best, any advice / pointer to doc that explains best way to do this? i have not moved my most important server over to this cluster yet so i can take some downtime to reinstall. thanks! sam --------------05F3D0780062C6D37C1619A7 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> Hi,<br> so i was running a 3.4 hosted engine two node setup on centos 6, had some disk issues so i tried to upgrade to centos 7 and follow the path 3.4 &gt; 3.5 &gt; 3.6 &gt; 4.0.  i screwed up dig time somewhere between 3.6 and 4.0, so i wiped the drives, installed a fresh 4.0.3, then created the database and restored the 3.6 engine backup before running engine-setup as per the docs.   things seemed to work, but i have the the following issues / symptoms:<br> - ovirt-ha-agent running 100% CPU on both nodes<br> - messages in the UI that the Hosted Engine storage Domain isn't active and Failed to import the Hosted Engine Storage Domain<br> - hosted engine is not visible in the UI<br> and the following repeating in the agent.log:<br> <br> MainThread::INFO::2016-10-03 12:38:27,718::hosted_engine::461::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)<br> MainThread::INFO::2016-10-03 12:38:27,720::hosted_engine::466::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host vmhost1.oracool.net (id: 1, score: 3400)<br> MainThread::INFO::2016-10-03 12:38:37,979::states::421::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost<br> MainThread::INFO::2016-10-03 12:38:37,985::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM<br> MainThread::INFO::2016-10-03 12:38:45,645::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage<br> MainThread::INFO::2016-10-03 12:38:45,647::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server<br> MainThread::INFO::2016-10-03 12:39:00,543::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server<br> MainThread::INFO::2016-10-03 12:39:00,562::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain<br> MainThread::INFO::2016-10-03 12:39:01,235::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images<br> MainThread::INFO::2016-10-03 12:39:01,236::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images<br> MainThread::INFO::2016-10-03 12:39:09,295::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Reloading vm.conf from the shared storage domain<br> MainThread::INFO::2016-10-03 12:39:09,296::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Trying to get a fresher copy of vm configuration from the OVF_STORE<br> MainThread::WARNING::2016-10-03 12:39:16,928::ovf_store::107::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE<br> MainThread::ERROR::2016-10-03 12:39:16,934::config::235::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf<br> <br> I have searched a bit and not really found a solution, and have come to the conclusion that i have made a mess of things, and am wondering if the best solution is to export the VMs, and reinstall everything then import them back?<br> i am using remote  NFS storage.<br> if i try and add the hosted engine storage domain it says it is already registered.<br> i have also upgraded and am now running <span class="gwt-InlineLabel">oVirt Engine Version: 4.0.4.4-1.el7.centos<br> hosts were installed using ovirt-node.  currently at  3.10.0-327.28.3.el7.x86_64<br> if a fresh install is best, any advice / pointer to doc that explains best way to do this?<br> i have not moved my most important server over to this cluster yet so i can take some downtime to reinstall.<br> thanks!<br> sam<br> <br> </span><br> </body> </html> --------------05F3D0780062C6D37C1619A7--