OVF_STORE issue after upgrading from 3.6.1 to 3.6.2
Hi, after upgrading from 3.6.1 to 3.6.2 I'm getting an error with OVF_STORE. How can I recreate the OVF_STORE? In another post someone said that i would be recreated after 1 hour, but not seems to be my case... Do I have a way to manually regenerate ? MainThread::WARNING::2016-01-26 12:07:56,734::ovf_store::105::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE MainThread::ERROR::2016-01-26 12:07:56,735::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf MainThread::INFO::2016-01-26 12:07:56,790::hosted_engine::464::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400) Thanks -Luiz
On Tue, Jan 26, 2016 at 1:11 PM, Luiz Claudio Prazeres Goncalves < luizcpg@gmail.com> wrote:
Hi, after upgrading from 3.6.1 to 3.6.2 I'm getting an error with OVF_STORE. How can I recreate the OVF_STORE? In another post someone said that i would be recreated after 1 hour, but not seems to be my case... Do I have a way to manually regenerate ?
The engine will create the OVF_STORE only if it correctly imported the hosted-engine storage domain; can you please check that? If it's still not there and you are upgrading, maybe you have just a left-over from previous release where the auto-import of the hosted-engine storage domain wasn't that stable. Please trying manually cleaning it.
MainThread::WARNING::2016-01-26 12:07:56,734::ovf_store::105::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE
MainThread::ERROR::2016-01-26 12:07:56,735::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::INFO::2016-01-26 12:07:56,790::hosted_engine::464::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
Thanks
-Luiz
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
In fact the hosted-engine storage domain was not created on 3.6.1 and , therefore, that's the OVF_STORE was not there. It's a gluster storage where the engine is being stored, so how can I "force" the generation of the hosted-engine storage domain ... it's not being generated automatically, even waiting > 1 h. Thanks -Luiz 2016-01-26 11:17 GMT-02:00 Simone Tiraboschi <stirabos@redhat.com>:
On Tue, Jan 26, 2016 at 1:11 PM, Luiz Claudio Prazeres Goncalves < luizcpg@gmail.com> wrote:
Hi, after upgrading from 3.6.1 to 3.6.2 I'm getting an error with OVF_STORE. How can I recreate the OVF_STORE? In another post someone said that i would be recreated after 1 hour, but not seems to be my case... Do I have a way to manually regenerate ?
The engine will create the OVF_STORE only if it correctly imported the hosted-engine storage domain; can you please check that? If it's still not there and you are upgrading, maybe you have just a left-over from previous release where the auto-import of the hosted-engine storage domain wasn't that stable. Please trying manually cleaning it.
MainThread::WARNING::2016-01-26 12:07:56,734::ovf_store::105::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE
MainThread::ERROR::2016-01-26 12:07:56,735::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::INFO::2016-01-26 12:07:56,790::hosted_engine::464::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
Thanks
-Luiz
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
By the way, I've tried to ,manually, add the engine storage domain , but failed... it's a wrong procedure, but I did it before talking to Simone :( ... it was manually removed later.... even after removing the wrong one.... the "correct one" was not automatically generated even waiting > 1 h. *as a colateral effect , I suppose, I've started to see the following errors on agent.log* MainThread::INFO::2016-01-26 13:03:32,827::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:03:37,842::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:03:42,858::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:03:47,873::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:03:52,889::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:03:57,904::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:04:02,920::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:04:07,936::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::INFO::2016-01-26 13:04:12,960::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING MainThread::ERROR::2016-01-26 13:04:12,961::hosted_engine::832::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_domain_monitor) Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 454, in start_monitoring self._initialize_domain_monitor() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 833, in _initialize_domain_monitor raise Exception(msg) Exception: Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition MainThread::WARNING::2016-01-26 13:04:12,961::hosted_engine::480::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Error while monitoring engine: Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition MainThread::WARNING::2016-01-26 13:04:12,961::hosted_engine::483::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Unexpected error Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 454, in start_monitoring self._initialize_domain_monitor() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 833, in _initialize_domain_monitor raise Exception(msg) Exception: Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition MainThread::ERROR::2016-01-26 13:04:12,961::hosted_engine::496::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Shutting down the agent because of 3 failures in a row! MainThread::WARNING::2016-01-26 13:04:14,979::hosted_engine::767::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_domain_monitor_if_possible) The VM is running locally or we have no data, keeping the domain monitor. MainThread::INFO::2016-01-26 13:04:14,980::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down MainThread::ERROR::2016-01-26 13:04:12,961::hosted_engine::496::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Shutting down the agent because of 3 failures in a row! MainThread::WARNING::2016-01-26 13:04:14,979::hosted_engine::767::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_domain_monitor_if_possible) The VM is running locally or we have no data, keeping the domain monitor. MainThread::INFO::2016-01-26 13:04:14,980::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down *and now the HA Agent do not start anymore... Anyone know how to fix both issues ? * Thanks -Luiz 2016-01-26 12:00 GMT-02:00 Luiz Claudio Prazeres Goncalves < luizcpg@gmail.com>:
In fact the hosted-engine storage domain was not created on 3.6.1 and , therefore, that's the OVF_STORE was not there.
It's a gluster storage where the engine is being stored, so how can I "force" the generation of the hosted-engine storage domain ... it's not being generated automatically, even waiting > 1 h.
Thanks -Luiz
2016-01-26 11:17 GMT-02:00 Simone Tiraboschi <stirabos@redhat.com>:
On Tue, Jan 26, 2016 at 1:11 PM, Luiz Claudio Prazeres Goncalves < luizcpg@gmail.com> wrote:
Hi, after upgrading from 3.6.1 to 3.6.2 I'm getting an error with OVF_STORE. How can I recreate the OVF_STORE? In another post someone said that i would be recreated after 1 hour, but not seems to be my case... Do I have a way to manually regenerate ?
The engine will create the OVF_STORE only if it correctly imported the hosted-engine storage domain; can you please check that? If it's still not there and you are upgrading, maybe you have just a left-over from previous release where the auto-import of the hosted-engine storage domain wasn't that stable. Please trying manually cleaning it.
MainThread::WARNING::2016-01-26 12:07:56,734::ovf_store::105::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE
MainThread::ERROR::2016-01-26 12:07:56,735::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::INFO::2016-01-26 12:07:56,790::hosted_engine::464::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
Thanks
-Luiz
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Tue, Jan 26, 2016 at 3:24 PM, Luiz Claudio Prazeres Goncalves < luizcpg@gmail.com> wrote:
By the way, I've tried to ,manually, add the engine storage domain , but failed... it's a wrong procedure, but I did it before talking to Simone :( ... it was manually removed later....
even after removing the wrong one.... the "correct one" was not automatically generated even waiting > 1 h.
*as a colateral effect , I suppose, I've started to see the following errors on agent.log*
MainThread::INFO::2016-01-26 13:03:32,827::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:03:37,842::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:03:42,858::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:03:47,873::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:03:52,889::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:03:57,904::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:04:02,920::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:04:07,936::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::INFO::2016-01-26 13:04:12,960::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_domain_monitor_status) VDSM domain monitor status: PENDING
MainThread::ERROR::2016-01-26 13:04:12,961::hosted_engine::832::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_domain_monitor) Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 454, in start_monitoring
self._initialize_domain_monitor()
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 833, in _initialize_domain_monitor
raise Exception(msg)
Exception: Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition
MainThread::WARNING::2016-01-26 13:04:12,961::hosted_engine::480::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Error while monitoring engine: Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition
MainThread::WARNING::2016-01-26 13:04:12,961::hosted_engine::483::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Unexpected error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 454, in start_monitoring
self._initialize_domain_monitor()
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 833, in _initialize_domain_monitor
raise Exception(msg)
Exception: Failed to start monitoring domain (sd_uuid=7149b31d-430b-4d91-a8a3-ae1024dfd0a9, host_id=2): timeout during domain acquisition
MainThread::ERROR::2016-01-26 13:04:12,961::hosted_engine::496::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Shutting down the agent because of 3 failures in a row!
MainThread::WARNING::2016-01-26 13:04:14,979::hosted_engine::767::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_domain_monitor_if_possible) The VM is running locally or we have no data, keeping the domain monitor.
MainThread::INFO::2016-01-26 13:04:14,980::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down
MainThread::ERROR::2016-01-26 13:04:12,961::hosted_engine::496::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Shutting down the agent because of 3 failures in a row!
MainThread::WARNING::2016-01-26 13:04:14,979::hosted_engine::767::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_domain_monitor_if_possible) The VM is running locally or we have no data, keeping the domain monitor.
MainThread::INFO::2016-01-26 13:04:14,980::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down
*and now the HA Agent do not start anymore... Anyone know how to fix both issues ? *
OK, [root@kvm2 ~]# vdsClient -s 0 getStorageDomainsList 2d0b57a7-8fe7-445d-af41-0c66c99ba962 0b4c94b1-aa8e-4415-b6f2-79c12ee9c4ed 6d50e33a-8f7c-4d75-a64b-07db36507b37 7149b31d-430b-4d91-a8a3-ae1024dfd0a9 7149b31d-430b-4d91-a8a3-ae1024dfd0a9 The 7149b31d-430b-4d91-a8a3-ae1024dfd0a9 appears twice in VDSM and so (or because that, we still need to understand) the auto-import fails. You hit this one: https://bugzilla.redhat.com/show_bug.cgi?id=1300749 Thanks
-Luiz
2016-01-26 12:00 GMT-02:00 Luiz Claudio Prazeres Goncalves < luizcpg@gmail.com>:
In fact the hosted-engine storage domain was not created on 3.6.1 and , therefore, that's the OVF_STORE was not there.
It's a gluster storage where the engine is being stored, so how can I "force" the generation of the hosted-engine storage domain ... it's not being generated automatically, even waiting > 1 h.
Thanks -Luiz
2016-01-26 11:17 GMT-02:00 Simone Tiraboschi <stirabos@redhat.com>:
On Tue, Jan 26, 2016 at 1:11 PM, Luiz Claudio Prazeres Goncalves < luizcpg@gmail.com> wrote:
Hi, after upgrading from 3.6.1 to 3.6.2 I'm getting an error with OVF_STORE. How can I recreate the OVF_STORE? In another post someone said that i would be recreated after 1 hour, but not seems to be my case... Do I have a way to manually regenerate ?
The engine will create the OVF_STORE only if it correctly imported the hosted-engine storage domain; can you please check that? If it's still not there and you are upgrading, maybe you have just a left-over from previous release where the auto-import of the hosted-engine storage domain wasn't that stable. Please trying manually cleaning it.
MainThread::WARNING::2016-01-26 12:07:56,734::ovf_store::105::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE
MainThread::ERROR::2016-01-26 12:07:56,735::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::INFO::2016-01-26 12:07:56,790::hosted_engine::464::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400)
Thanks
-Luiz
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (2)
-
Luiz Claudio Prazeres Goncalves -
Simone Tiraboschi