1:31 a.m.
Seeing issues with my hosted engine, it seems it's unable to extract
vm.conf from storage. My ovirt-hosted-engine-ha/agent.log is full of this
repeating over and over. This is happening on all 3 of my hosts. My storage
is glusterfs on the hosts themselves.
Hopefully this is enough info to get started.
Thanks!
MainThread::INFO::2017-02-07
19:27:33,063::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
Initializing VDSM
MainThread::INFO::2017-02-07
19:27:35,455::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Connecting the storage
MainThread::INFO::2017-02-07
19:27:35,456::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2017-02-07
19:27:40,169::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2017-02-07
19:27:40,202::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Refreshing the storage domain
MainThread::INFO::2017-02-07
19:27:40,418::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Preparing images
MainThread::INFO::2017-02-07
19:27:40,419::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images)
Preparing images
MainThread::INFO::2017-02-07
19:27:43,370::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Reloading vm.conf from the shared storage domain
MainThread::INFO::2017-02-07
19:27:43,371::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file)
Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::INFO::2017-02-07
19:27:45,968::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
Found OVF_STORE: imgUUID:3e14c1b5-5ade-4827-aad4-66c59824acd2,
volUUID:3cbeeb3b-f755-4d42-a654-8dab34213792
MainThread::INFO::2017-02-07
19:27:46,257::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
Found OVF_STORE: imgUUID:9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce,
volUUID:8f4d69c5-73a7-4e8c-a58f-909b55efec7d
MainThread::INFO::2017-02-07
19:27:46,355::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
Extracting Engine VM OVF from the OVF_STORE
MainThread::INFO::2017-02-07
19:27:46,366::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
OVF_STORE volume path:
/rhev/data-center/mnt/glusterSD/ovirt01-gluster.doonga.org:
_engine/536cd721-4396-4029-b1ea-8ce84738137e/images/9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce/8f4d69c5-73a7-4e8c-a58f-909b55efec7d
MainThread::ERROR::2017-02-07
19:27:46,389::ovf_store::124::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
Unable to extract HEVM OVF
MainThread::ERROR::2017-02-07
19:27:46,390::config::235::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file)
Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
8:59 a.m.
On Wed, Feb 8, 2017 at 2:31 AM, Todd Punderson <punderstorm93(a)gmail.com> wrote:
Seeing issues with my hosted engine, it seems it's unable to
extract vm.conf
from storage. My ovirt-hosted-engine-ha/agent.log is full of this repeating
over and over. This is happening on all 3 of my hosts. My storage is
glusterfs on the hosts themselves.
Hopefully this is enough info to get started.
Thanks!
MainThread::INFO::2017-02-07
19:27:33,063::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
Initializing VDSM
MainThread::INFO::2017-02-07
19:27:35,455::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Connecting the storage
MainThread::INFO::2017-02-07
19:27:35,456::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2017-02-07
19:27:40,169::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2017-02-07
19:27:40,202::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Refreshing the storage domain
MainThread::INFO::2017-02-07
19:27:40,418::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Preparing images
MainThread::INFO::2017-02-07
19:27:40,419::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images)
Preparing images
MainThread::INFO::2017-02-07
19:27:43,370::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Reloading vm.conf from the shared storage domain
MainThread::INFO::2017-02-07
19:27:43,371::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file)
Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::INFO::2017-02-07
19:27:45,968::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
Found OVF_STORE: imgUUID:3e14c1b5-5ade-4827-aad4-66c59824acd2,
volUUID:3cbeeb3b-f755-4d42-a654-8dab34213792
MainThread::INFO::2017-02-07
19:27:46,257::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
Found OVF_STORE: imgUUID:9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce,
volUUID:8f4d69c5-73a7-4e8c-a58f-909b55efec7d
MainThread::INFO::2017-02-07
19:27:46,355::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
Extracting Engine VM OVF from the OVF_STORE
MainThread::INFO::2017-02-07
19:27:46,366::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
OVF_STORE volume path:
/rhev/data-center/mnt/glusterSD/ovirt01-gluster.doonga.org:_engine/536cd721-4396-4029-b1ea-8ce84738137e/images/9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce/8f4d69c5-73a7-4e8c-a58f-909b55efec7d
MainThread::ERROR::2017-02-07
19:27:46,389::ovf_store::124::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
Unable to extract HEVM OVF
MainThread::ERROR::2017-02-07
19:27:46,390::config::235::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file)
Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
Can you please attach the output of:
sudo -u vdsm dd
if=/rhev/data-center/mnt/glusterSD/ovirt01-gluster.doonga.org:_engine/536cd721-4396-4029-b1ea-8ce84738137e/images/9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce/8f4d69c5-73a7-4e8c-a58f-909b55efec7d
| tar -tvf -
Thanks.
Did everything work well in 4.0? How did you upgrade?
Best,
--
Didi
9:27 a.m.
On Wed, Feb 8, 2017 at 8:59 AM, Yedidyah Bar David <didi(a)redhat.com> wrote:
On Wed, Feb 8, 2017 at 2:31 AM, Todd Punderson
<punderstorm93(a)gmail.com>
wrote:
> Seeing issues with my hosted engine, it seems it's unable to extract
vm.conf
> from storage. My ovirt-hosted-engine-ha/agent.log is full of this
repeating
> over and over. This is happening on all 3 of my hosts. My storage is
> glusterfs on the hosts themselves.
>
> Hopefully this is enough info to get started.
Another step is editing /etc/ovirt-hosted-engine-ha/agent-log.conf changing
from
[logger_root]
level=INFO
handlers=syslog,logfile
propagate=0
to
[logger_root]
level=DEBUG
handlers=syslog,logfile
propagate=0
and restart ovirt-ha-agent to get more detailed info about the issue.
>
> Thanks!
>
> MainThread::INFO::2017-02-07
> 19:27:33,063::hosted_engine::612::ovirt_hosted_engine_ha.
agent.hosted_engine.HostedEngine::(_initialize_vdsm)
> Initializing VDSM
> MainThread::INFO::2017-02-07
> 19:27:35,455::hosted_engine::639::ovirt_hosted_engine_ha.
agent.hosted_engine.HostedEngine::(_initialize_storage_images)
> Connecting the storage
> MainThread::INFO::2017-02-07
> 19:27:35,456::storage_server::219::ovirt_hosted_engine_ha.
lib.storage_server.StorageServer::(connect_storage_server)
> Connecting storage server
> MainThread::INFO::2017-02-07
> 19:27:40,169::storage_server::226::ovirt_hosted_engine_ha.
lib.storage_server.StorageServer::(connect_storage_server)
> Connecting storage server
> MainThread::INFO::2017-02-07
> 19:27:40,202::storage_server::233::ovirt_hosted_engine_ha.
lib.storage_server.StorageServer::(connect_storage_server)
> Refreshing the storage domain
> MainThread::INFO::2017-02-07
> 19:27:40,418::hosted_engine::666::ovirt_hosted_engine_ha.
agent.hosted_engine.HostedEngine::(_initialize_storage_images)
> Preparing images
> MainThread::INFO::2017-02-07
> 19:27:40,419::image::126::ovirt_hosted_engine_ha.lib.
image.Image::(prepare_images)
> Preparing images
> MainThread::INFO::2017-02-07
> 19:27:43,370::hosted_engine::669::ovirt_hosted_engine_ha.
agent.hosted_engine.HostedEngine::(_initialize_storage_images)
> Reloading vm.conf from the shared storage domain
> MainThread::INFO::2017-02-07
> 19:27:43,371::config::206::ovirt_hosted_engine_ha.agent.
hosted_engine.HostedEngine.config::(refresh_local_conf_file)
> Trying to get a fresher copy of vm configuration from the OVF_STORE
> MainThread::INFO::2017-02-07
> 19:27:45,968::ovf_store::103::ovirt_hosted_engine_ha.lib.
ovf.ovf_store.OVFStore::(scan)
> Found OVF_STORE: imgUUID:3e14c1b5-5ade-4827-aad4-66c59824acd2,
> volUUID:3cbeeb3b-f755-4d42-a654-8dab34213792
> MainThread::INFO::2017-02-07
> 19:27:46,257::ovf_store::103::ovirt_hosted_engine_ha.lib.
ovf.ovf_store.OVFStore::(scan)
> Found OVF_STORE: imgUUID:9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce,
> volUUID:8f4d69c5-73a7-4e8c-a58f-909b55efec7d
> MainThread::INFO::2017-02-07
> 19:27:46,355::ovf_store::112::ovirt_hosted_engine_ha.lib.
ovf.ovf_store.OVFStore::(getEngineVMOVF)
> Extracting Engine VM OVF from the OVF_STORE
> MainThread::INFO::2017-02-07
> 19:27:46,366::ovf_store::119::ovirt_hosted_engine_ha.lib.
ovf.ovf_store.OVFStore::(getEngineVMOVF)
> OVF_STORE volume path:
> /rhev/data-center/mnt/glusterSD/ovirt01-gluster.doonga.org:
_engine/536cd721-4396-4029-b1ea-8ce84738137e/images/9b49968b-5a62-4ab2-
a2c5-b94bc0b2d3ce/8f4d69c5-73a7-4e8c-a58f-909b55efec7d
> MainThread::ERROR::2017-02-07
> 19:27:46,389::ovf_store::124::ovirt_hosted_engine_ha.lib.
ovf.ovf_store.OVFStore::(getEngineVMOVF)
> Unable to extract HEVM OVF
> MainThread::ERROR::2017-02-07
> 19:27:46,390::config::235::ovirt_hosted_engine_ha.agent.
hosted_engine.HostedEngine.config::(refresh_local_conf_file)
> Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
Can you please attach the output of:
sudo -u vdsm dd
if=/rhev/data-center/mnt/glusterSD/ovirt01-gluster.doonga.org:
_engine/536cd721-4396-4029-b1ea-8ce84738137e/images/9b49968b-5a62-4ab2-
a2c5-b94bc0b2d3ce/8f4d69c5-73a7-4e8c-a58f-909b55efec7d
| tar -tvf -
Thanks.
Did everything work well in 4.0? How did you upgrade?
Best,
--
Didi
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
1:13 a.m.
Hi,
I was able to resolve the issue by reinstalling the hosts via
"Install/Reinstall" in the engine. I set the hosted engine to deploy, I did
that on all three hosts and it seems to be working fine now.
Thanks
On Wed, Feb 8, 2017 at 2:59 AM Yedidyah Bar David <didi(a)redhat.com> wrote:
On Wed, Feb 8, 2017 at 2:31 AM, Todd Punderson
<punderstorm93(a)gmail.com>
wrote:
> Seeing issues with my hosted engine, it seems it's unable to extract
vm.conf
> from storage. My ovirt-hosted-engine-ha/agent.log is full of this
repeating
> over and over. This is happening on all 3 of my hosts. My storage is
> glusterfs on the hosts themselves.
>
> Hopefully this is enough info to get started.
>
> Thanks!
>
> MainThread::INFO::2017-02-07
>
19:27:33,063::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
> Initializing VDSM
> MainThread::INFO::2017-02-07
>
19:27:35,455::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
> Connecting the storage
> MainThread::INFO::2017-02-07
>
19:27:35,456::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
> Connecting storage server
> MainThread::INFO::2017-02-07
>
19:27:40,169::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
> Connecting storage server
> MainThread::INFO::2017-02-07
>
19:27:40,202::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
> Refreshing the storage domain
> MainThread::INFO::2017-02-07
>
19:27:40,418::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
> Preparing images
> MainThread::INFO::2017-02-07
>
19:27:40,419::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images)
> Preparing images
> MainThread::INFO::2017-02-07
>
19:27:43,370::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
> Reloading vm.conf from the shared storage domain
> MainThread::INFO::2017-02-07
>
19:27:43,371::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file)
> Trying to get a fresher copy of vm configuration from the OVF_STORE
> MainThread::INFO::2017-02-07
>
19:27:45,968::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
> Found OVF_STORE: imgUUID:3e14c1b5-5ade-4827-aad4-66c59824acd2,
> volUUID:3cbeeb3b-f755-4d42-a654-8dab34213792
> MainThread::INFO::2017-02-07
>
19:27:46,257::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
> Found OVF_STORE: imgUUID:9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce,
> volUUID:8f4d69c5-73a7-4e8c-a58f-909b55efec7d
> MainThread::INFO::2017-02-07
>
19:27:46,355::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
> Extracting Engine VM OVF from the OVF_STORE
> MainThread::INFO::2017-02-07
>
19:27:46,366::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
> OVF_STORE volume path:
> /rhev/data-center/mnt/glusterSD/ovirt01-gluster.doonga.org:
_engine/536cd721-4396-4029-b1ea-8ce84738137e/images/9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce/8f4d69c5-73a7-4e8c-a58f-909b55efec7d
> MainThread::ERROR::2017-02-07
>
19:27:46,389::ovf_store::124::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF)
> Unable to extract HEVM OVF
> MainThread::ERROR::2017-02-07
>
19:27:46,390::config::235::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file)
> Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
Can you please attach the output of:
sudo -u vdsm dd
if=/rhev/data-center/mnt/glusterSD/ovirt01-gluster.doonga.org:
_engine/536cd721-4396-4029-b1ea-8ce84738137e/images/9b49968b-5a62-4ab2-a2c5-b94bc0b2d3ce/8f4d69c5-73a7-4e8c-a58f-909b55efec7d
| tar -tvf -
Thanks.
Did everything work well in 4.0? How did you upgrade?
Best,
--
Didi
3017
days inactive
3018
days old
3 comments
3 participants
participants (3)
-
Simone Tiraboschi -
Todd Punderson -
Yedidyah Bar David