[ovirt-users] Re: Broker fails to start after upgrade 4.1 to 4.2 metadata_image_UUID can't be ''

Monday, 25 June 2018

Hi,

yes there is a solution described directly in the bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1521011#c20

The provided script worked only for cases that had the necessary
disks, but where the uuids were not written to the config files.

You need to follow the procedure from comment 20 when no disks for
lockspace and metadata exist at all.

Best regards

Martin Sivak

On Mon, Jun 25, 2018 at 9:52 AM, Reznikov Alexei <reznikov_aa(a)soskol.com&gt; wrote:
> 21.06.2018 20:15, reznikov_aa(a)soskol.com пишет:
>>
>> Hi list!
>>
>> After upgrade my cluster from 4.1.9 to 4.2.2, agent and broker can't start
>> on host...
>>
>> cat /var/log/ovirt-hosted-engine-ha/agent.log
>> MainThread::ERROR::2018-06-21
>>
03:25:34,603::hosted_engine::538::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
>> Failed to start necessary monitors
>> MainThread::ERROR::2018-06-21
>> 03:25:34,604::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>> Traceback (most recent call last)
>>
>> cat /var/log/ovirt-hosted-engine-ha/broker.log
>> MainThread::INFO::2018-06-21
>>
03:25:40,406::monitor::50::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors)
>> Finished loading submonitors
>> MainThread::WARNING::2018-06-21
>>
03:25:40,406::storage_broker::97::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__)
>> Can't connect vdsm storage: 'metadata_image_UUID can't be ''
>>
>> cat /etc/ovirt-hosted-engine/hosted-engine.conf | grep metadata_image
>> metadata_image_UUID=
>>
>> Also is:
>> cat /etc/ovirt-hosted-engine/hosted-engine.conf | grep lock
>> lockspace_image_UUID=
>> lockspace_volume_UUID=
>>
>> This bug is very much like this
>> https://bugzilla.redhat.com/show_bug.cgi?id=1521011 My cluster started with
>> version 3.3...
>>
>> But i can't resolution this bug correctly.
>>
>> Guru please help me!!!
>>
>> Thanx, Alex!
>>
> Bump.
>
> I tried run workaround script from Simone Tiraboschi, but him not work
> properly for me.
>
> I not see volume ... hosted-engine.lockspace and hosted-engine.metada is
> null.
>
> [root@h4 ~]# ./workaround_1521011.sh
> + source /etc/ovirt-hosted-engine/hosted-engine.conf
> ++ fqdn=eng.lan
> ++ vm_disk_id=e9d7a377-e109-4b28-9a43-7a8c8b603749
> ++ vm_disk_vol_id=cd12a59e-7d84-4b4e-98c7-4c68e83ecd7b
> ++ vmid=ccdd675a-a58b-495a-9502-3e6a4b7e5228
> ++ storage=ssd:/ovirt
> ++ mnt_options=
> ++ conf=/var/run/ovirt-hosted-engine-ha/vm.conf
> ++ host_id=4
> ++ console=vnc
> ++ domainType=nfs3
> ++ spUUID=00000000-0000-0000-0000-000000000000
> ++ sdUUID=8905c9ac-d892-478d-8346-63b8fa1c5763
> ++ connectionUUID=ce84071b-86a2-4e82-b4d9-06abf23dfbc4
> ++ ca_cert=/etc/pki/vdsm/libvirt-spice/ca-cert.pem
> ++ ca_subject='C=EN, L=Test, O=Test, CN=Test'
> ++ vdsm_use_ssl=true
> ++ gateway=10.245.183.1
> ++ bridge=ovirtmgmt
> ++ lockspace_volume_UUID=
> ++ lockspace_image_UUID=
> ++ metadata_volume_UUID=
> ++ metadata_image_UUID=
> ++ conf_volume_UUID=a20d9700-1b9a-41d8-bb4b-f2b7c168104f
> ++ conf_image_UUID=b5f353f5-9357-4aad-b1a3-751d411e6278
> ++ iqn=
> ++ portal=
> ++ user=
> ++ password=
> ++ port=
> ++ vdsm-client StorageDomain getImages
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> ++ grep -
> ++ tr -d ,
> ++ xargs
> + for i in '$(vdsm-client StorageDomain getImages storagedomainID=${sdUUID}
> storagepoolID=${spUUID} | grep - | tr -d '\'','\'' |
xargs)'
> ++ vdsm-client StorageDomain getVolumes
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=83e0550b-0fc3-40b1-955d-b07ebfbb3994
> ++ grep -
> ++ tr -d ,
> ++ xargs
> + for v in '$(vdsm-client StorageDomain getVolumes storagedomainID=${sdUUID}
> storagepoolID=${spUUID} imageID=${i} | grep - | tr -d
'\'','\'' | xargs)'
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=83e0550b-0fc3-40b1-955d-b07ebfbb3994
> volumeID=5a26be32-6c5b-4dcc-ac67-5c442f24df55
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.uuid'
> ++ xargs
> + lockspace_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=83e0550b-0fc3-40b1-955d-b07ebfbb3994
> volumeID=5a26be32-6c5b-4dcc-ac67-5c442f24df55
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.image'
> ++ xargs
> + lockspace_img_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=83e0550b-0fc3-40b1-955d-b07ebfbb3994
> volumeID=5a26be32-6c5b-4dcc-ac67-5c442f24df55
> ++ jq '. | select(.description=="hosted-engine.metadata") | .uuid'
> ++ xargs
> + metadata_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=83e0550b-0fc3-40b1-955d-b07ebfbb3994
> volumeID=5a26be32-6c5b-4dcc-ac67-5c442f24df55
> ++ jq '. | select(.description=="hosted-engine.metadata") |
.image'
> ++ xargs
> + metadata_img_uuid=
> + [[ ! -z '' ]]
> + for i in '$(vdsm-client StorageDomain getImages storagedomainID=${sdUUID}
> storagepoolID=${spUUID} | grep - | tr -d '\'','\'' |
xargs)'
> ++ vdsm-client StorageDomain getVolumes
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=3abe2f7b-02b9-40a3-8feb-f2809c22c0fb
> ++ grep -
> ++ tr -d ,
> ++ xargs
> + for v in '$(vdsm-client StorageDomain getVolumes storagedomainID=${sdUUID}
> storagepoolID=${spUUID} imageID=${i} | grep - | tr -d
'\'','\'' | xargs)'
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=3abe2f7b-02b9-40a3-8feb-f2809c22c0fb
> volumeID=83186aa6-aaa7-4ce3-9967-7cd3d97d777b
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.uuid'
> ++ xargs
> + lockspace_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=3abe2f7b-02b9-40a3-8feb-f2809c22c0fb
> volumeID=83186aa6-aaa7-4ce3-9967-7cd3d97d777b
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.image'
> ++ xargs
> + lockspace_img_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=3abe2f7b-02b9-40a3-8feb-f2809c22c0fb
> volumeID=83186aa6-aaa7-4ce3-9967-7cd3d97d777b
> ++ jq '. | select(.description=="hosted-engine.metadata") | .uuid'
> ++ xargs
> + metadata_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=3abe2f7b-02b9-40a3-8feb-f2809c22c0fb
> volumeID=83186aa6-aaa7-4ce3-9967-7cd3d97d777b
> ++ jq '. | select(.description=="hosted-engine.metadata") |
.image'
> ++ xargs
> + metadata_img_uuid=
> + [[ ! -z '' ]]
> + for i in '$(vdsm-client StorageDomain getImages storagedomainID=${sdUUID}
> storagepoolID=${spUUID} | grep - | tr -d '\'','\'' |
xargs)'
> ++ vdsm-client StorageDomain getVolumes
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=b5f353f5-9357-4aad-b1a3-751d411e6278
> ++ grep -
> ++ tr -d ,
> ++ xargs
> + for v in '$(vdsm-client StorageDomain getVolumes storagedomainID=${sdUUID}
> storagepoolID=${spUUID} imageID=${i} | grep - | tr -d
'\'','\'' | xargs)'
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=b5f353f5-9357-4aad-b1a3-751d411e6278
> volumeID=a20d9700-1b9a-41d8-bb4b-f2b7c168104f
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.uuid'
> ++ xargs
> + lockspace_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=b5f353f5-9357-4aad-b1a3-751d411e6278
> volumeID=a20d9700-1b9a-41d8-bb4b-f2b7c168104f
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.image'
> ++ xargs
> + lockspace_img_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=b5f353f5-9357-4aad-b1a3-751d411e6278
> volumeID=a20d9700-1b9a-41d8-bb4b-f2b7c168104f
> ++ jq '. | select(.description=="hosted-engine.metadata") | .uuid'
> ++ xargs
> + metadata_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=b5f353f5-9357-4aad-b1a3-751d411e6278
> volumeID=a20d9700-1b9a-41d8-bb4b-f2b7c168104f
> ++ jq '. | select(.description=="hosted-engine.metadata") |
.image'
> ++ xargs
> + metadata_img_uuid=
> + [[ ! -z '' ]]
> + for i in '$(vdsm-client StorageDomain getImages storagedomainID=${sdUUID}
> storagepoolID=${spUUID} | grep - | tr -d '\'','\'' |
xargs)'
> ++ vdsm-client StorageDomain getVolumes
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=e9d7a377-e109-4b28-9a43-7a8c8b603749
> ++ grep -
> ++ tr -d ,
> ++ xargs
> + for v in '$(vdsm-client StorageDomain getVolumes storagedomainID=${sdUUID}
> storagepoolID=${spUUID} imageID=${i} | grep - | tr -d
'\'','\'' | xargs)'
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=e9d7a377-e109-4b28-9a43-7a8c8b603749
> volumeID=cd12a59e-7d84-4b4e-98c7-4c68e83ecd7b
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.uuid'
> ++ xargs
> + lockspace_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=e9d7a377-e109-4b28-9a43-7a8c8b603749
> volumeID=cd12a59e-7d84-4b4e-98c7-4c68e83ecd7b
> ++ jq '. | select(.description=="hosted-engine.lockspace") |
.image'
> ++ xargs
> + lockspace_img_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=e9d7a377-e109-4b28-9a43-7a8c8b603749
> volumeID=cd12a59e-7d84-4b4e-98c7-4c68e83ecd7b
> ++ jq '. | select(.description=="hosted-engine.metadata") | .uuid'
> ++ xargs
> + metadata_vol_uuid=
> + [[ ! -z '' ]]
> ++ vdsm-client Volume getInfo
> storagedomainID=8905c9ac-d892-478d-8346-63b8fa1c5763
> storagepoolID=00000000-0000-0000-0000-000000000000
> imageID=e9d7a377-e109-4b28-9a43-7a8c8b603749
> volumeID=cd12a59e-7d84-4b4e-98c7-4c68e83ecd7b
> ++ jq '. | select(.description=="hosted-engine.metadata") |
.image'
> ++ xargs
> + metadata_img_uuid=
> + [[ ! -z '' ]]
>
> How to create missing volume ? Is there any solution, or just reinstall HE?
>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/3Y7NKSBA3ZN...

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

[ovirt-users] Re: Broker fails to start after upgrade 4.1 to 4.2 metadata_image_UUID can't be ''