hosted-storage import fails on hyperconverged glusterFS
by Liebe, André-Sebastian
This is an S/MIME signed message
------69AE9CF32CF9461733367ED13D0233A1
Content-Language: de-DE
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hello List,
I run into trouble after moving our hosted engine from nfs to hyperconverge=
d glusterFS by backup/restore[1] procedure. The engine logs it can't impor=
t and activate the hosted-storage although I can see the storage.
Any Hints how to fix this?
- I created the ha-replica-3 gluster volume prior to hosted-engine-setup us=
ing the hosts short name.
- Then ran hosted-engine-setup to install an new hosted engine (by installi=
ng centOS7 and ovirt-engine amnually)
- inside the new hosted-engine I restored the last successfull backup (wich=
was in running state)
- then I connected to the engine-database and removed the old hosted-engine=
by hand (as part of this patch would do: https://gerrit.ovirt.org/#/c/6496=
6/) and all known hosts (after marking all vms as down, where I got ETL err=
or messages later on for this)
- then I finished up the engine installation by running the engine-setup in=
side the hosted_engine
- and finally completed the hosted-engine-setup
The new hosted engine came up successfully with all prior known storage and=
after enabling glusterFS, the cluster this HA-host is part of, I could see=
it in the volumes and storage tab. After adding the remaining two hosts, t=
he volume was marked as active.
But here's the the error message I get repeadately since then:
> 2017-01-19 08:49:36,652 WARN [org.ovirt.engine.core.bll.storage.domain.I=
mportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-6-thread-10) =
[3b955ecd] Validation of action 'ImportHostedEngineStorageDomain' failed fo=
r user SYSTEM. Reasons: VAR__ACTION__ADD,VAR__TYPE__STORAGE__DOMAIN,ACTION_=
TYPE_FAILED_STORAGE_DOMAIN_ALREADY_EXIST
There are also some repeating messages about this ha-replica-3 volume, beca=
use I used the hosts short name on volume creation, which I can't change af=
aik without a complete cluster shutdown.
> 2017-01-19 08:48:03,134 INFO [org.ovirt.engine.core.bll.AddUnmanagedVmsC=
ommand] (DefaultQuartzScheduler3) [7471d7de] Running command: AddUnmanagedV=
msCommand internal: true.
> 2017-01-19 08:48:03,134 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.=
FullListVDSCommand] (DefaultQuartzScheduler3) [7471d7de] START, FullListVDS=
Command(HostName =3D , FullListVDSCommandParameters:{runAsync=3D'true', hos=
tId=3D'f62c7d04-9c95-453f-92d5-6dabf9da874a', vds=3D'Host[,f62c7d04-9c95-45=
3f-92d5-6dabf9da874a]', vmIds=3D'[dfea96e8-e94a-407e-af46-3019fd3f2991]'}),=
log id: 2d0941f9
> 2017-01-19 08:48:03,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.=
FullListVDSCommand] (DefaultQuartzScheduler3) [7471d7de] FINISH, FullListVD=
SCommand, return: [{guestFQDN=3D, emulatedMachine=3Dpc, pid=3D0, guestDiskM=
apping=3D{}, devices=3D[Ljava.lang.Object;@4181d938, cpuType=3DHaswell-noTS=
X, smp=3D2, vmType=3Dkvm, memSize=3D8192, vmName=3DHostedEngine, username=
=3D, exitMessage=3DXML error: maximum vcpus count must be an integer, vmId=
=3Ddfea96e8-e94a-407e-af46-3019fd3f2991, displayIp=3D0, displayPort=3D-1, g=
uestIPs=3D, spiceSecureChannels=3Dsmain,sdisplay,sinputs,scursor,splayback,=
srecord,ssmartcard,susbredir, exitCode=3D1, nicModel=3Drtl8139,pv, exitReas=
on=3D1, status=3DDown, maxVCpus=3DNone, clientIp=3D, statusTime=3D667507178=
0, display=3Dvnc, displaySecurePort=3D-1}], log id: 2d0941f9
> 2017-01-19 08:48:03,163 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.=
VdsBrokerObjectsBuilder] (DefaultQuartzScheduler3) [7471d7de] null architec=
ture type, replacing with x86_64, %s
> 2017-01-19 08:48:17,779 INFO [org.ovirt.engine.core.vdsbroker.gluster.Gl=
usterServersListVDSCommand] (DefaultQuartzScheduler3) [7471d7de] START, Glu=
sterServersListVDSCommand(HostName =3D lvh2, VdsIdVDSCommandParametersBase:=
{runAsync=3D'true', hostId=3D'23297fc2-db12-4778-a5ff-b74d6fc9554b'}), log =
id: 57d029dc
> 2017-01-19 08:48:18,177 INFO [org.ovirt.engine.core.vdsbroker.gluster.Gl=
usterServersListVDSCommand] (DefaultQuartzScheduler3) [7471d7de] FINISH, Gl=
usterServersListVDSCommand, return: [172.31.1.22/24:CONNECTED, lvh3.lab.gem=
atik.de:CONNECTED, lvh4.lab.gematik.de:CONNECTED], log id: 57d029dc
> 2017-01-19 08:48:18,180 INFO [org.ovirt.engine.core.vdsbroker.gluster.Gl=
usterVolumesListVDSCommand] (DefaultQuartzScheduler3) [7471d7de] START, Glu=
sterVolumesListVDSCommand(HostName =3D lvh2, GlusterVolumesListVDSParameter=
s:{runAsync=3D'true', hostId=3D'23297fc2-db12-4778-a5ff-b74d6fc9554b'}), lo=
g id: 5cd11a39
> 2017-01-19 08:48:18,282 WARN [org.ovirt.engine.core.vdsbroker.gluster.Gl=
usterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler3) [7471d7de] Could=
not associate brick 'lvh2:/data/gluster/0/brick' of volume '7dc6410d-8f2a-=
406c-812a-8235fa6f721c' with correct network as no gluster network found in=
cluster '57ff41c2-0297-039d-039c-000000000362'
> 2017-01-19 08:48:18,284 WARN [org.ovirt.engine.core.vdsbroker.gluster.Gl=
usterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler3) [7471d7de] Could=
not associate brick 'lvh3:/data/gluster/0/brick' of volume '7dc6410d-8f2a-=
406c-812a-8235fa6f721c' with correct network as no gluster network found in=
cluster '57ff41c2-0297-039d-039c-000000000362'
> 2017-01-19 08:48:18,285 WARN [org.ovirt.engine.core.vdsbroker.gluster.Gl=
usterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler3) [7471d7de] Could=
not associate brick 'lvh4:/data/gluster/0/brick' of volume '7dc6410d-8f2a-=
406c-812a-8235fa6f721c' with correct network as no gluster network found in=
cluster '57ff41c2-0297-039d-039c-000000000362'
> 2017-01-19 08:48:18,286 INFO [org.ovirt.engine.core.vdsbroker.gluster.Gl=
usterVolumesListVDSCommand] (DefaultQuartzScheduler3) [7471d7de] FINISH, Gl=
usterVolumesListVDSCommand, return: {7dc6410d-8f2a-406c-812a-8235fa6f721c=
=3Dorg.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntit=
y@2459e231, 000391cc-9946-47b3-82c9-af17da69d182=3Dorg.ovirt.engine.core.co=
mmon.businessentities.gluster.GlusterVolumeEntity@42990505}, log id: 5cd11a=
39
[1] http://www.ovirt.org/documentation/admin-guide/hosted-engine-backup-and=
-restore seems to be moved to http://www.ovirt.org/documentation/self-hoste=
d/chap-Backing_up_and_Restoring_an_EL-Based_Self-Hosted_Environment/#creati=
ng-a-new-self-hosted-engine-environment-to-be-used-as-the-restored-environm=
ent
sincerely,
Andr=E9-Sebastian Liebe
Technik / Innovation
Telefon: +49 30 40041-197
Telefax: +49 30 40041-111
gematik- Gesellschaft f=FCr Telematikanwendungen der Gesundheitskarte mbH
Friedrichstra=DFe 136, 10117 Berlin
Amtsgericht Berlin-Charlottenburg HRB 96351 B
Gesch=E4ftsf=FChrer: Alexander Beyer
------69AE9CF32CF9461733367ED13D0233A1
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
MIIP1AYJKoZIhvcNAQcCoIIPxTCCD8ECAQExDzANBglghkgBZQMEAgEFADALBgkq
hkiG9w0BBwGggg0fMIIGuDCCBKCgAwIBAgIPGReV3CJ0GxId21RMXMvcMA0GCSqG
SIb3DQEBCwUAMEUxCzAJBgNVBAYTAkNIMRUwEwYDVQQKEwxTd2lzc1NpZ24gQUcx
HzAdBgNVBAMTFlN3aXNzU2lnbiBHb2xkIENBIC0gRzIwHhcNMTQwOTE5MTQxMDI1
WhcNMjkwOTE1MTQxMDI1WjBUMQswCQYDVQQGEwJDSDEVMBMGA1UEChMMU3dpc3NT
aWduIEFHMS4wLAYDVQQDEyVTd2lzc1NpZ24gUGVyc29uYWwgR29sZCBDQSAyMDE0
IC0gRzIyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAnrf0T2JFsx4N
Pb+TUBInPgv9nUyF9dMJh7sQL66HybrB2devqNc1+Gq+fFT4tfBa9hbI+xqRRqCO
0LoQOJ3v3XMlOzQ3zTL+b3wgDvs/d8tZ7LWGOSYF480/rL1aaBIQFlrE62GDQvcm
kH/C8QxsSS+T+ga2FQhC/Br/btveNcP6cz87WyshF8IU/7sOKYqh3o5mbxI641R1
u3+zaiGq8A9620pSoW3b9P1Mf5t4z51ifqb+/QsYtDt60dw+mVES6sk8cl9VRLej
cuiXFyVJaj7YyITior33buzheHvzZdxaQSgeq0mIrvmXqtplZoqXQ12irR7xhuf/
w9UtFBbqCwIDAQABo4IClDCCApAwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQI
MAYBAf8CAQAwHQYDVR0OBBYEFNoy+Un4UcyYcWYM2c6225I/CUvvMB8GA1UdIwQY
MBaAFFsle5akZVF+uDnzwHhmXug65/DuMIH/BgNVHR8EgfcwgfQwR6BFoEOGQWh0
dHA6Ly9jcmwuc3dpc3NzaWduLm5ldC81QjI1N0I5NkE0NjU1MTdFQjgzOUYzQzA3
ODY2NUVFODNBRTdGMEVFMIGooIGloIGihoGfbGRhcDovL2RpcmVjdG9yeS5zd2lz
c3NpZ24ubmV0L0NOPTVCMjU3Qjk2QTQ2NTUxN0VCODM5RjNDMDc4NjY1RUU4M0FF
N0YwRUUlMkNPPVN3aXNzU2lnbiUyQ0M9Q0g/Y2VydGlmaWNhdGVSZXZvY2F0aW9u
TGlzdD9iYXNlP29iamVjdENsYXNzPWNSTERpc3RyaWJ1dGlvblBvaW50MF8GA1Ud
IARYMFYwVAYJYIV0AVkBAgEGMEcwRQYIKwYBBQUHAgEWOWh0dHA6Ly9yZXBvc2l0
b3J5LnN3aXNzc2lnbi5jb20vU3dpc3NTaWduLUdvbGQtQ1AtQ1BTLnBkZjCBxgYI
KwYBBQUHAQEEgbkwgbYwZAYIKwYBBQUHMAKGWGh0dHA6Ly9zd2lzc3NpZ24ubmV0
L2NnaS1iaW4vYXV0aG9yaXR5L2Rvd25sb2FkLzVCMjU3Qjk2QTQ2NTUxN0VCODM5
RjNDMDc4NjY1RUU4M0FFN0YwRUUwTgYIKwYBBQUHMAGGQmh0dHA6Ly9vY3NwLnN3
aXNzc2lnbi5uZXQvNUIyNTdCOTZBNDY1NTE3RUI4MzlGM0MwNzg2NjVFRTgzQUU3
RjBFRTANBgkqhkiG9w0BAQsFAAOCAgEArevbqCOPH6va0x4oEb4UviyXJzIokl5M
jRADlGMDFXaKm52mU+IZlJ9PjcqBiEXhjjD1oR8uV5aVRWXH+auI6JgQZz5L+u2a
309bDu3bcineP45vZ+nAuoDvulmx2nqjloCOturaks727w/nHCoz723eYH46fg//
q/a5QnOoBe+WIiZWT71TcycX4hiVcPwVqB53tL9IM2qe2balnCpZKorZWVHQXYWS
O5MIBaw/iMOwCCpflDRU75TruHPjfE4Bu4U9Nm3NBt3zcc5ykcQ8ZbJzkWLnSi2L
Y8r71Ulc3jxqyWOZQbgS14x/BzVQj4Rk9+X8KovGpooAFVIBR/bdU/z2IdZlENgo
PNpJokuEH5OS6UZSyNmhhXjI2QGXn91G+QV4++kreVz6heVOhDPgOoOfRXLYG9O/
9wqPMthmUQgcTrymp57bPpM0zIYpHxcKfxhoYprQdO3LuU1F2xkZE2vFLWVB9ugH
zXd1ADIKrrGgJXPfMluXT9wK8BzTcI5cOqhESt4Awq5q9fmiW3OUAROL0Ca0Z/sR
QcvvJ/ewlZvLdsymcfjV6JtN7/IBgIJ4D/Js4DRWCwAv1jhBiEGsRAlhiJmep//U
1HEaT9oredt9PYS1iKbkcMuDEtPuYXrQ+8OQmMEwMbOFboZMMTOVNOenLH+EiG6B
UnowTwqqjtAwggZfMIIFR6ADAgECAg9L4pTQgb3rW5+eyDb7b8IwDQYJKoZIhvcN
AQELBQAwVDELMAkGA1UEBhMCQ0gxFTATBgNVBAoTDFN3aXNzU2lnbiBBRzEuMCwG
A1UEAxMlU3dpc3NTaWduIFBlcnNvbmFsIEdvbGQgQ0EgMjAxNCAtIEcyMjAeFw0x
NjEwMDYwODAyMDRaFw0xNzEwMDYwODAyMDRaMIG2MQswCQYDVQQGEwJERTEPMA0G
A1UECBMGQmVybGluMQ8wDQYDVQQHEwZCZXJsaW4xEDAOBgNVBAoTB2dlbWF0aWsx
LDAqBgNVBAsTIyhaMSBTZWN1cmVNYWlsIEdhdGV3YXkgQ2VydGlmaWNhdGUpMR4w
HAYDVQQDExVBbmRyZSBTZWJhc3RpYW4gTGllYmUxJTAjBgkqhkiG9w0BCQEWFmFu
ZHJlLmxpZWJlQGdlbWF0aWsuZGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQDDUYQmrsYEHuCqhIhh5ZdAEFX8PsubBQIB61sGiENO3ccmlkETeb7S4fvZ
q4+zBYZEi3XSPqgLxuC/FBG+KjQPdrRxe/rDjRJNFMdJZBXPF6ukky3ftHAsSNPO
ysj0e4pShWZKTGmOT6hyilaVHhY6ZbjXHtClJJVLafssLfR3Ti7WkqYv/xt7grDR
b19FjZxwgmwum2769cvCX9QkBqq6Z8IGn0RRVWpqx54hLkz3huIRf9HrXMVEEnn4
IWV5JebN4PnNkJiKHdMyr3LAsrk1E9tYtcMnQxyu2hx3Kr4XokK7oO2Lzpts4H7S
kuOMmfR02vXCBlyHmdzVScuooquNAgMBAAGjggLJMIICxTAhBgNVHREEGjAYgRZh
bmRyZS5saWViZUBnZW1hdGlrLmRlMA4GA1UdDwEB/wQEAwIEsDATBgNVHSUEDDAK
BggrBgEFBQcDBDAdBgNVHQ4EFgQU0YbsGMI0C0ealqDM5iWHFOxy21kwHwYDVR0j
BBgwFoAU2jL5SfhRzJhxZgzZzrbbkj8JS+8wgf8GA1UdHwSB9zCB9DBHoEWgQ4ZB
aHR0cDovL2NybC5zd2lzc3NpZ24ubmV0L0RBMzJGOTQ5Rjg1MUNDOTg3MTY2MENE
OUNFQjZEQjkyM0YwOTRCRUYwgaiggaWggaKGgZ9sZGFwOi8vZGlyZWN0b3J5LnN3
aXNzc2lnbi5uZXQvQ049REEzMkY5NDlGODUxQ0M5ODcxNjYwQ0Q5Q0VCNkRCOTIz
RjA5NEJFRiUyQ089U3dpc3NTaWduJTJDQz1DSD9jZXJ0aWZpY2F0ZVJldm9jYXRp
b25MaXN0P2Jhc2U/b2JqZWN0Q2xhc3M9Y1JMRGlzdHJpYnV0aW9uUG9pbnQwXwYD
VR0gBFgwVjBUBglghXQBWQECAQYwRzBFBggrBgEFBQcCARY5aHR0cDovL3JlcG9z
aXRvcnkuc3dpc3NzaWduLmNvbS9Td2lzc1NpZ24tR29sZC1DUC1DUFMucGRmMIHX
BggrBgEFBQcBAQSByjCBxzBkBggrBgEFBQcwAoZYaHR0cDovL3N3aXNzc2lnbi5u
ZXQvY2dpLWJpbi9hdXRob3JpdHkvZG93bmxvYWQvREEzMkY5NDlGODUxQ0M5ODcx
NjYwQ0Q5Q0VCNkRCOTIzRjA5NEJFRjBfBggrBgEFBQcwAYZTaHR0cDovL2dvbGQt
cGVyc29uYWwtZzIub2NzcC5zd2lzc3NpZ24ubmV0L0RBMzJGOTQ5Rjg1MUNDOTg3
MTY2MENEOUNFQjZEQjkyM0YwOTRCRUYwDQYJKoZIhvcNAQELBQADggEBACjxz8QL
pCQ51jLVWShJDyvEsG3oVmhajtEuHHxV1M/0TQ2CPZgJ37juNCeyrCTxSAYtAr0l
bYMSds5EPdupijDkM9QVRJIw7JSygWmOnLieL6iCcWvRI4yACmZ6zyEptp3KkJnh
dAgV5RTSoY6vNKlFG93wZXA7AhJxwP2n08atbU8Rp6mbn5WG+L7VV3ARxAMdNvsJ
ISvzzYxsACkD8+O7YV1jNq+BkHvr6Zf3BAPSA2rhe4T87PvI3VAuP9CiMwOdnF1s
rp59Ar39KtoHTm74TnzKbZVzmg5J3ktSFW4HjYZXe5jTzD/l7LHUwL1q/1T0H/cF
zYDFPQ7KaWSkrTMxggJ5MIICdQIBATBnMFQxCzAJBgNVBAYTAkNIMRUwEwYDVQQK
EwxTd2lzc1NpZ24gQUcxLjAsBgNVBAMTJVN3aXNzU2lnbiBQZXJzb25hbCBHb2xk
IENBIDIwMTQgLSBHMjICD0vilNCBvetbn57INvtvwjANBglghkgBZQMEAgEFAKCB
5DAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNzAx
MjAwOTMxNDhaMC8GCSqGSIb3DQEJBDEiBCAjDolzoeLgX4jlF98HvhJDpnFC4Gk4
k88WmLcwl+0zaDB5BgkqhkiG9w0BCQ8xbDBqMAsGCWCGSAFlAwQBKjALBglghkgB
ZQMEARYwCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDAN
BggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDANBgkqhkiG9w0B
AQEFAASCAQAW3q74vOdMmKjAe4hDYOJKdKZADU+NqbAHIsMvVvFwMCa5edrkxSUt
V66myL81uyR8B7zFnL/tf6eV6jN8b3l/kX6H8XZaKO44TRx9vCWJzCI8A0XUCOFH
irM1H7OvUlxT9o4OL3F92M68wMIKstm8gEHmzIjcLbliWOgq8RYDB3oIKpYHQmzD
pRoaldfXfaPUc69d19ASgUkOLB9+hU9JFNMPYbfc7x1FlhjmAJ2F6uH0Sq8w7MRC
Duj2EaSIU6+qBfmY4tFvlykXqUoW7y4M7cSHXm9rBZbiO30FfCc7F5mzlz3LHGwU
cMS911+MZ7gkbdm6O09Fyx1AKc1Ue1VJ
------69AE9CF32CF9461733367ED13D0233A1--