7:18 a.m.
I reinstalled OS all nodes CentoOS 8 streams.
installed cockpit engine and ran through HCI deploy wizard with gluster. this deployed
4.4 version of engine.. then said 4.5 version available.
I try to add hosts and get error "Error while executing action: Server
thor.penguinpages.local is already part of another cluster." (or other node).
I ssh into ovirt engine host and attempt to ssh from that host to three nodes of HCI
cluster and it works fine. I added ssh keys for passwordless login but no change
I change cluster then datacenter to 4.5. Then attempt to run upgrade but get same
error.
Seems like chicken / egg issue. I can't get nodes. I can't upgrade 4.4. to 4.5
because I have no nodes to allow maintenance mode. And engine shows "no
updates" needed
[root@ovirte01 ~]# yum update
Last metadata expiration check: 1:39:16 ago on Mon 08 Mar 2021 08:36:40 AM EST.
Dependencies resolved.
Nothing to do.
Complete!
Suggestions?
7:25 a.m.
New subject: HCI Fresh Deploy - Cannot upgrade 4.4. to 4.5 or add more hosts
<<< Update>>>
as I try to add node. I am able to add fingerprint .. and below is output from engine
[root@ovirte01 ~]# tail -f /var/log/ovirt-engine/server.log
...
2021-03-08 10:23:21,542-05 WARNING [javax.persistence.spi] (default task-6)
javax.persistence.spi::No valid providers found.
2021-03-08 10:23:21,719-05 INFO
[org.apache.sshd.common.io.DefaultIoServiceFactoryFactory] (default task-6) No
detected/configured IoServiceFactoryFactory using Nio2ServiceFactoryFactory
2021-03-08 10:23:21,744-05 INFO
[org.apache.sshd.client.config.hosts.DefaultConfigFileHostEntryResolver] (default task-6)
resolveEffectiveResolver(dummy@thor.penguinpages.local:22) loaded 0 entries from
/var/lib/ovirt-engine/.ssh/config
2021-03-08 10:23:30,737-05 WARN [org.apache.sshd.client.session.ClientConnectionService]
(sshd-SshClient[4f22a453]-nio2-thread-3)
globalRequest(ClientConnectionService[ClientSessionImpl[root@thor.penguinpages.local/172.16.100.101:22]])[hostkeys-00@openssh.com,
want-reply=false] failed (SshException) to process: EdDSA provider not supported
Does anyone know where else logs would be for engine failure to add node?
7:41 a.m.
New subject: HCI Fresh Deploy - Cannot upgrade 4.4. to 4.5 or add more hosts
Their may be aspect of this where HCI engine.. composes bricks of gluster into engine
hosted storage... and then is not able to add hosts into cluster.
Below is looping error :
[root@ovirte01 ~]# tail -f /var/log/ovirt-engine/engine.log
2021-03-08 10:40:19,192-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] START,
GlusterServersListVDSCommand(HostName = medusa.penguinpages.local,
VdsIdVDSCommandParametersBase:{hostId='ea26e8ad-e762-4852-bf71-b0a6b2b69853'}),
log id: 76bd303b
2021-03-08 10:40:19,395-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] FINISH,
GlusterServersListVDSCommand, return: [172.16.101.103/24:CONNECTED,
thorst.penguinpages.local:CONNECTED, odinst.penguinpages.local:CONNECTED], log id:
76bd303b
2021-03-08 10:40:19,400-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] START,
GlusterVolumesListVDSCommand(HostName = medusa.penguinpages.local,
GlusterVolumesListVDSParameters:{hostId='ea26e8ad-e762-4852-bf71-b0a6b2b69853'}),
log id: 2ebcd203
2021-03-08 10:40:19,507-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not add
brick 'thorst.penguinpages.local:/gluster_bricks/data/data' to volume
'78117441-467f-4c9c-ae10-4608de688404' - server uuid
'165ecdcd-10c1-4b34-aefa-9a0a6d3d8751' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,508-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not add
brick 'odinst.penguinpages.local:/gluster_bricks/data/data' to volume
'78117441-467f-4c9c-ae10-4608de688404' - server uuid
'835293d4-38b8-4e1b-80ff-9e654effa3c3' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,511-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not
associate brick 'medusast.penguinpages.local:/gluster_bricks/data/data' of volume
'78117441-467f-4c9c-ae10-4608de688404' with correct network as no gluster network
found in cluster '91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,511-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not add
brick 'thorst.penguinpages.local:/gluster_bricks/engine/engine' to volume
'40f9f0b8-1861-45d6-b63b-d59ec264422d' - server uuid
'165ecdcd-10c1-4b34-aefa-9a0a6d3d8751' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,512-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not add
brick 'odinst.penguinpages.local:/gluster_bricks/engine/engine' to volume
'40f9f0b8-1861-45d6-b63b-d59ec264422d' - server uuid
'835293d4-38b8-4e1b-80ff-9e654effa3c3' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,514-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not
associate brick 'medusast.penguinpages.local:/gluster_bricks/engine/engine' of
volume '40f9f0b8-1861-45d6-b63b-d59ec264422d' with correct network as no gluster
network found in cluster '91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,515-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not add
brick 'thorst.penguinpages.local:/gluster_bricks/vmstore/vmstore' to volume
'59cb23d6-325b-48ea-a03c-6fb43809f714' - server uuid
'165ecdcd-10c1-4b34-aefa-9a0a6d3d8751' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,515-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not add
brick 'odinst.penguinpages.local:/gluster_bricks/vmstore/vmstore' to volume
'59cb23d6-325b-48ea-a03c-6fb43809f714' - server uuid
'835293d4-38b8-4e1b-80ff-9e654effa3c3' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,517-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] Could not
associate brick 'medusast.penguinpages.local:/gluster_bricks/vmstore/vmstore' of
volume '59cb23d6-325b-48ea-a03c-6fb43809f714' with correct network as no gluster
network found in cluster '91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:19,518-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [] FINISH,
GlusterVolumesListVDSCommand, return:
{78117441-467f-4c9c-ae10-4608de688404=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b562e3bc,
59cb23d6-325b-48ea-a03c-6fb43809f714=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8eb20b93,
40f9f0b8-1861-45d6-b63b-d59ec264422d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d1f4bf10},
log id: 2ebcd203
2021-03-08 10:40:34,543-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] START,
GlusterServersListVDSCommand(HostName = medusa.penguinpages.local,
VdsIdVDSCommandParametersBase:{hostId='ea26e8ad-e762-4852-bf71-b0a6b2b69853'}),
log id: 2b2e9659
2021-03-08 10:40:34,747-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] FINISH,
GlusterServersListVDSCommand, return: [172.16.101.103/24:CONNECTED,
thorst.penguinpages.local:CONNECTED, odinst.penguinpages.local:CONNECTED], log id:
2b2e9659
2021-03-08 10:40:34,750-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] START,
GlusterVolumesListVDSCommand(HostName = medusa.penguinpages.local,
GlusterVolumesListVDSParameters:{hostId='ea26e8ad-e762-4852-bf71-b0a6b2b69853'}),
log id: 2c39f795
2021-03-08 10:40:34,858-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not add
brick 'thorst.penguinpages.local:/gluster_bricks/data/data' to volume
'78117441-467f-4c9c-ae10-4608de688404' - server uuid
'165ecdcd-10c1-4b34-aefa-9a0a6d3d8751' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,858-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not add
brick 'odinst.penguinpages.local:/gluster_bricks/data/data' to volume
'78117441-467f-4c9c-ae10-4608de688404' - server uuid
'835293d4-38b8-4e1b-80ff-9e654effa3c3' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,861-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not
associate brick 'medusast.penguinpages.local:/gluster_bricks/data/data' of volume
'78117441-467f-4c9c-ae10-4608de688404' with correct network as no gluster network
found in cluster '91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,862-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not add
brick 'thorst.penguinpages.local:/gluster_bricks/engine/engine' to volume
'40f9f0b8-1861-45d6-b63b-d59ec264422d' - server uuid
'165ecdcd-10c1-4b34-aefa-9a0a6d3d8751' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,863-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not add
brick 'odinst.penguinpages.local:/gluster_bricks/engine/engine' to volume
'40f9f0b8-1861-45d6-b63b-d59ec264422d' - server uuid
'835293d4-38b8-4e1b-80ff-9e654effa3c3' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,866-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not
associate brick 'medusast.penguinpages.local:/gluster_bricks/engine/engine' of
volume '40f9f0b8-1861-45d6-b63b-d59ec264422d' with correct network as no gluster
network found in cluster '91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,867-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not add
brick 'thorst.penguinpages.local:/gluster_bricks/vmstore/vmstore' to volume
'59cb23d6-325b-48ea-a03c-6fb43809f714' - server uuid
'165ecdcd-10c1-4b34-aefa-9a0a6d3d8751' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,867-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not add
brick 'odinst.penguinpages.local:/gluster_bricks/vmstore/vmstore' to volume
'59cb23d6-325b-48ea-a03c-6fb43809f714' - server uuid
'835293d4-38b8-4e1b-80ff-9e654effa3c3' not found in cluster
'91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,871-05 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] Could not
associate brick 'medusast.penguinpages.local:/gluster_bricks/vmstore/vmstore' of
volume '59cb23d6-325b-48ea-a03c-6fb43809f714' with correct network as no gluster
network found in cluster '91fde852-8013-11eb-8054-00163e7e2056'
2021-03-08 10:40:34,871-05 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87) [] FINISH,
GlusterVolumesListVDSCommand, return:
{78117441-467f-4c9c-ae10-4608de688404=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b562e3bc,
59cb23d6-325b-48ea-a03c-6fb43809f714=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8eb20b93,
40f9f0b8-1861-45d6-b63b-d59ec264422d=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d1f4bf10},
log id: 2c39f795
1145
days inactive
1145
days old
2 comments
1 participants
participants (1)
-
penguin pages