Cannot get Ovirt 4.5 to work, how ever I try. Virgin install: no pki ca-cert gen, restoring: no OVN connection
6:06 a.m.
Hi there,
over the last months I've hunkered down to update my companies antiquated Ovirt 4.3.
To manage this in an orderly fashion we replicated the setup.
In the update process I always arrive at the same problem. Once I managed to solve it by
chance, but I cannot reproduce the solution.
The setup is Ovirt Engine running on a dedicated Centos-Stream-8 virtual machine managed
in VirtManager. The nodes are either OvirtNode 4.4 or 4.5. The problem exists on both.
Issue1:
Updating to 4.4 works without issue. Then, regardless whether I update by restoring to
Ovirt 4.5 or by updating the engine through the update path networks stop functioning and,
very peculiarly I get a very strange keymap in the vm console. It's no real keymap.
It's quertz, but # resolves as 3 and all kind of strange stuff. However, this can be
resolved on individual basis by setting the vm-console keymap to de (german). Connected
hosts and new hosts always dispaly "OVN connected: No".
The error log hints at some kind of ssl error. I either get dropping connections, or
protocol miss-matches in the node log. I deactivated Ovirt4.4-repositories on the engine
and did a distro-sync, because I found an old bug-report that implicated protocol
mismatched may result from unclean python-library versioning.
I reenrolled certificates, I reinstalled the host and still cannot get a connection:
Logs on host:
/var/log/ovn-controller.log:
2023-12-19T11:27:14.245Z|00018|memory|INFO|6604 kB peak resident set size after 15.1
seconds
2023-12-19T11:27:14.245Z|00019|memory|INFO|idl-cells:100
2023-12-19T11:29:34.483Z|00001|vlog|INFO|opened log file /var/log/ovn/ovn-controller.log
2023-12-19T11:29:34.512Z|00002|reconnect|INFO|unix:/run/openvswitch/db.sock:
connecting...
2023-12-19T11:29:34.513Z|00003|reconnect|INFO|unix:/run/openvswitch/db.sock: connected
2023-12-19T11:29:34.517Z|00004|main|INFO|OVN internal version is : [21.12.3-20.21.0-61.4]
2023-12-19T11:29:34.517Z|00005|main|INFO|OVS IDL reconnected, force recompute.
2023-12-19T11:29:34.573Z|00006|reconnect|INFO|ssl:127.0.0.1:6642: connecting...
2023-12-19T11:29:34.573Z|00007|main|INFO|OVNSB IDL reconnected, force recompute.
2023-12-19T11:29:34.573Z|00008|reconnect|INFO|ssl:127.0.0.1:6642: connection attempt
failed (Connection refused)
2023-12-19T11:29:35.575Z|00009|reconnect|INFO|ssl:127.0.0.1:6642: connecting...
2023-12-19T11:29:35.589Z|00010|reconnect|INFO|ssl:127.0.0.1:6642: connection attempt
failed (Connection refused)
2023-12-19T11:29:35.589Z|00011|reconnect|INFO|ssl:127.0.0.1:6642: waiting 2 seconds before
reconnect
2023-12-19T11:29:37.592Z|00012|reconnect|INFO|ssl:127.0.0.1:6642: connecting...
2023-12-19T11:29:37.592Z|00013|reconnect|INFO|ssl:127.0.0.1:6642: connection attempt
failed (Connection refused)
2023-12-19T11:29:37.592Z|00014|reconnect|INFO|ssl:127.0.0.1:6642: waiting 4 seconds before
reconnect
2023-12-19T11:29:41.596Z|00015|reconnect|INFO|ssl:127.0.0.1:6642: connecting...
2023-12-19T11:29:41.596Z|00016|reconnect|INFO|ssl:127.0.0.1:6642: connection attempt
failed (Connection refused)
2023-12-19T11:29:41.596Z|00017|reconnect|INFO|ssl:127.0.0.1:6642: continuing to reconnect
in the background but suppressing further logging
/var/log/openvswitch/ovsdb-server.log:
2023-12-19T11:26:56.889Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovsdb-server.log
2023-12-19T11:26:56.915Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.15.8
2023-12-19T11:27:06.922Z|00003|memory|INFO|20624 kB peak resident set size after 10.0
seconds
2023-12-19T11:27:06.922Z|00004|memory|INFO|cells:128 monitors:5 sessions:3
2023-12-19T11:29:30.771Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovsdb-server.log
2023-12-19T11:29:30.813Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.15.8
2023-12-19T11:29:31.047Z|00003|jsonrpc|WARN|unix#0: receive error: Connection reset by
peer
2023-12-19T11:29:31.047Z|00004|reconnect|WARN|unix#0: connection dropped (Connection reset
by peer)
2023-12-19T11:29:32.821Z|00005|jsonrpc|WARN|unix#2: receive error: Connection reset by
peer
2023-12-19T11:29:32.821Z|00006|reconnect|WARN|unix#2: connection dropped (Connection reset
by peer)
2023-12-19T11:29:33.139Z|00007|jsonrpc|WARN|unix#4: receive error: Connection reset by
peer
2023-12-19T11:29:33.139Z|00008|reconnect|WARN|unix#4: connection dropped (Connection reset
by peer)
2023-12-19T11:29:40.864Z|00009|memory|INFO|23108 kB peak resident set size after 10.1
seconds
2023-12-19T11:29:40.864Z|00010|memory|INFO|cells:128 monitors:4 sessions:3
Logs on engine:
/var/log/ovn/ovsdb-server-nb.log:
2023-12-18T19:36:23.056Z|00001|vlog|INFO|opened log file /var/log/ovn/ovsdb-server-nb.log
2023-12-18T19:36:23.784Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.15.8
2023-12-18T19:36:24.275Z|00003|jsonrpc|WARN|unix#0: receive error: Connection reset by
peer
2023-12-18T19:36:24.276Z|00004|reconnect|WARN|unix#0: connection dropped (Connection reset
by peer)
2023-12-18T19:36:33.808Z|00005|memory|INFO|22528 kB peak resident set size after 10.8
seconds
2023-12-18T19:36:33.808Z|00006|memory|INFO|cells:99 monitors:2 sessions:1
/var/log/ovirt-engine/engine.log (currently unable to start vms. normally not the case in
my tests but error message seems related)
2023-12-19 06:49:17,982-05 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [43d1e22d]
EVENT_ID: PROVIDER_SYNCHRONIZATION_STARTED(223), Provider ovirt-provider-ovn
synchronization started.
2023-12-19 06:49:18,122-05 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [43d1e22d]
EVENT_ID: PROVIDER_SYNCHRONIZATION_ENDED(224), Provider ovirt-provider-ovn synchronization
ended.
2023-12-19 06:49:18,122-05 ERROR
[org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [43d1e22d]
Command 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand'
failed: EngineException: (Failed with error Unsupported or unrecognized SSL message and
code 5050)
Issue2:
When installing ovirt4.5 engine-setup always fails in pki-phase because no new root cert
is generated. I believe it ultimately say apache.ca is missing. This is also on a fresh
Centos-Stream-8 machine following official install instructions.
Please help. :)
4:53 a.m.
I have made progress on the network issue and it turns out, there is something missing
from the dependencies. It's a python library that redefines the behavior of a
function. This has the effect of not producing an interpreter error when missing...
I got the networks to function by installing
dnf install python3.11-netaddr.noarch python3-netaddr.noarch
I tried these packages before via pip, but it didn't seem to do the trick. Sure some
additional configuration was missing. This solution has been mentioned somewhere before,
but it was via pip and I also don't have the resource at hand. So there's no
link.
I haven't looked into which of the two packages is the actual missing one. However, I
find it odd that this seems to suggest that the devs are not running recurring automated
tests on virgin machines with their supposedly supported OS installed...
Will report back when I understand the keymap problem.
8:27 a.m.
Concerning the keymap issue:
setting ClientModeConsoleDefault to spice will evade the problem by using the keymap
configured on the remote system aka the virtual machine
# engine-config -s ClientModeConsoleDefault=spice
otherwise VNC is used as standard and will only function properly if the keymap property
in _ovirt-engine>virtual machines>edit>console>vnc keyboard layout_ is set to
the same keymap as your local keymap on the client aka your workstation.
Probably works as designed. Don't know if standard console has changed between major
releases.
121
days inactive
137
days old
3 comments
1 participants
participants (1)
-
julian.steiner＠conesphere.com