On Wed, 27 Feb 2019 13:24:14 +0200 Juhani Rautiainen <juhani.rautiainen@gmail.com> wrote:
On Wed, Feb 27, 2019 at 10:49 AM Dominik Holler <dholler@redhat.com> wrote:
I just copied that ifcfg-ovirtmgmt file from second node and fixed IP-address to correct one before doing ifdown/ifup. That file had changed to DHCP so my instinct was trying to correct that one.
Please let oVirt doing the work for you. If you interface to oVirt is the web UI, please use the dialog "Edit Managment Network: ovirtmgmt" which opens by clicking on the pencil symbol next to ovirtmgmt in Compute > Hosts > xxx > Network Interfaces > Setup Host Networks This will enable oVirt to recognize this change as intended.
Problem was that oVirt couldn't do the work anymore. It had for some reason switched that node to using DHCP addresses. DHCP gave totally different address to the node which was not known by ovirt engine. This is why I tried above change because I had lost connections to node after the resync. I had to use HP ILO console to see what's going on and found out that it had switched to DHCP and had wrong. And it used ILO fencing to boot the server because it couldn't reach it (which took many aftive vm's down). After the boot it still couldn't connect because address was still given by DHCP.
You did the right thing to establish a temporary connection of host and engine. As soon as the engine is able to communicate with the host, oVirt's web ui (or REST-API) should be used to configure the management network address and to ensure, that the host is in sync with oVirt's config.
What I'm wondering why it switched to DHCP when it had had static since first minute?
This is a valid question.
I noticed that one node had ovirtmgmt network unsynchronized. I tried
oVirt detected a difference between the expected configuration and applied configuration. This might happen if the interface configuration is change directly on the host instead of using oVirt Engine.
to resynchronize it.
If you have the vdsm.log, the relevant lines start at the pattern Calling 'Host.setupNetworks' and ends at the pattern FINISH getCapabilities
This led the node being rebooted by HP ILO. After reboot the node came up with DHCP address.
I have no idea why this happened, maybe there is a trace in /var/log/messages ? VDSM would revert a change in management networking configuration, if after the change communication to engine is broken. But if the host is rebooted during the change, there might be a problem. If you think you triggered Bug 1664106 - [Network] Host reboot while setup networks leads to host stuck on status 'connecting' you logfiles would be very helpful, if they are attached to the bug. Thanks!
Maybe no required anymore, since you described very precise what you did.
Or not clearly enough.
Thanks, -Juhani