12:45 a.m.
Your logs suggest that VDSM hasn't detected the removal of the network
yet. Did you try clicking on the "Sync Host Networks" button on the
non-operational hosts?
If you've changed the network config, VDSM may have failed to set up
the host properly. What does the output of "ip addr show" on these
hosts?
Manually removing the network from the hosts is only a temporary fix.
VDSM will attempt to recreate the network on restart if VDSM's config
isn't sync'd with the engine.
-Patrick Hibbs
On Wed, 2022-04-27 at 12:36 +0300, Andrei Verovski wrote:
Hi,
I run into nasty and unexpected problem (oVirt 4.4.7.6-1.el8 ) and 2
non-operational node hosts, which seem relate to ghost network
glitch.
Everything worked fine for a very long time until I did the
following.
1) Made a backup of several VMs which have network “CloudLink-ISP2”.
2) Moved VM disks into another node hosts, which don’t have link
“CloudLink-ISP2”, and forgot (preliminary to move) to re-configure
network interface (remove link to “CloudLink-ISP2”) as I did before..
3) Now I have 2 hosts in non-operational mode which is a really big
problem - there are only 3 nodes total in a cluster.
I tried to fix this by adding 3rd network “CloudLink-ISP2” to each
non-operational hosts, and connected ethernet interfaces to a switch.
Unfortunately, it didn’t helped either because for whatever reason
link did not go up.
How I can remove any reference to “CloudLink-ISP2” from node11 and
node14? They don’t have any VMs needed that “CloudLink-ISP2”. I
edited even inactive VMs and removed “CloudLink-ISP2”.
May be there is an option to ignore missing or downed links whn
activating node hosts?
Thanks in advance for any help.
——————————
Logs:
2022-04-27 11:16:59,769+03 ERROR
[org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (EE-
ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-51)
[5ab20328] Host 'node14' is set to Non-Operational, it is missing the
following networks: 'CloudLink-ISP2'
2022-04-27 11:16:59,790+03 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector
] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-
Thread-51) [5ab20328] EVENT_ID: VDS_SET_NONOPERATIONAL_NETWORK(519),
Host node14 does not comply with the cluster ClusterRiga11 networks,
the following networks are missing on host: 'CloudLink-ISP2'
2022-04-27 11:16:59,884+03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector
] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-
Thread-51) [55610e1e] EVENT_ID: VDS_DETECTED(13), Status of host
node14 was set to NonOperational.
2022-04-27 11:16:59,907+03 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-
ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-51)
[5d6ef791] Host 'node14'(aa871d44-94e2-4fdb-aeb3-ca0ae8dc568f) is
already in NonOperational status for reason 'NETWORK_UNREACHABLE'.
SetNonOperationalVds command is skipped.
2022-04-27 11:12:26,050+03 ERROR
[org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (EE-
ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-19)
[5400352e] Host 'node11' is set to Non-Operational, it is missing the
following networks: 'CloudLink-ISP2'
2022-04-27 11:12:26,070+03 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector
] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-
Thread-19) [5400352e] EVENT_ID: VDS_SET_NONOPERATIONAL_NETWORK(519),
Host node11 does not comply with the cluster ClusterRiga11 networks,
the following networks are missing on host: 'CloudLink-ISP2'
2022-04-27 11:12:26,192+03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector
] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-
Thread-19) [57d79e54] EVENT_ID: VDS_DETECTED(13), Status of host
node11 was set to NonOperational.
2022-04-27 11:12:26,214+03 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-
ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-19)
[151821e3] Host 'node11'(3c854f9c-2cdd-423e-bca0-37964ba76702) is
already in NonOperational status for reason 'NETWORK_UNREACHABLE'.
SetNonOperationalVds command is skipped.
2022-04-27 11:51:10,539+03 INFO
[org.ovirt.engine.core.vdsbroker.VdsManager] (EE-
ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-62)
[] Clearing domains data for host node14
2022-04-27 11:51:10,539+03 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-
ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-62)
[] Host 'node14' moved to Non-Operational state because interface/s
which are down are needed by required network/s in the current
cluster: 'eno3 (CloudLink-ISP2)'
2022-04-27 11:51:10,569+03 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector
] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-
Thread-62) [] EVENT_ID: VDS_SET_NONOPERATIONAL_IFACE_DOWN(603), Host
node14 moved to Non-Operational state because interfaces which are
down are needed by required networks in the current cluster: 'eno3
(CloudLink-ISP2)'.
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/EGENEHBHJYP...