[ovirt-users] ovirt 3.6.3.4 and VLAN are not friends

[Fabrice Bacchella]

I'm trying to bring some VLAN to ovirts hosts.

I have one native vlan and two tagged vlan.

When the tagged vlan are not configured in ovirt, I can add hosts easily. As soon as I add the two tagged one, every things run amok. New hosts network are broken, network configuration is untouched and the host is marked as non operationnal.

In engine.log, I get:

2016-03-09 10:25:34,376 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HostSetupNetworksVDSCommand] (default task-20) [da910e3] FINISH, HostSetupNetworksVDSCommand, log id: 7c267202
2016-03-09 10:25:34,383 INFO  [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to nb0104.prod.exalead.com/10.83.17.27
2016-03-09 10:25:34,404 INFO  [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to nb0104.prod.exalead.com/10.83.17.27
2016-03-09 10:25:36,401 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default task-20) [da910e3] Error: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request
2016-03-09 10:25:36,401 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default task-20) [da910e3] Exception: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_95]
        at java.util.concurrent.FutureTask.get(FutureTask.java:202) [rt.jar:1.7.0_95]
        at org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.get(FutureVDSCommand.java:72) [vdsbroker.jar:]

and a lot of them like that. I even get a :
2016-03-09 10:26:56,830 INFO  [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to nb0104.prod.exalead.com/10.83.17.27
2016-03-09 10:26:56,830 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) [] Unable to process messages
2016-03-09 10:26:56,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default task-20) [da910e3] Error: java.lang.NullPointerException
2016-03-09 10:26:56,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default task-20) [da910e3] Exception: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_95]
        at java.util.concurrent.FutureTask.get(FutureTask.java:202) [rt.jar:1.7.0_95]
        at org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.get(FutureVDSCommand.java:72) [vdsbroker.jar:]
        at org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.poll(HostSetupNetworkPoller.java:30) [bll.jar:]
        at org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand.invokeSetupNetworksCommand(HostSetupNetworksCommand.java:423) [bll.jar:]
        at org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand.executeCommand(HostSetupNetworksCommand.java:256) [bll.jar:]
....
Caused by: java.lang.NullPointerException
        at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:121) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:114) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:73) [vdsm-jsonrpc-java-client.jar:]
        at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:87) [vdsbroker.jar:]
        at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:1016) [vdsbroker.jar:]
        at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:1012) [vdsbroker.jar:]
...
2016-03-09 10:26:56,860 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (default task-20) [da910e3] Command 'PollVDSCommand(HostName = nb0104, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='8aed4c69-3a1d-4f00-95c4-79bcd3596ef9'})' execution failed: java.util.concurrent.ExecutionException: java.lang.NullPointerException


And latter:

    2016-03-09 10:37:28,635 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [47d52cbc] Correlation ID: 47d52cbc, Job ID: e1f27868-08c8-488a-b70c-7c9d8f06c08b, Call Stack: null, Custom Event ID: -1, Message: Host nb0104 does not comply with the cluster Default networks, the following networks are missing on host: 'DMZ,Extranet,ovirtmgmt'

I try to finish the network configuration manuall, , but some time it works, sometime it fails. When it fails I get a timeout, and operations are not possible any more on the host.
I'm getting a:

	Cannot setup Networks. Another Setup Networks process in progress on the host. Please try later after refreshing the host network capabilities.

In engine.log, I get:

    2016-03-09 10:38:13,470 WARN  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-15) [64ef0a3a] CanDoAction of action 'HostSetupNetworks' failed for user FA4 at apachesso. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,ACTION_TYPE_FAILED_SETUP_NETWORKS_IN_PROGRESS

The lock stay forever until i restart the ovirt-engine.

In the host, in mom.log, I got

2016-03-09 10:26:57,960 - mom.vdsmInterface - ERROR - Cannot connect to VDSM! [Errno 111] Connection refused
2016-03-09 10:26:57,960 - mom.GuestManager - INFO - Guest Manager starting: multi-thread
2016-03-09 10:26:57,963 - mom.vdsmInterface - ERROR - Cannot connect to VDSM! [Errno 111] Connection refused


I'm running ovirt 3.6.3.4 on Centos 7.2, I reinstalled everything, even the OS, with a empty database.