OVS switch support is experimental at this stage and in some cases when trying to change from one switch to the other, it fails. It was also not checked against a hosted engine setup, which handles networking a bit differently for the management network (ovirtmgmt). Nevertheless, we are interested in understanding all the problems that exists today, so if you can, please share the supervdsm log, it has the interesting networking traces. We plan to block cluster switch editing until these problems are resolved. It will be only allowed to define a new cluster as OVS, not convert an existing one from Linux Bridge to OVS. On Fri, Mar 9, 2018 at 9:54 AM, Thomas Davis <tadavis(a)lbl.gov&gt; wrote: > I'm getting further along with 4.2.2rc3 than the 4.2.1 when it comes to > hosted engine and vlans.. it actually does install > under 4.2.2rc3. > > But it's a complete failure when I switch the cluster from Linux > Bridge/Legacy to OVS. The first time I try, vdsm does > not properly configure the node, it's all messed up. > > I'm getting this in vdsmd logs: > > 2018-03-08 23:12:46,610-0800 INFO (jsonrpc/7) [api.network] START > setupNetworks(networks={u'ovirtmgmt': {u'ipv6autoconf': True, u'nic': > u'eno1', u'vlan': u'50', u'ipaddr': u'192.168.85.49', u'switch': u'ovs', > u'mtu': 1500, u'netmask': u'255.255.252.0', u'dhcpv6': False, u'STP': > u'no', u'bridged': u'true', u'gateway': u'192.168.85.254', u'defaultRoute': > True}}, bondings={}, options={u'connectivityCheck': u'true', > u'connectivityTimeout': 120}) from=::ffff:192.168.85.24,56806, > flow_id=4147e25f-0a23-4f47-a0a4-d424a3437d11 (api:46) > > 2018-03-08 23:12:52,449-0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] > RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573) > > 2018-03-08 23:12:52,511-0800 INFO (jsonrpc/7) [api.network] FINISH > setupNetworks error=[Errno 19] ovirtmgmt is not present in the system > from=::ffff:192.168.85.24,56806, flow_id=4147e25f-0a23-4f47-a0a4-d424a3437d11 > (api:50) > 2018-03-08 23:12:52,512-0800 ERROR (jsonrpc/7) [jsonrpc.JsonRpcServer] > Internal server error (__init__:611) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line > 606, in _handle_request > res = method(**params) > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, > in _dynamicMethod > result = fn(*methodArgs) > File "<string>", line 2, in setupNetworks > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, > in method > ret = func(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1527, in > setupNetworks > supervdsm.getProxy().setupNetworks(networks, bondings, options) > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line > 55, in __call__ > return callMethod() > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line > 53, in <lambda> > **kwargs) > File "<string>", line 2, in setupNetworks > File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in > _callmethod > raise convert_to_error(kind, result) > IOError: [Errno 19] ovirtmgmt is not present in the system > 2018-03-08 23:12:52,512-0800 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] > RPC call Host.setupNetworks failed (error -32603) in 5.90 seconds > (__init__:573) > 2018-03-08 23:12:54,769-0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] > RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573) > 2018-03-08 23:12:54,772-0800 INFO (jsonrpc/5) [api.host] START > getCapabilities() from=::1,45562 (api:46) > 2018-03-08 23:12:54,906-0800 INFO (jsonrpc/5) [api.host] FINISH > getCapabilities error=[Errno 19] ovirtmgmt is not present in the system > from=::1,45562 (api:50) > 2018-03-08 23:12:54,906-0800 ERROR (jsonrpc/5) [jsonrpc.JsonRpcServer] > Internal server error (__init__:611) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line > 606, in _handle_request > res = method(**params) > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, > in _dynamicMethod > result = fn(*methodArgs) > File "<string>", line 2, in getCapabilities > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, > in method > ret = func(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1339, in > getCapabilities > c = caps.get() > File "/usr/lib/python2.7/site-packages/vdsm/host/caps.py", line 168, > in get > net_caps = supervdsm.getProxy().network_caps() > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line > 55, in __call__ > return callMethod() > File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line > 53, in <lambda> > **kwargs) > File "<string>", line 2, in network_caps > File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in > _callmethod > raise convert_to_error(kind, result) > IOError: [Errno 19] ovirtmgmt is not present in the system > > So something is dreadfully wrong with the bridge to ovs conversion in > 4.2.2rc3. > > thomas > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users >

Well, I just hit https://bugzilla.redhat.com/show_bug.cgi?id=1513991 And it's been closed, which means with vdsm-4.20.17-1.el7.centos.x86_64 OVS networking is totally borked..

I know OVS is Experimental, but it worked in 4.1.x, and now we have to do a step back to legacy bridge just to use 4.2.x, which in a vlan environment just wreaks havoc (every VLAN need's a unique mac assigned to the bridge, which vdsm does not do, so suddenly you get the kernel complaining about seeing it's mac address several times.)

There is zero documentation on how to use OVN instead of OVS.

thomas On 03/13/2018 09:22 AM, Thomas Davis wrote: > > I'll work on it some more. I have 2 different clusters in the data center > (1 is the Hosted Engine systems, another is not..) I had trouble with both. > I'll try again on the non-hosted engine cluster to see what it is doing. I > have it working in 4.1, but we are trying to do a clean wipe since the 4.1 > engine has been upgraded so many times from v3.5 plus we want to move to > hosted-engine-ha from a single engine node and the ansible modules/roles > (which also have problems..) > > thomas > > On Tue, Mar 13, 2018 at 6:27 AM, Edward Haas <ehaas(a)redhat.com > <mailto:ehaas@redhat.com>> wrote: > > > OVS switch support is experimental at this stage and in some cases > when trying to change from one switch to the other, it fails. > It was also not checked against a hosted engine setup, which handles > networking a bit differently for the management network (ovirtmgmt). > Nevertheless, we are interested in understanding all the problems > that exists today, so if you can, please share the supervdsm log, it > has the interesting networking traces. > > We plan to block cluster switch editing until these problems are > resolved. It will be only allowed to define a new cluster as OVS, > not convert an existing one from Linux Bridge to OVS. > > On Fri, Mar 9, 2018 at 9:54 AM, Thomas Davis <tadavis(a)lbl.gov > <mailto:tadavis@lbl.gov>> wrote: > > I'm getting further along with 4.2.2rc3 than the 4.2.1 when it > comes to hosted engine and vlans.. it actually does install > under 4.2.2rc3. > > But it's a complete failure when I switch the cluster from Linux > Bridge/Legacy to OVS. The first time I try, vdsm does > not properly configure the node, it's all messed up. > > I'm getting this in vdsmd logs: > > 2018-03-08 23:12:46,610-0800 INFO (jsonrpc/7) [api.network] > START setupNetworks(networks={u'ovirtmgmt': {u'ipv6autoconf': > True, u'nic': u'eno1', u'vlan': u'50', u'ipaddr': > u'192.168.85.49', u'switch': u'ovs', u'mtu': 1500, u'netmask': > u'255.255.252.0', u'dhcpv6': False, u'STP': u'no', u'bridged': > u'true', u'gateway': u'192.168.85.254', u'defaultRoute': True}}, > bondings={}, options={u'connectivityCheck': u'true', > u'connectivityTimeout': 120}) from=::ffff:192.168.85.24,56806, > flow_id=4147e25f-0a23-4f47-a0a4-d424a3437d11 (api:46) > > 2018-03-08 23:12:52,449-0800 INFO (jsonrpc/2) > [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 > seconds (__init__:573) > > 2018-03-08 23:12:52,511-0800 INFO (jsonrpc/7) [api.network] > FINISH setupNetworks error=[Errno 19] ovirtmgmt is not present > in the system from=::ffff:192.168.85.24,56806, > flow_id=4147e25f-0a23-4f47-a0a4-d424a3437d11 (api:50) > 2018-03-08 23:12:52,512-0800 ERROR (jsonrpc/7) > [jsonrpc.JsonRpcServer] Internal server error (__init__:611) > Traceback (most recent call last): > File > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line > 606, in _handle_request > res = method(**params) > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", > line 201, in _dynamicMethod > result = fn(*methodArgs) > File "<string>", line 2, in setupNetworks > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", > line 48, in method > ret = func(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line > 1527, in setupNetworks > supervdsm.getProxy().setupNetworks(networks, bondings, > options) > File > "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", > line 55, in __call__ > return callMethod() > File > "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", > line 53, in <lambda> > **kwargs) > File "<string>", line 2, in setupNetworks > File "/usr/lib64/python2.7/multiprocessing/managers.py", line > 773, in _callmethod > raise convert_to_error(kind, result) > IOError: [Errno 19] ovirtmgmt is not present in the system > 2018-03-08 23:12:52,512-0800 INFO (jsonrpc/7) > [jsonrpc.JsonRpcServer] RPC call Host.setupNetworks failed > (error -32603) in 5.90 seconds (__init__:573) > 2018-03-08 23:12:54,769-0800 INFO (jsonrpc/1) > [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 > seconds (__init__:573) > 2018-03-08 23:12:54,772-0800 INFO (jsonrpc/5) [api.host] START > getCapabilities() from=::1,45562 (api:46) > 2018-03-08 23:12:54,906-0800 INFO (jsonrpc/5) [api.host] FINISH > getCapabilities error=[Errno 19] ovirtmgmt is not present in the > system from=::1,45562 (api:50) > 2018-03-08 23:12:54,906-0800 ERROR (jsonrpc/5) > [jsonrpc.JsonRpcServer] Internal server error (__init__:611) > Traceback (most recent call last): > File > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line > 606, in _handle_request > res = method(**params) > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", > line 201, in _dynamicMethod > result = fn(*methodArgs) > File "<string>", line 2, in getCapabilities > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", > line 48, in method > ret = func(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line > 1339, in getCapabilities > c = caps.get() > File "/usr/lib/python2.7/site-packages/vdsm/host/caps.py", > line 168, in get > net_caps = supervdsm.getProxy().network_caps() > File > "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", > line 55, in __call__ > return callMethod() > File > "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", > line 53, in <lambda> > **kwargs) > File "<string>", line 2, in network_caps > File "/usr/lib64/python2.7/multiprocessing/managers.py", line > 773, in _callmethod > raise convert_to_error(kind, result) > IOError: [Errno 19] ovirtmgmt is not present in the system > > So something is dreadfully wrong with the bridge to ovs > conversion in 4.2.2rc3. > > thomas > _______________________________________________ > Users mailing list > Users(a)ovirt.org <mailto:Users@ovirt.org> > http://lists.ovirt.org/mailman/listinfo/users > <http://lists.ovirt.org/mailman/listinfo/users> > > > _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

On Thu, Mar 15, 2018 at 1:50 AM, Thomas Davis <tadavis(a)lbl.gov&gt; wrote: > Well, I just hit > > https://bugzilla.redhat.com/show_bug.cgi?id=1513991 > > And it's been closed, which means with vdsm-4.20.17-1.el7.centos.x86_64 > OVS networking is totally borked.. You are welcome to reopen that bug, specifying your use case for OvS. I cannot promise fixing this bug, as our resources are limited, and that bug, which was introduced in 4.2, was not deemed as urgently needed. https://gerrit.ovirt.org/#/c/86932/ attempts to fix the bug, but it still needs a lot of work. > > I know OVS is Experimental, but it worked in 4.1.x, and now we have to do a > step back to legacy bridge just to use 4.2.x, which in a vlan environment > just wreaks havoc (every VLAN need's a unique mac assigned to the bridge, > which vdsm does not do, so suddenly you get the kernel complaining about > seeing it's mac address several times.) Could you elaborate on this issue? What is wrong with a bridge that learns its mac from its underlying device? What wold like Vdsm to do, in your opinion? You can file a bug (or even send a patch) if there is a functionality that you'd like to fix. > > There is zero documentation on how to use OVN instead of OVS. I hope that https://ovirt.org/develop/release-management/features/network/provider-ph... can help. > thomas > > On 03/13/2018 09:22 AM, Thomas Davis wrote: >> >> I'll work on it some more. I have 2 different clusters in the data center >> (1 is the Hosted Engine systems, another is not..) I had trouble with both. >> I'll try again on the non-hosted engine cluster to see what it is doing. I >> have it working in 4.1, but we are trying to do a clean wipe since the 4.1 >> engine has been upgraded so many times from v3.5 plus we want to move to >> hosted-engine-ha from a single engine node and the ansible modules/roles >> (which also have problems..) >> >> thomas >> >> On Tue, Mar 13, 2018 at 6:27 AM, Edward Haas <ehaas(a)redhat.com >> <mailto:ehaas@redhat.com>> wrote: >> >> >> OVS switch support is experimental at this stage and in some cases >> when trying to change from one switch to the other, it fails. >> It was also not checked against a hosted engine setup, which handles >> networking a bit differently for the management network (ovirtmgmt). >> Nevertheless, we are interested in understanding all the problems >> that exists today, so if you can, please share the supervdsm log, it >> has the interesting networking traces. >> >> We plan to block cluster switch editing until these problems are >> resolved. It will be only allowed to define a new cluster as OVS, >> not convert an existing one from Linux Bridge to OVS. >> >> On Fri, Mar 9, 2018 at 9:54 AM, Thomas Davis <tadavis(a)lbl.gov >> <mailto:tadavis@lbl.gov>> wrote: >> >> I'm getting further along with 4.2.2rc3 than the 4.2.1 when it >> comes to hosted engine and vlans.. it actually does install >> under 4.2.2rc3. >> >> But it's a complete failure when I switch the cluster from Linux >> Bridge/Legacy to OVS. The first time I try, vdsm does >> not properly configure the node, it's all messed up. >> >> I'm getting this in vdsmd logs: >> >> 2018-03-08 23:12:46,610-0800 INFO (jsonrpc/7) [api.network] >> START setupNetworks(networks={u'ovirtmgmt': {u'ipv6autoconf': >> True, u'nic': u'eno1', u'vlan': u'50', u'ipaddr': >> u'192.168.85.49', u'switch': u'ovs', u'mtu': 1500, u'netmask': >> u'255.255.252.0', u'dhcpv6': False, u'STP': u'no', u'bridged': >> u'true', u'gateway': u'192.168.85.254', u'defaultRoute': True}}, >> bondings={}, options={u'connectivityCheck': u'true', >> u'connectivityTimeout': 120}) from=::ffff:192.168.85.24,56806, >> flow_id=4147e25f-0a23-4f47-a0a4-d424a3437d11 (api:46) >> >> 2018-03-08 23:12:52,449-0800 INFO (jsonrpc/2) >> [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 >> seconds (__init__:573) >> >> 2018-03-08 23:12:52,511-0800 INFO (jsonrpc/7) [api.network] >> FINISH setupNetworks error=[Errno 19] ovirtmgmt is not present >> in the system from=::ffff:192.168.85.24,56806, >> flow_id=4147e25f-0a23-4f47-a0a4-d424a3437d11 (api:50) >> 2018-03-08 23:12:52,512-0800 ERROR (jsonrpc/7) >> [jsonrpc.JsonRpcServer] Internal server error (__init__:611) >> Traceback (most recent call last): >> File >> "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line >> 606, in _handle_request >> res = method(**params) >> File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", >> line 201, in _dynamicMethod >> result = fn(*methodArgs) >> File "<string>", line 2, in setupNetworks >> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", >> line 48, in method >> ret = func(*args, **kwargs) >> File "/usr/lib/python2.7/site-packages/vdsm/API.py", line >> 1527, in setupNetworks >> supervdsm.getProxy().setupNetworks(networks, bondings, >> options) >> File >> "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", >> line 55, in __call__ >> return callMethod() >> File >> "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", >> line 53, in <lambda> >> **kwargs) >> File "<string>", line 2, in setupNetworks >> File "/usr/lib64/python2.7/multiprocessing/managers.py", line >> 773, in _callmethod >> raise convert_to_error(kind, result) >> IOError: [Errno 19] ovirtmgmt is not present in the system >> 2018-03-08 23:12:52,512-0800 INFO (jsonrpc/7) >> [jsonrpc.JsonRpcServer] RPC call Host.setupNetworks failed >> (error -32603) in 5.90 seconds (__init__:573) >> 2018-03-08 23:12:54,769-0800 INFO (jsonrpc/1) >> [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 >> seconds (__init__:573) >> 2018-03-08 23:12:54,772-0800 INFO (jsonrpc/5) [api.host] START >> getCapabilities() from=::1,45562 (api:46) >> 2018-03-08 23:12:54,906-0800 INFO (jsonrpc/5) [api.host] FINISH >> getCapabilities error=[Errno 19] ovirtmgmt is not present in the >> system from=::1,45562 (api:50) >> 2018-03-08 23:12:54,906-0800 ERROR (jsonrpc/5) >> [jsonrpc.JsonRpcServer] Internal server error (__init__:611) >> Traceback (most recent call last): >> File >> "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line >> 606, in _handle_request >> res = method(**params) >> File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", >> line 201, in _dynamicMethod >> result = fn(*methodArgs) >> File "<string>", line 2, in getCapabilities >> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", >> line 48, in method >> ret = func(*args, **kwargs) >> File "/usr/lib/python2.7/site-packages/vdsm/API.py", line >> 1339, in getCapabilities >> c = caps.get() >> File "/usr/lib/python2.7/site-packages/vdsm/host/caps.py", >> line 168, in get >> net_caps = supervdsm.getProxy().network_caps() >> File >> "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", >> line 55, in __call__ >> return callMethod() >> File >> "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", >> line 53, in <lambda> >> **kwargs) >> File "<string>", line 2, in network_caps >> File "/usr/lib64/python2.7/multiprocessing/managers.py", line >> 773, in _callmethod >> raise convert_to_error(kind, result) >> IOError: [Errno 19] ovirtmgmt is not present in the system >> >> So something is dreadfully wrong with the bridge to ovs >> conversion in 4.2.2rc3. >> >> thomas >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org <mailto:Users@ovirt.org> >> http://lists.ovirt.org/mailman/listinfo/users >> <http://lists.ovirt.org/mailman/listinfo/users> >> >> >> > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users