<div dir="ltr"><div><div><div>Sverker,<br></div>I can see you as a user in gerrit (<a href="mailto:sverker@abrahamsson.com">sverker@abrahamsson.com</a>), but there are no patches for your name.<br></div>Please check for any errors after you issue:<br>git push gerrit.ovirt.org:ovirt-provider-ovn HEAD:refs/for/master<br><br></div>Also, please let me know if you need any other help on with gerrit.</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 16, 2017 at 8:49 PM, Sverker Abrahamsson <span dir="ltr"><<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamsson.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I've followed the instructions to best effort, so hopefully it's right..<br>
<br>
<br>
Den 2017-01-13 kl. 10:31, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Please push the patch into: <a href="https://gerrit.ovirt.org/ovirt-provider-ovn" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/ovirt<wbr>-provider-ovn</a><br>
(let me know if you need some directions)<br>
<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson" <<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamsson.com</a>><br>
To: "Marcin Mirecki" <<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users" <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Monday, January 9, 2017 1:45:37 PM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and mandatory ovirtmgmt network<br>
<br>
Ok, found it. The issue is right here:<br>
<br>
<interface type="bridge"><br>
<mac address="00:1a:4a:16:01:54" /><br>
<model type="virtio" /><br>
<source bridge="br-int" /><br>
<virtualport type="openvswitch" /><br>
<link state="up" /><br>
<boot order="2" /><br>
<bandwidth /><br>
<virtualport type="openvswitch"><br>
<parameters<br>
interfaceid="912cba79-982e-4a8<wbr>7-868e-241fedccb59a" /><br>
</virtualport><br>
</interface><br>
<br>
There are two elements for virtualport, the first without id and the<br>
second with. On h2 I had fixed this which was the patch I posted earlier<br>
although I switched back to use br-int after understanding that was the<br>
correct way. When that hook was copied to h1 the port gets attached fine.<br>
<br>
Patch with updated testcase attached.<br>
<br>
/Sverker<br>
<br>
<br>
Den 2017-01-09 kl. 10:41, skrev Sverker Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This is the content of vdsm.log on h1 at this time:<br>
<br>
2017-01-06 20:54:12,636 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC<br>
call VM.create succeeded in 0.01 seconds (__init__:515)<br>
2017-01-06 20:54:12,636 INFO (vm/6dd5291e) [virt.vm]<br>
(vmId='6dd5291e-6556-4d29-8b4e<wbr>-ea896e627645') VM wrapper has started<br>
(vm:1901)<br>
2017-01-06 20:54:12,636 INFO (vm/6dd5291e) [vds] prepared volume<br>
path:<br>
/rhev/data-center/mnt/h2-int.l<wbr>imetransit.com:_var_lib_export<wbr>s_iso/1d49c4bc-0fec-4503-a583-<wbr>d476fa3a370d/images/11111111-<wbr>1111-1111-1111-111111111111/<wbr>CentOS-7-x86_64-NetInstall-<wbr>1611.iso<br>
(clientIF:374)<br>
2017-01-06 20:54:12,743 INFO (vm/6dd5291e) [root] (hooks:108)<br>
2017-01-06 20:54:12,847 INFO (vm/6dd5291e) [root] (hooks:108)<br>
2017-01-06 20:54:12,863 INFO (vm/6dd5291e) [virt.vm]<br>
(vmId='6dd5291e-6556-4d29-8b4e<wbr>-ea896e627645') <?xml version='1.0'<br>
encoding='UTF-8'?><br>
<domain xmlns:ovirt="<a href="http://ovirt.org/vm/tune/1.0" rel="noreferrer" target="_blank">http://ovirt.org/<wbr>vm/tune/1.0</a>" type="kvm"><br>
<name>CentOS7_3</name><br>
<uuid>6dd5291e-6556-4d29-<wbr>8b4e-ea896e627645</uuid><br>
<memory>1048576</memory><br>
<currentMemory>1048576</curre<wbr>ntMemory><br>
<maxMemory slots="16">4294967296</maxMemo<wbr>ry><br>
<vcpu current="1">16</vcpu><br>
<devices><br>
<channel type="unix"><br>
<target name="com.redhat.rhevm.vdsm" type="virtio" /><br>
<source mode="bind"<br>
path="/var/lib/libvirt/qemu/ch<wbr>annels/6dd5291e-6556-4d29-8b4e<wbr>-ea896e627645.com.redhat.rhevm<wbr>.vdsm"<br>
/><br>
</channel><br>
<channel type="unix"><br>
<target name="org.qemu.guest_agent.0" type="virtio" /><br>
<source mode="bind"<br>
path="/var/lib/libvirt/qemu/ch<wbr>annels/6dd5291e-6556-4d29-8b4e<wbr>-ea896e627645.org.qemu.guest_<wbr>agent.0"<br>
/><br>
</channel><br>
<input bus="ps2" type="mouse" /><br>
<memballoon model="virtio" /><br>
<controller index="0" model="virtio-scsi" type="scsi" /><br>
<controller index="0" ports="16" type="virtio-serial" /><br>
<video><br>
<model heads="1" ram="65536" type="qxl" vgamem="16384"<br>
vram="32768" /><br>
</video><br>
<graphics autoport="yes" defaultMode="secure" passwd="*****"<br>
passwdValidTo="1970-01-01T00:0<wbr>0:01" port="-1" tlsPort="-1" type="spice"><br>
<channel mode="secure" name="main" /><br>
<channel mode="secure" name="inputs" /><br>
<channel mode="secure" name="cursor" /><br>
<channel mode="secure" name="playback" /><br>
<channel mode="secure" name="record" /><br>
<channel mode="secure" name="display" /><br>
<channel mode="secure" name="smartcard" /><br>
<channel mode="secure" name="usbredir" /><br>
<listen network="vdsm-ovirtmgmt" type="network" /><br>
</graphics><br>
<interface type="bridge"><br>
<mac address="00:1a:4a:16:01:54" /><br>
<model type="virtio" /><br>
<source bridge="br-int" /><br>
<virtualport type="openvswitch" /><br>
<link state="up" /><br>
<boot order="2" /><br>
<bandwidth /><br>
<virtualport type="openvswitch"><br>
<parameters<br>
interfaceid="912cba79-982e-4a8<wbr>7-868e-241fedccb59a" /><br>
</virtualport><br>
</interface><br>
<disk device="cdrom" snapshot="no" type="file"><br>
<source<br>
file="/rhev/data-center/mnt/h2<wbr>-int.limetransit.com:_var_lib_<wbr>exports_iso/1d49c4bc-0fec-4503<wbr>-a583-d476fa3a370d/images/1111<wbr>1111-1111-1111-1111-1111111111<wbr>11/CentOS-7-x86_64-NetInstall-<wbr>1611.iso"<br>
startupPolicy="optional" /><br>
<target bus="ide" dev="hdc" /><br>
<readonly /><br>
<boot order="1" /><br>
</disk><br>
<channel type="spicevmc"><br>
<target name="com.redhat.spice.0" type="virtio" /><br>
</channel><br>
</devices><br>
<metadata><br>
<ovirt:qos /><br>
</metadata><br>
<os><br>
<type arch="x86_64" machine="pc-i440fx-rhel7.2.0"><wbr>hvm</type><br>
<smbios mode="sysinfo" /><br>
<bootmenu enable="yes" timeout="10000" /><br>
</os><br>
<sysinfo type="smbios"><br>
<system><br>
<entry name="manufacturer">oVirt</ent<wbr>ry><br>
<entry name="product">oVirt Node</entry><br>
<entry name="version">7-3.1611.el7.ce<wbr>ntos</entry><br>
<entry<br>
name="serial">62f1adff-b29e-4a<wbr>7c-abba-c2c4c73248c6</entry><br>
<entry<br>
name="uuid">6dd5291e-6556-4d29<wbr>-8b4e-ea896e627645</entry><br>
</system><br>
</sysinfo><br>
<clock adjustment="0" offset="variable"><br>
<timer name="rtc" tickpolicy="catchup" /><br>
<timer name="pit" tickpolicy="delay" /><br>
<timer name="hpet" present="no" /><br>
</clock><br>
<features><br>
<acpi /><br>
</features><br>
<cpu match="exact"><br>
<model>SandyBridge</model><br>
<topology cores="1" sockets="16" threads="1" /><br>
<numa><br>
<cell cpus="0" memory="1048576" /><br>
</numa><br>
</cpu><br>
</domain><br>
(vm:1988)<br>
2017-01-06 20:54:13,046 INFO (libvirt/events) [virt.vm]<br>
(vmId='6dd5291e-6556-4d29-8b4e<wbr>-ea896e627645') CPU running: onResume<br>
(vm:4863)<br>
2017-01-06 20:54:13,058 INFO (vm/6dd5291e) [virt.vm]<br>
(vmId='6dd5291e-6556-4d29-8b4e<wbr>-ea896e627645') Starting connection<br>
(guestagent:245)<br>
2017-01-06 20:54:13,060 INFO (vm/6dd5291e) [virt.vm]<br>
(vmId='6dd5291e-6556-4d29-8b4e<wbr>-ea896e627645') CPU running: domain<br>
initialization (vm:4863)<br>
2017-01-06 20:54:15,154 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC<br>
call Host.getVMFullList succeeded in 0.01 seconds (__init__:515)<br>
2017-01-06 20:54:17,571 INFO (periodic/2) [dispatcher] Run and<br>
protect: getVolumeSize(sdUUID=u'2ee54fb<wbr>8-48f2-4576-8cff-f2346504b08b'<wbr>,<br>
spUUID=u'584ebd64-0268-0193-02<wbr>5b-00000000038e',<br>
imgUUID=u'5a3aae57-ffe0-4a3b-a<wbr>a87-8461669db7f9',<br>
volUUID=u'b6a88789-fcb1-4d3e-9<wbr>11b-2a4d3b6c69c7', options=None)<br>
(logUtils:49)<br>
2017-01-06 20:54:17,573 INFO (periodic/2) [dispatcher] Run and<br>
protect: getVolumeSize, Return response: {'truesize': '1859723264',<br>
'apparentsize': '21474836480'} (logUtils:52)<br>
2017-01-06 20:54:21,211 INFO (periodic/2) [dispatcher] Run and<br>
protect: repoStats(options=None) (logUtils:49)<br>
2017-01-06 20:54:21,212 INFO (periodic/2) [dispatcher] Run and<br>
protect: repoStats, Return response:<br>
{u'2ee54fb8-48f2-4576-8cff-f23<wbr>46504b08b': {'code': 0, 'actual': True,<br>
'version': 3, 'acquired': True, 'delay': '0.000936552', 'lastCheck':<br>
'1.4', 'valid': True}, u'1d49c4bc-0fec-4503-a583-d476<wbr>fa3a370d':<br>
{'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':<br>
'0.000960248', 'lastCheck': '1.4', 'valid': True}} (logUtils:52)<br>
2017-01-06 20:54:23,543 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC<br>
call Host.getAllVmStats succeeded in 0.00 seconds (__init__:515)<br>
2017-01-06 20:54:23,641 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC<br>
call Host.getAllVmIoTunePolicies succeeded in 0.00 seconds (__init__:515)<br>
2017-01-06 20:54:24,918 INFO (jsonrpc/0) [dispatcher] Run and<br>
protect: repoStats(options=None) (logUtils:49)<br>
2017-01-06 20:54:24,918 INFO (jsonrpc/0) [dispatcher] Run and<br>
protect: repoStats, Return response:<br>
{u'2ee54fb8-48f2-4576-8cff-f23<wbr>46504b08b': {'code': 0, 'actual': True,<br>
'version': 3, 'acquired': True, 'delay': '0.000936552', 'lastCheck':<br>
'5.1', 'valid': True}, u'1d49c4bc-0fec-4503-a583-d476<wbr>fa3a370d':<br>
{'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':<br>
'0.000960248', 'lastCheck': '2.1', 'valid': True}} (logUtils:52)<br>
2017-01-06 20:54:24,924 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC<br>
call Host.getStats succeeded in 0.01 seconds (__init__:515)<br>
<br>
Vdsm and the OVN driver must have been called as the port IS created,<br>
but with the wrong id. I don't find the faulty id in vdsm.log neither,<br>
the xml above have the correct id.<br>
/Sverker<br>
<br>
Den 2017-01-09 kl. 10:06, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The port is set up on the host by the ovirt-provider-ovn-driver.<br>
The driver is invoked by the vdsm hook whenever any operation on<br>
the port is done.<br>
Please ensure that this is installed properly.<br>
You can check the vdsm log (/var/log/vdsm/vdsm.log) to see if the<br>
hook was executed properly.<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson" <<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamsson.com</a>><br>
To: "Marcin Mirecki" <<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users" <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Friday, January 6, 2017 9:00:26 PM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and mandatory<br>
ovirtmgmt network<br>
<br>
I created a new VM in the ui and assigned it to host h1. In<br>
/var/log/ovirt-provider-ovn.lo<wbr>g I get the following:<br>
<br>
2017-01-06 20:54:11,940 Request: GET : /v2.0/ports<br>
2017-01-06 20:54:11,940 Connecting to remote ovn database:<br>
tcp:<a href="http://127.0.0.1:6641" rel="noreferrer" target="_blank">127.0.0.1:6641</a><br>
2017-01-06 20:54:12,157 Connected (number of retries: 2)<br>
2017-01-06 20:54:12,158 Response code: 200<br>
2017-01-06 20:54:12,158 Response body: {"ports": [{"name":<br>
"4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873", "network_id":<br>
"e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601", "device_owner": "oVirt",<br>
"mac_address": "00:1a:4a:16:01:51", "id":<br>
"4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873", "device_id":<br>
"40cd7328-d575-4c3d-b656-9ef9b<wbr>acc0078"}, {"name":<br>
"92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5", "network_id":<br>
"e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601", "device_owner": "oVirt",<br>
"mac_address": "00:1a:4a:16:01:52", "id":<br>
"92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5", "device_id":<br>
"4baefa8c-3822-4de0-9cd0-1d025<wbr>bab7844"}]}<br>
2017-01-06 20:54:12,160 Request: SHOW :<br>
/v2.0/networks/e53554cf-e553-4<wbr>0a1-8d22-9c8d95ec0601<br>
2017-01-06 20:54:12,160 Connecting to remote ovn database:<br>
tcp:<a href="http://127.0.0.1:6641" rel="noreferrer" target="_blank">127.0.0.1:6641</a><br>
2017-01-06 20:54:12,377 Connected (number of retries: 2)<br>
2017-01-06 20:54:12,378 Response code: 200<br>
2017-01-06 20:54:12,378 Response body: {"network": {"id":<br>
"e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601", "name": "ovirtbridge"}}<br>
2017-01-06 20:54:12,380 Request: POST : /v2.0/ports<br>
2017-01-06 20:54:12,380 Request body:<br>
{<br>
"port" : {<br>
"name" : "nic1",<br>
"binding:host_id" : "<a href="http://h1.limetransit.com" rel="noreferrer" target="_blank">h1.limetransit.com</a>",<br>
"admin_state_up" : true,<br>
"device_id" : "e8553a88-05f0-401d-8b9b-5fff7<wbr>7f7bbbe",<br>
"device_owner" : "oVirt",<br>
"mac_address" : "00:1a:4a:16:01:54",<br>
"network_id" : "e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601"<br>
}<br>
}<br>
2017-01-06 20:54:12,380 Connecting to remote ovn database:<br>
tcp:<a href="http://127.0.0.1:6641" rel="noreferrer" target="_blank">127.0.0.1:6641</a><br>
2017-01-06 20:54:12,610 Connected (number of retries: 2)<br>
2017-01-06 20:54:12,614 Response code: 200<br>
2017-01-06 20:54:12,614 Response body: {"port": {"name":<br>
"912cba79-982e-4a87-868e-241fe<wbr>dccb59a", "network_id":<br>
"e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601", "device_owner": "oVirt",<br>
"mac_address": "00:1a:4a:16:01:54", "id":<br>
"912cba79-982e-4a87-868e-241fe<wbr>dccb59a", "device_id":<br>
"e8553a88-05f0-401d-8b9b-5fff7<wbr>7f7bbbe"}}<br>
<br>
h1:/var/log/messages<br>
Jan 6 20:54:12 h1 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl<br>
--timeout=5 -- --if-exists del-port vnet1 -- add-port br-int vnet1 --<br>
set Interface vnet1<br>
"external-ids:attached-mac=\"0<wbr>0:1a:4a:16:01:54\"" --<br>
set Interface vnet1<br>
"external-ids:iface-id=\"20388<wbr>407-0f76-41d8-97aa-8e2b5978f90<wbr>8\"" -- set<br>
Interface vnet1<br>
"external-ids:vm-id=\"6dd5291e<wbr>-6556-4d29-8b4e-ea896e627645\"<wbr>" -- set<br>
Interface vnet1 external-ids:iface-status=acti<wbr>ve<br>
<br>
[root@h2 ~]# ovn-nbctl show<br>
switch e53554cf-e553-40a1-8d22-9c8d95<wbr>ec0601 (ovirtbridge)<br>
port 4981ee5f-6e15-4bd5-a1cf-7ead9b<wbr>dd5873<br>
addresses: ["00:1a:4a:16:01:51"]<br>
port 912cba79-982e-4a87-868e-241fed<wbr>ccb59a<br>
addresses: ["00:1a:4a:16:01:54"]<br>
port 92f6d3c8-68b3-4986-9c09-60bee0<wbr>4644b5<br>
addresses: ["00:1a:4a:16:01:52"]<br>
port ovirtbridge-port2<br>
addresses: ["unknown"]<br>
port ovirtbridge-port1<br>
addresses: ["unknown"]<br>
[root@h2 ~]# ovn-sbctl show<br>
Chassis "6e4dd29f-7607-48d7-8e5a-eef4c<wbr>6aeefb5"<br>
hostname: "<a href="http://h2.limetransit.com" rel="noreferrer" target="_blank">h2.limetransit.com</a>"<br>
Encap geneve<br>
ip: "148.251.126.50"<br>
options: {csum="true"}<br>
Port_Binding "4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873"<br>
Port_Binding "ovirtbridge-port1"<br>
Chassis "4f10fb04-8fb2-48d7-8a3f-ea644<wbr>4c02cf9"<br>
hostname: "<a href="http://h1.limetransit.com" rel="noreferrer" target="_blank">h1.limetransit.com</a>"<br>
Encap geneve<br>
ip: "144.76.84.73"<br>
options: {csum="true"}<br>
Port_Binding "ovirtbridge-port2"<br>
Port_Binding "92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5"<br>
<br>
I.e. same issue<br>
/Sverker<br>
<br>
Den 2017-01-06 kl. 20:49, skrev Sverker Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The port is created from Ovirt UI, the ovs-vsctl command below is<br>
executed when VM is started. In /var/log/ovirt-provider-ovn.lo<wbr>g on h2<br>
I get the following:<br>
<br>
2017-01-06 20:19:25,452 Request: GET : /v2.0/ports<br>
2017-01-06 20:19:25,452 Connecting to remote ovn database:<br>
tcp:<a href="http://127.0.0.1:6641" rel="noreferrer" target="_blank">127.0.0.1:6641</a><br>
2017-01-06 20:19:25,670 Connected (number of retries: 2)<br>
2017-01-06 20:19:25,670 Response code: 200<br>
2017-01-06 20:19:25,670 Response body: {"ports": [{"name":<br>
"4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873", "network_id":<br>
"e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601", "device_owner": "oVirt",<br>
"mac_address": "00:1a:4a:16:01:51", "id":<br>
"4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873", "device_id":<br>
"40cd7328-d575-4c3d-b656-9ef9b<wbr>acc0078"}, {"name":<br>
"92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5", "network_id":<br>
"e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601", "device_owner": "oVirt",<br>
"mac_address": "00:1a:4a:16:01:52", "id":<br>
"92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5", "device_id":<br>
"4baefa8c-3822-4de0-9cd0-1d025<wbr>bab7844"}]}<br>
2017-01-06 20:19:25,673 Request: PUT :<br>
/v2.0/ports/92f6d3c8-68b3-4986<wbr>-9c09-60bee04644b5<br>
2017-01-06 20:19:25,673 Request body:<br>
{<br>
"port" : {<br>
"binding:host_id" : "<a href="http://h1.limetransit.com" rel="noreferrer" target="_blank">h1.limetransit.com</a>",<br>
"security_groups" : null<br>
}<br>
}<br>
2017-01-06 20:19:25,673 Connecting to remote ovn database:<br>
tcp:<a href="http://127.0.0.1:6641" rel="noreferrer" target="_blank">127.0.0.1:6641</a><br>
2017-01-06 20:19:25,890 Connected (number of retries: 2)<br>
2017-01-06 20:19:25,891 Response code: 200<br>
2017-01-06 20:19:25,891 Response body: {"port": {"name":<br>
"92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5", "network_id":<br>
"e53554cf-e553-40a1-8d22-9c8d9<wbr>5ec0601", "device_owner": "oVirt",<br>
"mac_address": "00:1a:4a:16:01:52", "id":<br>
"92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5", "device_id":<br>
"4baefa8c-3822-4de0-9cd0-1d025<wbr>bab7844"}}<br>
<br>
In /var/log/messages on h1 I get the following:<br>
<br>
Jan 6 20:18:56 h1 dbus-daemon: dbus[1339]: [system] Successfully<br>
activated service 'org.freedesktop.problems'<br>
Jan 6 20:19:26 h1 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl<br>
--timeout=5 -- --if-exists del-port vnet0 -- add-port br-int vnet0 --<br>
set Interface vnet0 "external-ids:attached-mac=\"0<wbr>0:1a:4a:16:01:52\""<br>
-- set Interface vnet0<br>
"external-ids:iface-id=\"72daf<wbr>da5-03c2-4bb6-bcb6-241fa5c0a1f<wbr>3\"" --<br>
set Interface vnet0<br>
"external-ids:vm-id=\"4d0c134a<wbr>-11a0-40f4-b2fb-c13c17c7251c\"<wbr>" -- set<br>
Interface vnet0 external-ids:iface-status=acti<wbr>ve<br>
Jan 6 20:19:26 h1 kernel: device vnet0 entered promiscuous mode<br>
Jan 6 20:19:26 h1 avahi-daemon[1391]: Registering new address record<br>
for fe80::fc1a:4aff:fe16:152 on vnet0.*.<br>
Jan 6 20:19:26 h1 systemd-machined: New machine qemu-4-CentOS72.<br>
Jan 6 20:19:26 h1 systemd: Started Virtual Machine qemu-4-CentOS72.<br>
Jan 6 20:19:26 h1 systemd: Starting Virtual Machine qemu-4-CentOS72.<br>
<br>
[root@h2 ~]# ovn-nbctl show<br>
switch e53554cf-e553-40a1-8d22-9c8d95<wbr>ec0601 (ovirtbridge)<br>
port 4981ee5f-6e15-4bd5-a1cf-7ead9b<wbr>dd5873<br>
addresses: ["00:1a:4a:16:01:51"]<br>
port 92f6d3c8-68b3-4986-9c09-60bee0<wbr>4644b5<br>
addresses: ["00:1a:4a:16:01:52"]<br>
port ovirtbridge-port2<br>
addresses: ["unknown"]<br>
port ovirtbridge-port1<br>
addresses: ["unknown"]<br>
[root@h2 ~]# ovn-sbctl show<br>
Chassis "6e4dd29f-7607-48d7-8e5a-eef4c<wbr>6aeefb5"<br>
hostname: "<a href="http://h2.limetransit.com" rel="noreferrer" target="_blank">h2.limetransit.com</a>"<br>
Encap geneve<br>
ip: "148.251.126.50"<br>
options: {csum="true"}<br>
Port_Binding "4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873"<br>
Port_Binding "ovirtbridge-port1"<br>
Chassis "4f10fb04-8fb2-48d7-8a3f-ea644<wbr>4c02cf9"<br>
hostname: "<a href="http://h1.limetransit.com" rel="noreferrer" target="_blank">h1.limetransit.com</a>"<br>
Encap geneve<br>
ip: "144.76.84.73"<br>
options: {csum="true"}<br>
Port_Binding "ovirtbridge-port2"<br>
<br>
I.e. the port is set up with the wrong ID and not attached to OVN.<br>
<br>
If I correct external-ids:iface-id like this:<br>
[root@h1 ~]# ovs-vsctl set Interface vnet0<br>
"external-ids:iface-id=\"92f6d<wbr>3c8-68b3-4986-9c09-60bee04644b<wbr>5\""<br>
<br>
then sb is correct:<br>
[root@h2 ~]# ovn-sbctl show<br>
Chassis "6e4dd29f-7607-48d7-8e5a-eef4c<wbr>6aeefb5"<br>
hostname: "<a href="http://h2.limetransit.com" rel="noreferrer" target="_blank">h2.limetransit.com</a>"<br>
Encap geneve<br>
ip: "148.251.126.50"<br>
options: {csum="true"}<br>
Port_Binding "4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873"<br>
Port_Binding "ovirtbridge-port1"<br>
Chassis "4f10fb04-8fb2-48d7-8a3f-ea644<wbr>4c02cf9"<br>
hostname: "<a href="http://h1.limetransit.com" rel="noreferrer" target="_blank">h1.limetransit.com</a>"<br>
Encap geneve<br>
ip: "144.76.84.73"<br>
options: {csum="true"}<br>
Port_Binding "ovirtbridge-port2"<br>
Port_Binding "92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5"<br>
<br>
I don't know from where the ID 72dafda5-03c2-4bb6-bcb6-241fa5<wbr>c0a1f3<br>
comes from, doesn't show in any log other than /var/log/messages.<br>
<br>
If I do the same exercise on the same host as engine is running on<br>
then the port for the VM gets the right id and is working from<br>
beginning.<br>
/Sverker<br>
<br>
Den 2017-01-03 kl. 10:23, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
How did you create this port?<br>
From the oVirt engine UI?<br>
The OVN provider creates the port when you add the port in the<br>
engine UI,<br>
it is then plugged into the ovs bridge by the VIF driver.<br>
Please attach /var/log/ovirt-provider-ovn.lo<wbr>g<br>
<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Tuesday, January 3, 2017 2:06:22 AM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and mandatory<br>
ovirtmgmt<br>
network<br>
<br>
Found an issue with Ovirt - OVN integration.<br>
<br>
Engine and OVN central db running on host h2. Created VM to run<br>
on host<br>
h1, which is started. Ovn db state:<br>
<br>
[root@h2 env3]# ovn-nbctl show<br>
switch e53554cf-e553-40a1-8d22-9c8d95<wbr>ec0601 (ovirtbridge)<br>
port 4981ee5f-6e15-4bd5-a1cf-7ead9b<wbr>dd5873<br>
addresses: ["00:1a:4a:16:01:51"]<br>
port 92f6d3c8-68b3-4986-9c09-60bee0<wbr>4644b5<br>
addresses: ["00:1a:4a:16:01:52"]<br>
port ovirtbridge-port2<br>
addresses: ["unknown"]<br>
port ovirtbridge-port1<br>
addresses: ["unknown"]<br>
[root@h2 env3]# ovn-sbctl show<br>
Chassis "6e4dd29f-7607-48d7-8e5a-eef4c<wbr>6aeefb5"<br>
hostname: "<a href="http://h2.limetransit.com" rel="noreferrer" target="_blank">h2.limetransit.com</a>"<br>
Encap geneve<br>
ip: "148.251.126.50"<br>
options: {csum="true"}<br>
Port_Binding "4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873"<br>
Port_Binding "ovirtbridge-port1"<br>
Chassis "4f10fb04-8fb2-48d7-8a3f-ea644<wbr>4c02cf9"<br>
hostname: "<a href="http://h1.limetransit.com" rel="noreferrer" target="_blank">h1.limetransit.com</a>"<br>
Encap geneve<br>
ip: "144.76.84.73"<br>
options: {csum="true"}<br>
Port_Binding "ovirtbridge-port2"<br>
<br>
Port 92f6d3c8-68b3-4986-9c09-60bee0<wbr>4644b5 is for the new VM which is<br>
started on h1, but it is not assigned to that chassis. The reason is<br>
that on h1 the port on br-int is created like this:<br>
<br>
ovs-vsctl --timeout=5 -- --if-exists del-port vnet0 -- add-port<br>
br-int<br>
vnet0 -- set Interface vnet0<br>
"external-ids:attached-mac=\"0<wbr>0:1a:4a:16:01:52\"" -- set<br>
Interface vnet0<br>
"external-ids:iface-id=\"35bcb<wbr>e31-2c7e-4d97-add9-ce150eeb2f1<wbr>1\""<br>
-- set<br>
Interface vnet0<br>
"external-ids:vm-id=\"4d0c134a<wbr>-11a0-40f4-b2fb-c13c17c7251c\"<wbr>" -- set<br>
Interface vnet0 external-ids:iface-status=acti<wbr>ve<br>
<br>
I.e. the extrernal id of interface is wrong. When I manually<br>
change to<br>
the right id like this the port works fine:<br>
<br>
ovs-vsctl --timeout=5 -- --if-exists del-port vnet0 -- add-port<br>
br-int<br>
vnet0 -- set Interface vnet0<br>
"external-ids:attached-mac=\"0<wbr>0:1a:4a:16:01:52\"" -- set<br>
Interface vnet0<br>
"external-ids:iface-id=\"92f6d<wbr>3c8-68b3-4986-9c09-60bee04644b<wbr>5\""<br>
-- set<br>
Interface vnet0<br>
"external-ids:vm-id=\"4d0c134a<wbr>-11a0-40f4-b2fb-c13c17c7251c\"<wbr>" -- set<br>
Interface vnet0 external-ids:iface-status=acti<wbr>ve<br>
<br>
sb db after correcting the port:<br>
<br>
Chassis "6e4dd29f-7607-48d7-8e5a-eef4c<wbr>6aeefb5"<br>
hostname: "<a href="http://h2.limetransit.com" rel="noreferrer" target="_blank">h2.limetransit.com</a>"<br>
Encap geneve<br>
ip: "148.251.126.50"<br>
options: {csum="true"}<br>
Port_Binding "4981ee5f-6e15-4bd5-a1cf-7ead9<wbr>bdd5873"<br>
Port_Binding "ovirtbridge-port1"<br>
Chassis "4f10fb04-8fb2-48d7-8a3f-ea644<wbr>4c02cf9"<br>
hostname: "<a href="http://h1.limetransit.com" rel="noreferrer" target="_blank">h1.limetransit.com</a>"<br>
Encap geneve<br>
ip: "144.76.84.73"<br>
options: {csum="true"}<br>
Port_Binding "ovirtbridge-port2"<br>
Port_Binding "92f6d3c8-68b3-4986-9c09-60bee<wbr>04644b5"<br>
<br>
I don't know from where the faulty id comes from, it's not in any<br>
logs.<br>
In the domain xml as printed in vdsm.log the id is correct:<br>
<br>
<interface type="bridge"><br>
<mac address="00:1a:4a:16:01:52" /><br>
<model type="virtio" /><br>
<source bridge="br-int" /><br>
<virtualport type="openvswitch" /><br>
<link state="up" /><br>
<boot order="2" /><br>
<bandwidth /><br>
<virtualport type="openvswitch"><br>
<parameters<br>
interfaceid="92f6d3c8-68b3-498<wbr>6-9c09-60bee04644b5" /><br>
</virtualport><br>
</interface><br>
<br>
Where is the ovs-vsctl command line built for this call?<br>
<br>
/Sverker<br>
<br>
<br>
Den 2017-01-02 kl. 13:40, skrev Sverker Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Got it to work now by following the env8 example in OVN tutorial,<br>
where a port is added with type l2gateway. Not sure how that is<br>
different from the localnet variant, but didn't suceed in<br>
getting that<br>
one working. Now I'm able to ping and telnet over the tunnel,<br>
but not<br>
ssh even when the port is answering on telnet. Neither does nfs<br>
traffic work even though mount did. Suspecting MTU issue. I did<br>
notice<br>
that ovn-controller starts too early, before network interfaces are<br>
established and hence can't reach the db. As these is a purely<br>
OVS/OVN<br>
issue I'll ask about it on their mailing list.<br>
<br>
Getting back to the original issue with Ovirt, I've now added the<br>
second host h1 to ovirt-engine. Had to do the same as with h2 to<br>
create a dummy ovirtmgmt network but configured access via the<br>
public<br>
IP. My firewall settings was replaced with iptables config and<br>
vdsm.conf was overwritten when engine was set up, so those had<br>
to be<br>
manually restored. It would be preferable if it would be<br>
possible to<br>
configure ovirt-engine that it does not "own" the host and instead<br>
comply with the settings it has instead of enforcing it's own<br>
view..<br>
<br>
Apart from that it seems the second host works, although I need to<br>
resolve the traffic issue over the OVS tunnel.<br>
/Sverker<br>
<br>
Den 2017-01-02 kl. 01:13, skrev Sverker Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
1. That is not possible as ovirt (or vdsm) will rewrite the<br>
network<br>
configuration to a non-working state. That is why I've set that<br>
if as<br>
hidden to vdsm and is why I'm keen on getting OVS/OVN to work<br>
<br>
2. I've been reading the doc for OVN and starting to connect the<br>
dots, which is not trivial as it is complex. Some insights<br>
reached:<br>
<br>
First step is the OVN database, installed by<br>
openvswitch-ovn-central,<br>
which I currently have running on h2 host. The 'ovn-nbctl' and<br>
'ovn-sbctl' commands are only possible to execute on a database<br>
node.<br>
Two ip's are given to 'vdsm-tool ovn-config <ip to database><br>
<tunnel<br>
ip>' as arguments, where <ip to database> is how this OVN node<br>
reaches the database and <tunnel ip> is the ip to which other OVN<br>
nodes sets up a tunnel to this node. I.e. it is not for creating a<br>
tunnel to the database which I thought first from the<br>
description in<br>
blog post.<br>
<br>
The tunnel between OVN nodes is of type geneve which is a UDP<br>
based<br>
protocol but I have not been able to find anywhere which port<br>
is used<br>
so that I can open it in firewalld. I have added OVN on another<br>
host,<br>
called h1, and connected it to the db. I see there is traffic<br>
to the<br>
db port, but I don't see any geneve traffic between the nodes.<br>
<br>
Ovirt is now able to create it's vnet0 interface on the br-int ovs<br>
bridge, but then I run into the next issue. How do I create a<br>
connection from the logical switch to the physical host? I need<br>
that<br>
to a) get a connection out to the internet through a<br>
masqueraded if<br>
or ipv6 and b) be able to run a dhcp server to give ip's to the<br>
VM's.<br>
<br>
/Sverker<br>
<br>
Den 2016-12-30 kl. 18:05, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
1. Why not use your physical nic for ovirtmgmt then?<br>
<br>
2. "ovn-nbctl ls-add" does not add a bridge, but a logical<br>
switch.<br>
br-int is an internal OVN implementation detail, which<br>
the user<br>
should not care about. What you see in the ovirt UI are<br>
logical<br>
networks. They are implemented as OVN logical switches<br>
in case<br>
of the OVN provider.<br>
<br>
Please look at:<br>
<a href="http://www.ovirt.org/blog/2016/11/ovirt-provider-ovn/" rel="noreferrer" target="_blank">http://www.ovirt.org/blog/2016<wbr>/11/ovirt-provider-ovn/</a><br>
You can get the latest rpms from here:<br>
<a href="http://resources.ovirt.org/repos/ovirt/experimental/master/ovirt-provider-ovn_fc24_46/rpm/fc24/noarch/" rel="noreferrer" target="_blank">http://resources.ovirt.org/rep<wbr>os/ovirt/experimental/master/<wbr>ovirt-provider-ovn_fc24_46/<wbr>rpm/fc24/noarch/</a><br>
<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Friday, December 30, 2016 4:25:58 PM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and mandatory<br>
ovirtmgmt network<br>
<br>
1. No, I did not want to put the ovirtmgmt bridge on my physical<br>
nic as<br>
it always messed up the network config making the host<br>
unreachable. I<br>
have put a ovs bridge on this nic which I will use to make<br>
tunnels<br>
when<br>
I add other hosts. Maybe br-int will be used for that<br>
instead, will<br>
see<br>
when I get that far.<br>
<br>
As it is now I have a dummy if for ovirtmgmt bridge but this<br>
will<br>
probably not work when I add other hosts as that bridge cannot<br>
connect<br>
to the other hosts. I'm considering keeping this just as a<br>
dummy to<br>
keep<br>
ovirt engine satisfied while the actual communication will<br>
happen<br>
over<br>
OVN/OVS bridges and tunnels.<br>
<br>
2. On<br>
<a href="https://www.ovirt.org//develop/release-management/features/ovirt-ovn-provider/" rel="noreferrer" target="_blank">https://www.ovirt.org//develop<wbr>/release-management/features/<wbr>ovirt-ovn-provider/</a><br>
<br>
<br>
there is instructions how to add an OVS bridge to OVN with<br>
|ovn-nbctl<br>
ls-add <network name>|. If you want to use br-int then it makes<br>
sense to<br>
make that bridge visible in ovirt webui under networks so<br>
that it<br>
can be<br>
selected for VM's.<br>
<br>
It quite doesn't make sense to me that I can select other<br>
network<br>
for my<br>
VM but then that setting is not used when setting up the<br>
network.<br>
<br>
/Sverker<br>
<br>
Den 2016-12-30 kl. 15:34, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
The OVN provider does not require you to add any bridges<br>
manually.<br>
As I understand we were dealing with two problems:<br>
1. You only had one physical nic and wanted to put a bridge<br>
on it,<br>
attaching the management network to the bridge. This<br>
was the<br>
reason for<br>
creating the bridge (the recommended setup would be<br>
to used a<br>
separate<br>
physical nic for the management network). This bridge<br>
has<br>
nothing to<br>
do with the OVN bridge.<br>
2. OVN - you want to use OVN on this system. For this you<br>
have to<br>
install<br>
OVN on your hosts. This should create the br-int bridge,<br>
which are<br>
then used by the OVN provider. This br-int bridge<br>
must be<br>
configured<br>
to connect to other hosts using the geneve tunnels.<br>
<br>
In both cases the systems will not be aware of any bridges you<br>
create.<br>
They need a nic (be it physical or virtual) to connect to other<br>
system.<br>
Usually this is the physical nic. In your case you decided<br>
to put<br>
a bridge<br>
on the physical nic, and give oVirt a virtual nic attached<br>
to this<br>
bridge.<br>
This works, but keep in mind that the bridge you have<br>
introduced<br>
is outside<br>
of oVirt's (and OVN) control (and as such is not supported).<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
What is the purpose of<br>
adding my bridges to Ovirt through the external provider and<br>
configure<br>
them on my VM<br>
</blockquote>
I am not quite sure I understand.<br>
The external provider (OVN provider to be specific), does<br>
not add<br>
any<br>
bridges<br>
to the system. It is using the br-int bridge created by OVN.<br>
The<br>
networks<br>
created by the OVN provider are purely logical entities,<br>
implemented using<br>
the OVN br-int bridge.<br>
<br>
Marcin<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Friday, December 30, 2016 12:15:43 PM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and mandatory<br>
ovirtmgmt<br>
network<br>
<br>
Hi<br>
That is the logic I quite don't understand. What is the<br>
purpose of<br>
adding my bridges to Ovirt through the external provider and<br>
configure<br>
them on my VM if you are disregarding that and using br-int<br>
anyway?<br>
<br>
/Sverker<br>
<br>
Den 2016-12-30 kl. 10:53, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Sverker,<br>
<br>
br-int is the integration bridge created by default in<br>
OVN. This<br>
is the<br>
bridge we use for the OVN provider. As OVN is required to be<br>
installed,<br>
we assume that this bridge is present.<br>
Using any other ovs bridge is not supported, and will require<br>
custom code<br>
changes (such as the ones you created).<br>
<br>
The proper setup in your case would probably be to create<br>
br-int<br>
and<br>
connect<br>
this to your ovirtbridge, although I don't know the<br>
details of<br>
your env,<br>
so<br>
this is just my best guess.<br>
<br>
Marcin<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>>, "Numan Siddique"<br>
<<a href="mailto:nusiddiq@redhat.com" target="_blank">nusiddiq@redhat.com</a>><br>
Sent: Friday, December 30, 2016 1:14:50 AM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and mandatory<br>
ovirtmgmt<br>
network<br>
<br>
Even better, if the value is not hardcoded then the<br>
configured<br>
value is<br>
used. Might be that I'm missunderstanding something but<br>
this is<br>
the<br>
behaviour I expected instead of that it is using br-int.<br>
<br>
Attached is a patch which properly sets up the xml, in case<br>
there is<br>
already a virtual port there + testcode of some variants<br>
<br>
/Sverker<br>
<br>
Den 2016-12-29 kl. 22:55, skrev Sverker Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
When I change<br>
/usr/libexec/vdsm/hooks/before<wbr>_device_create/ovirt_provider_<wbr>ovn_hook<br>
<br>
<br>
to instead of hardcoded to br-int use BRIDGE_NAME =<br>
'ovirtbridge' then<br>
I get the expected behaviour and I get a working network<br>
connectivity<br>
in my VM with IP provided by dhcp.<br>
<br>
/Sverker<br>
<br>
Den 2016-12-29 kl. 22:07, skrev Sverker Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
By default the vNic profile of my OVN bridge<br>
ovirtbridge gets a<br>
Network filter named vdsm-no-mac-spoofing. If I instead<br>
set<br>
No filter<br>
then I don't get those ebtables / iptables messages. It<br>
seems<br>
that<br>
there is some issue between ovirt/vdsm and firewalld,<br>
which<br>
we can<br>
put to the side for now.<br>
<br>
It is not clear for me why the port is added on br-int<br>
instead of the<br>
bridge I've assigned to the VM, which is ovirtbridge??<br>
<br>
/Sverker<br>
<br>
Den 2016-12-29 kl. 14:20, skrev Sverker Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The specific command most likely fails because there<br>
is no<br>
chain<br>
named libvirt-J-vnet0, but when should that have been<br>
created?<br>
/Sverker<br>
<br>
-------- Vidarebefordrat meddelande --------<br>
Ämne: Re: [ovirt-users] Issue with OVN/OVS and<br>
mandatory<br>
ovirtmgmt<br>
network<br>
Datum: Thu, 29 Dec 2016 08:06:29 -0500 (EST)<br>
Från: Marcin Mirecki<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Till: Sverker Abrahamsson<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamsso<wbr>n.com</a>><br>
Kopia: Ovirt Users<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>>, Lance Richardson<br>
<<a href="mailto:lrichard@redhat.com" target="_blank">lrichard@redhat.com</a>>, Numan<br>
Siddique<<a href="mailto:nusiddiq@redhat.com" target="_blank">nusiddiq@redhat.com</a>><br>
<br>
<br>
<br>
Let me add the OVN team.<br>
<br>
Lance, Numan,<br>
<br>
Can you please look at this?<br>
<br>
Trying to plug a vNIC results in:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Dec 28 23:31:35 h2 ovs-vsctl:<br>
ovs|00001|vsctl|INFO|Called as<br>
ovs-vsctl<br>
--timeout=5 -- --if-exists del-port vnet0 --<br>
add-port<br>
br-int<br>
vnet0 --<br>
set Interface vnet0<br>
"external-ids:attached-mac=\"0<wbr>0:1a:4a:16:01:51\""<br>
-- set Interface vnet0<br>
"external-ids:iface-id=\"e8853<wbr>aac-8a75-41b0-8010-e630017dcdd<wbr>8\""<br>
<br>
<br>
--<br>
set Interface vnet0<br>
"external-ids:vm-id=\"b9440d60<wbr>-ef5a-4e2b-83cf-081df7c09e6f\"<wbr>"<br>
<br>
<br>
--<br>
set<br>
Interface vnet0 external-ids:iface-status=acti<wbr>ve<br>
Dec 28 23:31:35 h2 kernel: device vnet0 entered<br>
promiscuous<br>
mode<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -D<br>
PREROUTING<br>
-i vnet0<br>
-j<br>
libvirt-J-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
</blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote>
More details below<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Thursday, December 29, 2016 1:42:11 PM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and<br>
mandatory<br>
ovirtmgmt<br>
network<br>
<br>
Hi<br>
Same problem still..<br>
/Sverker<br>
<br>
Den 2016-12-29 kl. 13:34, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
The tunnels are created to connect multiple OVN<br>
controllers.<br>
If there is only one, there is no need for the<br>
tunnels, so<br>
none<br>
will be created, this is the correct behavior.<br>
<br>
Does the problem still occur after setting<br>
configuring the<br>
OVN-controller?<br>
<br>
Marcin<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Thursday, December 29, 2016 11:44:32 AM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and<br>
mandatory<br>
ovirtmgmt<br>
network<br>
<br>
Hi<br>
The rpm packages you listed in the other mail are<br>
installed but I<br>
had<br>
not run vdsm-tool ovn-config to create tunnel as<br>
the OVN<br>
controller<br>
is<br>
on the same host.<br>
<br>
[root@h2 ~]# rpm -q openvswitch-ovn-common<br>
openvswitch-ovn-common-2.6.90-<wbr>1.el7.centos.x86_64<br>
[root@h2 ~]# rpm -q openvswitch-ovn-host<br>
openvswitch-ovn-host-2.6.90-1.<wbr>el7.centos.x86_64<br>
[root@h2 ~]# rpm -q python-openvswitch<br>
python-openvswitch-2.6.90-1.el<wbr>7.centos.noarch<br>
<br>
After removing my manually created br-int and run<br>
<br>
vdsm-tool ovn-config 127.0.0.1 172.27.1.1<br>
<br>
then I have the br-int but 'ip link show' does not<br>
show<br>
any link<br>
'genev_sys_' nor does 'ovs-vsctl show' any port for<br>
ovn.<br>
I assume<br>
these<br>
are when there is an actual tunnel?<br>
<br>
[root@h2 ~]# ovs-vsctl show<br>
ebb6aede-cbbc-4f4f-a88a-a9cd72<wbr>b2bd23<br>
Bridge br-int<br>
fail_mode: secure<br>
Port br-int<br>
Interface br-int<br>
type: internal<br>
Bridge ovirtbridge<br>
Port ovirtbridge<br>
Interface ovirtbridge<br>
type: internal<br>
Bridge "ovsbridge0"<br>
Port "ovsbridge0"<br>
Interface "ovsbridge0"<br>
type: internal<br>
Port "eth0"<br>
Interface "eth0"<br>
ovs_version: "2.6.90"<br>
<br>
[root@h2 ~]# ip link show<br>
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc<br>
noqueue state<br>
UNKNOWN<br>
mode<br>
DEFAULT qlen 1<br>
link/loopback 00:00:00:00:00:00 brd<br>
00:00:00:00:00:00<br>
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_<wbr>UP> mtu 1500<br>
qdisc<br>
pfifo_fast<br>
master ovs-system state UP mode DEFAULT qlen 1000<br>
link/ether 44:8a:5b:84:7d:b3 brd<br>
ff:ff:ff:ff:ff:ff<br>
3: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc<br>
noop<br>
state<br>
DOWN<br>
mode<br>
DEFAULT qlen 1000<br>
link/ether 5a:14:cf:28:47:e2 brd<br>
ff:ff:ff:ff:ff:ff<br>
4: ovsbridge0: <BROADCAST,MULTICAST,UP,LOWER_<wbr>UP><br>
mtu 1500<br>
qdisc<br>
noqueue<br>
state UNKNOWN mode DEFAULT qlen 1000<br>
link/ether 44:8a:5b:84:7d:b3 brd<br>
ff:ff:ff:ff:ff:ff<br>
5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop<br>
state DOWN<br>
mode<br>
DEFAULT qlen 1000<br>
link/ether 9e:b0:3a:9d:f2:4b brd<br>
ff:ff:ff:ff:ff:ff<br>
6: ovirtbridge: <BROADCAST,MULTICAST,UP,LOWER_<wbr>UP> mtu<br>
1500 qdisc<br>
noqueue<br>
state UNKNOWN mode DEFAULT qlen 1000<br>
link/ether a6:f6:e5:a4:5b:45 brd<br>
ff:ff:ff:ff:ff:ff<br>
7: dummy0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500<br>
qdisc<br>
noqueue<br>
master<br>
ovirtmgmt state UNKNOWN mode DEFAULT qlen 1000<br>
link/ether 66:e0:1c:c3:a9:d8 brd<br>
ff:ff:ff:ff:ff:ff<br>
8: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_<wbr>UP> mtu<br>
1500<br>
qdisc<br>
noqueue<br>
state UP mode DEFAULT qlen 1000<br>
link/ether 66:e0:1c:c3:a9:d8 brd<br>
ff:ff:ff:ff:ff:ff<br>
<br>
Firewall settings:<br>
[root@h2 ~]# firewall-cmd --list-all-zones<br>
work<br>
target: default<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services: dhcpv6-client ssh<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
drop<br>
target: DROP<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services:<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
internal<br>
target: default<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services: dhcpv6-client mdns samba-client ssh<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
external<br>
target: default<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services: ssh<br>
ports:<br>
protocols:<br>
masquerade: yes<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
trusted<br>
target: ACCEPT<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services:<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
home<br>
target: default<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services: dhcpv6-client mdns samba-client ssh<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
dmz<br>
target: default<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services: ssh<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
public (active)<br>
target: default<br>
icmp-block-inversion: no<br>
interfaces: eth0 ovsbridge0<br>
sources:<br>
services: dhcpv6-client ssh<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
block<br>
target: %%REJECT%%<br>
icmp-block-inversion: no<br>
interfaces:<br>
sources:<br>
services:<br>
ports:<br>
protocols:<br>
masquerade: no<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
<br>
<br>
ovirt (active)<br>
target: default<br>
icmp-block-inversion: no<br>
interfaces: ovirtbridge ovirtmgmt<br>
sources:<br>
services: dhcp ovirt-fence-kdump-listener<br>
ovirt-http<br>
ovirt-https<br>
ovirt-imageio-proxy ovirt-postgres ovirt-provider-ovn<br>
ovirt-vmconsole-proxy ovirt-websocket-proxy ssh vdsm<br>
ports:<br>
protocols:<br>
masquerade: yes<br>
forward-ports:<br>
sourceports:<br>
icmp-blocks:<br>
rich rules:<br>
rule family="ipv4" port port="6641"<br>
protocol="tcp"<br>
accept<br>
rule family="ipv4" port port="6642"<br>
protocol="tcp"<br>
accept<br>
<br>
The db dump is attached<br>
/Sverker<br>
Den 2016-12-29 kl. 09:50, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
Can you please do: "sudo ovsdb-client dump"<br>
on the host and send me the output?<br>
<br>
Have you configured the ovn controller to connect<br>
to the<br>
OVN north? You can do it using "vdsm-tool<br>
ovn-config" or<br>
using the OVN tools directly.<br>
Please check<br>
out:<a href="https://www.ovirt.org/blog/2016/11/ovirt-provider-ovn/" rel="noreferrer" target="_blank">https://www.ovirt.org/blog<wbr>/2016/11/ovirt-provider-ovn/</a><br>
<br>
for details.<br>
<br>
Also please note that the OVN provider is completely<br>
different<br>
from the neutron-openvswitch plugin. Please don't mix<br>
the two.<br>
<br>
Marcin<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
To: "Sverker Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Thursday, December 29, 2016 9:27:19 AM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and<br>
mandatory<br>
ovirtmgmt<br>
network<br>
<br>
Hi,<br>
<br>
br-int is the OVN integration bridge, it should<br>
have been<br>
created<br>
when installing OVN. I assume you have the following<br>
packages<br>
installed<br>
on the host:<br>
openvswitch-ovn-common<br>
openvswitch-ovn-host<br>
python-openvswitch<br>
<br>
Please give me some time to look at the connectivity<br>
problem.<br>
<br>
Marcin<br>
<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker<br>
Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Thursday, December 29, 2016 12:47:04 AM<br>
Subject: Re: [ovirt-users] Issue with OVN/OVS and<br>
mandatory<br>
ovirtmgmt<br>
network<br>
<br>
From<br>
/usr/libexec/vdsm/hooks/before<wbr>_device_create/ovirt_provider_<wbr>ovn_hook<br>
<br>
<br>
(installed by ovirt-provider-ovn-driver rpm):<br>
<br>
BRIDGE_NAME = 'br-int'<br>
<br>
<br>
Den 2016-12-28 kl. 23:56, skrev Sverker<br>
Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Googling on the message about br-int suggested<br>
adding<br>
that<br>
bridge to<br>
ovs:<br>
<br>
ovs-vsctl add-br br-int<br>
<br>
Then the VM is able to boot, but it fails to get<br>
network<br>
connectivity.<br>
Output in /var/log/messages:<br>
<br>
Dec 28 23:31:35 h2 ovs-vsctl:<br>
ovs|00001|vsctl|INFO|Called as<br>
ovs-vsctl<br>
--timeout=5 -- --if-exists del-port vnet0 --<br>
add-port<br>
br-int<br>
vnet0 --<br>
set Interface vnet0<br>
"external-ids:attached-mac=\"0<wbr>0:1a:4a:16:01:51\""<br>
-- set Interface vnet0<br>
"external-ids:iface-id=\"e8853<wbr>aac-8a75-41b0-8010-e630017dcdd<wbr>8\""<br>
<br>
<br>
--<br>
set Interface vnet0<br>
"external-ids:vm-id=\"b9440d60<wbr>-ef5a-4e2b-83cf-081df7c09e6f\"<wbr>"<br>
<br>
<br>
--<br>
set<br>
Interface vnet0 external-ids:iface-status=acti<wbr>ve<br>
Dec 28 23:31:35 h2 kernel: device vnet0 entered<br>
promiscuous<br>
mode<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -D<br>
PREROUTING<br>
-i vnet0<br>
-j<br>
libvirt-J-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -D<br>
POSTROUTING -o<br>
vnet0<br>
-j<br>
libvirt-P-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -L<br>
libvirt-J-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -L<br>
libvirt-P-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
libvirt-J-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
libvirt-J-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
libvirt-P-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
libvirt-P-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
J-vnet0-mac'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
J-vnet0-mac'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
J-vnet0-arp-mac'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
J-vnet0-arp-mac'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -D libvirt-out -m<br>
physdev<br>
--physdev-is-bridged --physdev-out vnet0 -g<br>
FO-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -D libvirt-out -m<br>
physdev<br>
--physdev-out<br>
vnet0 -g FO-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -D libvirt-in -m<br>
physdev<br>
--physdev-in<br>
vnet0<br>
-g FI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -D libvirt-host-in -m<br>
physdev<br>
--physdev-in<br>
vnet0 -g HI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -F FO-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -X FO-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -F FI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -X FI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -F HI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -X HI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -E FP-vnet0 FO-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -E FJ-vnet0 FI-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/iptables -w2 -w -E HJ-vnet0 HI-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -D libvirt-out -m<br>
physdev<br>
--physdev-is-bridged --physdev-out vnet0 -g<br>
FO-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -D libvirt-out -m<br>
physdev<br>
--physdev-out<br>
vnet0 -g FO-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -D libvirt-in -m<br>
physdev<br>
--physdev-in<br>
vnet0 -g FI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m<br>
physdev<br>
--physdev-in<br>
vnet0 -g HI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -F FO-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -X FO-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -F FI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -X FI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -F HI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -X HI-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -E FP-vnet0 FO-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -E FJ-vnet0 FI-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ip6tables -w2 -w -E HJ-vnet0 HI-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -D<br>
PREROUTING<br>
-i vnet0<br>
-j<br>
libvirt-I-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -D<br>
POSTROUTING -o<br>
vnet0<br>
-j<br>
libvirt-O-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -L<br>
libvirt-I-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -L<br>
libvirt-O-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
libvirt-I-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
libvirt-I-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
libvirt-O-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
libvirt-O-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -L<br>
libvirt-P-vnet0'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -E<br>
libvirt-P-vnet0<br>
libvirt-O-vnet0' failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
I-vnet0-mac'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
I-vnet0-mac'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -F<br>
I-vnet0-arp-mac'<br>
failed:<br>
Dec 28 23:31:35 h2 firewalld: WARNING:<br>
COMMAND_FAILED:<br>
'/usr/sbin/ebtables --concurrent -t nat -X<br>
I-vnet0-arp-mac'<br>
failed:<br>
<br>
<br>
[root@h2 etc]# ovs-vsctl show<br>
ebb6aede-cbbc-4f4f-a88a-a9cd72<wbr>b2bd23<br>
Bridge ovirtbridge<br>
Port "ovirtport0"<br>
Interface "ovirtport0"<br>
type: internal<br>
Port ovirtbridge<br>
Interface ovirtbridge<br>
type: internal<br>
Bridge "ovsbridge0"<br>
Port "ovsbridge0"<br>
Interface "ovsbridge0"<br>
type: internal<br>
Port "eth0"<br>
Interface "eth0"<br>
Bridge br-int<br>
Port br-int<br>
Interface br-int<br>
type: internal<br>
Port "vnet0"<br>
Interface "vnet0"<br>
ovs_version: "2.6.90"<br>
<br>
Searching through the code it appears that br-int<br>
comes from<br>
neutron-openvswitch plugin ??<br>
<br>
[root@h2 share]# rpm -qf<br>
/usr/share/otopi/plugins/ovirt<wbr>-host-deploy/openstack/neutron<wbr>_openvswitch.py<br>
<br>
<br>
<a href="http://ovirt-host-deploy-1.6.0-0.0.ma">ovirt-host-deploy-1.6.0-0.0.ma</a><wbr>ster.20161215101008.gitb76ad50<wbr>.el7.centos.noarch<br>
<br>
<br>
<br>
<br>
/Sverker<br>
<br>
Den 2016-12-28 kl. 23:24, skrev Sverker<br>
Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
In addition I had to add an alias to modprobe:<br>
<br>
[root@h2 modprobe.d]# cat dummy.conf<br>
alias dummy0 dummy<br>
<br>
<br>
Den 2016-12-28 kl. 23:03, skrev Sverker<br>
Abrahamsson:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi<br>
I first tried to set device name to dummy_0, but<br>
then ifup<br>
did<br>
not<br>
succeed in creating the device unless I first<br>
did<br>
'ip link<br>
add<br>
dummy_0 type dummy' but then it would not<br>
suceed to<br>
establish<br>
the if<br>
on reboot.<br>
<br>
Setting fake_nics = dummy0 would not work<br>
neither,<br>
but this<br>
works:<br>
<br>
fake_nics = dummy*<br>
<br>
The engine is now able to find the if and assign<br>
bridge<br>
ovirtmgmt to<br>
it.<br>
<br>
However, I then run into the next issue when<br>
starting a VM:<br>
<br>
2016-12-28 22:28:23,897 ERROR<br>
[org.ovirt.engine.core.dal.dbb<wbr>roker.auditloghandling.AuditLo<wbr>gDirector]<br>
<br>
<br>
(ForkJoinPool-1-worker-2) [] Correlation ID:<br>
null,<br>
Call<br>
Stack:<br>
null,<br>
Custom Event ID: -1, Message: VM CentOS7 is down<br>
with error.<br>
Exit<br>
message: Cannot get interface MTU on<br>
'br-int': No<br>
such<br>
device.<br>
<br>
This VM has a nic on ovirtbridge, which comes<br>
from<br>
the OVN<br>
provider.<br>
<br>
/Sverker<br>
<br>
Den 2016-12-28 kl. 14:38, skrev Marcin Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Sverker,<br>
<br>
Can you try adding a vnic named veth_* or<br>
dummy_*,<br>
(or alternatively add the name of the vnic to<br>
vdsm.config fake_nics), and setup the<br>
management<br>
network using this vnic?<br>
I suppose adding the vnic you use for<br>
connecting<br>
to the engine to fake_nics should make it<br>
visible<br>
to the engine, and you should be able to use<br>
it for<br>
the setup.<br>
<br>
Marcin<br>
<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
To: "Sverker<br>
Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Wednesday, December 28, 2016 12:06:26 PM<br>
Subject: Re: [ovirt-users] Issue with<br>
OVN/OVS and<br>
mandatory<br>
ovirtmgmt network<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I have an internal OVS bridge called<br>
ovirtbridge<br>
which<br>
has<br>
a port<br>
with<br>
IP address, but in the host network settings<br>
that port is<br>
not<br>
visible.<br>
</blockquote>
I just verified and unfortunately the virtual<br>
ports are<br>
not<br>
visible in engine<br>
to assign a network to :(<br>
I'm afraid that the engine is not ready for<br>
such a<br>
scenario<br>
(even<br>
if it<br>
works).<br>
Please give me some time to look for a<br>
solution.<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker<br>
Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
To: "Marcin Mirecki"<<a href="mailto:mmirecki@redhat.com" target="_blank">mmirecki@redhat.com</a>><br>
Cc: "Ovirt Users"<<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Wednesday, December 28, 2016<br>
11:48:24 AM<br>
Subject: Re: [ovirt-users] Issue with<br>
OVN/OVS and<br>
mandatory<br>
ovirtmgmt<br>
network<br>
<br>
Hi Marcin<br>
Yes, that is my issue. I don't want to let<br>
ovirt/vdsm see<br>
eth0<br>
nor<br>
ovsbridge0 since as soon as it sees them it<br>
messes up the<br>
network<br>
config<br>
so that the host will be unreachable.<br>
<br>
I have an internal OVS bridge called<br>
ovirtbridge<br>
which<br>
has<br>
a port<br>
with<br>
IP address, but in the host network settings<br>
that port is<br>
not<br>
visible.<br>
It doesn't help to name it ovirtmgmt.<br>
<br>
The engine is able to communicate with the<br>
host<br>
on the ip<br>
it has<br>
been<br>
given, it's just that it believes that it<br>
HAS to<br>
have a<br>
ovirtmgmt<br>
network which can't be on OVN.<br>
<br>
/Sverker<br>
<br>
<br>
Den 2016-12-28 kl. 10:45, skrev Marcin<br>
Mirecki:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Sverker,<br>
<br>
The management network is mandatory on each<br>
host. It's<br>
used by<br>
the<br>
engine to communicate with the host.<br>
Looking at your description and the<br>
exception<br>
it looks<br>
like it<br>
is<br>
missing.<br>
The error is caused by not having any<br>
network<br>
for the<br>
host<br>
(network list retrieved in<br>
InterfaceDaoImpl.getHostNetwor<wbr>ksByCluster -<br>
which<br>
gets all the networks on nics for a host<br>
from<br>
vds_interface<br>
table in the<br>
DB).<br>
<br>
Could you maybe create a virtual nic<br>
connected to<br>
ovsbridge0 (as<br>
I<br>
understand you<br>
have no physical nic available) and use this<br>
for the<br>
management<br>
network?<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I then create a bridge for use with<br>
ovirt, with<br>
a<br>
private<br>
address.<br>
</blockquote>
I'm not quite sure I understand. Is this yet<br>
another<br>
bridge<br>
connected to<br>
ovsbridge0?<br>
You could also attach the vnic for the<br>
management<br>
network<br>
here<br>
if need<br>
be.<br>
<br>
Please keep in mind that OVN has no use in<br>
setting up<br>
the<br>
management<br>
network.<br>
The OVN provider can only handle external<br>
networks,<br>
which<br>
can<br>
not be used<br>
for a<br>
management network.<br>
<br>
Marcin<br>
<br>
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Sverker<br>
Abrahamsson"<<a href="mailto:sverker@abrahamsson.com" target="_blank">sverker@abrahamss<wbr>on.com</a>><br>
<a href="mailto:To%3Ausers@ovirt.org" target="_blank">To:users@ovirt.org</a><br>
Sent: Wednesday, December 28, 2016<br>
12:39:59 AM<br>
Subject: [ovirt-users] Issue with<br>
OVN/OVS and<br>
mandatory<br>
ovirtmgmt<br>
network<br>
<br>
<br>
<br>
Hi<br>
For long time I've been looking for proper<br>
support in<br>
ovirt for<br>
Open<br>
vSwitch<br>
so I'm happy that it is moving in the right<br>
direction.<br>
However,<br>
there<br>
seems<br>
to still be a dependency on a ovirtmgmt<br>
bridge<br>
and I'm<br>
unable<br>
to move<br>
that<br>
to the OVN provider.<br>
<br>
The hosting center where I rent hw<br>
instances<br>
has a bit<br>
special<br>
network<br>
setup,<br>
so I have one physical network port with<br>
a /32<br>
netmask<br>
and<br>
point-to-point<br>
config to router. The physical port I<br>
connect<br>
to a ovs<br>
bridge<br>
which has<br>
the<br>
public ip. Since ovirt always messes up the<br>
network<br>
config when<br>
I've<br>
tried<br>
to let it have access to the network config<br>
for the<br>
physical<br>
port, I've<br>
set<br>
eht0 and ovsbridge0 as hidden in vdsm.conf.<br>
<br>
<br>
I then create a bridge for use with<br>
ovirt, with<br>
a<br>
private<br>
address. With<br>
the<br>
OVN provider I am now able to import these<br>
into the<br>
engine and<br>
it looks<br>
good. When creating a VM I can select<br>
that it<br>
will have<br>
a<br>
vNic<br>
on my OVS<br>
bridge.<br>
<br>
However, I can't start the VM as an<br>
exception<br>
is thrown<br>
in the<br>
log:<br>
<br>
2016-12-28 00:13:33,350 ERROR<br>
[org.ovirt.engine.core.bll.Run<wbr>VmCommand]<br>
(default task-5) [3c882d53] Error during<br>
ValidateFailure.:<br>
java.lang.NullPointerException<br>
at<br>
org.ovirt.engine.core.bll.sche<wbr>duling.policyunits.NetworkPoli<wbr>cyUnit.validateRequiredNetwork<wbr>sAvailable(NetworkPolicyUnit.<wbr>java:140)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.sche<wbr>duling.policyunits.NetworkPoli<wbr>cyUnit.filter(NetworkPolicyUni<wbr>t.java:69)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.sche<wbr>duling.SchedulingManager.runIn<wbr>ternalFilters(SchedulingManage<wbr>r.java:597)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.sche<wbr>duling.SchedulingManager.runFi<wbr>lters(SchedulingManager.java:<wbr>564)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.sche<wbr>duling.SchedulingManager.canSc<wbr>hedule(SchedulingManager.java:<wbr>494)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.vali<wbr>dator.RunVmValidator.canRunVm(<wbr>RunVmValidator.java:133)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.RunV<wbr>mCommand.validate(RunVmCommand<wbr>.java:940)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.Comm<wbr>andBase.internalValidate(Comma<wbr>ndBase.java:886)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.Comm<wbr>andBase.validateOnly(CommandBa<wbr>se.java:366)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.Prev<wbr>alidatingMultipleActionsRunner<wbr>.canRunActions(PrevalidatingMu<wbr>ltipleActionsRunner.java:113)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.Prev<wbr>alidatingMultipleActionsRunner<wbr>.invokeCommands(PrevalidatingM<wbr>ultipleActionsRunner.java:99)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.Prev<wbr>alidatingMultipleActionsRunner<wbr>.execute(PrevalidatingMultiple<wbr>ActionsRunner.java:76)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.Back<wbr>end.runMultipleActionsImpl(Bac<wbr>kend.java:613)<br>
<br>
<br>
<br>
[bll.jar:]<br>
at<br>
org.ovirt.engine.core.bll.Back<wbr>end.runMultipleActions(Backend<wbr>.java:583)<br>
<br>
<br>
<br>
[bll.jar:]<br>
<br>
<br>
Looking at that section of code where the<br>
exception is<br>
thrown,<br>
I see<br>
that<br>
it<br>
iterates over host networks to find<br>
required<br>
networks,<br>
which I<br>
assume is<br>
ovirtmgmt. In the host network setup<br>
dialog I<br>
don't see<br>
any<br>
networks at<br>
all<br>
but it lists ovirtmgmt as required. It also<br>
list the<br>
OVN<br>
networks but<br>
these<br>
can't be statically assigned as they are<br>
added<br>
dynamically when<br>
needed,<br>
which is fine.<br>
<br>
I believe that I either need to remove<br>
ovirtmgmt<br>
network<br>
or<br>
configure<br>
that<br>
it<br>
is provided by the OVN provider, but<br>
neither is<br>
possible.<br>
Preferably it<br>
shouldn't be hardcoded which network is<br>
management and<br>
mandatory but be<br>
possible to configure.<br>
<br>
/Sverker<br>
Den 2016-12-27 kl. 17:10, skrev Marcin<br>
Mirecki:<br>
<br>
<br>
</blockquote></blockquote></blockquote>
______________________________<wbr>_________________<br>
<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br>
</blockquote></blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote></blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br>
</blockquote></blockquote></blockquote></blockquote></blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote></blockquote></blockquote>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</blockquote></blockquote></blockquote>
</blockquote></blockquote></blockquote>
</blockquote></div><br></div>