<div dir="ltr"><div class="gmail_default">I put the management and storage on separate VLANs to try avoid the floating IP address issue temporarily. I also bonded the two nics, but I don't think that shouldn't matter.</div>
<div class="gmail_default"><br></div><div class="gmail_default">The other server got brought down the other day for some maintenance, I hope to get it back up in a few days. But I can tell you a few things I noticed:</div>
<div class="gmail_default"><br></div><div class="gmail_default">ip a - it'll list the floating IP on both servers even if only active on one.</div><div class="gmail_default"><br></div><div class="gmail_default">I've got about 10 other networks so I've snipped out quite a bit.</div>
<div class="gmail_default"><br></div><div class="gmail_default"># ip a</div><div class="gmail_default"><snip></div><div class="gmail_default"><div class="gmail_default">130: bond0.2@bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP </div>
<div class="gmail_default"> link/ether 00:10:18:2e:6a:cb brd ff:ff:ff:ff:ff:ff</div><div class="gmail_default"> inet <a href="http://172.16.0.11/24">172.16.0.11/24</a> brd 172.16.0.255 scope global bond0.2</div><div class="gmail_default">
inet6 fe80::210:18ff:fe2e:6acb/64 scope link </div><div class="gmail_default"> valid_lft forever preferred_lft forever</div><div class="gmail_default">131: bond0.3@bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP </div>
<div class="gmail_default"> link/ether 00:10:18:2e:6a:cb brd ff:ff:ff:ff:ff:ff</div><div class="gmail_default"> inet <a href="http://172.16.1.11/24">172.16.1.11/24</a> brd 172.16.1.255 scope global bond0.3</div><div class="gmail_default">
inet <a href="http://172.16.1.5/32">172.16.1.5/32</a> scope global bond0.3</div><div class="gmail_default"> inet6 fe80::210:18ff:fe2e:6acb/64 scope link </div><div class="gmail_default"> valid_lft forever preferred_lft forever</div>
<div></snip></div></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default"># <span style="font-family:arial,sans-serif;font-size:13px">vdsClient -s 0 getVdsCaps</span></div>
<div class="gmail_default"><snip></div><div class="gmail_default"><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'storage_network': {'addr': '172.16.1.5',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'bridged': False,</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'gateway': '172.16.1.1',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'iface': 'bond0.3',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'interface': 'bond0.3',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'ipv6addrs': ['fe80::210:18ff:fe2e:6acb/64'],</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'ipv6gateway': '::',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'mtu': '1500',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'netmask': '255.255.255.255',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'qosInbound': '',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'qosOutbound': ''},</div>
<div><snip></div></div><div class="gmail_default"><div class="gmail_default"><span class="" style="white-space:pre"> </span>vlans = {'bond0.2': {'addr': '172.16.0.11',</div><div class="gmail_default">
<span class="" style="white-space:pre"> </span> 'cfg': {'BOOTPROTO': 'none',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'DEFROUTE': 'yes',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'DEVICE': 'bond0.2',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'GATEWAY': '172.16.0.1',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'IPADDR': '172.16.0.11',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'NETMASK': '255.255.255.0',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'NM_CONTROLLED': 'no',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'ONBOOT': 'yes',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'VLAN': 'yes'},</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'iface': 'bond0',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'ipv6addrs': ['fe80::210:18ff:fe2e:6acb/64'],</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'mtu': '1500',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'netmask': '255.255.255.0',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'vlanid': 2},</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'bond0.3': {'addr': '172.16.1.5',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'cfg': {'BOOTPROTO': 'none',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'DEFROUTE': 'no',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'DEVICE': 'bond0.3',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'IPADDR': '172.16.1.11',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'NETMASK': '255.255.255.0',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'NM_CONTROLLED': 'no',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'ONBOOT': 'yes',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'VLAN': 'yes'},</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'iface': 'bond0',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'ipv6addrs': ['fe80::210:18ff:fe2e:6acb/64'],</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'mtu': '1500',</div>
<div class="gmail_default"><span class="" style="white-space:pre"> </span> 'netmask': '255.255.255.255',</div><div class="gmail_default"><span class="" style="white-space:pre"> </span> 'vlanid': 3},</div>
<div><br></div><div>I hope that's enough info, if not I'll post the full config on both when I can bring it back up.</div><div><br></div><div>Cheers,<br>Andrew.</div><div><br></div></div><div class="gmail_default" style="font-family:tahoma,sans-serif">
<span style="font-family:arial"><br></span></div><div class="gmail_default" style="font-family:tahoma,sans-serif"><span style="font-family:arial">On Sun, Dec 1, 2013 at 7:15 PM, Assaf Muller </span><span dir="ltr" style="font-family:arial"><<a href="mailto:amuller@redhat.com" target="_blank">amuller@redhat.com</a>></span><span style="font-family:arial"> wrote:</span><br>
</div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Could you please attach the output of:<br>
"vdsClient -s 0 getVdsCaps"<br>
(Or without the -s, whichever works)<br>
And:<br>
"ip a"<br>
<br>
On both hosts?<br>
You seem to have made changes since the documentation on the link you provided, like separating the management and storage via VLANs on eth0. Any other changes?<br>
<br>
<br>
Assaf Muller, Cloud Networking Engineer<br>
Red Hat<br>
<div class=""><div class="h5"><br>
----- Original Message -----<br>
From: "Andrew Lau" <<a href="mailto:andrew@andrewklau.com">andrew@andrewklau.com</a>><br>
To: "users" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>><br>
Sent: Sunday, December 1, 2013 4:55:32 AM<br>
Subject: [Users] Keepalived on oVirt Hosts has engine networking issues<br>
<br>
Hi,<br>
<br>
I have the scenario where I have gluster and ovirt hosts on the same box, to keep the gluster volumes highly available incase a box drops I'm using keepalived across the boxes and using that IP as the means for the storage domain. I documented my setup here in case anyone needs a little more info <a href="http://www.andrewklau.com/returning-to-glusterized-ovirt-3-3/" target="_blank">http://www.andrewklau.com/returning-to-glusterized-ovirt-3-3/</a><br>
<br>
However, the engine seems to be picking up the floating IP assigned to keepalived as the interface and messing with the ovirtmgmt migration network, so migrations are failing as my floating IP gets assigned to the ovirtmgmt bridge in the engine however it's not actually there on most hosts (except one) so vdsm seems to report destination same as source.<br>
<br>
I've since created a new vlan interface just for storage to avoid the ovirtmgmt conflict, but the engine will still pick up the wrong IP on the storage vlan because of keepalived. This means I can't use the save network feature within the engine as it'll save the floating ip rather than the one already there. Is this a bug or just the way it's designed.<br>
<br>
eth0.2 -> ovirtmgmt (172.16.0.11) -> management and migration network -> engine sees, sets and saves 172.16.0.11<br>
eth0.3 -> storagenetwork (172.16.1.11) -> gluster network -> engine sees, sets and saves 172.16.1.5 (my floating IP)<br>
<br>
I hope this makes sense.<br>
<br>
p.s. can anyone also confirm, does gluster support multi pathing by default? If I'm using this keepalived method, am I bottle necking myself to one host?<br>
<br>
Thanks,<br>
Andrew<br>
<br>
</div></div><div class=""><div class="h5">_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</div></div></blockquote></div><br></div></div>