<div dir="ltr"><div>I'm a little confused by your explanation of 'just do the bonding at the guest level'. I apologize for my ignorance here, but I'm trying to prepare myself for a similar configuration where I'm going to need to get all much bandwidth out of the bond as possible. How would bonding multiple interfaces at the VM level provide a better balance than at the hypervisor level? Wouldn't the traffic more or less end up traveling the same path regardless of the virtual interface?<br></div><div><br></div><div>I'm trying to plan out an oVirt implementation where I would like to bond multiple interfaces on my hypervisor nodes for balancing/redundancy, and I'm very curious what others have done with Cisco hardware (in my case, a pair of 3650's with MEC) in order to get the best solution.</div><div><br></div><div>I will read through these threads and see if I can gain a better understanding, but if you happen to have an easy explanation that would help my understand, I would greatly appreciate it.</div><div><br></div><div><br></div><div class="gmail_extra"><div class="gmail_quote">On Wed, Dec 31, 2014 at 1:01 AM, Blaster <span dir="ltr"><<a href="mailto:blaster@556nato.com" target="_blank">blaster@556nato.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><br></div><div>Thanks for your thoughts. The problem is, most of the data is transmitted from a couple apps to a couple systems. The chance of a hash collision (i.e., most of the data going out the same interface anyway) is quite high. On Solaris, I just created two physical interfaces each with their own IP, and bound the apps to the appropriate interfaces. This worked great. Imagine my surprise when I discovered this doesn’t work on Linux and my crash course on weak host models.</div><div><br></div><div>Interesting that no one commented on my thought to just do the bonding at the guest level (and use balance-alb) instead of at the hypervisor level. Some ESXi experts I have talked to say this is actually the preferred method with ESXi and not to do it at the hypervisor level, as the VM knows better than VMware.</div><div><br></div><div>Or is the bonding mode issue with balance-alb/tlb more with the Linux TCP stack itself and not with oVirt and VDSM?</div><div><div class="h5"><div><br></div><div><br></div><br><div><div>On Dec 30, 2014, at 4:34 AM, Nikolai Sednev <<a href="mailto:nsednev@redhat.com" target="_blank">nsednev@redhat.com</a>> wrote:</div><br><blockquote type="cite"><div><div style="font-family:georgia,serif;font-size:12pt"><div><span style="color:#333333;font-family:monospace;font-size:13.142857551574707px;background-color:#fdfdfd">Mode 2 will do the job the best way for you in case of static LAG supported only at the switch's side, I'd advise using of xmit_hash_policy layer3+4, so you'll get better distribution for your DC.</span></div><div><br></div><div><span name="x"></span><br>Thanks in advance.<br><div><br></div>Best regards,<br>Nikolai<br>____________________<br>Nikolai Sednev<br>Senior Quality Engineer at Compute team<br>Red Hat Israel<br>34 Jerusalem Road,<br>Ra'anana, Israel 43501<br><div><br></div>Tel: <a href="tel:%2B972%20%C2%A0%209%207692043" value="+97297692043" target="_blank">+972 9 7692043</a><br>Mobile: <a href="tel:%2B972%2052%207342734" value="+972527342734" target="_blank">+972 52 7342734</a><br>Email: <a href="mailto:nsednev@redhat.com" target="_blank">nsednev@redhat.com</a><br>IRC: nsednev<span name="x"></span><br></div><div><br></div><hr><div style="font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt"><b>From: </b><a href="mailto:users-request@ovirt.org" target="_blank">users-request@ovirt.org</a><br><b>To: </b><a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br><b>Sent: </b>Tuesday, December 30, 2014 2:12:58 AM<br><b>Subject: </b>Users Digest, Vol 39, Issue 173<br><div><br></div>Send Users mailing list submissions to<br> <a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br><div><br></div>To subscribe or unsubscribe via the World Wide Web, visit<br> <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>or, via email, send a message with subject or body 'help' to<br> <a href="mailto:users-request@ovirt.org" target="_blank">users-request@ovirt.org</a><br><div><br></div>You can reach the person managing the list at<br> <a href="mailto:users-owner@ovirt.org" target="_blank">users-owner@ovirt.org</a><br><div><br></div>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Users digest..."<br><div><br></div><br>Today's Topics:<br><div><br></div> 1. Re: ??: bond mode balance-alb (Jorick Astrego)<br> 2. Re: ??: bond mode balance-alb (Jorick Astrego)<br> 3. HostedEngine Deployment Woes (Mikola Rose)<br><div><br></div><br>----------------------------------------------------------------------<br><div><br></div>Message: 1<br>Date: Mon, 29 Dec 2014 20:13:40 +0100<br>From: Jorick Astrego <<a href="mailto:j.astrego@netbulae.eu" target="_blank">j.astrego@netbulae.eu</a>><br>To: <a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>Subject: Re: [ovirt-users] ??: bond mode balance-alb<br>Message-ID: <<a href="mailto:54A1A7E4.90308@netbulae.eu" target="_blank">54A1A7E4.90308@netbulae.eu</a>><br>Content-Type: text/plain; charset="utf-8"<br><div><br></div><br>On 12/29/2014 12:56 AM, Dan Kenigsberg wrote:<br>> On Fri, Dec 26, 2014 at 12:39:45PM -0600, Blaster wrote:<br>>> On 12/23/2014 2:55 AM, Dan Kenigsberg wrote:<br>>>> Bug 1094842 - Bonding modes 0, 5 and 6 should be avoided for VM networks<br>>>> <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1094842#c0" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1094842#c0</a><br>>> Dan,<br>>><br>>> What is bad about these modes that oVirt can't use them?<br>> I can only quote jpirko's workds from the link above:<br>><br>> Do not use tlb or alb in bridge, never! It does not work, that's it. The reason<br>> is it mangles source macs in xmit frames and arps. When it is possible, just<br>> use mode 4 (lacp). That should be always possible because all enterprise<br>> switches support that. Generally, for 99% of use cases, you *should* use mode<br>> 4. There is no reason to use other modes.<br>><br>This switch is more of an office switch and only supports part of the<br>802.3ad standard:<br><div><br></div><br> PowerConnect* *2824<br><div><br></div> Scalable from small workgroups to dense access solutions, the 2824<br> offers 24-port flexibility plus two combo small?form?factor<br> pluggable (SFP) ports for connecting the switch to other networking<br> equipment located beyond the 100 m distance limitations of copper<br> cabling.<br><div><br></div> Industry-standard link aggregation adhering to IEEE 802.3ad<br> standards (static support only, LACP not supported)<br><div><br></div><br>So the only way to have some kind of bonding without buying more<br>expensive switches, is using balance-rr (mode=0), balance-xor (mode=2)<br>or broadcast (modes=3).<br>>> I just tested mode 4, and the LACP with Fedora 20 appears to not be<br>>> compatible with the LAG mode on my Dell 2824.<br>>><br>>> Would there be any issues with bringing two NICS into the VM and doing<br>>> balance-alb at the guest level?<br>>><br>Kind regards,<br><div><br></div>Jorick Astrego<br><div><br></div><br><div><br></div>Met vriendelijke groet, With kind regards,<br><div><br></div>Jorick Astrego<br><div><br></div>Netbulae Virtualization Experts <br><div><br></div>----------------<br><div><br></div> Tel: 053 20 30 270 <a href="mailto:info@netbulae.eu" target="_blank">info@netbulae.eu</a> Staalsteden 4-3A KvK 08198180<br> Fax: 053 20 30 271 <a href="http://www.netbulae.eu" target="_blank">www.netbulae.eu</a> 7547 TA Enschede BTW NL821234584B01<br><div><br></div>----------------<br><div><br></div>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://lists.ovirt.org/pipermail/users/attachments/20141229/dfacba22/attachment-0001.html" target="_blank">http://lists.ovirt.org/pipermail/users/attachments/20141229/dfacba22/attachment-0001.html</a>><br><div><br></div>------------------------------<br><div><br></div>Message: 2<br>Date: Mon, 29 Dec 2014 20:14:55 +0100<br>From: Jorick Astrego <<a href="mailto:j.astrego@netbulae.eu" target="_blank">j.astrego@netbulae.eu</a>><br>To: <a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>Subject: Re: [ovirt-users] ??: bond mode balance-alb<br>Message-ID: <<a href="mailto:54A1A82F.1090100@netbulae.eu" target="_blank">54A1A82F.1090100@netbulae.eu</a>><br>Content-Type: text/plain; charset="utf-8"<br><div><br></div><br>On 12/29/2014 12:56 AM, Dan Kenigsberg wrote:<br>> On Fri, Dec 26, 2014 at 12:39:45PM -0600, Blaster wrote:<br>>> On 12/23/2014 2:55 AM, Dan Kenigsberg wrote:<br>>>> Bug 1094842 - Bonding modes 0, 5 and 6 should be avoided for VM networks<br>>>> <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1094842#c0" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1094842#c0</a><br>>><br>Sorry, no mode 0. So only mode 2 or 3 for your environment....<br><div><br></div>Kind regards,<br><div><br></div>Jorick<br><div><br></div><br><div><br></div>Met vriendelijke groet, With kind regards,<br><div><br></div>Jorick Astrego<br><div><br></div>Netbulae Virtualization Experts <br><div><br></div>----------------<br><div><br></div> Tel: 053 20 30 270 <a href="mailto:info@netbulae.eu" target="_blank">info@netbulae.eu</a> Staalsteden 4-3A KvK 08198180<br> Fax: 053 20 30 271 <a href="http://www.netbulae.eu" target="_blank">www.netbulae.eu</a> 7547 TA Enschede BTW NL821234584B01<br><div><br></div>----------------<br><div><br></div>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://lists.ovirt.org/pipermail/users/attachments/20141229/41da033b/attachment-0001.html" target="_blank">http://lists.ovirt.org/pipermail/users/attachments/20141229/41da033b/attachment-0001.html</a>><br><div><br></div>------------------------------<br><div><br></div>Message: 3<br>Date: Tue, 30 Dec 2014 00:12:52 +0000<br>From: Mikola Rose <<a href="mailto:mrose@power-soft.com" target="_blank">mrose@power-soft.com</a>><br>To: "<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>" <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>Subject: [ovirt-users] HostedEngine Deployment Woes<br>Message-ID: <<a href="mailto:F992C848-E4EB-468E-83F4-37646EDB3E62@power-soft.com" target="_blank">F992C848-E4EB-468E-83F4-37646EDB3E62@power-soft.com</a>><br>Content-Type: text/plain; charset="us-ascii"<br><div><br></div><br>Hi List Members;<br><div><br></div>I have been struggling with deploying oVirt hosted engine I keep running into a timeout during the "Misc Configuration" any suggestion on how I can trouble shoot this?<br><div><br></div>Redhat 2.6.32-504.3.3.el6.x86_64<br><div><br></div>Installed Packages<br>ovirt-host-deploy.noarch 1.2.5-1.el6ev @rhel-6-server-rhevm-3.4-rpms<br>ovirt-host-deploy-java.noarch 1.2.5-1.el6ev @rhel-6-server-rhevm-3.4-rpms<br>ovirt-hosted-engine-ha.noarch 1.1.6-3.el6ev @rhel-6-server-rhevm-3.4-rpms<br>ovirt-hosted-engine-setup.noarch 1.1.5-1.el6ev @rhel-6-server-rhevm-3.4-rpms<br>rhevm-setup-plugin-ovirt-engine.noarch 3.4.4-2.2.el6ev @rhel-6-server-rhevm-3.4-rpms<br>rhevm-setup-plugin-ovirt-engine-common.noarch 3.4.4-2.2.el6ev @rhel-6-server-rhevm-3.4-rpms<br><div><br></div><br> Please confirm installation settings (Yes, No)[No]: Yes<br>[ INFO ] Stage: Transaction setup<br>[ INFO ] Stage: Misc configuration<br>[ INFO ] Stage: Package installation<br>[ INFO ] Stage: Misc configuration<br>[ INFO ] Configuring libvirt<br>[ INFO ] Configuring VDSM<br>[ INFO ] Starting vdsmd<br>[ INFO ] Waiting for VDSM hardware info<br>[ INFO ] Waiting for VDSM hardware info<br>[ INFO ] Connecting Storage Domain<br>[ INFO ] Connecting Storage Pool<br>[ INFO ] Verifying sanlock lockspace initialization<br>[ INFO ] sanlock lockspace already initialized<br>[ INFO ] sanlock metadata already initialized<br>[ INFO ] Creating VM Image<br>[ INFO ] Disconnecting Storage Pool<br>[ INFO ] Start monitoring domain<br>[ ERROR ] Failed to execute stage 'Misc configuration': The read operation timed out<br>[ INFO ] Stage: Clean up<br>[ INFO ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf'<br>[ INFO ] Stage: Pre-termination<br>[ INFO ] Stage: Termination<br><div><br></div><br><div><br></div>2014-12-29 14:53:41 DEBUG otopi.plugins.ovirt_hosted_engine_setup.sanlock.lockspace lockspace._misc:133 Ensuring lease for lockspace hosted-engine, host id 1 is acquired (file: /rhev/data-center/mnt/192.168.0.75:_Volumes_Raid1/8094d528-7aa2-4c28-839f-73d7c8bcfebb/ha_agent/hosted-engine.lockspace)<br>2014-12-29 14:53:41 INFO otopi.plugins.ovirt_hosted_engine_setup.sanlock.lockspace lockspace._misc:144 sanlock lockspace already initialized<br>2014-12-29 14:53:41 INFO otopi.plugins.ovirt_hosted_engine_setup.sanlock.lockspace lockspace._misc:157 sanlock metadata already initialized<br>2014-12-29 14:53:41 DEBUG otopi.context context._executeMethod:138 Stage misc METHOD otopi.plugins.ovirt_hosted_engine_setup.vm.image.Plugin._misc<br>2014-12-29 14:53:41 INFO otopi.plugins.ovirt_hosted_engine_setup.vm.image image._misc:162 Creating VM Image<br>2014-12-29 14:53:41 DEBUG otopi.plugins.ovirt_hosted_engine_setup.vm.image image._misc:163 createVolume<br>2014-12-29 14:53:42 DEBUG otopi.plugins.ovirt_hosted_engine_setup.vm.image image._misc:184 Created volume d8e7eed4-c763-4b3d-8a71-35f2d692a73d, request was:<br>- image: 9043e535-ea94-41f8-98df-6fdbfeb107c3<br>- volume: e6a9291d-ac21-4a95-b43c-0d6e552baaa2<br>2014-12-29 14:53:42 DEBUG otopi.ovirt_hosted_engine_setup.tasks tasks.wait:48 Waiting for existing tasks to complete<br>2014-12-29 14:53:43 DEBUG otopi.ovirt_hosted_engine_setup.tasks tasks.wait:48 Waiting for existing tasks to complete<br>2014-12-29 14:53:43 DEBUG otopi.context context._executeMethod:138 Stage misc METHOD otopi.plugins.ovirt_hosted_engine_setup.vm.boot_disk.Plugin._misc<br>2014-12-29 14:53:43 DEBUG otopi.context context._executeMethod:144 condition False<br>2014-12-29 14:53:43 DEBUG otopi.context context._executeMethod:138 Stage misc METHOD otopi.plugins.ovirt_hosted_engine_setup.storage.storage.Plugin._disconnect_pool<br>2014-12-29 14:53:43 INFO otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._disconnect_pool:971 Disconnecting Storage Pool<br>2014-12-29 14:53:43 DEBUG otopi.ovirt_hosted_engine_setup.tasks tasks.wait:48 Waiting for existing tasks to complete<br>2014-12-29 14:53:43 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._spmStop:602 spmStop<br>2014-12-29 14:53:43 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._spmStop:611<br>2014-12-29 14:53:43 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._storagePoolConnection:573 disconnectStoragePool<br>2014-12-29 14:53:45 INFO otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._disconnect_pool:975 Start monitoring domain<br>2014-12-29 14:53:45 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._startMonitoringDomain:529 _startMonitoringDomain<br>2014-12-29 14:53:46 DEBUG otopi.plugins.ovirt_hosted_engine_setup.storage.storage storage._startMonitoringDomain:534 {'status': {'message': 'OK', 'code': 0}}<br>2014-12-29 14:53:51 DEBUG otopi.ovirt_hosted_engine_setup.tasks tasks.wait:127 Waiting for domain monitor<br>2014-12-29 14:54:51 DEBUG otopi.context context._executeMethod:152 method exception<br>Traceback (most recent call last):<br> File "/usr/lib/python2.6/site-packages/otopi/context.py", line 142, in _executeMethod<br> method['method']()<br> File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/storage.py", line 976, in _disconnect_pool<br> self._startMonitoringDomain()<br> File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/storage.py", line 539, in _startMonitoringDomain<br> waiter.wait(self.environment[ohostedcons.StorageEnv.SD_UUID])<br> File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/tasks.py", line 128, in wait<br> response = serv.s.getVdsStats()<br> File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__<br> return self.__send(self.__name, args)<br> File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request<br> verbose=self.__verbose<br> File "/usr/lib64/python2.6/xmlrpclib.py", line 1237, in request<br> errcode, errmsg, headers = h.getreply()<br> File "/usr/lib64/python2.6/httplib.py", line 1064, in getreply<br> response = self._conn.getresponse()<br> File "/usr/lib64/python2.6/httplib.py", line 990, in getresponse<br> response.begin()<br> File "/usr/lib64/python2.6/httplib.py", line 391, in begin<br> version, status, reason = self._read_status()<br> File "/usr/lib64/python2.6/httplib.py", line 349, in _read_status<br> line = self.fp.readline()<br> File "/usr/lib64/python2.6/socket.py", line 433, in readline<br> data = recv(1)<br> File "/usr/lib64/python2.6/ssl.py", line 215, in recv<br> return self.read(buflen)<br> File "/usr/lib64/python2.6/ssl.py", line 136, in read<br> return self._sslobj.read(len)<br>SSLError: The read operation timed out<br><div><br></div><br><div><br></div><br>var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20141229145137-g8d2or.log<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="http://lists.ovirt.org/pipermail/users/attachments/20141230/899f724c/attachment.html" target="_blank">http://lists.ovirt.org/pipermail/users/attachments/20141230/899f724c/attachment.html</a>><br><div><br></div>------------------------------<br><div><br></div>_______________________________________________<br>Users mailing list<br><a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br><a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br><div><br></div><br>End of Users Digest, Vol 39, Issue 173<br>**************************************<br></div><div><br></div></div></div>_______________________________________________<br>Users mailing list<br><a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br><a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br></blockquote></div><br></div></div></div><br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div></div>