<html><body><div style="font-family: arial, helvetica, sans-serif; font-size: 12pt; color: #000000"><div>Hi Gianluca,<br></div><div><br data-mce-bogus="1"></div><div>My answers below,<br data-mce-bogus="1"></div><div><br></div><div data-marker="__SIG_PRE__"><div>cordialement, regards,</div><div><br data-mce-bogus="1"></div><table style="margin: 0px; table-layout: fixed; border: none; width: 345pt; padding: 0px;" data-mce-style="margin: 0px; table-layout: fixed; border: none; width: 345pt; padding: 0px;" data-mce-selected="1" width="460" cellspacing="0" cellpadding="0" border="0"><tbody><tr><td style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 86.25pt; text-align: left;" data-mce-style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 86.25pt; text-align: left;" width="115" valign="top"><a href="https://www.lyra-network.com/" style="border: none; text-decoration: none; width: 100%; display: block; clear: both;" data-mce-href="https://www.lyra-network.com/" data-mce-style="border: none; text-decoration: none; width: 100%; display: block; clear: both;"><img src="cid:25cd0e595e0f025605ddb995496484dcb70629d9@zimbra" alt="" data-mce-src="/home/admin@lyra-network.com/corporate-documents/element-signature_logo_lyra_115x94.jpg" doc="corporate-documents/element-signature_logo_lyra_115x94.jpg" height="94" width="115" border="0"></a><br data-mce-bogus="1"></td><td style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 258.75pt;" data-mce-style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 258.75pt;" width="345" valign="top"><table style="margin: 0; width: 258.75pt; table-layout: fixed; border: none;" data-mce-style="margin: 0; width: 258.75pt; table-layout: fixed; border: none;" width="345" cellspacing="0" cellpadding="0" border="0"><tbody><tr><td style="font-size: 16px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 18px; font-weight: bold;" data-mce-style="font-size: 16px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 18px; font-weight: bold;">Nelson LAMEIRAS</td></tr><tr><td style="font-family: Arial,sans-serif; color: #2b3c7a; font-size: 11px; line-height: 16px;" data-mce-style="font-family: Arial,sans-serif; color: #2b3c7a; font-size: 11px; line-height: 16px;">Ingénieur Systèmes et Réseaux<span style="font-family: Arial,sans-serif; color: #337ab7; font-size: 11px;" data-mce-style="font-family: Arial,sans-serif; color: #337ab7; font-size: 11px;"> / Systems and Networks engineer</span></td></tr><tr><td style="line-height: 16px; font-family: 'Arial',sans-serif; color: #2b3c7a; font-size: 11px;" data-mce-style="line-height: 16px; font-family: 'Arial',sans-serif; color: #2b3c7a; font-size: 11px;"><span style="font-family: Arial,sans-serif; color: #2b3c7a; font-size: 11px;" data-mce-style="font-family: Arial,sans-serif; color: #2b3c7a; font-size: 11px;">Tel: +33 5 32 09 09 70</span></td></tr><tr><td style="line-height: 16px; font-family: 'Arial',sans-serif; color: #2b3c7a; font-size: 11px;" data-mce-style="line-height: 16px; font-family: 'Arial',sans-serif; color: #2b3c7a; font-size: 11px;"><a href="mailto:nelson.lameiras@lyra-network.com" style="font-size: 11px; font-family: Arial,sans-serif; line-height: 16px; text-decoration: none;" data-mce-href="mailto:nelson.lameiras@lyra-network.com" data-mce-style="font-size: 11px; font-family: Arial,sans-serif; line-height: 16px; text-decoration: none;">nelson.lameiras@lyra-network.com</a><br data-mce-bogus="1"></td></tr><tr><td style="line-height: 16px; font-family: 'Arial',sans-serif; color: #2b3c7a; font-size: 11px;" data-mce-style="line-height: 16px; font-family: 'Arial',sans-serif; color: #2b3c7a; font-size: 11px;"><a href="https://www.lyra-network.com/" style="font-size: 11px; font-family: Arial,sans-serif; line-height: 16px; border: none; text-decoration: none;" data-mce-href="https://www.lyra-network.com/" data-mce-style="font-size: 11px; font-family: Arial,sans-serif; line-height: 16px; border: none; text-decoration: none;">www.lyra-network.com</a> <a href="https://payzen.eu" style="font-size: 11px; font-family: Arial,sans-serif; line-height: 16px; border: none; text-decoration: none;" data-mce-href="https://payzen.eu" data-mce-style="font-size: 11px; font-family: Arial,sans-serif; line-height: 16px; border: none; text-decoration: none;"> | www.payzen.eu</a><br data-mce-bogus="1"></td></tr></tbody></table></td></tr></tbody></table><table style="margin: 0; table-layout: fixed; border: none; width: 345pt;" data-mce-style="margin: 0; table-layout: fixed; border: none; width: 345pt;" data-mce-selected="1" width="460" cellspacing="0" cellpadding="0"><tbody><tr><td style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 24pt; text-align: left;" data-mce-style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 24pt; text-align: left;" width="32" valign="top"><a href="https://www.youtube.com/channel/UCrVl1CO_Jlu3KbiRH-tQ_vA" style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 32px;" data-mce-href="https://www.youtube.com/channel/UCrVl1CO_Jlu3KbiRH-tQ_vA" data-mce-style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 32px;"><img src="cid:4c5918308790526a8092b3bb40ae8dab65d1777b@zimbra" alt="" data-mce-src="/home/admin@lyra-network.com/corporate-documents/element-signature_logo_YouTube_32x28.jpg" doc="corporate-documents/element-signature_logo_YouTube_32x28.jpg" height="28" width="32" border="0"></a><br data-mce-bogus="1"></td><td style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 30.75pt; text-align: left;" data-mce-style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 30.75pt; text-align: left;" width="41" valign="top"><a href="https://www.linkedin.com/company/lyra-network_2" style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 41px;" data-mce-href="https://www.linkedin.com/company/lyra-network_2" data-mce-style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 41px;"><img src="cid:0c1a4f5470e19b606b0a0890da695ad9d2808a60@zimbra" data-mce-src="/home/admin@lyra-network.com/corporate-documents/element-signature_logo_LinkedIn_41x28.jpg" doc="corporate-documents/element-signature_logo_LinkedIn_41x28.jpg" height="28" width="41" border="0"></a><br data-mce-bogus="1"></td><td style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 31.5pt; text-align: left;" data-mce-style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 31.5pt; text-align: left;" width="42" valign="top"><a href="https://twitter.com/LyraNetwork" style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 42px;" data-mce-href="https://twitter.com/LyraNetwork" data-mce-style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 42px;"><img src="cid:5f6dbbaff6dac1a2e25c3c96f02d8020be9c5268@zimbra" alt="" data-mce-src="/home/admin@lyra-network.com/corporate-documents/element-signature_logo_Twitter_42x28.jpg" doc="corporate-documents/element-signature_logo_Twitter_42x28.jpg" height="28" width="42" border="0"></a><br data-mce-bogus="1"></td><td style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 258.75pt; text-align: left;" data-mce-style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 258.75pt; text-align: left;" width="345" valign="top"><a href="https://payzen.eu" style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 61px;" data-mce-href="https://payzen.eu" data-mce-style="border: none; text-decoration: none; margin: 0; float: left; display: inline-block; width: 61px;"><img src="cid:7003ef2927fa834b99af5bd467d27eb6d701e6b1@zimbra" alt="" data-mce-src="/home/admin@lyra-network.com/corporate-documents/element-signature_payzen_61x28.jpg" doc="corporate-documents/element-signature_payzen_61x28.jpg" height="28" width="61" border="0"></a><br data-mce-bogus="1"></td></tr></tbody></table><table style="margin: 0; table-layout: fixed; border: none; width: 345pt;" data-mce-style="margin: 0; table-layout: fixed; border: none; width: 345pt;" width="460" cellspacing="0" cellpadding="0"><tbody><tr><td><hr style="background: #293b79 none repeat scroll 0% 0%; width: 100%; height: 4px; color: #293b79; margin: 0px;" data-mce-style="background: #293b79 none repeat scroll 0% 0%; width: 100%; height: 4px; color: #293b79; margin: 0px;"></td></tr></tbody></table><table style="margin: 0; table-layout: fixed; border: none; width: 345pt;" data-mce-style="margin: 0; table-layout: fixed; border: none; width: 345pt;" width="460" cellspacing="0" cellpadding="0"><tbody><tr><td style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 30.75pt; text-align: left;" data-mce-style="margin: 0; font-size: 11px; font-family: Arial,sans-serif; color: #2b3c7a; line-height: 16px; width: 30.75pt; text-align: left;"><div>Lyra Network, 109 rue de l'innovation, 31670 Labège, FRANCE</div></td></tr></tbody></table><div><br></div></div><div><br></div><hr id="zwchr" data-marker="__DIVIDER__"><div data-marker="__HEADERS__"><b>From: </b>"Gianluca Cecchi" <gianluca.cecchi@gmail.com><br><b>To: </b>"Nelson Lameiras" <nelson.lameiras@lyra-network.com><br><b>Cc: </b>"ovirt users" <users@ovirt.org><br><b>Sent: </b>Thursday, June 8, 2017 9:36:33 AM<br><b>Subject: </b>Re: [ovirt-users] ISCSI storage with multiple nics on same subnet disabled on host activation<br></div><div><br></div><div data-marker="__QUOTED_TEXT__"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote>On Wed, Jun 7, 2017 at 2:59 PM, Nelson Lameiras <span dir="ltr"><<a href="mailto:nelson.lameiras@lyra-network.com" target="_blank">nelson.lameiras@lyra-network.com</a>></span> wrote:</blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:arial,helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><div>Hello,<br></div><br><div>In our oVirt hosts, we are using DELL equallogic SAN with each server connecting to SAN via 2 physical interfaces. Since both interfaces share the same network (Equalogic limitation) we must patch the linux kernel to to allow iSCSI multipath with multiple NICs in the same subnet with sysctl :<br></div><br><div>--------------------------------------------------------------------------------------------<br></div><div><pre>net.ipv4.conf.p2p1.arp_ignore=1
net.ipv4.conf.p2p1.arp_announce=2
net.ipv4.conf.p2p1.rp_filter=2
net.ipv4.conf.p2p2.arp_ignore=1
net.ipv4.conf.p2p2.arp_announce=2
net.ipv4.conf.p2p2.rp_filter=2<br><br></pre></div></div></div></blockquote><blockquote><div>Hello Nelson,</div><div>nice to see another environment based on EQL iSCSI SAN.</div><div>The systctl.conf customized lines above are not what you mean with "we must patch the linux kernel", is it?</div><div>What do you mean exactly?</div></blockquote></div><div class="gmail_quote"><br data-mce-bogus="1"></div><div class="gmail_quote">Sorry, by "patch" I meant "sysctl.conf configuration" indeed. <br></div><div class="gmail_quote">This configuration is what we use to be able to configure 2 network interfaces in the same subnet.<br><div> <br></div><div><div style="font-family:arial,helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><div><blockquote><p>--------------------------------------------------------------------------------------------<br><br></p></blockquote></div></div></div><blockquote><div><div style="font-family:arial,helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><div><p>This works great in most setups, but for a strange reason, on some of our setups, the sysctl configuration is updated by VDSM when activating a host and the second interface stops working immeadiatly :</p><div>--------------------------------------------------------------------------------------------</div><div>vdsm.log<br></div><div><pre id="gmail-m_3172160333806361248preview"><code>2017-06-07 11:51:51,063+0200 INFO (jsonrpc/5) [storage.ISCSI] Setting strict mode rp_filter for device 'p2p2'. (iscsi:602)
2017-06-07 11:51:51,064+0200 ERROR (jsonrpc/5) [storage.HSM] Could not connect to storageServer (hsm:2392)
Traceback (most recent call last):
File "/usr/share/vdsm/storage/hsm.py", line 2389, in connectStorageServer
conObj.connect()
File "/usr/share/vdsm/storage/storageServer.py", line 433, in connect
iscsi.addIscsiNode(self._iface, self._target, self._cred)
File "/usr/lib/python2.7/site-packages/vdsm/storage/iscsi.py", line 232, in addIscsiNode
iscsiadm.node_login(<a href="http://iface.name" target="_blank">iface.name</a>, target.address, target.iqn)
File "/usr/lib/python2.7/site-packages/vdsm/storage/iscsiadm.py", line 337, in node_login
raise IscsiNodeError(rc, out, err)</code></pre><pre><br></pre></div></div></div></div><div>In my environment I don't see these kinds of problems.</div><div>I'm using plain CentOS 7.3 hosts as hypervisors; is this your case too or are you using ovirt-ng nodes?</div><div><br data-mce-bogus="1"></div></blockquote><div>Plain CentOS 7.3 hosts also. Built from a Minimal centos and adding oVirt repositories.<br></div><div><br data-mce-bogus="1"></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:arial,helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><div><div><p style="margin:0px">--------------------------------------------------------------------------------------------</p></div></div><div><br><div>"strict mode" is enforced for second interface, and it no longuer works...</div><div>Which means - at least - that there is no redundancy in case of hardware faillure and this is not acceptable for our production needs.<br></div><br><div>What is really strange is that we have another "twin" site on another geographic region with simillar hardware configuration and same oVirt installation, and this problem does not happen.</div><br><div>What can be the root cause of this behaviour? How can I correct it?<br></div><br><br></div></div></div></blockquote><blockquote><div>Actually it is very strange to me to understand how you don't have problems on the other side.</div><div>My workaround to get multipath with EQL in oVirt was the one explained here with screenshots:</div><div><a href="http://lists.ovirt.org/pipermail/users/2017-March/080992.html" target="_blank">http://lists.ovirt.org/pipermail/users/2017-March/080992.html</a><br data-mce-bogus="1"></div><div>It has been working now for about 3 months without problems. We have several Oracle RDBMS server VMs on it, from 2virtual threads and 8Gb of ram to 16 threads (2-4-2) and 64Gb of ram </div><div>How did you configure multipath instead? As far as I know oVirt will not let you do it.</div><div>The only way would be to use bonding but that is not certified/supported by Dell.</div><div>HIH sharing experiences,</div><div>Gianluca</div><div><br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div></blockquote><div>I've read the link you gave and it seems to me that you are using the same physical interface for SAN and vm network.<br></div><div>In my setup : </div><div> - SAN network is using two disctinct physical interfaces (p2pX) on same subnet (therefore the need for sysctl patch)<br></div><div> - VM network (p1pX) is using 2 other distinct physical interfaces on "trunk" mode<br data-mce-bogus="1"></div><div> - Administration network (emX) is using 2 distinct bonded network interfaces <br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>Our setup, using a total of 6 network interfaces is "simpler" than yours since each interface has a "disctinct" role.<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>To make multipath work with oVirt on this setup, we had to configure SAN network on system before installing oVirt (using sysctl.conf above), </div><div> - next declare 2 SAN networks on oVirt that are "dragued" to each SAN interface in host network configuration, </div><div> - and finally, in datacenter listing, chosing a datacenter and selecting "iSCSI Multipathing" tab below I can add a multipath configuration concerning both SAN oVirt networks and relevant iqn storage targets.<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>Hope it's clear, if not, please ask me.<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>It's very strange to me also since this configuration did work great on our labs. But half of production sites have the problem decribed above : second SAN interface stops working uppon host activation because vdsm forces sysctl net.ipv4.conf.p2p2.rp_filter to 1 (strict mode).<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>Still searching for clues as why this behaviour...<br data-mce-bogus="1"></div></div></div></div><br></div></div></body></html>