<div dir="ltr">Hi,<div><br></div><div>1) The storage device has 2 controllers. Each controller has 4 nics. All 8 nics are connected to a level 2 switch that has ethernet connections to the hosts. The storage device nics enumerated are 0, 1, 2, 3. Controller 0, nic 0 are the primary target for a group which includes controller 1, nic 0. In /var/lib/iscsi/nodes/{IQN}/{nic}/default, the difference between c0n0 and c1n0 is :</div><div><div># diff 10.251.6.10?,3260,?/default </div><div>3c3</div><div>< node.tpgt = 1</div><div>---</div><div>> node.tpgt = 2</div><div>47c47</div><div>< node.conn[0].address = 10.251.6.101</div><div>---</div><div>> node.conn[0].address = 10.251.6.102</div></div><div><br></div><div>The 4 groups of nics have different subnets. Dell docs recommend this, and the storage devices keeps packets separated by these subnets. For my setup, I have this config:</div><div> controller 0 controller 1 host-1 host-2</div><div>nic0 10.251.6.101 10.251.6.102 10.251.6.135 10.251.6.136</div><div>nic1 10.251.7.101 10.251.7.102 10.251.7.135 10.251.7.136<br></div><div>nic2 10.251.8.101 10.251.8.102 10.251.8.135 10.251.8.136<br></div><div>nic3 10.251.9.101 10.251.9.102 10.251.9.135 10.251.9.136<br></div><div><br></div><div>On each virtualization host, I have 1 nics configured on each of the 4 subnets.</div><div><br></div><div>2) We are using rhev 3.5. I have deployed this as round-robin, not bond. According to Dell support, iscsi works best with round-robin, whereas FC NAS works best with bond. I follow their recommendations. The rdac driver is setup for this, and the prio= below separates the 8 nic paths into 2 groups.</div><div><br></div><div><div><font face="monospace, monospace"># multipath -ll</font></div><div><font face="monospace, monospace">36f01faf000d7ddeb000002085258bce5 dm-1 DELL,MD32xxi</font></div><div><font face="monospace, monospace">size=756G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw</font></div><div><font face="monospace, monospace">|-+- policy='round-robin 0' <b>prio=14</b> status=active</font></div><div><font face="monospace, monospace">| |- 7:0:0:1 sdc 8:32 active ready running</font></div><div><font face="monospace, monospace">| |- 8:0:0:1 sdj 8:144 active ready running</font></div><div><font face="monospace, monospace">| |- 11:0:0:1 sdu 65:64 active ready running</font></div><div><font face="monospace, monospace">| `- 12:0:0:1 sdv 65:80 active ready running</font></div><div><font face="monospace, monospace">`-+- policy='round-robin 0' <b>prio=9</b> status=enabled</font></div><div><font face="monospace, monospace"> |- 6:0:0:1 sdf 8:80 active ready running</font></div><div><font face="monospace, monospace"> |- 10:0:0:1 sdk 8:160 active ready running</font></div><div><font face="monospace, monospace"> |- 5:0:0:1 sdo 8:224 active ready running</font></div><div><font face="monospace, monospace"> `- 9:0:0:1 sdt 65:48 active ready running</font></div></div><div><br></div><div>Hope this helps explain my setup. I am not sure how to file a bug. Is this done on bugzilla or somewhere else?</div><div><br></div><div>Paul</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Mar 15, 2015 at 7:44 AM, Elad Ben Aharon <span dir="ltr"><<a href="mailto:ebenahar@redhat.com" target="_blank">ebenahar@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-family:times new roman,new york,times,serif;font-size:12pt;color:#000000"><div>Hi Paul,<br></div><div><br></div><div>I would like to know the following details:<br></div><div>1) Are the hosts's NICs connected to the storage server located in the same network subnet as the storage server itself?<br></div><div>2) Have you tried to deploy the connection to the storage server using the 'iSCSI multipath' bond that available in RHEV-3.4? <br><br></div><hr><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt"><b>From: </b>"Nir Soffer" <<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>><br><b>To: </b>"Paul Dyer" <<a href="mailto:pmdyermms@gmail.com" target="_blank">pmdyermms@gmail.com</a>><br><b>Cc: </b><a href="mailto:devel@ovirt.org" target="_blank">devel@ovirt.org</a>, "Elad Ben Aharon" <<a href="mailto:ebenahar@redhat.com" target="_blank">ebenahar@redhat.com</a>><br><b>Sent: </b>Sunday, 15 March, 2015 12:54:44 PM<div><div class="h5"><br><b>Subject: </b>Re: [ovirt-devel] adding new paths to iscsi data storage domains<br><div><br></div>Adding Elad who tested this feature lately to add more info.<br><div><br></div>----- Original Message -----<br>> From: "Paul Dyer" <<a href="mailto:pmdyermms@gmail.com" target="_blank">pmdyermms@gmail.com</a>><br>> To: "Nir Soffer" <<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>><br>> Cc: <a href="mailto:devel@ovirt.org" target="_blank">devel@ovirt.org</a><br>> Sent: Friday, March 13, 2015 6:25:05 PM<br>> Subject: Re: [ovirt-devel] adding new paths to iscsi data storage domains<br>> <br>> Nir,<br>> <br>> we have added 2 more nics to each virtualization host. In order to get<br>> this working, I had to add an after_network_setup hook. The shell script<br>> simply does "/sbin/iscsiadm -m node -L all", to get the extra targets<br>> login after reboot.<br>> <br>> I looked in the engine table storage_server_connections and found that only<br>> the iscsi targets selected during the original storage domain create were<br>> present. If ovirt-engine added rows here, then most of the work would<br>> have been done.<br>> <br>> I say mostly, because the Dell MD3200i did not return exactly the correct<br>> portal target values. The device has 2 controllers, with 4 nics each.<br>> Controller 0, ports 0,1,2,3 uses portal target 1. Controller 1, the<br>> ports uses portal target 2. After iscsiadm discovery, the portal targets<br>> for ports 1,2,3 were all 1. Ports 0 had targets 1 and 2, correctly. I<br>> adjusted the values saved on the filesystem, and login/logoff works fine.<br>> <br>> Paul<br>> <br>> <br>> <br>> On Fri, Feb 6, 2015 at 11:57 AM, Paul Dyer <<a href="mailto:pmdyermms@gmail.com" target="_blank">pmdyermms@gmail.com</a>> wrote:<br>> <br>> > First of all, thank you for your time. I must apologize that in this<br>> > install, I am using RHEV 3.4.5. I will try to reproduce this on an ovirt<br>> > install. I just need to create some paths to iscsi targets.<br>> ><br>> > 1. This configuration has 2 physical hosts, Dell PE-R715 servers, with a<br>> > Dell PowerVault MD3200i iSCSI data storage domain. The EM3 nic was the<br>> > original link. EM4 was the new connection to storage.<br>> ><br>> > 2. From the manager interface, I selected Storage tab, then the<br>> > kvm5DataDomain, then edit. From the popup, I added the IP address under<br>> > Discovery Targets, then clicked the Discover button. Then, clicked the<br>> > login arrow on the new targets discovered.<br>> ><br>> > I have attached the engine and vdsm logs. I was working on this at about<br>> > 11:40am Feb 4th.<br>> ><br>> > When setting the host in maintenance mode, then reboot, and Activate, the<br>> > new paths do not get a login.<br>> ><br>> > Thanks,<br>> > Paul<br>> ><br>> ><br>> ><br>> > On Fri, Feb 6, 2015 at 5:38 AM, Nir Soffer <<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>> wrote:<br>> ><br>> >> ----- Original Message -----<br>> >> > From: "Paul Dyer" <<a href="mailto:pmdyermms@gmail.com" target="_blank">pmdyermms@gmail.com</a>><br>> >> > To: <a href="mailto:devel@ovirt.org" target="_blank">devel@ovirt.org</a><br>> >> > Sent: Friday, February 6, 2015 12:20:23 AM<br>> >> > Subject: [ovirt-devel] adding new paths to iscsi data storage domains<br>> >> ><br>> >> > Hi,<br>> >> ><br>> >> > I have been reading the devel list for months now, and would like to<br>> >> ask a<br>> >> > question.<br>> >> ><br>> >> > In version 3.4.5, adding new paths to an existing iSCSI data storage<br>> >> domain<br>> >> > does not work from the manager.<br>> >><br>> >> It works on ovirt 3.5 and master and it should work also in all previous<br>> >> versions.<br>> >><br>> >> Please open a bug for this:<br>> >> 1. Describe the configuration you are modifying<br>> >> 2. Describe the steps you take<br>> >> 3. Include engine log<br>> >> 4. Include vdsm log from the host trying to add new devices<br>> >><br>> >> > I have been able to add the paths with<br>> >> > command line "iscsiadm -m discovery -t st" and "iscsiadm -m node -L<br>> >> all".<br>> >> ><br>> >> > Is there any plan to allow for adding new storage paths after the data<br>> >> domain<br>> >> > has been created?<br>> >> ><br>> >> > Thanks,<br>> >> > Paul<br>> >> ><br>> >> ><br>> >> > --<br>> >> > Paul Dyer,<br>> >> > Mercury Consulting Group, RHCE<br>> >> > <a href="tel:504-302-8750" value="+15043028750" target="_blank">504-302-8750</a><br>> >> ><br>> >> > _______________________________________________<br>> >> > Devel mailing list<br>> >> > <a href="mailto:Devel@ovirt.org" target="_blank">Devel@ovirt.org</a><br>> >> > <a href="http://lists.ovirt.org/mailman/listinfo/devel" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br>> >><br>> ><br>> ><br>> ><br>> > --<br>> > Paul Dyer,<br>> > Mercury Consulting Group, RHCE<br>> > <a href="tel:504-302-8750" value="+15043028750" target="_blank">504-302-8750</a><br>> ><br>> <br>> <br>> <br>> --<br>> Paul Dyer,<br>> Mercury Consulting Group, RHCE<br>> <a href="tel:504-302-8750" value="+15043028750" target="_blank">504-302-8750</a><br>> <br></div></div></div><div><br></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Paul Dyer,<br>Mercury Consulting Group, RHCE<br>504-302-8750</div>
</div>