This is a multi-part message in MIME format. ------=_NextPartTM-000-55ab7768-e2e6-4bb1-a471-e6535eb723ff Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable on "Ren=E9 Koch [rkoch(a)linuxland.at]=0A= band=0A= =0A= =0A= =0A= =0A= =0A= =0A= =0A= Welcome to Ovirt.=0A= =0A= We are on infiniband IPoIB for about 3 years now (started with VMWare). So = we=0A= jumped on the 10GBit path long before 10GB ethernet had reasonable prices. = =0A= We are fine with it but you should consider the following aspects.=0A= =0A= - Go for ConnectX-2 cards or higher. They allow for IPoIB TCP offloading. D= o not=0A= believe the other information you will find on the internet. A helful statm= ent here:=0A= http://permalink.gmane.org/gmane.linux.drivers.rdma/18032=0A= =0A= - With IPoIB infiniband you should always be happy to achieve more than 1GB= it =0A= throughput. A hear a lot of people complaining that they do not get wire sp= eed.=0A= The stack is quite complex and from our experience we can reach 5-7GBit spe= eds =0A= easily on a DDR backbone (16 Gbps payload). Better CPUs will increase the = =0A= throughput. A jump from Xeon 5400 to 5600 gave a nice boost.=0A= =0A= - Everything around NFS over RDMA is currently a mess. =0A= =0A= - Cable prices are a real pain (at least in Europe). =0A= =0A= - Older CX4 Switches are very cheap and rock solid. We have a CISCO 7000D.= =0A= Though limited to 2044MTU they are fine.=0A= =0A= - Infiniband requires a subnet manager on the network. Cards will only get = a=0A= link if that process is available. Either you need a switch with an builtin= SM or=0A= in case of direct connection it must run on one of the hosts.=0A= =0A= - always remember that GBit bonding will not improve the throughput of one= =0A= connection. So you can easily migrate a single VM or disks over infiniband = with =0A= much more than 1GBit while an ethernet bond will be limited to 1Gbit.=0A= =0A= Maybe you should have a look at the 10GBit mellanox ethernet adapters. They= are =0A= floating around for cheap prices ~$60 have a SFP+ port and are build from t= he=0A= same silicone as the infiniband cards. For direct connections they should b= e fine=0A= but switching hardware is expensive.=0A= =0A= Markus=0A= ------=_NextPartTM-000-55ab7768-e2e6-4bb1-a471-e6535eb723ff Content-Type: text/plain; name="InterScan_Disclaimer.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="InterScan_Disclaimer.txt" **************************************************************************** Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet. Über das Internet versandte E-Mails können unter fremden Namen erstellt oder manipuliert werden. Deshalb ist diese als E-Mail verschickte Nachricht keine rechtsverbindliche Willenserklärung. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln Vorstand: Kadir Akin Dr. Michael Höhnerbach Vorsitzender des Aufsichtsrates: Hans Kristian Langva Registergericht: Amtsgericht Köln Registernummer: HRB 52 497 This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. e-mails sent over the internet may have been written under a wrong name or been manipulated. That is why this message sent as an e-mail is not a legally binding declaration of intention. Collogia Unternehmensberatung AG Ubierring 11 D-50678 Köln executive board: Kadir Akin Dr. Michael Höhnerbach President of the supervisory board: Hans Kristian Langva Registry office: district court Cologne Register number: HRB 52 497 **************************************************************************** ------=_NextPartTM-000-55ab7768-e2e6-4bb1-a471-e6535eb723ff--