Interesting.
I have a lot of stability issues with IPoIB in EL7. In Windows it’s works flawlessly, but
on Linux is just unstable. So I never had the courage to put an hypervisor Storage network
over IPoIB.
Are you using Mellanox OFED, or the upstream one?
The iSCSI is only through software? No RDMA involved? Aren’t the CPU being wasted in this
scenario since Connect-X3 does not offload TCP/IP connections?
Finally the IPoIB mode is connected or dataram? Have you tunned the MTU settings?
Thanks,
Sent from my iPhone
On 1 Mar 2019, at 21:36, Mike Blanchard <darkytoo(a)gmail.com>
wrote:
Hi Vinicius, i'm sorry, I thought I had replied already.
I'm mostly using IPoIB, but gluster is using some over RDMA. I'm using it a few
different ways,
5 IB nodes, all have dual port ConnectX-3 cards @ 40GB
2 nodes are running windows, one port on each is used for a sync channel for starwind
Vsan to mirror ~20tb of date. The second ports are using for iSCS sharing SMB fiile
shares for other IB nodes.
3 nides are running CentOS 7 and ovirt. One of the ports on each are used for gluster
synchronzaton, gluster is set to use both IPoIB & RDMA. Then the other ports are
using for migratons, iSCSI sharing, and a backup admin channel.
> On Fri, Mar 1, 2019 at 1:36 PM Vinícius Ferrão <ferrao(a)versatushpc.com.br>
wrote:
> Darkytoo have you received my message.
>
> I’m really curious about your setup using Infiniband.
>
> Thanks,
>
> > On 14 Feb 2019, at 07:42, Vinícius Ferrão <ferrao(a)versatushpc.com.br>
wrote:
> >
> > Are you guys running IPoIB for Infiniband Storage? Is it true IB mode? Or
Ethernet over IB?
> >
> > If using NFS on top of it, RDMA is working
> > with NFSoRDMA?
> >
> > I’m really curious about your setup.
> >
> > Thanks,
> >
> > Sent from my iPhone
> >
> >> On 10 Feb 2019, at 23:55, darkytoo(a)gmail.com wrote:
> >>
> >> You were just like me, I also thought it would pick up settings from
cockpit, but once I realized you had to configure the network again when adding to the
host, everything sort of fell into line. I would make sure and monitor the networks so
you can see the traffic going through the infiniband and not the ethernet.
> >> _______________________________________________
> >> Users mailing list -- users(a)ovirt.org
> >> To unsubscribe send an email to users-leave(a)ovirt.org
> >> Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
> >> oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
> >> List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/WC6W7KRXDEN...