My gluster volume is mounted with *.rdma. I cannot make changes to the switch as it doesn’t have any management to it as it is unmanaged. I can transfer files and run VMs from the gluster volume. I just receive these messages. I am using IBoIP. I have no problems doing a NFS over RDMA share from a second server that is also hooked up to the Infiniband switch.

From: Arman Khalatyan <arm2arm@gmail.com>
Date: Monday, August 13, 2018 at 2:21 PM
To: Matthew Hoberg <housemouse@squeakz.net>
Cc: users <users@ovirt.org>
Subject: Re: [ovirt-users] Connection issues when using gluster + infiniband + RDMA

try to leave datagramm mode, do not change mtu. it looks like gluster is connected over the tcp/ipoib. you might get packets drops with mtu> 2k. as i remember you should tune your ib switch for the mtu size,at least on the mellanox managed switch.

if the gluster connected over rdma it should not use any of tcp settings.

Is your mount -l shows that the gluster mounted with .rdma?

btw opensm should run only once, the others are going to stay in the  standby mode.

<housemouse@squeakz.net> schrieb am Mo., 13. Aug. 2018, 20:13:

I am trying to setup gluster using Infiniband and mounting it in RDMA mode. I have created a network hook so that it configures the interfaces as MTU=65520 and CONNECTED_MODE=yes. I have a second server doing NFS with RDMA over Infiniband with some VMs on it. When I try and transfer files to the gluster storage it is taking a while and I am seeing the message “VDSM {hostname} command GetGlusterVolumeHealInfoVDS failed: Message timeout which can be caused by communication issues” This is usually followed by “Host {hostname} is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued.” I just installed the Infiniband hardware. The switch is a Qlogic 12200-18 with QLE7340 single port Infiniband cards in each of the 3 servers. The error message varies on which of the 3 nodes it comes from. Each of the 3 servers is running the opensm service.
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KGR2CL7M34IHL6XJPHUYUSNJCKF63UQS/