On Tue, 2013-08-20 at 00:24 +0200, René Koch wrote:
-----Original message-----
> From:Dan Kenigsberg <danken(a)redhat.com>
> Sent: Monday 19th August 2013 23:48
> To: René Koch <r.koch(a)ovido.at>; Itzik Brown <ItzikB(a)mellanox.com>
> Cc: ovirt-users <users(a)ovirt.org>
> Subject: Re: [Users] oVirt and Infiniband
>
> On Tue, Aug 13, 2013 at 03:48:14PM +0200, René Koch (ovido) wrote:
> > Hi,
> >
> > Does anyone have oVirt (or a plain KVM hypervisor) running on Infiniband
> > where the virtual machines are bridged to an Infiniband interface (with
> > EoIB kernel module)?
>
> As far as I recall the EoIB module is not yet in upstream kernel. Could
> you give more details on your setup (distro, kernel, module)?
> Do you get connectivity between the hosts? Maybe Itzik and his
> colleagues could help us here.
Thanks for your answer.
That's right - EoIB isn't in the kernel.
I installed module from Mellanox OFED package and even there it's marked as
unstable...
OS is CentOS 6.4.
At the moment I've also an open support case with Mellanox trying to solve the
connectivity issue.
So I have support from official site, but was thinking maybe someone has already
experience with such a setup and also had some issues/tipps for me...
My problem is some sort of a bridge or maybe MAC translation issue (unsure cause of a lot
of unanswered ARP requests in tcpdump and bridge is working on Ethernet interfaces).
I have bridge ovirtmgmt on eth2 interface (eth2 is a (virtual) ethernet interface bound
to the Infiniband-interface) with an IP address configured on it.
Hosts can reach each other, so ethernet to infiniband translation seems to work.
But when I create a vm in oVirt this vm can only communicate with IPs/vms on the bridge.
This mean I can ping the IP of the host and other vms on this bridge, but no host behind
the bridge (like e.g. other hosts or vms on other hosts)...
Mellanox support team found the issue:
Daemon openibd requires write access to libvirt which is restricted by
oVirt per default.
When changing auth_unix_rw to "none" networking of the vms is working fine
over the inifinband network.
So my question is now:
I think there's a good reason why write access to libvirt is restricted.
In my particular setup no one will do a virsh start/stop/whatever so
from a user point I can live with an open libvirt.
But are there any troubles I can run into from oVirt side with
auth_unix_rw="none" beside users doing evil virsh stuff?
Today I tested it and ran into a first issue which I didn't investigate
so far (will do this week):
- Started vm from oVirt on node
- Changed auth_unix_rw to "none"
- Restarted libvirt on node
- VM was running according to "virsh list"
- VM was stopped in oVirt
- Started vm in oVirt on another node
- vm was running twice
Will have a look at the logs what append exactly this vm...
Regards,
René
>
> >
> > I'm having issues in such a setup where vms can't communicate over
this
> > bridge...
> >
> >
> > Regards,
> > René
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> >
http://lists.ovirt.org/mailman/listinfo/users
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users