Re: [ovirt-users] VM support for Infiniband
On 9/30/2016 10:10 AM, Kevin Hung wrote:
oVirt sees the HCA as SR-IOV capable and it sees the VFs, so I
believe
I have set up SR-IOV correctly. I was able to do a generic passthrough
of the HCA, but I was not able to pass through the VFs to more than
one VM probably because they are all in the same IOMMU group. I first
tried following the oVirt 3.6 Deep Dive video on SR-IOV [1] but was
stuck because of the bridging problem.
[1] https://www.youtube.com/watch?v=A-MROZ8D06Y
I was able to pass through the VFs after fixing the IOMMU issue. I had
to read up on IOMMU groups and discovered that the PCIe root ports on
Xeon E3 processors do not support ACS [1], which is needed for the VFs
to be in different IOMMU groups. I was able to plug the HCA into a
different PCIe slot which was connected to the PCH instead, allowing
VFIO to see the VFs in different IOMMU groups.
The way SR-IOV works for Ethernet network adapters in oVirt seems to be
really nice as the scheduler takes care of the pass-through. Is this
possible for Infiniband HCAs or is that a technical/architectural
limitation?
[1] http://vfio.blogspot.com.ar/2014/08/iommu-groups-inside-and-out.html
Thanks,
Kevin Hung