[ovirt-users] VM support for Infiniband

[Kevin Hung]

On 9/30/2016 10:10 AM, Kevin Hung wrote:
> oVirt sees the HCA as SR-IOV capable and it sees the VFs, so I believe 
> I have set up SR-IOV correctly. I was able to do a generic passthrough 
> of the HCA, but I was not able to pass through the VFs to more than 
> one VM probably because they are all in the same IOMMU group. I first 
> tried following the oVirt 3.6 Deep Dive video on SR-IOV [1] but was 
> stuck because of the bridging problem.
> [1] https://www.youtube.com/watch?v=A-MROZ8D06Y

I was able to pass through the VFs after fixing the IOMMU issue. I had 
to read up on IOMMU groups and discovered that the PCIe root ports on 
Xeon E3 processors do not support ACS [1], which is needed for the VFs 
to be in different IOMMU groups. I was able to plug the HCA into a 
different PCIe slot which was connected to the PCH instead, allowing 
VFIO to see the VFs in different IOMMU groups.

The way SR-IOV works for Ethernet network adapters in oVirt seems to be 
really nice as the scheduler takes care of the pass-through. Is this 
possible for Infiniband HCAs or is that a technical/architectural 
limitation?

[1] http://vfio.blogspot.com.ar/2014/08/iommu-groups-inside-and-out.html

Thanks,
Kevin Hung