8:56 p.m.
Hello,
I have been trying (unsuccessfully) to add my Infiniband HCAs on my VM
hosts to oVirt 4.0.4. I am not using IBoIP, just plain Infiniband. I
would like to be able to pass through the card (or better yet the VF
from SR-IOV) to the guest VM.
From what I can figure out, oVirt is trying to bridge the interface but
I don't think that will work as shown here [1]. Does oVirt currently
have support for passing through Infiniband to VMs? If not, are there
any plans to support it?
[1] http://lists.ovirt.org/pipermail/users/2016-July/040898.html
Thanks,
Kevin Hung
8:28 a.m.
On 30 Sep 2016, at 03:56, Kevin Hung <khung(a)nullaxiom.com>
wrote:
Hello,
I have been trying (unsuccessfully) to add my Infiniband HCAs on my VM hosts to oVirt
4.0.4. I am not using IBoIP, just plain Infiniband. I would like to be able to pass
through the card (or better yet the VF from SR-IOV) to the guest VM.
SR-IOV should work (or general PCI passthrough), it’s then transparent to oVirt. Did you
not succeed with setting up SR-IOV or generic passthrough of the HCA?
From what I can figure out, oVirt is trying to bridge the interface but I don't think
that will work as shown here [1]. Does oVirt currently have support for passing through
Infiniband to VMs? If not, are there any plans to support it?
Well, at least for generic passthrough the PCI device would be detached from the host so
should not be recognized/touched by oVirt networking code at all.
Thanks,
michal
[1] http://lists.ovirt.org/pipermail/users/2016-July/040898.html
Thanks,
Kevin Hung
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
9:10 a.m.
On 9/30/2016 9:28 AM, Michal Skrivanek wrote:
> On 30 Sep 2016, at 03:56, Kevin Hung <khung(a)nullaxiom.com>
wrote:
>
> Hello,
>
> I have been trying (unsuccessfully) to add my Infiniband HCAs on my VM hosts to oVirt
4.0.4. I am not using IBoIP, just plain Infiniband. I would like to be able to pass
through the card (or better yet the VF from SR-IOV) to the guest VM.
SR-IOV should work (or general PCI passthrough), it’s then transparent to oVirt. Did you
not succeed with setting up SR-IOV or generic passthrough of the HCA?
oVirt sees the
HCA as SR-IOV capable and it sees the VFs, so I believe I
have set up SR-IOV correctly. I was able to do a generic passthrough of
the HCA, but I was not able to pass through the VFs to more than one VM
probably because they are all in the same IOMMU group. I first tried
following the oVirt 3.6 Deep Dive video on SR-IOV [1] but was stuck
because of the bridging problem.
> From what I can figure out, oVirt is trying to bridge the interface but I don't
think that will work as shown here [1]. Does oVirt currently have support for passing
through Infiniband to VMs? If not, are there any plans to support it?
Well, at least for generic passthrough the PCI device would be detached from the host so
should not be recognized/touched by oVirt networking code at all.
Thanks,
michal
> [1] http://lists.ovirt.org/pipermail/users/2016-July/040898.html
>
> Thanks,
> Kevin Hung
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
[1] https://www.youtube.com/watch?v=A-MROZ8D06Y
1:25 p.m.
On 9/30/2016 10:10 AM, Kevin Hung wrote:
oVirt sees the HCA as SR-IOV capable and it sees the VFs, so I
believe
I have set up SR-IOV correctly. I was able to do a generic passthrough
of the HCA, but I was not able to pass through the VFs to more than
one VM probably because they are all in the same IOMMU group. I first
tried following the oVirt 3.6 Deep Dive video on SR-IOV [1] but was
stuck because of the bridging problem.
[1] https://www.youtube.com/watch?v=A-MROZ8D06Y
I was able to pass through the VFs after fixing the IOMMU issue. I had
to read up on IOMMU groups and discovered that the PCIe root ports on
Xeon E3 processors do not support ACS [1], which is needed for the VFs
to be in different IOMMU groups. I was able to plug the HCA into a
different PCIe slot which was connected to the PCH instead, allowing
VFIO to see the VFs in different IOMMU groups.
The way SR-IOV works for Ethernet network adapters in oVirt seems to be
really nice as the scheduler takes care of the pass-through. Is this
possible for Infiniband HCAs or is that a technical/architectural
limitation?
[1] http://vfio.blogspot.com.ar/2014/08/iommu-groups-inside-and-out.html
Thanks,
Kevin Hung
2976
days inactive
2977
days old
3 comments
2 participants
participants (2)
-
Kevin Hung -
Michal Skrivanek