As a follow-up to this, I made some headway in sorting out the source of the issue, but hoping someone can give me a pointer to where this is happening in the code, as well as some understanding for why.

In oVirt, when I allocate a virtual function to a guest, a new MAC address is generated for the VF (as it should be) from the MAC address pool in oVirt, and then that MAC address is written to the VF on the hypervisor.  Thus I end up with something like:

: ens11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master i40e state UP mode DEFAULT qlen 1000
    link/ether 3c:fd:fe:9d:a1:38 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:1a:4a:16:01:52, spoof checking on, link-state auto

This *is not* how it happens under libvirt/virt-manager, however.  When allocating a VF to a guest under libvirt, a random MAC address is generated and associated with the VF under the guest, but it is not written back to the hypervisor, and is instead left as 00:00:00:00:00:00.

I am pretty sure this writing of the MAC address at the hypervisor is causing at least some of the issues I'm seeing, as with the Intel cards, that prevents the guest from changing/adding a new MAC address, which is what happens with the instantiation of a macvlan interface.

So can anyone point me to where in the oVirt code this MAC address assignment is occurring?  Also curious why oVirt does this assignment, but libvirt does not.

Thanks!

- jkt

On Mon, Feb 22, 2016 at 2:51 PM Jay Turner <jkt@iix.net> wrote:
Hoping someone can help with a problem my team is seeing under oVirt.

We are making heavy use of macvlan interfaces (in VEPA mode) on-top of virtual functions, under VMs being managed by oVirt.  In this scenario IPv6 is not playing nicely, with no traffic going through, and messages about neighbor solicitation.  There are some pointers out there indicating the issue stems from the fact IPv6 utilizing multicast for neighbor solicitation, but nothing we have tried seems to work around this issue.

The problem is made all the most bizarre by the fact that on the same hardware libvirt + virt-manager works perfectly fine.  I have looked at the corresponding xml for the guests, and nothing seems to point to the underlying cause for oVirt to fail, but libvirt to succeed.

* Intel XL710 40G NICs (i40e/i40evf drivers)
* CentOS 7.1 (fully up-to-date)
* oVirt 3.6
* libvirt-1.2.17
* virt-manager-1.2.1-8

Thanks for any pointers folks can provide.

- jkt