I don't create the VMs directly in the UI but use the API with Foreman. If I change the MACs on the NICs I have to go back and update Foreman as well. My workaround today is unplug the NIC I want to be second, briefly power-up the VM so the "first" NIC gets assigned a PCI address, shutdown and re-plug the "second" NIC. On power-up the "second" NIC will be assigned a later PCI address and so achieve the ordering I want. It's just fiddly and can only be done on first boot. I was hoping for a more elegant solution that works when you have large numbers of VMs to provision. ---- On Wed, 24 May 2023 08:09:21 +0100 Guillaume Pavese <guillaume.pavese(a)interactiv-group.com&gt; wrote --- I think it depends on the mac addresses. If you see that the MAC addresses are not sequential, delete the NICs and recreate them in the order you want them to be it world for us Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group On Wed, May 24, 2023 at 1:46 AM Alan G <mailto:alan%2Bovirt@griff.me.uk> wrote: Ce message et toutes les pièces jointes (ci-après le “message”) sont établis à l’intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d’en avertir immédiatement l’expéditeur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L’internet ne permettant pas d’assurer l’intégrité de ce message . Interactiv-group (et ses filiales) décline(nt) toute responsabilité au titre de ce message, dans l’hypothèse ou il aurait été modifié. https://interactiv-group.com/disclaimer.html Is there any way to enforce NIC ordering so the vNICs match the ordering in the Engine UI? I found this but not clear if it was ever implemented? https://www.ovirt.org/develop/release-management/features/network/predict... _______________________________________________ Users mailing list -- mailto:users@ovirt.org To unsubscribe send an email to mailto:users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/I3CSHZWQBSI...

Perhaps it's more of a Foreman/API limitation. In Foreman I provision two interfaces, that in the Foreman UI at least, have an ordering. Sometimes when they are provisioned in oVirt the "second" interface has a lower MAC than the "first" interface and so the ordering in Foreman is different to the ordering in oVIrt. This breaks a bunch of stuff that relies on the networks being on specific interface (eth0, eth1). ---- On Wed, 24 May 2023 14:09:33 +0100 Volenbovskyi, Konstantin <Konstantin.Volenbovskyi(a)haufe.com&gt; wrote --- Hi, Do I understand it correctly that:                 -your OS uses PCI addresses to order NICSs?                 -PCI addresses of NICs are in fact non-consistent between VMs? So you boot VM1 (with NICs vNIC1 and vNIC2) and VM2 (vNIC3 and vNIC4) with same order in eg. API (->GUI) and you sometimes see different PCI addresses (vNIC2 has higher PCI address than vNIC1; however vNIC3 has higher PCI address than vNIC4)? I am surprised about that…   BR, Konstantin   Von: Alan G <mailto:alan+ovirt@griff.me.uk> Datum: Mittwoch, 24. Mai 2023 um 10:50 An: Guillaume Pavese <mailto:guillaume.pavese@interactiv-group.com> Cc: users <mailto:users@ovirt.org> Betreff: [ovirt-users] Re: NIC Ordering   I don't create the VMs directly in the UI but use the API with Foreman. If I change the MACs on the NICs I have to go back and update Foreman as well.   My workaround today is unplug the NIC I want to be second, briefly power-up the VM so the "first" NIC gets assigned a PCI address, shutdown and re-plug the "second" NIC. On power-up the "second" NIC will be assigned a later PCI address and so achieve the ordering I want.   It's just fiddly and can only be done on first boot. I was hoping for a more elegant solution that works when you have large numbers of VMs to provision.       ---- On Wed, 24 May 2023 08:09:21 +0100 Guillaume Pavese <mailto:guillaume.pavese@interactiv-group.com> wrote ---   I think it depends on the mac addresses. If you see that the MAC addresses are not sequential, delete the NICs and recreate them in the order you want them to be it world for us     Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group     On Wed, May 24, 2023 at 1:46 AM Alan G <mailto:alan%2Bovirt@griff.me.uk> wrote:     Ce message et toutes les pièces jointes (ci-après le “message”) sont établis à l’intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d’en avertir immédiatement l’expéditeur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L’internet ne permettant pas d’assurer l’intégrité de ce message . Interactiv-group (et ses filiales) décline(nt) toute responsabilité au titre de ce message, dans l’hypothèse ou il aurait été modifié. https://interactiv-group.com/disclaimer.html   Is there any way to enforce NIC ordering so the vNICs match the ordering in the Engine UI?   I found this but not clear if it was ever implemented?   https://www.ovirt.org/develop/release-management/features/network/predict...     _______________________________________________ Users mailing list -- mailto:users@ovirt.org To unsubscribe send an email to mailto:users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/I3CSHZWQBSI...  

My guess is that Foreman is deterministic, but the MAC addresses are allocated by oVirt and I don't know what rules are used there. For a long time it was fine and it worked as desired, but since we deleted a few VMs we have fragmentation in the MAC pool and now sometimes the "second" NIC gets allocated a lower MAC than the "first" NIC. I probably need to capture the requests from Foreman to oVirt but not sure how to do that with it being HTTPS. Your idea of using custom MTU size is interesting. thanks. ---- On Thu, 25 May 2023 11:23:31 +0100 Volenbovskyi, Konstantin <Konstantin.Volenbovskyi(a)haufe.com&gt; wrote --- Hi, the situation is significantly different in case you say that Foreman is not deterministic in supplying NICs via API. So all this determinism in ‘first interface gets lowest MAC/lowest PCI’ is not helpful because you essential don’t know which interface becomes the first one. In fact, I am not sure if your OS uses ‘lowest PCI’ or ‘lowest MAC’ for ordering, I think that ‘lowest PCI’ is much more widespread.   A few ideas:                 -I think that slightly more elegant: maybe the idea is to create VM with first NIC, then add second NIC and then start your VM? -I am not sure what is your OS, but udev rules is a powerful mechanism to control NIC assignment. In case you say that behaviour of Foreman can’t be addressed – you might change MTU by 1 byte for certain network (1499) then use this attribute to find that NIC ATTR{mtu} or modify creation of VM to create one vNIC with link state=down and then use ATTR{operstate}   BR, Konstantin   Von: Alan G <mailto:alan+ovirt@griff.me.uk> Datum: Mittwoch, 24. Mai 2023 um 15:46 An: "Volenbovskyi, Konstantin" <mailto:Konstantin.Volenbovskyi@haufe.com> Cc: Guillaume Pavese <mailto:guillaume.pavese@interactiv-group.com>, users <mailto:users@ovirt.org> Betreff: [ovirt-users] Re: NIC Ordering   Perhaps it's more of a Foreman/API limitation.   In Foreman I provision two interfaces, that in the Foreman UI at least, have an ordering. Sometimes when they are provisioned in oVirt the "second" interface has a lower MAC than the "first" interface and so the ordering in Foreman is different to the ordering in oVIrt.   This breaks a bunch of stuff that relies on the networks being on specific interface (eth0, eth1).       ---- On Wed, 24 May 2023 14:09:33 +0100 Volenbovskyi, Konstantin <mailto:Konstantin.Volenbovskyi@haufe.com> wrote ---   Hi, Do I understand it correctly that:                 -your OS uses PCI addresses to order NICSs?                 -PCI addresses of NICs are in fact non-consistent between VMs? So you boot VM1 (with NICs vNIC1 and vNIC2) and VM2 (vNIC3 and vNIC4) with same order in eg. API (->GUI) and you sometimes see different PCI addresses (vNIC2 has higher PCI address than vNIC1; however vNIC3 has higher PCI address than vNIC4)? I am surprised about that…   BR, Konstantin   Von: Alan G <mailto:alan+ovirt@griff.me.uk> Datum: Mittwoch, 24. Mai 2023 um 10:50 An: Guillaume Pavese <mailto:guillaume.pavese@interactiv-group.com> Cc: users <mailto:users@ovirt.org> Betreff: [ovirt-users] Re: NIC Ordering   I don't create the VMs directly in the UI but use the API with Foreman. If I change the MACs on the NICs I have to go back and update Foreman as well.   My workaround today is unplug the NIC I want to be second, briefly power-up the VM so the "first" NIC gets assigned a PCI address, shutdown and re-plug the "second" NIC. On power-up the "second" NIC will be assigned a later PCI address and so achieve the ordering I want.   It's just fiddly and can only be done on first boot. I was hoping for a more elegant solution that works when you have large numbers of VMs to provision.       ---- On Wed, 24 May 2023 08:09:21 +0100 Guillaume Pavese <mailto:guillaume.pavese@interactiv-group.com> wrote ---   I think it depends on the mac addresses. If you see that the MAC addresses are not sequential, delete the NICs and recreate them in the order you want them to be it world for us     Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group     On Wed, May 24, 2023 at 1:46 AM Alan G <mailto:alan%2Bovirt@griff.me.uk> wrote:     Ce message et toutes les pièces jointes (ci-après le “message”) sont établis à l’intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d’en avertir immédiatement l’expéditeur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L’internet ne permettant pas d’assurer l’intégrité de ce message . Interactiv-group (et ses filiales) décline(nt) toute responsabilité au titre de ce message, dans l’hypothèse ou il aurait été modifié. https://interactiv-group.com/disclaimer.html   Is there any way to enforce NIC ordering so the vNICs match the ordering in the Engine UI?   I found this but not clear if it was ever implemented?   https://www.ovirt.org/develop/release-management/features/network/predict...     _______________________________________________ Users mailing list -- mailto:users@ovirt.org To unsubscribe send an email to mailto:users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/I3CSHZWQBSI...