Guest PXE boot with 2 NICs : MAC order
Hello, I can't remember if I already asked that here, but my researches returned nothing obvious. It is frequent in our setup to boot a VM setup with two NICS : - one in a network with a DHCP server, leading to a PXE/tftp/kickstart setup - a second one with a network with no DHCP, nor routed. We use this network in some cases to access iSCSI data. When PXE booting such a VM, I realized that in gPXE, the NICs are order according to their MAC address (net0, net1...) I found no mean to explicitly tell gPXE to use a specific interface, and it is a pain, because my setup workflow implies I could choose the correct interface. In the oVirt web GUI, I found nothing about that. In the ovirt-shell, I had no better luck. In the gPXE wiki, I found nothing more that what's above. This is not a big issue, but was worth asking. -- Nicolas Ecarnot
You should file an RFE on bugzilla for this kind of stuff and maybe raise awareness during the weekly meeting on irc. this increases the chance of an implementation taking place dramatically ;) HTH -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
Le 31/03/2015 09:21, Sven Kieske a écrit :
You should file an RFE on bugzilla for this kind of stuff and maybe raise awareness during the weekly meeting on irc.
this increases the chance of an implementation taking place dramatically ;)
HTH
Hi Sven, I found and append this : https://bugzilla.redhat.com/show_bug.cgi?id=1045022 -- Nicolas Ecarnot
Indeed, if you're using oVirt 3.4 and up, and you supply all the NICs whenever you create the VM (not afterwards - as part of the new VM dialog), the NICs should receive MAC addresses according to their ordering by names, e.g. nic1 will always get a lower MAC address than nic2. For a newly-created VM this should guarantee that nic1 will indeed end up the first NIC inside the guest OS and be used by PXE. If this is something you do a lot, then I'd suggest creating a new template as such and create VMs from that. On 31/03/15 11:23, Nicolas Ecarnot wrote:
Le 31/03/2015 09:21, Sven Kieske a écrit :
You should file an RFE on bugzilla for this kind of stuff and maybe raise awareness during the weekly meeting on irc.
this increases the chance of an implementation taking place dramatically ;)
HTH
Hi Sven,
I found and append this : https://bugzilla.redhat.com/show_bug.cgi?id=1045022
Hi Lior, Le 31/03/2015 12:55, Lior Vernia a écrit :
Indeed, if you're using oVirt 3.4 and up, and you supply all the NICs whenever you create the VM (not afterwards - as part of the new VM dialog), the NICs should receive MAC addresses according to their ordering by names, e.g. nic1 will always get a lower MAC address than nic2. For a newly-created VM this should guarantee that nic1 will indeed end up the first NIC inside the guest OS and be used by PXE.
In the BZ I provided above, I explain that my tests are showing that this is not true : the nics are not yet sorted according to their MAC nor their names. I'm using 3.5.1
If this is something you do a lot, then I'd suggest creating a new template as such and create VMs from that.
I know that could be a way, but this is not our strategy, as we prefer using kickstart and we have so many different setups that factorisation is not possible.
On 31/03/15 11:23, Nicolas Ecarnot wrote:
Le 31/03/2015 09:21, Sven Kieske a écrit :
You should file an RFE on bugzilla for this kind of stuff and maybe raise awareness during the weekly meeting on irc.
this increases the chance of an implementation taking place dramatically ;)
HTH
Hi Sven,
I found and append this : https://bugzilla.redhat.com/show_bug.cgi?id=1045022
-- Nicolas Ecarnot
On 31/03/15 14:21, Nicolas Ecarnot wrote:
Hi Lior,
Le 31/03/2015 12:55, Lior Vernia a écrit :
Indeed, if you're using oVirt 3.4 and up, and you supply all the NICs whenever you create the VM (not afterwards - as part of the new VM dialog), the NICs should receive MAC addresses according to their ordering by names, e.g. nic1 will always get a lower MAC address than nic2. For a newly-created VM this should guarantee that nic1 will indeed end up the first NIC inside the guest OS and be used by PXE.
In the BZ I provided above, I explain that my tests are showing that this is not true : the nics are not yet sorted according to their MAC nor their names.
I'm using 3.5.1
I see. It seems MAC addresses are allocated according to the NIC name order, and that they are also named in the corresponding order within the guest OS. So the only problem here is how gPXE chooses a NIC to boot from... I'm not familiar with the behavior of gPXE, but oVirt seems to behave alright.
If this is something you do a lot, then I'd suggest creating a new template as such and create VMs from that.
I know that could be a way, but this is not our strategy, as we prefer using kickstart and we have so many different setups that factorisation is not possible.
On 31/03/15 11:23, Nicolas Ecarnot wrote:
Le 31/03/2015 09:21, Sven Kieske a écrit :
You should file an RFE on bugzilla for this kind of stuff and maybe raise awareness during the weekly meeting on irc.
this increases the chance of an implementation taking place dramatically ;)
HTH
Hi Sven,
I found and append this : https://bugzilla.redhat.com/show_bug.cgi?id=1045022
Le 31/03/2015 13:57, Lior Vernia a écrit :
On 31/03/15 14:21, Nicolas Ecarnot wrote:
Hi Lior,
Le 31/03/2015 12:55, Lior Vernia a écrit :
Indeed, if you're using oVirt 3.4 and up, and you supply all the NICs whenever you create the VM (not afterwards - as part of the new VM dialog), the NICs should receive MAC addresses according to their ordering by names, e.g. nic1 will always get a lower MAC address than nic2. For a newly-created VM this should guarantee that nic1 will indeed end up the first NIC inside the guest OS and be used by PXE.
In the BZ I provided above, I explain that my tests are showing that this is not true : the nics are not yet sorted according to their MAC nor their names.
I'm using 3.5.1
I see. It seems MAC addresses are allocated according to the NIC name order,
I'm not sure this is exact. According to what I'm witnessing, MAC addresses are allocated in incremental order, from the first free MAC address in the MAC pool range.
and that they are also named in the corresponding order within the guest OS.
Er.. what gives when I'm naming my interfaces ethX, and amongst that, oVirt comes trying to add some nicX???
So the only problem here is how gPXE chooses a NIC to boot from... I'm not familiar with the behavior of gPXE, but oVirt seems to behave alright.
I mostly agree with that, in the sense that one has to dig in which way gPXE is sorting the NICs, ie mapping the MACs to its "net0", "net1", and so on. And then, decide whether oVirt could have a control upon this sorting, and then issue a RFE to master it from the web GUI. -- Nicolas ECARNOT
If this is something you do a lot, then I'd suggest creating a new template as such and create VMs from that.
I know that could be a way, but this is not our strategy, as we prefer using kickstart and we have so many different setups that factorisation is not possible.
On 31/03/15 11:23, Nicolas Ecarnot wrote:
Le 31/03/2015 09:21, Sven Kieske a écrit :
You should file an RFE on bugzilla for this kind of stuff and maybe raise awareness during the weekly meeting on irc.
this increases the chance of an implementation taking place dramatically ;)
HTH
Hi Sven,
I found and append this : https://bugzilla.redhat.com/show_bug.cgi?id=1045022
-- Nicolas Ecarnot
On 31/03/15 15:34, Nicolas Ecarnot wrote:
Le 31/03/2015 13:57, Lior Vernia a écrit :
On 31/03/15 14:21, Nicolas Ecarnot wrote:
Hi Lior,
Le 31/03/2015 12:55, Lior Vernia a écrit :
Indeed, if you're using oVirt 3.4 and up, and you supply all the NICs whenever you create the VM (not afterwards - as part of the new VM dialog), the NICs should receive MAC addresses according to their ordering by names, e.g. nic1 will always get a lower MAC address than nic2. For a newly-created VM this should guarantee that nic1 will indeed end up the first NIC inside the guest OS and be used by PXE.
In the BZ I provided above, I explain that my tests are showing that this is not true : the nics are not yet sorted according to their MAC nor their names.
I'm using 3.5.1
I see. It seems MAC addresses are allocated according to the NIC name order,
I'm not sure this is exact. According to what I'm witnessing, MAC addresses are allocated in incremental order, from the first free MAC address in the MAC pool range.
My question is as follows: when you create a VM, and add NICs as part of the new VM dialog (using the widget at the bottom), have you ever encountered a situation where a "higher-named" NIC got a lower MAC address?
and that they are also named in the corresponding order within the guest OS.
Er.. what gives when I'm naming my interfaces ethX, and amongst that, oVirt comes trying to add some nicX???
When NICs are added *after* the VM had already been created, it really depends whether the VM is currently running and they're being hotplugged or whether it's not running, and also on the guest OS (this last bit is always true). If the VM is running and NICs are being hotplugged as soon as they're created, I'd expect them to receive "increasing" names within the guest OS according to the order of creation, regardless of MAC address (at least if we're talking about newer RHEL and Fedora versions).
So the only problem here is how gPXE chooses a NIC to boot from... I'm not familiar with the behavior of gPXE, but oVirt seems to behave alright.
I mostly agree with that, in the sense that one has to dig in which way gPXE is sorting the NICs, ie mapping the MACs to its "net0", "net1", and so on.
That's the key point here for me, as it's difficult to predict the behavior of n'importe quel piece of software... :)
And then, decide whether oVirt could have a control upon this sorting, and then issue a RFE to master it from the web GUI.
Le 31/03/2015 14:48, Lior Vernia a écrit :
I see. It seems MAC addresses are allocated according to the NIC name order,
I'm not sure this is exact. According to what I'm witnessing, MAC addresses are allocated in incremental order, from the first free MAC address in the MAC pool range.
My question is as follows: when you create a VM, and add NICs as part of the new VM dialog (using the widget at the bottom), have you ever encountered a situation where a "higher-named" NIC got a lower MAC address?
Absolutely YES. I just repeated this test, and it is easy to reproduce : in a MAC pool, manage to free some lower MAC addresses, and the will get allocated from the first free in the pool. And I found this behaviour SOUND. Don't change it.
So the only problem here is how gPXE chooses a NIC to boot from... I'm not familiar with the behavior of gPXE, but oVirt seems to behave alright.
I mostly agree with that, in the sense that one has to dig in which way gPXE is sorting the NICs, ie mapping the MACs to its "net0", "net1", and so on.
That's the key point here for me, as it's difficult to predict the behavior of n'importe quel piece of software... :)
Thanks for your explanation, though I think the present way oVirt is assigning the MACs and naming the interfaces are not bad and should not change. The key point is how to make oVirt force gPXE to use the interface we want, whatever its name or its MAC. If possible? It seems I'm not the first one to deal with this issue : http://etherboot.org/pipermail/gpxe/2011-September/001742.html -- Nicolas Ecarnot
On 31/03/15 16:02, Nicolas Ecarnot wrote:
Le 31/03/2015 14:48, Lior Vernia a écrit :
I see. It seems MAC addresses are allocated according to the NIC name order,
I'm not sure this is exact. According to what I'm witnessing, MAC addresses are allocated in incremental order, from the first free MAC address in the MAC pool range.
My question is as follows: when you create a VM, and add NICs as part of the new VM dialog (using the widget at the bottom), have you ever encountered a situation where a "higher-named" NIC got a lower MAC address?
Absolutely YES. I just repeated this test, and it is easy to reproduce : in a MAC pool, manage to free some lower MAC addresses, and the will get allocated from the first free in the pool.
So the lower ones are allocated. But I would expect nic0 to receive a lower MAC address than nic1 on the same VM. Have you encountered a situation where that is not the case?
And I found this behaviour SOUND. Don't change it.
So the only problem here is how gPXE chooses a NIC to boot from... I'm not familiar with the behavior of gPXE, but oVirt seems to behave alright.
I mostly agree with that, in the sense that one has to dig in which way gPXE is sorting the NICs, ie mapping the MACs to its "net0", "net1", and so on.
That's the key point here for me, as it's difficult to predict the behavior of n'importe quel piece of software... :)
Thanks for your explanation, though I think the present way oVirt is assigning the MACs and naming the interfaces are not bad and should not change. The key point is how to make oVirt force gPXE to use the interface we want, whatever its name or its MAC. If possible?
It seems I'm not the first one to deal with this issue : http://etherboot.org/pipermail/gpxe/2011-September/001742.html
Le 31/03/2015 15:55, Lior Vernia a écrit :
So the lower ones are allocated. But I would expect nic0 to receive a lower MAC address than nic1 on the same VM. Have you encountered a situation where that is not the case?
Once again : YES. What I'm witnessing is that oVirt only uses the first free MAC, and does not mind the name. -- Nicolas Ecarnot
On 31/03/15 17:59, Nicolas Ecarnot wrote:
Le 31/03/2015 15:55, Lior Vernia a écrit :
So the lower ones are allocated. But I would expect nic0 to receive a lower MAC address than nic1 on the same VM. Have you encountered a situation where that is not the case?
Once again : YES. What I'm witnessing is that oVirt only uses the first free MAC, and does not mind the name.
Once again, what you're answering sounds unrelated to what I'm asking... :) If you're creating two NICs at the same time (from the new VM dialog), and oVirt uses the first free MAC address whenever it needs to allocate one... Then it allocates MAC addresses according to the order of the NICs, so nic0 would still always get a MAC address lower than that of nic1.
Le 01/04/2015 09:00, Lior Vernia a écrit :
On 31/03/15 17:59, Nicolas Ecarnot wrote:
Le 31/03/2015 15:55, Lior Vernia a écrit :
So the lower ones are allocated. But I would expect nic0 to receive a lower MAC address than nic1 on the same VM. Have you encountered a situation where that is not the case?
Once again : YES. What I'm witnessing is that oVirt only uses the first free MAC, and does not mind the name.
Once again, what you're answering sounds unrelated to what I'm asking... :) If you're creating two NICs at the same time (from the new VM dialog), and oVirt uses the first free MAC address whenever it needs to allocate one... Then it allocates MAC addresses according to the order of the NICs, so nic0 would still always get a MAC address lower than that of nic1.
Indeed. To be precise, your question did not implied : - two creations in a row, with no other user in between adding or deleting some MAC - it was a brand new and fresh VM I'm sure I can find plenty of weird cases were the simplest way we ALL understood from the start are not working the way we think. But once again, THIS is not my point. My point is : Once the MACs are allocated (whatever the way), how are they mapped to tell which gets named net0, net1, aso... -- Nicolas Ecarnot
participants (3)
-
Lior Vernia -
Nicolas Ecarnot -
Sven Kieske