[ovirt-users] oVirt / ROM images / PXE

[Paul Heinlein]

Summary: To get oVirt-managed VMs to boot using PXE, I had to replace 
the rhel6-*.rom files with their ipxe equivalents.

Setup: Nodes running CentOS 7, fully current. Engine 3.5.0.1 running 
on Fedora 19. Storage via NFS. 1GB ethernet network. DHCP server on 
separate subnet, relying on dhcp-relay switch settings.

PXE is our preferred method for installing VMs. We rarely use 
templates.

I recently upgraded our oVirt hypervisor nodes from Fedora 19 to 
CentOS 7 (where "upgrade" obviously implies a full OS 
re-installation). I failed to test creating VMs using PXE in this 
setup because -- and this is crucial -- it worked fine on our 
non-oVirt CentOS 7 hypervisors, which are managed using virsh and 
virt-install.

So PXE booting a new VM failed. The symptom was the dreaded "dhcp 
connection timeout" at the seabios prompt. The timout would persist 
even when I pressed Ctrl-B and re-tried "dhcp net0" several times over 
the course of the next few minutes.

Significantly, existing VMs experienced no DHCP troubles at all, in 
the sense that dhclient or its equivalent within the VM could get 
and renew leases as expected.

I poked around the forward-delay settings on the oVirt-managed 
bridges, but they had the same charactistics of the bridges on our 
non-oVirt hypervisors. Lots of other network-oriented troubleshooting 
likewise failed.

It got to the point that I took one of my oVirt nodes, removed it from 
oVirt, and reinstalled it with plain CentOS 7. Its physical 
connections and IP addresses stayed exactly the same. Without oVirt, 
virt-install used PXE without a hitch. I re-added that machine to the 
oVirt cluster, and PXE failed once again.

I noticed that the BIOS PXE prompt was slightly different in the ovirt 
and non-ovirt environment, which led me to poke around the ROM images 
in /usr/share/qemu-kvm.

I think the rhel6-*.rom images are those used by oVirt, while the 
pxe-*.rom images (actually symlinks into ../ipxe/) are used by, e.g., 
the local virt-install utility.

My workaround is to replace the rhel6-*.rom files with the same 
symlinks used by the pxe-*.rom files:

----- %< -----
# short version of script, minus error checking
cd /usr/share/qemu-kvm
for NIC in e1000 ne2k_pci pcnet rtl8139 virtio; do
   mv rhel6-${NIC}.rom rhel6-${NIC}.rom.dist
   ln -s $(readlink pxe-${NIC}.rom) rhel6-${NIC}.rom
done
----- %< -----

Now oVirt VMs can boot from PXE without any issue.

I'm wildly curious about what's going on here.

-- 
Paul Heinlein
heinlein at madboa.com
45°38' N, 122°6' W