Hello Arik,
we have upgrade one of our oVirt engines to 4.2.1 (from 4.1.9) and afterwards all nodes too. The cluster compatibility level has been set to 4.2.
Now we can't start a VM after it has been powered off. The only hint we found in engine.log is:
2018-03-07 14:51:52,504+01 INFO
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b54a-56d9-4708-8a67-5609438ddadb] START, UpdateVmDynamicDataVDSCommand( UpdateVmDynamicDataVDSCommandParameters:{hostId='null', vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vmDynamic='org.ovirt.engine.co <http://org.ovirt.engine.co>
<http://org.ovirt.engine.co>re.common.businessentities.VmDynamic@491983e9'}),
log id: 7d49849e 2018-03-07 14:51:52,509+01 INFO
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b54a-56d9-4708-8a67-5609438ddadb] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7d49849e 2018-03-07 14:51:52,531+01 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b54a-56d9-4708-8a67-5609438ddadb] START, CreateVDSCommand(
CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b', vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM [prod-hub-201]'}), log id: 4af1f227 2018-03-07 14:51:52,533+01 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b54a-56d9-4708-8a67-5609438ddadb] START, CreateBrokerVDSCommand(HostName = prod-node-210,
CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b', vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM [prod-hub-201]'}), log id: 71dcc8e7 2018-03-07 14:51:52,545+01 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b54a-56d9-4708-8a67-5609438ddadb] Failed in 'CreateBrokerVDS' method, for vds: 'prod-node-210'; host: 'prod-node-210': null 2018-03-07 14:51:52,546+01 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b54a-56d9-4708-8a67-5609438ddadb] Command 'CreateBrokerVDSCommand(HostName = prod-node-210,
CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b', vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM [prod-hub-201]'})' execution failed: null 2018-03-07 14:51:52,546+01 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b54a-56d9-4708-8a67-5609438ddadb] FINISH, CreateBrokerVDSCommand, log id: 71dcc8e7 2018-03-07 14:51:52,546+01 ERROR [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (EE-ManagedThreadFactory-engine-Thread-25) [f855b5 4a-56d9-4708-8a67-5609438ddadb] Failed to create VM: java.lang.NullPointerException at
org.ovirt.engine.core.vdsbroker.builder.vminfo.LibvirtVmXmlBuilder.lambda$writeInterfaces$23(LibvirtVmXmlBuilder.java:1066) [vdsbroker.jar:]
[...]
But this doesn't lead us to the root cause. I haven't found any matching bug tickets in release notes for upcoming 4.2.1. Can anyone help here?
What's the mac address of that VM? You can find it in the UI or with:
select mac_addr from vm_interface where vm_guid in (select vm_guid from vm_static where vm_name='<vm_name>');
Actually, different question - does this VM has unplugged network interface?
The VM has two NICs. Both are plugged.
The MAC addresses are 00:1a:4a:18:01:52 for nic1 and 00:1a:4a:36:01:67 for nic2.
OK, those seem like two valid mac addresses so maybe something is wrong with the vm devices. Could you please provide the output of:
select type, device, address, is_managed, is_plugged, alias from vm_device where vm_id in (select vm_guid from vm_static where vm_name='<vm_name>');
sure: engine=# select type, device, address, is_managed, is_plugged, alias from vm_device where vm_id in (select vm_guid from vm_static where vm_name='prod-hub-201'); type | device | address | is_managed | is_plugged | alias ------------+---------------+--------------------------------------------------- -----------+------------+------------+---------------- video | qxl | | t | t | controller | virtio-scsi | | t | t | balloon | memballoon | | t | f | balloon0 graphics | spice | | t | t | controller | virtio-serial | {slot=0x06, bus=0x00, domain=0x0000, type=pci, fun ction=0x0} | t | t | virtio-serial0 disk | disk | {slot=0x07, bus=0x00, domain=0x0000, type=pci, fun ction=0x0} | f | t | virtio-disk0 memballoon | memballoon | {slot=0x08, bus=0x00, domain=0x0000, type=pci, fun ction=0x0} | f | t | balloon0 interface | bridge | {slot=0x03, bus=0x00, domain=0x0000, type=pci, fun ction=0x0} | f | t | net0 interface | bridge | {slot=0x09, bus=0x00, domain=0x0000, type=pci, fun ction=0x0} | f | t | net1 controller | scsi | {slot=0x05, bus=0x00, domain=0x0000, type=pci, fun ction=0x0} | f | t | scsi0 controller | ide | {slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1} | f | t | ide controller | usb | {slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x2} | t | t | usb channel | unix | {bus=0, controller=0, type=virtio-serial, port=1} | f | t | channel0 channel | unix | {bus=0, controller=0, type=virtio-serial, port=2} | f | t | channel1 channel | spicevmc | {bus=0, controller=0, type=virtio-serial, port=3} | f | t | channel2 interface | bridge | | t | t | net1 interface | bridge | | t | t | net0 disk | cdrom | | t | f | ide0-1-0 disk | cdrom | {bus=1, controller=0, type=drive, target=0, unit=0} | f | t | ide0-1-0 disk | disk | | t | t | virtio-disk0 (20 rows) Kind regards Jan