I ve got 4 Hosts after changeing the harddrive on every Host and normal updates I am not able anymore to start VM's on 2 of these 4 hosts. I am also not able to migrate a running vm to these hosts. This is the error log for start up 2019-12-17 16:01:28,326+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] Lock Acquired to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:28,609+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b'}), log id: 5706d380 2019-12-17 16:01:28,611+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5706d380 2019-12-17 16:01:28,854+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] Running command: RunVmOnceCommand internal: false. Entities affected : ID: 40cf6c27-6464-4fa7-bc01-9158cb03328b Type: VMAction group RUN_VM with role type USER, ID: 40cf6c27-6464-4fa7-bc01-9158cb03328b Type: VMAction group EDIT_ADMIN_VM_PROPERTIES with role type ADMIN 2019-12-17 16:01:28,999+01 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, UpdateVmDynamicDataVDSCommand( UpdateVmDynamicDataVDSCommandParameters:{hostId='null', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', vmDynamic='org.ovirt.engine.core.common.businessentities.VmDynamic@147620b'}), log id: 612dabdf 2019-12-17 16:01:29,097+01 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, UpdateVmDynamicDataVDSCommand, return: , log id: 612dabdf 2019-12-17 16:01:29,106+01 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, CreateVDSCommand( CreateVDSCommandParameters:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', vm='VM [z-push]'}), log id: 56abde11 2019-12-17 16:01:29,110+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, CreateBrokerVDSCommand(HostName = kvm360.durchhalten.intern, CreateVDSCommandParameters:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', vm='VM [z-push]'}), log id: 36d088ab 2019-12-17 16:01:29,163+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] VM <?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> <name>z-push</name> <uuid>40cf6c27-6464-4fa7-bc01-9158cb03328b</uuid> <memory>4194304</memory> <currentMemory>4194304</currentMemory> <iothreads>1</iothreads> <maxMemory slots="16">16777216</maxMemory> <vcpu current="4">16</vcpu> <sysinfo type="smbios"> <system> <entry name="manufacturer">oVirt</entry> <entry name="product">OS-NAME:</entry> <entry name="version">OS-VERSION:</entry> <entry name="serial">HOST-SERIAL:</entry> <entry name="uuid">40cf6c27-6464-4fa7-bc01-9158cb03328b</entry> </system> </sysinfo> <clock offset="variable" adjustment="0"> <timer name="rtc" tickpolicy="catchup"/> <timer name="pit" tickpolicy="delay"/> <timer name="hpet" present="no"/> </clock> <features> <acpi/> </features> <cpu match="exact"> <model>SandyBridge</model> <feature name="pcid" policy="require"/> <feature name="spec-ctrl" policy="require"/> <feature name="ssbd" policy="require"/> <topology cores="1" threads="1" sockets="16"/> <numa> <cell id="0" cpus="0,1,2,3" memory="4194304"/> </numa> </cpu> <cputune/> <devices> <input type="mouse" bus="ps2"/> <channel type="unix"> <target type="virtio" name="ovirt-guest-agent.0"/> <source mode="bind" path="/var/lib/libvirt/qemu/channels/40cf6c27-6464-4fa7-bc01-9158cb03328b.ovirt-guest-agent.0"/> </channel> <channel type="unix"> <target type="virtio" name="org.qemu.guest_agent.0"/> <source mode="bind" path="/var/lib/libvirt/qemu/channels/40cf6c27-6464-4fa7-bc01-9158cb03328b.org.qemu.guest_agent.0"/> </channel> <rng model="virtio"> <backend model="random">/dev/urandom</backend> <alias name="ua-375334ff-0392-4314-879d-d89d81af5b69"/> </rng> <watchdog model="i6300esb" action="none"> <alias name="ua-5a4b6796-47bd-40e4-99ac-81b9546e6824"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x08" type="pci"/> </watchdog> <memballoon model="virtio"> <stats period="5"/> <alias name="ua-6d47e6f5-fd4f-452d-bcca-895b08d896f8"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x06" type="pci"/> </memballoon> <video> <model type="qxl" vram="32768" heads="1" ram="65536" vgamem="16384"/> <alias name="ua-79117281-c935-4cfc-aa21-9f028d817d79"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci"/> </video> <controller type="usb" model="piix3-uhci" index="0"> <address bus="0x00" domain="0x0000" function="0x2" slot="0x01" type="pci"/> </controller> <controller type="virtio-serial" index="0" ports="16"> <alias name="ua-b82337cf-7f59-4390-ad5b-4c940e205fae"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x05" type="pci"/> </controller> <graphics type="spice" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" tlsPort="-1"> <channel name="main" mode="secure"/> <channel name="inputs" mode="secure"/> <channel name="cursor" mode="secure"/> <channel name="playback" mode="secure"/> <channel name="record" mode="secure"/> <channel name="display" mode="secure"/> <channel name="smartcard" mode="secure"/> <channel name="usbredir" mode="secure"/> <listen type="network" network="vdsm-ovirtmgmt"/> </graphics> <channel type="spicevmc"> <target type="virtio" name="com.redhat.spice.0"/> </channel> <interface type="bridge"> <model type="virtio"/> <link state="up"/> <source bridge="ovirtmgmt"/> <driver queues="4" name="vhost"/> <alias name="ua-db1017dd-9b76-4245-85fd-7490185434ac"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci"/> <boot order="2"/> <mac address="56:6f:c1:d7:00:02"/> <mtu size="1500"/> <filterref filter="vdsm-no-mac-spoofing"/> <bandwidth/> </interface> <disk type="file" device="cdrom" snapshot="no"> <driver name="qemu" type="raw" error_policy="report"/> <source file="" startupPolicy="optional"> <seclabel model="dac" type="none" relabel="no"/> </source> <target dev="hdc" bus="ide"/> <readonly/> <alias name="ua-c2fa60b6-be89-487c-b01f-ed14c2d4624f"/> <address bus="1" controller="0" unit="0" type="drive" target="0"/> </disk> <disk snapshot="no" type="file" device="disk"> <target dev="sda" bus="scsi"/> <source file="/rhev/data-center/0e279f02-0c73-11e9-bc47-00163e150480/80364cc8-8afd-4482-884d-d63d5a7988df/images/38f2e3bb-687f-4f01-b769-3a5403136181/1a9627d8-181b-4790-aa47-569dd1e6e9fb"> <seclabel model="dac" type="none" relabel="no"/> </source> <driver name="qemu" io="threads" type="qcow2" error_policy="stop" cache="none"/> <alias name="ua-38f2e3bb-687f-4f01-b769-3a5403136181"/> <address bus="0" controller="0" unit="0" type="drive" target="0"/> <boot order="1"/> <serial>38f2e3bb-687f-4f01-b769-3a5403136181</serial> </disk> </devices> <pm> <suspend-to-disk enabled="no"/> <suspend-to-mem enabled="no"/> </pm> <os> <type arch="x86_64" machine="pc-i440fx-rhel7.6.0">hvm</type> <smbios mode="sysinfo"/> <bootmenu enable="yes" timeout="30000"/> </os> <metadata> <ovirt-tune:qos/> <ovirt-vm:vm> <ovirt-vm:minGuaranteedMemoryMb type="int">4096</ovirt-vm:minGuaranteedMemoryMb> <ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion> <ovirt-vm:custom/> <ovirt-vm:device mac_address="56:6f:c1:d7:00:02"> <ovirt-vm:custom/> </ovirt-vm:device> <ovirt-vm:device devtype="disk" name="sda"> <ovirt-vm:poolID>0e279f02-0c73-11e9-bc47-00163e150480</ovirt-vm:poolID> <ovirt-vm:volumeID>1a9627d8-181b-4790-aa47-569dd1e6e9fb</ovirt-vm:volumeID> <ovirt-vm:imageID>38f2e3bb-687f-4f01-b769-3a5403136181</ovirt-vm:imageID> <ovirt-vm:domainID>80364cc8-8afd-4482-884d-d63d5a7988df</ovirt-vm:domainID> </ovirt-vm:device> <ovirt-vm:launchPaused>false</ovirt-vm:launchPaused> <ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior> </ovirt-vm:vm> </metadata> </domain> 2019-12-17 16:01:29,198+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, CreateBrokerVDSCommand, return: , log id: 36d088ab 2019-12-17 16:01:29,252+01 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, CreateVDSCommand, return: WaitForLaunch, log id: 56abde11 2019-12-17 16:01:29,252+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] Lock freed to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:29,319+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] EVENT_ID: USER_STARTED_VM(153), VM z-push was started by admin@internal-authz (Host: kvm360.durchhalten.intern). 2019-12-17 16:01:30,412+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] Fetched 1 VMs from VDS 'd38bae4c-8494-4861-ae5a-38992db338e5' 2019-12-17 16:01:30,620+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] EVENT_ID: VM_CONSOLE_DISCONNECTED(168), User <UNKNOWN> got disconnected from VM z-push. 2019-12-17 16:01:30,766+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DumpXmlsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] START, DumpXmlsVDSCommand(HostName = kvm360.durchhalten.intern, Params:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmIds='[40cf6c27-6464-4fa7-bc01-9158cb03328b]'}), log id: 4be2467a 2019-12-17 16:01:30,774+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DumpXmlsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] FINISH, DumpXmlsVDSCommand, return: {40cf6c27-6464-4fa7-bc01-9158cb03328b=<?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"><name>z-push&... slots="16">16777216</maxMemory><vcpu current="4">16</vcpu><sysinfo type="smbios"><system><entry name="manufacturer">oVirt</entry><entry name="product">OS-NAME:</entry><entry name="version">OS-VERSION:</entry><entry name="serial">HOST-SERIAL:</entry><entry name="uuid">40cf6c27-6464-4fa7-bc01-9158cb03328b</entry></system></sysinfo><clock offset="variable" adjustment="0"><timer name="rtc" tickpolicy="catchup"></timer><timer name="pit" tickpolicy="delay"></timer -4314-879d-d89d81af5b69"></alias></rng><watchdog model="i6300esb" action="none"><alias name="ua-5a4b6796-47bd-40e4-99ac-81b9546e6824"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x08" type="pci"></address></watchdog><memballoon model="virtio"><stats period="5"></stats><alias name="ua-6d47e6f5-fd4f-452d-bcca-895b08d896f8"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x06" type="pci"></address></memballoon><video><model type="qxl" vram="32768" heads="1" ram="65536" vgamem="16384"></model><alias name="ua-79117281-c935-4cfc-aa21-9f028d817d79"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci"></address></video><controller type="usb" model="piix3-uhci" index="0"><address bus="0x00" domain="0x0000" function="0x2" slot="0x01" type="pci"></address></controller><controller type="virtio-serial" index="0" ports="16"><alias name="ua-b82337cf-7f59-4390-ad5b-4c940e205fae"></alias><address bus="0x00" domain="0x0000" function=" 0x0" slot="0x05" type="pci"></address></controller><graphics type="spice" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" tlsPort="-1"><channel name="main" mode="secure"></channel><channel name="inputs" mode="secure"></channel><channel name="cursor" mode="secure"></channel><channel name="playback" mode="secure"></channel><channel name="record" mode="secure"></channel><channel name="display" mode="secure"></channel><channel name="smartcard" mode="secure"></channel><channel name="usbredir" mode="secure"></channel><listen type="network" network="vdsm-ovirtmgmt"></listen></graphics><channel type="spicevmc"><target type="virtio" name="com.redhat.spice.0"></target></channel><interface type="bridge"><model type="virtio"></model><link state="up"></link><source bridge="ovirtmgmt"></source><driver queues="4" name="vhost"></driver><alias name="ua-db1017dd-9b76-4245-85fd-7490185434ac"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci">< /address><boot order="2"></boot><mac address="56:6f:c1:d7:00:02"></mac><mtu size="1500"></mtu><filterref filter="vdsm-no-mac-spoofing"></filterref><bandwidth></bandwidth></interface><disk type="file" device="cdrom" snapshot="no"><driver name="qemu" type="raw" error_policy="report"></driver><source file="" startupPolicy="optional"><seclabel model="dac" type="none" relabel="no"></seclabel></source><target dev="hdc" bus="ide"></target><readonly></readonly><alias name="ua-c2fa60b6-be89-487c-b01f-ed14c2d4624f"></alias><address bus="1" controller="0" unit="0" type="drive" target="0"></address></disk><disk snapshot="no" type="file" device="disk"><target dev="sda" bus="scsi"></target><source file="/rhev/data-center/0e279f02-0c73-11e9-bc47-00163e150480/80364cc8-8afd-4482-884d-d63d5a7988df/images/38f2e3bb-687f-4f01-b769-3a5403136181/1a9627d8-181b-4790-aa47-569dd1e6e9fb"><seclabel model="dac" type="none" relabel="no"></seclabel></source><driver name="qemu" io="threads" type="qcow2" error_polic y="stop" cache="none"></driver><alias name="ua-38f2e3bb-687f-4f01-b769-3a5403136181"></alias><address bus="0" controller="0" unit="0" type="drive" target="0"></address><boot order="1"></boot><serial>38f2e3bb-687f-4f01-b769-3a5403136181</serial></disk></devices><pm><suspend-to-disk enabled="no"></suspend-to-disk><suspend-to-mem enabled="no"></suspend-to-mem></pm><os><type arch="x86_64" machine="pc-i440fx-rhel7.6.0">hvm</type><smbios mode="sysinfo"></smbios><bootmenu enable="yes" timeout="30000"></bootmenu></os><metadata><ovirt-tune:qos></ovirt-tune:qos><ovirt-vm:vm><ovirt-vm:minGuaranteedMemoryMb type="int">4096</ovirt-vm:minGuaranteedMemoryMb><ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion><ovirt-vm:custom></ovirt-vm:custom><ovirt-vm:device mac_address="56:6f:c1:d7:00:02"><ovirt-vm:custom></ovirt-vm:custom></ovirt-vm:device><ovirt-vm:device devtype="disk" name="sda"><ovirt-vm:poolID>0e279f02-0c73-11e9-bc47-00163e150480</ovirt-vm:poolID><ovirt-vm:volumeID>1a9627d8-181b-4790-aa4 7-569dd1e6e9fb</ovirt-vm:volumeID><ovirt-vm:imageID>38f2e3bb-687f-4f01-b769-3a5403136181</ovirt-vm:imageID><ovirt-vm:domainID>80364cc8-8afd-4482-884d-d63d5a7988df</ovirt-vm:domainID></ovirt-vm:device><ovirt-vm:launchPaused>false</ovirt-vm:launchPaused><ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior></ovirt-vm:vm></metadata></domain>}, log id: 4be2467a 2019-12-17 16:01:30,918+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] VM '40cf6c27-6464-4fa7-bc01-9158cb03328b' managed non pluggable device was removed unexpectedly from libvirt: 'VmDevice:{id='VmDeviceId:{deviceId='5f17a444-36e8-4854-a51b-f355aab2d8a8', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b'}', device='virtio-scsi', type='CONTROLLER', specParams='[ioThreadId=1]', address='', managed='true', plugged='false', readOnly='false', deviceAlias='', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}' 2019-12-17 16:01:32,762+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] VM '40cf6c27-6464-4fa7-bc01-9158cb03328b' was reported as Down on VDS 'd38bae4c-8494-4861-ae5a-38992db338e5'(kvm360.durchhalten.intern) 2019-12-17 16:01:32,764+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-12) [] START, DestroyVDSCommand(HostName = kvm360.durchhalten.intern, DestroyVmVDSCommandParameters:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 3478fcba 2019-12-17 16:01:33,283+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-12) [] FINISH, DestroyVDSCommand, return: , log id: 3478fcba 2019-12-17 16:01:33,283+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] VM '40cf6c27-6464-4fa7-bc01-9158cb03328b'(z-push) moved from 'WaitForLaunch' --> 'Down' 2019-12-17 16:01:33,510+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM z-push is down with error. Exit message: The name org.fedoraproject.FirewallD1 was not provided by any .service files. 2019-12-17 16:01:33,512+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] add VM '40cf6c27-6464-4fa7-bc01-9158cb03328b'(z-push) to rerun treatment 2019-12-17 16:01:33,743+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-12) [] Rerun VM '40cf6c27-6464-4fa7-bc01-9158cb03328b'. Called from VDS 'kvm360.durchhalten.intern' 2019-12-17 16:01:33,877+01 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-753) [] EVENT_ID: USER_INITIATED_RUN_VM_FAILED(151), Failed to run VM z-push on Host kvm360.durchhalten.intern. 2019-12-17 16:01:34,057+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] Lock Acquired to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:34,085+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b'}), log id: 4074b804 2019-12-17 16:01:34,086+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 4074b804 2019-12-17 16:01:34,134+01 WARN [org.ovirt.engine.core.bll.RunVmOnceCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] Validation of action 'RunVmOnce' failed for user admin@internal-authz. Reasons: VAR__ACTION__RUN,VAR__TYPE__VM,VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_NO_HOSTS 2019-12-17 16:01:34,135+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] Lock freed to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:34,300+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-753) [] EVENT_ID: USER_FAILED_RUN_VM(54), Failed to run VM z-push (User: admin@internal-authz). 2019-12-17 16:01:34,314+01 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-754) [2ecacb63] Running command: ProcessDownVmCommand internal: true. 2019-12-17 16:01:46,128+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Fetched 0 VMs from VDS 'd38bae4c-8494-4861-ae5a-38992db338e5' and this for migration 2019-12-17 16:05:42,411+01 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] Lock Acquired to object 'EngineLock:{exclusiveLocks='[46755b18-ac33-4e62-b1f0-a746bfeed59b=VM]', sharedLocks=''}' 2019-12-17 16:05:42,685+01 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 46755b18-ac33-4e62-b1f0-a746bfeed59b Type: VMAction group MIGRATE_VM with role type USER 2019-12-17 16:05:43,043+01 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId='7895de21-d174-471f-a0e5-340d59be1bb0', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b', srcHost='kvm380.durchhalten.intern', dstVdsId='d104ef2f-98a4-4a99-b28d-2725294f8626', dstHost='kvm320.durchhalten.intern:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='62', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {li mit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.200.231'}), log id: 57fb24a3 2019-12-17 16:05:43,048+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] START, MigrateBrokerVDSCommand(HostName = kvm380.durchhalten.intern, MigrateVDSCommandParameters:{hostId='7895de21-d174-471f-a0e5-340d59be1bb0', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b', srcHost='kvm380.durchhalten.intern', dstVdsId='d104ef2f-98a4-4a99-b28d-2725294f8626', dstHost='kvm320.durchhalten.intern:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='62', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400] }}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.200.231'}), log id: 6765eb33 2019-12-17 16:05:44,068+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] FINISH, MigrateBrokerVDSCommand, return: , log id: 6765eb33 2019-12-17 16:05:44,126+01 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 57fb24a3 2019-12-17 16:05:44,205+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] EVENT_ID: VM_MIGRATION_START(62), Migration started (VM: TestVM, Source: kvm380.durchhalten.intern, Destination: kvm320.durchhalten.intern, User: admin@internal-authz). 2019-12-17 16:05:44,588+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-2) [] VM '46755b18-ac33-4e62-b1f0-a746bfeed59b' was reported as Down on VDS 'd104ef2f-98a4-4a99-b28d-2725294f8626'(kvm320.durchhalten.intern) 2019-12-17 16:05:44,591+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-2) [] START, DestroyVDSCommand(HostName = kvm320.durchhalten.intern, DestroyVmVDSCommandParameters:{hostId='d104ef2f-98a4-4a99-b28d-2725294f8626', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 1afbc395 2019-12-17 16:05:44,853+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-2) [] Failed to destroy VM '46755b18-ac33-4e62-b1f0-a746bfeed59b' because VM does not exist, ignoring 2019-12-17 16:05:44,854+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-2) [] FINISH, DestroyVDSCommand, return: , log id: 1afbc395 2019-12-17 16:05:44,854+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-2) [] VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'(TestVM) was unexpectedly detected as 'Down' on VDS 'd104ef2f-98a4-4a99-b28d-2725294f8626'(kvm320.durchhalten.intern) (expected on '7895de21-d174-471f-a0e5-340d59be1bb0') 2019-12-17 16:05:44,854+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-2) [] Migration of VM 'TestVM' to host 'kvm320.durchhalten.intern' failed: VM destroyed during the startup. 2019-12-17 16:05:44,861+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-9) [] VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'(TestVM) moved from 'MigratingFrom' --> 'Up' 2019-12-17 16:05:44,862+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-9) [] Adding VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'(TestVM) to re-run list 2019-12-17 16:05:44,927+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-9) [] Rerun VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'. Called from VDS 'kvm380.durchhalten.intern' 2019-12-17 16:05:44,944+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-879) [] START, MigrateStatusVDSCommand(HostName = kvm380.durchhalten.intern, MigrateStatusVDSCommandParameters:{hostId='7895de21-d174-471f-a0e5-340d59be1bb0', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b'}), log id: 127dcc70 2019-12-17 16:05:44,952+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-879) [] FINISH, MigrateStatusVDSCommand, return: , log id: 127dcc70 2019-12-17 16:05:45,228+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-879) [] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: TestVM, Source: kvm380.durchhalten.intern, Destination: kvm320.durchhalten.intern). 2019-12-17 16:05:45,329+01 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (EE-ManagedThreadFactory-engine-Thread-879) [] Lock freed to object 'EngineLock:{exclusiveLocks='[46755b18-ac33-4e62-b1f0-a746bfeed59b=VM]', sharedLocks=''}' maybe someone can give me a hint thx shb