Migration failed, No available host found
Hi. I have 3 nodes in one cluster and 1 VM running on node2. I'm trying to move the VM to node 1 or node 3, and it fails with the error: Migration failed, No available host found I'm unable to decipher engine.log to determine the cause of the problem. Below is what seems to be the relevant lines from the log. Any help would be appreciated. Thank you! Jason. --- 2015-04-06 08:31:56,554 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp--127.0.0.1-8702-5) [3b191496] Lock Acquired to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:31:56,686 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:31:56,703 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.HaReservationWeightPolicyUnit] (org.ovirt.thread.pool-8-thread-20) [3b191496] Started HA reservation scoring method 2015-04-06 08:31:56,727 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 7555acbd 2015-04-06 08:31:56,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 6d98fb94 2015-04-06 08:31:56,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateBrokerVDSCommand, log id: 6d98fb94 2015-04-06 08:31:56,769 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 7555acbd 2015-04-06 08:31:56,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-20) [3b191496] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt3, User: admin@internal). 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:33:17,661 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:33:17,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 6c3c9923 2015-04-06 08:33:17,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Failed in MigrateStatusVDS method 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] HostName = virt2 2015-04-06 08:33:17,670 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:33:17,671 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateStatusVDSCommand, log id: 6c3c9923 2015-04-06 08:33:17,685 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt3). 2015-04-06 08:33:17,776 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:33:17,825 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 5642c909 2015-04-06 08:33:17,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1d4ead35 2015-04-06 08:33:17,833 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateBrokerVDSCommand, log id: 1d4ead35 2015-04-06 08:33:17,843 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5642c909 2015-04-06 08:33:17,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt1, User: admin@internal). 2015-04-06 08:34:42,969 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:34:42,970 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:34:42,983 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:34:43,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 7b89f823 2015-04-06 08:34:43,029 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Failed in MigrateStatusVDS method 2015-04-06 08:34:43,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] HostName = virt2 2015-04-06 08:34:43,030 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] FINISH, MigrateStatusVDSCommand, log id: 7b89f823 2015-04-06 08:34:43,049 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt1). 2015-04-06 08:34:43,132 WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] CanDoAction of action MigrateVm failed for user admin@internal. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_NO_HOSTS 2015-04-06 08:34:43,132 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Lock freed to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:34:43,165 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed, No available host found (VM: nindigo, Source: virt2).
Engine try to migrate vm on some available host, but migration failed, so engine try another host. From some reason migration failed on all hosts: (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 For future investigation we need vdsm logs(/var/log/vdsm/vdsm.log) from source and also from destination hosts. Thanks ----- Original Message ----- From: "Jason Keltz" <jas@cse.yorku.ca> To: "users" <users@ovirt.org> Sent: Monday, April 6, 2015 3:47:23 PM Subject: [ovirt-users] Migration failed, No available host found Hi. I have 3 nodes in one cluster and 1 VM running on node2. I'm trying to move the VM to node 1 or node 3, and it fails with the error: Migration failed, No available host found I'm unable to decipher engine.log to determine the cause of the problem. Below is what seems to be the relevant lines from the log. Any help would be appreciated. Thank you! Jason. --- 2015-04-06 08:31:56,554 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp--127.0.0.1-8702-5) [3b191496] Lock Acquired to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:31:56,686 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:31:56,703 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.HaReservationWeightPolicyUnit] (org.ovirt.thread.pool-8-thread-20) [3b191496] Started HA reservation scoring method 2015-04-06 08:31:56,727 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 7555acbd 2015-04-06 08:31:56,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 6d98fb94 2015-04-06 08:31:56,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateBrokerVDSCommand, log id: 6d98fb94 2015-04-06 08:31:56,769 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 7555acbd 2015-04-06 08:31:56,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-20) [3b191496] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt3, User: admin@internal). 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:33:17,661 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:33:17,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 6c3c9923 2015-04-06 08:33:17,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Failed in MigrateStatusVDS method 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] HostName = virt2 2015-04-06 08:33:17,670 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:33:17,671 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateStatusVDSCommand, log id: 6c3c9923 2015-04-06 08:33:17,685 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt3). 2015-04-06 08:33:17,776 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:33:17,825 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 5642c909 2015-04-06 08:33:17,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1d4ead35 2015-04-06 08:33:17,833 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateBrokerVDSCommand, log id: 1d4ead35 2015-04-06 08:33:17,843 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5642c909 2015-04-06 08:33:17,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt1, User: admin@internal). 2015-04-06 08:34:42,969 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:34:42,970 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:34:42,983 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:34:43,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 7b89f823 2015-04-06 08:34:43,029 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Failed in MigrateStatusVDS method 2015-04-06 08:34:43,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] HostName = virt2 2015-04-06 08:34:43,030 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] FINISH, MigrateStatusVDSCommand, log id: 7b89f823 2015-04-06 08:34:43,049 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt1). 2015-04-06 08:34:43,132 WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] CanDoAction of action MigrateVm failed for user admin@internal. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_NO_HOSTS 2015-04-06 08:34:43,132 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Lock freed to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:34:43,165 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed, No available host found (VM: nindigo, Source: virt2). _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Artyom, Here are the vdsm logs from virt1, virt2 (where the node is running), and virt3. The logs from virt2 look suspicious, but still not sure the problem. http://goo.gl/GjbWUP Jason. On 04/06/2015 09:42 AM, Artyom Lukianov wrote:
Engine try to migrate vm on some available host, but migration failed, so engine try another host. From some reason migration failed on all hosts: (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12
For future investigation we need vdsm logs(/var/log/vdsm/vdsm.log) from source and also from destination hosts. Thanks
----- Original Message ----- From: "Jason Keltz" <jas@cse.yorku.ca> To: "users" <users@ovirt.org> Sent: Monday, April 6, 2015 3:47:23 PM Subject: [ovirt-users] Migration failed, No available host found
Hi.
I have 3 nodes in one cluster and 1 VM running on node2. I'm trying to move the VM to node 1 or node 3, and it fails with the error: Migration failed, No available host found
I'm unable to decipher engine.log to determine the cause of the problem. Below is what seems to be the relevant lines from the log. Any help would be appreciated.
Thank you!
Jason.
---
2015-04-06 08:31:56,554 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp--127.0.0.1-8702-5) [3b191496] Lock Acquired to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:31:56,686 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:31:56,703 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.HaReservationWeightPolicyUnit] (org.ovirt.thread.pool-8-thread-20) [3b191496] Started HA reservation scoring method 2015-04-06 08:31:56,727 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 7555acbd 2015-04-06 08:31:56,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 6d98fb94 2015-04-06 08:31:56,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateBrokerVDSCommand, log id: 6d98fb94 2015-04-06 08:31:56,769 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 7555acbd 2015-04-06 08:31:56,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-20) [3b191496] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt3, User: admin@internal). 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:33:17,661 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:33:17,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 6c3c9923 2015-04-06 08:33:17,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Failed in MigrateStatusVDS method 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] HostName = virt2 2015-04-06 08:33:17,670 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:33:17,671 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateStatusVDSCommand, log id: 6c3c9923 2015-04-06 08:33:17,685 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt3). 2015-04-06 08:33:17,776 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:33:17,825 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 5642c909 2015-04-06 08:33:17,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1d4ead35 2015-04-06 08:33:17,833 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateBrokerVDSCommand, log id: 1d4ead35 2015-04-06 08:33:17,843 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5642c909 2015-04-06 08:33:17,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt1, User: admin@internal). 2015-04-06 08:34:42,969 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:34:42,970 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:34:42,983 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:34:43,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 7b89f823 2015-04-06 08:34:43,029 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Failed in MigrateStatusVDS method 2015-04-06 08:34:43,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] HostName = virt2 2015-04-06 08:34:43,030 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] FINISH, MigrateStatusVDSCommand, log id: 7b89f823 2015-04-06 08:34:43,049 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt1). 2015-04-06 08:34:43,132 WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] CanDoAction of action MigrateVm failed for user admin@internal. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_NO_HOSTS 2015-04-06 08:34:43,132 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Lock freed to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:34:43,165 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed, No available host found (VM: nindigo, Source: virt2). _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Artyom, The problems were caused by an issue with MTU on the hosts. I have rectified the issue and can now migrate hosts. Jason. On 04/06/2015 10:57 AM, Jason Keltz wrote:
Hi Artyom,
Here are the vdsm logs from virt1, virt2 (where the node is running), and virt3. The logs from virt2 look suspicious, but still not sure the problem.
Jason.
On 04/06/2015 09:42 AM, Artyom Lukianov wrote:
Engine try to migrate vm on some available host, but migration failed, so engine try another host. From some reason migration failed on all hosts: (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12
For future investigation we need vdsm logs(/var/log/vdsm/vdsm.log) from source and also from destination hosts. Thanks
----- Original Message ----- From: "Jason Keltz" <jas@cse.yorku.ca> To: "users" <users@ovirt.org> Sent: Monday, April 6, 2015 3:47:23 PM Subject: [ovirt-users] Migration failed, No available host found
Hi.
I have 3 nodes in one cluster and 1 VM running on node2. I'm trying to move the VM to node 1 or node 3, and it fails with the error: Migration failed, No available host found
I'm unable to decipher engine.log to determine the cause of the problem. Below is what seems to be the relevant lines from the log. Any help would be appreciated.
Thank you!
Jason.
---
2015-04-06 08:31:56,554 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp--127.0.0.1-8702-5) [3b191496] Lock Acquired to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:31:56,686 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:31:56,703 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.HaReservationWeightPolicyUnit]
(org.ovirt.thread.pool-8-thread-20) [3b191496] Started HA reservation scoring method 2015-04-06 08:31:56,727 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 7555acbd 2015-04-06 08:31:56,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 6d98fb94 2015-04-06 08:31:56,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateBrokerVDSCommand, log id: 6d98fb94 2015-04-06 08:31:56,769 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 7555acbd 2015-04-06 08:31:56,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-20) [3b191496] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt3, User: admin@internal). 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:33:17,661 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:33:17,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 6c3c9923 2015-04-06 08:33:17,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Failed in MigrateStatusVDS method 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] HostName = virt2 2015-04-06 08:33:17,670 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:33:17,671 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateStatusVDSCommand, log id: 6c3c9923 2015-04-06 08:33:17,685 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt3). 2015-04-06 08:33:17,776 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:33:17,825 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 5642c909 2015-04-06 08:33:17,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1d4ead35 2015-04-06 08:33:17,833 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateBrokerVDSCommand, log id: 1d4ead35 2015-04-06 08:33:17,843 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5642c909 2015-04-06 08:33:17,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt1, User: admin@internal). 2015-04-06 08:34:42,969 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:34:42,970 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:34:42,983 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:34:43,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 7b89f823 2015-04-06 08:34:43,029 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Failed in MigrateStatusVDS method 2015-04-06 08:34:43,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] HostName = virt2 2015-04-06 08:34:43,030 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] FINISH, MigrateStatusVDSCommand, log id: 7b89f823 2015-04-06 08:34:43,049 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt1). 2015-04-06 08:34:43,132 WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] CanDoAction of action MigrateVm failed for user admin@internal. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_NO_HOSTS
2015-04-06 08:34:43,132 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Lock freed to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:34:43,165 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed, No available host found (VM: nindigo, Source: virt2). _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
HI, Jason Could u explain the detailed way of changing MTU? And how to check the root cause of MTU from log? -----邮件原件----- 发件人: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] 代表 Jason Keltz 发送时间: 2015年4月7日 0:14 收件人: users@ovirt.org 主题: Re: [ovirt-users] Migration failed, No available host found Hi Artyom, The problems were caused by an issue with MTU on the hosts. I have rectified the issue and can now migrate hosts. Jason. On 04/06/2015 10:57 AM, Jason Keltz wrote:
Hi Artyom,
Here are the vdsm logs from virt1, virt2 (where the node is running), and virt3. The logs from virt2 look suspicious, but still not sure the problem.
Jason.
On 04/06/2015 09:42 AM, Artyom Lukianov wrote:
Engine try to migrate vm on some available host, but migration failed, so engine try another host. From some reason migration failed on all hosts: (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12
For future investigation we need vdsm logs(/var/log/vdsm/vdsm.log) from source and also from destination hosts. Thanks
----- Original Message ----- From: "Jason Keltz" <jas@cse.yorku.ca> To: "users" <users@ovirt.org> Sent: Monday, April 6, 2015 3:47:23 PM Subject: [ovirt-users] Migration failed, No available host found
Hi.
I have 3 nodes in one cluster and 1 VM running on node2. I'm trying to move the VM to node 1 or node 3, and it fails with the error: Migration failed, No available host found
I'm unable to decipher engine.log to determine the cause of the problem. Below is what seems to be the relevant lines from the log. Any help would be appreciated.
Thank you!
Jason.
---
2015-04-06 08:31:56,554 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp--127.0.0.1-8702-5) [3b191496] Lock Acquired to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:31:56,686 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:31:56,703 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.HaReservationWeight PolicyUnit]
(org.ovirt.thread.pool-8-thread-20) [3b191496] Started HA reservation scoring method 2015-04-06 08:31:56,727 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 7555acbd 2015-04-06 08:31:56,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=3429b1fc-36d5-4078-831c-a5b4370a8bfc, dstHost=192.168.0.36:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 6d98fb94 2015-04-06 08:31:56,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateBrokerVDSCommand, log id: 6d98fb94 2015-04-06 08:31:56,769 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-20) [3b191496] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 7555acbd 2015-04-06 08:31:56,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector ] (org.ovirt.thread.pool-8-thread-20) [3b191496] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt3, User: admin@internal). 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:33:17,633 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:33:17,661 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [71f97a52] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:33:17,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 6c3c9923 2015-04-06 08:33:17,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Failed in MigrateStatusVDS method 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:33:17,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] HostName = virt2 2015-04-06 08:33:17,670 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:33:17,671 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateStatusVDSCommand, log id: 6c3c9923 2015-04-06 08:33:17,685 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector ] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt3). 2015-04-06 08:33:17,776 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Running command: MigrateVmCommand internal: false. Entities affected : ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group MIGRATE_VM with role type USER, ID: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 8d432949-e03c-4950-a91a-160727f7bdf2 Type: VdsGroupsAction group CREATE_VM with role type USER 2015-04-06 08:33:17,825 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 5642c909 2015-04-06 08:33:17,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] START, MigrateBrokerVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af, srcHost=192.168.0.35, dstVdsId=e21cf0b9-a7e7-41e7-96d0-27006b42ad84, dstHost=192.168.0.34:54321, migrationMethod=ONLINE, tunnelMigration=false, migrationDowntime=0), log id: 1d4ead35 2015-04-06 08:33:17,833 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateBrokerVDSCommand, log id: 1d4ead35 2015-04-06 08:33:17,843 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71f97a52] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5642c909 2015-04-06 08:33:17,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector ] (org.ovirt.thread.pool-8-thread-38) [71f97a52] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: nindigo, Source: virt2, Destination: virt1, User: admin@internal). 2015-04-06 08:34:42,969 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] VM nindigo 9de649ca-c9a9-4ba7-bb2c-61c44e2819af moved from MigratingFrom --> Up 2015-04-06 08:34:42,970 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Adding VM 9de649ca-c9a9-4ba7-bb2c-61c44e2819af to re-run list 2015-04-06 08:34:42,983 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [2b07d5b3] Rerun vm 9de649ca-c9a9-4ba7-bb2c-61c44e2819af. Called from vds virt2 2015-04-06 08:34:43,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] START, MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af), log id: 7b89f823 2015-04-06 08:34:43,029 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Failed in MigrateStatusVDS method 2015-04-06 08:34:43,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]] 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] HostName = virt2 2015-04-06 08:34:43,030 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Command MigrateStatusVDSCommand(HostName = virt2, HostId = 1d1d1fbb-3067-4703-8b51-e0a231d344e6, vmId=9de649ca-c9a9-4ba7-bb2c-61c44e2819af) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration, code = 12 2015-04-06 08:34:43,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] FINISH, MigrateStatusVDSCommand, log id: 7b89f823 2015-04-06 08:34:43,049 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector ] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nindigo, Source: virt2, Destination: virt1). 2015-04-06 08:34:43,132 WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] CanDoAction of action MigrateVm failed for user admin@internal. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM ,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__V M,SCHEDULING_NO_HOSTS
2015-04-06 08:34:43,132 INFO [org.ovirt.engine.core.bll.MigrateVmCommand] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Lock freed to object EngineLock [exclusiveLocks= key: 9de649ca-c9a9-4ba7-bb2c-61c44e2819af value: VM , sharedLocks= ] 2015-04-06 08:34:43,165 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector ] (org.ovirt.thread.pool-8-thread-14) [2b07d5b3] Correlation ID: 3b191496, Job ID: 0f8c2d21-201e-454f-9876-dce9a1ca56fd, Call Stack: null, Custom Event ID: -1, Message: Migration failed, No available host found (VM: nindigo, Source: virt2). _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (3)
-
Artyom Lukianov -
Jason Keltz -
Xie, Chao