Migration not working
Hi, I'm having issues with migration to some hosts. I have a 4 node cluster and I tried updating to see if it fixes but it's still failing. The reason is nos clear in the engine log. 2020-08-14 09:45:32,322-03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-429) [364e7777] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: medialist2-videoteca, Source: virt2.tnu.com.uy, Destination: virt3.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,336-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Lock Acquired to object 'EngineLock:{exclusiveLocks='[aac7468 5-c969-4761-923f-043400176edf=VM]', sharedLocks=''}' 2020-08-14 09:45:32,355-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Running command: MigrateVmToServerCommand internal: true. Ent ities affected : ID: aac74685-c969-4761-923f-043400176edf Type: VMAction group MIGRATE_VM with role type USER 2020-08-14 09:45:32,383-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId= '634f3f64-8945-470c-b31c-b8d4c73109e6', vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e441a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4. tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth= '125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action= {name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, pa rams=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.16.100.45'}), log id: 1db80717 2020-08-14 09:45:32,385-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] START, MigrateBrokerVDSCommand(HostName = virt 2.tnu.com.uy, MigrateVDSCommandParameters:{hostId='634f3f64-8945-470c-b31c-b8d4c73109e6', vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e4 41a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4.tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompre ssed='false', consoleAddress='null', maxBandwidth='125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDow ntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, para ms=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.1 6.100.45'}), log id: 4085b503 2020-08-14 09:45:32,433-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateBrokerVDSCommand, return: , log id: 4085b503 2020-08-14 09:45:32,437-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1db8 0717 2020-08-14 09:45:32,446-03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-429) [535a8bdb] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: reverse_proxy, Source: virt2.tnu.com.uy, Destination: virt4.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,462-03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-429) [535a8bdb] EVENT_ID: USER_VDS_MAINTENANCE_WITHOUT_REASON(620), Host virt2.tnu.com.uy was switched to Maintenance mode by jplorier@tnu.com.uy. 2020-08-14 09:45:32,681-03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [7667d149-886e-406b-9cd6-9b7472fa9944] Command 'MaintenanceNumberOfVdss' id: 'db97a747-115b-4f01-aa3d-6277080e6a44' child commands '[]' executions were completed, status 'SUCCEEDED' 2020-08-14 09:45:33,024-03 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] Received first domain report for host virt2.tnu.com.uy 2020-08-14 09:45:33,686-03 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [7667d149-886e-406b-9cd6-9b7472fa9944] Ending command 'org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand' successfully. 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) moved from 'MigratingFrom' --> 'Up' 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Adding VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) to re-run list 2020-08-14 09:45:35,144-03 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Rerun VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'. Called from VDS 'virt2.tnu.com.uy' *In the host running the vm, I see no error related to the migration, only about Engine VM which seems to be working fine otherwise:* 2020-08-14 09:06:23,337-0400 ERROR (jsonrpc/3) [root] failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? (api:19 *VDSM version in node:* vdsm.x86_64 4.30.46-1.el7 @ovirt-4.3 All nodes are up to date. Thanks
On Fri, Aug 14, 2020, 16:16 Juan Pablo Lorier <jplorier@gmail.com> wrote:
Hi,
I'm having issues with migration to some hosts. I have a 4 node cluster and I tried updating to see if it fixes but it's still failing. The reason is nos clear in the engine log.
2020-08-14 09:45:32,322-03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-429) [364e7777] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: medialist2-videoteca, Source: virt2.tnu.com.uy, Destination: virt3.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,336-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Lock Acquired to object 'EngineLock:{exclusiveLocks='[aac7468 5-c969-4761-923f-043400176edf=VM]', sharedLocks=''}' 2020-08-14 09:45:32,355-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Running command: MigrateVmToServerCommand internal: true. Ent ities affected : ID: aac74685-c969-4761-923f-043400176edf Type: VMAction group MIGRATE_VM with role type USER 2020-08-14 09:45:32,383-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId= '634f3f64-8945-470c-b31c-b8d4c73109e6', vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e441a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4. tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth= '125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action= {name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, pa rams=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.16.100.45'}), log id: 1db80717 2020-08-14 09:45:32,385-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] START, MigrateBrokerVDSCommand(HostName = virt 2.tnu.com.uy, MigrateVDSCommandParameters:{hostId='634f3f64-8945-470c-b31c-b8d4c73109e6', vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e4 41a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4.tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompre ssed='false', consoleAddress='null', maxBandwidth='125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDow ntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, para ms=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.1 6.100.45'}), log id: 4085b503 2020-08-14 09:45:32,433-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateBrokerVDSCommand, return: , log id: 4085b503 2020-08-14 09:45:32,437-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1db8 0717 2020-08-14 09:45:32,446-03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-429) [535a8bdb] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: reverse_proxy, Source: virt2.tnu.com.uy, Destination: virt4.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,462-03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-429) [535a8bdb] EVENT_ID: USER_VDS_MAINTENANCE_WITHOUT_REASON(620), Host virt2.tnu.com.uy was switched to Maintenance mode by jplorier@tnu.com.uy. 2020-08-14 09:45:32,681-03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [7667d149-886e-406b-9cd6-9b7472fa9944] Command 'MaintenanceNumberOfVdss' id: 'db97a747-115b-4f01-aa3d-6277080e6a44' child commands '[]' executions were completed, status 'SUCCEEDED' 2020-08-14 09:45:33,024-03 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] Received first domain report for host virt2.tnu.com.uy 2020-08-14 09:45:33,686-03 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [7667d149-886e-406b-9cd6-9b7472fa9944] Ending command 'org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand' successfully. 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) moved from 'MigratingFrom' --> 'Up' 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Adding VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) to re-run list 2020-08-14 09:45:35,144-03 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Rerun VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'. Called from VDS 'virt2.tnu.com.uy'
*In the host running the vm, I see no error related to the migration, only about Engine VM which seems to be working fine otherwise:*
2020-08-14 09:06:23,337-0400 ERROR (jsonrpc/3) [root] failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? (api:19
I would check also vdsm logs at source and destination host.
*VDSM version in node:*
vdsm.x86_64 4.30.46-1.el7 @ovirt-4.3
All nodes are up to date.
Thanks
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4IQCJE363EEJP7...
What is the output of 'hosted-engine --vm-status' on all nodes that are supposed to host the HostedEngine ? Usually issues there can be debugged , when you check the ovirt-ha-broker and ovirt-ha-agent logs (in /var/log on the affected Host). Best Regards, Strahil Nikolov На 15 август 2020 г. 9:29:20 GMT+03:00, Alex K <rightkicktech@gmail.com> написа:
On Fri, Aug 14, 2020, 16:16 Juan Pablo Lorier <jplorier@gmail.com> wrote:
Hi,
I'm having issues with migration to some hosts. I have a 4 node cluster and I tried updating to see if it fixes but it's still failing. The reason is nos clear in the engine log.
2020-08-14 09:45:32,322-03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-429) [364e7777] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: medialist2-videoteca, Source: virt2.tnu.com.uy, Destination: virt3.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,336-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Lock Acquired to object 'EngineLock:{exclusiveLocks='[aac7468 5-c969-4761-923f-043400176edf=VM]', sharedLocks=''}' 2020-08-14 09:45:32,355-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Running command: MigrateVmToServerCommand internal: true. Ent ities affected : ID: aac74685-c969-4761-923f-043400176edf Type: VMAction group MIGRATE_VM with role type USER 2020-08-14 09:45:32,383-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId= '634f3f64-8945-470c-b31c-b8d4c73109e6', vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e441a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4. tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth= '125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action= {name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, pa rams=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.16.100.45'}), log id: 1db80717 2020-08-14 09:45:32,385-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] START, MigrateBrokerVDSCommand(HostName = virt 2.tnu.com.uy,
MigrateVDSCommandParameters:{hostId='634f3f64-8945-470c-b31c-b8d4c73109e6',
vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e4 41a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4.tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompre ssed='false', consoleAddress='null', maxBandwidth='125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDow ntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, para ms=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.1 6.100.45'}), log id: 4085b503 2020-08-14 09:45:32,433-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateBrokerVDSCommand, return: , log id: 4085b503 2020-08-14 09:45:32,437-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1db8 0717 2020-08-14 09:45:32,446-03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-429) [535a8bdb] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: reverse_proxy, Source: virt2.tnu.com.uy, Destination: virt4.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,462-03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-429) [535a8bdb] EVENT_ID: USER_VDS_MAINTENANCE_WITHOUT_REASON(620), Host virt2.tnu.com.uy was switched to Maintenance mode by jplorier@tnu.com.uy. 2020-08-14 09:45:32,681-03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [7667d149-886e-406b-9cd6-9b7472fa9944] Command 'MaintenanceNumberOfVdss' id: 'db97a747-115b-4f01-aa3d-6277080e6a44' child commands '[]' executions were completed, status 'SUCCEEDED' 2020-08-14 09:45:33,024-03 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] Received first domain report for host virt2.tnu.com.uy 2020-08-14 09:45:33,686-03 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [7667d149-886e-406b-9cd6-9b7472fa9944] Ending command 'org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand' successfully. 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) moved from 'MigratingFrom' --> 'Up' 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Adding VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) to re-run list 2020-08-14 09:45:35,144-03 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Rerun VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'. Called from VDS 'virt2.tnu.com.uy'
*In the host running the vm, I see no error related to the migration, only about Engine VM which seems to be working fine otherwise:*
2020-08-14 09:06:23,337-0400 ERROR (jsonrpc/3) [root] failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? (api:19
I would check also vdsm logs at source and destination host.
*VDSM version in node:*
vdsm.x86_64 4.30.46-1.el7 @ovirt-4.3
All nodes are up to date.
Thanks
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/4IQCJE363EEJP7...
Hi, It seems to be fixed now. I updated all the nodes to the last version as well as hosted engine. I'll see to look at VDSM logs if it happens again. Regards El 16/8/20 a las 02:44, Strahil Nikolov escribió:
What is the output of 'hosted-engine --vm-status' on all nodes that are supposed to host the HostedEngine ?
Usually issues there can be debugged , when you check the ovirt-ha-broker and ovirt-ha-agent logs (in /var/log on the affected Host).
Best Regards, Strahil Nikolov
На 15 август 2020 г. 9:29:20 GMT+03:00, Alex K <rightkicktech@gmail.com> написа:
On Fri, Aug 14, 2020, 16:16 Juan Pablo Lorier <jplorier@gmail.com> wrote:
Hi,
I'm having issues with migration to some hosts. I have a 4 node cluster and I tried updating to see if it fixes but it's still failing. The reason is nos clear in the engine log.
2020-08-14 09:45:32,322-03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-429) [364e7777] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: medialist2-videoteca, Source: virt2.tnu.com.uy, Destination: virt3.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,336-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Lock Acquired to object 'EngineLock:{exclusiveLocks='[aac7468 5-c969-4761-923f-043400176edf=VM]', sharedLocks=''}' 2020-08-14 09:45:32,355-03 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-429) [535a8bdb] Running command: MigrateVmToServerCommand internal: true. Ent ities affected : ID: aac74685-c969-4761-923f-043400176edf Type: VMAction group MIGRATE_VM with role type USER 2020-08-14 09:45:32,383-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId= '634f3f64-8945-470c-b31c-b8d4c73109e6', vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e441a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4. tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth= '125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action= {name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, pa rams=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.16.100.45'}), log id: 1db80717 2020-08-14 09:45:32,385-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] START, MigrateBrokerVDSCommand(HostName = virt 2.tnu.com.uy,
MigrateVDSCommandParameters:{hostId='634f3f64-8945-470c-b31c-b8d4c73109e6',
vmId='aac74685-c969-4761-923f-043400176edf', srcHost='virt2.tnu.com.uy', dstVdsId='f9e4 41a0-a4e6-4548-8c6f-83a958120f02', dstHost='virt4.tnu.com.uy:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompre ssed='false', consoleAddress='null', maxBandwidth='125', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDow ntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, para ms=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='172.1 6.100.45'}), log id: 4085b503 2020-08-14 09:45:32,433-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateBrokerVDSCommand, return: , log id: 4085b503 2020-08-14 09:45:32,437-03 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-429) [535a8bdb] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1db8 0717 2020-08-14 09:45:32,446-03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-429) [535a8bdb] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIAT ED(67), Migration initiated by system (VM: reverse_proxy, Source: virt2.tnu.com.uy, Destination: virt4.tnu.com.uy, Reason: ). 2020-08-14 09:45:32,462-03 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-429) [535a8bdb] EVENT_ID: USER_VDS_MAINTENANCE_WITHOUT_REASON(620), Host virt2.tnu.com.uy was switched to Maintenance mode by jplorier@tnu.com.uy. 2020-08-14 09:45:32,681-03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [7667d149-886e-406b-9cd6-9b7472fa9944] Command 'MaintenanceNumberOfVdss' id: 'db97a747-115b-4f01-aa3d-6277080e6a44' child commands '[]' executions were completed, status 'SUCCEEDED' 2020-08-14 09:45:33,024-03 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [] Received first domain report for host virt2.tnu.com.uy 2020-08-14 09:45:33,686-03 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [7667d149-886e-406b-9cd6-9b7472fa9944] Ending command 'org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand' successfully. 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) moved from 'MigratingFrom' --> 'Up' 2020-08-14 09:45:35,137-03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Adding VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'(medialist2-videoteca) to re-run list 2020-08-14 09:45:35,144-03 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Rerun VM 'fd854ebe-144f-455f-bf0c-48167d7aae9f'. Called from VDS 'virt2.tnu.com.uy'
*In the host running the vm, I see no error related to the migration, only about Engine VM which seems to be working fine otherwise:*
2020-08-14 09:06:23,337-0400 ERROR (jsonrpc/3) [root] failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? (api:19
I would check also vdsm logs at source and destination host.
*VDSM version in node:*
vdsm.x86_64 4.30.46-1.el7 @ovirt-4.3
All nodes are up to date.
Thanks
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/4IQCJE363EEJP7...
participants (3)
-
Alex K -
Juan Pablo Lorier -
Strahil Nikolov