VMs stuck in migrating state

Monday, 26 February 2018

Hi,

We're running 4.1.9 and during the weekend we had a storage issue that 
seemed to leave some hosts in an strange state. One of the hosts has 
almost all VMs migrating (although it seems to not actually being 
migrating them) and the migration state cannot be cancelled.

When clicking on one of those machines and selecting 'Cancel migration', 
in the ovirt-engine log I see:

2018-02-26 08:52:07,588Z INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.CancelMigrateVDSCommand] 
(org.ovirt.thread.pool-6-thread-36) 
[887dfbf9-dece-4f7b-90a8-dac02b849b7f] HostName = host2.domain.com
2018-02-26 08:52:07,588Z ERROR 
[org.ovirt.engine.core.vdsbroker.vdsbroker.CancelMigrateVDSCommand] 
(org.ovirt.thread.pool-6-thread-36) 
[887dfbf9-dece-4f7b-90a8-dac02b849b7f] Command 
'CancelMigrateVDSCommand(HostName = host2.domain.com, 
CancelMigrationVDSParameters:{runAsync='true', 
hostId='e63b9146-10c4-47ad-bd6c-f053a8c5b4eb', 
vmId='26d37e43-32e2-4e55-9c1e-1438518d5021'})' execution failed: 
VDSGenericException: VDSErrorException: Failed to CancelMigrateVDS, 
error = Migration process cancelled, code = 82

On the vdsm side I see:

2018-02-26 08:56:19,396+0000 INFO  (jsonrpc/0) [vdsm.api] START 
migrateCancel() from=::ffff:10.X.X.X,54654, 
flow_id=874d36d7-63f5-4b71-8a4d-6d9f3ec65858 (api:46)
2018-02-26 08:56:19,398+0000 INFO  (jsonrpc/0) [vdsm.api] FINISH 
migrateCancel return={'status': {'message': 'Migration process 
cancelled', 'code': 82}, 'progress': 0} from=::ffff:10.X.X.X,54654, 
flow_id=874d36d7-63f5-4b71-8a4d-6d9f3ec65858 (api:52)

So no error on the vdsm side log.

I already tried restarting ovirt-engine but it didn't work.

Could someone shed some light on how to cancel the migration status for 
these machines? All of them seem to be running on the same host.

Thanks.

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011