[Users] VM Migration failed
Dafna Ron
dron at redhat.com
Sat Jan 18 12:05:58 UTC 2014
I looked at the logs log and only vdsm from node2 (which appears to be
the src node) seems to have full info.
Please attach the following complete logs:
vdsm log from dst node
engine log
libvirt logs from both nodes.
Also, can you please answer the following?
libvirt and vdsm you are using?
what is selinux status on both hosts (enforcing/permissive)?
do you have snapshots on the vm or does it happen on a newly created disk?
when you create the disk, is it from a template or is it a new image?
Thanks,
Dafna
On 01/17/2014 10:21 PM, Itamar Heim wrote:
> On 01/18/2014 12:07 AM, Meital Bourvine wrote:
>> I opened a bug about missing info in UI:
>> https://bugzilla.redhat.com/show_bug.cgi?id=1054994
>>
>> It actually failed with this error:
>> Thread-1417::DEBUG::2014-01-17 17:01:28,344::vm::768::vm.Vm::(run)
>> vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::starting migration
>> monitor thread
>> Thread-1415::DEBUG::2014-01-17
>> 17:01:28,409::libvirtconnection::108::libvirtconnection::(wrapper)
>> Unknown libvirterror: ecode: 38 edom: 42 level: 2 message: Failed to
>> inquire lock: No such process
>> Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::745::vm.Vm::(cancel)
>> vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::canceling migration
>> downtime thread
>> Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::815::vm.Vm::(stop)
>> vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::stopping migration
>> monitor thread
>> Thread-1416::DEBUG::2014-01-17 17:01:28,409::vm::742::vm.Vm::(run)
>> vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::migration downtime
>> thread exiting
>> Thread-1415::ERROR::2014-01-17
>> 17:01:28,410::vm::238::vm.Vm::(_recover)
>> vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to inquire lock:
>> No such process
>> Thread-1415::ERROR::2014-01-17 17:01:28,619::vm::337::vm.Vm::(run)
>> vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to migrate
>> Traceback (most recent call last):
>> File "/usr/share/vdsm/vm.py", line 323, in run
>> self._startUnderlyingMigration()
>> File "/usr/share/vdsm/vm.py", line 400, in _startUnderlyingMigration
>> None, maxBandwidth)
>> File "/usr/share/vdsm/vm.py", line 838, in f
>> ret = attr(*args, **kwargs)
>> File
>> "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line
>> 76, in wrapper
>> ret = f(*args, **kwargs)
>> File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1178,
>> in migrateToURI2
>> if ret == -1: raise libvirtError ('virDomainMigrateToURI2()
>> failed', dom=self)
>> libvirtError: Failed to inquire lock: No such process
>> Thread-26::ERROR::2014-01-17
>> 17:01:28,917::sampling::355::vm.Vm::(collect)
>> vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Stats function failed:
>> <AdvancedStatsFunction _highWrite at 0x26efb58>
>>
>> The problem is that it doesn't say which process...
>
> this looks like noise post the migration failing, with probably a more
> relevant error in libvirt log
>
>>
>> ----- Original Message -----
>>> From: "Itamar Heim" <iheim at redhat.com>
>>> To: "Edgars M." <edgars.mazurs at gmail.com>, "Meital Bourvine"
>>> <mbourvin at redhat.com>
>>> Cc: users at ovirt.org, "Michal Skrivanek" <mskrivan at redhat.com>
>>> Sent: Friday, January 17, 2014 9:47:11 PM
>>> Subject: Re: [Users] VM Migration failed
>>>
>>> On 01/17/2014 06:25 PM, Edgars M. wrote:
>>>> Hi Meital
>>>>
>>>> I tried to migrate another VM and it also failed.
>>>>
>>>> This is what I get in UI:
>>>>
>>>>
>>>> Migration started (VM: nophpapp01, Source: novmnode1, Destination:
>>>> novmnode2, User: edgarsm).
>>>> Migration failed due to Error: Fatal error during migration. Trying to
>>>> migrate to another Host (VM: nophpapp01, Source: novmnode1,
>>>> Destination:
>>>> novmnode2).
>>>> Migration failed due to Error: Fatal error during migration (VM:
>>>> nophpapp01, Source: novmnode1, Destination: novmnode2).
>>>>
>>>> There is nothing in /var/log/messages, neither on engine server nor
>>>> nodes.
>>>>
>>>>
>>>> See attachments for engine and vdsm logs. (I am migrating from vmnode1
>>>> to vmnode2)
>>>
>>> whatever the issue is here, can you please open a bug on trying to
>>> return more info to user on the migration error itself (for easier
>>> troubleshooting).
>>>
>>> thanks,
>>> Itamar
>>>
>>>>
>>>> Thanks for the help
>>>> Edgars
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Jan 17, 2014 at 4:46 PM, Meital Bourvine <mbourvin at redhat.com
>>>> <mailto:mbourvin at redhat.com>> wrote:
>>>>
>>>> Which error are you getting in the UI?
>>>> Can you please attach the full engine and vdsm logs?
>>>> Also, please check if there is a relevant error in
>>>> /var/log/messages
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> *From: *"Edgars M." <edgars.mazurs at gmail.com
>>>> <mailto:edgars.mazurs at gmail.com>>
>>>> *To: *users at ovirt.org <mailto:users at ovirt.org>
>>>> *Sent: *Friday, January 17, 2014 3:42:37 PM
>>>> *Subject: *[Users] VM Migration failed
>>>>
>>>>
>>>> Hi
>>>>
>>>> I am experiencing issues with manual VM migration. VM
>>>> fails to
>>>> migrate to other node in the same Cluster. Here are some
>>>> relevant engine.log entries:
>>>>
>>>> ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
>>>> (DefaultQuartzScheduler_Worker-73) Rerun vm
>>>> a31cfd62-26fc-4396-8a83-1aed68c7fd39. Called from vds
>>>> novmnode1
>>>>
>>>> ERROR
>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
>>>> (pool-6-thread-49) Failed in MigrateStatusVDS method
>>>>
>>>> ERROR
>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
>>>> (pool-6-thread-49) Error code migrateErr and error message
>>>> VDSGenericException: VDSErrorException: Failed to
>>>> MigrateStatusVDS, error = Fatal error during migration
>>>>
>>>> ERROR
>>>> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
>>>> (pool-6-thread-49) Command MigrateStatusVDS execution failed.
>>>> Exception: VDSErrorException: VDSGenericException:
>>>> VDSErrorException: Failed to MigrateStatusVDS, error = Fatal
>>>> error during migration
>>>>
>>>> Both Engine and Node are running CentOS 6.5 x64. oVirt Engine
>>>> Version: 3.3.2-1.el6. VDSM version 4.13.2-1. I restarted
>>>> engine
>>>> and vdsm, that did not help.
>>>>
>>>> Also in vdsm log file I see the following errors related
>>>> to the
>>>> same VM ID:
>>>>
>>>> Thread-27::ERROR::2014-01-17
>>>> 16:37:06,271::sampling::355::vm.Vm::(collect)
>>>> vmId=`a31cfd62-26fc-4396-8a83-1aed68c7fd39`::Stats function
>>>> failed: <AdvancedStatsFunction _highWrite at 0x26efb58>
>>>>
>>>> Any hints?
>>>>
>>>> BR
>>>> Edgars
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>
>>>
>>>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
--
Dafna Ron
More information about the Users
mailing list