Re: [Users] VM Migration failed

On 01/18/2014 12:07 AM, Meital Bourvine wrote: > I opened a bug about missing info in UI: > https://bugzilla.redhat.com/show_bug.cgi?id=1054994 > > It actually failed with this error: > Thread-1417::DEBUG::2014-01-17 17:01:28,344::vm::768::vm.Vm::(run) > vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::starting migration > monitor thread > Thread-1415::DEBUG::2014-01-17 > 17:01:28,409::libvirtconnection::108::libvirtconnection::(wrapper) > Unknown libvirterror: ecode: 38 edom: 42 level: 2 message: Failed to > inquire lock: No such process > Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::745::vm.Vm::(cancel) > vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::canceling migration > downtime thread > Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::815::vm.Vm::(stop) > vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::stopping migration > monitor thread > Thread-1416::DEBUG::2014-01-17 17:01:28,409::vm::742::vm.Vm::(run) > vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::migration downtime > thread exiting > Thread-1415::ERROR::2014-01-17 > 17:01:28,410::vm::238::vm.Vm::(_recover) > vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to inquire lock: > No such process > Thread-1415::ERROR::2014-01-17 17:01:28,619::vm::337::vm.Vm::(run) > vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to migrate > Traceback (most recent call last): > File "/usr/share/vdsm/vm.py", line 323, in run > self._startUnderlyingMigration() > File "/usr/share/vdsm/vm.py", line 400, in _startUnderlyingMigration > None, maxBandwidth) > File "/usr/share/vdsm/vm.py", line 838, in f > ret = attr(*args, **kwargs) > File > "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line > 76, in wrapper > ret = f(*args, **kwargs) > File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1178, > in migrateToURI2 > if ret == -1: raise libvirtError ('virDomainMigrateToURI2() > failed', dom=self) > libvirtError: Failed to inquire lock: No such process > Thread-26::ERROR::2014-01-17 > 17:01:28,917::sampling::355::vm.Vm::(collect) > vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Stats function failed: > <AdvancedStatsFunction _highWrite at 0x26efb58> > > The problem is that it doesn't say which process... this looks like noise post the migration failing, with probably a more relevant error in libvirt log > > ----- Original Message ----- >> From: "Itamar Heim" <iheim(a)redhat.com&gt; >> To: "Edgars M." <edgars.mazurs(a)gmail.com&gt;, "Meital Bourvine" >> <mbourvin(a)redhat.com&gt; >> Cc: users(a)ovirt.org, "Michal Skrivanek" <mskrivan(a)redhat.com&gt; >> Sent: Friday, January 17, 2014 9:47:11 PM >> Subject: Re: [Users] VM Migration failed >> >> On 01/17/2014 06:25 PM, Edgars M. wrote: >>> Hi Meital >>> >>> I tried to migrate another VM and it also failed. >>> >>> This is what I get in UI: >>> >>> >>> Migration started (VM: nophpapp01, Source: novmnode1, Destination: >>> novmnode2, User: edgarsm). >>> Migration failed due to Error: Fatal error during migration. Trying to >>> migrate to another Host (VM: nophpapp01, Source: novmnode1, >>> Destination: >>> novmnode2). >>> Migration failed due to Error: Fatal error during migration (VM: >>> nophpapp01, Source: novmnode1, Destination: novmnode2). >>> >>> There is nothing in /var/log/messages, neither on engine server nor >>> nodes. >>> >>> >>> See attachments for engine and vdsm logs. (I am migrating from vmnode1 >>> to vmnode2) >> >> whatever the issue is here, can you please open a bug on trying to >> return more info to user on the migration error itself (for easier >> troubleshooting). >> >> thanks, >> Itamar >> >>> >>> Thanks for the help >>> Edgars >>> >>> >>> >>> >>> >>> >>> >>> On Fri, Jan 17, 2014 at 4:46 PM, Meital Bourvine <mbourvin(a)redhat.com >>> <mailto:mbourvin@redhat.com>> wrote: >>> >>> Which error are you getting in the UI? >>> Can you please attach the full engine and vdsm logs? >>> Also, please check if there is a relevant error in >>> /var/log/messages >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >>> *From: *"Edgars M." <edgars.mazurs(a)gmail.com >>> <mailto:edgars.mazurs@gmail.com>> >>> *To: *users(a)ovirt.org <mailto:users@ovirt.org> >>> *Sent: *Friday, January 17, 2014 3:42:37 PM >>> *Subject: *[Users] VM Migration failed >>> >>> >>> Hi >>> >>> I am experiencing issues with manual VM migration. VM >>> fails to >>> migrate to other node in the same Cluster. Here are some >>> relevant engine.log entries: >>> >>> ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] >>> (DefaultQuartzScheduler_Worker-73) Rerun vm >>> a31cfd62-26fc-4396-8a83-1aed68c7fd39. Called from vds >>> novmnode1 >>> >>> ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] >>> (pool-6-thread-49) Failed in MigrateStatusVDS method >>> >>> ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] >>> (pool-6-thread-49) Error code migrateErr and error message >>> VDSGenericException: VDSErrorException: Failed to >>> MigrateStatusVDS, error = Fatal error during migration >>> >>> ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] >>> (pool-6-thread-49) Command MigrateStatusVDS execution failed. >>> Exception: VDSErrorException: VDSGenericException: >>> VDSErrorException: Failed to MigrateStatusVDS, error = Fatal >>> error during migration >>> >>> Both Engine and Node are running CentOS 6.5 x64. oVirt Engine >>> Version: 3.3.2-1.el6. VDSM version 4.13.2-1. I restarted >>> engine >>> and vdsm, that did not help. >>> >>> Also in vdsm log file I see the following errors related >>> to the >>> same VM ID: >>> >>> Thread-27::ERROR::2014-01-17 >>> 16:37:06,271::sampling::355::vm.Vm::(collect) >>> vmId=`a31cfd62-26fc-4396-8a83-1aed68c7fd39`::Stats function >>> failed: <AdvancedStatsFunction _highWrite at 0x26efb58> >>> >>> Any hints? >>> >>> BR >>> Edgars >>> >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org <mailto:Users@ovirt.org> >>> http://lists.ovirt.org/mailman/listinfo/users >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >>> >> >> >> _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users