Re: [Users] Problem with live migration.

Wednesday, 18 April 2012

This is a multi-part message in MIME format.
--------------020800060704090308040504
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

On 04/18/2012 11:30 AM, Martin Hovmöller wrote:
...

 On Tue, Apr 17, 2012 at 4:38 PM, Rami Vaknin <rvaknin(a)redhat.com 
 <mailto:rvaknin@redhat.com>> wrote:

     On 04/17/2012 05:36 PM, Martin Hovmöller wrote:
>
>
>     On Tue, Apr 17, 2012 at 2:55 PM, Rami Vaknin <rvaknin(a)redhat.com
>     <mailto:rvaknin@redhat.com>> wrote:
>
>         On 04/17/2012 03:45 PM, Martin Hovmöller wrote:
>>
>>
>>         On Tue, Apr 17, 2012 at 2:11 PM, Itamar Heim
>>         <iheim(a)redhat.com <mailto:iheim@redhat.com>> wrote:
>>
>>             On 04/17/2012 12:47 PM, Martin Hovmöller wrote:
>>
>>                 I can't live migrate the vm's in my cluster:
>>
>>                 012-04-17 04:06:39,728 INFO
>>                  [org.ovirt.engine.core.bll.MigrateVmCommand]
>>                 (pool-5-thread-48)
>>                 Running command: MigrateVmCommand internal: false.
>>                 Entities affected :
>>                  ID: ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a Type: VM
>>                 2012-04-17 04:06:39,743 INFO
>>                  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
>>                 (pool-5-thread-48)
>>                 START, MigrateVDSCommand(vdsId =
>>                 f33ffc14-87ba-11e1-b610-e3aeca1e8008,
>>                 vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
>>                 srcHost=10.23.30.130,
>>                 dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,
>>                 dstHost=10.23.30.110:54321
>>                 <http://10.23.30.110:54321>
>>                 <http://10.23.30.110:54321>;,
>>
>>                 migrationMethod=ONLINE), log id: 7c7201be
>>                 2012-04-17 04:06:39,748 INFO
>>                 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
>>                 (pool-5-thread-48) VdsBroker::migrate::Entered
>>                 (vm_guid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
>>                 srcHost=10.23.30.130,
>>                 dstHost=10.23.30.110:54321
>>                 <http://10.23.30.110:54321>
>>                 <http://10.23.30.110:54321>;,  method=online
>>
>>                 2012-04-17 04:06:39,749 INFO
>>                 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
>>                 (pool-5-thread-48) START,
>>                 MigrateBrokerVDSCommand(vdsId =
>>                 f33ffc14-87ba-11e1-b610-e3aeca1e8008,
>>                 vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
>>                 srcHost=10.23.30.130,
>>                 dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,
>>                 dstHost=10.23.30.110:54321
>>                 <http://10.23.30.110:54321>
>>                 <http://10.23.30.110:54321>;,
>>
>>                 migrationMethod=ONLINE), log id: 3c40c2de
>>                 2012-04-17 04:06:39,834 INFO
>>                 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
>>                 (pool-5-thread-48) FINISH, MigrateBrokerVDSCommand,
>>                 log id: 3c40c2de
>>                 2012-04-17 04:06:39,840 INFO
>>                  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
>>                 (pool-5-thread-48)
>>                 FINISH, MigrateVDSCommand, return: MigratingFrom,
>>                 log id: 7c7201be
>>                 2012-04-17 04:06:42,166 INFO
>>                  [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
>>                 (QuartzScheduler_Worker-18) vds::refreshVmList vm id
>>                 ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a is migrating to
>>                 vds rhevh1.<domain>
>>                 ignoring it in the refresh till migration is done
>>                 [...]
>>                 2012-04-17 04:07:33,852 INFO
>>                  [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
>>                 (QuartzScheduler_Worker-34) vds::refreshVmList vm id
>>                 ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a status = Paused
>>                 on vds
>>                 rhevh1.<domain> ignoring it in the refresh till
>>                 migration is done
>>                 2012-04-17 04:07:37,407 ERROR
>>                 [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
>>                 (QuartzScheduler_Worker-71) Rerun vm
>>                 ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a. Called from
>>                 vds rhevh2.<domain>
>>                 2012-04-17 04:07:37,414 INFO
>>                 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
>>                 (pool-5-thread-47) START,
>>                 MigrateStatusVDSCommand(vdsId =
>>                 f33ffc14-87ba-11e1-b610-e3aeca1e8008,
>>                 vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a), log id:
>>                 8dfce1b
>>                 2012-04-17 04:07:37,496 ERROR
>>                 [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>>                 (pool-5-thread-47) Failed in MigrateStatusVDS method
>>                 2012-04-17 04:07:37,497 ERROR
>>                 [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>>                 (pool-5-thread-47) Error code createErr and error
>>                 message
>>                 VDSGenericException: VDSErrorException: Failed to
>>                 MigrateStatusVDS,
>>                 error = Error creating the requested virtual machine
>>                 2012-04-17 04:07:37,497 INFO
>>                  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>>                 (pool-5-thread-47) Command
>>                
org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand
>>                 return
>>                 value
>>                  Class Name:
>>                
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>>                 mStatus                       Class Name:
>>                 org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>>                 mCode                         9
>>                 mMessage                      Error creating the
>>                 requested virtual machine
>>
>>
>>                 2012-04-17 04:07:37,497 INFO
>>                  [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>>                 (pool-5-thread-47) Vds: rhevh2.<domain>
>>                 2012-04-17 04:07:37,497 ERROR
>>                 [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
>>                 (pool-5-thread-47)
>>                 Command MigrateStatusVDS execution failed.
>>                 Exception: VDSErrorException:
>>                 VDSGenericException: VDSErrorException: Failed to
>>                 MigrateStatusVDS,
>>                 error = Error creating the requested virtual machine
>>                 2012-04-17 04:07:37,497 INFO
>>                 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
>>                 (pool-5-thread-47) FINISH, MigrateStatusVDSCommand,
>>                 log id: 8dfce1b
>>                 2012-04-17 04:07:37,528 WARN
>>                  [org.ovirt.engine.core.bll.MigrateVmCommand]
>>                 (pool-5-thread-47)
>>                 CanDoAction of action MigrateVm failed.
>>                
Reasons:ACTION_TYPE_FAILED_VDS_VM_CLUSTER,VAR__ACTION__MIGRATE,VAR__TYPE__VM
>>
>>
>>                 There is no problem regarding storage or anything
>>                 like that. If I shut
>>                 down the vm I can start it on the other host without
>>                 any problems
>>                 whatsoever.
>>                 Do I need to do something to make live migration
>>                 work? Trying to dig in
>>                 the logs on the hypervisors, but there's so much
>>                 stuff being logged there...
>>
>>
>>                 _______________________________________________
>>                 Users mailing list
>>                 Users(a)ovirt.org <mailto:Users@ovirt.org>
>>                 http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>             vdsm log from the host that failed to create the VM?
>>
>>
>>         The vdsm log is frighteningly long, so I put it in pastebin:
>>         http://pastebin.com/BEeqFCCb
>>         I don't really see anything worthwhile myself, but on the
>>         other hand I'm not sure what I'm looking for, so it's mostly
>>         a massive wall of text to me, but it should be somewhere in
>>         that time frame.
>         Here is the failure, "code 9" is not too informative, can you
>         attach the libvirtd.log file too?
>
>         Thread-49287::DEBUG::2012-04-17
>         14:25:39,834::clientIF::81::vds::(wrapper) return
>         migrationCreate with {'status': {'message': 'Error
creating
>         the requested virtual machine', 'code': 9}}
>
>>
>>
>>         _______________________________________________
>>         Users mailing list
>>         Users(a)ovirt.org  <mailto:Users@ovirt.org>
>>         http://lists.ovirt.org/mailman/listinfo/users
>
>
>         -- 
>
>         Thanks,
>
>         Rami Vaknin, QE @ Red Hat, TLV, IL.
>
>
>     I think this is the relevant info from the libvirt.log:
>     2012-04-17 14:30:53.768+0000: 8506: debug :
>     virDomainLookupByUUIDString:1984 : conn=0x7fa9a00bcde0,
>     uuidstr=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
>     2012-04-17 14:30:53.768+0000: 8506: debug :
>     virDomainLookupByUUID:1941 : conn=0x7fa9a00bcde0,
>     uuid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
>     2012-04-17 14:30:53.771+0000: 8506: error :
>     virNetClientProgramDispatchError:170 : Domain not found: no
>     domain with matching uuid 'ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a'
>
>     What domain is this, is it the storage domain?

     Domain in libvirt is simply a vm.

>
>
>     _______________________________________________
>     Users mailing list
>     Users(a)ovirt.org  <mailto:Users@ovirt.org>
>     http://lists.ovirt.org/mailman/listinfo/users

     -- 

     Thanks,

     Rami Vaknin, QE @ Red Hat, TLV, IL.

 Alright, so what does this mean then? The machine I'm migrating to 
 can't see the vm?
 That doesn't make sense since I can manually start it on that host.. 
On second look at the vdsm.log I see that disk preparation in the 
destination host failed (timed out), try to take a look at vdsm.log at 
the destination.
In addition, for some reason getVmStats returns "Virtual machine does 
not exist" right before the migration.

Thread-49286::DEBUG::2012-04-17 
14:25:07,054::clientIF::76::vds::(wrapper) [10.23.30.130]::call 
getVmStats with ('ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a',) {}
Thread-49286::DEBUG::2012-04-17 
14:25:07,206::clientIF::81::vds::(wrapper) return getVmStats with 
{'status': {'message': 'Virtual machine does not exist',
'code': 1}}
Thread-49287::DEBUG::2012-04-17 
14:25:07,381::clientIF::76::vds::(wrapper) [10.23.30.130]::call 
migrationCreate with ({'bridge': 'ovirtmgmt', 'acpiEnable':
'true',.....
Thread-49287::DEBUG::2012-04-17 
14:25:08,156::libvirtvm::1504::vm.Vm::(waitForMigrationDestinationPrepare) 
vmId=`ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a`::migration destination: 
waiting 31s for path preparation

Thread-49287::DEBUG::2012-04-17 
14:25:39,458::libvirtvm::1507::vm.Vm::(waitForMigrationDestinationPrepare) 
vmId=`ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a`::Timeout while waiting for 
path preparation

14:25:07,054::clientIF::76::vds::(wrapper) [10.23.30.130]::call 
getVmStats with ('ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a',) {}
Thread-49286::DEBUG::2012-04-17 
14:25:07,206::clientIF::81::vds::(wrapper) return getVmStats with 
{'status': {'message': 'Virtual machine does not exist',
'code': 1}}
...

 _______________________________________________
 Users mailing list
 Users(a)ovirt.org
 http://lists.ovirt.org/mailman/listinfo/users 

-- 

Thanks,

Rami Vaknin, QE @ Red Hat, TLV, IL.

--------------020800060704090308040504
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    On 04/18/2012 11:30 AM, Martin Hovm&ouml;ller wrote:
    <blockquote
cite="mid:CAGKvcKXco_=isKREAGyq+CTV_-=xVUruBHfBHDmkREJLc-W+7w@mail.gmail.com"
      type="cite"><br>
      <br>
      <div class="gmail_quote">On Tue, Apr 17, 2012 at 4:38 PM, Rami
        Vaknin <span dir="ltr">&lt;<a
moz-do-not-send="true"

href="mailto:rvaknin@redhat.com">rvaknin@redhat.com</a>&gt;</span>
        wrote:<br>
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div bgcolor="#FFFFFF" text="#000000">
            <div>
              <div class="h5"> On 04/17/2012 05:36 PM, Martin
Hovm&ouml;ller
                wrote:
                <blockquote type="cite"><br>
                  <br>
                  <div class="gmail_quote">On Tue, Apr 17, 2012 at 2:55
                    PM, Rami Vaknin <span dir="ltr">&lt;<a
                        moz-do-not-send="true"
                        href="mailto:rvaknin@redhat.com"
target=&quot;_blank&quot;&gt;rvaknin(a)redhat.com&lt;/a&gt;&amp;gt;&lt;/span&gt;
                    wrote:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0
                      .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      <div bgcolor="#FFFFFF" text="#000000">
                        <div>
                          <div> On 04/17/2012 03:45 PM, Martin Hovm&ouml;ller
                            wrote:
                            <blockquote type="cite"><br>
                              <br>
                              <div class="gmail_quote">On Tue, Apr 17,
                                2012 at 2:11 PM, Itamar Heim <span
                                  dir="ltr">&lt;<a
                                    moz-do-not-send="true"
                                    href="mailto:iheim@redhat.com"

target=&quot;_blank&quot;&gt;iheim(a)redhat.com&lt;/a&gt;&amp;gt;&lt;/span&gt;
                                wrote:<br>
                                <blockquote class="gmail_quote"
                                  style="margin:0 0 0
                                  .8ex;border-left:1px #ccc
                                  solid;padding-left:1ex">
                                  <div>On 04/17/2012 12:47 PM, Martin
                                    Hovm&ouml;ller wrote:<br>
                                  </div>
                                  <blockquote class="gmail_quote"
                                    style="margin:0 0 0
                                    .8ex;border-left:1px #ccc
                                    solid;padding-left:1ex">
                                    <div> I can't live migrate the vm's
                                      in my cluster:<br>
                                      <br>
                                      012-04-17 04:06:39,728 INFO<br>

&nbsp;[org.ovirt.engine.core.bll.MigrateVmCommand]

                                      (pool-5-thread-48)<br>
                                      Running command: MigrateVmCommand
                                      internal: false. Entities affected
                                      :<br>
                                      &nbsp;ID:
                                      ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
                                      Type: VM<br>
                                      2012-04-17 04:06:39,743 INFO<br>

&nbsp;[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]

                                      (pool-5-thread-48)<br>
                                      START, MigrateVDSCommand(vdsId =
                                      f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
                                      vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,

                                      srcHost=10.23.30.130,<br>
dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,<br>
                                    </div>
                                    dstHost=<a moz-do-not-send="true"
                                      href="http://10.23.30.110:54321"

target="_blank">10.23.30.110:54321</a>
                                    &lt;<a moz-do-not-send="true"
                                      href="http://10.23.30.110:54321"

target="_blank">http://10.23.30.110:54321</a>&gt;,

                                    <div><br>
                                      migrationMethod=ONLINE), log id:
                                      7c7201be<br>
                                      2012-04-17 04:06:39,748 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
                                      (pool-5-thread-48)
                                      VdsBroker::migrate::Entered<br>
                                      (vm_guid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,

                                      srcHost=10.23.30.130,<br>
                                    </div>
                                    dstHost=<a moz-do-not-send="true"
                                      href="http://10.23.30.110:54321"

target="_blank">10.23.30.110:54321</a>
                                    &lt;<a moz-do-not-send="true"
                                      href="http://10.23.30.110:54321"

target="_blank">http://10.23.30.110:54321</a>&gt;,

                                    &nbsp;method=online
                                    <div><br>
                                      2012-04-17 04:06:39,749 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
                                      (pool-5-thread-48) START,
                                      MigrateBrokerVDSCommand(vdsId =<br>
f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
                                      vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,

                                      srcHost=10.23.30.130,<br>
dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,<br>
                                    </div>
                                    dstHost=<a moz-do-not-send="true"
                                      href="http://10.23.30.110:54321"

target="_blank">10.23.30.110:54321</a>
                                    &lt;<a moz-do-not-send="true"
                                      href="http://10.23.30.110:54321"

target="_blank">http://10.23.30.110:54321</a>&gt;,

                                    <div>
                                      <div><br>
                                        migrationMethod=ONLINE), log id:
                                        3c40c2de<br>
                                        2012-04-17 04:06:39,834 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
                                        (pool-5-thread-48) FINISH,
                                        MigrateBrokerVDSCommand, log id:
                                        3c40c2de<br>
                                        2012-04-17 04:06:39,840 INFO<br>

&nbsp;[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]

                                        (pool-5-thread-48)<br>
                                        FINISH, MigrateVDSCommand,
                                        return: MigratingFrom, log id:
                                        7c7201be<br>
                                        2012-04-17 04:06:42,166 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
                                        (QuartzScheduler_Worker-18)
                                        vds::refreshVmList vm id<br>
                                        ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
                                        is migrating to vds
                                        rhevh1.&lt;domain&gt;<br>
                                        ignoring it in the refresh till
                                        migration is done<br>
                                        [...]<br>
                                        2012-04-17 04:07:33,852 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
                                        (QuartzScheduler_Worker-34)
                                        vds::refreshVmList vm id<br>
                                        ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
                                        status = Paused on vds<br>
                                        rhevh1.&lt;domain&gt; ignoring
                                        it in the refresh till migration
                                        is done<br>
                                        2012-04-17 04:07:37,407 ERROR<br>
[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
                                        (QuartzScheduler_Worker-71)
                                        Rerun vm<br>
                                        ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a.
                                        Called from vds
                                        rhevh2.&lt;domain&gt;<br>
                                        2012-04-17 04:07:37,414 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]<br>
                                        (pool-5-thread-47) START,
                                        MigrateStatusVDSCommand(vdsId =<br>
f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
                                        vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a),

                                        log id: 8dfce1b<br>
                                        2012-04-17 04:07:37,496 ERROR<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                                        (pool-5-thread-47) Failed in
                                        MigrateStatusVDS method<br>
                                        2012-04-17 04:07:37,497 ERROR<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                                        (pool-5-thread-47) Error code
                                        createErr and error message<br>
                                        VDSGenericException:
                                        VDSErrorException: Failed to
                                        MigrateStatusVDS,<br>
                                        error = Error creating the
                                        requested virtual machine<br>
                                        2012-04-17 04:07:37,497 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                                        (pool-5-thread-47) Command<br>

org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand

                                        return<br>
                                        value<br>
                                        &nbsp;Class Name:<br>
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc<br>
                                        mStatus &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                        Class Name:<br>
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc<br>
                                        mCode &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
9<br>
                                        mMessage &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
                                        &nbsp;Error creating the requested
                                        virtual machine<br>
                                        <br>
                                        <br>
                                        2012-04-17 04:07:37,497 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                                        (pool-5-thread-47) Vds:
                                        rhevh2.&lt;domain&gt;<br>
                                        2012-04-17 04:07:37,497 ERROR<br>
                                        [org.ovirt.engine.core.vdsbroker.VDSCommandBase]

                                        (pool-5-thread-47)<br>
                                        Command MigrateStatusVDS
                                        execution failed. Exception:
                                        VDSErrorException:<br>
                                        VDSGenericException:
                                        VDSErrorException: Failed to
                                        MigrateStatusVDS,<br>
                                        error = Error creating the
                                        requested virtual machine<br>
                                        2012-04-17 04:07:37,497 INFO<br>
&nbsp;[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]<br>
                                        (pool-5-thread-47) FINISH,
                                        MigrateStatusVDSCommand, log id:
                                        8dfce1b<br>
                                        2012-04-17 04:07:37,528 WARN<br>

&nbsp;[org.ovirt.engine.core.bll.MigrateVmCommand]

                                        (pool-5-thread-47)<br>
                                        CanDoAction of action MigrateVm
                                        failed.<br>
Reasons:ACTION_TYPE_FAILED_VDS_VM_CLUSTER,VAR__ACTION__MIGRATE,VAR__TYPE__VM<br>
                                        <br>
                                        <br>
                                        There is no problem regarding
                                        storage or anything like that.
                                        If I shut<br>
                                        down the vm I can start it on
                                        the other host without any
                                        problems<br>
                                        whatsoever.<br>
                                        Do I need to do something to
                                        make live migration work? Trying
                                        to dig in<br>
                                        the logs on the hypervisors, but
                                        there's so much stuff being
                                        logged there...<br>
                                        <br>
                                        <br>
                                      </div>
                                    </div>
_______________________________________________<br>
                                    Users mailing list<br>
                                    <a moz-do-not-send="true"
                                      href="mailto:Users@ovirt.org"

target=&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt;&lt;br&gt;
                                    <a moz-do-not-send="true"

href="http://lists.ovirt.org/mailman/listinfo/users"

target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
                                  </blockquote>
                                  <br>
                                  vdsm log from the host that failed to
                                  create the VM?<br>
                                </blockquote>
                              </div>
                              <br>
                              <div>The vdsm log is&nbsp;frighteningly long,
                                so I put it in pastebin:&nbsp;<a
                                  moz-do-not-send="true"
                                  href="http://pastebin.com/BEeqFCCb"

target="_blank">http://pastebin.com/BEeqFCCb</a></d...
                              <div>I don't really see anything
                                worthwhile myself, but on the other hand
                                I'm not sure what I'm looking for, so
                                it's mostly a massive wall of text to
                                me, but it should be somewhere in that
                                time frame.</div>
                            </blockquote>
                          </div>
                        </div>
                        Here is the failure, "code 9" is not too
                        informative, can you attach the libvirtd.log
                        file too?<br>
                        <br>
                        Thread-49287::DEBUG::2012-04-17
                        14:25:39,834::clientIF::81::vds::(wrapper)
                        return migrationCreate with {'status':
                        {'message': 'Error creating the requested
                        virtual machine', 'code': 9}}
                        <div><br>
                          <blockquote type="cite"> <br>
                            <fieldset></fieldset>
                            <br>
                            <pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" href="mailto:Users@ovirt.org"
target=&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt;
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
</pre>
                          </blockquote>
                          <br>
                          <br>
                        </div>
                        <span><font color="#888888">
                            <pre cols="72">-- 

Thanks,

Rami Vaknin, QE @ Red Hat, TLV, IL.
</pre>
                          </font></span></div>
                    </blockquote>
                  </div>
                  <div><br>
                  </div>
                  <div>I think this is the relevant info from the
                    libvirt.log:</div>
                  <div>
                    <div>2012-04-17 14:30:53.768+0000: 8506: debug :
                      virDomainLookupByUUIDString:1984 :
                      conn=0x7fa9a00bcde0,
                      uuidstr=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a</div>
                    <div>2012-04-17 14:30:53.768+0000: 8506: debug :
                      virDomainLookupByUUID:1941 : conn=0x7fa9a00bcde0,
                      uuid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a</div>
                    <div>2012-04-17 14:30:53.771+0000: 8506: error :
                      virNetClientProgramDispatchError:170 : Domain not
                      found: no domain with matching uuid
                      'ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a'</div>
                  </div>
                  <div><br>
                  </div>
                  <div>What domain is this, is it the storage domain?</div>
                </blockquote>
                <br>
              </div>
            </div>
            Domain in libvirt is simply a vm.
            <div class="im"><br>
              <blockquote type="cite"> <br>
                <fieldset></fieldset>
                <br>
                <pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" href="mailto:Users@ovirt.org"
target=&quot;_blank&quot;&gt;Users(a)ovirt.org&lt;/a&gt;
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
</pre>
              </blockquote>
              <br>
              <br>
              <pre cols="72">-- 

Thanks,

Rami Vaknin, QE @ Red Hat, TLV, IL.
</pre>
            </div>
          </div>
        </blockquote>
      </div>
      <br>
      <div>Alright, so what does this mean then? The machine I'm
        migrating to can't see the vm?<br>
        That doesn't make sense since I can manually start it on that
        host..</div>
    </blockquote>
    <br>
    On second look at the vdsm.log I see that disk preparation in the
    destination host failed (timed out), try to take a look at vdsm.log
    at the destination.<br>
    In addition, for some reason getVmStats returns "Virtual machine
    does not exist" right before the migration.<br>
    <br>
    <br>
    Thread-49286::DEBUG::2012-04-17
    14:25:07,054::clientIF::76::vds::(wrapper) [10.23.30.130]::call
    getVmStats with ('ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a',) {}<br>
    Thread-49286::DEBUG::2012-04-17
    14:25:07,206::clientIF::81::vds::(wrapper) return getVmStats with
    {'status': {'message': 'Virtual machine does not exist',
'code': 1}}<br>
    Thread-49287::DEBUG::2012-04-17
    14:25:07,381::clientIF::76::vds::(wrapper) [10.23.30.130]::call
    migrationCreate with ({'bridge': 'ovirtmgmt', 'acpiEnable':
    'true',.....<br>
    Thread-49287::DEBUG::2012-04-17
    14:25:08,156::libvirtvm::1504::vm.Vm::(waitForMigrationDestinationPrepare)
    vmId=`ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a`::migration destination:
    waiting 31s for path preparation<br>
    <br>
    Thread-49287::DEBUG::2012-04-17
    14:25:39,458::libvirtvm::1507::vm.Vm::(waitForMigrationDestinationPrepare)
    vmId=`ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a`::Timeout while waiting
    for path preparation<br>
    <br>
    14:25:07,054::clientIF::76::vds::(wrapper) [10.23.30.130]::call
    getVmStats with ('ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a',) {}<br>
    Thread-49286::DEBUG::2012-04-17
    14:25:07,206::clientIF::81::vds::(wrapper) return getVmStats with
    {'status': {'message': 'Virtual machine does not exist',
'code': 1}}<br>
    <blockquote
cite="mid:CAGKvcKXco_=isKREAGyq+CTV_-=xVUruBHfBHDmkREJLc-W+7w@mail.gmail.com"
      type="cite">
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
    </blockquote>
    <br>
    <br>
    <pre class="moz-signature" cols="72">-- 

Thanks,

Rami Vaknin, QE @ Red Hat, TLV, IL.
</pre>
  </body>
</html>

--------------020800060704090308040504--

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Re: [Users] Problem with live migration.