<br><br><div class="gmail_quote">On Tue, Apr 17, 2012 at 2:55 PM, Rami Vaknin <span dir="ltr">&lt;<a href="mailto:rvaknin@redhat.com">rvaknin@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

  
    
  
  <div bgcolor="#FFFFFF" text="#000000"><div><div class="h5">
    On 04/17/2012 03:45 PM, Martin Hovmöller wrote:
    <blockquote type="cite"><br>
      <br>
      <div class="gmail_quote">On Tue, Apr 17, 2012 at 2:11 PM, Itamar
        Heim <span dir="ltr">&lt;<a href="mailto:iheim@redhat.com" target="_blank">iheim@redhat.com</a>&gt;</span>
        wrote:<br>
        <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div>On 04/17/2012 12:47 PM, Martin Hovmöller
            wrote:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div>
              I can&#39;t live migrate the vm&#39;s in my cluster:<br>
              <br>
              012-04-17 04:06:39,728 INFO<br>
               [org.ovirt.engine.core.bll.MigrateVmCommand]
              (pool-5-thread-48)<br>
              Running command: MigrateVmCommand internal: false.
              Entities affected :<br>
               ID: ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a Type: VM<br>
              2012-04-17 04:06:39,743 INFO<br>
               [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
              (pool-5-thread-48)<br>
              START, MigrateVDSCommand(vdsId = f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
              vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
              srcHost=10.23.30.130,<br>
              dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,<br>
            </div>
            dstHost=<a href="http://10.23.30.110:54321" target="_blank">10.23.30.110:54321</a>
            &lt;<a href="http://10.23.30.110:54321" target="_blank">http://10.23.30.110:54321</a>&gt;,
            <div><br>
              migrationMethod=ONLINE), log id: 7c7201be<br>
              2012-04-17 04:06:39,748 INFO<br>
               [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
              (pool-5-thread-48) VdsBroker::migrate::Entered<br>
              (vm_guid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
              srcHost=10.23.30.130,<br>
            </div>
            dstHost=<a href="http://10.23.30.110:54321" target="_blank">10.23.30.110:54321</a>
            &lt;<a href="http://10.23.30.110:54321" target="_blank">http://10.23.30.110:54321</a>&gt;,
             method=online
            <div><br>
              2012-04-17 04:06:39,749 INFO<br>
               [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
              (pool-5-thread-48) START, MigrateBrokerVDSCommand(vdsId =<br>
              f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
              vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
              srcHost=10.23.30.130,<br>
              dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,<br>
            </div>
            dstHost=<a href="http://10.23.30.110:54321" target="_blank">10.23.30.110:54321</a>
            &lt;<a href="http://10.23.30.110:54321" target="_blank">http://10.23.30.110:54321</a>&gt;,
            <div>
              <div><br>
                migrationMethod=ONLINE), log id: 3c40c2de<br>
                2012-04-17 04:06:39,834 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
                (pool-5-thread-48) FINISH, MigrateBrokerVDSCommand, log
                id: 3c40c2de<br>
                2012-04-17 04:06:39,840 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
                (pool-5-thread-48)<br>
                FINISH, MigrateVDSCommand, return: MigratingFrom, log
                id: 7c7201be<br>
                2012-04-17 04:06:42,166 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
                (QuartzScheduler_Worker-18) vds::refreshVmList vm id<br>
                ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a is migrating to vds
                rhevh1.&lt;domain&gt;<br>
                ignoring it in the refresh till migration is done<br>
                [...]<br>
                2012-04-17 04:07:33,852 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
                (QuartzScheduler_Worker-34) vds::refreshVmList vm id<br>
                ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a status = Paused on
                vds<br>
                rhevh1.&lt;domain&gt; ignoring it in the refresh till
                migration is done<br>
                2012-04-17 04:07:37,407 ERROR<br>
                [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
                (QuartzScheduler_Worker-71) Rerun vm<br>
                ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a. Called from vds
                rhevh2.&lt;domain&gt;<br>
                2012-04-17 04:07:37,414 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]<br>
                (pool-5-thread-47) START, MigrateStatusVDSCommand(vdsId
                =<br>
                f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
                vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a), log id:
                8dfce1b<br>
                2012-04-17 04:07:37,496 ERROR<br>
                [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                (pool-5-thread-47) Failed in MigrateStatusVDS method<br>
                2012-04-17 04:07:37,497 ERROR<br>
                [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                (pool-5-thread-47) Error code createErr and error
                message<br>
                VDSGenericException: VDSErrorException: Failed to
                MigrateStatusVDS,<br>
                error = Error creating the requested virtual machine<br>
                2012-04-17 04:07:37,497 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                (pool-5-thread-47) Command<br>
                org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand
                return<br>
                value<br>
                 Class Name:<br>
                org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc<br>
                mStatus                       Class Name:<br>
                org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc<br>
                mCode                         9<br>
                mMessage                      Error creating the
                requested virtual machine<br>
                <br>
                <br>
                2012-04-17 04:07:37,497 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
                (pool-5-thread-47) Vds: rhevh2.&lt;domain&gt;<br>
                2012-04-17 04:07:37,497 ERROR<br>
                [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
                (pool-5-thread-47)<br>
                Command MigrateStatusVDS execution failed. Exception:
                VDSErrorException:<br>
                VDSGenericException: VDSErrorException: Failed to
                MigrateStatusVDS,<br>
                error = Error creating the requested virtual machine<br>
                2012-04-17 04:07:37,497 INFO<br>
                 [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]<br>
                (pool-5-thread-47) FINISH, MigrateStatusVDSCommand, log
                id: 8dfce1b<br>
                2012-04-17 04:07:37,528 WARN<br>
                 [org.ovirt.engine.core.bll.MigrateVmCommand]
                (pool-5-thread-47)<br>
                CanDoAction of action MigrateVm failed.<br>
                Reasons:ACTION_TYPE_FAILED_VDS_VM_CLUSTER,VAR__ACTION__MIGRATE,VAR__TYPE__VM<br>
                <br>
                <br>
                There is no problem regarding storage or anything like
                that. If I shut<br>
                down the vm I can start it on the other host without any
                problems<br>
                whatsoever.<br>
                Do I need to do something to make live migration work?
                Trying to dig in<br>
                the logs on the hypervisors, but there&#39;s so much stuff
                being logged there...<br>
                <br>
                <br>
              </div>
            </div>
            _______________________________________________<br>
            Users mailing list<br>
            <a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
            <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
          </blockquote>
          <br>
          vdsm log from the host that failed to create the VM?<br>
        </blockquote>
      </div>
      <br>
      <div>The vdsm log is frighteningly long, so I put it in pastebin: <a href="http://pastebin.com/BEeqFCCb" target="_blank">http://pastebin.com/BEeqFCCb</a></div>
      <div>I don&#39;t really see anything worthwhile myself, but on the
        other hand I&#39;m not sure what I&#39;m looking for, so it&#39;s mostly a
        massive wall of text to me, but it should be somewhere in that
        time frame.</div>
    </blockquote></div></div>
    Here is the failure, &quot;code 9&quot; is not too informative, can you attach
    the libvirtd.log file too?<br>
    <br>
    Thread-49287::DEBUG::2012-04-17
    14:25:39,834::clientIF::81::vds::(wrapper) return migrationCreate
    with {&#39;status&#39;: {&#39;message&#39;: &#39;Error creating the requested virtual
    machine&#39;, &#39;code&#39;: 9}}<div class="im"><br>
    <blockquote type="cite">
      <br>
      <fieldset></fieldset>
      <br>
      <pre>_______________________________________________
Users mailing list
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
    <br>
    </div><span class="HOEnZb"><font color="#888888"><pre cols="72">-- 

Thanks,

Rami Vaknin, QE @ Red Hat, TLV, IL.
</pre>
  </font></span></div>

</blockquote></div><div><br></div><div>I think this is the relevant info from the libvirt.log:</div><div><div>2012-04-17 14:30:53.768+0000: 8506: debug : virDomainLookupByUUIDString:1984 : conn=0x7fa9a00bcde0, uuidstr=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a</div>
<div>2012-04-17 14:30:53.768+0000: 8506: debug : virDomainLookupByUUID:1941 : conn=0x7fa9a00bcde0, uuid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a</div><div>2012-04-17 14:30:53.771+0000: 8506: error : virNetClientProgramDispatchError:170 : Domain not found: no domain with matching uuid &#39;ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a&#39;</div>
</div><div><br></div><div>What domain is this, is it the storage domain?</div>