This is a multi-part message in MIME format.
--------------050105090406090809010800
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
On 04/17/2012 05:36 PM, Martin Hovmöller wrote:
On Tue, Apr 17, 2012 at 2:55 PM, Rami Vaknin <rvaknin(a)redhat.com
<mailto:rvaknin@redhat.com>> wrote:
On 04/17/2012 03:45 PM, Martin Hovmöller wrote:
>
>
> On Tue, Apr 17, 2012 at 2:11 PM, Itamar Heim <iheim(a)redhat.com
> <mailto:iheim@redhat.com>> wrote:
>
> On 04/17/2012 12:47 PM, Martin Hovmöller wrote:
>
> I can't live migrate the vm's in my cluster:
>
> 012-04-17 04:06:39,728 INFO
> [org.ovirt.engine.core.bll.MigrateVmCommand]
> (pool-5-thread-48)
> Running command: MigrateVmCommand internal: false.
> Entities affected :
> ID: ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a Type: VM
> 2012-04-17 04:06:39,743 INFO
> [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
> (pool-5-thread-48)
> START, MigrateVDSCommand(vdsId =
> f33ffc14-87ba-11e1-b610-e3aeca1e8008,
> vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
> srcHost=10.23.30.130,
> dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,
> dstHost=10.23.30.110:54321 <
http://10.23.30.110:54321>
> <
http://10.23.30.110:54321>,
>
> migrationMethod=ONLINE), log id: 7c7201be
> 2012-04-17 04:06:39,748 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
> (pool-5-thread-48) VdsBroker::migrate::Entered
> (vm_guid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
> srcHost=10.23.30.130,
> dstHost=10.23.30.110:54321 <
http://10.23.30.110:54321>
> <
http://10.23.30.110:54321>, method=online
>
> 2012-04-17 04:06:39,749 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
> (pool-5-thread-48) START, MigrateBrokerVDSCommand(vdsId =
> f33ffc14-87ba-11e1-b610-e3aeca1e8008,
> vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
> srcHost=10.23.30.130,
> dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,
> dstHost=10.23.30.110:54321 <
http://10.23.30.110:54321>
> <
http://10.23.30.110:54321>,
>
> migrationMethod=ONLINE), log id: 3c40c2de
> 2012-04-17 04:06:39,834 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
> (pool-5-thread-48) FINISH, MigrateBrokerVDSCommand, log
> id: 3c40c2de
> 2012-04-17 04:06:39,840 INFO
> [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
> (pool-5-thread-48)
> FINISH, MigrateVDSCommand, return: MigratingFrom, log id:
> 7c7201be
> 2012-04-17 04:06:42,166 INFO
> [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> (QuartzScheduler_Worker-18) vds::refreshVmList vm id
> ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a is migrating to vds
> rhevh1.<domain>
> ignoring it in the refresh till migration is done
> [...]
> 2012-04-17 04:07:33,852 INFO
> [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> (QuartzScheduler_Worker-34) vds::refreshVmList vm id
> ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a status = Paused on vds
> rhevh1.<domain> ignoring it in the refresh till migration
> is done
> 2012-04-17 04:07:37,407 ERROR
> [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> (QuartzScheduler_Worker-71) Rerun vm
> ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a. Called from vds
> rhevh2.<domain>
> 2012-04-17 04:07:37,414 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
> (pool-5-thread-47) START, MigrateStatusVDSCommand(vdsId =
> f33ffc14-87ba-11e1-b610-e3aeca1e8008,
> vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a), log id: 8dfce1b
> 2012-04-17 04:07:37,496 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (pool-5-thread-47) Failed in MigrateStatusVDS method
> 2012-04-17 04:07:37,497 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (pool-5-thread-47) Error code createErr and error message
> VDSGenericException: VDSErrorException: Failed to
> MigrateStatusVDS,
> error = Error creating the requested virtual machine
> 2012-04-17 04:07:37,497 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (pool-5-thread-47) Command
> org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand
> return
> value
> Class Name:
> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
> mStatus Class Name:
> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
> mCode 9
> mMessage Error creating the
> requested virtual machine
>
>
> 2012-04-17 04:07:37,497 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
> (pool-5-thread-47) Vds: rhevh2.<domain>
> 2012-04-17 04:07:37,497 ERROR
> [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
> (pool-5-thread-47)
> Command MigrateStatusVDS execution failed. Exception:
> VDSErrorException:
> VDSGenericException: VDSErrorException: Failed to
> MigrateStatusVDS,
> error = Error creating the requested virtual machine
> 2012-04-17 04:07:37,497 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
> (pool-5-thread-47) FINISH, MigrateStatusVDSCommand, log
> id: 8dfce1b
> 2012-04-17 04:07:37,528 WARN
> [org.ovirt.engine.core.bll.MigrateVmCommand]
> (pool-5-thread-47)
> CanDoAction of action MigrateVm failed.
>
Reasons:ACTION_TYPE_FAILED_VDS_VM_CLUSTER,VAR__ACTION__MIGRATE,VAR__TYPE__VM
>
>
> There is no problem regarding storage or anything like
> that. If I shut
> down the vm I can start it on the other host without any
> problems
> whatsoever.
> Do I need to do something to make live migration work?
> Trying to dig in
> the logs on the hypervisors, but there's so much stuff
> being logged there...
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org <mailto:Users@ovirt.org>
>
http://lists.ovirt.org/mailman/listinfo/users
>
>
> vdsm log from the host that failed to create the VM?
>
>
> The vdsm log is frighteningly long, so I put it in pastebin:
>
http://pastebin.com/BEeqFCCb
> I don't really see anything worthwhile myself, but on the other
> hand I'm not sure what I'm looking for, so it's mostly a massive
> wall of text to me, but it should be somewhere in that time frame.
Here is the failure, "code 9" is not too informative, can you
attach the libvirtd.log file too?
Thread-49287::DEBUG::2012-04-17
14:25:39,834::clientIF::81::vds::(wrapper) return migrationCreate
with {'status': {'message': 'Error creating the requested
virtual
machine', 'code': 9}}
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org <mailto:Users@ovirt.org>
>
http://lists.ovirt.org/mailman/listinfo/users
--
Thanks,
Rami Vaknin, QE @ Red Hat, TLV, IL.
I think this is the relevant info from the libvirt.log:
2012-04-17 14:30:53.768+0000: 8506: debug :
virDomainLookupByUUIDString:1984 : conn=0x7fa9a00bcde0,
uuidstr=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
2012-04-17 14:30:53.768+0000: 8506: debug : virDomainLookupByUUID:1941
: conn=0x7fa9a00bcde0, uuid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
2012-04-17 14:30:53.771+0000: 8506: error :
virNetClientProgramDispatchError:170 : Domain not found: no domain
with matching uuid 'ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a'
What domain is this, is it the storage domain?
Domain in libvirt is simply a vm.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Thanks,
Rami Vaknin, QE @ Red Hat, TLV, IL.
--------------050105090406090809010800
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 04/17/2012 05:36 PM, Martin Hovmöller wrote:
<blockquote
cite="mid:CAGKvcKUS=mWsNRVq=EqxXiQWXgxqSftz0Yr9f_OKuim5Yoe-UQ@mail.gmail.com"
type="cite"><br>
<br>
<div class="gmail_quote">On Tue, Apr 17, 2012 at 2:55 PM, Rami
Vaknin <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:rvaknin@redhat.com">rvaknin@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div class="h5"> On 04/17/2012 03:45 PM, Martin
Hovmöller
wrote:
<blockquote type="cite"><br>
<br>
<div class="gmail_quote">On Tue, Apr 17, 2012 at 2:11
PM, Itamar Heim <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:iheim@redhat.com"
target="_blank">iheim(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On 04/17/2012 12:47 PM, Martin Hovmöller
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div> I can't live migrate the vm's in my
cluster:<br>
<br>
012-04-17 04:06:39,728 INFO<br>
[org.ovirt.engine.core.bll.MigrateVmCommand]
(pool-5-thread-48)<br>
Running command: MigrateVmCommand internal:
false. Entities affected :<br>
ID: ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a
Type: VM<br>
2012-04-17 04:06:39,743 INFO<br>
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
(pool-5-thread-48)<br>
START, MigrateVDSCommand(vdsId =
f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
srcHost=10.23.30.130,<br>
dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,<br>
</div>
dstHost=<a moz-do-not-send="true"
href="http://10.23.30.110:54321"
target="_blank">10.23.30.110:54321</a>
<<a
moz-do-not-send="true"
href="http://10.23.30.110:54321"
target="_blank">http://10.23.30.110:54321</a>>,
<div><br>
migrationMethod=ONLINE), log id: 7c7201be<br>
2012-04-17 04:06:39,748 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
(pool-5-thread-48) VdsBroker::migrate::Entered<br>
(vm_guid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
srcHost=10.23.30.130,<br>
</div>
dstHost=<a moz-do-not-send="true"
href="http://10.23.30.110:54321"
target="_blank">10.23.30.110:54321</a>
<<a
moz-do-not-send="true"
href="http://10.23.30.110:54321"
target="_blank">http://10.23.30.110:54321</a>>,
method=online
<div><br>
2012-04-17 04:06:39,749 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
(pool-5-thread-48) START,
MigrateBrokerVDSCommand(vdsId =<br>
f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a,
srcHost=10.23.30.130,<br>
dstVdsId=b4d329b4-87b6-11e1-ac0a-b70ec8cc50f0,<br>
</div>
dstHost=<a moz-do-not-send="true"
href="http://10.23.30.110:54321"
target="_blank">10.23.30.110:54321</a>
<<a
moz-do-not-send="true"
href="http://10.23.30.110:54321"
target="_blank">http://10.23.30.110:54321</a>>,
<div>
<div><br>
migrationMethod=ONLINE), log id: 3c40c2de<br>
2012-04-17 04:06:39,834 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]<br>
(pool-5-thread-48) FINISH,
MigrateBrokerVDSCommand, log id: 3c40c2de<br>
2012-04-17 04:06:39,840 INFO<br>
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
(pool-5-thread-48)<br>
FINISH, MigrateVDSCommand, return:
MigratingFrom, log id: 7c7201be<br>
2012-04-17 04:06:42,166 INFO<br>
[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
(QuartzScheduler_Worker-18)
vds::refreshVmList vm id<br>
ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a is
migrating to vds rhevh1.<domain><br>
ignoring it in the refresh till migration is
done<br>
[...]<br>
2012-04-17 04:07:33,852 INFO<br>
[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
(QuartzScheduler_Worker-34)
vds::refreshVmList vm id<br>
ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a status
= Paused on vds<br>
rhevh1.<domain> ignoring it in the
refresh till migration is done<br>
2012-04-17 04:07:37,407 ERROR<br>
[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]<br>
(QuartzScheduler_Worker-71) Rerun vm<br>
ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a. Called
from vds rhevh2.<domain><br>
2012-04-17 04:07:37,414 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]<br>
(pool-5-thread-47) START,
MigrateStatusVDSCommand(vdsId =<br>
f33ffc14-87ba-11e1-b610-e3aeca1e8008,<br>
vmId=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a),
log id: 8dfce1b<br>
2012-04-17 04:07:37,496 ERROR<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
(pool-5-thread-47) Failed in
MigrateStatusVDS method<br>
2012-04-17 04:07:37,497 ERROR<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
(pool-5-thread-47) Error code createErr and
error message<br>
VDSGenericException: VDSErrorException:
Failed to MigrateStatusVDS,<br>
error = Error creating the requested virtual
machine<br>
2012-04-17 04:07:37,497 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
(pool-5-thread-47) Command<br>
org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand
return<br>
value<br>
Class Name:<br>
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc<br>
mStatus
Class Name:<br>
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc<br>
mCode
9<br>
mMessage
Error
creating
the requested virtual machine<br>
<br>
<br>
2012-04-17 04:07:37,497 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]<br>
(pool-5-thread-47) Vds:
rhevh2.<domain><br>
2012-04-17 04:07:37,497 ERROR<br>
[org.ovirt.engine.core.vdsbroker.VDSCommandBase]
(pool-5-thread-47)<br>
Command MigrateStatusVDS execution failed.
Exception: VDSErrorException:<br>
VDSGenericException: VDSErrorException:
Failed to MigrateStatusVDS,<br>
error = Error creating the requested virtual
machine<br>
2012-04-17 04:07:37,497 INFO<br>
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]<br>
(pool-5-thread-47) FINISH,
MigrateStatusVDSCommand, log id: 8dfce1b<br>
2012-04-17 04:07:37,528 WARN<br>
[org.ovirt.engine.core.bll.MigrateVmCommand]
(pool-5-thread-47)<br>
CanDoAction of action MigrateVm failed.<br>
Reasons:ACTION_TYPE_FAILED_VDS_VM_CLUSTER,VAR__ACTION__MIGRATE,VAR__TYPE__VM<br>
<br>
<br>
There is no problem regarding storage or
anything like that. If I shut<br>
down the vm I can start it on the other host
without any problems<br>
whatsoever.<br>
Do I need to do something to make live
migration work? Trying to dig in<br>
the logs on the hypervisors, but there's so
much stuff being logged there...<br>
<br>
<br>
</div>
</div>
_______________________________________________<br>
Users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
</blockquote>
<br>
vdsm log from the host that failed to create the
VM?<br>
</blockquote>
</div>
<br>
<div>The vdsm log is frighteningly long, so I put it
in pastebin: <a moz-do-not-send="true"
href="http://pastebin.com/BEeqFCCb"
target="_blank">http://pastebin.com/BEeqFCCb</a></d...
<div>I don't really see anything worthwhile myself,
but on the other hand I'm not sure what I'm looking
for, so it's mostly a massive wall of text to me,
but it should be somewhere in that time frame.</div>
</blockquote>
</div>
</div>
Here is the failure, "code 9" is not too informative, can
you attach the libvirtd.log file too?<br>
<br>
Thread-49287::DEBUG::2012-04-17
14:25:39,834::clientIF::81::vds::(wrapper) return
migrationCreate with {'status': {'message': 'Error
creating
the requested virtual machine', 'code': 9}}
<div class="im"><br>
<blockquote type="cite"> <br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a>
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
</pre>
</blockquote>
<br>
<br>
</div>
<span class="HOEnZb"><font color="#888888">
<pre cols="72">--
Thanks,
Rami Vaknin, QE @ Red Hat, TLV, IL.
</pre>
</font></span></div>
</blockquote>
</div>
<div><br>
</div>
<div>I think this is the relevant info from the libvirt.log:</div>
<div>
<div>2012-04-17 14:30:53.768+0000: 8506: debug :
virDomainLookupByUUIDString:1984 : conn=0x7fa9a00bcde0,
uuidstr=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a</div>
<div>2012-04-17 14:30:53.768+0000: 8506: debug :
virDomainLookupByUUID:1941 : conn=0x7fa9a00bcde0,
uuid=ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a</div>
<div>2012-04-17 14:30:53.771+0000: 8506: error :
virNetClientProgramDispatchError:170 : Domain not found: no
domain with matching uuid
'ce9bb531-4d2a-4f5e-9935-3c8a4f5db94a'</div>
</div>
<div><br>
</div>
<div>What domain is this, is it the storage domain?</div>
</blockquote>
<br>
Domain in libvirt is simply a vm.<br>
<blockquote
cite="mid:CAGKvcKUS=mWsNRVq=EqxXiQWXgxqSftz0Yr9f_OKuim5Yoe-UQ@mail.gmail.com"
type="cite">
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Thanks,
Rami Vaknin, QE @ Red Hat, TLV, IL.
</pre>
</body>
</html>
--------------050105090406090809010800--