From michal.skrivanek at redhat.com Fri Oct 4 15:30:34 2013 Content-Type: multipart/mixed; boundary="===============7714051177634271967==" MIME-Version: 1.0 From: Michal Skrivanek To: users at ovirt.org Subject: Re: [Users] Migration issues with ovirt 3.3 Date: Fri, 04 Oct 2013 21:30:09 +0200 Message-ID: <32425E32-C174-4860-B7D4-E1C8B0AADB6F@redhat.com> In-Reply-To: CAPJ7RaQnd_VwoaTyMxs8zAkyGrt_zGNLXrt5iTBEg-QiucOVug@mail.gmail.com --===============7714051177634271967== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable --Apple-Mail=3D_80349546-76AA-4918-9B45-6194CB54F8F2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=3Dwindows-1252 On 4 Oct 2013, at 19:55, emitor(a)gmail.com wrote: > Dan, I've made a whole new installation of oVirt 3.3 using the stable =3D repos, I've updated libvirt to 1.1.2-1 and configured a glusterFS =3D cluster. >=3D20 > I've just tried the live migration of the Windows Host, here i send =3D you the vdsm, libvir and qemu logs hmm=3D85I see both PCI address for CDROM as we've been seeing on gluster = =3D setups. So far no update from gluster AFAIK=3D85it seems to be broken on = =3D their side somewhere=3D85 and I also see error on migrations dst which seems to be =3D another/different disk device mess up The W2K8 create VM call in the ovirt1's log is already wrong, can you =3D capture/include the initial create call? In the previous cases it was =3D always the case that the VM was created(from engine) with IDE address =3D but then after the first run it somehow changed to PCI on the first =3D status update from vdsm Thanks, michal >=3D20 >=3D20 > 2013/10/4 Dan Kenigsberg > On Wed, Oct 02, 2013 at 01:36:56PM -0300, emitor(a)gmail.com wrote: > > I've configured the host as a NFS cluster. I've imported the VMs, =3D the > > debian now it's migrating without problems, the pfSense now gives me = =3D an > > error (Migration failed due to Error: Desktop does not exist) and =3D the > > Windows VM doesn't start in the new cluster after import it. > > > > I've attached the logs of the vdsm >=3D20 > Thanks for providing your, but they begin too late. >=3D20 > The bug is still a mystery to me. Something garbles Vdsm's internal > representation of virtual disks. And it's bad. >=3D20 > I would appreciate if you could dig backward in your logs and find the > last successfull startup of a VM, say W2K8_cluster1 > (86f10137-d917-4508-a07a-3f83e6f24a95). Then, browse forward to find =3D the > first failure (be it on next startup of VM or during migration). >=3D20 > Best regards, > Dan. >=3D20 >=3D20 >=3D20 > --=3D20 > Emiliano Tortorella > +598 98941176 > emitor(a)gmail.com > =3D ____________________________________________= =3D ___ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users --Apple-Mail=3D_80349546-76AA-4918-9B45-6194CB54F8F2 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=3Dwindows-1252
On 4 Oct 2013, at 19:55, emitor(a)gmail.com wrote:
Dan, I've made a whole new installation of oVirt 3.3 using = =3D the stable repos, I've updated libvirt to 1.1.2-1 and configured a =3D glusterFS cluster.

I've just tried the live migration = =3D of the Windows Host, here i send you the vdsm, libvir and qemu =3D logs

hmm=3D85I see both PCI address = =3D for CDROM as we've been seeing on gluster setups. So far no update from =3D gluster AFAIK=3D85it seems to be broken on their side =3D somewhere=3D85
and I also see error on migrations dst which =3D seems to be another/different disk device mess =3D up

The W2K8 create VM call in the ovirt1's = =3D log is already wrong, can you capture/include the initial create call? =3D In the previous cases it was always the case that the VM was =3D created(from engine) with IDE address but then after the first run it =3D somehow changed to PCI on the first status update from =3D vdsm

Thanks,
michal



2013/10/4 Dan Kenigsberg <danken(a)redhat.com>
On Wed, Oct 02, 2013 at 01:36:56PM -0300, emitor(a)gmail.com wrote:
> I've configured the host as a NFS cluster. I've imported the VMs, =3D the
> debian now it's migrating without problems, the pfSense now gives =3D me an
> error (Migration failed due to Error: Desktop does not exist) and =3D the
> Windows VM doesn't start in the new cluster after import it.
>
> I've attached the logs of the vdsm

Thanks for providing your, but they begin too late.

The bug is still a mystery to me. Something garbles Vdsm's internal
representation of virtual disks. And it's bad.

I would appreciate if you could dig backward in your logs and find =3D the
last successfull startup of a VM, say W2K8_cluster1
(86f10137-d917-4508-a07a-3f83e6f24a95). Then, browse forward to find =3D the
first failure (be it on next startup of VM or during migration).

Best regards,
Dan.



--
Emiliano Tortorella
+598 98941176
emitor(a)gmail.com
=3D <ovirt_logs_clusterGFS.tar.gz>_________________________= =3D ______________________
Users mailing list
Users(a)ovirt.org
http://lists.o= virt.=3D org/mailman/listinfo/users

=3D --Apple-Mail=3D_80349546-76AA-4918-9B45-6194CB54F8F2-- --===============7714051177634271967== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" Ci0tQXBwbGUtTWFpbD1fODAzNDk1NDYtNzZBQS00OTE4LTlCNDUtNjE5NENCNTRGOEYyCkNvbnRl bnQtVHJhbnNmZXItRW5jb2Rpbmc6IHF1b3RlZC1wcmludGFibGUKQ29udGVudC1UeXBlOiB0ZXh0 L3BsYWluOwoJY2hhcnNldD13aW5kb3dzLTEyNTIKCgpPbiA0IE9jdCAyMDEzLCBhdCAxOTo1NSwg ZW1pdG9yQGdtYWlsLmNvbSB3cm90ZToKCj4gRGFuLCBJJ3ZlIG1hZGUgYSB3aG9sZSBuZXcgaW5z dGFsbGF0aW9uIG9mIG9WaXJ0IDMuMyB1c2luZyB0aGUgc3RhYmxlID0KcmVwb3MsIEkndmUgdXBk YXRlZCBsaWJ2aXJ0IHRvIDEuMS4yLTEgYW5kIGNvbmZpZ3VyZWQgYSBnbHVzdGVyRlMgPQpjbHVz dGVyLgo+PTIwCj4gSSd2ZSBqdXN0IHRyaWVkIHRoZSBsaXZlIG1pZ3JhdGlvbiBvZiB0aGUgV2lu ZG93cyBIb3N0LCBoZXJlIGkgc2VuZCA9CnlvdSB0aGUgdmRzbSwgbGlidmlyIGFuZCBxZW11IGxv Z3MKCmhtbT04NUkgc2VlIGJvdGggUENJIGFkZHJlc3MgZm9yIENEUk9NIGFzIHdlJ3ZlIGJlZW4g c2VlaW5nIG9uIGdsdXN0ZXIgPQpzZXR1cHMuIFNvIGZhciBubyB1cGRhdGUgZnJvbSBnbHVzdGVy IEFGQUlLPTg1aXQgc2VlbXMgdG8gYmUgYnJva2VuIG9uID0KdGhlaXIgc2lkZSBzb21ld2hlcmU9 ODUKYW5kIEkgYWxzbyBzZWUgZXJyb3Igb24gbWlncmF0aW9ucyBkc3Qgd2hpY2ggc2VlbXMgdG8g YmUgPQphbm90aGVyL2RpZmZlcmVudCBkaXNrIGRldmljZSBtZXNzIHVwCgpUaGUgVzJLOCBjcmVh dGUgVk0gY2FsbCBpbiB0aGUgb3ZpcnQxJ3MgbG9nIGlzIGFscmVhZHkgd3JvbmcsIGNhbiB5b3Ug PQpjYXB0dXJlL2luY2x1ZGUgdGhlIGluaXRpYWwgY3JlYXRlIGNhbGw/IEluIHRoZSBwcmV2aW91 cyBjYXNlcyBpdCB3YXMgPQphbHdheXMgdGhlIGNhc2UgdGhhdCB0aGUgVk0gd2FzIGNyZWF0ZWQo ZnJvbSBlbmdpbmUpIHdpdGggSURFIGFkZHJlc3MgPQpidXQgdGhlbiBhZnRlciB0aGUgZmlyc3Qg cnVuIGl0IHNvbWVob3cgY2hhbmdlZCB0byBQQ0kgb24gdGhlIGZpcnN0ID0Kc3RhdHVzIHVwZGF0 ZSBmcm9tIHZkc20KClRoYW5rcywKbWljaGFsCgo+PTIwCj49MjAKPiAyMDEzLzEwLzQgRGFuIEtl bmlnc2JlcmcgPGRhbmtlbkByZWRoYXQuY29tPgo+IE9uIFdlZCwgT2N0IDAyLCAyMDEzIGF0IDAx OjM2OjU2UE0gLTAzMDAsIGVtaXRvckBnbWFpbC5jb20gd3JvdGU6Cj4gPiBJJ3ZlIGNvbmZpZ3Vy ZWQgdGhlIGhvc3QgYXMgYSBORlMgY2x1c3Rlci4gSSd2ZSBpbXBvcnRlZCB0aGUgVk1zLCA9CnRo ZQo+ID4gZGViaWFuIG5vdyBpdCdzIG1pZ3JhdGluZyB3aXRob3V0IHByb2JsZW1zLCB0aGUgcGZT ZW5zZSBub3cgZ2l2ZXMgbWUgPQphbgo+ID4gZXJyb3IgKE1pZ3JhdGlvbiBmYWlsZWQgZHVlIHRv IEVycm9yOiBEZXNrdG9wIGRvZXMgbm90IGV4aXN0KSBhbmQgPQp0aGUKPiA+IFdpbmRvd3MgVk0g ZG9lc24ndCBzdGFydCBpbiB0aGUgbmV3IGNsdXN0ZXIgYWZ0ZXIgaW1wb3J0IGl0Lgo+ID4KPiA+ IEkndmUgYXR0YWNoZWQgdGhlIGxvZ3Mgb2YgdGhlIHZkc20KPj0yMAo+IFRoYW5rcyBmb3IgcHJv dmlkaW5nIHlvdXIsIGJ1dCB0aGV5IGJlZ2luIHRvbyBsYXRlLgo+PTIwCj4gVGhlIGJ1ZyBpcyBz dGlsbCBhIG15c3RlcnkgdG8gbWUuIFNvbWV0aGluZyBnYXJibGVzIFZkc20ncyBpbnRlcm5hbAo+ IHJlcHJlc2VudGF0aW9uIG9mIHZpcnR1YWwgZGlza3MuIEFuZCBpdCdzIGJhZC4KPj0yMAo+IEkg d291bGQgYXBwcmVjaWF0ZSBpZiB5b3UgY291bGQgZGlnIGJhY2t3YXJkIGluIHlvdXIgbG9ncyBh bmQgZmluZCB0aGUKPiBsYXN0IHN1Y2Nlc3NmdWxsIHN0YXJ0dXAgb2YgYSBWTSwgc2F5IFcySzhf Y2x1c3RlcjEKPiAoODZmMTAxMzctZDkxNy00NTA4LWEwN2EtM2Y4M2U2ZjI0YTk1KS4gVGhlbiwg YnJvd3NlIGZvcndhcmQgdG8gZmluZCA9CnRoZQo+IGZpcnN0IGZhaWx1cmUgKGJlIGl0IG9uIG5l eHQgc3RhcnR1cCBvZiBWTSBvciBkdXJpbmcgbWlncmF0aW9uKS4KPj0yMAo+IEJlc3QgcmVnYXJk cywKPiBEYW4uCj49MjAKPj0yMAo+PTIwCj4gLS09MjAKPiBFbWlsaWFubyBUb3J0b3JlbGxhCj4g KzU5OCA5ODk0MTE3Ngo+IGVtaXRvckBnbWFpbC5jb20KPiA9CjxvdmlydF9sb2dzX2NsdXN0ZXJH RlMudGFyLmd6Pl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPQpf X18KPiBVc2VycyBtYWlsaW5nIGxpc3QKPiBVc2Vyc0BvdmlydC5vcmcKPiBodHRwOi8vbGlzdHMu b3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMKCgotLUFwcGxlLU1haWw9XzgwMzQ5NTQ2 LTc2QUEtNDkxOC05QjQ1LTYxOTRDQjU0RjhGMgpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiBx dW90ZWQtcHJpbnRhYmxlCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOwoJY2hhcnNldD13aW5kb3dz LTEyNTIKCjxodG1sPjxoZWFkPjwvaGVhZD48Ym9keSBzdHlsZT0zRCJ3b3JkLXdyYXA6IGJyZWFr LXdvcmQ7ID0KLXdlYmtpdC1uYnNwLW1vZGU6IHNwYWNlOyAtd2Via2l0LWxpbmUtYnJlYWs6IGFm dGVyLXdoaXRlLXNwYWNlOyA9CiI+PGJyPjxkaXY+PGRpdj5PbiA0IE9jdCAyMDEzLCBhdCAxOTo1 NSwgPGEgPQpocmVmPTNEIm1haWx0bzplbWl0b3JAZ21haWwuY29tIj5lbWl0b3JAZ21haWwuY29t PC9hPiB3cm90ZTo8L2Rpdj48YnIgPQpjbGFzcz0zRCJBcHBsZS1pbnRlcmNoYW5nZS1uZXdsaW5l Ij48YmxvY2txdW90ZSB0eXBlPTNEImNpdGUiPjxkaXYgPQpkaXI9M0QibHRyIj5EYW4sIEkndmUg bWFkZSBhIHdob2xlIG5ldyBpbnN0YWxsYXRpb24gb2Ygb1ZpcnQgMy4zIHVzaW5nID0KdGhlIHN0 YWJsZSByZXBvcywgSSd2ZSB1cGRhdGVkIGxpYnZpcnQgdG8mbmJzcDsxLjEuMi0xIGFuZCBjb25m aWd1cmVkIGEgPQpnbHVzdGVyRlMgY2x1c3Rlci48ZGl2Pjxicj48L2Rpdj48ZGl2PkkndmUganVz dCB0cmllZCB0aGUgbGl2ZSBtaWdyYXRpb24gPQpvZiB0aGUgV2luZG93cyBIb3N0LCBoZXJlIGkg c2VuZCB5b3UgdGhlIHZkc20sIGxpYnZpciBhbmQgcWVtdSA9CmxvZ3M8L2Rpdj48L2Rpdj48L2Js b2NrcXVvdGU+PGRpdj48YnI+PC9kaXY+aG1tPTg1SSBzZWUgYm90aCBQQ0kgYWRkcmVzcyA9CmZv ciBDRFJPTSBhcyB3ZSd2ZSBiZWVuIHNlZWluZyBvbiBnbHVzdGVyIHNldHVwcy4gU28gZmFyIG5v IHVwZGF0ZSBmcm9tID0KZ2x1c3RlciBBRkFJSz04NWl0IHNlZW1zIHRvIGJlIGJyb2tlbiBvbiB0 aGVpciBzaWRlID0Kc29tZXdoZXJlPTg1PC9kaXY+PGRpdj5hbmQgSSBhbHNvIHNlZSBlcnJvciBv biBtaWdyYXRpb25zIGRzdCB3aGljaCA9CnNlZW1zIHRvIGJlIGFub3RoZXIvZGlmZmVyZW50IGRp c2sgZGV2aWNlIG1lc3MgPQp1cDwvZGl2PjxkaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5UaGUgVzJL OCBjcmVhdGUgVk0gY2FsbCBpbiB0aGUgb3ZpcnQxJ3MgPQpsb2cgaXMgYWxyZWFkeSB3cm9uZywg Y2FuIHlvdSBjYXB0dXJlL2luY2x1ZGUgdGhlIGluaXRpYWwgY3JlYXRlIGNhbGw/ID0KSW4gdGhl IHByZXZpb3VzIGNhc2VzIGl0IHdhcyBhbHdheXMgdGhlIGNhc2UgdGhhdCB0aGUgVk0gd2FzID0K Y3JlYXRlZChmcm9tIGVuZ2luZSkgd2l0aCBJREUgYWRkcmVzcyBidXQgdGhlbiBhZnRlciB0aGUg Zmlyc3QgcnVuIGl0ID0Kc29tZWhvdyBjaGFuZ2VkIHRvIFBDSSBvbiB0aGUgZmlyc3Qgc3RhdHVz IHVwZGF0ZSBmcm9tID0KdmRzbTwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+VGhhbmtzLDwvZGl2 PjxkaXY+bWljaGFsPC9kaXY+PGRpdj48YnI+PC9kaXY9Cj48YmxvY2txdW90ZSB0eXBlPTNEImNp dGUiPjxkaXYgZGlyPTNEImx0ciI+CjwvZGl2PjxkaXYgY2xhc3M9M0QiZ21haWxfZXh0cmEiPjxi cj48YnI+PGRpdiA9CmNsYXNzPTNEImdtYWlsX3F1b3RlIj4yMDEzLzEwLzQgRGFuIEtlbmlnc2Jl cmcgPHNwYW4gZGlyPTNEImx0ciI+Jmx0OzxhID0KaHJlZj0zRCJtYWlsdG86ZGFua2VuQHJlZGhh dC5jb20iID0KdGFyZ2V0PTNEIl9ibGFuayI+ZGFua2VuQHJlZGhhdC5jb208L2E+Jmd0Ozwvc3Bh bj48YnI+PGJsb2NrcXVvdGUgPQpjbGFzcz0zRCJnbWFpbF9xdW90ZSIgc3R5bGU9M0QibWFyZ2lu OjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2MgPQpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4 Ij4KPGRpdiBjbGFzcz0zRCJpbSI+T24gV2VkLCBPY3QgMDIsIDIwMTMgYXQgMDE6MzY6NTZQTSAt MDMwMCwgPGEgPQpocmVmPTNEIm1haWx0bzplbWl0b3JAZ21haWwuY29tIj5lbWl0b3JAZ21haWwu Y29tPC9hPiB3cm90ZTo8YnI+CiZndDsgSSd2ZSBjb25maWd1cmVkIHRoZSBob3N0IGFzIGEgTkZT IGNsdXN0ZXIuIEkndmUgaW1wb3J0ZWQgdGhlIFZNcywgPQp0aGU8YnI+CiZndDsgZGViaWFuIG5v dyBpdCdzIG1pZ3JhdGluZyB3aXRob3V0IHByb2JsZW1zLCB0aGUgcGZTZW5zZSBub3cgZ2l2ZXMg PQptZSBhbjxicj4KJmd0OyBlcnJvciAoTWlncmF0aW9uIGZhaWxlZCBkdWUgdG8gRXJyb3I6IERl c2t0b3AgZG9lcyBub3QgZXhpc3QpIGFuZCA9CnRoZTxicj4KJmd0OyBXaW5kb3dzIFZNIGRvZXNu J3Qgc3RhcnQgaW4gdGhlIG5ldyBjbHVzdGVyIGFmdGVyIGltcG9ydCBpdC48YnI+CiZndDs8YnI+ CiZndDsgSSd2ZSBhdHRhY2hlZCB0aGUgbG9ncyBvZiB0aGUgdmRzbTxicj4KPGJyPgo8L2Rpdj5U aGFua3MgZm9yIHByb3ZpZGluZyB5b3VyLCBidXQgdGhleSBiZWdpbiB0b28gbGF0ZS48YnI+Cjxi cj4KVGhlIGJ1ZyBpcyBzdGlsbCBhIG15c3RlcnkgdG8gbWUuIFNvbWV0aGluZyBnYXJibGVzIFZk c20ncyBpbnRlcm5hbDxicj4KcmVwcmVzZW50YXRpb24gb2YgdmlydHVhbCBkaXNrcy4gQW5kIGl0 J3MgYmFkLjxicj4KPGJyPgpJIHdvdWxkIGFwcHJlY2lhdGUgaWYgeW91IGNvdWxkIGRpZyBiYWNr d2FyZCBpbiB5b3VyIGxvZ3MgYW5kIGZpbmQgPQp0aGU8YnI+Cmxhc3Qgc3VjY2Vzc2Z1bGwgc3Rh cnR1cCBvZiBhIFZNLCBzYXkgVzJLOF9jbHVzdGVyMTxicj4KKDg2ZjEwMTM3LWQ5MTctNDUwOC1h MDdhLTNmODNlNmYyNGE5NSkuIFRoZW4sIGJyb3dzZSBmb3J3YXJkIHRvIGZpbmQgPQp0aGU8YnI+ CmZpcnN0IGZhaWx1cmUgKGJlIGl0IG9uIG5leHQgc3RhcnR1cCBvZiBWTSBvciBkdXJpbmcgbWln cmF0aW9uKS48YnI+Cjxicj4KQmVzdCByZWdhcmRzLDxicj4KRGFuLjxicj4KPC9ibG9ja3F1b3Rl PjwvZGl2Pjxicj48YnIgY2xlYXI9M0QiYWxsIj48ZGl2Pjxicj48L2Rpdj4tLSA8YnI+PGRpdiA9 CmRpcj0zRCJsdHIiPjxiPkVtaWxpYW5vIFRvcnRvcmVsbGE8L2I+PGJyPis1OTggOTg5NDExNzY8 YnI+PGEgPQpocmVmPTNEIm1haWx0bzplbWl0b3JAZ21haWwuY29tIiA9CnRhcmdldD0zRCJfYmxh bmsiPmVtaXRvckBnbWFpbC5jb208L2E+PC9kaXY+CjwvZGl2Pgo9CjxzcGFuPiZsdDtvdmlydF9s b2dzX2NsdXN0ZXJHRlMudGFyLmd6Jmd0Ozwvc3Bhbj5fX19fX19fX19fX19fX19fX19fX19fX19f PQpfX19fX19fX19fX19fX19fX19fX19fPGJyPlVzZXJzIG1haWxpbmcgbGlzdDxicj48YSA9Cmhy ZWY9M0QibWFpbHRvOlVzZXJzQG92aXJ0Lm9yZyI+VXNlcnNAb3ZpcnQub3JnPC9hPjxicj5odHRw Oi8vbGlzdHMub3ZpcnQuPQpvcmcvbWFpbG1hbi9saXN0aW5mby91c2Vyczxicj48L2Jsb2NrcXVv dGU+PC9kaXY+PGJyPjwvYm9keT48L2h0bWw+PQoKLS1BcHBsZS1NYWlsPV84MDM0OTU0Ni03NkFB LTQ5MTgtOUI0NS02MTk0Q0I1NEY4RjItLQo= --===============7714051177634271967==--