From vfeenstr at redhat.com Mon Jan 7 07:19:53 2013 Content-Type: multipart/mixed; boundary="===============1373177785782240019==" MIME-Version: 1.0 From: Vinzenz Feenstra To: users at ovirt.org Subject: Re: [Users] node loses contact with rhev agent Date: Mon, 07 Jan 2013 13:19:49 +0100 Message-ID: <50EABD65.5040902@redhat.com> In-Reply-To: 50E6A199.8050604@redhat.com --===============1373177785782240019== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------020604000405040203010901 Content-Type: text/plain; charset=3DISO-8859-1; format=3Dflowed Content-Transfer-Encoding: 7bit Hi Jonathan, Could you provide the logs for it? Thanks. On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote: > Hi, > > On 01/03/2013 08:53 PM, Jonathan Horne wrote: >> i have 2 nodes now that have both lost contact with the rhev agents = >> on guest VMs. if i migrate the guest to another node, the IP address = >> and memory usage immediately show up. migrate a guest back, and the = >> IP/memory info disappears. >> >> how can i troubleshoot what is causing this? > > It'd be nice if you could send me the logs from /var/log/vdsm/vdsm* = > which are on the node where it looses the contact. I am currently = > trying to find exactly the cause of such an issue as well and it would = > be helpful if we could be provided with log files. Currently I am = > still lacking them. > I will check them and let you know what I will be able to find. > > If you wouldn't mind I would be referring to you for more questions if = > some will come up during the search for the cause and I will let you = > know the results asap. >> >> thanks, >> jonathan >> > > -- = > Regards, > > Vinzenz Feenstra | Senior Software Engineer > RedHat Engineering Virtualization R & D > Phone: +420 532 294 625 > IRC: vfeenstr or evilissimo > > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users -- = Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com --------------020604000405040203010901 Content-Type: text/html; charset=3DISO-8859-1 Content-Transfer-Encoding: 7bit

Hi Jonathan,

Could you provide the logs for it?
Thanks.

On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote:

Hi,

On 01/03/2013 08:53 PM, Jonathan Horne wrote:

i have 2 nodes now that have both lost contact with the rhev agents on guest VMs. if i migrate the guest to another node, the IP address and memory usage immediately show up. migrate a guest back, and the IP/memory info disappears.

how can i troubleshoot what is causing this?

It'd be nice if you could send me the logs from /var/log/vdsm/vdsm* which are on the node where it looses the contact. I am currently trying to find exactly the cause of such an issue as well and it would be helpful if we could be provided with log files. Currently I am still lacking them.
I will check them and let you know what I will be able to find.

If you wouldn't mind I would be referring to you for more questions if some will come up during the search for the cause and I will let you know the results asap.

thanks,

jonathan
-- =

Regards,

Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo

Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
_______________________________________________
Users mailing list
Use=
rs(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

-- =

Regards,

Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo

Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com

--------------020604000405040203010901-- --===============1373177785782240019== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMjA2MDQwMDA0MDUwNDAyMDMwMTA5MDEKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PUlTTy04ODU5LTE7IGZvcm1hdD1mbG93ZWQKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzog N2JpdAoKSGkgSm9uYXRoYW4sCgpDb3VsZCB5b3UgcHJvdmlkZSB0aGUgbG9ncyBmb3IgaXQ/ClRo YW5rcy4KCk9uIDAxLzA0LzIwMTMgMTA6MzIgQU0sIFZpbnplbnogRmVlbnN0cmEgd3JvdGU6Cj4g SGksCj4KPiBPbiAwMS8wMy8yMDEzIDA4OjUzIFBNLCBKb25hdGhhbiBIb3JuZSB3cm90ZToKPj4g aSBoYXZlIDIgbm9kZXMgbm93IHRoYXQgaGF2ZSBib3RoIGxvc3QgY29udGFjdCB3aXRoIHRoZSBy aGV2IGFnZW50cyAKPj4gb24gZ3Vlc3QgVk1zLiAgaWYgaSBtaWdyYXRlIHRoZSBndWVzdCB0byBh bm90aGVyIG5vZGUsIHRoZSBJUCBhZGRyZXNzIAo+PiBhbmQgbWVtb3J5IHVzYWdlIGltbWVkaWF0 ZWx5IHNob3cgdXAuICBtaWdyYXRlIGEgZ3Vlc3QgYmFjaywgYW5kIHRoZSAKPj4gSVAvbWVtb3J5 IGluZm8gZGlzYXBwZWFycy4KPj4KPj4gaG93IGNhbiBpIHRyb3VibGVzaG9vdCB3aGF0IGlzIGNh dXNpbmcgdGhpcz8KPgo+IEl0J2QgYmUgbmljZSBpZiB5b3UgY291bGQgc2VuZCBtZSB0aGUgbG9n cyBmcm9tIC92YXIvbG9nL3Zkc20vdmRzbSogCj4gd2hpY2ggYXJlIG9uIHRoZSBub2RlIHdoZXJl IGl0IGxvb3NlcyB0aGUgY29udGFjdC4gSSBhbSBjdXJyZW50bHkgCj4gdHJ5aW5nIHRvIGZpbmQg ZXhhY3RseSB0aGUgY2F1c2Ugb2Ygc3VjaCBhbiBpc3N1ZSBhcyB3ZWxsIGFuZCBpdCB3b3VsZCAK PiBiZSBoZWxwZnVsIGlmIHdlIGNvdWxkIGJlIHByb3ZpZGVkIHdpdGggbG9nIGZpbGVzLiBDdXJy ZW50bHkgSSBhbSAKPiBzdGlsbCBsYWNraW5nIHRoZW0uCj4gSSB3aWxsIGNoZWNrIHRoZW0gYW5k IGxldCB5b3Uga25vdyB3aGF0IEkgd2lsbCBiZSBhYmxlIHRvIGZpbmQuCj4KPiBJZiB5b3Ugd291 bGRuJ3QgbWluZCBJIHdvdWxkIGJlIHJlZmVycmluZyB0byB5b3UgZm9yIG1vcmUgcXVlc3Rpb25z IGlmIAo+IHNvbWUgd2lsbCBjb21lIHVwIGR1cmluZyB0aGUgc2VhcmNoIGZvciB0aGUgY2F1c2Ug YW5kIEkgd2lsbCBsZXQgeW91IAo+IGtub3cgdGhlIHJlc3VsdHMgYXNhcC4KPj4KPj4gdGhhbmtz LAo+PiBqb25hdGhhbgo+Pgo+Cj4gLS0gCj4gUmVnYXJkcywKPgo+IFZpbnplbnogRmVlbnN0cmEg fCBTZW5pb3IgU29mdHdhcmUgRW5naW5lZXIKPiBSZWRIYXQgRW5naW5lZXJpbmcgVmlydHVhbGl6 YXRpb24gUiAmIEQKPiBQaG9uZTogKzQyMCA1MzIgMjk0IDYyNQo+IElSQzogdmZlZW5zdHIgb3Ig ZXZpbGlzc2ltbwo+Cj4gQmV0dGVyIHRlY2hub2xvZ3kuIEZhc3RlciBpbm5vdmF0aW9uLiBQb3dl cmVkIGJ5IGNvbW11bml0eSBjb2xsYWJvcmF0aW9uLgo+IFNlZSBob3cgaXQgd29ya3MgYXQgcmVk aGF0LmNvbQo+Cj4KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwo+IFVzZXJzIG1haWxpbmcgbGlzdAo+IFVzZXJzQG92aXJ0Lm9yZwo+IGh0dHA6Ly9saXN0 cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycwoKCi0tIApSZWdhcmRzLAoKVmluemVu eiBGZWVuc3RyYSB8IFNlbmlvciBTb2Z0d2FyZSBFbmdpbmVlcgpSZWRIYXQgRW5naW5lZXJpbmcg VmlydHVhbGl6YXRpb24gUiAmIEQKUGhvbmU6ICs0MjAgNTMyIDI5NCA2MjUKSVJDOiB2ZmVlbnN0 ciBvciBldmlsaXNzaW1vCgpCZXR0ZXIgdGVjaG5vbG9neS4gRmFzdGVyIGlubm92YXRpb24uIFBv d2VyZWQgYnkgY29tbXVuaXR5IGNvbGxhYm9yYXRpb24uClNlZSBob3cgaXQgd29ya3MgYXQgcmVk aGF0LmNvbQoKCi0tLS0tLS0tLS0tLS0tMDIwNjA0MDAwNDA1MDQwMjAzMDEwOTAxCkNvbnRlbnQt VHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PUlTTy04ODU5LTEKQ29udGVudC1UcmFuc2Zlci1FbmNv ZGluZzogN2JpdAoKPGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBjb250ZW50PSJ0ZXh0L2h0bWw7 IGNoYXJzZXQ9SVNPLTg4NTktMSIKICAgICAgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICA8 L2hlYWQ+CiAgPGJvZHkgYmdjb2xvcj0iI0ZGRkZGRiIgdGV4dD0iIzAwMDAwMCI+CiAgICA8ZGl2 IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPkhpIEpvbmF0aGFuLDxicj4KICAgICAgPGJyPgogICAg ICBDb3VsZCB5b3UgcHJvdmlkZSB0aGUgbG9ncyBmb3IgaXQ/PGJyPgogICAgICBUaGFua3MuPGJy PgogICAgICA8YnI+CiAgICAgIE9uIDAxLzA0LzIwMTMgMTA6MzIgQU0sIFZpbnplbnogRmVlbnN0 cmEgd3JvdGU6PGJyPgogICAgPC9kaXY+CiAgICA8YmxvY2txdW90ZSBjaXRlPSJtaWQ6NTBFNkEx OTkuODA1MDYwNEByZWRoYXQuY29tIiB0eXBlPSJjaXRlIj4KICAgICAgPG1ldGEgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PUlTTy04ODU5LTEiCiAgICAgICAgaHR0cC1lcXVpdj0iQ29udGVu dC1UeXBlIj4KICAgICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5IaSw8YnI+CiAgICAg ICAgPGJyPgogICAgICAgIE9uIDAxLzAzLzIwMTMgMDg6NTMgUE0sIEpvbmF0aGFuIEhvcm5lIHdy b3RlOjxicj4KICAgICAgPC9kaXY+CiAgICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDo5QkU2RjQ5 M0Y4M0E1OTREQTYwQzQ1RTZBMDlEQzVBQzAxNzAxRTZDQEFVU1AwMURBRzAyMDEuY29sbGFib3Jh dGlvbmhvc3QubmV0IgogICAgICAgIHR5cGU9ImNpdGUiPgogICAgICAgIDxtZXRhIGh0dHAtZXF1 aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0idGV4dC9odG1sOwogICAgICAgICAgY2hhcnNldD1J U08tODg1OS0xIj4KICAgICAgICA8ZGl2PgogICAgICAgICAgPGRpdj4KICAgICAgICAgICAgPGRp dj5pIGhhdmUgMiBub2RlcyBub3cgdGhhdCBoYXZlIGJvdGggbG9zdCBjb250YWN0IHdpdGggdGhl CiAgICAgICAgICAgICAgcmhldiBhZ2VudHMgb24gZ3Vlc3QgVk1zLiAmbmJzcDtpZiBpIG1pZ3Jh dGUgdGhlIGd1ZXN0IHRvCiAgICAgICAgICAgICAgYW5vdGhlciBub2RlLCB0aGUgSVAgYWRkcmVz cyBhbmQgbWVtb3J5IHVzYWdlIGltbWVkaWF0ZWx5CiAgICAgICAgICAgICAgc2hvdyB1cC4gJm5i c3A7bWlncmF0ZSBhIGd1ZXN0IGJhY2ssIGFuZCB0aGUgSVAvbWVtb3J5IGluZm8KICAgICAgICAg ICAgICBkaXNhcHBlYXJzLjwvZGl2PgogICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAg PC9kaXY+CiAgICAgICAgICAgIDxkaXY+aG93IGNhbiBpIHRyb3VibGVzaG9vdCB3aGF0IGlzIGNh dXNpbmcgdGhpcz88L2Rpdj4KICAgICAgICAgIDwvZGl2PgogICAgICAgIDwvZGl2PgogICAgICA8 L2Jsb2NrcXVvdGU+CiAgICAgIDxicj4KICAgICAgSXQnZCBiZSBuaWNlIGlmIHlvdSBjb3VsZCBz ZW5kIG1lIHRoZSBsb2dzIGZyb20KICAgICAgL3Zhci9sb2cvdmRzbS92ZHNtKiB3aGljaCBhcmUg b24gdGhlIG5vZGUgd2hlcmUgaXQgbG9vc2VzIHRoZQogICAgICBjb250YWN0LiBJIGFtIGN1cnJl bnRseSB0cnlpbmcgdG8gZmluZCBleGFjdGx5IHRoZSBjYXVzZSBvZiBzdWNoCiAgICAgIGFuIGlz c3VlIGFzIHdlbGwgYW5kIGl0IHdvdWxkIGJlIGhlbHBmdWwgaWYgd2UgY291bGQgYmUgcHJvdmlk ZWQKICAgICAgd2l0aCBsb2cgZmlsZXMuIEN1cnJlbnRseSBJIGFtIHN0aWxsIGxhY2tpbmcgdGhl bS48YnI+CiAgICAgIEkgd2lsbCBjaGVjayB0aGVtIGFuZCBsZXQgeW91IGtub3cgd2hhdCBJIHdp bGwgYmUgYWJsZSB0byBmaW5kLjxicj4KICAgICAgPGJyPgogICAgICBJZiB5b3Ugd291bGRuJ3Qg bWluZCBJIHdvdWxkIGJlIHJlZmVycmluZyB0byB5b3UgZm9yIG1vcmUKICAgICAgcXVlc3Rpb25z IGlmIHNvbWUgd2lsbCBjb21lIHVwIGR1cmluZyB0aGUgc2VhcmNoIGZvciB0aGUgY2F1c2UgYW5k CiAgICAgIEkgd2lsbCBsZXQgeW91IGtub3cgdGhlIHJlc3VsdHMgYXNhcC48YnI+CiAgICAgIDxi bG9ja3F1b3RlCmNpdGU9Im1pZDo5QkU2RjQ5M0Y4M0E1OTREQTYwQzQ1RTZBMDlEQzVBQzAxNzAx RTZDQEFVU1AwMURBRzAyMDEuY29sbGFib3JhdGlvbmhvc3QubmV0IgogICAgICAgIHR5cGU9ImNp dGUiPgogICAgICAgIDxkaXY+CiAgICAgICAgICA8ZGl2PgogICAgICAgICAgICA8ZGl2Pjxicj4K ICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+dGhhbmtzLDwvZGl2PgogICAgICAg ICAgICA8ZGl2PmpvbmF0aGFuPC9kaXY+CiAgICAgICAgICAgIDxkaXY+IDwvZGl2PgogICAgICAg ICAgPC9kaXY+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGJyPgogICAgICA8L2Jsb2NrcXVvdGU+ CiAgICAgIDxicj4KICAgICAgPHByZSBjbGFzcz0ibW96LXNpZ25hdHVyZSIgY29scz0iNzIiPi0t IApSZWdhcmRzLAoKVmluemVueiBGZWVuc3RyYSB8IFNlbmlvciBTb2Z0d2FyZSBFbmdpbmVlcgpS ZWRIYXQgRW5naW5lZXJpbmcgVmlydHVhbGl6YXRpb24gUiAmYW1wOyBEClBob25lOiArNDIwIDUz MiAyOTQgNjI1CklSQzogdmZlZW5zdHIgb3IgZXZpbGlzc2ltbwoKQmV0dGVyIHRlY2hub2xvZ3ku IEZhc3RlciBpbm5vdmF0aW9uLiBQb3dlcmVkIGJ5IGNvbW11bml0eSBjb2xsYWJvcmF0aW9uLgpT ZWUgaG93IGl0IHdvcmtzIGF0IHJlZGhhdC5jb208L3ByZT4KICAgICAgPGJyPgogICAgICA8Zmll bGRzZXQgY2xhc3M9Im1pbWVBdHRhY2htZW50SGVhZGVyIj48L2ZpZWxkc2V0PgogICAgICA8YnI+ CiAgICAgIDxwcmUgd3JhcD0iIj5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpVc2VycyBtYWlsaW5nIGxpc3QKPGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJy ZXZpYXRlZCIgaHJlZj0ibWFpbHRvOlVzZXJzQG92aXJ0Lm9yZyI+VXNlcnNAb3ZpcnQub3JnPC9h Pgo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJodHRwOi8vbGlzdHMub3Zp cnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMiPmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFp bG1hbi9saXN0aW5mby91c2VyczwvYT4KPC9wcmU+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+ CiAgICA8YnI+CiAgICA8cHJlIGNsYXNzPSJtb3otc2lnbmF0dXJlIiBjb2xzPSI3MiI+LS0gClJl Z2FyZHMsCgpWaW56ZW56IEZlZW5zdHJhIHwgU2VuaW9yIFNvZnR3YXJlIEVuZ2luZWVyClJlZEhh dCBFbmdpbmVlcmluZyBWaXJ0dWFsaXphdGlvbiBSICZhbXA7IEQKUGhvbmU6ICs0MjAgNTMyIDI5 NCA2MjUKSVJDOiB2ZmVlbnN0ciBvciBldmlsaXNzaW1vCgpCZXR0ZXIgdGVjaG5vbG9neS4gRmFz dGVyIGlubm92YXRpb24uIFBvd2VyZWQgYnkgY29tbXVuaXR5IGNvbGxhYm9yYXRpb24uClNlZSBo b3cgaXQgd29ya3MgYXQgcmVkaGF0LmNvbTwvcHJlPgogIDwvYm9keT4KPC9odG1sPgoKLS0tLS0t LS0tLS0tLS0wMjA2MDQwMDA0MDUwNDAyMDMwMTA5MDEtLQo= --===============1373177785782240019==-- From jhorne at skopos.us Mon Jan 14 10:37:00 2013 Content-Type: multipart/mixed; boundary="===============4544593229980092146==" MIME-Version: 1.0 From: Jonathan Horne To: users at ovirt.org Subject: Re: [Users] node loses contact with rhev agent Date: Mon, 14 Jan 2013 15:36:57 +0000 Message-ID: <9BE6F493F83A594DA60C45E6A09DC5AC0171A20A@AUSP01DAG0203> In-Reply-To: 50EABD65.5040902@redhat.com --===============4544593229980092146== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable --_000_9BE6F493F83A594DA60C45E6A09DC5AC0171A20AAUSP01DAG0203_ Content-Type: text/plain; charset=3D"us-ascii" Content-Transfer-Encoding: quoted-printable Its happening again, what logs do I need to provide? From: Vinzenz Feenstra [mailto:vfeenstr(a)redhat.com] Sent: Monday, January 07, 2013 6:20 AM To: users(a)ovirt.org Cc: Jonathan Horne Subject: Re: [Users] node loses contact with rhev agent Hi Jonathan, Could you provide the logs for it? Thanks. On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote: Hi, On 01/03/2013 08:53 PM, Jonathan Horne wrote: i have 2 nodes now that have both lost contact with the rhev agents on gues= =3D t VMs. if i migrate the guest to another node, the IP address and memory u= =3D sage immediately show up. migrate a guest back, and the IP/memory info dis= =3D appears. how can i troubleshoot what is causing this? It'd be nice if you could send me the logs from /var/log/vdsm/vdsm* which a= =3D re on the node where it looses the contact. I am currently trying to find e= =3D xactly the cause of such an issue as well and it would be helpful if we cou= =3D ld be provided with log files. Currently I am still lacking them. I will check them and let you know what I will be able to find. If you wouldn't mind I would be referring to you for more questions if some= =3D will come up during the search for the cause and I will let you know the r= =3D esults asap. thanks, jonathan -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ________________________________ This is a PRIVATE message. If you are not the intended recipient, please de= =3D lete without copying and kindly advise us by e-mail of the mistake in deliv= =3D ery. NOTE: Regardless of content, this e-mail shall not operate to bind SKO= =3D POS to any order or other contract unless pursuant to explicit written agre= =3D ement or government initiative expressly permitting the use of e-mail for s= =3D uch purpose. --_000_9BE6F493F83A594DA60C45E6A09DC5AC0171A20AAUSP01DAG0203_ Content-Type: text/html; charset=3D"us-ascii" Content-Transfer-Encoding: quoted-printable

Its happening again, wh= =3D at logs do I need to provide?

From: Vinzenz Feenstra [mailto:vfeenstr(a)redhat.co= m]
Sent: Monday, January 07, 2013 6:20 AM
To: users(a)ovirt.org
Cc: Jonathan Horne
Subject: Re: [Users] node loses contact with rhev agent

Hi Jonathan,

Could you provide the logs for it?
Thanks.

On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote:

Hi,

On 01/03/2013 08:53 PM, Jonathan Horne wrote:

i have 2 nodes now that have both lost contact wit= h =3D the rhev agents on guest VMs. if i migrate the guest to another node,= =3D the IP address and memory usage immediately show up. migrate a guest= =3D back, and the IP/memory info disappears.

how can i troubleshoot what is causing this?

It'd be nice if you could send me the logs from /var/log/vdsm/vdsm* which a= =3D re on the node where it looses the contact. I am currently trying to find e= =3D xactly the cause of such an issue as well and it would be helpful if we cou= =3D ld be provided with log files. Currently I am still lacking them.
I will check them and let you know what I will be able to find.

If you wouldn't mind I would be referring to you for more questions if some= =3D will come up during the search for the cause and I will let you know the r= =3D esults asap.

thanks,

jonathan
-- 
Regards,
 
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
 
Better technology. Faster innovation. Powered by community collaborati=
=3D
on.
See how it works at redhat.com
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lis=
ts=3D
.ovirt.org/mailman/listinfo/users

--

Regards,

Vinzenz Feenstra | Senior Software Engineer

RedHat Engineering Virtualization R & D

Phone: +420 532 294 625

IRC: vfeenstr or evilissimo

Better technology. Faster innovation. Powered by community collaborati=
=3D
on.

See how it works at redhat.com

This is a PRIVATE mess= age. I=3D f you are not the intended recipient, please delete without copying and kin= =3D dly advise us by e-mail of the mistake in delivery. NOTE: Regardless of con= =3D tent, this e-mail shall not operate to bind SKOPOS to any order or other contract unless pursuant to explicit wri= =3D tten agreement or government initiative expressly permitting the use of e-m= =3D ail for such purpose. --_000_9BE6F493F83A594DA60C45E6A09DC5AC0171A20AAUSP01DAG0203_-- --===============4544593229980092146== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" LS1fMDAwXzlCRTZGNDkzRjgzQTU5NERBNjBDNDVFNkEwOURDNUFDMDE3MUEyMEFBVVNQMDFEQUcw MjAzXwpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9InVzLWFzY2lpIgpDb250ZW50 LVRyYW5zZmVyLUVuY29kaW5nOiBxdW90ZWQtcHJpbnRhYmxlCgpJdHMgaGFwcGVuaW5nIGFnYWlu LCB3aGF0IGxvZ3MgZG8gSSBuZWVkIHRvIHByb3ZpZGU/CgpGcm9tOiBWaW56ZW56IEZlZW5zdHJh IFttYWlsdG86dmZlZW5zdHJAcmVkaGF0LmNvbV0KU2VudDogTW9uZGF5LCBKYW51YXJ5IDA3LCAy MDEzIDY6MjAgQU0KVG86IHVzZXJzQG92aXJ0Lm9yZwpDYzogSm9uYXRoYW4gSG9ybmUKU3ViamVj dDogUmU6IFtVc2Vyc10gbm9kZSBsb3NlcyBjb250YWN0IHdpdGggcmhldiBhZ2VudAoKSGkgSm9u YXRoYW4sCgpDb3VsZCB5b3UgcHJvdmlkZSB0aGUgbG9ncyBmb3IgaXQ/ClRoYW5rcy4KCk9uIDAx LzA0LzIwMTMgMTA6MzIgQU0sIFZpbnplbnogRmVlbnN0cmEgd3JvdGU6CkhpLAoKT24gMDEvMDMv MjAxMyAwODo1MyBQTSwgSm9uYXRoYW4gSG9ybmUgd3JvdGU6CmkgaGF2ZSAyIG5vZGVzIG5vdyB0 aGF0IGhhdmUgYm90aCBsb3N0IGNvbnRhY3Qgd2l0aCB0aGUgcmhldiBhZ2VudHMgb24gZ3Vlcz0K dCBWTXMuICBpZiBpIG1pZ3JhdGUgdGhlIGd1ZXN0IHRvIGFub3RoZXIgbm9kZSwgdGhlIElQIGFk ZHJlc3MgYW5kIG1lbW9yeSB1PQpzYWdlIGltbWVkaWF0ZWx5IHNob3cgdXAuICBtaWdyYXRlIGEg Z3Vlc3QgYmFjaywgYW5kIHRoZSBJUC9tZW1vcnkgaW5mbyBkaXM9CmFwcGVhcnMuCgpob3cgY2Fu IGkgdHJvdWJsZXNob290IHdoYXQgaXMgY2F1c2luZyB0aGlzPwoKSXQnZCBiZSBuaWNlIGlmIHlv dSBjb3VsZCBzZW5kIG1lIHRoZSBsb2dzIGZyb20gL3Zhci9sb2cvdmRzbS92ZHNtKiB3aGljaCBh PQpyZSBvbiB0aGUgbm9kZSB3aGVyZSBpdCBsb29zZXMgdGhlIGNvbnRhY3QuIEkgYW0gY3VycmVu dGx5IHRyeWluZyB0byBmaW5kIGU9CnhhY3RseSB0aGUgY2F1c2Ugb2Ygc3VjaCBhbiBpc3N1ZSBh cyB3ZWxsIGFuZCBpdCB3b3VsZCBiZSBoZWxwZnVsIGlmIHdlIGNvdT0KbGQgYmUgcHJvdmlkZWQg d2l0aCBsb2cgZmlsZXMuIEN1cnJlbnRseSBJIGFtIHN0aWxsIGxhY2tpbmcgdGhlbS4KSSB3aWxs IGNoZWNrIHRoZW0gYW5kIGxldCB5b3Uga25vdyB3aGF0IEkgd2lsbCBiZSBhYmxlIHRvIGZpbmQu CgpJZiB5b3Ugd291bGRuJ3QgbWluZCBJIHdvdWxkIGJlIHJlZmVycmluZyB0byB5b3UgZm9yIG1v cmUgcXVlc3Rpb25zIGlmIHNvbWU9CiB3aWxsIGNvbWUgdXAgZHVyaW5nIHRoZSBzZWFyY2ggZm9y IHRoZSBjYXVzZSBhbmQgSSB3aWxsIGxldCB5b3Uga25vdyB0aGUgcj0KZXN1bHRzIGFzYXAuCgoK dGhhbmtzLApqb25hdGhhbgoKCgoKLS0KClJlZ2FyZHMsCgoKClZpbnplbnogRmVlbnN0cmEgfCBT ZW5pb3IgU29mdHdhcmUgRW5naW5lZXIKClJlZEhhdCBFbmdpbmVlcmluZyBWaXJ0dWFsaXphdGlv biBSICYgRAoKUGhvbmU6ICs0MjAgNTMyIDI5NCA2MjUKCklSQzogdmZlZW5zdHIgb3IgZXZpbGlz c2ltbwoKCgpCZXR0ZXIgdGVjaG5vbG9neS4gRmFzdGVyIGlubm92YXRpb24uIFBvd2VyZWQgYnkg Y29tbXVuaXR5IGNvbGxhYm9yYXRpb24uCgpTZWUgaG93IGl0IHdvcmtzIGF0IHJlZGhhdC5jb20K CgoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCgpVc2Vy cyBtYWlsaW5nIGxpc3QKClVzZXJzQG92aXJ0Lm9yZzxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPgoK aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzCgoKCgotLQoKUmVn YXJkcywKCgoKVmluemVueiBGZWVuc3RyYSB8IFNlbmlvciBTb2Z0d2FyZSBFbmdpbmVlcgoKUmVk SGF0IEVuZ2luZWVyaW5nIFZpcnR1YWxpemF0aW9uIFIgJiBECgpQaG9uZTogKzQyMCA1MzIgMjk0 IDYyNQoKSVJDOiB2ZmVlbnN0ciBvciBldmlsaXNzaW1vCgoKCkJldHRlciB0ZWNobm9sb2d5LiBG YXN0ZXIgaW5ub3ZhdGlvbi4gUG93ZXJlZCBieSBjb21tdW5pdHkgY29sbGFib3JhdGlvbi4KClNl ZSBob3cgaXQgd29ya3MgYXQgcmVkaGF0LmNvbQoKX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KVGhpcyBpcyBhIFBSSVZBVEUgbWVzc2FnZS4gSWYgeW91IGFyZSBub3QgdGhlIGludGVu ZGVkIHJlY2lwaWVudCwgcGxlYXNlIGRlPQpsZXRlIHdpdGhvdXQgY29weWluZyBhbmQga2luZGx5 IGFkdmlzZSB1cyBieSBlLW1haWwgb2YgdGhlIG1pc3Rha2UgaW4gZGVsaXY9CmVyeS4gTk9URTog UmVnYXJkbGVzcyBvZiBjb250ZW50LCB0aGlzIGUtbWFpbCBzaGFsbCBub3Qgb3BlcmF0ZSB0byBi aW5kIFNLTz0KUE9TIHRvIGFueSBvcmRlciBvciBvdGhlciBjb250cmFjdCB1bmxlc3MgcHVyc3Vh bnQgdG8gZXhwbGljaXQgd3JpdHRlbiBhZ3JlPQplbWVudCBvciBnb3Zlcm5tZW50IGluaXRpYXRp dmUgZXhwcmVzc2x5IHBlcm1pdHRpbmcgdGhlIHVzZSBvZiBlLW1haWwgZm9yIHM9CnVjaCBwdXJw b3NlLgoKLS1fMDAwXzlCRTZGNDkzRjgzQTU5NERBNjBDNDVFNkEwOURDNUFDMDE3MUEyMEFBVVNQ MDFEQUcwMjAzXwpDb250ZW50LVR5cGU6IHRleHQvaHRtbDsgY2hhcnNldD0idXMtYXNjaWkiCkNv bnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IHF1b3RlZC1wcmludGFibGUKCjxodG1sPgo8aGVhZD4K PG1ldGEgaHR0cC1lcXVpdj0zRCJDb250ZW50LVR5cGUiIGNvbnRlbnQ9M0QidGV4dC9odG1sOyBj aGFyc2V0PTNEdXMtYXNjaWkiPQo+CjxzdHlsZT4KPCEtLQpAZm9udC1mYWNlCgl7Zm9udC1mYW1p bHk6Q2FsaWJyaX0KQGZvbnQtZmFjZQoJe2ZvbnQtZmFtaWx5OlRhaG9tYX0KQGZvbnQtZmFjZQoJ e2ZvbnQtZmFtaWx5OkNvbnNvbGFzfQpwLk1zb05vcm1hbCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNv Tm9ybWFsCgl7bWFyZ2luOjBpbjsKCW1hcmdpbi1ib3R0b206LjAwMDFwdDsKCWZvbnQtc2l6ZTox Mi4wcHQ7Cglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIiwic2VyaWYiOwoJY29sb3I6Ymxh Y2t9CmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsKCXtjb2xvcjpibHVlOwoJdGV4dC1kZWNvcmF0 aW9uOnVuZGVybGluZX0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkCgl7Y29s b3I6cHVycGxlOwoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZX0KcHJlCgl7bWFyZ2luOjBpbjsK CW1hcmdpbi1ib3R0b206LjAwMDFwdDsKCWZvbnQtc2l6ZToxMC4wcHQ7Cglmb250LWZhbWlseToi Q291cmllciBOZXciOwoJY29sb3I6YmxhY2t9CnNwYW4uSFRNTFByZWZvcm1hdHRlZENoYXIKCXtm b250LWZhbWlseToiQ29uc29sYXMiLCJzZXJpZiI7Cgljb2xvcjpibGFja30Kc3Bhbi5FbWFpbFN0 eWxlMTkKCXtmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiOwoJY29sb3I6IzFGNDk3 RH0KLk1zb0NocERlZmF1bHQKCXtmb250LXNpemU6MTAuMHB0fQpAcGFnZSBXb3JkU2VjdGlvbjEK CXttYXJnaW46MS4waW4gMS4waW4gMS4waW4gMS4waW59CmRpdi5Xb3JkU2VjdGlvbjEKCXt9Ci0t Pgo8L3N0eWxlPgo8L2hlYWQ+Cjxib2R5IGJnY29sb3I9M0Qid2hpdGUiIGxhbmc9M0QiRU4tVVMi IGxpbms9M0QiYmx1ZSIgdmxpbms9M0QicHVycGxlIj4KPGRpdiBjbGFzcz0zRCJXb3JkU2VjdGlv bjEiPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPTNEImZvbnQtc2l6ZToxMS4w cHQ7IGZvbnQtZmFtaWx5OiZxdW90O0M9CmFsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1 b3Q7OyBjb2xvcjojMUY0OTdEIj5JdHMgaGFwcGVuaW5nIGFnYWluLCB3aD0KYXQgbG9ncyBkbyBJ IG5lZWQgdG8gcHJvdmlkZT88L3NwYW4+PC9wPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxzcGFu IHN0eWxlPTNEImZvbnQtc2l6ZToxMS4wcHQ7IGZvbnQtZmFtaWx5OiZxdW90O0M9CmFsaWJyaSZx dW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7OyBjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+ PC9wPgo8ZGl2Pgo8ZGl2IHN0eWxlPTNEImJvcmRlcjpub25lOyBib3JkZXItdG9wOnNvbGlkICNC NUM0REYgMS4wcHQ7IHBhZGRpbmc6My4wcHQgMGk9Cm4gMGluIDBpbiI+CjxwIGNsYXNzPTNEIk1z b05vcm1hbCI+PGI+PHNwYW4gc3R5bGU9M0QiZm9udC1zaXplOjEwLjBwdDsgZm9udC1mYW1pbHk6 JnF1bz0KdDtUYWhvbWEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90OzsgY29sb3I6d2luZG93 dGV4dCI+RnJvbTo8L3NwYW4+PC9iPjxzPQpwYW4gc3R5bGU9M0QiZm9udC1zaXplOjEwLjBwdDsg Zm9udC1mYW1pbHk6JnF1b3Q7VGFob21hJnF1b3Q7LCZxdW90O3NhbnMtc2U9CnJpZiZxdW90Ozsg Y29sb3I6d2luZG93dGV4dCI+IFZpbnplbnogRmVlbnN0cmEgW21haWx0bzp2ZmVlbnN0ckByZWRo YXQuY29tXQo8YnI+CjxiPlNlbnQ6PC9iPiBNb25kYXksIEphbnVhcnkgMDcsIDIwMTMgNjoyMCBB TTxicj4KPGI+VG86PC9iPiB1c2Vyc0BvdmlydC5vcmc8YnI+CjxiPkNjOjwvYj4gSm9uYXRoYW4g SG9ybmU8YnI+CjxiPlN1YmplY3Q6PC9iPiBSZTogW1VzZXJzXSBub2RlIGxvc2VzIGNvbnRhY3Qg d2l0aCByaGV2IGFnZW50PC9zcGFuPjwvcD4KPC9kaXY+CjwvZGl2Pgo8cCBjbGFzcz0zRCJNc29O b3JtYWwiPiZuYnNwOzwvcD4KPGRpdj4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj5IaSBKb25hdGhh biw8YnI+Cjxicj4KQ291bGQgeW91IHByb3ZpZGUgdGhlIGxvZ3MgZm9yIGl0Pzxicj4KVGhhbmtz Ljxicj4KPGJyPgpPbiAwMS8wNC8yMDEzIDEwOjMyIEFNLCBWaW56ZW56IEZlZW5zdHJhIHdyb3Rl OjwvcD4KPC9kaXY+CjxibG9ja3F1b3RlIHN0eWxlPTNEIm1hcmdpbi10b3A6NS4wcHQ7IG1hcmdp bi1ib3R0b206NS4wcHQiPgo8ZGl2Pgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPkhpLDxicj4KPGJy PgpPbiAwMS8wMy8yMDEzIDA4OjUzIFBNLCBKb25hdGhhbiBIb3JuZSB3cm90ZTo8L3A+CjwvZGl2 Pgo8YmxvY2txdW90ZSBzdHlsZT0zRCJtYXJnaW4tdG9wOjUuMHB0OyBtYXJnaW4tYm90dG9tOjUu MHB0Ij4KPGRpdj4KPGRpdj4KPGRpdj4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj5pIGhhdmUgMiBu b2RlcyBub3cgdGhhdCBoYXZlIGJvdGggbG9zdCBjb250YWN0IHdpdGggPQp0aGUgcmhldiBhZ2Vu dHMgb24gZ3Vlc3QgVk1zLiAmbmJzcDtpZiBpIG1pZ3JhdGUgdGhlIGd1ZXN0IHRvIGFub3RoZXIg bm9kZSw9CiB0aGUgSVAgYWRkcmVzcyBhbmQgbWVtb3J5IHVzYWdlIGltbWVkaWF0ZWx5IHNob3cg dXAuICZuYnNwO21pZ3JhdGUgYSBndWVzdD0KIGJhY2ssIGFuZCB0aGUgSVAvbWVtb3J5IGluZm8g ZGlzYXBwZWFycy48L3A+CjwvZGl2Pgo8ZGl2Pgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPiZuYnNw OzwvcD4KPC9kaXY+CjxkaXY+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCI+aG93IGNhbiBpIHRyb3Vi bGVzaG9vdCB3aGF0IGlzIGNhdXNpbmcgdGhpcz88L3A+CjwvZGl2Pgo8L2Rpdj4KPC9kaXY+Cjwv YmxvY2txdW90ZT4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj48YnI+Ckl0J2QgYmUgbmljZSBpZiB5 b3UgY291bGQgc2VuZCBtZSB0aGUgbG9ncyBmcm9tIC92YXIvbG9nL3Zkc20vdmRzbSogd2hpY2gg YT0KcmUgb24gdGhlIG5vZGUgd2hlcmUgaXQgbG9vc2VzIHRoZSBjb250YWN0LiBJIGFtIGN1cnJl bnRseSB0cnlpbmcgdG8gZmluZCBlPQp4YWN0bHkgdGhlIGNhdXNlIG9mIHN1Y2ggYW4gaXNzdWUg YXMgd2VsbCBhbmQgaXQgd291bGQgYmUgaGVscGZ1bCBpZiB3ZSBjb3U9CmxkIGJlIHByb3ZpZGVk IHdpdGggbG9nIGZpbGVzLiBDdXJyZW50bHkKIEkgYW0gc3RpbGwgbGFja2luZyB0aGVtLjxicj4K SSB3aWxsIGNoZWNrIHRoZW0gYW5kIGxldCB5b3Uga25vdyB3aGF0IEkgd2lsbCBiZSBhYmxlIHRv IGZpbmQuPGJyPgo8YnI+CklmIHlvdSB3b3VsZG4ndCBtaW5kIEkgd291bGQgYmUgcmVmZXJyaW5n IHRvIHlvdSBmb3IgbW9yZSBxdWVzdGlvbnMgaWYgc29tZT0KIHdpbGwgY29tZSB1cCBkdXJpbmcg dGhlIHNlYXJjaCBmb3IgdGhlIGNhdXNlIGFuZCBJIHdpbGwgbGV0IHlvdSBrbm93IHRoZSByPQpl c3VsdHMgYXNhcC48YnI+Cjxicj4KPC9wPgo8ZGl2Pgo8ZGl2Pgo8ZGl2Pgo8cCBjbGFzcz0zRCJN c29Ob3JtYWwiPiZuYnNwOzwvcD4KPC9kaXY+CjxkaXY+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCI+ dGhhbmtzLDwvcD4KPC9kaXY+CjxkaXY+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCI+am9uYXRoYW48 L3A+CjwvZGl2Pgo8L2Rpdj4KPC9kaXY+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCI+Jm5ic3A7PC9w Pgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxicj4KPGJyPgo8L3A+CjxwcmU+LS0gPC9wcmU+Cjxw cmU+UmVnYXJkcyw8L3ByZT4KPHByZT4mbmJzcDs8L3ByZT4KPHByZT5WaW56ZW56IEZlZW5zdHJh IHwgU2VuaW9yIFNvZnR3YXJlIEVuZ2luZWVyPC9wcmU+CjxwcmU+UmVkSGF0IEVuZ2luZWVyaW5n IFZpcnR1YWxpemF0aW9uIFIgJmFtcDsgRDwvcHJlPgo8cHJlPlBob25lOiAmIzQzOzQyMCA1MzIg Mjk0IDYyNTwvcHJlPgo8cHJlPklSQzogdmZlZW5zdHIgb3IgZXZpbGlzc2ltbzwvcHJlPgo8cHJl PiZuYnNwOzwvcHJlPgo8cHJlPkJldHRlciB0ZWNobm9sb2d5LiBGYXN0ZXIgaW5ub3ZhdGlvbi4g UG93ZXJlZCBieSBjb21tdW5pdHkgY29sbGFib3JhdGk9Cm9uLjwvcHJlPgo8cHJlPlNlZSBob3cg aXQgd29ya3MgYXQgcmVkaGF0LmNvbTwvcHJlPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxicj4K PGJyPgo8YnI+CjwvcD4KPHByZT5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXzwvcHJlPgo8cHJlPlVzZXJzIG1haWxpbmcgbGlzdDwvcHJlPgo8cHJlPjxhIGhy ZWY9M0QibWFpbHRvOlVzZXJzQG92aXJ0Lm9yZyI+VXNlcnNAb3ZpcnQub3JnPC9hPjwvcHJlPgo8 cHJlPjxhIGhyZWY9M0QiaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3Vz ZXJzIj5odHRwOi8vbGlzdHM9Ci5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VyczwvYT48 L3ByZT4KPC9ibG9ja3F1b3RlPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxicj4KPGJyPgo8YnI+ CjwvcD4KPHByZT4tLSA8L3ByZT4KPHByZT5SZWdhcmRzLDwvcHJlPgo8cHJlPiZuYnNwOzwvcHJl Pgo8cHJlPlZpbnplbnogRmVlbnN0cmEgfCBTZW5pb3IgU29mdHdhcmUgRW5naW5lZXI8L3ByZT4K PHByZT5SZWRIYXQgRW5naW5lZXJpbmcgVmlydHVhbGl6YXRpb24gUiAmYW1wOyBEPC9wcmU+Cjxw cmU+UGhvbmU6ICYjNDM7NDIwIDUzMiAyOTQgNjI1PC9wcmU+CjxwcmU+SVJDOiB2ZmVlbnN0ciBv ciBldmlsaXNzaW1vPC9wcmU+CjxwcmU+Jm5ic3A7PC9wcmU+CjxwcmU+QmV0dGVyIHRlY2hub2xv Z3kuIEZhc3RlciBpbm5vdmF0aW9uLiBQb3dlcmVkIGJ5IGNvbW11bml0eSBjb2xsYWJvcmF0aT0K b24uPC9wcmU+CjxwcmU+U2VlIGhvdyBpdCB3b3JrcyBhdCByZWRoYXQuY29tPC9wcmU+CjwvZGl2 Pgo8YnI+Cjxocj4KPGZvbnQgY29sb3I9M0QiR3JheSIgZmFjZT0zRCJBcmlhbCIgc2l6ZT0zRCIx Ij5UaGlzIGlzIGEgUFJJVkFURSBtZXNzYWdlLiBJPQpmIHlvdSBhcmUgbm90IHRoZSBpbnRlbmRl ZCByZWNpcGllbnQsIHBsZWFzZSBkZWxldGUgd2l0aG91dCBjb3B5aW5nIGFuZCBraW49CmRseSBh ZHZpc2UgdXMgYnkgZS1tYWlsIG9mIHRoZSBtaXN0YWtlIGluIGRlbGl2ZXJ5LiBOT1RFOiBSZWdh cmRsZXNzIG9mIGNvbj0KdGVudCwgdGhpcyBlLW1haWwgc2hhbGwgbm90IG9wZXJhdGUgdG8KIGJp bmQgU0tPUE9TIHRvIGFueSBvcmRlciBvciBvdGhlciBjb250cmFjdCB1bmxlc3MgcHVyc3VhbnQg dG8gZXhwbGljaXQgd3JpPQp0dGVuIGFncmVlbWVudCBvciBnb3Zlcm5tZW50IGluaXRpYXRpdmUg ZXhwcmVzc2x5IHBlcm1pdHRpbmcgdGhlIHVzZSBvZiBlLW09CmFpbCBmb3Igc3VjaCBwdXJwb3Nl LjwvZm9udD4KPC9ib2R5Pgo8L2h0bWw+CgotLV8wMDBfOUJFNkY0OTNGODNBNTk0REE2MEM0NUU2 QTA5REM1QUMwMTcxQTIwQUFVU1AwMURBRzAyMDNfLS0K --===============4544593229980092146==-- From jhorne at skopos.us Mon Jan 14 11:28:21 2013 Content-Type: multipart/mixed; boundary="===============5910694068487950865==" MIME-Version: 1.0 From: Jonathan Horne To: users at ovirt.org Subject: Re: [Users] node loses contact with rhev agent Date: Mon, 14 Jan 2013 16:28:01 +0000 Message-ID: <9BE6F493F83A594DA60C45E6A09DC5AC0171A577@AUSP01DAG0203> In-Reply-To: 9BE6F493F83A594DA60C45E6A09DC5AC0171A20A@AUSP01DAG0203 --===============5910694068487950865== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable --_000_9BE6F493F83A594DA60C45E6A09DC5AC0171A577AUSP01DAG0203_ Content-Type: text/plain; charset=3D"us-ascii" Content-Transfer-Encoding: quoted-printable Ok this time instead of brutally rebuilding the node... I just rebooted tha= =3D t one node and its reading agents again. Crisis averted. From: users-bounces(a)ovirt.org [mailto:users-bounces(a)ovirt.org] On Behal= f Of=3D Jonathan Horne Sent: Monday, January 14, 2013 9:37 AM To: Vinzenz Feenstra; users(a)ovirt.org Subject: Re: [Users] node loses contact with rhev agent Its happening again, what logs do I need to provide? From: Vinzenz Feenstra [mailto:vfeenstr(a)redhat.com] Sent: Monday, January 07, 2013 6:20 AM To: users(a)ovirt.org Cc: Jonathan Horne Subject: Re: [Users] node loses contact with rhev agent Hi Jonathan, Could you provide the logs for it? Thanks. On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote: Hi, On 01/03/2013 08:53 PM, Jonathan Horne wrote: i have 2 nodes now that have both lost contact with the rhev agents on gues= =3D t VMs. if i migrate the guest to another node, the IP address and memory u= =3D sage immediately show up. migrate a guest back, and the IP/memory info dis= =3D appears. how can i troubleshoot what is causing this? It'd be nice if you could send me the logs from /var/log/vdsm/vdsm* which a= =3D re on the node where it looses the contact. I am currently trying to find e= =3D xactly the cause of such an issue as well and it would be helpful if we cou= =3D ld be provided with log files. Currently I am still lacking them. I will check them and let you know what I will be able to find. If you wouldn't mind I would be referring to you for more questions if some= =3D will come up during the search for the cause and I will let you know the r= =3D esults asap. thanks, jonathan -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ________________________________ This is a PRIVATE message. If you are not the intended recipient, please de= =3D lete without copying and kindly advise us by e-mail of the mistake in deliv= =3D ery. NOTE: Regardless of content, this e-mail shall not operate to bind SKO= =3D POS to any order or other contract unless pursuant to explicit written agre= =3D ement or government initiative expressly permitting the use of e-mail for s= =3D uch purpose. ________________________________ This is a PRIVATE message. If you are not the intended recipient, please de= =3D lete without copying and kindly advise us by e-mail of the mistake in deliv= =3D ery. NOTE: Regardless of content, this e-mail shall not operate to bind SKO= =3D POS to any order or other contract unless pursuant to explicit written agre= =3D ement or government initiative expressly permitting the use of e-mail for s= =3D uch purpose. --_000_9BE6F493F83A594DA60C45E6A09DC5AC0171A577AUSP01DAG0203_ Content-Type: text/html; charset=3D"us-ascii" Content-Transfer-Encoding: quoted-printable

Ok this time instead of= =3D brutally rebuilding the node… I just rebooted that one node and its = =3D reading agents again. Crisis averted.

From: users-bounces(a)ovirt.org [mailto:users-bounc= es=3D @ovirt.org] On Behalf Of Jonathan Horne
Sent: Monday, January 14, 2013 9:37 AM
To: Vinzenz Feenstra; users(a)ovirt.org
Subject: Re: [Users] node loses contact with rhev agent

Its happening again, wh= =3D at logs do I need to provide?

From: Vinzenz Feenstra [mailto:vfeenstr(a)redhat.com]
Sent: Monday, January 07, 2013 6:20 AM
To: users(a)ovirt.org
Cc: Jonathan Horne
Subject: Re: [Users] node loses contact with rhev agent

Hi Jonathan,

Could you provide the logs for it?
Thanks.

On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote:

Hi,

On 01/03/2013 08:53 PM, Jonathan Horne wrote:

i have 2 nodes now that have both lost contact wit= h =3D the rhev agents on guest VMs. if i migrate the guest to another node,= =3D the IP address and memory usage immediately show up. migrate a guest= =3D back, and the IP/memory info disappears.

how can i troubleshoot what is causing this?

It'd be nice if you could send me the logs from /var/log/vdsm/vdsm* which a= =3D re on the node where it looses the contact. I am currently trying to find e= =3D xactly the cause of such an issue as well and it would be helpful if we cou= =3D ld be provided with log files. Currently I am still lacking them.
I will check them and let you know what I will be able to find.

If you wouldn't mind I would be referring to you for more questions if some= =3D will come up during the search for the cause and I will let you know the r= =3D esults asap.

thanks,

jonathan
-- 
Regards,
 
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
 
Better technology. Faster innovation. Powered by community collaborati=
=3D
on.
See how it works at redhat.com
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lis=
ts=3D
.ovirt.org/mailman/listinfo/users

--

Regards,

Vinzenz Feenstra | Senior Software Engineer

RedHat Engineering Virtualization R & D

Phone: +420 532 294 625

IRC: vfeenstr or evilissimo

Better technology. Faster innovation. Powered by community collaborati=
=3D
on.

See how it works at redhat.com

This is a PRIVATE message. If= =3D you are not the intended recipient, please delete without copying and kind= =3D ly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind SKOPOS to any= =3D order or other contract unless pursuant to explicit written agreement or g= =3D overnment initiative expressly permitting the use of e-mail for such purpos= =3D e.

This is a PRIVATE mess= age. I=3D f you are not the intended recipient, please delete without copying and kin= =3D dly advise us by e-mail of the mistake in delivery. NOTE: Regardless of con= =3D tent, this e-mail shall not operate to bind SKOPOS to any order or other contract unless pursuant to explicit wri= =3D tten agreement or government initiative expressly permitting the use of e-m= =3D ail for such purpose. --_000_9BE6F493F83A594DA60C45E6A09DC5AC0171A577AUSP01DAG0203_-- --===============5910694068487950865== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" LS1fMDAwXzlCRTZGNDkzRjgzQTU5NERBNjBDNDVFNkEwOURDNUFDMDE3MUE1NzdBVVNQMDFEQUcw MjAzXwpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9InVzLWFzY2lpIgpDb250ZW50 LVRyYW5zZmVyLUVuY29kaW5nOiBxdW90ZWQtcHJpbnRhYmxlCgpPayB0aGlzIHRpbWUgaW5zdGVh ZCBvZiBicnV0YWxseSByZWJ1aWxkaW5nIHRoZSBub2RlLi4uIEkganVzdCByZWJvb3RlZCB0aGE9 CnQgb25lIG5vZGUgYW5kIGl0cyByZWFkaW5nIGFnZW50cyBhZ2Fpbi4gIENyaXNpcyBhdmVydGVk LgoKRnJvbTogdXNlcnMtYm91bmNlc0BvdmlydC5vcmcgW21haWx0bzp1c2Vycy1ib3VuY2VzQG92 aXJ0Lm9yZ10gT24gQmVoYWxmIE9mPQogSm9uYXRoYW4gSG9ybmUKU2VudDogTW9uZGF5LCBKYW51 YXJ5IDE0LCAyMDEzIDk6MzcgQU0KVG86IFZpbnplbnogRmVlbnN0cmE7IHVzZXJzQG92aXJ0Lm9y ZwpTdWJqZWN0OiBSZTogW1VzZXJzXSBub2RlIGxvc2VzIGNvbnRhY3Qgd2l0aCByaGV2IGFnZW50 CgpJdHMgaGFwcGVuaW5nIGFnYWluLCB3aGF0IGxvZ3MgZG8gSSBuZWVkIHRvIHByb3ZpZGU/CgpG cm9tOiBWaW56ZW56IEZlZW5zdHJhIFttYWlsdG86dmZlZW5zdHJAcmVkaGF0LmNvbV0KU2VudDog TW9uZGF5LCBKYW51YXJ5IDA3LCAyMDEzIDY6MjAgQU0KVG86IHVzZXJzQG92aXJ0Lm9yZzxtYWls dG86dXNlcnNAb3ZpcnQub3JnPgpDYzogSm9uYXRoYW4gSG9ybmUKU3ViamVjdDogUmU6IFtVc2Vy c10gbm9kZSBsb3NlcyBjb250YWN0IHdpdGggcmhldiBhZ2VudAoKSGkgSm9uYXRoYW4sCgpDb3Vs ZCB5b3UgcHJvdmlkZSB0aGUgbG9ncyBmb3IgaXQ/ClRoYW5rcy4KCk9uIDAxLzA0LzIwMTMgMTA6 MzIgQU0sIFZpbnplbnogRmVlbnN0cmEgd3JvdGU6CkhpLAoKT24gMDEvMDMvMjAxMyAwODo1MyBQ TSwgSm9uYXRoYW4gSG9ybmUgd3JvdGU6CmkgaGF2ZSAyIG5vZGVzIG5vdyB0aGF0IGhhdmUgYm90 aCBsb3N0IGNvbnRhY3Qgd2l0aCB0aGUgcmhldiBhZ2VudHMgb24gZ3Vlcz0KdCBWTXMuICBpZiBp IG1pZ3JhdGUgdGhlIGd1ZXN0IHRvIGFub3RoZXIgbm9kZSwgdGhlIElQIGFkZHJlc3MgYW5kIG1l bW9yeSB1PQpzYWdlIGltbWVkaWF0ZWx5IHNob3cgdXAuICBtaWdyYXRlIGEgZ3Vlc3QgYmFjaywg YW5kIHRoZSBJUC9tZW1vcnkgaW5mbyBkaXM9CmFwcGVhcnMuCgpob3cgY2FuIGkgdHJvdWJsZXNo b290IHdoYXQgaXMgY2F1c2luZyB0aGlzPwoKSXQnZCBiZSBuaWNlIGlmIHlvdSBjb3VsZCBzZW5k IG1lIHRoZSBsb2dzIGZyb20gL3Zhci9sb2cvdmRzbS92ZHNtKiB3aGljaCBhPQpyZSBvbiB0aGUg bm9kZSB3aGVyZSBpdCBsb29zZXMgdGhlIGNvbnRhY3QuIEkgYW0gY3VycmVudGx5IHRyeWluZyB0 byBmaW5kIGU9CnhhY3RseSB0aGUgY2F1c2Ugb2Ygc3VjaCBhbiBpc3N1ZSBhcyB3ZWxsIGFuZCBp dCB3b3VsZCBiZSBoZWxwZnVsIGlmIHdlIGNvdT0KbGQgYmUgcHJvdmlkZWQgd2l0aCBsb2cgZmls ZXMuIEN1cnJlbnRseSBJIGFtIHN0aWxsIGxhY2tpbmcgdGhlbS4KSSB3aWxsIGNoZWNrIHRoZW0g YW5kIGxldCB5b3Uga25vdyB3aGF0IEkgd2lsbCBiZSBhYmxlIHRvIGZpbmQuCgpJZiB5b3Ugd291 bGRuJ3QgbWluZCBJIHdvdWxkIGJlIHJlZmVycmluZyB0byB5b3UgZm9yIG1vcmUgcXVlc3Rpb25z IGlmIHNvbWU9CiB3aWxsIGNvbWUgdXAgZHVyaW5nIHRoZSBzZWFyY2ggZm9yIHRoZSBjYXVzZSBh bmQgSSB3aWxsIGxldCB5b3Uga25vdyB0aGUgcj0KZXN1bHRzIGFzYXAuCgp0aGFua3MsCmpvbmF0 aGFuCgoKCi0tCgpSZWdhcmRzLAoKCgpWaW56ZW56IEZlZW5zdHJhIHwgU2VuaW9yIFNvZnR3YXJl IEVuZ2luZWVyCgpSZWRIYXQgRW5naW5lZXJpbmcgVmlydHVhbGl6YXRpb24gUiAmIEQKClBob25l OiArNDIwIDUzMiAyOTQgNjI1CgpJUkM6IHZmZWVuc3RyIG9yIGV2aWxpc3NpbW8KCgoKQmV0dGVy IHRlY2hub2xvZ3kuIEZhc3RlciBpbm5vdmF0aW9uLiBQb3dlcmVkIGJ5IGNvbW11bml0eSBjb2xs YWJvcmF0aW9uLgoKU2VlIGhvdyBpdCB3b3JrcyBhdCByZWRoYXQuY29tCgoKCl9fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCgpVc2VycyBtYWlsaW5nIGxpc3QK ClVzZXJzQG92aXJ0Lm9yZzxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPgoKaHR0cDovL2xpc3RzLm92 aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzCgoKCi0tCgpSZWdhcmRzLAoKCgpWaW56ZW56 IEZlZW5zdHJhIHwgU2VuaW9yIFNvZnR3YXJlIEVuZ2luZWVyCgpSZWRIYXQgRW5naW5lZXJpbmcg VmlydHVhbGl6YXRpb24gUiAmIEQKClBob25lOiArNDIwIDUzMiAyOTQgNjI1CgpJUkM6IHZmZWVu c3RyIG9yIGV2aWxpc3NpbW8KCgoKQmV0dGVyIHRlY2hub2xvZ3kuIEZhc3RlciBpbm5vdmF0aW9u LiBQb3dlcmVkIGJ5IGNvbW11bml0eSBjb2xsYWJvcmF0aW9uLgoKU2VlIGhvdyBpdCB3b3JrcyBh dCByZWRoYXQuY29tCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpUaGlzIGlzIGEg UFJJVkFURSBtZXNzYWdlLiBJZiB5b3UgYXJlIG5vdCB0aGUgaW50ZW5kZWQgcmVjaXBpZW50LCBw bGVhc2UgZGU9CmxldGUgd2l0aG91dCBjb3B5aW5nIGFuZCBraW5kbHkgYWR2aXNlIHVzIGJ5IGUt bWFpbCBvZiB0aGUgbWlzdGFrZSBpbiBkZWxpdj0KZXJ5LiBOT1RFOiBSZWdhcmRsZXNzIG9mIGNv bnRlbnQsIHRoaXMgZS1tYWlsIHNoYWxsIG5vdCBvcGVyYXRlIHRvIGJpbmQgU0tPPQpQT1MgdG8g YW55IG9yZGVyIG9yIG90aGVyIGNvbnRyYWN0IHVubGVzcyBwdXJzdWFudCB0byBleHBsaWNpdCB3 cml0dGVuIGFncmU9CmVtZW50IG9yIGdvdmVybm1lbnQgaW5pdGlhdGl2ZSBleHByZXNzbHkgcGVy bWl0dGluZyB0aGUgdXNlIG9mIGUtbWFpbCBmb3Igcz0KdWNoIHB1cnBvc2UuCgpfX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwpUaGlzIGlzIGEgUFJJVkFURSBtZXNzYWdlLiBJZiB5b3Ug YXJlIG5vdCB0aGUgaW50ZW5kZWQgcmVjaXBpZW50LCBwbGVhc2UgZGU9CmxldGUgd2l0aG91dCBj b3B5aW5nIGFuZCBraW5kbHkgYWR2aXNlIHVzIGJ5IGUtbWFpbCBvZiB0aGUgbWlzdGFrZSBpbiBk ZWxpdj0KZXJ5LiBOT1RFOiBSZWdhcmRsZXNzIG9mIGNvbnRlbnQsIHRoaXMgZS1tYWlsIHNoYWxs IG5vdCBvcGVyYXRlIHRvIGJpbmQgU0tPPQpQT1MgdG8gYW55IG9yZGVyIG9yIG90aGVyIGNvbnRy YWN0IHVubGVzcyBwdXJzdWFudCB0byBleHBsaWNpdCB3cml0dGVuIGFncmU9CmVtZW50IG9yIGdv dmVybm1lbnQgaW5pdGlhdGl2ZSBleHByZXNzbHkgcGVybWl0dGluZyB0aGUgdXNlIG9mIGUtbWFp bCBmb3Igcz0KdWNoIHB1cnBvc2UuCgotLV8wMDBfOUJFNkY0OTNGODNBNTk0REE2MEM0NUU2QTA5 REM1QUMwMTcxQTU3N0FVU1AwMURBRzAyMDNfCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBjaGFy c2V0PSJ1cy1hc2NpaSIKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogcXVvdGVkLXByaW50YWJs ZQoKPGh0bWw+CjxoZWFkPgo8bWV0YSBodHRwLWVxdWl2PTNEIkNvbnRlbnQtVHlwZSIgY29udGVu dD0zRCJ0ZXh0L2h0bWw7IGNoYXJzZXQ9M0R1cy1hc2NpaSI9Cj4KPHN0eWxlPgo8IS0tCkBmb250 LWZhY2UKCXtmb250LWZhbWlseTpDYWxpYnJpfQpAZm9udC1mYWNlCgl7Zm9udC1mYW1pbHk6VGFo b21hfQpAZm9udC1mYWNlCgl7Zm9udC1mYW1pbHk6Q29uc29sYXN9CnAuTXNvTm9ybWFsLCBsaS5N c29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwKCXttYXJnaW46MGluOwoJbWFyZ2luLWJvdHRvbTouMDAw MXB0OwoJZm9udC1zaXplOjEyLjBwdDsKCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLCJz ZXJpZiI7Cgljb2xvcjpibGFja30KYTpsaW5rLCBzcGFuLk1zb0h5cGVybGluawoJe2NvbG9yOmJs dWU7Cgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lfQphOnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJs aW5rRm9sbG93ZWQKCXtjb2xvcjpwdXJwbGU7Cgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lfQpw cmUKCXttYXJnaW46MGluOwoJbWFyZ2luLWJvdHRvbTouMDAwMXB0OwoJZm9udC1zaXplOjEwLjBw dDsKCWZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7Cgljb2xvcjpibGFja30KcC5Nc29BY2V0YXRl LCBsaS5Nc29BY2V0YXRlLCBkaXYuTXNvQWNldGF0ZQoJe21hcmdpbjowaW47CgltYXJnaW4tYm90 dG9tOi4wMDAxcHQ7Cglmb250LXNpemU6OC4wcHQ7Cglmb250LWZhbWlseToiVGFob21hIiwic2Fu cy1zZXJpZiI7Cgljb2xvcjpibGFja30Kc3Bhbi5IVE1MUHJlZm9ybWF0dGVkQ2hhcgoJe2ZvbnQt ZmFtaWx5OkNvbnNvbGFzOwoJY29sb3I6YmxhY2t9CnAubXNvY2hwZGVmYXVsdCwgbGkubXNvY2hw ZGVmYXVsdCwgZGl2Lm1zb2NocGRlZmF1bHQKCXttYXJnaW4tcmlnaHQ6MGluOwoJbWFyZ2luLWxl ZnQ6MGluOwoJZm9udC1zaXplOjEwLjBwdDsKCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4i LCJzZXJpZiJ9CnNwYW4uaHRtbHByZWZvcm1hdHRlZGNoYXIwCgl7Zm9udC1mYW1pbHk6Q29uc29s YXM7Cgljb2xvcjpibGFja30Kc3Bhbi5lbWFpbHN0eWxlMTkKCXtmb250LWZhbWlseToiQ2FsaWJy aSIsInNhbnMtc2VyaWYiOwoJY29sb3I6IzFGNDk3RH0Kc3Bhbi5FbWFpbFN0eWxlMjIKCXtmb250 LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiOwoJY29sb3I6IzFGNDk3RH0Kc3Bhbi5CYWxs b29uVGV4dENoYXIKCXtmb250LWZhbWlseToiVGFob21hIiwic2Fucy1zZXJpZiI7Cgljb2xvcjpi bGFja30KLk1zb0NocERlZmF1bHQKCXtmb250LXNpemU6MTAuMHB0fQpAcGFnZSBXb3JkU2VjdGlv bjEKCXttYXJnaW46MS4waW4gMS4waW4gMS4waW4gMS4waW59CmRpdi5Xb3JkU2VjdGlvbjEKCXt9 Ci0tPgo8L3N0eWxlPgo8L2hlYWQ+Cjxib2R5IGJnY29sb3I9M0Qid2hpdGUiIGxhbmc9M0QiRU4t VVMiIGxpbms9M0QiYmx1ZSIgdmxpbms9M0QicHVycGxlIj4KPGRpdiBjbGFzcz0zRCJXb3JkU2Vj dGlvbjEiPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPTNEImZvbnQtc2l6ZTox MS4wcHQ7IGZvbnQtZmFtaWx5OiZxdW90O0M9CmFsaWJyaSZxdW90OywmcXVvdDtzYW5zLXNlcmlm JnF1b3Q7OyBjb2xvcjojMUY0OTdEIj5PayB0aGlzIHRpbWUgaW5zdGVhZCBvZj0KIGJydXRhbGx5 IHJlYnVpbGRpbmcgdGhlIG5vZGUmIzgyMzA7IEkganVzdCByZWJvb3RlZCB0aGF0IG9uZSBub2Rl IGFuZCBpdHMgPQpyZWFkaW5nIGFnZW50cyBhZ2Fpbi4mbmJzcDsgQ3Jpc2lzIGF2ZXJ0ZWQuPC9z cGFuPjwvcD4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0zRCJmb250LXNpemU6 MTEuMHB0OyBmb250LWZhbWlseTomcXVvdDtDPQphbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJp ZiZxdW90OzsgY29sb3I6IzFGNDk3RCI+Jm5ic3A7PC9zcGFuPjwvcD4KPGRpdj4KPGRpdiBzdHls ZT0zRCJib3JkZXI6bm9uZTsgYm9yZGVyLXRvcDpzb2xpZCAjQjVDNERGIDEuMHB0OyBwYWRkaW5n OjMuMHB0IDBpPQpuIDBpbiAwaW4iPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxiPjxzcGFuIHN0 eWxlPTNEImZvbnQtc2l6ZToxMC4wcHQ7IGZvbnQtZmFtaWx5OiZxdW89CnQ7VGFob21hJnF1b3Q7 LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7IGNvbG9yOndpbmRvd3RleHQiPkZyb206PC9zcGFuPjwv Yj48cz0KcGFuIHN0eWxlPTNEImZvbnQtc2l6ZToxMC4wcHQ7IGZvbnQtZmFtaWx5OiZxdW90O1Rh aG9tYSZxdW90OywmcXVvdDtzYW5zLXNlPQpyaWYmcXVvdDs7IGNvbG9yOndpbmRvd3RleHQiPiB1 c2Vycy1ib3VuY2VzQG92aXJ0Lm9yZyBbbWFpbHRvOnVzZXJzLWJvdW5jZXM9CkBvdmlydC5vcmdd CjxiPk9uIEJlaGFsZiBPZiA8L2I+Sm9uYXRoYW4gSG9ybmU8YnI+CjxiPlNlbnQ6PC9iPiBNb25k YXksIEphbnVhcnkgMTQsIDIwMTMgOTozNyBBTTxicj4KPGI+VG86PC9iPiBWaW56ZW56IEZlZW5z dHJhOyB1c2Vyc0BvdmlydC5vcmc8YnI+CjxiPlN1YmplY3Q6PC9iPiBSZTogW1VzZXJzXSBub2Rl IGxvc2VzIGNvbnRhY3Qgd2l0aCByaGV2IGFnZW50PC9zcGFuPjwvcD4KPC9kaXY+CjwvZGl2Pgo8 cCBjbGFzcz0zRCJNc29Ob3JtYWwiPiZuYnNwOzwvcD4KPGRpdj4KPHAgY2xhc3M9M0QiTXNvTm9y bWFsIj48c3BhbiBzdHlsZT0zRCJmb250LXNpemU6MTEuMHB0OyBmb250LWZhbWlseTomcXVvdDtD PQphbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90OzsgY29sb3I6IzFGNDk3RCI+SXRz IGhhcHBlbmluZyBhZ2Fpbiwgd2g9CmF0IGxvZ3MgZG8gSSBuZWVkIHRvIHByb3ZpZGU/PC9zcGFu PjwvcD4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0zRCJmb250LXNpemU6MTEu MHB0OyBmb250LWZhbWlseTomcXVvdDtDPQphbGlicmkmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZx dW90OzsgY29sb3I6IzFGNDk3RCI+Jm5ic3A7PC9zcGFuPjwvcD4KPGRpdj4KPGRpdiBzdHlsZT0z RCJib3JkZXI6bm9uZTsgYm9yZGVyLXRvcDpzb2xpZCAjQjVDNERGIDEuMHB0OyBwYWRkaW5nOjMu MHB0IDBpPQpuIDBpbiAwaW4iPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPjxiPjxzcGFuIHN0eWxl PTNEImZvbnQtc2l6ZToxMC4wcHQ7IGZvbnQtZmFtaWx5OiZxdW89CnQ7VGFob21hJnF1b3Q7LCZx dW90O3NhbnMtc2VyaWYmcXVvdDs7IGNvbG9yOndpbmRvd3RleHQiPkZyb206PC9zcGFuPjwvYj48 cz0KcGFuIHN0eWxlPTNEImZvbnQtc2l6ZToxMC4wcHQ7IGZvbnQtZmFtaWx5OiZxdW90O1RhaG9t YSZxdW90OywmcXVvdDtzYW5zLXNlPQpyaWYmcXVvdDs7IGNvbG9yOndpbmRvd3RleHQiPiBWaW56 ZW56IEZlZW5zdHJhIFs8YSBocmVmPTNEIm1haWx0bzp2ZmVlbnN0ckA9CnJlZGhhdC5jb20iPm1h aWx0bzp2ZmVlbnN0ckByZWRoYXQuY29tPC9hPl0KPGJyPgo8Yj5TZW50OjwvYj4gTW9uZGF5LCBK YW51YXJ5IDA3LCAyMDEzIDY6MjAgQU08YnI+CjxiPlRvOjwvYj4gPGEgaHJlZj0zRCJtYWlsdG86 dXNlcnNAb3ZpcnQub3JnIj51c2Vyc0BvdmlydC5vcmc8L2E+PGJyPgo8Yj5DYzo8L2I+IEpvbmF0 aGFuIEhvcm5lPGJyPgo8Yj5TdWJqZWN0OjwvYj4gUmU6IFtVc2Vyc10gbm9kZSBsb3NlcyBjb250 YWN0IHdpdGggcmhldiBhZ2VudDwvc3Bhbj48L3A+CjwvZGl2Pgo8L2Rpdj4KPHAgY2xhc3M9M0Qi TXNvTm9ybWFsIj4mbmJzcDs8L3A+CjxkaXY+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCI+SGkgSm9u YXRoYW4sPGJyPgo8YnI+CkNvdWxkIHlvdSBwcm92aWRlIHRoZSBsb2dzIGZvciBpdD88YnI+ClRo YW5rcy48YnI+Cjxicj4KT24gMDEvMDQvMjAxMyAxMDozMiBBTSwgVmluemVueiBGZWVuc3RyYSB3 cm90ZTo8L3A+CjwvZGl2Pgo8YmxvY2txdW90ZSBzdHlsZT0zRCJtYXJnaW4tdG9wOjUuMHB0OyBt YXJnaW4tYm90dG9tOjUuMHB0Ij4KPGRpdj4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj5IaSw8YnI+ Cjxicj4KT24gMDEvMDMvMjAxMyAwODo1MyBQTSwgSm9uYXRoYW4gSG9ybmUgd3JvdGU6PC9wPgo8 L2Rpdj4KPGJsb2NrcXVvdGUgc3R5bGU9M0QibWFyZ2luLXRvcDo1LjBwdDsgbWFyZ2luLWJvdHRv bTo1LjBwdCI+CjxkaXY+CjxkaXY+CjxkaXY+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCI+aSBoYXZl IDIgbm9kZXMgbm93IHRoYXQgaGF2ZSBib3RoIGxvc3QgY29udGFjdCB3aXRoID0KdGhlIHJoZXYg YWdlbnRzIG9uIGd1ZXN0IFZNcy4gJm5ic3A7aWYgaSBtaWdyYXRlIHRoZSBndWVzdCB0byBhbm90 aGVyIG5vZGUsPQogdGhlIElQIGFkZHJlc3MgYW5kIG1lbW9yeSB1c2FnZSBpbW1lZGlhdGVseSBz aG93IHVwLiAmbmJzcDttaWdyYXRlIGEgZ3Vlc3Q9CiBiYWNrLCBhbmQgdGhlIElQL21lbW9yeSBp bmZvIGRpc2FwcGVhcnMuPC9wPgo8L2Rpdj4KPGRpdj4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj4m bmJzcDs8L3A+CjwvZGl2Pgo8ZGl2Pgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPmhvdyBjYW4gaSB0 cm91Ymxlc2hvb3Qgd2hhdCBpcyBjYXVzaW5nIHRoaXM/PC9wPgo8L2Rpdj4KPC9kaXY+CjwvZGl2 Pgo8L2Jsb2NrcXVvdGU+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCIgc3R5bGU9M0QibWFyZ2luLWJv dHRvbToxMi4wcHQiPjxicj4KSXQnZCBiZSBuaWNlIGlmIHlvdSBjb3VsZCBzZW5kIG1lIHRoZSBs b2dzIGZyb20gL3Zhci9sb2cvdmRzbS92ZHNtKiB3aGljaCBhPQpyZSBvbiB0aGUgbm9kZSB3aGVy ZSBpdCBsb29zZXMgdGhlIGNvbnRhY3QuIEkgYW0gY3VycmVudGx5IHRyeWluZyB0byBmaW5kIGU9 CnhhY3RseSB0aGUgY2F1c2Ugb2Ygc3VjaCBhbiBpc3N1ZSBhcyB3ZWxsIGFuZCBpdCB3b3VsZCBi ZSBoZWxwZnVsIGlmIHdlIGNvdT0KbGQgYmUgcHJvdmlkZWQgd2l0aCBsb2cgZmlsZXMuIEN1cnJl bnRseQogSSBhbSBzdGlsbCBsYWNraW5nIHRoZW0uPGJyPgpJIHdpbGwgY2hlY2sgdGhlbSBhbmQg bGV0IHlvdSBrbm93IHdoYXQgSSB3aWxsIGJlIGFibGUgdG8gZmluZC48YnI+Cjxicj4KSWYgeW91 IHdvdWxkbid0IG1pbmQgSSB3b3VsZCBiZSByZWZlcnJpbmcgdG8geW91IGZvciBtb3JlIHF1ZXN0 aW9ucyBpZiBzb21lPQogd2lsbCBjb21lIHVwIGR1cmluZyB0aGUgc2VhcmNoIGZvciB0aGUgY2F1 c2UgYW5kIEkgd2lsbCBsZXQgeW91IGtub3cgdGhlIHI9CmVzdWx0cyBhc2FwLjwvcD4KPGRpdj4K PGRpdj4KPGRpdj4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj4mbmJzcDs8L3A+CjwvZGl2Pgo8ZGl2 Pgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiPnRoYW5rcyw8L3A+CjwvZGl2Pgo8ZGl2Pgo8cCBjbGFz cz0zRCJNc29Ob3JtYWwiPmpvbmF0aGFuPC9wPgo8L2Rpdj4KPC9kaXY+CjwvZGl2Pgo8cCBjbGFz cz0zRCJNc29Ob3JtYWwiPiZuYnNwOzwvcD4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIiBzdHlsZT0z RCJtYXJnaW4tYm90dG9tOjEyLjBwdCI+Jm5ic3A7PC9wPgo8cHJlPi0tIDwvcHJlPgo8cHJlPlJl Z2FyZHMsPC9wcmU+CjxwcmU+Jm5ic3A7PC9wcmU+CjxwcmU+VmluemVueiBGZWVuc3RyYSB8IFNl bmlvciBTb2Z0d2FyZSBFbmdpbmVlcjwvcHJlPgo8cHJlPlJlZEhhdCBFbmdpbmVlcmluZyBWaXJ0 dWFsaXphdGlvbiBSICZhbXA7IEQ8L3ByZT4KPHByZT5QaG9uZTogJiM0Mzs0MjAgNTMyIDI5NCA2 MjU8L3ByZT4KPHByZT5JUkM6IHZmZWVuc3RyIG9yIGV2aWxpc3NpbW88L3ByZT4KPHByZT4mbmJz cDs8L3ByZT4KPHByZT5CZXR0ZXIgdGVjaG5vbG9neS4gRmFzdGVyIGlubm92YXRpb24uIFBvd2Vy ZWQgYnkgY29tbXVuaXR5IGNvbGxhYm9yYXRpPQpvbi48L3ByZT4KPHByZT5TZWUgaG93IGl0IHdv cmtzIGF0IHJlZGhhdC5jb208L3ByZT4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIiBzdHlsZT0zRCJt YXJnaW4tYm90dG9tOjEyLjBwdCI+PGJyPgo8YnI+CjwvcD4KPHByZT5fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXzwvcHJlPgo8cHJlPlVzZXJzIG1haWxpbmcg bGlzdDwvcHJlPgo8cHJlPjxhIGhyZWY9M0QibWFpbHRvOlVzZXJzQG92aXJ0Lm9yZyI+VXNlcnNA b3ZpcnQub3JnPC9hPjwvcHJlPgo8cHJlPjxhIGhyZWY9M0QiaHR0cDovL2xpc3RzLm92aXJ0Lm9y Zy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzIj5odHRwOi8vbGlzdHM9Ci5vdmlydC5vcmcvbWFpbG1h bi9saXN0aW5mby91c2VyczwvYT48L3ByZT4KPC9ibG9ja3F1b3RlPgo8cCBjbGFzcz0zRCJNc29O b3JtYWwiIHN0eWxlPTNEIm1hcmdpbi1ib3R0b206MTIuMHB0Ij48YnI+Cjxicj4KPC9wPgo8cHJl Pi0tIDwvcHJlPgo8cHJlPlJlZ2FyZHMsPC9wcmU+CjxwcmU+Jm5ic3A7PC9wcmU+CjxwcmU+Vmlu emVueiBGZWVuc3RyYSB8IFNlbmlvciBTb2Z0d2FyZSBFbmdpbmVlcjwvcHJlPgo8cHJlPlJlZEhh dCBFbmdpbmVlcmluZyBWaXJ0dWFsaXphdGlvbiBSICZhbXA7IEQ8L3ByZT4KPHByZT5QaG9uZTog JiM0Mzs0MjAgNTMyIDI5NCA2MjU8L3ByZT4KPHByZT5JUkM6IHZmZWVuc3RyIG9yIGV2aWxpc3Np bW88L3ByZT4KPHByZT4mbmJzcDs8L3ByZT4KPHByZT5CZXR0ZXIgdGVjaG5vbG9neS4gRmFzdGVy IGlubm92YXRpb24uIFBvd2VyZWQgYnkgY29tbXVuaXR5IGNvbGxhYm9yYXRpPQpvbi48L3ByZT4K PHByZT5TZWUgaG93IGl0IHdvcmtzIGF0IHJlZGhhdC5jb208L3ByZT4KPC9kaXY+CjxwIGNsYXNz PTNEIk1zb05vcm1hbCI+PHNwYW4gc3R5bGU9M0QiY29sb3I6d2luZG93dGV4dCI+Jm5ic3A7PC9z cGFuPjwvcD4KPGRpdiBjbGFzcz0zRCJNc29Ob3JtYWwiIGFsaWduPTNEImNlbnRlciIgc3R5bGU9 M0QidGV4dC1hbGlnbjpjZW50ZXIiPjxzcGFuPQogc3R5bGU9M0QiY29sb3I6d2luZG93dGV4dCI+ CjxociBzaXplPTNEIjIiIHdpZHRoPTNEIjEwMCUiIGFsaWduPTNEImNlbnRlciI+Cjwvc3Bhbj48 L2Rpdj4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0zRCJmb250LXNpemU6Ny41 cHQ7IGZvbnQtZmFtaWx5OiZxdW90O0FyPQppYWwmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiZxdW90 OzsgY29sb3I6Z3JheSI+VGhpcyBpcyBhIFBSSVZBVEUgbWVzc2FnZS4gSWY9CiB5b3UgYXJlIG5v dCB0aGUgaW50ZW5kZWQgcmVjaXBpZW50LCBwbGVhc2UgZGVsZXRlIHdpdGhvdXQgY29weWluZyBh bmQga2luZD0KbHkgYWR2aXNlIHVzIGJ5IGUtbWFpbCBvZiB0aGUgbWlzdGFrZSBpbiBkZWxpdmVy eS4gTk9URToKIFJlZ2FyZGxlc3Mgb2YgY29udGVudCwgdGhpcyBlLW1haWwgc2hhbGwgbm90IG9w ZXJhdGUgdG8gYmluZCBTS09QT1MgdG8gYW55PQogb3JkZXIgb3Igb3RoZXIgY29udHJhY3QgdW5s ZXNzIHB1cnN1YW50IHRvIGV4cGxpY2l0IHdyaXR0ZW4gYWdyZWVtZW50IG9yIGc9Cm92ZXJubWVu dCBpbml0aWF0aXZlIGV4cHJlc3NseSBwZXJtaXR0aW5nIHRoZSB1c2Ugb2YgZS1tYWlsIGZvciBz dWNoIHB1cnBvcz0KZS48L3NwYW4+PHNwYW4gc3R5bGU9M0QiY29sb3I6d2luZG93dGV4dCI+Cjwv c3Bhbj48L3A+CjwvZGl2Pgo8YnI+Cjxocj4KPGZvbnQgY29sb3I9M0QiR3JheSIgZmFjZT0zRCJB cmlhbCIgc2l6ZT0zRCIxIj5UaGlzIGlzIGEgUFJJVkFURSBtZXNzYWdlLiBJPQpmIHlvdSBhcmUg bm90IHRoZSBpbnRlbmRlZCByZWNpcGllbnQsIHBsZWFzZSBkZWxldGUgd2l0aG91dCBjb3B5aW5n IGFuZCBraW49CmRseSBhZHZpc2UgdXMgYnkgZS1tYWlsIG9mIHRoZSBtaXN0YWtlIGluIGRlbGl2 ZXJ5LiBOT1RFOiBSZWdhcmRsZXNzIG9mIGNvbj0KdGVudCwgdGhpcyBlLW1haWwgc2hhbGwgbm90 IG9wZXJhdGUgdG8KIGJpbmQgU0tPUE9TIHRvIGFueSBvcmRlciBvciBvdGhlciBjb250cmFjdCB1 bmxlc3MgcHVyc3VhbnQgdG8gZXhwbGljaXQgd3JpPQp0dGVuIGFncmVlbWVudCBvciBnb3Zlcm5t ZW50IGluaXRpYXRpdmUgZXhwcmVzc2x5IHBlcm1pdHRpbmcgdGhlIHVzZSBvZiBlLW09CmFpbCBm b3Igc3VjaCBwdXJwb3NlLjwvZm9udD4KPC9ib2R5Pgo8L2h0bWw+CgotLV8wMDBfOUJFNkY0OTNG ODNBNTk0REE2MEM0NUU2QTA5REM1QUMwMTcxQTU3N0FVU1AwMURBRzAyMDNfLS0K --===============5910694068487950865==-- From vfeenstr at redhat.com Thu Jan 17 02:46:56 2013 Content-Type: multipart/mixed; boundary="===============8234729336360563580==" MIME-Version: 1.0 From: Vinzenz Feenstra To: users at ovirt.org Subject: Re: [Users] node loses contact with rhev agent Date: Thu, 17 Jan 2013 08:46:52 +0100 Message-ID: <50F7AC6C.5050108@redhat.com> In-Reply-To: 9BE6F493F83A594DA60C45E6A09DC5AC0171A577@AUSP01DAG0203 --===============8234729336360563580== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------090206080107070400030502 Content-Type: text/plain; charset=3DISO-8859-1; format=3Dflowed Content-Transfer-Encoding: 7bit Hi, Great to hear that you solved the issue. As a suggestion for next time: Execute: service vdsmd restart on the node. You'll not loose your = running VMs and it will work afterwards. I think I managed to pinpoint the issue, however we're still trying to = verify the patch I have implemented as a proposal for it. I noticed that logs won't help me that much on that issue as this is = some condition where the connection between host and guest times out due = to the server being busy or something. I am not 100%ly certain about the = cause, however the solution will be to reissue connects on time outs. Regards, On 01/14/2013 05:28 PM, Jonathan Horne wrote: > > Ok this time instead of brutally rebuilding the node... I just = > rebooted that one node and its reading agents again. Crisis averted. > > *From:*users-bounces(a)ovirt.org [mailto:users-bounces(a)ovirt.org] *On = > Behalf Of *Jonathan Horne > *Sent:* Monday, January 14, 2013 9:37 AM > *To:* Vinzenz Feenstra; users(a)ovirt.org > *Subject:* Re: [Users] node loses contact with rhev agent > > Its happening again, what logs do I need to provide? > > *From:*Vinzenz Feenstra [mailto:vfeenstr(a)redhat.com] > *Sent:* Monday, January 07, 2013 6:20 AM > *To:* users(a)ovirt.org > *Cc:* Jonathan Horne > *Subject:* Re: [Users] node loses contact with rhev agent > > Hi Jonathan, > > Could you provide the logs for it? > Thanks. > > On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote: > > Hi, > > On 01/03/2013 08:53 PM, Jonathan Horne wrote: > > i have 2 nodes now that have both lost contact with the rhev > agents on guest VMs. if i migrate the guest to another node, > the IP address and memory usage immediately show up. migrate > a guest back, and the IP/memory info disappears. > > how can i troubleshoot what is causing this? > > > It'd be nice if you could send me the logs from > /var/log/vdsm/vdsm* which are on the node where it looses the > contact. I am currently trying to find exactly the cause of such > an issue as well and it would be helpful if we could be provided > with log files. Currently I am still lacking them. > I will check them and let you know what I will be able to find. > > If you wouldn't mind I would be referring to you for more > questions if some will come up during the search for the cause and > I will let you know the results asap. > > thanks, > > jonathan > > -- = > > Regards, > > = > > Vinzenz Feenstra | Senior Software Engineer > > RedHat Engineering Virtualization R & D > > Phone: +420 532 294 625 > > IRC: vfeenstr or evilissimo > > = > > Better technology. Faster innovation. Powered by community collaborat= ion. > > See how it works at redhat.com > > > > _______________________________________________ > > Users mailing list > > Users(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/users > > > > -- = > Regards, > = > Vinzenz Feenstra | Senior Software Engineer > RedHat Engineering Virtualization R & D > Phone: +420 532 294 625 > IRC: vfeenstr or evilissimo > = > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com > > ------------------------------------------------------------------------ > > This is a PRIVATE message. If you are not the intended recipient, = > please delete without copying and kindly advise us by e-mail of the = > mistake in delivery. NOTE: Regardless of content, this e-mail shall = > not operate to bind SKOPOS to any order or other contract unless = > pursuant to explicit written agreement or government initiative = > expressly permitting the use of e-mail for such purpose. > > > ------------------------------------------------------------------------ > This is a PRIVATE message. If you are not the intended recipient, = > please delete without copying and kindly advise us by e-mail of the = > mistake in delivery. NOTE: Regardless of content, this e-mail shall = > not operate to bind SKOPOS to any order or other contract unless = > pursuant to explicit written agreement or government initiative = > expressly permitting the use of e-mail for such purpose. > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users -- = Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R & D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com --------------090206080107070400030502 Content-Type: text/html; charset=3DISO-8859-1 Content-Transfer-Encoding: 7bit

Hi,

Great to hear that you solved the issue. As a suggestion for next time:

Execute: service vdsmd restart on the node. You'll not loose your running VMs and it will work afterwards.
I think I managed to pinpoint the issue, however we're still trying to verify the patch I have implemented as a proposal for it.

I noticed that logs won't help me that much on that issue as this is some condition where the connection between host and guest times out due to the server being busy or something. I am not 100%ly certain about the cause, however the solution will be to reissue connects on time outs.

Regards,

On 01/14/2013 05:28 PM, Jonathan Horne wrote:

Ok this time instead of brutally rebuilding the node… I just rebooted that one node and its reading agents again. Crisis averted.

From: users-bounces(a)ovirt.org [mailto:users-bounces(a)ovirt.org] On Behalf Of Jonathan Horne
Sent: Monday, January 14, 2013 9:37 AM
To: Vinzenz Feenstra; users(a)ovirt.org
Subject: Re: [Users] node loses contact with rhev agent
Its happening again, what logs do I need to provide?

From: Vinzenz Feenstra [mailto:vfeenstr(a= )redhat.com]
Sent: Monday, January 07, 2013 6:20 AM
To: users(a)ovirt.org=
Cc: Jonathan Horne
Subject: Re: [Users] node loses contact with rhev agent

Hi Jonathan,

Could you provide the logs for it?
Thanks.

On 01/04/2013 10:32 AM, Vinzenz Feenstra wrote:
Hi,

On 01/03/2013 08:53 PM, Jonathan Horne wrote:

i have 2 nodes now that have both lost contact with the rhev agents on guest VMs. if i migrate the guest to another node, the IP address and memory usage immediately show up. migrate a guest back, and the IP/memory info disappears.

how can i troubleshoot what is causing this?

It'd be nice if you could send me the logs from /var/log/vdsm/vdsm* which are on the node where it looses the contact. I am currently trying to find exactly the cause of such an issue as well and it would be helpful if we could be provided with log files. Currently I am still lacking them.
I will check them and let you know what I will be able to find.

If you wouldn't mind I would be referring to you for more questions if some will come up during the search for the cause and I will let you know the results asap.

thanks,

jonathan
-- 
Regards,
 
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
 
Better technology. Faster innovation. Powered by community=
 collaboration.
See how it works at redhat.com
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users<=
/pre>
          
-- 
Regards,
 
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
 
Better technology. Faster innovation. Powered by community c=
ollaboration.
See how it works at redhat.com
This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind SKOPOS to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.=
This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind SKOPOS to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
_______________________________________________
Users mailing list
Use=
rs(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

-- =

Regards,

Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo

Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com

--------------090206080107070400030502-- --===============8234729336360563580== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wOTAyMDYwODAxMDcwNzA0MDAwMzA1MDIKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PUlTTy04ODU5LTE7IGZvcm1hdD1mbG93ZWQKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzog N2JpdAoKSGksCgpHcmVhdCB0byBoZWFyIHRoYXQgeW91IHNvbHZlZCB0aGUgaXNzdWUuIEFzIGEg c3VnZ2VzdGlvbiBmb3IgbmV4dCB0aW1lOgoKRXhlY3V0ZTogc2VydmljZSB2ZHNtZCByZXN0YXJ0 IG9uIHRoZSBub2RlLiBZb3UnbGwgbm90IGxvb3NlIHlvdXIgCnJ1bm5pbmcgVk1zIGFuZCBpdCB3 aWxsIHdvcmsgYWZ0ZXJ3YXJkcy4KSSB0aGluayBJIG1hbmFnZWQgdG8gcGlucG9pbnQgdGhlIGlz c3VlLCBob3dldmVyIHdlJ3JlIHN0aWxsIHRyeWluZyB0byAKdmVyaWZ5IHRoZSBwYXRjaCBJIGhh dmUgaW1wbGVtZW50ZWQgYXMgYSBwcm9wb3NhbCBmb3IgaXQuCgpJIG5vdGljZWQgdGhhdCBsb2dz IHdvbid0IGhlbHAgbWUgdGhhdCBtdWNoIG9uIHRoYXQgaXNzdWUgYXMgdGhpcyBpcyAKc29tZSBj b25kaXRpb24gd2hlcmUgdGhlIGNvbm5lY3Rpb24gYmV0d2VlbiBob3N0IGFuZCBndWVzdCB0aW1l cyBvdXQgZHVlIAp0byB0aGUgc2VydmVyIGJlaW5nIGJ1c3kgb3Igc29tZXRoaW5nLiBJIGFtIG5v dCAxMDAlbHkgY2VydGFpbiBhYm91dCB0aGUgCmNhdXNlLCBob3dldmVyIHRoZSBzb2x1dGlvbiB3 aWxsIGJlIHRvIHJlaXNzdWUgY29ubmVjdHMgb24gdGltZSBvdXRzLgoKUmVnYXJkcywKCk9uIDAx LzE0LzIwMTMgMDU6MjggUE0sIEpvbmF0aGFuIEhvcm5lIHdyb3RlOgo+Cj4gT2sgdGhpcyB0aW1l IGluc3RlYWQgb2YgYnJ1dGFsbHkgcmVidWlsZGluZyB0aGUgbm9kZS4uLiBJIGp1c3QgCj4gcmVi b290ZWQgdGhhdCBvbmUgbm9kZSBhbmQgaXRzIHJlYWRpbmcgYWdlbnRzIGFnYWluLiAgQ3Jpc2lz IGF2ZXJ0ZWQuCj4KPiAqRnJvbToqdXNlcnMtYm91bmNlc0BvdmlydC5vcmcgW21haWx0bzp1c2Vy cy1ib3VuY2VzQG92aXJ0Lm9yZ10gKk9uIAo+IEJlaGFsZiBPZiAqSm9uYXRoYW4gSG9ybmUKPiAq U2VudDoqIE1vbmRheSwgSmFudWFyeSAxNCwgMjAxMyA5OjM3IEFNCj4gKlRvOiogVmluemVueiBG ZWVuc3RyYTsgdXNlcnNAb3ZpcnQub3JnCj4gKlN1YmplY3Q6KiBSZTogW1VzZXJzXSBub2RlIGxv c2VzIGNvbnRhY3Qgd2l0aCByaGV2IGFnZW50Cj4KPiBJdHMgaGFwcGVuaW5nIGFnYWluLCB3aGF0 IGxvZ3MgZG8gSSBuZWVkIHRvIHByb3ZpZGU/Cj4KPiAqRnJvbToqVmluemVueiBGZWVuc3RyYSBb bWFpbHRvOnZmZWVuc3RyQHJlZGhhdC5jb21dCj4gKlNlbnQ6KiBNb25kYXksIEphbnVhcnkgMDcs IDIwMTMgNjoyMCBBTQo+ICpUbzoqIHVzZXJzQG92aXJ0Lm9yZyA8bWFpbHRvOnVzZXJzQG92aXJ0 Lm9yZz4KPiAqQ2M6KiBKb25hdGhhbiBIb3JuZQo+ICpTdWJqZWN0OiogUmU6IFtVc2Vyc10gbm9k ZSBsb3NlcyBjb250YWN0IHdpdGggcmhldiBhZ2VudAo+Cj4gSGkgSm9uYXRoYW4sCj4KPiBDb3Vs ZCB5b3UgcHJvdmlkZSB0aGUgbG9ncyBmb3IgaXQ/Cj4gVGhhbmtzLgo+Cj4gT24gMDEvMDQvMjAx MyAxMDozMiBBTSwgVmluemVueiBGZWVuc3RyYSB3cm90ZToKPgo+ICAgICBIaSwKPgo+ICAgICBP biAwMS8wMy8yMDEzIDA4OjUzIFBNLCBKb25hdGhhbiBIb3JuZSB3cm90ZToKPgo+ICAgICAgICAg aSBoYXZlIDIgbm9kZXMgbm93IHRoYXQgaGF2ZSBib3RoIGxvc3QgY29udGFjdCB3aXRoIHRoZSBy aGV2Cj4gICAgICAgICBhZ2VudHMgb24gZ3Vlc3QgVk1zLiAgaWYgaSBtaWdyYXRlIHRoZSBndWVz dCB0byBhbm90aGVyIG5vZGUsCj4gICAgICAgICB0aGUgSVAgYWRkcmVzcyBhbmQgbWVtb3J5IHVz YWdlIGltbWVkaWF0ZWx5IHNob3cgdXAuICBtaWdyYXRlCj4gICAgICAgICBhIGd1ZXN0IGJhY2ss IGFuZCB0aGUgSVAvbWVtb3J5IGluZm8gZGlzYXBwZWFycy4KPgo+ICAgICAgICAgaG93IGNhbiBp IHRyb3VibGVzaG9vdCB3aGF0IGlzIGNhdXNpbmcgdGhpcz8KPgo+Cj4gICAgIEl0J2QgYmUgbmlj ZSBpZiB5b3UgY291bGQgc2VuZCBtZSB0aGUgbG9ncyBmcm9tCj4gICAgIC92YXIvbG9nL3Zkc20v dmRzbSogd2hpY2ggYXJlIG9uIHRoZSBub2RlIHdoZXJlIGl0IGxvb3NlcyB0aGUKPiAgICAgY29u dGFjdC4gSSBhbSBjdXJyZW50bHkgdHJ5aW5nIHRvIGZpbmQgZXhhY3RseSB0aGUgY2F1c2Ugb2Yg c3VjaAo+ICAgICBhbiBpc3N1ZSBhcyB3ZWxsIGFuZCBpdCB3b3VsZCBiZSBoZWxwZnVsIGlmIHdl IGNvdWxkIGJlIHByb3ZpZGVkCj4gICAgIHdpdGggbG9nIGZpbGVzLiBDdXJyZW50bHkgSSBhbSBz dGlsbCBsYWNraW5nIHRoZW0uCj4gICAgIEkgd2lsbCBjaGVjayB0aGVtIGFuZCBsZXQgeW91IGtu b3cgd2hhdCBJIHdpbGwgYmUgYWJsZSB0byBmaW5kLgo+Cj4gICAgIElmIHlvdSB3b3VsZG4ndCBt aW5kIEkgd291bGQgYmUgcmVmZXJyaW5nIHRvIHlvdSBmb3IgbW9yZQo+ICAgICBxdWVzdGlvbnMg aWYgc29tZSB3aWxsIGNvbWUgdXAgZHVyaW5nIHRoZSBzZWFyY2ggZm9yIHRoZSBjYXVzZSBhbmQK PiAgICAgSSB3aWxsIGxldCB5b3Uga25vdyB0aGUgcmVzdWx0cyBhc2FwLgo+Cj4gICAgIHRoYW5r cywKPgo+ICAgICBqb25hdGhhbgo+Cj4gICAgIC0tIAo+Cj4gICAgIFJlZ2FyZHMsCj4KPiAgICAg ICAKPgo+ICAgICBWaW56ZW56IEZlZW5zdHJhIHwgU2VuaW9yIFNvZnR3YXJlIEVuZ2luZWVyCj4K PiAgICAgUmVkSGF0IEVuZ2luZWVyaW5nIFZpcnR1YWxpemF0aW9uIFIgJiBECj4KPiAgICAgUGhv bmU6ICs0MjAgNTMyIDI5NCA2MjUKPgo+ICAgICBJUkM6IHZmZWVuc3RyIG9yIGV2aWxpc3NpbW8K Pgo+ICAgICAgIAo+Cj4gICAgIEJldHRlciB0ZWNobm9sb2d5LiBGYXN0ZXIgaW5ub3ZhdGlvbi4g UG93ZXJlZCBieSBjb21tdW5pdHkgY29sbGFib3JhdGlvbi4KPgo+ICAgICBTZWUgaG93IGl0IHdv cmtzIGF0IHJlZGhhdC5jb20KPgo+Cj4KPiAgICAgX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX18KPgo+ICAgICBVc2VycyBtYWlsaW5nIGxpc3QKPgo+ICAgICBV c2Vyc0BvdmlydC5vcmcgIDxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPgo+Cj4gICAgIGh0dHA6Ly9s aXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2Vycwo+Cj4KPgo+IC0tIAo+IFJlZ2Fy ZHMsCj4gICAKPiBWaW56ZW56IEZlZW5zdHJhIHwgU2VuaW9yIFNvZnR3YXJlIEVuZ2luZWVyCj4g UmVkSGF0IEVuZ2luZWVyaW5nIFZpcnR1YWxpemF0aW9uIFIgJiBECj4gUGhvbmU6ICs0MjAgNTMy IDI5NCA2MjUKPiBJUkM6IHZmZWVuc3RyIG9yIGV2aWxpc3NpbW8KPiAgIAo+IEJldHRlciB0ZWNo bm9sb2d5LiBGYXN0ZXIgaW5ub3ZhdGlvbi4gUG93ZXJlZCBieSBjb21tdW5pdHkgY29sbGFib3Jh dGlvbi4KPiBTZWUgaG93IGl0IHdvcmtzIGF0IHJlZGhhdC5jb20KPgo+IC0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LQo+Cj4gVGhpcyBpcyBhIFBSSVZBVEUgbWVzc2FnZS4gSWYgeW91IGFyZSBub3QgdGhlIGludGVu ZGVkIHJlY2lwaWVudCwgCj4gcGxlYXNlIGRlbGV0ZSB3aXRob3V0IGNvcHlpbmcgYW5kIGtpbmRs eSBhZHZpc2UgdXMgYnkgZS1tYWlsIG9mIHRoZSAKPiBtaXN0YWtlIGluIGRlbGl2ZXJ5LiBOT1RF OiBSZWdhcmRsZXNzIG9mIGNvbnRlbnQsIHRoaXMgZS1tYWlsIHNoYWxsIAo+IG5vdCBvcGVyYXRl IHRvIGJpbmQgU0tPUE9TIHRvIGFueSBvcmRlciBvciBvdGhlciBjb250cmFjdCB1bmxlc3MgCj4g cHVyc3VhbnQgdG8gZXhwbGljaXQgd3JpdHRlbiBhZ3JlZW1lbnQgb3IgZ292ZXJubWVudCBpbml0 aWF0aXZlIAo+IGV4cHJlc3NseSBwZXJtaXR0aW5nIHRoZSB1c2Ugb2YgZS1tYWlsIGZvciBzdWNo IHB1cnBvc2UuCj4KPgo+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQo+IFRoaXMgaXMgYSBQUklWQVRFIG1lc3Nh Z2UuIElmIHlvdSBhcmUgbm90IHRoZSBpbnRlbmRlZCByZWNpcGllbnQsIAo+IHBsZWFzZSBkZWxl dGUgd2l0aG91dCBjb3B5aW5nIGFuZCBraW5kbHkgYWR2aXNlIHVzIGJ5IGUtbWFpbCBvZiB0aGUg Cj4gbWlzdGFrZSBpbiBkZWxpdmVyeS4gTk9URTogUmVnYXJkbGVzcyBvZiBjb250ZW50LCB0aGlz IGUtbWFpbCBzaGFsbCAKPiBub3Qgb3BlcmF0ZSB0byBiaW5kIFNLT1BPUyB0byBhbnkgb3JkZXIg b3Igb3RoZXIgY29udHJhY3QgdW5sZXNzIAo+IHB1cnN1YW50IHRvIGV4cGxpY2l0IHdyaXR0ZW4g YWdyZWVtZW50IG9yIGdvdmVybm1lbnQgaW5pdGlhdGl2ZSAKPiBleHByZXNzbHkgcGVybWl0dGlu ZyB0aGUgdXNlIG9mIGUtbWFpbCBmb3Igc3VjaCBwdXJwb3NlLgo+Cj4KPiBfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwo+IFVzZXJzIG1haWxpbmcgbGlzdAo+ IFVzZXJzQG92aXJ0Lm9yZwo+IGh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5m by91c2VycwoKCi0tIApSZWdhcmRzLAoKVmluemVueiBGZWVuc3RyYSB8IFNlbmlvciBTb2Z0d2Fy ZSBFbmdpbmVlcgpSZWRIYXQgRW5naW5lZXJpbmcgVmlydHVhbGl6YXRpb24gUiAmIEQKUGhvbmU6 ICs0MjAgNTMyIDI5NCA2MjUKSVJDOiB2ZmVlbnN0ciBvciBldmlsaXNzaW1vCgpCZXR0ZXIgdGVj aG5vbG9neS4gRmFzdGVyIGlubm92YXRpb24uIFBvd2VyZWQgYnkgY29tbXVuaXR5IGNvbGxhYm9y YXRpb24uClNlZSBob3cgaXQgd29ya3MgYXQgcmVkaGF0LmNvbQoKCi0tLS0tLS0tLS0tLS0tMDkw MjA2MDgwMTA3MDcwNDAwMDMwNTAyCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PUlT Ty04ODU5LTEKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdAoKPGh0bWw+CiAgPGhlYWQ+ CiAgICA8bWV0YSBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9SVNPLTg4NTktMSIKICAgICAg aHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICA8L2hlYWQ+CiAgPGJvZHkgYmdjb2xvcj0iI0ZG RkZGRiIgdGV4dD0iIzAwMDAwMCI+CiAgICA8ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPkhp LDxicj4KICAgICAgPGJyPgogICAgICBHcmVhdCB0byBoZWFyIHRoYXQgeW91IHNvbHZlZCB0aGUg aXNzdWUuIEFzIGEgc3VnZ2VzdGlvbiBmb3IgbmV4dAogICAgICB0aW1lOjxicj4KICAgICAgPGJy PgogICAgICBFeGVjdXRlOiBzZXJ2aWNlIHZkc21kIHJlc3RhcnQgb24gdGhlIG5vZGUuIFlvdSds bCBub3QgbG9vc2UgeW91cgogICAgICBydW5uaW5nIFZNcyBhbmQgaXQgd2lsbCB3b3JrIGFmdGVy d2FyZHMuPGJyPgogICAgICBJIHRoaW5rIEkgbWFuYWdlZCB0byBwaW5wb2ludCB0aGUgaXNzdWUs IGhvd2V2ZXIgd2UncmUgc3RpbGwKICAgICAgdHJ5aW5nIHRvIHZlcmlmeSB0aGUgcGF0Y2ggSSBo YXZlIGltcGxlbWVudGVkIGFzIGEgcHJvcG9zYWwgZm9yCiAgICAgIGl0Ljxicj4KICAgICAgPGJy PgogICAgICBJIG5vdGljZWQgdGhhdCBsb2dzIHdvbid0IGhlbHAgbWUgdGhhdCBtdWNoIG9uIHRo YXQgaXNzdWUgYXMgdGhpcwogICAgICBpcyBzb21lIGNvbmRpdGlvbiB3aGVyZSB0aGUgY29ubmVj dGlvbiBiZXR3ZWVuIGhvc3QgYW5kIGd1ZXN0CiAgICAgIHRpbWVzIG91dCBkdWUgdG8gdGhlIHNl cnZlciBiZWluZyBidXN5IG9yIHNvbWV0aGluZy4gSSBhbSBub3QKICAgICAgMTAwJWx5IGNlcnRh aW4gYWJvdXQgdGhlIGNhdXNlLCBob3dldmVyIHRoZSBzb2x1dGlvbiB3aWxsIGJlIHRvCiAgICAg IHJlaXNzdWUgY29ubmVjdHMgb24gdGltZSBvdXRzLjxicj4KICAgICAgPGJyPgogICAgICBSZWdh cmRzLDxicj4KICAgICAgPGJyPgogICAgICBPbiAwMS8xNC8yMDEzIDA1OjI4IFBNLCBKb25hdGhh biBIb3JuZSB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAgIDxibG9ja3F1b3RlCiAgICAgIGNpdGU9 Im1pZDo5QkU2RjQ5M0Y4M0E1OTREQTYwQzQ1RTZBMDlEQzVBQzAxNzFBNTc3QEFVU1AwMURBRzAy MDMiCiAgICAgIHR5cGU9ImNpdGUiPgogICAgICA8bWV0YSBodHRwLWVxdWl2PSJDb250ZW50LVR5 cGUiIGNvbnRlbnQ9InRleHQvaHRtbDsKICAgICAgICBjaGFyc2V0PUlTTy04ODU5LTEiPgogICAg ICA8c3R5bGU+CjwhLS0KQGZvbnQtZmFjZQoJe2ZvbnQtZmFtaWx5OkNhbGlicml9CkBmb250LWZh Y2UKCXtmb250LWZhbWlseTpUYWhvbWF9CkBmb250LWZhY2UKCXtmb250LWZhbWlseTpDb25zb2xh c30KcC5Nc29Ob3JtYWwsIGxpLk1zb05vcm1hbCwgZGl2Lk1zb05vcm1hbAoJe21hcmdpbjowaW47 CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7Cglmb250LXNpemU6MTIuMHB0OwoJZm9udC1mYW1pbHk6 IlRpbWVzIE5ldyBSb21hbiIsInNlcmlmIjsKCWNvbG9yOmJsYWNrfQphOmxpbmssIHNwYW4uTXNv SHlwZXJsaW5rCgl7Y29sb3I6Ymx1ZTsKCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmV9CmE6dmlz aXRlZCwgc3Bhbi5Nc29IeXBlcmxpbmtGb2xsb3dlZAoJe2NvbG9yOnB1cnBsZTsKCXRleHQtZGVj b3JhdGlvbjp1bmRlcmxpbmV9CnByZQoJe21hcmdpbjowaW47CgltYXJnaW4tYm90dG9tOi4wMDAx cHQ7Cglmb250LXNpemU6MTAuMHB0OwoJZm9udC1mYW1pbHk6IkNvdXJpZXIgTmV3IjsKCWNvbG9y OmJsYWNrfQpwLk1zb0FjZXRhdGUsIGxpLk1zb0FjZXRhdGUsIGRpdi5Nc29BY2V0YXRlCgl7bWFy Z2luOjBpbjsKCW1hcmdpbi1ib3R0b206LjAwMDFwdDsKCWZvbnQtc2l6ZTo4LjBwdDsKCWZvbnQt ZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIjsKCWNvbG9yOmJsYWNrfQpzcGFuLkhUTUxQcmVm b3JtYXR0ZWRDaGFyCgl7Zm9udC1mYW1pbHk6Q29uc29sYXM7Cgljb2xvcjpibGFja30KcC5tc29j aHBkZWZhdWx0LCBsaS5tc29jaHBkZWZhdWx0LCBkaXYubXNvY2hwZGVmYXVsdAoJe21hcmdpbi1y aWdodDowaW47CgltYXJnaW4tbGVmdDowaW47Cglmb250LXNpemU6MTAuMHB0OwoJZm9udC1mYW1p bHk6IlRpbWVzIE5ldyBSb21hbiIsInNlcmlmIn0Kc3Bhbi5odG1scHJlZm9ybWF0dGVkY2hhcjAK CXtmb250LWZhbWlseTpDb25zb2xhczsKCWNvbG9yOmJsYWNrfQpzcGFuLmVtYWlsc3R5bGUxOQoJ e2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Cgljb2xvcjojMUY0OTdEfQpzcGFu LkVtYWlsU3R5bGUyMgoJe2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Cgljb2xv cjojMUY0OTdEfQpzcGFuLkJhbGxvb25UZXh0Q2hhcgoJe2ZvbnQtZmFtaWx5OiJUYWhvbWEiLCJz YW5zLXNlcmlmIjsKCWNvbG9yOmJsYWNrfQouTXNvQ2hwRGVmYXVsdAoJe2ZvbnQtc2l6ZToxMC4w cHR9CkBwYWdlIFdvcmRTZWN0aW9uMQoJe21hcmdpbjoxLjBpbiAxLjBpbiAxLjBpbiAxLjBpbn0K ZGl2LldvcmRTZWN0aW9uMQoJe30KLS0+Cjwvc3R5bGU+CiAgICAgIDxkaXYgY2xhc3M9IldvcmRT ZWN0aW9uMSI+CiAgICAgICAgPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQt c2l6ZToxMS4wcHQ7CiAgICAgICAgICAgIGZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss JnF1b3Q7c2Fucy1zZXJpZiZxdW90OzsKICAgICAgICAgICAgY29sb3I6IzFGNDk3RCI+T2sgdGhp cyB0aW1lIGluc3RlYWQgb2YgYnJ1dGFsbHkgcmVidWlsZGluZwogICAgICAgICAgICB0aGUgbm9k ZSYjODIzMDsgSSBqdXN0IHJlYm9vdGVkIHRoYXQgb25lIG5vZGUgYW5kIGl0cyByZWFkaW5nCiAg ICAgICAgICAgIGFnZW50cyBhZ2Fpbi4mbmJzcDsgQ3Jpc2lzIGF2ZXJ0ZWQuPC9zcGFuPjwvcD4K ICAgICAgICA8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBw dDsKICAgICAgICAgICAgZm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVvdDtzYW5z LXNlcmlmJnF1b3Q7OwogICAgICAgICAgICBjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PC9w PgogICAgICAgIDxkaXY+CiAgICAgICAgICA8ZGl2IHN0eWxlPSJib3JkZXI6bm9uZTsgYm9yZGVy LXRvcDpzb2xpZCAjQjVDNERGIDEuMHB0OwogICAgICAgICAgICBwYWRkaW5nOjMuMHB0IDBpbiAw aW4gMGluIj4KICAgICAgICAgICAgPHAgY2xhc3M9Ik1zb05vcm1hbCI+PGI+PHNwYW4gc3R5bGU9 ImZvbnQtc2l6ZToxMC4wcHQ7CiAgICAgICAgICAgICAgICAgIGZvbnQtZmFtaWx5OiZxdW90O1Rh aG9tYSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7OwogICAgICAgICAgICAgICAgICBjb2xv cjp3aW5kb3d0ZXh0Ij5Gcm9tOjwvc3Bhbj48L2I+PHNwYW4KICAgICAgICAgICAgICAgIHN0eWxl PSJmb250LXNpemU6MTAuMHB0OwogICAgICAgICAgICAgICAgZm9udC1mYW1pbHk6JnF1b3Q7VGFo b21hJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7CiAgICAgICAgICAgICAgICBjb2xvcjp3 aW5kb3d0ZXh0Ij4gPGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIgaHJlZj0ibWFp bHRvOnVzZXJzLWJvdW5jZXNAb3ZpcnQub3JnIj51c2Vycy1ib3VuY2VzQG92aXJ0Lm9yZzwvYT4K ICAgICAgICAgICAgICAgIFs8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJt YWlsdG86dXNlcnMtYm91bmNlc0BvdmlydC5vcmciPm1haWx0bzp1c2Vycy1ib3VuY2VzQG92aXJ0 Lm9yZzwvYT5dCiAgICAgICAgICAgICAgICA8Yj5PbiBCZWhhbGYgT2YgPC9iPkpvbmF0aGFuIEhv cm5lPGJyPgogICAgICAgICAgICAgICAgPGI+U2VudDo8L2I+IE1vbmRheSwgSmFudWFyeSAxNCwg MjAxMyA5OjM3IEFNPGJyPgogICAgICAgICAgICAgICAgPGI+VG86PC9iPiBWaW56ZW56IEZlZW5z dHJhOyA8YSBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJtYWlsdG86dXNl cnNAb3ZpcnQub3JnIj51c2Vyc0BvdmlydC5vcmc8L2E+PGJyPgogICAgICAgICAgICAgICAgPGI+ U3ViamVjdDo8L2I+IFJlOiBbVXNlcnNdIG5vZGUgbG9zZXMgY29udGFjdCB3aXRoIHJoZXYKICAg ICAgICAgICAgICAgIGFnZW50PC9zcGFuPjwvcD4KICAgICAgICAgIDwvZGl2PgogICAgICAgIDwv ZGl2PgogICAgICAgIDxwIGNsYXNzPSJNc29Ob3JtYWwiPiZuYnNwOzwvcD4KICAgICAgICA8ZGl2 PgogICAgICAgICAgPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox MS4wcHQ7CiAgICAgICAgICAgICAgZm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OywmcXVv dDtzYW5zLXNlcmlmJnF1b3Q7OwogICAgICAgICAgICAgIGNvbG9yOiMxRjQ5N0QiPkl0cyBoYXBw ZW5pbmcgYWdhaW4sIHdoYXQgbG9ncyBkbyBJIG5lZWQgdG8KICAgICAgICAgICAgICBwcm92aWRl Pzwvc3Bhbj48L3A+CiAgICAgICAgICA8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0i Zm9udC1zaXplOjExLjBwdDsKICAgICAgICAgICAgICBmb250LWZhbWlseTomcXVvdDtDYWxpYnJp JnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7CiAgICAgICAgICAgICAgY29sb3I6IzFGNDk3 RCI+Jm5ic3A7PC9zcGFuPjwvcD4KICAgICAgICAgIDxkaXY+CiAgICAgICAgICAgIDxkaXYgc3R5 bGU9ImJvcmRlcjpub25lOyBib3JkZXItdG9wOnNvbGlkICNCNUM0REYgMS4wcHQ7CiAgICAgICAg ICAgICAgcGFkZGluZzozLjBwdCAwaW4gMGluIDBpbiI+CiAgICAgICAgICAgICAgPHAgY2xhc3M9 Ik1zb05vcm1hbCI+PGI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7CiAgICAgICAgICAg ICAgICAgICAgZm9udC1mYW1pbHk6JnF1b3Q7VGFob21hJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYm cXVvdDs7CiAgICAgICAgICAgICAgICAgICAgY29sb3I6d2luZG93dGV4dCI+RnJvbTo8L3NwYW4+ PC9iPjxzcGFuCiAgICAgICAgICAgICAgICAgIHN0eWxlPSJmb250LXNpemU6MTAuMHB0OwogICAg ICAgICAgICAgICAgICBmb250LWZhbWlseTomcXVvdDtUYWhvbWEmcXVvdDssJnF1b3Q7c2Fucy1z ZXJpZiZxdW90OzsKICAgICAgICAgICAgICAgICAgY29sb3I6d2luZG93dGV4dCI+IFZpbnplbnog RmVlbnN0cmEgWzxhCiAgICAgICAgICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIgog ICAgICAgICAgICAgICAgICAgIGhyZWY9Im1haWx0bzp2ZmVlbnN0ckByZWRoYXQuY29tIj5tYWls dG86dmZlZW5zdHJAcmVkaGF0LmNvbTwvYT5dCiAgICAgICAgICAgICAgICAgIDxicj4KICAgICAg ICAgICAgICAgICAgPGI+U2VudDo8L2I+IE1vbmRheSwgSmFudWFyeSAwNywgMjAxMyA2OjIwIEFN PGJyPgogICAgICAgICAgICAgICAgICA8Yj5Ubzo8L2I+IDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1 ZSIKICAgICAgICAgICAgICAgICAgICBocmVmPSJtYWlsdG86dXNlcnNAb3ZpcnQub3JnIj51c2Vy c0BvdmlydC5vcmc8L2E+PGJyPgogICAgICAgICAgICAgICAgICA8Yj5DYzo8L2I+IEpvbmF0aGFu IEhvcm5lPGJyPgogICAgICAgICAgICAgICAgICA8Yj5TdWJqZWN0OjwvYj4gUmU6IFtVc2Vyc10g bm9kZSBsb3NlcyBjb250YWN0IHdpdGgKICAgICAgICAgICAgICAgICAgcmhldiBhZ2VudDwvc3Bh bj48L3A+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAgICAgICA8cCBj bGFzcz0iTXNvTm9ybWFsIj4mbmJzcDs8L3A+CiAgICAgICAgICA8ZGl2PgogICAgICAgICAgICA8 cCBjbGFzcz0iTXNvTm9ybWFsIj5IaSBKb25hdGhhbiw8YnI+CiAgICAgICAgICAgICAgPGJyPgog ICAgICAgICAgICAgIENvdWxkIHlvdSBwcm92aWRlIHRoZSBsb2dzIGZvciBpdD88YnI+CiAgICAg ICAgICAgICAgVGhhbmtzLjxicj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgT24g MDEvMDQvMjAxMyAxMDozMiBBTSwgVmluemVueiBGZWVuc3RyYSB3cm90ZTo8L3A+CiAgICAgICAg ICA8L2Rpdj4KICAgICAgICAgIDxibG9ja3F1b3RlIHN0eWxlPSJtYXJnaW4tdG9wOjUuMHB0OyBt YXJnaW4tYm90dG9tOjUuMHB0Ij4KICAgICAgICAgICAgPGRpdj4KICAgICAgICAgICAgICA8cCBj bGFzcz0iTXNvTm9ybWFsIj5IaSw8YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgICBPbiAwMS8wMy8yMDEzIDA4OjUzIFBNLCBKb25hdGhhbiBIb3JuZSB3cm90ZTo8L3A+CiAg ICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8YmxvY2txdW90ZSBzdHlsZT0ibWFyZ2luLXRv cDo1LjBwdDsgbWFyZ2luLWJvdHRvbTo1LjBwdCI+CiAgICAgICAgICAgICAgPGRpdj4KICAgICAg ICAgICAgICAgIDxkaXY+CiAgICAgICAgICAgICAgICAgIDxkaXY+CiAgICAgICAgICAgICAgICAg ICAgPHAgY2xhc3M9Ik1zb05vcm1hbCI+aSBoYXZlIDIgbm9kZXMgbm93IHRoYXQgaGF2ZQogICAg ICAgICAgICAgICAgICAgICAgYm90aCBsb3N0IGNvbnRhY3Qgd2l0aCB0aGUgcmhldiBhZ2VudHMg b24gZ3Vlc3QKICAgICAgICAgICAgICAgICAgICAgIFZNcy4gJm5ic3A7aWYgaSBtaWdyYXRlIHRo ZSBndWVzdCB0byBhbm90aGVyIG5vZGUsIHRoZQogICAgICAgICAgICAgICAgICAgICAgSVAgYWRk cmVzcyBhbmQgbWVtb3J5IHVzYWdlIGltbWVkaWF0ZWx5IHNob3cgdXAuCiAgICAgICAgICAgICAg ICAgICAgICAmbmJzcDttaWdyYXRlIGEgZ3Vlc3QgYmFjaywgYW5kIHRoZSBJUC9tZW1vcnkgaW5m bwogICAgICAgICAgICAgICAgICAgICAgZGlzYXBwZWFycy48L3A+CiAgICAgICAgICAgICAgICAg IDwvZGl2PgogICAgICAgICAgICAgICAgICA8ZGl2PgogICAgICAgICAgICAgICAgICAgIDxwIGNs YXNzPSJNc29Ob3JtYWwiPiZuYnNwOzwvcD4KICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAg ICAgICAgICAgICAgIDxkaXY+CiAgICAgICAgICAgICAgICAgICAgPHAgY2xhc3M9Ik1zb05vcm1h bCI+aG93IGNhbiBpIHRyb3VibGVzaG9vdCB3aGF0IGlzCiAgICAgICAgICAgICAgICAgICAgICBj YXVzaW5nIHRoaXM/PC9wPgogICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAg IDwvZGl2PgogICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAg ICAgICAgICAgIDxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBw dCI+PGJyPgogICAgICAgICAgICAgIEl0J2QgYmUgbmljZSBpZiB5b3UgY291bGQgc2VuZCBtZSB0 aGUgbG9ncyBmcm9tCiAgICAgICAgICAgICAgL3Zhci9sb2cvdmRzbS92ZHNtKiB3aGljaCBhcmUg b24gdGhlIG5vZGUgd2hlcmUgaXQgbG9vc2VzCiAgICAgICAgICAgICAgdGhlIGNvbnRhY3QuIEkg YW0gY3VycmVudGx5IHRyeWluZyB0byBmaW5kIGV4YWN0bHkgdGhlCiAgICAgICAgICAgICAgY2F1 c2Ugb2Ygc3VjaCBhbiBpc3N1ZSBhcyB3ZWxsIGFuZCBpdCB3b3VsZCBiZSBoZWxwZnVsIGlmCiAg ICAgICAgICAgICAgd2UgY291bGQgYmUgcHJvdmlkZWQgd2l0aCBsb2cgZmlsZXMuIEN1cnJlbnRs eSBJIGFtIHN0aWxsCiAgICAgICAgICAgICAgbGFja2luZyB0aGVtLjxicj4KICAgICAgICAgICAg ICBJIHdpbGwgY2hlY2sgdGhlbSBhbmQgbGV0IHlvdSBrbm93IHdoYXQgSSB3aWxsIGJlIGFibGUg dG8KICAgICAgICAgICAgICBmaW5kLjxicj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgSWYgeW91IHdvdWxkbid0IG1pbmQgSSB3b3VsZCBiZSByZWZlcnJpbmcgdG8geW91IGZvciBt b3JlCiAgICAgICAgICAgICAgcXVlc3Rpb25zIGlmIHNvbWUgd2lsbCBjb21lIHVwIGR1cmluZyB0 aGUgc2VhcmNoIGZvciB0aGUKICAgICAgICAgICAgICBjYXVzZSBhbmQgSSB3aWxsIGxldCB5b3Ug a25vdyB0aGUgcmVzdWx0cyBhc2FwLjwvcD4KICAgICAgICAgICAgPGRpdj4KICAgICAgICAgICAg ICA8ZGl2PgogICAgICAgICAgICAgICAgPGRpdj4KICAgICAgICAgICAgICAgICAgPHAgY2xhc3M9 Ik1zb05vcm1hbCI+Jm5ic3A7PC9wPgogICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAg ICAgICA8ZGl2PgogICAgICAgICAgICAgICAgICA8cCBjbGFzcz0iTXNvTm9ybWFsIj50aGFua3Ms PC9wPgogICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICA8ZGl2PgogICAgICAg ICAgICAgICAgICA8cCBjbGFzcz0iTXNvTm9ybWFsIj5qb25hdGhhbjwvcD4KICAgICAgICAgICAg ICAgIDwvZGl2PgogICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8L2Rpdj4KICAgICAg ICAgICAgPHAgY2xhc3M9Ik1zb05vcm1hbCI+Jm5ic3A7PC9wPgogICAgICAgICAgICA8cCBjbGFz cz0iTXNvTm9ybWFsIiBzdHlsZT0ibWFyZ2luLWJvdHRvbToxMi4wcHQiPiZuYnNwOzwvcD4KICAg ICAgICAgICAgPHByZT4tLSA8L3ByZT4KICAgICAgICAgICAgPHByZT5SZWdhcmRzLDwvcHJlPgog ICAgICAgICAgICA8cHJlPiZuYnNwOzwvcHJlPgogICAgICAgICAgICA8cHJlPlZpbnplbnogRmVl bnN0cmEgfCBTZW5pb3IgU29mdHdhcmUgRW5naW5lZXI8L3ByZT4KICAgICAgICAgICAgPHByZT5S ZWRIYXQgRW5naW5lZXJpbmcgVmlydHVhbGl6YXRpb24gUiAmYW1wOyBEPC9wcmU+CiAgICAgICAg ICAgIDxwcmU+UGhvbmU6ICs0MjAgNTMyIDI5NCA2MjU8L3ByZT4KICAgICAgICAgICAgPHByZT5J UkM6IHZmZWVuc3RyIG9yIGV2aWxpc3NpbW88L3ByZT4KICAgICAgICAgICAgPHByZT4mbmJzcDs8 L3ByZT4KICAgICAgICAgICAgPHByZT5CZXR0ZXIgdGVjaG5vbG9neS4gRmFzdGVyIGlubm92YXRp b24uIFBvd2VyZWQgYnkgY29tbXVuaXR5IGNvbGxhYm9yYXRpb24uPC9wcmU+CiAgICAgICAgICAg IDxwcmU+U2VlIGhvdyBpdCB3b3JrcyBhdCByZWRoYXQuY29tPC9wcmU+CiAgICAgICAgICAgIDxw IGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBwdCI+PGJyPgogICAg ICAgICAgICAgIDxicj4KICAgICAgICAgICAgPC9wPgogICAgICAgICAgICA8cHJlPl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPC9wcmU+CiAgICAgICAgICAg IDxwcmU+VXNlcnMgbWFpbGluZyBsaXN0PC9wcmU+CiAgICAgICAgICAgIDxwcmU+PGEgbW96LWRv LW5vdC1zZW5kPSJ0cnVlIiBocmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIj5Vc2Vyc0Bvdmly dC5vcmc8L2E+PC9wcmU+CiAgICAgICAgICAgIDxwcmU+PGEgbW96LWRvLW5vdC1zZW5kPSJ0cnVl IiBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMiPmh0 dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VyczwvYT48L3ByZT4KICAg ICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICAgIDxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxl PSJtYXJnaW4tYm90dG9tOjEyLjBwdCI+PGJyPgogICAgICAgICAgICA8YnI+CiAgICAgICAgICA8 L3A+CiAgICAgICAgICA8cHJlPi0tIDwvcHJlPgogICAgICAgICAgPHByZT5SZWdhcmRzLDwvcHJl PgogICAgICAgICAgPHByZT4mbmJzcDs8L3ByZT4KICAgICAgICAgIDxwcmU+VmluemVueiBGZWVu c3RyYSB8IFNlbmlvciBTb2Z0d2FyZSBFbmdpbmVlcjwvcHJlPgogICAgICAgICAgPHByZT5SZWRI YXQgRW5naW5lZXJpbmcgVmlydHVhbGl6YXRpb24gUiAmYW1wOyBEPC9wcmU+CiAgICAgICAgICA8 cHJlPlBob25lOiArNDIwIDUzMiAyOTQgNjI1PC9wcmU+CiAgICAgICAgICA8cHJlPklSQzogdmZl ZW5zdHIgb3IgZXZpbGlzc2ltbzwvcHJlPgogICAgICAgICAgPHByZT4mbmJzcDs8L3ByZT4KICAg ICAgICAgIDxwcmU+QmV0dGVyIHRlY2hub2xvZ3kuIEZhc3RlciBpbm5vdmF0aW9uLiBQb3dlcmVk IGJ5IGNvbW11bml0eSBjb2xsYWJvcmF0aW9uLjwvcHJlPgogICAgICAgICAgPHByZT5TZWUgaG93 IGl0IHdvcmtzIGF0IHJlZGhhdC5jb208L3ByZT4KICAgICAgICA8L2Rpdj4KICAgICAgICA8cCBj bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iY29sb3I6d2luZG93dGV4dCI+Jm5ic3A7PC9z cGFuPjwvcD4KICAgICAgICA8ZGl2IGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJ0ZXh0LWFsaWdu OmNlbnRlciIgYWxpZ249ImNlbnRlciI+PHNwYW4KICAgICAgICAgICAgc3R5bGU9ImNvbG9yOndp bmRvd3RleHQiPgogICAgICAgICAgICA8aHIgYWxpZ249ImNlbnRlciIgc2l6ZT0iMiIgd2lkdGg9 IjEwMCUiPgogICAgICAgICAgPC9zcGFuPjwvZGl2PgogICAgICAgIDxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6Ny41cHQ7CiAgICAgICAgICAgIGZvbnQtZmFtaWx5 OiZxdW90O0FyaWFsJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmcXVvdDs7CiAgICAgICAgICAgIGNv bG9yOmdyYXkiPlRoaXMgaXMgYSBQUklWQVRFIG1lc3NhZ2UuIElmIHlvdSBhcmUgbm90IHRoZQog ICAgICAgICAgICBpbnRlbmRlZCByZWNpcGllbnQsIHBsZWFzZSBkZWxldGUgd2l0aG91dCBjb3B5 aW5nIGFuZCBraW5kbHkKICAgICAgICAgICAgYWR2aXNlIHVzIGJ5IGUtbWFpbCBvZiB0aGUgbWlz dGFrZSBpbiBkZWxpdmVyeS4gTk9URToKICAgICAgICAgICAgUmVnYXJkbGVzcyBvZiBjb250ZW50 LCB0aGlzIGUtbWFpbCBzaGFsbCBub3Qgb3BlcmF0ZSB0byBiaW5kCiAgICAgICAgICAgIFNLT1BP UyB0byBhbnkgb3JkZXIgb3Igb3RoZXIgY29udHJhY3QgdW5sZXNzIHB1cnN1YW50IHRvCiAgICAg ICAgICAgIGV4cGxpY2l0IHdyaXR0ZW4gYWdyZWVtZW50IG9yIGdvdmVybm1lbnQgaW5pdGlhdGl2 ZQogICAgICAgICAgICBleHByZXNzbHkgcGVybWl0dGluZyB0aGUgdXNlIG9mIGUtbWFpbCBmb3Ig c3VjaCBwdXJwb3NlLjwvc3Bhbj48c3BhbgogICAgICAgICAgICBzdHlsZT0iY29sb3I6d2luZG93 dGV4dCI+CiAgICAgICAgICA8L3NwYW4+PC9wPgogICAgICA8L2Rpdj4KICAgICAgPGJyPgogICAg ICA8aHI+CiAgICAgIDxmb250IGNvbG9yPSJHcmF5IiBmYWNlPSJBcmlhbCIgc2l6ZT0iMSI+VGhp cyBpcyBhIFBSSVZBVEUKICAgICAgICBtZXNzYWdlLiBJZiB5b3UgYXJlIG5vdCB0aGUgaW50ZW5k ZWQgcmVjaXBpZW50LCBwbGVhc2UgZGVsZXRlCiAgICAgICAgd2l0aG91dCBjb3B5aW5nIGFuZCBr aW5kbHkgYWR2aXNlIHVzIGJ5IGUtbWFpbCBvZiB0aGUgbWlzdGFrZSBpbgogICAgICAgIGRlbGl2 ZXJ5LiBOT1RFOiBSZWdhcmRsZXNzIG9mIGNvbnRlbnQsIHRoaXMgZS1tYWlsIHNoYWxsIG5vdAog ICAgICAgIG9wZXJhdGUgdG8gYmluZCBTS09QT1MgdG8gYW55IG9yZGVyIG9yIG90aGVyIGNvbnRy YWN0IHVubGVzcwogICAgICAgIHB1cnN1YW50IHRvIGV4cGxpY2l0IHdyaXR0ZW4gYWdyZWVtZW50 IG9yIGdvdmVybm1lbnQgaW5pdGlhdGl2ZQogICAgICAgIGV4cHJlc3NseSBwZXJtaXR0aW5nIHRo ZSB1c2Ugb2YgZS1tYWlsIGZvciBzdWNoIHB1cnBvc2UuPC9mb250PgogICAgICA8YnI+CiAgICAg IDxmaWVsZHNldCBjbGFzcz0ibWltZUF0dGFjaG1lbnRIZWFkZXIiPjwvZmllbGRzZXQ+CiAgICAg IDxicj4KICAgICAgPHByZSB3cmFwPSIiPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fClVzZXJzIG1haWxpbmcgbGlzdAo8YSBjbGFzcz0ibW96LXR4dC1saW5r LWFiYnJldmlhdGVkIiBocmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIj5Vc2Vyc0BvdmlydC5v cmc8L2E+CjxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJlZXRleHQiIGhyZWY9Imh0dHA6Ly9saXN0 cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyI+aHR0cDovL2xpc3RzLm92aXJ0Lm9y Zy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzPC9hPgo8L3ByZT4KICAgIDwvYmxvY2txdW90ZT4KICAg IDxicj4KICAgIDxicj4KICAgIDxwcmUgY2xhc3M9Im1vei1zaWduYXR1cmUiIGNvbHM9IjcyIj4t LSAKUmVnYXJkcywKClZpbnplbnogRmVlbnN0cmEgfCBTZW5pb3IgU29mdHdhcmUgRW5naW5lZXIK UmVkSGF0IEVuZ2luZWVyaW5nIFZpcnR1YWxpemF0aW9uIFIgJmFtcDsgRApQaG9uZTogKzQyMCA1 MzIgMjk0IDYyNQpJUkM6IHZmZWVuc3RyIG9yIGV2aWxpc3NpbW8KCkJldHRlciB0ZWNobm9sb2d5 LiBGYXN0ZXIgaW5ub3ZhdGlvbi4gUG93ZXJlZCBieSBjb21tdW5pdHkgY29sbGFib3JhdGlvbi4K U2VlIGhvdyBpdCB3b3JrcyBhdCByZWRoYXQuY29tPC9wcmU+CiAgPC9ib2R5Pgo8L2h0bWw+Cgot LS0tLS0tLS0tLS0tLTA5MDIwNjA4MDEwNzA3MDQwMDAzMDUwMi0tCg== --===============8234729336360563580==--