From mskrivan at redhat.com Thu Apr 20 08:35:27 2017 Content-Type: multipart/mixed; boundary="===============2227388091000325532==" MIME-Version: 1.0 From: Michal Skrivanek To: users at ovirt.org Subject: Re: [ovirt-users] Question about Huge Pages Date: Thu, 20 Apr 2017 10:35:23 +0200 Message-ID: In-Reply-To: CAG2kNCwqF2zgFEscObnab0-cs+FvVnY=hr+CoDg1H0mny-y1+g@mail.gmail.com --===============2227388091000325532== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable --Apple-Mail=3D_2B6E5EAC-A5EA-4F24-8549-C3234A9C6DD6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=3Dutf-8 > On 19 Apr 2017, at 16:28, Gianluca Cecchi = =3D wrote: >=3D20 > On Wed, Apr 19, 2017 at 3:44 PM, Martin Polednik > wrote: >=3D20 >=3D20 > If you are using recent CentOS (or I guess Fedora), there isn't any > extra setup required. Just create the custom property: >=3D20 > Both my engine and my hosts are CentOS 7.3 + updates that=3DE2=3D80=3D99s good > =3D20 >=3D20 > On the host where engine is running: >=3D20 > $ engine-config -s "UserDefinedVMProperties=3D3Dhugepages=3D3D^.*$" > $ service ovirt-engine restart >=3D20 > and you should see 'hugepages' when editing a VM under custom =3D properties. >=3D20 > So no vdsm hook at all to install? today you still need the hook. >=3D20 > =3D20 > Set the number to (desired memory / 2048) and you're good to go. The > VM will run with it's memory backed by hugepages. >=3D20 > As in sysctl.conf? So that if I want 4Gb of Huge Pages I have to set =3D 2048? yes. there might be some=3D20 >=3D20 > =3D20 > If you need > hugepages even inside the VM, do whatever you would do on a physical > host. >=3D20 > mpolednik >=3D20 >=3D20 > yes, the main subject is to have Huge Pages inside the guest, so that =3D Oracle RDBMS at startup detect them and use them yes, so if you do that via sysctl.conf on real HW just do the same here, = =3D or modify kernel cmdline. Note that those are two separate things the hook is making QEMU process use hugepages memory in the host - that =3D improves performance of any VM then how it looks in guest is no concern to oVirt, it=3DE2=3D80=3D99s =3D guest-side hugepages. You can enable/set them regardless the previous =3D step, which may be fine if you just want to expose the capability to =3D some app - e.g. in testing that the guest-side Oracle can work with =3D hugepages in the guest. But you probably want both Oracle to see hugepages and also actually use = =3D them - then you need both reserve that on host for qemu process and then = =3D inside guest reserve that for oracle. I.e. you need to add a =3D =3DE2=3D80=3D9Cbuffer=3DE2=3D80=3D9D on host side to accommodate the non-hu= gepages =3D parts of the guest e.g. on 24GB host you can reserve 20GB hugepages for =3D VMs to use, and then run a VM with 20GB memory, reserving 16GB hugepages = =3D inside the guest for oracle to use. Thanks, michal >=3D20 > Gianluca=3D20 --Apple-Mail=3D_2B6E5EAC-A5EA-4F24-8549-C3234A9C6DD6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=3Dutf-8

On 19 Apr 2017, at 16:28, Gianluca Cecchi <gianluca.cecchi(a)gmail.com> wrote:

On W= ed, =3D Apr 19, 2017 at 3:44 PM, Martin Polednik <mpolednik(a)redhat.com> wrote:

If you are using recent CentOS (or I guess Fedora), there isn't any
extra setup required. Just create the custom property:

Both my engine and my hosts are CentOS 7.3 + =3D updates

that=3DE2=3D80=3D99s good

On the host where engine is running:

$ engine-config -s "UserDefinedVMProperties=3D3Dhugepages=3D3D^.*$"
$ service ovirt-engine restart

and you should see 'hugepages' when editing a VM under custom =3D properties.

So no vdsm hook at all to =3D install?

today you still need the hook.

Set the number to (desired memory / 2048) and you're good to go. The
VM will run with it's memory backed by hugepages.

As in sysctl.conf? So= =3D that if I want 4Gb of Huge Pages I have to set =3D 2048?

yes. there might be some

If you need
hugepages even inside the VM, do whatever you would do on a physical
host.

mpolednik

yes, the main subject is to have Huge = =3D Pages inside the guest, so that Oracle RDBMS at startup detect them and =3D use them

yes, so if you do that via sysctl.conf on real HW just = =3D do the same here, or modify kernel cmdline.

Note that those are two separate =3D things

the hook is making QEMU process use hugepages memory in = =3D the host - that improves performance of any VM

then how it =3D looks in guest is no concern to oVirt, it=3DE2=3D80=3D99s guest-side =3D hugepages. You can enable/set them regardless the previous step, which =3D may be fine if you just want to expose the capability to some app =3D - e.g. in testing that the guest-side Oracle can work with =3D hugepages in the guest.

But you probably want both Oracle to =3D see hugepages and also actually use them - then you need both reserve =3D that on host for qemu process and then inside guest reserve that for =3D oracle. I.e. you need to add a =3DE2=3D80=3D9Cbuffer=3DE2=3D80=3D9D on host= side to =3D accommodate the non-hugepages parts of the guest e.g. on 24GB host you =3D can reserve 20GB hugepages for VMs to use, and then run a VM with 20GB =3D memory, reserving 16GB hugepages inside the guest for oracle to =3D use.

Thanks,

michal

Gianluca

=3D --Apple-Mail=3D_2B6E5EAC-A5EA-4F24-8549-C3234A9C6DD6-- --===============2227388091000325532== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" Ci0tQXBwbGUtTWFpbD1fMkI2RTVFQUMtQTVFQS00RjI0LTg1NDktQzMyMzRBOUM2REQ2CkNvbnRl bnQtVHJhbnNmZXItRW5jb2Rpbmc6IHF1b3RlZC1wcmludGFibGUKQ29udGVudC1UeXBlOiB0ZXh0 L3BsYWluOwoJY2hhcnNldD11dGYtOAoKCj4gT24gMTkgQXByIDIwMTcsIGF0IDE2OjI4LCBHaWFu bHVjYSBDZWNjaGkgPGdpYW5sdWNhLmNlY2NoaUBnbWFpbC5jb20+ID0Kd3JvdGU6Cj49MjAKPiBP biBXZWQsIEFwciAxOSwgMjAxNyBhdCAzOjQ0IFBNLCBNYXJ0aW4gUG9sZWRuaWsgPG1wb2xlZG5p a0ByZWRoYXQuY29tID0KPG1haWx0bzptcG9sZWRuaWtAcmVkaGF0LmNvbT4+IHdyb3RlOgo+PTIw Cj49MjAKPiBJZiB5b3UgYXJlIHVzaW5nIHJlY2VudCBDZW50T1MgKG9yIEkgZ3Vlc3MgRmVkb3Jh KSwgdGhlcmUgaXNuJ3QgYW55Cj4gZXh0cmEgc2V0dXAgcmVxdWlyZWQuIEp1c3QgY3JlYXRlIHRo ZSBjdXN0b20gcHJvcGVydHk6Cj49MjAKPiBCb3RoIG15IGVuZ2luZSBhbmQgbXkgaG9zdHMgYXJl IENlbnRPUyA3LjMgKyB1cGRhdGVzCgp0aGF0PUUyPTgwPTk5cyBnb29kCgo+ID0yMAo+PTIwCj4g T24gdGhlIGhvc3Qgd2hlcmUgZW5naW5lIGlzIHJ1bm5pbmc6Cj49MjAKPiAkIGVuZ2luZS1jb25m aWcgLXMgIlVzZXJEZWZpbmVkVk1Qcm9wZXJ0aWVzPTNEaHVnZXBhZ2VzPTNEXi4qJCIKPiAkIHNl cnZpY2Ugb3ZpcnQtZW5naW5lIHJlc3RhcnQKPj0yMAo+IGFuZCB5b3Ugc2hvdWxkIHNlZSAnaHVn ZXBhZ2VzJyB3aGVuIGVkaXRpbmcgYSBWTSB1bmRlciBjdXN0b20gPQpwcm9wZXJ0aWVzLgo+PTIw Cj4gU28gbm8gdmRzbSBob29rIGF0IGFsbCB0byBpbnN0YWxsPwoKdG9kYXkgeW91IHN0aWxsIG5l ZWQgdGhlIGhvb2suCgo+PTIwCj4gPTIwCj4gU2V0IHRoZSBudW1iZXIgdG8gKGRlc2lyZWQgbWVt b3J5IC8gMjA0OCkgYW5kIHlvdSdyZSBnb29kIHRvIGdvLiBUaGUKPiBWTSB3aWxsIHJ1biB3aXRo IGl0J3MgbWVtb3J5IGJhY2tlZCBieSBodWdlcGFnZXMuCj49MjAKPiBBcyBpbiBzeXNjdGwuY29u Zj8gU28gdGhhdCBpZiBJIHdhbnQgNEdiIG9mIEh1Z2UgUGFnZXMgSSBoYXZlIHRvIHNldCA9CjIw NDg/Cgp5ZXMuIHRoZXJlIG1pZ2h0IGJlIHNvbWU9MjAKCj49MjAKPiA9MjAKPiBJZiB5b3UgbmVl ZAo+IGh1Z2VwYWdlcyBldmVuIGluc2lkZSB0aGUgVk0sIGRvIHdoYXRldmVyIHlvdSB3b3VsZCBk byBvbiBhIHBoeXNpY2FsCj4gaG9zdC4KPj0yMAo+IG1wb2xlZG5pawo+PTIwCj49MjAKPiB5ZXMs IHRoZSBtYWluIHN1YmplY3QgaXMgdG8gaGF2ZSBIdWdlIFBhZ2VzIGluc2lkZSB0aGUgZ3Vlc3Qs IHNvIHRoYXQgPQpPcmFjbGUgUkRCTVMgYXQgc3RhcnR1cCBkZXRlY3QgdGhlbSBhbmQgdXNlIHRo ZW0KCnllcywgc28gaWYgeW91IGRvIHRoYXQgdmlhIHN5c2N0bC5jb25mIG9uIHJlYWwgSFcganVz dCBkbyB0aGUgc2FtZSBoZXJlLCA9Cm9yIG1vZGlmeSBrZXJuZWwgY21kbGluZS4KCk5vdGUgdGhh dCB0aG9zZSBhcmUgdHdvIHNlcGFyYXRlIHRoaW5ncwp0aGUgaG9vayBpcyBtYWtpbmcgUUVNVSBw cm9jZXNzIHVzZSBodWdlcGFnZXMgbWVtb3J5IGluIHRoZSBob3N0IC0gdGhhdCA9CmltcHJvdmVz IHBlcmZvcm1hbmNlIG9mIGFueSBWTQp0aGVuIGhvdyBpdCBsb29rcyBpbiBndWVzdCBpcyBubyBj b25jZXJuIHRvIG9WaXJ0LCBpdD1FMj04MD05OXMgPQpndWVzdC1zaWRlIGh1Z2VwYWdlcy4gWW91 IGNhbiBlbmFibGUvc2V0IHRoZW0gcmVnYXJkbGVzcyB0aGUgcHJldmlvdXMgPQpzdGVwLCB3aGlj aCBtYXkgYmUgZmluZSBpZiB5b3UganVzdCB3YW50IHRvIGV4cG9zZSB0aGUgY2FwYWJpbGl0eSB0 byA9CnNvbWUgYXBwICAtIGUuZy4gaW4gdGVzdGluZyB0aGF0IHRoZSBndWVzdC1zaWRlIE9yYWNs ZSBjYW4gd29yayB3aXRoID0KaHVnZXBhZ2VzIGluIHRoZSBndWVzdC4KQnV0IHlvdSBwcm9iYWJs eSB3YW50IGJvdGggT3JhY2xlIHRvIHNlZSBodWdlcGFnZXMgYW5kIGFsc28gYWN0dWFsbHkgdXNl ID0KdGhlbSAtIHRoZW4geW91IG5lZWQgYm90aCByZXNlcnZlIHRoYXQgb24gaG9zdCBmb3IgcWVt dSBwcm9jZXNzIGFuZCB0aGVuID0KaW5zaWRlIGd1ZXN0IHJlc2VydmUgdGhhdCBmb3Igb3JhY2xl LiBJLmUuIHlvdSBuZWVkIHRvIGFkZCBhID0KPUUyPTgwPTlDYnVmZmVyPUUyPTgwPTlEIG9uIGhv c3Qgc2lkZSB0byBhY2NvbW1vZGF0ZSB0aGUgbm9uLWh1Z2VwYWdlcyA9CnBhcnRzIG9mIHRoZSBn dWVzdCBlLmcuIG9uIDI0R0IgaG9zdCB5b3UgY2FuIHJlc2VydmUgMjBHQiBodWdlcGFnZXMgZm9y ID0KVk1zIHRvIHVzZSwgYW5kIHRoZW4gcnVuIGEgVk0gd2l0aCAyMEdCIG1lbW9yeSwgcmVzZXJ2 aW5nIDE2R0IgaHVnZXBhZ2VzID0KaW5zaWRlIHRoZSBndWVzdCBmb3Igb3JhY2xlIHRvIHVzZS4K ClRoYW5rcywKbWljaGFsCgo+PTIwCj4gR2lhbmx1Y2E9MjAKCgotLUFwcGxlLU1haWw9XzJCNkU1 RUFDLUE1RUEtNEYyNC04NTQ5LUMzMjM0QTlDNkRENgpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5n OiBxdW90ZWQtcHJpbnRhYmxlCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOwoJY2hhcnNldD11dGYt OAoKPGh0bWw+PGhlYWQ+PG1ldGEgaHR0cC1lcXVpdj0zRCJDb250ZW50LVR5cGUiIGNvbnRlbnQ9 M0QidGV4dC9odG1sID0KY2hhcnNldD0zRHV0Zi04Ij48L2hlYWQ+PGJvZHkgc3R5bGU9M0Qid29y ZC13cmFwOiBicmVhay13b3JkOyA9Ci13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgLXdlYmtpdC1s aW5lLWJyZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsiID0KY2xhc3M9M0QiIj48YnIgY2xhc3M9M0Qi Ij48ZGl2PjxibG9ja3F1b3RlIHR5cGU9M0QiY2l0ZSIgY2xhc3M9M0QiIj48ZGl2ID0KY2xhc3M9 M0QiIj5PbiAxOSBBcHIgMjAxNywgYXQgMTY6MjgsIEdpYW5sdWNhIENlY2NoaSAmbHQ7PGEgPQpo cmVmPTNEIm1haWx0bzpnaWFubHVjYS5jZWNjaGlAZ21haWwuY29tIiA9CmNsYXNzPTNEIiI+Z2lh bmx1Y2EuY2VjY2hpQGdtYWlsLmNvbTwvYT4mZ3Q7IHdyb3RlOjwvZGl2PjxiciA9CmNsYXNzPTNE IkFwcGxlLWludGVyY2hhbmdlLW5ld2xpbmUiPjxkaXYgY2xhc3M9M0QiIj48ZGl2IGRpcj0zRCJs dHIiID0KY2xhc3M9M0QiIj48ZGl2IGNsYXNzPTNEImdtYWlsX2V4dHJhIj48ZGl2IGNsYXNzPTNE ImdtYWlsX3F1b3RlIj5PbiBXZWQsID0KQXByIDE5LCAyMDE3IGF0IDM6NDQgUE0sIE1hcnRpbiBQ b2xlZG5payA8c3BhbiBkaXI9M0QibHRyIiA9CmNsYXNzPTNEIiI+Jmx0OzxhIGhyZWY9M0QibWFp bHRvOm1wb2xlZG5pa0ByZWRoYXQuY29tIiB0YXJnZXQ9M0QiX2JsYW5rIiA9CmNsYXNzPTNEIiI+ bXBvbGVkbmlrQHJlZGhhdC5jb208L2E+Jmd0Ozwvc3Bhbj4gd3JvdGU6PGJyID0KY2xhc3M9M0Qi Ij48YmxvY2txdW90ZSBjbGFzcz0zRCJnbWFpbF9xdW90ZSIgc3R5bGU9M0QibWFyZ2luOjAgMCAw ID0KLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2 ID0KY2xhc3M9M0QiSE9FblpiIj48ZGl2IGNsYXNzPTNEImg1Ij48YmxvY2txdW90ZSBjbGFzcz0z RCJnbWFpbF9xdW90ZSIgPQpzdHlsZT0zRCJtYXJnaW46MCAwIDAgLjhleDtib3JkZXItbGVmdDox cHggI2NjYyA9CnNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPjxiciBjbGFzcz0zRCIiPgo8L2Jsb2Nr cXVvdGU+CjxiciBjbGFzcz0zRCIiPjwvZGl2PjwvZGl2PgpJZiB5b3UgYXJlIHVzaW5nIHJlY2Vu dCBDZW50T1MgKG9yIEkgZ3Vlc3MgRmVkb3JhKSwgdGhlcmUgaXNuJ3QgYW55PGJyID0KY2xhc3M9 M0QiIj4KZXh0cmEgc2V0dXAgcmVxdWlyZWQuIEp1c3QgY3JlYXRlIHRoZSBjdXN0b20gcHJvcGVy dHk6PGJyID0KY2xhc3M9M0QiIj48L2Jsb2NrcXVvdGU+PGRpdiBjbGFzcz0zRCIiPjxiciBjbGFz cz0zRCIiPjwvZGl2PjxkaXYgPQpjbGFzcz0zRCIiPkJvdGggbXkgZW5naW5lIGFuZCBteSBob3N0 cyBhcmUgQ2VudE9TIDcuMyArID0KdXBkYXRlczwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2 PjwvYmxvY2txdW90ZT48ZGl2PjxiciA9CmNsYXNzPTNEIiI+PC9kaXY+dGhhdD1FMj04MD05OXMg Z29vZDwvZGl2PjxkaXY+PGJyIGNsYXNzPTNEIiI+PGJsb2NrcXVvdGUgPQp0eXBlPTNEImNpdGUi IGNsYXNzPTNEIiI+PGRpdiBjbGFzcz0zRCIiPjxkaXYgZGlyPTNEImx0ciIgY2xhc3M9M0QiIj48 ZGl2ID0KY2xhc3M9M0QiZ21haWxfZXh0cmEiPjxkaXYgY2xhc3M9M0QiZ21haWxfcXVvdGUiPjxk aXYgPQpjbGFzcz0zRCIiPiZuYnNwOzwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPTNEImdtYWlsX3F1 b3RlIiBzdHlsZT0zRCJtYXJnaW46MD0KIDAgMCAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNv bGlkO3BhZGRpbmctbGVmdDoxZXgiPgo8YnIgY2xhc3M9M0QiIj4KT24gdGhlIGhvc3Qgd2hlcmUg ZW5naW5lIGlzIHJ1bm5pbmc6PGJyIGNsYXNzPTNEIiI+CjxiciBjbGFzcz0zRCIiPgokIGVuZ2lu ZS1jb25maWcgLXMgIlVzZXJEZWZpbmVkVk1Qcm9wZXJ0aWVzPTNEaHVnZXA8d2JyID0KY2xhc3M9 M0QiIj5hZ2VzPTNEXi4qJCI8YnIgY2xhc3M9M0QiIj4KJCBzZXJ2aWNlIG92aXJ0LWVuZ2luZSBy ZXN0YXJ0PGJyIGNsYXNzPTNEIiI+CjxiciBjbGFzcz0zRCIiPgphbmQgeW91IHNob3VsZCBzZWUg J2h1Z2VwYWdlcycgd2hlbiBlZGl0aW5nIGEgVk0gdW5kZXIgY3VzdG9tID0KcHJvcGVydGllcy48 YnIgY2xhc3M9M0QiIj48L2Jsb2NrcXVvdGU+PGRpdiBjbGFzcz0zRCIiPjxiciA9CmNsYXNzPTNE IiI+PC9kaXY+PGRpdiBjbGFzcz0zRCIiPlNvIG5vIHZkc20gaG9vayBhdCBhbGwgdG8gPQppbnN0 YWxsPzwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2PjwvYmxvY2txdW90ZT48ZGl2PjxiciA9 CmNsYXNzPTNEIiI+PC9kaXY+dG9kYXkgeW91IHN0aWxsIG5lZWQgdGhlIGhvb2suPC9kaXY+PGRp dj48YnIgPQpjbGFzcz0zRCIiPjxibG9ja3F1b3RlIHR5cGU9M0QiY2l0ZSIgY2xhc3M9M0QiIj48 ZGl2IGNsYXNzPTNEIiI+PGRpdiA9CmRpcj0zRCJsdHIiIGNsYXNzPTNEIiI+PGRpdiBjbGFzcz0z RCJnbWFpbF9leHRyYSI+PGRpdiA9CmNsYXNzPTNEImdtYWlsX3F1b3RlIj48ZGl2IGNsYXNzPTNE IiI+PGJyIGNsYXNzPTNEIiI+PC9kaXY+PGRpdiA9CmNsYXNzPTNEIiI+Jm5ic3A7PC9kaXY+PGJs b2NrcXVvdGUgY2xhc3M9M0QiZ21haWxfcXVvdGUiIHN0eWxlPTNEIm1hcmdpbjowPQogMCAwIC44 ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+ClNldCB0aGUg bnVtYmVyIHRvIChkZXNpcmVkIG1lbW9yeSAvIDIwNDgpIGFuZCB5b3UncmUgZ29vZCB0byBnby4g VGhlPGJyID0KY2xhc3M9M0QiIj4KVk0gd2lsbCBydW4gd2l0aCBpdCdzIG1lbW9yeSBiYWNrZWQg YnkgaHVnZXBhZ2VzLiA8L2Jsb2NrcXVvdGU+PGRpdiA9CmNsYXNzPTNEIiI+PGJyIGNsYXNzPTNE IiI+PC9kaXY+PGRpdiBjbGFzcz0zRCIiPkFzIGluIHN5c2N0bC5jb25mPyBTbyA9CnRoYXQgaWYg SSB3YW50IDRHYiBvZiBIdWdlIFBhZ2VzIEkgaGF2ZSB0byBzZXQgPQoyMDQ4PzwvZGl2PjwvZGl2 PjwvZGl2PjwvZGl2PjwvZGl2PjwvYmxvY2txdW90ZT48ZGl2PjxiciA9CmNsYXNzPTNEIiI+PC9k aXY+eWVzLiB0aGVyZSBtaWdodCBiZSBzb21lJm5ic3A7PC9kaXY+PGRpdj48YnIgPQpjbGFzcz0z RCIiPjxibG9ja3F1b3RlIHR5cGU9M0QiY2l0ZSIgY2xhc3M9M0QiIj48ZGl2IGNsYXNzPTNEIiI+ PGRpdiA9CmRpcj0zRCJsdHIiIGNsYXNzPTNEIiI+PGRpdiBjbGFzcz0zRCJnbWFpbF9leHRyYSI+ PGRpdiA9CmNsYXNzPTNEImdtYWlsX3F1b3RlIj48ZGl2IGNsYXNzPTNEIiI+PGJyIGNsYXNzPTNE IiI+PC9kaXY+PGRpdiA9CmNsYXNzPTNEIiI+Jm5ic3A7PC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9 M0QiZ21haWxfcXVvdGUiIHN0eWxlPTNEIm1hcmdpbjowPQogMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6 MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+SWYgeW91IG5lZWQ8YnIgPQpjbGFzcz0z RCIiPgpodWdlcGFnZXMgZXZlbiBpbnNpZGUgdGhlIFZNLCBkbyB3aGF0ZXZlciB5b3Ugd291bGQg ZG8gb24gYSBwaHlzaWNhbDxiciA9CmNsYXNzPTNEIiI+Cmhvc3QuPGJyIGNsYXNzPTNEIiI+Cjxi ciBjbGFzcz0zRCIiPgptcG9sZWRuaWs8ZGl2IGNsYXNzPTNEIkhPRW5aYiI+PGRpdiBjbGFzcz0z RCJoNSI+PGJyID0KY2xhc3M9M0QiIj48L2Rpdj48L2Rpdj48L2Jsb2NrcXVvdGU+PGRpdiBjbGFz cz0zRCIiPjxiciA9CmNsYXNzPTNEIiI+PC9kaXY+PGRpdiBjbGFzcz0zRCIiPnllcywgdGhlIG1h aW4gc3ViamVjdCBpcyB0byBoYXZlIEh1Z2UgPQpQYWdlcyBpbnNpZGUgdGhlIGd1ZXN0LCBzbyB0 aGF0IE9yYWNsZSBSREJNUyBhdCBzdGFydHVwIGRldGVjdCB0aGVtIGFuZCA9CnVzZSB0aGVtPC9k aXY+PC9kaXY+PC9kaXY+PC9kaXY+PC9kaXY+PC9ibG9ja3F1b3RlPjxkaXY+PGJyID0KY2xhc3M9 M0QiIj48L2Rpdj55ZXMsIHNvIGlmIHlvdSBkbyB0aGF0IHZpYSBzeXNjdGwuY29uZiBvbiByZWFs IEhXIGp1c3QgPQpkbyB0aGUgc2FtZSBoZXJlLCBvciBtb2RpZnkga2VybmVsIGNtZGxpbmUuPC9k aXY+PGRpdj48YnIgPQpjbGFzcz0zRCIiPjwvZGl2PjxkaXY+Tm90ZSB0aGF0IHRob3NlIGFyZSB0 d28gc2VwYXJhdGUgPQp0aGluZ3M8L2Rpdj48ZGl2PnRoZSBob29rIGlzIG1ha2luZyBRRU1VIHBy b2Nlc3MgdXNlIGh1Z2VwYWdlcyBtZW1vcnkgaW4gPQp0aGUgaG9zdCAtIHRoYXQgaW1wcm92ZXMg cGVyZm9ybWFuY2Ugb2YgYW55IFZNPC9kaXY+PGRpdj50aGVuIGhvdyBpdCA9Cmxvb2tzIGluIGd1 ZXN0IGlzIG5vIGNvbmNlcm4gdG8gb1ZpcnQsIGl0PUUyPTgwPTk5cyBndWVzdC1zaWRlID0KaHVn ZXBhZ2VzLiBZb3UgY2FuIGVuYWJsZS9zZXQgdGhlbSByZWdhcmRsZXNzIHRoZSBwcmV2aW91cyBz dGVwLCB3aGljaCA9Cm1heSBiZSBmaW5lIGlmIHlvdSBqdXN0IHdhbnQgdG8gZXhwb3NlIHRoZSBj YXBhYmlsaXR5IHRvIHNvbWUgYXBwID0KJm5ic3A7LSBlLmcuIGluIHRlc3RpbmcgdGhhdCB0aGUg Z3Vlc3Qtc2lkZSBPcmFjbGUgY2FuIHdvcmsgd2l0aCA9Cmh1Z2VwYWdlcyBpbiB0aGUgZ3Vlc3Qu PC9kaXY+PGRpdj5CdXQgeW91IHByb2JhYmx5IHdhbnQgYm90aCBPcmFjbGUgdG8gPQpzZWUgaHVn ZXBhZ2VzIGFuZCBhbHNvIGFjdHVhbGx5IHVzZSB0aGVtIC0gdGhlbiB5b3UgbmVlZCBib3RoIHJl c2VydmUgPQp0aGF0IG9uIGhvc3QgZm9yIHFlbXUgcHJvY2VzcyBhbmQgdGhlbiBpbnNpZGUgZ3Vl c3QgcmVzZXJ2ZSB0aGF0IGZvciA9Cm9yYWNsZS4gSS5lLiB5b3UgbmVlZCB0byBhZGQgYSA9RTI9 ODA9OUNidWZmZXI9RTI9ODA9OUQgb24gaG9zdCBzaWRlIHRvID0KYWNjb21tb2RhdGUgdGhlIG5v bi1odWdlcGFnZXMgcGFydHMgb2YgdGhlIGd1ZXN0IGUuZy4gb24gMjRHQiBob3N0IHlvdSA9CmNh biByZXNlcnZlIDIwR0IgaHVnZXBhZ2VzIGZvciBWTXMgdG8gdXNlLCBhbmQgdGhlbiBydW4gYSBW TSB3aXRoIDIwR0IgPQptZW1vcnksIHJlc2VydmluZyAxNkdCIGh1Z2VwYWdlcyBpbnNpZGUgdGhl IGd1ZXN0IGZvciBvcmFjbGUgdG8gPQp1c2UuPC9kaXY+PGRpdj48YnIgPQpjbGFzcz0zRCIiPjwv ZGl2PjxkaXY+VGhhbmtzLDwvZGl2PjxkaXY+bWljaGFsPC9kaXY+PGRpdj48YnIgPQpjbGFzcz0z RCIiPjxibG9ja3F1b3RlIHR5cGU9M0QiY2l0ZSIgY2xhc3M9M0QiIj48ZGl2IGNsYXNzPTNEIiI+ PGRpdiA9CmRpcj0zRCJsdHIiIGNsYXNzPTNEIiI+PGRpdiBjbGFzcz0zRCJnbWFpbF9leHRyYSI+ PGRpdiA9CmNsYXNzPTNEImdtYWlsX3F1b3RlIj48ZGl2IGNsYXNzPTNEIiI+PGJyIGNsYXNzPTNE IiI+PC9kaXY+PGRpdiA9CmNsYXNzPTNEIiI+R2lhbmx1Y2EmbmJzcDs8L2Rpdj48L2Rpdj48L2Rp dj48L2Rpdj4KPC9kaXY+PC9ibG9ja3F1b3RlPjwvZGl2PjxiciBjbGFzcz0zRCIiPjwvYm9keT48 L2h0bWw+PQoKLS1BcHBsZS1NYWlsPV8yQjZFNUVBQy1BNUVBLTRGMjQtODU0OS1DMzIzNEE5QzZE RDYtLQo= --===============2227388091000325532==--