From deweyto at gmail.com Mon Jun 20 21:38:07 2016 Content-Type: multipart/mixed; boundary="===============1704740968678166690==" MIME-Version: 1.0 From: Dewey Du To: users at ovirt.org Subject: [ovirt-users] Is oVirt 3.6 with GlusterFS 3.7 recommended for production usage? Date: Tue, 21 Jun 2016 09:38:05 +0800 Message-ID: --===============1704740968678166690== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online servers. Is it recommended for production usage? Thx. --===============1704740968678166690== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+SSB3YW50IHRvIGRlcGxveSBvVmlydCAzLjYgd2l0aCBHbHVzdGVyRlMg My43IHRvIG15IG9ubGluZSBzZXJ2ZXJzLiBJcyBpdCByZWNvbW1lbmRlZCBmb3IgcHJvZHVjdGlv biB1c2FnZT/CoDxkaXY+PGJyPjwvZGl2PjxkaXY+VGh4LsKgPGJyPjwvZGl2PjwvZGl2Pgo= --===============1704740968678166690==-- From sabose at redhat.com Tue Jun 21 07:15:02 2016 Content-Type: multipart/mixed; boundary="===============6102599457828049905==" MIME-Version: 1.0 From: Sahina Bose To: users at ovirt.org Subject: Re: [ovirt-users] Is oVirt 3.6 with GlusterFS 3.7 recommended for production usage? Date: Tue, 21 Jun 2016 16:45:00 +0530 Message-ID: <576921B4.2050607@redhat.com> In-Reply-To: CAMSiEm5JAxmVJnMhD1-YsjS7siXbdyccr99KBBRBwKej4P0Vow@mail.gmail.com --===============6102599457828049905== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------030703040300030502060605 Content-Type: text/plain; charset=3Dwindows-1252; format=3Dflowed Content-Transfer-Encoding: 7bit Make sure that you use replica 3 gluster volumes for storing VM images. = Are you planning to deploy as a hyperconverged setup? Either way, try and use the latest ovirt 3.6 and glusterfs 3.7 (3.7.12 = that addresses bugs related to sharding and o-direct is due to be = released soon) On 06/21/2016 07:08 AM, Dewey Du wrote: > I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online servers. Is = > it recommended for production usage? > > Thx. > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users --------------030703040300030502060605 Content-Type: text/html; charset=3Dwindows-1252 Content-Transfer-Encoding: 8bit Make sure that you use replica 3 gluster volumes for storing VM images. Are you planning to deploy as a hyperconverged setup?
Either way, try and use the latest ovirt 3.6 and glusterfs 3.7 (3.7.12 that addresses bugs related to sharding and o-direct is due to be released soon)

On 06/21/2016 07:08 AM, Dewey Du wrote:<= br>

I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online servers. Is it recommended for production usage?=C2=A0

Thx.=C2=A0
_______________________________________________
Users mailing list
Use=
rs(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

--------------030703040300030502060605-- --===============6102599457828049905== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMzA3MDMwNDAzMDAwMzA1MDIwNjA2MDUKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXdpbmRvd3MtMTI1MjsgZm9ybWF0PWZsb3dlZApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5n OiA3Yml0CgpNYWtlIHN1cmUgdGhhdCB5b3UgdXNlIHJlcGxpY2EgMyBnbHVzdGVyIHZvbHVtZXMg Zm9yIHN0b3JpbmcgVk0gaW1hZ2VzLiAKQXJlIHlvdSBwbGFubmluZyB0byBkZXBsb3kgYXMgYSBo eXBlcmNvbnZlcmdlZCBzZXR1cD8KRWl0aGVyIHdheSwgdHJ5IGFuZCB1c2UgdGhlIGxhdGVzdCBv dmlydCAzLjYgYW5kIGdsdXN0ZXJmcyAzLjcgKDMuNy4xMiAKdGhhdCBhZGRyZXNzZXMgYnVncyBy ZWxhdGVkIHRvIHNoYXJkaW5nIGFuZCBvLWRpcmVjdCBpcyBkdWUgdG8gYmUgCnJlbGVhc2VkIHNv b24pCgpPbiAwNi8yMS8yMDE2IDA3OjA4IEFNLCBEZXdleSBEdSB3cm90ZToKPiBJIHdhbnQgdG8g ZGVwbG95IG9WaXJ0IDMuNiB3aXRoIEdsdXN0ZXJGUyAzLjcgdG8gbXkgb25saW5lIHNlcnZlcnMu IElzIAo+IGl0IHJlY29tbWVuZGVkIGZvciBwcm9kdWN0aW9uIHVzYWdlPwo+Cj4gVGh4Lgo+Cj4K PiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwo+IFVzZXJz IG1haWxpbmcgbGlzdAo+IFVzZXJzQG92aXJ0Lm9yZwo+IGh0dHA6Ly9saXN0cy5vdmlydC5vcmcv bWFpbG1hbi9saXN0aW5mby91c2VycwoKCi0tLS0tLS0tLS0tLS0tMDMwNzAzMDQwMzAwMDMwNTAy MDYwNjA1CkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PXdpbmRvd3MtMTI1MgpDb250 ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0Cgo8aHRtbD4KICA8aGVhZD4KICAgIDxtZXRhIGNv bnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD13aW5kb3dzLTEyNTIiCiAgICAgIGh0dHAtZXF1aXY9 IkNvbnRlbnQtVHlwZSI+CiAgPC9oZWFkPgogIDxib2R5IHRleHQ9IiMwMDAwMDAiIGJnY29sb3I9 IiNGRkZGRkYiPgogICAgTWFrZSBzdXJlIHRoYXQgeW91IHVzZSByZXBsaWNhIDMgZ2x1c3RlciB2 b2x1bWVzIGZvciBzdG9yaW5nIFZNCiAgICBpbWFnZXMuIEFyZSB5b3UgcGxhbm5pbmcgdG8gZGVw bG95IGFzIGEgaHlwZXJjb252ZXJnZWQgc2V0dXA/PGJyPgogICAgRWl0aGVyIHdheSwgdHJ5IGFu ZCB1c2UgdGhlIGxhdGVzdCBvdmlydCAzLjYgYW5kIGdsdXN0ZXJmcyAzLjcKICAgICgzLjcuMTIg dGhhdCBhZGRyZXNzZXMgYnVncyByZWxhdGVkIHRvIHNoYXJkaW5nIGFuZCBvLWRpcmVjdCBpcyBk dWUKICAgIHRvIGJlIHJlbGVhc2VkIHNvb24pPGJyPgogICAgPGJyPgogICAgPGRpdiBjbGFzcz0i bW96LWNpdGUtcHJlZml4Ij5PbiAwNi8yMS8yMDE2IDA3OjA4IEFNLCBEZXdleSBEdSB3cm90ZTo8 YnI+CiAgICA8L2Rpdj4KICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQU1TaUVtNUpBeG1WSm5N aEQxLVlzalM3c2lYYmR5Y2NyOTlLQkJSQndLZWo0UDBWb3dAbWFpbC5nbWFpbC5jb20iCiAgICAg IHR5cGU9ImNpdGUiPgogICAgICA8ZGl2IGRpcj0ibHRyIj5JIHdhbnQgdG8gZGVwbG95IG9WaXJ0 IDMuNiB3aXRoIEdsdXN0ZXJGUyAzLjcgdG8gbXkKICAgICAgICBvbmxpbmUgc2VydmVycy4gSXMg aXQgcmVjb21tZW5kZWQgZm9yIHByb2R1Y3Rpb24gdXNhZ2U/oAogICAgICAgIDxkaXY+PGJyPgog ICAgICAgIDwvZGl2PgogICAgICAgIDxkaXY+VGh4LqA8YnI+CiAgICAgICAgPC9kaXY+CiAgICAg IDwvZGl2PgogICAgICA8YnI+CiAgICAgIDxmaWVsZHNldCBjbGFzcz0ibWltZUF0dGFjaG1lbnRI ZWFkZXIiPjwvZmllbGRzZXQ+CiAgICAgIDxicj4KICAgICAgPHByZSB3cmFwPSIiPl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClVzZXJzIG1haWxpbmcgbGlz dAo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJtYWlsdG86VXNlcnNA b3ZpcnQub3JnIj5Vc2Vyc0BvdmlydC5vcmc8L2E+CjxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJl ZXRleHQiIGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2Vy cyI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzPC9hPgo8L3By ZT4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxicj4KICA8L2JvZHk+CjwvaHRtbD4KCi0tLS0tLS0t LS0tLS0tMDMwNzAzMDQwMzAwMDMwNTAyMDYwNjA1LS0K --===============6102599457828049905==-- From deweyto at gmail.com Fri Jun 24 01:55:53 2016 Content-Type: multipart/mixed; boundary="===============4892962417229881845==" MIME-Version: 1.0 From: Dewey Du To: users at ovirt.org Subject: Re: [ovirt-users] Is oVirt 3.6 with GlusterFS 3.7 recommended for production usage? Date: Fri, 24 Jun 2016 13:55:51 +0800 Message-ID: In-Reply-To: 576921B4.2050607@redhat.com --===============4892962417229881845== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable I prefer deploying as a hyperconverged setup, but it is still under experim= ent, right? So I try to separate vm service and storage. I added new storage domain( Domain Type "Data", Storage Type "GlusterFS", Use Host "host-01". But then I can't add another new storage domain( Domain Type "Data", Storage Type "GlusterFS", Use Host "host-02"). The input field "path" is unwritable (gray) on the Popup New Domain Window. My question is, should we add a new storage domain for each ovirt-node? On Tue, Jun 21, 2016 at 7:15 PM, Sahina Bose wrote: > Make sure that you use replica 3 gluster volumes for storing VM images. > Are you planning to deploy as a hyperconverged setup? > Either way, try and use the latest ovirt 3.6 and glusterfs 3.7 (3.7.12 > that addresses bugs related to sharding and o-direct is due to be released > soon) > > On 06/21/2016 07:08 AM, Dewey Du wrote: > > I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online servers. Is it > recommended for production usage? > > Thx. > > > _______________________________________________ > Users mailing listUsers(a)ovirt.orghttp://lists.ovirt.org/mailman/listinf= o/users > > > --===============4892962417229881845== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPkkgcHJlZmVyIGRl cGxveWluZyBhcyBhIGg8L3NwYW4+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPnlwZXJj b252ZXJnZWQgc2V0dXAsIGJ1dCBpdCBpcyBzdGlsbCB1bmRlcsKgPC9zcGFuPmV4cGVyaW1lbnQs IHJpZ2h0P8KgPGJyPjxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPjxicj48L3Nw YW4+PC9kaXY+PGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+U28gSSB0cnkgdG/C oDwvc3Bhbj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+c2VwYXJhdGXCoDwvc3Bhbj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+dm0gc2VydmljZSBhbmQgc3RvcmFnZS7CoDwv c3Bhbj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+SSBhZGRlZCBuZXfCoDwvc3Bhbj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+c3RvcmFnZSBkb21haW4oIERvbWFpbiBUeXBl ICZxdW90O0RhdGEmcXVvdDssIFN0b3JhZ2UgVHlwZSAmcXVvdDtHbHVzdGVyRlMmcXVvdDssIFVz ZSBIb3N0ICZxdW90O2hvc3QtMDEmcXVvdDsuIEJ1dCB0aGVuPC9zcGFuPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTIuOHB4Ij7CoEkgY2FuJiMzOTt0IGFkZCBhbm90aGVyIG5ldyBzdG9yYWdlIGRv bWFpbiggRG9tYWluIFR5cGUgJnF1b3Q7RGF0YSZxdW90OywgU3RvcmFnZSBUeXBlICZxdW90O0ds dXN0ZXJGUyZxdW90OywgVXNlIEhvc3QgJnF1b3Q7aG9zdC0wMiZxdW90OykuIFRoZSBpbnB1dCBm aWVsZCAmcXVvdDtwYXRoJnF1b3Q7IGlzIHVud3JpdGFibGUgKGdyYXkpIG9uIHRoZSBQb3B1cCBO ZXcgRG9tYWluIFdpbmRvdy48L3NwYW4+PC9kaXY+PGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXpl OjEyLjhweCI+PGJyPjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIu OHB4Ij5NeSBxdWVzdGlvbiBpcywgc2hvdWxkIHdlIGFkZCBhIG5ldyBzdG9yYWdlIGRvbWFpbiBm b3IgZWFjaCBvdmlydC1ub2RlPzwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNp emU6MTIuOHB4Ij48YnI+PC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox Mi44cHgiPjxicj48L3NwYW4+PC9kaXY+PC9kaXY+PGRpdiBjbGFzcz0iZ21haWxfZXh0cmEiPjxi cj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gVHVlLCBKdW4gMjEsIDIwMTYgYXQgNzoxNSBQ TSwgU2FoaW5hIEJvc2UgPHNwYW4gZGlyPSJsdHIiPiZsdDs8YSBocmVmPSJtYWlsdG86c2Fib3Nl QHJlZGhhdC5jb20iIHRhcmdldD0iX2JsYW5rIj5zYWJvc2VAcmVkaGF0LmNvbTwvYT4mZ3Q7PC9z cGFuPiB3cm90ZTo8YnI+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFy Z2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFl eCI+CiAgCiAgICAKICAKICA8ZGl2IHRleHQ9IiMwMDAwMDAiIGJnY29sb3I9IiNGRkZGRkYiPgog ICAgTWFrZSBzdXJlIHRoYXQgeW91IHVzZSByZXBsaWNhIDMgZ2x1c3RlciB2b2x1bWVzIGZvciBz dG9yaW5nIFZNCiAgICBpbWFnZXMuIEFyZSB5b3UgcGxhbm5pbmcgdG8gZGVwbG95IGFzIGEgaHlw ZXJjb252ZXJnZWQgc2V0dXA/PGJyPgogICAgRWl0aGVyIHdheSwgdHJ5IGFuZCB1c2UgdGhlIGxh dGVzdCBvdmlydCAzLjYgYW5kIGdsdXN0ZXJmcyAzLjcKICAgICgzLjcuMTIgdGhhdCBhZGRyZXNz ZXMgYnVncyByZWxhdGVkIHRvIHNoYXJkaW5nIGFuZCBvLWRpcmVjdCBpcyBkdWUKICAgIHRvIGJl IHJlbGVhc2VkIHNvb24pPHNwYW4gY2xhc3M9IiI+PGJyPgogICAgPGJyPgogICAgPGRpdj5PbiAw Ni8yMS8yMDE2IDA3OjA4IEFNLCBEZXdleSBEdSB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAgIDwv c3Bhbj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3BhbiBjbGFzcz0iIj4KICAgICAgPGRpdiBk aXI9Imx0ciI+SSB3YW50IHRvIGRlcGxveSBvVmlydCAzLjYgd2l0aCBHbHVzdGVyRlMgMy43IHRv IG15CiAgICAgICAgb25saW5lIHNlcnZlcnMuIElzIGl0IHJlY29tbWVuZGVkIGZvciBwcm9kdWN0 aW9uIHVzYWdlP8KgCiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRp dj5UaHguwqA8YnI+CiAgICAgICAgPC9kaXY+CiAgICAgIDwvZGl2PgogICAgICA8YnI+CiAgICAg IDxmaWVsZHNldD48L2ZpZWxkc2V0PgogICAgICA8YnI+CiAgICAgIDwvc3Bhbj48cHJlPl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClVzZXJzIG1haWxpbmcg bGlzdAo8YSBocmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIiB0YXJnZXQ9Il9ibGFuayI+VXNl cnNAb3ZpcnQub3JnPC9hPgo8YSBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4v bGlzdGluZm8vdXNlcnMiIHRhcmdldD0iX2JsYW5rIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21h aWxtYW4vbGlzdGluZm8vdXNlcnM8L2E+CjwvcHJlPgogICAgPC9ibG9ja3F1b3RlPgogICAgPGJy PgogIDwvZGl2PgoKPC9ibG9ja3F1b3RlPjwvZGl2Pjxicj48L2Rpdj4K --===============4892962417229881845==-- From sabose at redhat.com Fri Jun 24 03:16:05 2016 Content-Type: multipart/mixed; boundary="===============4269130387058928876==" MIME-Version: 1.0 From: Sahina Bose To: users at ovirt.org Subject: Re: [ovirt-users] Is oVirt 3.6 with GlusterFS 3.7 recommended for production usage? Date: Fri, 24 Jun 2016 12:45:55 +0530 Message-ID: <576CDE2B.8080607@redhat.com> In-Reply-To: CAMSiEm6JROcd72wHyjtcYo-9k_3ri1jHNADOcT0sgnp3hWTnwQ@mail.gmail.com --===============4269130387058928876== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------050908070101090104000706 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 7bit On 06/24/2016 11:25 AM, Dewey Du wrote: > I prefer deploying as a hyperconverged setup, but it is still under = > experiment, right? Hyperconverged deployment with oVirt and Gluster has been tested and is = currently offered as a preview feature with guidance on do's/don'ts & = recommended gluster volume settings. We're working on enhancing this = further to make it easier to setup, integrate better in the oVirt UI in 4.1 > > So I try to separate vm service and storage. I added new storage = > domain( Domain Type "Data", Storage Type "GlusterFS", Use Host = > "host-01". But then I can't add another new storage domain( Domain = > Type "Data", Storage Type "GlusterFS", Use Host "host-02"). The input = > field "path" is unwritable (gray) on the Popup New Domain Window. > > My question is, should we add a new storage domain for each ovirt-node? No, you dont need to. This seems like a bug in the Create storage domain UI. Does refreshing = your browser fix the greyed-out input field? Any errors seen in engine logs? > > > > On Tue, Jun 21, 2016 at 7:15 PM, Sahina Bose > wrote: > > Make sure that you use replica 3 gluster volumes for storing VM > images. Are you planning to deploy as a hyperconverged setup? > Either way, try and use the latest ovirt 3.6 and glusterfs 3.7 > (3.7.12 that addresses bugs related to sharding and o-direct is > due to be released soon) > > On 06/21/2016 07:08 AM, Dewey Du wrote: >> I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online >> servers. Is it recommended for production usage? >> >> Thx. >> >> >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users > > --------------050908070101090104000706 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit

On 06/24/2016 11:25 AM, Dewey Du wrote:<= br>

I prefer deploying as a hyperconverged setup, but it is still under=C2=A0experiment, right?

Hyperconverged deployment with oVirt and Gluster has been tested and is currently offered as a preview feature with guidance on do's/don'ts & recommended gluster volume settings. We're working on enhancing this further to make it easier to setup, integrate better in the oVirt UI in 4.1

So I try to=C2=A0separate=C2=A0vm service and storage.=C2=A0= I added new=C2=A0storage domain( Domain Type "Data", Storage Type "GlusterFS", Use Host "host-01". But then=C2=A0I can't add another new storage domain( Domain Type "Data", Storage Type "GlusterFS", Use Host "host-02"). The input field "path" is unwritable (gray) on the Popup New Domain Window.

My question is, should we add a new storage domain for each ovirt-node?

No, you dont need to.
This seems like a bug in the Create storage domain UI. Does refreshing your browser fix the greyed-out input field? Any errors seen in engine logs?

On Tue, Jun 21, 2016 at 7:15 PM, Sahina Bose <sabose(= a)redhat.com> wrote:
Make sure that you use replica 3 gluster volumes for storing VM images. Are you planning to deploy as a hyperconverged setup?
Either way, try and use the latest ovirt 3.6 and glusterfs 3.7 (3.7.12 that addresses bugs related to sharding and o-direct is due to be released soon)

On 06/21/2016 07:08 AM, Dewey Du wrote:
I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online servers. Is it recommended for production usage?=C2=A0

Thx.=C2=A0
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

--------------050908070101090104000706-- --===============4269130387058928876== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wNTA5MDgwNzAxMDEwOTAxMDQwMDA3MDYKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQK CgoKT24gMDYvMjQvMjAxNiAxMToyNSBBTSwgRGV3ZXkgRHUgd3JvdGU6Cj4gSSBwcmVmZXIgZGVw bG95aW5nIGFzIGEgaHlwZXJjb252ZXJnZWQgc2V0dXAsIGJ1dCBpdCBpcyBzdGlsbCB1bmRlciAK PiBleHBlcmltZW50LCByaWdodD8KCkh5cGVyY29udmVyZ2VkIGRlcGxveW1lbnQgd2l0aCBvVmly dCBhbmQgR2x1c3RlciBoYXMgYmVlbiB0ZXN0ZWQgYW5kIGlzIApjdXJyZW50bHkgb2ZmZXJlZCBh cyBhIHByZXZpZXcgZmVhdHVyZSB3aXRoIGd1aWRhbmNlIG9uIGRvJ3MvZG9uJ3RzICYgCnJlY29t bWVuZGVkIGdsdXN0ZXIgdm9sdW1lIHNldHRpbmdzLiBXZSdyZSB3b3JraW5nIG9uIGVuaGFuY2lu ZyB0aGlzIApmdXJ0aGVyIHRvIG1ha2UgaXQgZWFzaWVyIHRvIHNldHVwLCBpbnRlZ3JhdGUgYmV0 dGVyIGluIHRoZSBvVmlydCBVSSBpbiA0LjEKCj4KPiBTbyBJIHRyeSB0byBzZXBhcmF0ZSB2bSBz ZXJ2aWNlIGFuZCBzdG9yYWdlLiBJIGFkZGVkIG5ldyBzdG9yYWdlIAo+IGRvbWFpbiggRG9tYWlu IFR5cGUgIkRhdGEiLCBTdG9yYWdlIFR5cGUgIkdsdXN0ZXJGUyIsIFVzZSBIb3N0IAo+ICJob3N0 LTAxIi4gQnV0IHRoZW4gSSBjYW4ndCBhZGQgYW5vdGhlciBuZXcgc3RvcmFnZSBkb21haW4oIERv bWFpbiAKPiBUeXBlICJEYXRhIiwgU3RvcmFnZSBUeXBlICJHbHVzdGVyRlMiLCBVc2UgSG9zdCAi aG9zdC0wMiIpLiBUaGUgaW5wdXQgCj4gZmllbGQgInBhdGgiIGlzIHVud3JpdGFibGUgKGdyYXkp IG9uIHRoZSBQb3B1cCBOZXcgRG9tYWluIFdpbmRvdy4KPgo+IE15IHF1ZXN0aW9uIGlzLCBzaG91 bGQgd2UgYWRkIGEgbmV3IHN0b3JhZ2UgZG9tYWluIGZvciBlYWNoIG92aXJ0LW5vZGU/CgpObywg eW91IGRvbnQgbmVlZCB0by4KVGhpcyBzZWVtcyBsaWtlIGEgYnVnIGluIHRoZSBDcmVhdGUgc3Rv cmFnZSBkb21haW4gVUkuIERvZXMgcmVmcmVzaGluZyAKeW91ciBicm93c2VyIGZpeCB0aGUgZ3Jl eWVkLW91dCBpbnB1dCBmaWVsZD8gQW55IGVycm9ycyBzZWVuIGluIGVuZ2luZSBsb2dzPwoKCj4K Pgo+Cj4gT24gVHVlLCBKdW4gMjEsIDIwMTYgYXQgNzoxNSBQTSwgU2FoaW5hIEJvc2UgPHNhYm9z ZUByZWRoYXQuY29tIAo+IDxtYWlsdG86c2Fib3NlQHJlZGhhdC5jb20+PiB3cm90ZToKPgo+ICAg ICBNYWtlIHN1cmUgdGhhdCB5b3UgdXNlIHJlcGxpY2EgMyBnbHVzdGVyIHZvbHVtZXMgZm9yIHN0 b3JpbmcgVk0KPiAgICAgaW1hZ2VzLiBBcmUgeW91IHBsYW5uaW5nIHRvIGRlcGxveSBhcyBhIGh5 cGVyY29udmVyZ2VkIHNldHVwPwo+ICAgICBFaXRoZXIgd2F5LCB0cnkgYW5kIHVzZSB0aGUgbGF0 ZXN0IG92aXJ0IDMuNiBhbmQgZ2x1c3RlcmZzIDMuNwo+ICAgICAoMy43LjEyIHRoYXQgYWRkcmVz c2VzIGJ1Z3MgcmVsYXRlZCB0byBzaGFyZGluZyBhbmQgby1kaXJlY3QgaXMKPiAgICAgZHVlIHRv IGJlIHJlbGVhc2VkIHNvb24pCj4KPiAgICAgT24gMDYvMjEvMjAxNiAwNzowOCBBTSwgRGV3ZXkg RHUgd3JvdGU6Cj4+ICAgICBJIHdhbnQgdG8gZGVwbG95IG9WaXJ0IDMuNiB3aXRoIEdsdXN0ZXJG UyAzLjcgdG8gbXkgb25saW5lCj4+ICAgICBzZXJ2ZXJzLiBJcyBpdCByZWNvbW1lbmRlZCBmb3Ig cHJvZHVjdGlvbiB1c2FnZT8KPj4KPj4gICAgIFRoeC4KPj4KPj4KPj4gICAgIF9fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCj4+ICAgICBVc2VycyBtYWlsaW5n IGxpc3QKPj4gICAgIFVzZXJzQG92aXJ0Lm9yZyA8bWFpbHRvOlVzZXJzQG92aXJ0Lm9yZz4KPj4g ICAgIGh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2Vycwo+Cj4KCgot LS0tLS0tLS0tLS0tLTA1MDkwODA3MDEwMTA5MDEwNDAwMDcwNgpDb250ZW50LVR5cGU6IHRleHQv aHRtbDsgY2hhcnNldD11dGYtOApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0Cgo8aHRt bD4KICA8aGVhZD4KICAgIDxtZXRhIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCIg aHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICA8L2hlYWQ+CiAgPGJvZHkgdGV4dD0iIzAwMDAw MCIgYmdjb2xvcj0iI0ZGRkZGRiI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8ZGl2IGNsYXNzPSJt b3otY2l0ZS1wcmVmaXgiPk9uIDA2LzI0LzIwMTYgMTE6MjUgQU0sIERld2V5IER1IHdyb3RlOjxi cj4KICAgIDwvZGl2PgogICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVNpRW02SlJPY2Q3MndI eWp0Y1lvLTlrXzNyaTFqSE5BRE9jVDBzZ25wM2hXVG53UUBtYWlsLmdtYWlsLmNvbSIKICAgICAg dHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYgZGlyPSJsdHIiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6 MTIuOHB4Ij5JIHByZWZlciBkZXBsb3lpbmcKICAgICAgICAgIGFzIGEgaDwvc3Bhbj48c3BhbiBz dHlsZT0iZm9udC1zaXplOjEyLjhweCI+eXBlcmNvbnZlcmdlZAogICAgICAgICAgc2V0dXAsIGJ1 dCBpdCBpcyBzdGlsbCB1bmRlcsKgPC9zcGFuPmV4cGVyaW1lbnQsIHJpZ2h0PyA8YnI+CiAgICAg IDwvZGl2PgogICAgPC9ibG9ja3F1b3RlPgogICAgPGJyPgogICAgSHlwZXJjb252ZXJnZWQgZGVw bG95bWVudCB3aXRoIG9WaXJ0IGFuZCBHbHVzdGVyIGhhcyBiZWVuIHRlc3RlZCBhbmQKICAgIGlz IGN1cnJlbnRseSBvZmZlcmVkIGFzIGEgcHJldmlldyBmZWF0dXJlIHdpdGggZ3VpZGFuY2Ugb24K ICAgIGRvJ3MvZG9uJ3RzICZhbXA7IHJlY29tbWVuZGVkIGdsdXN0ZXIgdm9sdW1lIHNldHRpbmdz LiBXZSdyZSB3b3JraW5nCiAgICBvbiBlbmhhbmNpbmcgdGhpcyBmdXJ0aGVyIHRvIG1ha2UgaXQg ZWFzaWVyIHRvIHNldHVwLCBpbnRlZ3JhdGUKICAgIGJldHRlciBpbiB0aGUgb1ZpcnQgVUkgaW4g NC4xPGJyPgogICAgPGJyPgogICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVNpRW02SlJPY2Q3 MndIeWp0Y1lvLTlrXzNyaTFqSE5BRE9jVDBzZ25wM2hXVG53UUBtYWlsLmdtYWlsLmNvbSIKICAg ICAgdHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYgZGlyPSJsdHIiPgogICAgICAgIDxkaXY+PHNwYW4g c3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPjxicj4KICAgICAgICAgIDwvc3Bhbj48L2Rpdj4KICAg ICAgICA8ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij5TbyBJIHRyeSB0b8KgPC9z cGFuPjxzcGFuCiAgICAgICAgICAgIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij5zZXBhcmF0ZcKg PC9zcGFuPjxzcGFuCiAgICAgICAgICAgIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij52bSBzZXJ2 aWNlIGFuZCBzdG9yYWdlLsKgPC9zcGFuPjxzcGFuCiAgICAgICAgICAgIHN0eWxlPSJmb250LXNp emU6MTIuOHB4Ij5JIGFkZGVkIG5ld8KgPC9zcGFuPjxzcGFuCiAgICAgICAgICAgIHN0eWxlPSJm b250LXNpemU6MTIuOHB4Ij5zdG9yYWdlIGRvbWFpbiggRG9tYWluIFR5cGUgIkRhdGEiLAogICAg ICAgICAgICBTdG9yYWdlIFR5cGUgIkdsdXN0ZXJGUyIsIFVzZSBIb3N0ICJob3N0LTAxIi4gQnV0 IHRoZW48L3NwYW4+PHNwYW4KICAgICAgICAgICAgc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPsKg SSBjYW4ndCBhZGQgYW5vdGhlciBuZXcgc3RvcmFnZQogICAgICAgICAgICBkb21haW4oIERvbWFp biBUeXBlICJEYXRhIiwgU3RvcmFnZSBUeXBlICJHbHVzdGVyRlMiLCBVc2UKICAgICAgICAgICAg SG9zdCAiaG9zdC0wMiIpLiBUaGUgaW5wdXQgZmllbGQgInBhdGgiIGlzIHVud3JpdGFibGUgKGdy YXkpCiAgICAgICAgICAgIG9uIHRoZSBQb3B1cCBOZXcgRG9tYWluIFdpbmRvdy48L3NwYW4+PC9k aXY+CiAgICAgICAgPGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+PGJyPgogICAg ICAgICAgPC9zcGFuPjwvZGl2PgogICAgICAgIDxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox Mi44cHgiPk15IHF1ZXN0aW9uIGlzLCBzaG91bGQgd2UKICAgICAgICAgICAgYWRkIGEgbmV3IHN0 b3JhZ2UgZG9tYWluIGZvciBlYWNoIG92aXJ0LW5vZGU/PC9zcGFuPjwvZGl2PgogICAgICA8L2Rp dj4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxicj4KICAgIE5vLCB5b3UgZG9udCBuZWVkIHRvLjxi cj4KICAgIFRoaXMgc2VlbXMgbGlrZSBhIGJ1ZyBpbiB0aGUgQ3JlYXRlIHN0b3JhZ2UgZG9tYWlu IFVJLiBEb2VzCiAgICByZWZyZXNoaW5nIHlvdXIgYnJvd3NlciBmaXggdGhlIGdyZXllZC1vdXQg aW5wdXQgZmllbGQ/IEFueSBlcnJvcnMKICAgIHNlZW4gaW4gZW5naW5lIGxvZ3M/PGJyPgogICAg PGJyPgogICAgPGJyPgogICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVNpRW02SlJPY2Q3MndI eWp0Y1lvLTlrXzNyaTFqSE5BRE9jVDBzZ25wM2hXVG53UUBtYWlsLmdtYWlsLmNvbSIKICAgICAg dHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYgZGlyPSJsdHIiPgogICAgICAgIDxkaXY+PHNwYW4gc3R5 bGU9ImZvbnQtc2l6ZToxMi44cHgiPjxicj4KICAgICAgICAgIDwvc3Bhbj48L2Rpdj4KICAgICAg ICA8ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij48YnI+CiAgICAgICAgICA8L3Nw YW4+PC9kaXY+CiAgICAgIDwvZGl2PgogICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGJy PgogICAgICAgIDxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIj5PbiBUdWUsIEp1biAyMSwgMjAxNiBh dCA3OjE1IFBNLCBTYWhpbmEKICAgICAgICAgIEJvc2UgPHNwYW4gZGlyPSJsdHIiPiZsdDs8YSBt b3otZG8tbm90LXNlbmQ9InRydWUiCiAgICAgICAgICAgICAgaHJlZj0ibWFpbHRvOnNhYm9zZUBy ZWRoYXQuY29tIiB0YXJnZXQ9Il9ibGFuayI+c2Fib3NlQHJlZGhhdC5jb208L2E+Jmd0Ozwvc3Bh bj4KICAgICAgICAgIHdyb3RlOjxicj4KICAgICAgICAgIDxibG9ja3F1b3RlIGNsYXNzPSJnbWFp bF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAgMAogICAgICAgICAgICAuOGV4O2JvcmRlci1sZWZ0 OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPgogICAgICAgICAgICA8ZGl2IHRleHQ9 IiMwMDAwMDAiIGJnY29sb3I9IiNGRkZGRkYiPiBNYWtlIHN1cmUgdGhhdCB5b3UKICAgICAgICAg ICAgICB1c2UgcmVwbGljYSAzIGdsdXN0ZXIgdm9sdW1lcyBmb3Igc3RvcmluZyBWTSBpbWFnZXMu IEFyZQogICAgICAgICAgICAgIHlvdSBwbGFubmluZyB0byBkZXBsb3kgYXMgYSBoeXBlcmNvbnZl cmdlZCBzZXR1cD88YnI+CiAgICAgICAgICAgICAgRWl0aGVyIHdheSwgdHJ5IGFuZCB1c2UgdGhl IGxhdGVzdCBvdmlydCAzLjYgYW5kIGdsdXN0ZXJmcwogICAgICAgICAgICAgIDMuNyAoMy43LjEy IHRoYXQgYWRkcmVzc2VzIGJ1Z3MgcmVsYXRlZCB0byBzaGFyZGluZyBhbmQKICAgICAgICAgICAg ICBvLWRpcmVjdCBpcyBkdWUgdG8gYmUgcmVsZWFzZWQgc29vbik8c3BhbiBjbGFzcz0iIj48YnI+ CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICA8ZGl2Pk9uIDA2LzIxLzIwMTYg MDc6MDggQU0sIERld2V5IER1IHdyb3RlOjxicj4KICAgICAgICAgICAgICAgIDwvZGl2PgogICAg ICAgICAgICAgIDwvc3Bhbj4KICAgICAgICAgICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48 c3BhbiBjbGFzcz0iIj4KICAgICAgICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+SSB3YW50IHRv IGRlcGxveSBvVmlydCAzLjYgd2l0aAogICAgICAgICAgICAgICAgICAgIEdsdXN0ZXJGUyAzLjcg dG8gbXkgb25saW5lIHNlcnZlcnMuIElzIGl0CiAgICAgICAgICAgICAgICAgICAgcmVjb21tZW5k ZWQgZm9yIHByb2R1Y3Rpb24gdXNhZ2U/wqAKICAgICAgICAgICAgICAgICAgICA8ZGl2Pjxicj4K ICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICA8ZGl2PlRoeC7C oDxicj4KICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgPC9kaXY+ CiAgICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgICAgPGZpZWxkc2V0PjwvZmll bGRzZXQ+CiAgICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgIDwvc3Bhbj4KICAg ICAgICAgICAgICAgIDxwcmU+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KVXNlcnMgbWFpbGluZyBsaXN0CjxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgaHJl Zj0ibWFpbHRvOlVzZXJzQG92aXJ0Lm9yZyIgdGFyZ2V0PSJfYmxhbmsiPlVzZXJzQG92aXJ0Lm9y ZzwvYT4KPGEgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIiBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQu b3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMiIHRhcmdldD0iX2JsYW5rIj5odHRwOi8vbGlzdHMu b3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnM8L2E+CjwvcHJlPgogICAgICAgICAgICAg IDwvYmxvY2txdW90ZT4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAg ICAgICAgPC9ibG9ja3F1b3RlPgogICAgICAgIDwvZGl2PgogICAgICAgIDxicj4KICAgICAgPC9k aXY+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+CiAgPC9ib2R5Pgo8L2h0bWw+CgotLS0tLS0t LS0tLS0tLTA1MDkwODA3MDEwMTA5MDEwNDAwMDcwNi0tCg== --===============4269130387058928876==-- From deweyto at gmail.com Fri Jun 24 05:46:41 2016 Content-Type: multipart/mixed; boundary="===============7739109306397786916==" MIME-Version: 1.0 From: Dewey Du To: users at ovirt.org Subject: Re: [ovirt-users] Is oVirt 3.6 with GlusterFS 3.7 recommended for production usage? Date: Fri, 24 Jun 2016 17:46:39 +0800 Message-ID: In-Reply-To: 576CDE2B.8080607@redhat.com --===============7739109306397786916== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable You mean we dont need to add new storage domain for each ovirt-node? Maybe I choose the "Use Host" option wrongly? I thought this is the way to tie the storage of "Host-01" to GlusterFS by selecting "Host-01" on the "Use Host" option. And I should do the same thing for "Host-02", which adding new storage domain with choosing "Host-02" on the "Use Host" option. So do others. Host-03, Host-04 ... On Fri, Jun 24, 2016 at 3:15 PM, Sahina Bose wrote: > > > On 06/24/2016 11:25 AM, Dewey Du wrote: > > I prefer deploying as a hyperconverged setup, but it is still under exper= iment, > right? > > > Hyperconverged deployment with oVirt and Gluster has been tested and is > currently offered as a preview feature with guidance on do's/don'ts & > recommended gluster volume settings. We're working on enhancing this > further to make it easier to setup, integrate better in the oVirt UI in 4= .1 > > > So I try to separate vm service and storage. I added new storage domain( > Domain Type "Data", Storage Type "GlusterFS", Use Host "host-01". But the= n I > can't add another new storage domain( Domain Type "Data", Storage Type > "GlusterFS", Use Host "host-02"). The input field "path" is unwritable > (gray) on the Popup New Domain Window. > > My question is, should we add a new storage domain for each ovirt-node? > > > No, you dont need to. > This seems like a bug in the Create storage domain UI. Does refreshing > your browser fix the greyed-out input field? Any errors seen in engine lo= gs? > > > > > > On Tue, Jun 21, 2016 at 7:15 PM, Sahina Bose wrote: > >> Make sure that you use replica 3 gluster volumes for storing VM images. >> Are you planning to deploy as a hyperconverged setup? >> Either way, try and use the latest ovirt 3.6 and glusterfs 3.7 (3.7.12 >> that addresses bugs related to sharding and o-direct is due to be releas= ed >> soon) >> >> On 06/21/2016 07:08 AM, Dewey Du wrote: >> >> I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online servers. Is it >> recommended for production usage? >> >> Thx. >> >> >> _______________________________________________ >> Users mailing listUsers(a)ovirt.orghttp://lists.ovirt.org/mailman/listin= fo/users >> >> >> > > --===============7739109306397786916== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+WW91IG1lYW4gd2UgZG9udCBuZWVkIHRvIGFkZCBuZXcgc3RvcmFnZSBk b21haW4gZm9yIGVhY2ggb3ZpcnQtbm9kZT/CoDxkaXY+TWF5YmUgSSBjaG9vc2UgdGhlICZxdW90 O1VzZSBIb3N0JnF1b3Q7IG9wdGlvbiB3cm9uZ2x5PyBJIHRob3VnaHQgdGhpcyBpcyB0aGUgd2F5 IHRvIHRpZSB0aGUgc3RvcmFnZSBvZiAmcXVvdDtIb3N0LTAxJnF1b3Q7IHRvIEdsdXN0ZXJGUyBi eSBzZWxlY3RpbmcgJnF1b3Q7SG9zdC0wMSZxdW90OyBvbiB0aGUgJnF1b3Q7VXNlIEhvc3QmcXVv dDsgb3B0aW9uLsKgPC9kaXY+PGRpdj5BbmQgSSBzaG91bGQgZG8gdGhlIHNhbWUgdGhpbmcgZm9y ICZxdW90O0hvc3QtMDImcXVvdDssIHdoaWNoIGFkZGluZyBuZXcgc3RvcmFnZSBkb21haW4gd2l0 aCBjaG9vc2luZyAmcXVvdDtIb3N0LTAyJnF1b3Q7IG9uIHRoZSAmcXVvdDtVc2UgSG9zdCZxdW90 OyBvcHRpb24uwqA8L2Rpdj48ZGl2PlNvIGRvIG90aGVycy4gSG9zdC0wMywgSG9zdC0wNCAuLi48 L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48L2Rpdj48ZGl2IGNsYXNzPSJnbWFp bF9leHRyYSI+PGJyPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIj5PbiBGcmksIEp1biAyNCwgMjAx NiBhdCAzOjE1IFBNLCBTYWhpbmEgQm9zZSA8c3BhbiBkaXI9Imx0ciI+Jmx0OzxhIGhyZWY9Im1h aWx0bzpzYWJvc2VAcmVkaGF0LmNvbSIgdGFyZ2V0PSJfYmxhbmsiPnNhYm9zZUByZWRoYXQuY29t PC9hPiZndDs8L3NwYW4+IHdyb3RlOjxicj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUi IHN0eWxlPSJtYXJnaW46MCAwIDAgLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRk aW5nLWxlZnQ6MWV4Ij4KICAKICAgIAogIAogIDxkaXYgdGV4dD0iIzAwMDAwMCIgYmdjb2xvcj0i I0ZGRkZGRiI+PHNwYW4gY2xhc3M9IiI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8ZGl2Pk9uIDA2 LzI0LzIwMTYgMTE6MjUgQU0sIERld2V5IER1IHdyb3RlOjxicj4KICAgIDwvZGl2PgogICAgPGJs b2NrcXVvdGUgdHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYgZGlyPSJsdHIiPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTIuOHB4Ij5JIHByZWZlciBkZXBsb3lpbmcKICAgICAgICAgIGFzIGEgaDwvc3Bh bj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+eXBlcmNvbnZlcmdlZAogICAgICAgICAg c2V0dXAsIGJ1dCBpdCBpcyBzdGlsbCB1bmRlcsKgPC9zcGFuPmV4cGVyaW1lbnQsIHJpZ2h0PyA8 YnI+CiAgICAgIDwvZGl2PgogICAgPC9ibG9ja3F1b3RlPgogICAgPGJyPjwvc3Bhbj4KICAgIEh5 cGVyY29udmVyZ2VkIGRlcGxveW1lbnQgd2l0aCBvVmlydCBhbmQgR2x1c3RlciBoYXMgYmVlbiB0 ZXN0ZWQgYW5kCiAgICBpcyBjdXJyZW50bHkgb2ZmZXJlZCBhcyBhIHByZXZpZXcgZmVhdHVyZSB3 aXRoIGd1aWRhbmNlIG9uCiAgICBkbyYjMzk7cy9kb24mIzM5O3RzICZhbXA7IHJlY29tbWVuZGVk IGdsdXN0ZXIgdm9sdW1lIHNldHRpbmdzLiBXZSYjMzk7cmUgd29ya2luZwogICAgb24gZW5oYW5j aW5nIHRoaXMgZnVydGhlciB0byBtYWtlIGl0IGVhc2llciB0byBzZXR1cCwgaW50ZWdyYXRlCiAg ICBiZXR0ZXIgaW4gdGhlIG9WaXJ0IFVJIGluIDQuMTxzcGFuIGNsYXNzPSIiPjxicj4KICAgIDxi cj4KICAgIDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPgogICAgICA8ZGl2IGRpcj0ibHRyIj4KICAg ICAgICA8ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij48YnI+CiAgICAgICAgICA8 L3NwYW4+PC9kaXY+CiAgICAgICAgPGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+ U28gSSB0cnkgdG/CoDwvc3Bhbj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+c2VwYXJh dGXCoDwvc3Bhbj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+dm0gc2VydmljZSBhbmQg c3RvcmFnZS7CoDwvc3Bhbj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+SSBhZGRlZCBu ZXfCoDwvc3Bhbj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+c3RvcmFnZSBkb21haW4o IERvbWFpbiBUeXBlICZxdW90O0RhdGEmcXVvdDssCiAgICAgICAgICAgIFN0b3JhZ2UgVHlwZSAm cXVvdDtHbHVzdGVyRlMmcXVvdDssIFVzZSBIb3N0ICZxdW90O2hvc3QtMDEmcXVvdDsuIEJ1dCB0 aGVuPC9zcGFuPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij7CoEkgY2FuJiMzOTt0IGFk ZCBhbm90aGVyIG5ldyBzdG9yYWdlCiAgICAgICAgICAgIGRvbWFpbiggRG9tYWluIFR5cGUgJnF1 b3Q7RGF0YSZxdW90OywgU3RvcmFnZSBUeXBlICZxdW90O0dsdXN0ZXJGUyZxdW90OywgVXNlCiAg ICAgICAgICAgIEhvc3QgJnF1b3Q7aG9zdC0wMiZxdW90OykuIFRoZSBpbnB1dCBmaWVsZCAmcXVv dDtwYXRoJnF1b3Q7IGlzIHVud3JpdGFibGUgKGdyYXkpCiAgICAgICAgICAgIG9uIHRoZSBQb3B1 cCBOZXcgRG9tYWluIFdpbmRvdy48L3NwYW4+PC9kaXY+CiAgICAgICAgPGRpdj48c3BhbiBzdHls ZT0iZm9udC1zaXplOjEyLjhweCI+PGJyPgogICAgICAgICAgPC9zcGFuPjwvZGl2PgogICAgICAg IDxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPk15IHF1ZXN0aW9uIGlzLCBzaG91 bGQgd2UKICAgICAgICAgICAgYWRkIGEgbmV3IHN0b3JhZ2UgZG9tYWluIGZvciBlYWNoIG92aXJ0 LW5vZGU/PC9zcGFuPjwvZGl2PgogICAgICA8L2Rpdj4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxi cj48L3NwYW4+CiAgICBObywgeW91IGRvbnQgbmVlZCB0by48YnI+CiAgICBUaGlzIHNlZW1zIGxp a2UgYSBidWcgaW4gdGhlIENyZWF0ZSBzdG9yYWdlIGRvbWFpbiBVSS4gRG9lcwogICAgcmVmcmVz aGluZyB5b3VyIGJyb3dzZXIgZml4IHRoZSBncmV5ZWQtb3V0IGlucHV0IGZpZWxkPyBBbnkgZXJy b3JzCiAgICBzZWVuIGluIGVuZ2luZSBsb2dzPzxzcGFuIGNsYXNzPSIiPjxicj4KICAgIDxicj4K ICAgIDxicj4KICAgIDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPgogICAgICA8ZGl2IGRpcj0ibHRy Ij4KICAgICAgICA8ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij48YnI+CiAgICAg ICAgICA8L3NwYW4+PC9kaXY+CiAgICAgICAgPGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEy LjhweCI+PGJyPgogICAgICAgICAgPC9zcGFuPjwvZGl2PgogICAgICA8L2Rpdj4KICAgICAgPGRp diBjbGFzcz0iZ21haWxfZXh0cmEiPjxicj4KICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9xdW90 ZSI+T24gVHVlLCBKdW4gMjEsIDIwMTYgYXQgNzoxNSBQTSwgU2FoaW5hCiAgICAgICAgICBCb3Nl IDxzcGFuIGRpcj0ibHRyIj4mbHQ7PGEgaHJlZj0ibWFpbHRvOnNhYm9zZUByZWRoYXQuY29tIiB0 YXJnZXQ9Il9ibGFuayI+c2Fib3NlQHJlZGhhdC5jb208L2E+Jmd0Ozwvc3Bhbj4KICAgICAgICAg IHdyb3RlOjxicj4KICAgICAgICAgIDxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5 bGU9Im1hcmdpbjowIDAgMCAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRpbmct bGVmdDoxZXgiPgogICAgICAgICAgICA8ZGl2IHRleHQ9IiMwMDAwMDAiIGJnY29sb3I9IiNGRkZG RkYiPiBNYWtlIHN1cmUgdGhhdCB5b3UKICAgICAgICAgICAgICB1c2UgcmVwbGljYSAzIGdsdXN0 ZXIgdm9sdW1lcyBmb3Igc3RvcmluZyBWTSBpbWFnZXMuIEFyZQogICAgICAgICAgICAgIHlvdSBw bGFubmluZyB0byBkZXBsb3kgYXMgYSBoeXBlcmNvbnZlcmdlZCBzZXR1cD88YnI+CiAgICAgICAg ICAgICAgRWl0aGVyIHdheSwgdHJ5IGFuZCB1c2UgdGhlIGxhdGVzdCBvdmlydCAzLjYgYW5kIGds dXN0ZXJmcwogICAgICAgICAgICAgIDMuNyAoMy43LjEyIHRoYXQgYWRkcmVzc2VzIGJ1Z3MgcmVs YXRlZCB0byBzaGFyZGluZyBhbmQKICAgICAgICAgICAgICBvLWRpcmVjdCBpcyBkdWUgdG8gYmUg cmVsZWFzZWQgc29vbik8c3Bhbj48YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgICA8ZGl2Pk9uIDA2LzIxLzIwMTYgMDc6MDggQU0sIERld2V5IER1IHdyb3RlOjxicj4KICAg ICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICAgIDwvc3Bhbj4KICAgICAgICAgICAgICA8 YmxvY2txdW90ZSB0eXBlPSJjaXRlIj48c3Bhbj4KICAgICAgICAgICAgICAgICAgPGRpdiBkaXI9 Imx0ciI+SSB3YW50IHRvIGRlcGxveSBvVmlydCAzLjYgd2l0aAogICAgICAgICAgICAgICAgICAg IEdsdXN0ZXJGUyAzLjcgdG8gbXkgb25saW5lIHNlcnZlcnMuIElzIGl0CiAgICAgICAgICAgICAg ICAgICAgcmVjb21tZW5kZWQgZm9yIHByb2R1Y3Rpb24gdXNhZ2U/wqAKICAgICAgICAgICAgICAg ICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAg ICAgICA8ZGl2PlRoeC7CoDxicj4KICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAg ICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgICAg PGZpZWxkc2V0PjwvZmllbGRzZXQ+CiAgICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAg ICAgIDwvc3Bhbj4KICAgICAgICAgICAgICAgIDxwcmU+X19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KVXNlcnMgbWFpbGluZyBsaXN0CjxhIGhyZWY9Im1haWx0 bzpVc2Vyc0BvdmlydC5vcmciIHRhcmdldD0iX2JsYW5rIj5Vc2Vyc0BvdmlydC5vcmc8L2E+Cjxh IGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyIgdGFy Z2V0PSJfYmxhbmsiPmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2Vy czwvYT4KPC9wcmU+CiAgICAgICAgICAgICAgPC9ibG9ja3F1b3RlPgogICAgICAgICAgICAgIDxi cj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgPC9k aXY+CiAgICAgICAgPGJyPgogICAgICA8L2Rpdj4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxicj4K ICA8L3NwYW4+PC9kaXY+Cgo8L2Jsb2NrcXVvdGU+PC9kaXY+PGJyPjwvZGl2Pgo= --===============7739109306397786916==-- From sabose at redhat.com Fri Jun 24 06:23:25 2016 Content-Type: multipart/mixed; boundary="===============6048425413915353151==" MIME-Version: 1.0 From: Sahina Bose To: users at ovirt.org Subject: Re: [ovirt-users] Is oVirt 3.6 with GlusterFS 3.7 recommended for production usage? Date: Fri, 24 Jun 2016 15:53:20 +0530 Message-ID: <576D0A18.2070300@redhat.com> In-Reply-To: CAMSiEm6VWA7YYM0VrhwxQMa5pcuY9c4Zgz8tjkdE2OsLKH+A-g@mail.gmail.com --===============6048425413915353151== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------000209030506000905010408 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 7bit On 06/24/2016 03:16 PM, Dewey Du wrote: > You mean we dont need to add new storage domain for each ovirt-node? > Maybe I choose the "Use Host" option wrongly? I thought this is the = > way to tie the storage of "Host-01" to GlusterFS by selecting = > "Host-01" on the "Use Host" option. > And I should do the same thing for "Host-02", which adding new storage = > domain with choosing "Host-02" on the "Use Host" option. > So do others. Host-03, Host-04 ... While creating storage domain, the host provided in "Use host" is the = one used to initially setup the connection to the storage. In case of gluster storage domain, if like you said, you want to = separate storage servers from hypervisor hosts - you would set up the = gluster volume using bricks on Host-04, Host-05, Host-06. And in oVirt create a cluster with your hypervisor hosts - Host-01, = Host-02 and Host-03. To create a data storage domain - use any of the = hosts (01-03) and provide path to gluster volume as host-04:/ In case of hyperconverged setup, the gluster volume can be setup on = Host-01, Host-02, Host-03 itself. Check = http://blogs-ramesh.blogspot.in/2016/01/ovirt-and-gluster-hyperconvergence.= html > > > > On Fri, Jun 24, 2016 at 3:15 PM, Sahina Bose > wrote: > > > > On 06/24/2016 11:25 AM, Dewey Du wrote: >> I prefer deploying as a hyperconverged setup, but it is still >> under experiment, right? > > Hyperconverged deployment with oVirt and Gluster has been tested > and is currently offered as a preview feature with guidance on > do's/don'ts & recommended gluster volume settings. We're working > on enhancing this further to make it easier to setup, integrate > better in the oVirt UI in 4.1 > >> >> So I try to separate vm service and storage. I added new storage >> domain( Domain Type "Data", Storage Type "GlusterFS", Use Host >> "host-01". But then I can't add another new storage domain( >> Domain Type "Data", Storage Type "GlusterFS", Use Host >> "host-02"). The input field "path" is unwritable (gray) on the >> Popup New Domain Window. >> >> My question is, should we add a new storage domain for each >> ovirt-node? > > No, you dont need to. > This seems like a bug in the Create storage domain UI. Does > refreshing your browser fix the greyed-out input field? Any errors > seen in engine logs? > > >> >> >> >> On Tue, Jun 21, 2016 at 7:15 PM, Sahina Bose > > wrote: >> >> Make sure that you use replica 3 gluster volumes for storing >> VM images. Are you planning to deploy as a hyperconverged setup? >> Either way, try and use the latest ovirt 3.6 and glusterfs >> 3.7 (3.7.12 that addresses bugs related to sharding and >> o-direct is due to be released soon) >> >> On 06/21/2016 07:08 AM, Dewey Du wrote: >>> I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online >>> servers. Is it recommended for production usage? >>> >>> Thx. >>> >>> >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >> >> > > --------------000209030506000905010408 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit

On 06/24/2016 03:16 PM, Dewey Du wrote:<= br>

You mean we dont need to add new storage domain for each ovirt-node?=C2=A0
Maybe I choose the "Use Host" option wrongly? I thought this is the way to tie the storage of "Host-01" to GlusterFS by selecting "Host-01" on the "Use Host" option.=C2=A0

And I should do the same thing for "Host-02", which adding new storage domain with choosing "Host-02" on the "Use Host" option.=C2=A0

So do others. Host-03, Host-04 ...

While creating storage domain, the host provided in "Use host" is the one used to initially setup the connection to the storage.
In case of gluster storage domain, if like you said, you want to separate storage servers from hypervisor hosts - you would set up the gluster volume using bricks on Host-04, Host-05, Host-06.
And in oVirt create a cluster with your hypervisor hosts - Host-01, Host-02 and Host-03. To create a data storage domain - use any of the hosts (01-03) and provide path to gluster volume as host-04:/<volname>

In case of hyperconverged setup, the gluster volume can be setup on Host-01, Host-02, Host-03 itself. Check http://blogs-ramesh.blogsp= ot.in/2016/01/ovirt-and-gluster-hyperconvergence.html

On Fri, Jun 24, 2016 at 3:15 PM, Sahina Bose <sabose(= a)redhat.com> wrote:
On 06/24/2016 11:25 AM, Dewey Du wrote:

I prefer deploying as a hyperconverged setup, but it is still under=C2=A0experiment, right?

Hyperconverged deployment with oVirt and Gluster has been tested and is currently offered as a preview feature with guidance on do's/don'ts & recommended gluster volume settings. We're working on enhancing this further to make it easier to setup, integrate better in the oVirt UI in 4.1

So I try to=C2=A0= separate=C2=A0vm service and storage.=C2=A0I added new=C2=A0storage domain( Domain Type "Data", Storage Type "GlusterFS", Use Host "host-01". But then=C2=A0I can't add another new storage domain( Domain Type "Data", Storage Type "GlusterFS", Use Host "host-02"). The input field "path" is unwritable (gray) on the Popup New Domain Window.

My question is, should we add a new storage domain for each ovirt-node?

No, you dont need to.
This seems like a bug in the Create storage domain UI. Does refreshing your browser fix the greyed-out input field? Any errors seen in engine logs?

On Tue, Jun 21, 2016 at 7:15 PM, Sahina Bose <sabose(a)redhat.com><= /span> wrote:

Make sure that you use replica 3 gluster volumes for storing VM images. Are you planning to deploy as a hyperconverged setup?
Either way, try and use the latest ovirt 3.6 and glusterfs 3.7 (3.7.12 that addresses bugs related to sharding and o-direct is due to be released soon)

On 06/21/2016 07:08 AM, Dewey Du wrote:

I want to deploy oVirt 3.6 with GlusterFS 3.7 to my online servers. Is it recommended for production usage?=C2= =A0

Thx.=C2=A0

__________________________________________= _____ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

--------------000209030506000905010408-- --===============6048425413915353151== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMDAyMDkwMzA1MDYwMDA5MDUwMTA0MDgKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQK CgoKT24gMDYvMjQvMjAxNiAwMzoxNiBQTSwgRGV3ZXkgRHUgd3JvdGU6Cj4gWW91IG1lYW4gd2Ug ZG9udCBuZWVkIHRvIGFkZCBuZXcgc3RvcmFnZSBkb21haW4gZm9yIGVhY2ggb3ZpcnQtbm9kZT8K PiBNYXliZSBJIGNob29zZSB0aGUgIlVzZSBIb3N0IiBvcHRpb24gd3JvbmdseT8gSSB0aG91Z2h0 IHRoaXMgaXMgdGhlIAo+IHdheSB0byB0aWUgdGhlIHN0b3JhZ2Ugb2YgIkhvc3QtMDEiIHRvIEds dXN0ZXJGUyBieSBzZWxlY3RpbmcgCj4gIkhvc3QtMDEiIG9uIHRoZSAiVXNlIEhvc3QiIG9wdGlv bi4KPiBBbmQgSSBzaG91bGQgZG8gdGhlIHNhbWUgdGhpbmcgZm9yICJIb3N0LTAyIiwgd2hpY2gg YWRkaW5nIG5ldyBzdG9yYWdlIAo+IGRvbWFpbiB3aXRoIGNob29zaW5nICJIb3N0LTAyIiBvbiB0 aGUgIlVzZSBIb3N0IiBvcHRpb24uCj4gU28gZG8gb3RoZXJzLiBIb3N0LTAzLCBIb3N0LTA0IC4u LgoKV2hpbGUgY3JlYXRpbmcgc3RvcmFnZSBkb21haW4sIHRoZSBob3N0IHByb3ZpZGVkIGluICJV c2UgaG9zdCIgaXMgdGhlIApvbmUgdXNlZCB0byBpbml0aWFsbHkgc2V0dXAgdGhlIGNvbm5lY3Rp b24gdG8gdGhlIHN0b3JhZ2UuCkluIGNhc2Ugb2YgZ2x1c3RlciBzdG9yYWdlIGRvbWFpbiwgaWYg bGlrZSB5b3Ugc2FpZCwgeW91IHdhbnQgdG8gCnNlcGFyYXRlIHN0b3JhZ2Ugc2VydmVycyBmcm9t IGh5cGVydmlzb3IgaG9zdHMgLSB5b3Ugd291bGQgc2V0IHVwIHRoZSAKZ2x1c3RlciB2b2x1bWUg dXNpbmcgYnJpY2tzIG9uIEhvc3QtMDQsIEhvc3QtMDUsIEhvc3QtMDYuCkFuZCBpbiBvVmlydCBj cmVhdGUgYSBjbHVzdGVyIHdpdGggeW91ciBoeXBlcnZpc29yIGhvc3RzIC0gSG9zdC0wMSwgCkhv c3QtMDIgYW5kIEhvc3QtMDMuIFRvIGNyZWF0ZSBhIGRhdGEgc3RvcmFnZSBkb21haW4gLSB1c2Ug YW55IG9mIHRoZSAKaG9zdHMgKDAxLTAzKSBhbmQgcHJvdmlkZSBwYXRoIHRvIGdsdXN0ZXIgdm9s dW1lIGFzIGhvc3QtMDQ6Lzx2b2xuYW1lPgoKSW4gY2FzZSBvZiBoeXBlcmNvbnZlcmdlZCBzZXR1 cCwgdGhlIGdsdXN0ZXIgdm9sdW1lIGNhbiBiZSBzZXR1cCBvbiAKSG9zdC0wMSwgSG9zdC0wMiwg SG9zdC0wMyBpdHNlbGYuIENoZWNrIApodHRwOi8vYmxvZ3MtcmFtZXNoLmJsb2dzcG90LmluLzIw MTYvMDEvb3ZpcnQtYW5kLWdsdXN0ZXItaHlwZXJjb252ZXJnZW5jZS5odG1sCgo+Cj4KPgo+IE9u IEZyaSwgSnVuIDI0LCAyMDE2IGF0IDM6MTUgUE0sIFNhaGluYSBCb3NlIDxzYWJvc2VAcmVkaGF0 LmNvbSAKPiA8bWFpbHRvOnNhYm9zZUByZWRoYXQuY29tPj4gd3JvdGU6Cj4KPgo+Cj4gICAgIE9u IDA2LzI0LzIwMTYgMTE6MjUgQU0sIERld2V5IER1IHdyb3RlOgo+PiAgICAgSSBwcmVmZXIgZGVw bG95aW5nIGFzIGEgaHlwZXJjb252ZXJnZWQgc2V0dXAsIGJ1dCBpdCBpcyBzdGlsbAo+PiAgICAg dW5kZXIgZXhwZXJpbWVudCwgcmlnaHQ/Cj4KPiAgICAgSHlwZXJjb252ZXJnZWQgZGVwbG95bWVu dCB3aXRoIG9WaXJ0IGFuZCBHbHVzdGVyIGhhcyBiZWVuIHRlc3RlZAo+ICAgICBhbmQgaXMgY3Vy cmVudGx5IG9mZmVyZWQgYXMgYSBwcmV2aWV3IGZlYXR1cmUgd2l0aCBndWlkYW5jZSBvbgo+ICAg ICBkbydzL2Rvbid0cyAmIHJlY29tbWVuZGVkIGdsdXN0ZXIgdm9sdW1lIHNldHRpbmdzLiBXZSdy ZSB3b3JraW5nCj4gICAgIG9uIGVuaGFuY2luZyB0aGlzIGZ1cnRoZXIgdG8gbWFrZSBpdCBlYXNp ZXIgdG8gc2V0dXAsIGludGVncmF0ZQo+ICAgICBiZXR0ZXIgaW4gdGhlIG9WaXJ0IFVJIGluIDQu MQo+Cj4+Cj4+ICAgICBTbyBJIHRyeSB0byBzZXBhcmF0ZSB2bSBzZXJ2aWNlIGFuZCBzdG9yYWdl LiBJIGFkZGVkIG5ldyBzdG9yYWdlCj4+ICAgICBkb21haW4oIERvbWFpbiBUeXBlICJEYXRhIiwg U3RvcmFnZSBUeXBlICJHbHVzdGVyRlMiLCBVc2UgSG9zdAo+PiAgICAgImhvc3QtMDEiLiBCdXQg dGhlbiBJIGNhbid0IGFkZCBhbm90aGVyIG5ldyBzdG9yYWdlIGRvbWFpbigKPj4gICAgIERvbWFp biBUeXBlICJEYXRhIiwgU3RvcmFnZSBUeXBlICJHbHVzdGVyRlMiLCBVc2UgSG9zdAo+PiAgICAg Imhvc3QtMDIiKS4gVGhlIGlucHV0IGZpZWxkICJwYXRoIiBpcyB1bndyaXRhYmxlIChncmF5KSBv biB0aGUKPj4gICAgIFBvcHVwIE5ldyBEb21haW4gV2luZG93Lgo+Pgo+PiAgICAgTXkgcXVlc3Rp b24gaXMsIHNob3VsZCB3ZSBhZGQgYSBuZXcgc3RvcmFnZSBkb21haW4gZm9yIGVhY2gKPj4gICAg IG92aXJ0LW5vZGU/Cj4KPiAgICAgTm8sIHlvdSBkb250IG5lZWQgdG8uCj4gICAgIFRoaXMgc2Vl bXMgbGlrZSBhIGJ1ZyBpbiB0aGUgQ3JlYXRlIHN0b3JhZ2UgZG9tYWluIFVJLiBEb2VzCj4gICAg IHJlZnJlc2hpbmcgeW91ciBicm93c2VyIGZpeCB0aGUgZ3JleWVkLW91dCBpbnB1dCBmaWVsZD8g QW55IGVycm9ycwo+ICAgICBzZWVuIGluIGVuZ2luZSBsb2dzPwo+Cj4KPj4KPj4KPj4KPj4gICAg IE9uIFR1ZSwgSnVuIDIxLCAyMDE2IGF0IDc6MTUgUE0sIFNhaGluYSBCb3NlIDxzYWJvc2VAcmVk aGF0LmNvbQo+PiAgICAgPG1haWx0bzpzYWJvc2VAcmVkaGF0LmNvbT4+IHdyb3RlOgo+Pgo+PiAg ICAgICAgIE1ha2Ugc3VyZSB0aGF0IHlvdSB1c2UgcmVwbGljYSAzIGdsdXN0ZXIgdm9sdW1lcyBm b3Igc3RvcmluZwo+PiAgICAgICAgIFZNIGltYWdlcy4gQXJlIHlvdSBwbGFubmluZyB0byBkZXBs b3kgYXMgYSBoeXBlcmNvbnZlcmdlZCBzZXR1cD8KPj4gICAgICAgICBFaXRoZXIgd2F5LCB0cnkg YW5kIHVzZSB0aGUgbGF0ZXN0IG92aXJ0IDMuNiBhbmQgZ2x1c3RlcmZzCj4+ICAgICAgICAgMy43 ICgzLjcuMTIgdGhhdCBhZGRyZXNzZXMgYnVncyByZWxhdGVkIHRvIHNoYXJkaW5nIGFuZAo+PiAg ICAgICAgIG8tZGlyZWN0IGlzIGR1ZSB0byBiZSByZWxlYXNlZCBzb29uKQo+Pgo+PiAgICAgICAg IE9uIDA2LzIxLzIwMTYgMDc6MDggQU0sIERld2V5IER1IHdyb3RlOgo+Pj4gICAgICAgICBJIHdh bnQgdG8gZGVwbG95IG9WaXJ0IDMuNiB3aXRoIEdsdXN0ZXJGUyAzLjcgdG8gbXkgb25saW5lCj4+ PiAgICAgICAgIHNlcnZlcnMuIElzIGl0IHJlY29tbWVuZGVkIGZvciBwcm9kdWN0aW9uIHVzYWdl Pwo+Pj4KPj4+ICAgICAgICAgVGh4Lgo+Pj4KPj4+Cj4+PiAgICAgICAgIF9fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCj4+PiAgICAgICAgIFVzZXJzIG1haWxp bmcgbGlzdAo+Pj4gICAgICAgICBVc2Vyc0BvdmlydC5vcmcgPG1haWx0bzpVc2Vyc0BvdmlydC5v cmc+Cj4+PiAgICAgICAgIGh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91 c2Vycwo+Pgo+Pgo+Cj4KCgotLS0tLS0tLS0tLS0tLTAwMDIwOTAzMDUwNjAwMDkwNTAxMDQwOApD b250ZW50LVR5cGU6IHRleHQvaHRtbDsgY2hhcnNldD11dGYtOApDb250ZW50LVRyYW5zZmVyLUVu Y29kaW5nOiA4Yml0Cgo8aHRtbD4KICA8aGVhZD4KICAgIDxtZXRhIGNvbnRlbnQ9InRleHQvaHRt bDsgY2hhcnNldD11dGYtOCIgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICA8L2hlYWQ+CiAg PGJvZHkgdGV4dD0iIzAwMDAwMCIgYmdjb2xvcj0iI0ZGRkZGRiI+CiAgICA8YnI+CiAgICA8YnI+ CiAgICA8ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPk9uIDA2LzI0LzIwMTYgMDM6MTYgUE0s IERld2V5IER1IHdyb3RlOjxicj4KICAgIDwvZGl2PgogICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlk OkNBTVNpRW02VldBN1lZTTBWcmh3eFFNYTVwY3VZOWM0Wmd6OHRqa2RFMk9zTEtIK0EtZ0BtYWls LmdtYWlsLmNvbSIKICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYgZGlyPSJsdHIiPllvdSBt ZWFuIHdlIGRvbnQgbmVlZCB0byBhZGQgbmV3IHN0b3JhZ2UgZG9tYWluIGZvcgogICAgICAgIGVh Y2ggb3ZpcnQtbm9kZT/CoAogICAgICAgIDxkaXY+TWF5YmUgSSBjaG9vc2UgdGhlICJVc2UgSG9z dCIgb3B0aW9uIHdyb25nbHk/IEkgdGhvdWdodAogICAgICAgICAgdGhpcyBpcyB0aGUgd2F5IHRv IHRpZSB0aGUgc3RvcmFnZSBvZiAiSG9zdC0wMSIgdG8gR2x1c3RlckZTCiAgICAgICAgICBieSBz ZWxlY3RpbmcgIkhvc3QtMDEiIG9uIHRoZSAiVXNlIEhvc3QiIG9wdGlvbi7CoDwvZGl2PgogICAg ICAgIDxkaXY+QW5kIEkgc2hvdWxkIGRvIHRoZSBzYW1lIHRoaW5nIGZvciAiSG9zdC0wMiIsIHdo aWNoIGFkZGluZwogICAgICAgICAgbmV3IHN0b3JhZ2UgZG9tYWluIHdpdGggY2hvb3NpbmcgIkhv c3QtMDIiIG9uIHRoZSAiVXNlIEhvc3QiCiAgICAgICAgICBvcHRpb24uwqA8L2Rpdj4KICAgICAg ICA8ZGl2PlNvIGRvIG90aGVycy4gSG9zdC0wMywgSG9zdC0wNCAuLi48L2Rpdj4KICAgICAgPC9k aXY+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+CiAgICBXaGlsZSBjcmVhdGluZyBzdG9yYWdl IGRvbWFpbiwgdGhlIGhvc3QgcHJvdmlkZWQgaW4gIlVzZSBob3N0IiBpcwogICAgdGhlIG9uZSB1 c2VkIHRvIGluaXRpYWxseSBzZXR1cCB0aGUgY29ubmVjdGlvbiB0byB0aGUgc3RvcmFnZS4gPGJy PgogICAgSW4gY2FzZSBvZiBnbHVzdGVyIHN0b3JhZ2UgZG9tYWluLCBpZiBsaWtlIHlvdSBzYWlk LCB5b3Ugd2FudCB0bwogICAgc2VwYXJhdGUgc3RvcmFnZSBzZXJ2ZXJzIGZyb20gaHlwZXJ2aXNv ciBob3N0cyAtIHlvdSB3b3VsZCBzZXQgdXAKICAgIHRoZSBnbHVzdGVyIHZvbHVtZSB1c2luZyBi cmlja3Mgb24gSG9zdC0wNCwgSG9zdC0wNSwgSG9zdC0wNi48YnI+CiAgICBBbmQgaW4gb1ZpcnQg Y3JlYXRlIGEgY2x1c3RlciB3aXRoIHlvdXIgaHlwZXJ2aXNvciBob3N0cyAtIEhvc3QtMDEsCiAg ICBIb3N0LTAyIGFuZCBIb3N0LTAzLiBUbyBjcmVhdGUgYSBkYXRhIHN0b3JhZ2UgZG9tYWluIC0g dXNlIGFueSBvZgogICAgdGhlIGhvc3RzICgwMS0wMykgYW5kIHByb3ZpZGUgcGF0aCB0byBnbHVz dGVyIHZvbHVtZSBhcwogICAgaG9zdC0wNDovJmx0O3ZvbG5hbWUmZ3Q7PGJyPgogICAgPGJyPgog ICAgSW4gY2FzZSBvZiBoeXBlcmNvbnZlcmdlZCBzZXR1cCwgdGhlIGdsdXN0ZXIgdm9sdW1lIGNh biBiZSBzZXR1cCBvbgogICAgSG9zdC0wMSwgSG9zdC0wMiwgSG9zdC0wMyBpdHNlbGYuIENoZWNr CjxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJlZXRleHQiIGhyZWY9Imh0dHA6Ly9ibG9ncy1yYW1l c2guYmxvZ3Nwb3QuaW4vMjAxNi8wMS9vdmlydC1hbmQtZ2x1c3Rlci1oeXBlcmNvbnZlcmdlbmNl Lmh0bWwiPmh0dHA6Ly9ibG9ncy1yYW1lc2guYmxvZ3Nwb3QuaW4vMjAxNi8wMS9vdmlydC1hbmQt Z2x1c3Rlci1oeXBlcmNvbnZlcmdlbmNlLmh0bWw8L2E+PGJyPgogICAgPGJyPgogICAgPGJsb2Nr cXVvdGUKY2l0ZT0ibWlkOkNBTVNpRW02VldBN1lZTTBWcmh3eFFNYTVwY3VZOWM0Wmd6OHRqa2RF Mk9zTEtIK0EtZ0BtYWlsLmdtYWlsLmNvbSIKICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgIDxkaXYg ZGlyPSJsdHIiPgogICAgICAgIDxkaXY+PGJyPgogICAgICAgIDwvZGl2PgogICAgICAgIDxkaXY+ PGJyPgogICAgICAgIDwvZGl2PgogICAgICA8L2Rpdj4KICAgICAgPGRpdiBjbGFzcz0iZ21haWxf ZXh0cmEiPjxicj4KICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gRnJpLCBKdW4g MjQsIDIwMTYgYXQgMzoxNSBQTSwgU2FoaW5hCiAgICAgICAgICBCb3NlIDxzcGFuIGRpcj0ibHRy Ij4mbHQ7PGEgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIgogICAgICAgICAgICAgIGhyZWY9Im1haWx0 bzpzYWJvc2VAcmVkaGF0LmNvbSIgdGFyZ2V0PSJfYmxhbmsiPnNhYm9zZUByZWRoYXQuY29tPC9h PiZndDs8L3NwYW4+CiAgICAgICAgICB3cm90ZTo8YnI+CiAgICAgICAgICA8YmxvY2txdW90ZSBj bGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAKICAgICAgICAgICAgLjhleDti b3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4KICAgICAgICAgICAg PGRpdiB0ZXh0PSIjMDAwMDAwIiBiZ2NvbG9yPSIjRkZGRkZGIj48c3BhbiBjbGFzcz0iIj4gPGJy PgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgPGRpdj5PbiAwNi8yNC8yMDE2 IDExOjI1IEFNLCBEZXdleSBEdSB3cm90ZTo8YnI+CiAgICAgICAgICAgICAgICA8L2Rpdj4KICAg ICAgICAgICAgICAgIDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPgogICAgICAgICAgICAgICAgICA8 ZGl2IGRpcj0ibHRyIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+SSBwcmVmZXIKICAg ICAgICAgICAgICAgICAgICAgIGRlcGxveWluZyBhcyBhIGg8L3NwYW4+PHNwYW4KICAgICAgICAg ICAgICAgICAgICAgIHN0eWxlPSJmb250LXNpemU6MTIuOHB4Ij55cGVyY29udmVyZ2VkIHNldHVw LCBidXQKICAgICAgICAgICAgICAgICAgICAgIGl0IGlzIHN0aWxsIHVuZGVywqA8L3NwYW4+ZXhw ZXJpbWVudCwgcmlnaHQ/IDxicj4KICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAg ICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgPC9z cGFuPiBIeXBlcmNvbnZlcmdlZCBkZXBsb3ltZW50IHdpdGggb1ZpcnQgYW5kIEdsdXN0ZXIKICAg ICAgICAgICAgICBoYXMgYmVlbiB0ZXN0ZWQgYW5kIGlzIGN1cnJlbnRseSBvZmZlcmVkIGFzIGEg cHJldmlldwogICAgICAgICAgICAgIGZlYXR1cmUgd2l0aCBndWlkYW5jZSBvbiBkbydzL2Rvbid0 cyAmYW1wOyByZWNvbW1lbmRlZAogICAgICAgICAgICAgIGdsdXN0ZXIgdm9sdW1lIHNldHRpbmdz LiBXZSdyZSB3b3JraW5nIG9uIGVuaGFuY2luZyB0aGlzCiAgICAgICAgICAgICAgZnVydGhlciB0 byBtYWtlIGl0IGVhc2llciB0byBzZXR1cCwgaW50ZWdyYXRlIGJldHRlciBpbgogICAgICAgICAg ICAgIHRoZSBvVmlydCBVSSBpbiA0LjE8c3BhbiBjbGFzcz0iIj48YnI+CiAgICAgICAgICAgICAg ICA8YnI+CiAgICAgICAgICAgICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4KICAgICAgICAg ICAgICAgICAgPGRpdiBkaXI9Imx0ciI+CiAgICAgICAgICAgICAgICAgICAgPGRpdj48c3BhbiBz dHlsZT0iZm9udC1zaXplOjEyLjhweCI+PGJyPgogICAgICAgICAgICAgICAgICAgICAgPC9zcGFu PjwvZGl2PgogICAgICAgICAgICAgICAgICAgIDxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox Mi44cHgiPlNvIEkgdHJ5IHRvwqA8L3NwYW4+PHNwYW4KICAgICAgICAgICAgICAgICAgICAgICAg c3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPnNlcGFyYXRlwqA8L3NwYW4+PHNwYW4KICAgICAgICAg ICAgICAgICAgICAgICAgc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPnZtIHNlcnZpY2UgYW5kCiAg ICAgICAgICAgICAgICAgICAgICAgIHN0b3JhZ2UuwqA8L3NwYW4+PHNwYW4gc3R5bGU9ImZvbnQt c2l6ZToxMi44cHgiPkkKICAgICAgICAgICAgICAgICAgICAgICAgYWRkZWQgbmV3wqA8L3NwYW4+ PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPnN0b3JhZ2UKICAgICAgICAgICAgICAgICAg ICAgICAgZG9tYWluKCBEb21haW4gVHlwZSAiRGF0YSIsIFN0b3JhZ2UgVHlwZQogICAgICAgICAg ICAgICAgICAgICAgICAiR2x1c3RlckZTIiwgVXNlIEhvc3QgImhvc3QtMDEiLiBCdXQgdGhlbjwv c3Bhbj48c3BhbgogICAgICAgICAgICAgICAgICAgICAgICBzdHlsZT0iZm9udC1zaXplOjEyLjhw eCI+wqBJIGNhbid0IGFkZCBhbm90aGVyCiAgICAgICAgICAgICAgICAgICAgICAgIG5ldyBzdG9y YWdlIGRvbWFpbiggRG9tYWluIFR5cGUgIkRhdGEiLCBTdG9yYWdlCiAgICAgICAgICAgICAgICAg ICAgICAgIFR5cGUgIkdsdXN0ZXJGUyIsIFVzZSBIb3N0ICJob3N0LTAyIikuIFRoZSBpbnB1dAog ICAgICAgICAgICAgICAgICAgICAgICBmaWVsZCAicGF0aCIgaXMgdW53cml0YWJsZSAoZ3JheSkg b24gdGhlIFBvcHVwCiAgICAgICAgICAgICAgICAgICAgICAgIE5ldyBEb21haW4gV2luZG93Ljwv c3Bhbj48L2Rpdj4KICAgICAgICAgICAgICAgICAgICA8ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNp emU6MTIuOHB4Ij48YnI+CiAgICAgICAgICAgICAgICAgICAgICA8L3NwYW4+PC9kaXY+CiAgICAg ICAgICAgICAgICAgICAgPGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjhweCI+TXkgcXVl c3Rpb24gaXMsCiAgICAgICAgICAgICAgICAgICAgICAgIHNob3VsZCB3ZSBhZGQgYSBuZXcgc3Rv cmFnZSBkb21haW4gZm9yIGVhY2gKICAgICAgICAgICAgICAgICAgICAgICAgb3ZpcnQtbm9kZT88 L3NwYW4+PC9kaXY+CiAgICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICAgICAgPC9i bG9ja3F1b3RlPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIDwvc3Bhbj4gTm8s IHlvdSBkb250IG5lZWQgdG8uPGJyPgogICAgICAgICAgICAgIFRoaXMgc2VlbXMgbGlrZSBhIGJ1 ZyBpbiB0aGUgQ3JlYXRlIHN0b3JhZ2UgZG9tYWluIFVJLgogICAgICAgICAgICAgIERvZXMgcmVm cmVzaGluZyB5b3VyIGJyb3dzZXIgZml4IHRoZSBncmV5ZWQtb3V0IGlucHV0CiAgICAgICAgICAg ICAgZmllbGQ/IEFueSBlcnJvcnMgc2VlbiBpbiBlbmdpbmUgbG9ncz88c3BhbiBjbGFzcz0iIj48 YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4KICAgICAgICAgICAgICAgICAgPGRpdiBkaXI9 Imx0ciI+CiAgICAgICAgICAgICAgICAgICAgPGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEy LjhweCI+PGJyPgogICAgICAgICAgICAgICAgICAgICAgPC9zcGFuPjwvZGl2PgogICAgICAgICAg ICAgICAgICAgIDxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi44cHgiPjxicj4KICAgICAg ICAgICAgICAgICAgICAgIDwvc3Bhbj48L2Rpdj4KICAgICAgICAgICAgICAgICAgPC9kaXY+CiAg ICAgICAgICAgICAgICAgIDxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj48YnI+CiAgICAgICAgICAg ICAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPk9uIFR1ZSwgSnVuIDIxLCAyMDE2IGF0 CiAgICAgICAgICAgICAgICAgICAgICA3OjE1IFBNLCBTYWhpbmEgQm9zZSA8c3BhbiBkaXI9Imx0 ciI+Jmx0OzxhCiAgICAgICAgICAgICAgICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVl IgogICAgICAgICAgICAgICAgICAgICAgICAgIGhyZWY9Im1haWx0bzpzYWJvc2VAcmVkaGF0LmNv bSIKICAgICAgICAgICAgICAgICAgICAgICAgICB0YXJnZXQ9Il9ibGFuayI+PGEgY2xhc3M9Im1v ei10eHQtbGluay1hYmJyZXZpYXRlZCIgaHJlZj0ibWFpbHRvOnNhYm9zZUByZWRoYXQuY29tIj5z YWJvc2VAcmVkaGF0LmNvbTwvYT48L2E+Jmd0Ozwvc3Bhbj4KICAgICAgICAgICAgICAgICAgICAg IHdyb3RlOjxicj4KICAgICAgICAgICAgICAgICAgICAgIDxibG9ja3F1b3RlIGNsYXNzPSJnbWFp bF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAKICAgICAgICAgICAgICAgICAgICAgICAgMCAuOGV4 O2JvcmRlci1sZWZ0OjFweCAjY2NjCiAgICAgICAgICAgICAgICAgICAgICAgIHNvbGlkO3BhZGRp bmctbGVmdDoxZXgiPgogICAgICAgICAgICAgICAgICAgICAgICA8ZGl2IHRleHQ9IiMwMDAwMDAi IGJnY29sb3I9IiNGRkZGRkYiPiBNYWtlIHN1cmUKICAgICAgICAgICAgICAgICAgICAgICAgICB0 aGF0IHlvdSB1c2UgcmVwbGljYSAzIGdsdXN0ZXIgdm9sdW1lcyBmb3IKICAgICAgICAgICAgICAg ICAgICAgICAgICBzdG9yaW5nIFZNIGltYWdlcy4gQXJlIHlvdSBwbGFubmluZyB0byBkZXBsb3kK ICAgICAgICAgICAgICAgICAgICAgICAgICBhcyBhIGh5cGVyY29udmVyZ2VkIHNldHVwPzxicj4K ICAgICAgICAgICAgICAgICAgICAgICAgICBFaXRoZXIgd2F5LCB0cnkgYW5kIHVzZSB0aGUgbGF0 ZXN0IG92aXJ0IDMuNgogICAgICAgICAgICAgICAgICAgICAgICAgIGFuZCBnbHVzdGVyZnMgMy43 ICgzLjcuMTIgdGhhdCBhZGRyZXNzZXMgYnVncwogICAgICAgICAgICAgICAgICAgICAgICAgIHJl bGF0ZWQgdG8gc2hhcmRpbmcgYW5kIG8tZGlyZWN0IGlzIGR1ZSB0byBiZQogICAgICAgICAgICAg ICAgICAgICAgICAgIHJlbGVhc2VkIHNvb24pPHNwYW4+PGJyPgogICAgICAgICAgICAgICAgICAg ICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdj5PbiAwNi8yMS8y MDE2IDA3OjA4IEFNLCBEZXdleSBEdSB3cm90ZTo8YnI+CiAgICAgICAgICAgICAgICAgICAgICAg ICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICA8L3NwYW4+CiAgICAgICAgICAg ICAgICAgICAgICAgICAgPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+PHNwYW4+CiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIDxkaXYgZGlyPSJsdHIiPkkgd2FudCB0byBkZXBsb3kgb1ZpcnQg My42CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgd2l0aCBHbHVzdGVyRlMgMy43IHRv IG15IG9ubGluZSBzZXJ2ZXJzLgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIElzIGl0 IHJlY29tbWVuZGVkIGZvciBwcm9kdWN0aW9uIHVzYWdlP8KgCiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9k aXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdj5UaHguwqA8YnI+CiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIDwvZGl2PgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIDxmaWVsZHNldD48L2ZpZWxkc2V0PgogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICA8L3NwYW4+ CiAgICAgICAgICAgICAgICAgICAgICAgICAgICA8cHJlPl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fClVzZXJzIG1haWxpbmcgbGlzdAo8YSBtb3otZG8tbm90 LXNlbmQ9InRydWUiIGhyZWY9Im1haWx0bzpVc2Vyc0BvdmlydC5vcmciIHRhcmdldD0iX2JsYW5r Ij5Vc2Vyc0BvdmlydC5vcmc8L2E+CjxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgaHJlZj0iaHR0 cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzIiB0YXJnZXQ9Il9ibGFu ayI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzPC9hPgo8L3By ZT4KICAgICAgICAgICAgICAgICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgICAg ICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAg ICAgICAgICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAg ICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAg ICAgICAgPC9ibG9ja3F1b3RlPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIDwv c3Bhbj48L2Rpdj4KICAgICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICA8L2Rpdj4KICAgICAg ICA8YnI+CiAgICAgIDwvZGl2PgogICAgPC9ibG9ja3F1b3RlPgogICAgPGJyPgogIDwvYm9keT4K PC9odG1sPgoKLS0tLS0tLS0tLS0tLS0wMDAyMDkwMzA1MDYwMDA5MDUwMTA0MDgtLQo= --===============6048425413915353151==--