From paf1 at email.cz Tue Mar 1 10:07:52 2016 Content-Type: multipart/mixed; boundary="===============0931395479680017672==" MIME-Version: 1.0 From: paf1 at email.cz To: users at ovirt.org Subject: [ovirt-users] open error -13 = sanlock Date: Tue, 01 Mar 2016 16:07:47 +0100 Message-ID: <56D5B043.8090605@email.cz> --===============0931395479680017672== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------070800050409070604090600 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 7bit Hello, can anybody explain this error no.13 ( open file ) in sanlock.log . The size of "ids" file is zero (0) 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 = /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk = /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids = error -13 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace = 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 If the main problem is about zero file size, can I regenerate this file = online securely , with no VM dependence ???? dist =3D RHEL - 7 - 2.1511 kernel =3D 3.10.0 - 327.10.1.el7.x86_64 KVM =3D 2.3.0 - 29.1.el7 libvirt =3D libvirt-1.2.17-13.el7_2.3 vdsm =3D vdsm-4.16.30-0.el7 GlusterFS =3D glusterfs-3.7.8-1.el7 regs. Pavel --------------070800050409070604090600 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit Hello,=C2=A0 can anybody=C2=A0 explain this error no.13 ( open file ) in sanlock.log .

The size of=C2=A0 "ids" file is zero (0)

2016-02-28 03:25:46+0100 269626 [1951]: open error -13 /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids
2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58= -89aa-102d44327124/dom_md/ids error -13
2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0

If the main problem is about zero file size, can I regenerate=C2=A0 this file online securely , with no VM dependence=C2=A0 ????

dist =3D RHEL - 7 - 2.1511
kernel =3D 3.10.0 - 327.10.1.el7.x86_64
KVM =3D 2.3.0 - 29.1.el7
libvirt =3D libvirt-1.2.17-13.el7_2.3
vdsm =3D vdsm-4.16.30-0.el7
GlusterFS =3D glusterfs-3.7.8-1.el7

regs.
Pavel
--------------070800050409070604090600-- --===============0931395479680017672== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wNzA4MDAwNTA0MDkwNzA2MDQwOTA2MDAKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQK CkhlbGxvLCAgY2FuIGFueWJvZHkgIGV4cGxhaW4gdGhpcyBlcnJvciBuby4xMyAoIG9wZW4gZmls ZSApIGluIHNhbmxvY2subG9nIC4KClRoZSBzaXplIG9mICAiaWRzIiBmaWxlIGlzIHplcm8gKDAp CgoyMDE2LTAyLTI4IDAzOjI1OjQ2KzAxMDAgMjY5NjI2IFsxOTUxXTogb3BlbiBlcnJvciAtMTMg Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1 MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHMKMjAxNi0wMi0yOCAw MzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IHMxODc5ODUgb3Blbl9kaXNrIAovcmhldi9kYXRh LWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMxOTkt NGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzIAplcnJvciAtMTMKMjAxNi0wMi0yOCAw MzoyNTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBzMTg3OTkyIGxvY2tzcGFjZSAKN2Y1MmI2OTct YzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0OjE6L3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQz MjcxMjQvZG9tX21kL2lkczowCgpJZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmls ZSBzaXplLCBjYW4gSSByZWdlbmVyYXRlICB0aGlzIGZpbGUgCm9ubGluZSBzZWN1cmVseSAsIHdp dGggbm8gVk0gZGVwZW5kZW5jZSAgPz8/PwoKCmRpc3QgPSBSSEVMIC0gNyAtIDIuMTUxMQprZXJu ZWwgPSAzLjEwLjAgLSAzMjcuMTAuMS5lbDcueDg2XzY0CktWTSA9IDIuMy4wIC0gMjkuMS5lbDcK bGlidmlydCA9IGxpYnZpcnQtMS4yLjE3LTEzLmVsN18yLjMKdmRzbSA9IHZkc20tNC4xNi4zMC0w LmVsNwpHbHVzdGVyRlMgPSBnbHVzdGVyZnMtMy43LjgtMS5lbDcKCgpyZWdzLgpQYXZlbAoKLS0t LS0tLS0tLS0tLS0wNzA4MDAwNTA0MDkwNzA2MDQwOTA2MDAKQ29udGVudC1UeXBlOiB0ZXh0L2h0 bWw7IGNoYXJzZXQ9dXRmLTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogOGJpdAoKPGh0bWw+ CiAgPGhlYWQ+CgogICAgPG1ldGEgaHR0cC1lcXVpdj0iY29udGVudC10eXBlIiBjb250ZW50PSJ0 ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPgogIDwvaGVhZD4KICA8Ym9keSB0ZXh0PSIjMDAwMDY2 IiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIEhlbGxvLMKgIGNhbiBhbnlib2R5wqAgZXhwbGFpbiB0 aGlzIGVycm9yIG5vLjEzICggb3BlbiBmaWxlICkgaW4KICAgIHNhbmxvY2subG9nIC48YnI+CiAg ICA8YnI+CiAgICBUaGUgc2l6ZSBvZsKgICJpZHMiIGZpbGUgaXMgemVybyAoMCk8YnI+CiAgICA8 YnI+CiAgICAyMDE2LTAyLTI4IDAzOjI1OjQ2KzAxMDAgMjY5NjI2IFsxOTUxXTogb3BlbiBlcnJv ciAtMTMKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1Q NC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkczxicj4KICAg IDIwMTYtMDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBzMTg3OTg1IG9wZW5fZGlz awogICAgL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1Q NC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkcwogICAgZXJy b3IgLTEzPGJyPgogICAgMjAxNi0wMi0yOCAwMzoyNTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBz MTg3OTkyIGxvY2tzcGFjZQo3ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQ6MTov cmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJi Njk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzOjA8YnI+CiAgICA8YnI+ CiAgICBJZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXplLCBjYW4gSSBy ZWdlbmVyYXRlwqAgdGhpcwogICAgZmlsZSBvbmxpbmUgc2VjdXJlbHkgLCB3aXRoIG5vIFZNIGRl cGVuZGVuY2XCoCA/Pz8/PGJyPgogICAgPGJyPgogICAgPGJyPgogICAgZGlzdCA9IFJIRUwgLSA3 IC0gMi4xNTExPGJyPgogICAga2VybmVsID0gMy4xMC4wIC0gMzI3LjEwLjEuZWw3Lng4Nl82NDxi cj4KICAgIEtWTSA9IDIuMy4wIC0gMjkuMS5lbDc8YnI+CiAgICBsaWJ2aXJ0ID0gbGlidmlydC0x LjIuMTctMTMuZWw3XzIuMzxicj4KICAgIHZkc20gPSB2ZHNtLTQuMTYuMzAtMC5lbDc8YnI+CiAg ICBHbHVzdGVyRlMgPSBnbHVzdGVyZnMtMy43LjgtMS5lbDc8YnI+CiAgICA8YnI+CiAgICA8YnI+ CiAgICByZWdzLjxicj4KICAgIFBhdmVsPGJyPgogIDwvYm9keT4KPC9odG1sPgoKLS0tLS0tLS0t LS0tLS0wNzA4MDAwNTA0MDkwNzA2MDQwOTA2MDAtLQo= --===============0931395479680017672==-- From frolland at redhat.com Tue Mar 1 10:34:07 2016 Content-Type: multipart/mixed; boundary="===============5374439313871475781==" MIME-Version: 1.0 From: Fred Rolland To: users at ovirt.org Subject: Re: [ovirt-users] open error -13 = sanlock Date: Tue, 01 Mar 2016 17:33:47 +0200 Message-ID: In-Reply-To: 56D5B043.8090605@email.cz --===============5374439313871475781== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Hi, Can you share VDSM logs ? There was something similar in this thread: http://lists.ovirt.org/pipermail/users/2016-February/038046.html Thanks, Fred On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz wrote: > Hello, can anybody explain this error no.13 ( open file ) in sanlock.log > . > > The size of "ids" file is zero (0) > > 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids > 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids > error -13 > 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace > 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/lo= calhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 > > If the main problem is about zero file size, can I regenerate this file > online securely , with no VM dependence ???? > > > dist =3D RHEL - 7 - 2.1511 > kernel =3D 3.10.0 - 327.10.1.el7.x86_64 > KVM =3D 2.3.0 - 29.1.el7 > libvirt =3D libvirt-1.2.17-13.el7_2.3 > vdsm =3D vdsm-4.16.30-0.el7 > GlusterFS =3D glusterfs-3.7.8-1.el7 > > > regs. > Pavel > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > --===============5374439313871475781== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+PGRpdj48ZGl2PjxkaXY+PGRpdj5IaSw8YnI+PC9kaXY+PGJyPkNhbiB5 b3Ugc2hhcmUgVkRTTSBsb2dzID88YnI+PGJyPjwvZGl2PlRoZXJlIHdhcyBzb21ldGhpbmcgc2lt aWxhciBpbiB0aGlzIHRocmVhZDo8YnI+PGEgaHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9w aXBlcm1haWwvdXNlcnMvMjAxNi1GZWJydWFyeS8wMzgwNDYuaHRtbCIgdGFyZ2V0PSJfYmxhbmsi Pmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvcGlwZXJtYWlsL3VzZXJzLzIwMTYtRmVicnVhcnkvMDM4 MDQ2Lmh0bWw8L2E+PGJyPjxicj48L2Rpdj5UaGFua3MsPGJyPjwvZGl2PkZyZWQ8YnI+PC9kaXY+ PGRpdiBjbGFzcz0iZ21haWxfZXh0cmEiPjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24g VHVlLCBNYXIgMSwgMjAxNiBhdCA1OjA3IFBNLCA8YSBocmVmPSJtYWlsdG86cGFmMUBlbWFpbC5j eiI+cGFmMUBlbWFpbC5jejwvYT4gPHNwYW4gZGlyPSJsdHIiPiZsdDs8YSBocmVmPSJtYWlsdG86 cGFmMUBlbWFpbC5jeiIgdGFyZ2V0PSJfYmxhbmsiPnBhZjFAZW1haWwuY3o8L2E+Jmd0Ozwvc3Bh bj4gd3JvdGU6PGJyPjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdp bjowIDAgMCAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgi PgogIAoKICAgIAogIAogIDxkaXYgdGV4dD0iIzAwMDA2NiIgYmdjb2xvcj0iI0ZGRkZGRiI+CiAg ICBIZWxsbyzCoCBjYW4gYW55Ym9kecKgIGV4cGxhaW4gdGhpcyBlcnJvciBuby4xMyAoIG9wZW4g ZmlsZSApIGluCiAgICBzYW5sb2NrLmxvZyAuPGJyPgogICAgPGJyPgogICAgVGhlIHNpemUgb2bC oCAmcXVvdDtpZHMmcXVvdDsgZmlsZSBpcyB6ZXJvICgwKTxicj4KICAgIDxicj4KICAgIDIwMTYt MDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBvcGVuIGVycm9yIC0xMwovcmhldi9k YXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMx OTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzPGJyPgogICAgMjAxNi0wMi0yOCAw MzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IHMxODc5ODUgb3Blbl9kaXNrCiAgICAvcmhldi9k YXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMx OTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzCiAgICBlcnJvciAtMTM8YnI+CiAg ICAyMDE2LTAyLTI4IDAzOjI1OjU2KzAxMDAgMjY5NjM2IFsxMTMwNF06IHMxODc5OTIgbG9ja3Nw YWNlCjdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNDoxOi9yaGV2L2RhdGEtY2Vu dGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4 LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM6MDxicj4KICAgIDxicj4KICAgIElmIHRoZSBt YWluIHByb2JsZW0gaXMgYWJvdXQgemVybyBmaWxlIHNpemUsIGNhbiBJIHJlZ2VuZXJhdGXCoCB0 aGlzCiAgICBmaWxlIG9ubGluZSBzZWN1cmVseSAsIHdpdGggbm8gVk0gZGVwZW5kZW5jZcKgID8/ Pz88YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICBkaXN0ID0gUkhFTCAtIDcgLSAyLjE1MTE8YnI+ CiAgICBrZXJuZWwgPSAzLjEwLjAgLSAzMjcuMTAuMS5lbDcueDg2XzY0PGJyPgogICAgS1ZNID0g Mi4zLjAgLSAyOS4xLmVsNzxicj4KICAgIGxpYnZpcnQgPSBsaWJ2aXJ0LTEuMi4xNy0xMy5lbDdf Mi4zPGJyPgogICAgdmRzbSA9IHZkc20tNC4xNi4zMC0wLmVsNzxicj4KICAgIEdsdXN0ZXJGUyA9 IGdsdXN0ZXJmcy0zLjcuOC0xLmVsNzxicj4KICAgIDxicj4KICAgIDxicj4KICAgIHJlZ3MuPGJy PgogICAgUGF2ZWw8YnI+CiAgPC9kaXY+Cgo8YnI+X19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX188YnI+ClVzZXJzIG1haWxpbmcgbGlzdDxicj4KPGEgaHJlZj0i bWFpbHRvOlVzZXJzQG92aXJ0Lm9yZyI+VXNlcnNAb3ZpcnQub3JnPC9hPjxicj4KPGEgaHJlZj0i aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzIiByZWw9Im5vcmVm ZXJyZXIiIHRhcmdldD0iX2JsYW5rIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlz dGluZm8vdXNlcnM8L2E+PGJyPgo8YnI+PC9ibG9ja3F1b3RlPjwvZGl2Pjxicj48L2Rpdj4K --===============5374439313871475781==-- From nsoffer at redhat.com Tue Mar 1 12:38:56 2016 Content-Type: multipart/mixed; boundary="===============1535756435803799399==" MIME-Version: 1.0 From: Nir Soffer To: users at ovirt.org Subject: Re: [ovirt-users] open error -13 = sanlock Date: Tue, 01 Mar 2016 19:38:55 +0200 Message-ID: In-Reply-To: 56D5B043.8090605@email.cz --===============1535756435803799399== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz wrote: > Hello, can anybody explain this error no.13 ( open file ) in sanlock.log > . > This is EACCES Can you share the outoput of: ls -lh /rhev/data-center/mnt/:<_path>//dom_md > > The size of "ids" file is zero (0) > This is how we create the ids file when initializing it. But then we use sanlock to initialize the ids file, and it should be 1MiB after that. Is this ids files created by vdsm, or one you created yourself? > 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids > 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids > error -13 > 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace > 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/lo= calhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 > > If the main problem is about zero file size, can I regenerate this file > online securely , with no VM dependence ???? > Yes, I think I already referred to the instructions how to do that in a previous mail. > > dist =3D RHEL - 7 - 2.1511 > kernel =3D 3.10.0 - 327.10.1.el7.x86_64 > KVM =3D 2.3.0 - 29.1.el7 > libvirt =3D libvirt-1.2.17-13.el7_2.3 > vdsm =3D vdsm-4.16.30-0.el7 > GlusterFS =3D glusterfs-3.7.8-1.el7 > > > regs. > Pavel > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > --===============1535756435803799399== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+PGRpdiBjbGFzcz0iZ21haWxfZXh0cmEiPjxkaXYgY2xhc3M9ImdtYWls X3F1b3RlIj5PbiBUdWUsIE1hciAxLCAyMDE2IGF0IDU6MDcgUE0sIDxhIGhyZWY9Im1haWx0bzpw YWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiA8c3BhbiBkaXI9Imx0ciI+Jmx0OzxhIGhy ZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6IiB0YXJnZXQ9Il9ibGFuayI+cGFmMUBlbWFpbC5jejwv YT4mZ3Q7PC9zcGFuPiB3cm90ZTo8YnI+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBz dHlsZT0ibWFyZ2luOjBweCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0LXdpZHRoOjFweDtib3Jk ZXItbGVmdC1jb2xvcjpyZ2IoMjA0LDIwNCwyMDQpO2JvcmRlci1sZWZ0LXN0eWxlOnNvbGlkO3Bh ZGRpbmctbGVmdDoxZXgiPgogIAoKICAgIAogIAogIDxkaXYgdGV4dD0iIzAwMDA2NiIgYmdjb2xv cj0iI0ZGRkZGRiI+CiAgICBIZWxsbyzCoCBjYW4gYW55Ym9kecKgIGV4cGxhaW4gdGhpcyBlcnJv ciBuby4xMyAoIG9wZW4gZmlsZSApIGluCiAgICBzYW5sb2NrLmxvZyAuPGJyPjwvZGl2PjwvYmxv Y2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PlRoaXMgaXMgRUFDQ0VTPC9kaXY+PGRpdj48YnI+ PC9kaXY+PGRpdj5DYW4geW91IHNoYXJlIHRoZSBvdXRvcHV0IG9mOjwvZGl2PjxkaXY+PGJyPjwv ZGl2PjxkaXY+wqAgwqAgbHMgLWxoIC9yaGV2L2RhdGEtY2VudGVyL21udC8mbHQ7c2VydmVyJmd0 OzombHQ7X3BhdGgmZ3Q7LyZsdDtzZF91dWlkJmd0Oy9kb21fbWQ8L2Rpdj48ZGl2PsKgPC9kaXY+ PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjBweCAwcHggMHB4 IDAuOGV4O2JvcmRlci1sZWZ0LXdpZHRoOjFweDtib3JkZXItbGVmdC1jb2xvcjpyZ2IoMjA0LDIw NCwyMDQpO2JvcmRlci1sZWZ0LXN0eWxlOnNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPjxkaXYgdGV4 dD0iIzAwMDA2NiIgYmdjb2xvcj0iI0ZGRkZGRiI+CiAgICA8YnI+CiAgICBUaGUgc2l6ZSBvZsKg ICZxdW90O2lkcyZxdW90OyBmaWxlIGlzIHplcm8gKDApPGJyPjwvZGl2PjwvYmxvY2txdW90ZT48 ZGl2Pjxicj48L2Rpdj48ZGl2PlRoaXMgaXMgaG93IHdlIGNyZWF0ZSB0aGUgaWRzIGZpbGUgd2hl biBpbml0aWFsaXppbmcgaXQuPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5CdXQgdGhlbiB3ZSB1 c2Ugc2FubG9jayB0byBpbml0aWFsaXplIHRoZSBpZHMgZmlsZSwgYW5kIGl0IHNob3VsZCBiZSAx TWlCIGFmdGVyIHRoYXQuPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5JcyB0aGlzIGlkcyBmaWxl cyBjcmVhdGVkIGJ5IHZkc20sIG9yIG9uZSB5b3UgY3JlYXRlZCB5b3Vyc2VsZj88L2Rpdj48ZGl2 PsKgPC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjBw eCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0LXdpZHRoOjFweDtib3JkZXItbGVmdC1jb2xvcjpy Z2IoMjA0LDIwNCwyMDQpO2JvcmRlci1sZWZ0LXN0eWxlOnNvbGlkO3BhZGRpbmctbGVmdDoxZXgi PjxkaXYgdGV4dD0iIzAwMDA2NiIgYmdjb2xvcj0iI0ZGRkZGRiI+CiAgICAyMDE2LTAyLTI4IDAz OjI1OjQ2KzAxMDAgMjY5NjI2IFsxOTUxXTogb3BlbiBlcnJvciAtMTMKL3JoZXYvZGF0YS1jZW50 ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgt ODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkczxicj4KICAgIDIwMTYtMDItMjggMDM6MjU6NDYr MDEwMCAyNjk2MjYgWzE5NTFdOiBzMTg3OTg1IG9wZW5fZGlzawogICAgL3JoZXYvZGF0YS1jZW50 ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgt ODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkcwogICAgZXJyb3IgLTEzPGJyPgogICAgMjAxNi0w Mi0yOCAwMzoyNTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBzMTg3OTkyIGxvY2tzcGFjZQo3ZjUy YjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQ6MTovcmhldi9kYXRhLWNlbnRlci9tbnQv Z2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEw MmQ0NDMyNzEyNC9kb21fbWQvaWRzOjA8YnI+CiAgICA8YnI+CiAgICBJZiB0aGUgbWFpbiBwcm9i bGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXplLCBjYW4gSSByZWdlbmVyYXRlwqAgdGhpcwogICAg ZmlsZSBvbmxpbmUgc2VjdXJlbHkgLCB3aXRoIG5vIFZNIGRlcGVuZGVuY2XCoCA/Pz8/PGJyPjwv ZGl2PjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PlllcywgSSB0aGluayBJIGFscmVh ZHkgcmVmZXJyZWQgdG8gdGhlIGluc3RydWN0aW9ucyBob3cgdG8gZG8gdGhhdCBpbiBhIHByZXZp b3VzIG1haWwuPGJyPjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFp bF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdC13aWR0 aDoxcHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVmdC1zdHls ZTpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IHRleHQ9IiMwMDAwNjYiIGJnY29sb3I9IiNG RkZGRkYiPgogICAgPGJyPgogICAgPGJyPgogICAgZGlzdCA9IFJIRUwgLSA3IC0gMi4xNTExPGJy PgogICAga2VybmVsID0gMy4xMC4wIC0gMzI3LjEwLjEuZWw3Lng4Nl82NDxicj4KICAgIEtWTSA9 IDIuMy4wIC0gMjkuMS5lbDc8YnI+CiAgICBsaWJ2aXJ0ID0gbGlidmlydC0xLjIuMTctMTMuZWw3 XzIuMzxicj4KICAgIHZkc20gPSB2ZHNtLTQuMTYuMzAtMC5lbDc8YnI+CiAgICBHbHVzdGVyRlMg PSBnbHVzdGVyZnMtMy43LjgtMS5lbDc8YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICByZWdzLjxi cj4KICAgIFBhdmVsPGJyPgogIDwvZGl2PgoKPGJyPl9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fPGJyPgpVc2VycyBtYWlsaW5nIGxpc3Q8YnI+CjxhIGhyZWY9 Im1haWx0bzpVc2Vyc0BvdmlydC5vcmciPlVzZXJzQG92aXJ0Lm9yZzwvYT48YnI+CjxhIGhyZWY9 Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyIgcmVsPSJub3Jl ZmVycmVyIiB0YXJnZXQ9Il9ibGFuayI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xp c3RpbmZvL3VzZXJzPC9hPjxicj4KPGJyPjwvYmxvY2txdW90ZT48L2Rpdj48YnI+PC9kaXY+PC9k aXY+Cg== --===============1535756435803799399==-- From paf1 at email.cz Tue Mar 1 15:51:21 2016 Content-Type: multipart/mixed; boundary="===============4946150822962749594==" MIME-Version: 1.0 From: paf1 at email.cz To: users at ovirt.org Subject: Re: [ovirt-users] open error -13 = sanlock Date: Tue, 01 Mar 2016 21:51:15 +0100 Message-ID: <56D600C3.8010405@email.cz> In-Reply-To: CAMRbyyus-rS0RtOWzePwrWp7xoOH8yRgT0-_wtcuM8WiWdO4VA@mail.gmail.com --===============4946150822962749594== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------080108070503070703050100 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit HI, requested output: # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-be= f3-0bc0b7a99fb7/dom_md: total 2,1M -rw-rw---- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 inbox -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.17 leases -rw-r--r-- 1 vdsm kvm 335 7. lis 22.17 metadata -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 outbox /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a57= 9-4cabeb55ded4/dom_md: total 1,1M -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.41 ids -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 inbox -rw-rw---- 1 vdsm kvm 2,0M 7. lis 03.56 leases -rw-r--r-- 1 vdsm kvm 333 7. lis 03.56 metadata -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 outbox /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-999= 2-b04464a04112/dom_md: total 1,1M -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 inbox -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.14 leases -rw-r--r-- 1 vdsm kvm 333 7. lis 22.14 metadata -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 outbox /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-ab4= 6-084f3d70b147/dom_md: total 1,1M -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.51 inbox -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases -rw-r--r-- 1 vdsm kvm 998 25. =C3=BAno 00.35 metadata -rw-rw---- 1 vdsm kvm 16M 7. lis 00.16 outbox /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md: total 1,1M -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.44 ids -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 inbox -rw-rw---- 1 vdsm kvm 2,0M 7. lis 00.18 leases -rw-r--r-- 1 vdsm kvm 333 7. lis 00.18 metadata -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 outbox /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-be4= 1-77002d1cd553/dom_md: total 1,1M -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.32 ids -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 inbox -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.18 leases -rw-r--r-- 1 vdsm kvm 333 7. lis 22.18 metadata -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 outbox /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9bf= e-c3da888e47f0/dom_md: total 3,0M -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.42 inbox -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases -rw-r--r-- 1 vdsm kvm 997 24. =C3=BAno 02.46 metadata -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.44 outbox /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-ba9= a-e6d9bdecb4b4/dom_md: total 2,1M -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.34 ids -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.35 inbox -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.27 outbox /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-9b= b1-1df2130c7c97/dom_md: total 3,0M -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids -rw-rw-r-- 1 vdsm kvm 16M 6. lis 23.50 inbox -rw-rw-r-- 1 vdsm kvm 2,0M 6. lis 23.51 leases -rw-rw-r-- 1 vdsm kvm 734 7. lis 02.13 metadata -rw-rw-r-- 1 vdsm kvm 16M 6. lis 16.55 outbox /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9f5= 9-3aebea4d8477/dom_md: total 1,1M -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.35 ids -rw-rw-r-- 1 vdsm kvm 16M 24. =C3=BAno 01.06 inbox -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases -rw-r--r-- 1 vdsm kvm 998 24. =C3=BAno 19.07 metadata -rw-rw-r-- 1 vdsm kvm 16M 7. lis 22.20 outbox The ids files was generated by "touch" command after deleting them due = "sanlock locking hang" gluster crash & reboot I expected that they will be filled automaticaly after gluster reboot ( = the shadow copy from ".gluster " directory was deleted & created = empty too ) OK, it looks that sanlock can't work with empty file or rewrite them . Am I right ?? The last point - about "ids" workaround - this is offline version =3D VMs = have to be moved out from for continual running with maintenance volume mode But this is not acceptable in current situation, so the question again, = is it safe to do it online ?? ( YES / NO ) regs. Pavel On 1.3.2016 18:38, Nir Soffer wrote: > On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz = > > wrote: > > Hello, can anybody explain this error no.13 ( open file ) in > sanlock.log . > > > This is EACCES > > Can you share the outoput of: > > ls -lh /rhev/data-center/mnt/:<_path>//dom_md > > > The size of "ids" file is zero (0) > > > This is how we create the ids file when initializing it. > > But then we use sanlock to initialize the ids file, and it should be = > 1MiB after that. > > Is this ids files created by vdsm, or one you created yourself? > > 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f= 58-89aa-102d44327124/dom_md/ids > 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f= 58-89aa-102d44327124/dom_md/ids > error -13 > 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace > 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterS= D/localhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 > > If the main problem is about zero file size, can I regenerate = > this file online securely , with no VM dependence ???? > > > Yes, I think I already referred to the instructions how to do that in = > a previous mail. > > > > dist =3D RHEL - 7 - 2.1511 > kernel =3D 3.10.0 - 327.10.1.el7.x86_64 > KVM =3D 2.3.0 - 29.1.el7 > libvirt =3D libvirt-1.2.17-13.el7_2.3 > vdsm =3D vdsm-4.16.30-0.el7 > GlusterFS =3D glusterfs-3.7.8-1.el7 > > > regs. > Pavel > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > --------------080108070503070703050100 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit HI,
requested output:

# ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md
=C2=A0
/rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-be= f3-0bc0b7a99fb7/dom_md:
total 2,1M
-rw-rw---- 1 vdsm kvm 1,0M=C2=A0 1.=C2=A0b=C5=99e 21.28 ids
-rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 22.16 inbox
-rw-rw---- 1 vdsm kvm 2,0M=C2=A0 7.=C2=A0lis 22.17 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 335=C2=A0 7.=C2=A0lis 22.17 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 22.16 outbox

/rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a57= 9-4cabeb55ded4/dom_md:
total 1,1M
-rw-r--r-- 1 vdsm kvm=C2=A0=C2=A0=C2=A0 0 24.=C2=A0=C3=BAno 07.41 ids -rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 00.14 inbox
-rw-rw---- 1 vdsm kvm 2,0M=C2=A0 7.=C2=A0lis 03.56 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 333=C2=A0 7.=C2=A0lis 03.56 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 00.14 outbox

/rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-999= 2-b04464a04112/dom_md:
total 1,1M
-rw-r--r-- 1 vdsm kvm=C2=A0=C2=A0=C2=A0 0 24.=C2=A0=C3=BAno 07.43 ids -rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 00.15 inbox
-rw-rw---- 1 vdsm kvm 2,0M=C2=A0 7.=C2=A0lis 22.14 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 333=C2=A0 7.=C2=A0lis 22.14 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 00.15 outbox

/rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-ab4= 6-084f3d70b147/dom_md:
total 1,1M
-rw-r--r-- 1 vdsm kvm=C2=A0=C2=A0=C2=A0 0 24.=C2=A0=C3=BAno 07.43 ids -rw-rw---- 1 vdsm kvm=C2=A0 16M 23.=C2=A0=C3=BAno 22.51 inbox
-rw-rw---- 1 vdsm kvm 2,0M 23.=C2=A0=C3=BAno 23.12 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 998 25.=C2=A0=C3=BAno 00.35 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 00.16 outbox

/rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md:
total 1,1M
-rw-r--r-- 1 vdsm kvm=C2=A0=C2=A0=C2=A0 0 24.=C2=A0=C3=BAno 07.44 ids -rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 00.17 inbox
-rw-rw---- 1 vdsm kvm 2,0M=C2=A0 7.=C2=A0lis 00.18 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 333=C2=A0 7.=C2=A0lis 00.18 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 00.17 outbox

/rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-be4= 1-77002d1cd553/dom_md:
total 1,1M
-rw-rw-r-- 1 vdsm kvm=C2=A0=C2=A0=C2=A0 0 24.=C2=A0=C3=BAno 07.32 ids -rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 22.18 inbox
-rw-rw---- 1 vdsm kvm 2,0M=C2=A0 7.=C2=A0lis 22.18 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 333=C2=A0 7.=C2=A0lis 22.18 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 22.18 outbox

/rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9bf= e-c3da888e47f0/dom_md:
total 3,0M
-rw-rw-r-- 1 vdsm kvm 1,0M=C2=A0 1.=C2=A0b=C5=99e 21.28 ids
-rw-rw---- 1 vdsm kvm=C2=A0 16M 25.=C2=A0=C3=BAno 00.42 inbox
-rw-rw---- 1 vdsm kvm 2,0M 25.=C2=A0=C3=BAno 00.44 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 997 24.=C2=A0=C3=BAno 02.46 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M 25.=C2=A0=C3=BAno 00.44 outbox

/rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-ba9= a-e6d9bdecb4b4/dom_md:
total 2,1M
-rw-r--r-- 1 vdsm kvm=C2=A0=C2=A0=C2=A0 0 24.=C2=A0=C3=BAno 07.34 ids -rw-rw---- 1 vdsm kvm=C2=A0 16M 23.=C2=A0=C3=BAno 22.35 inbox
-rw-rw---- 1 vdsm kvm 2,0M 23.=C2=A0=C3=BAno 22.38 leases
-rw-r--r-- 1 vdsm kvm 1,1K 24.=C2=A0=C3=BAno 19.07 metadata
-rw-rw---- 1 vdsm kvm=C2=A0 16M 23.=C2=A0=C3=BAno 22.27 outbox

/rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-9b= b1-1df2130c7c97/dom_md:
total 3,0M
-rw-rw-r-- 1 vdsm kvm 1,0M=C2=A0 1.=C2=A0b=C5=99e 21.28 ids
-rw-rw-r-- 1 vdsm kvm=C2=A0 16M=C2=A0 6.=C2=A0lis 23.50 inbox
-rw-rw-r-- 1 vdsm kvm 2,0M=C2=A0 6.=C2=A0lis 23.51 leases
-rw-rw-r-- 1 vdsm kvm=C2=A0 734=C2=A0 7.=C2=A0lis 02.13 metadata
-rw-rw-r-- 1 vdsm kvm=C2=A0 16M=C2=A0 6.=C2=A0lis 16.55 outbox

/rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9f5= 9-3aebea4d8477/dom_md:
total 1,1M
-rw-rw-r-- 1 vdsm kvm=C2=A0=C2=A0=C2=A0 0 24.=C2=A0=C3=BAno 07.35 ids -rw-rw-r-- 1 vdsm kvm=C2=A0 16M 24.=C2=A0=C3=BAno 01.06 inbox
-rw-rw-r-- 1 vdsm kvm 2,0M 24.=C2=A0=C3=BAno 02.44 leases
-rw-r--r-- 1 vdsm kvm=C2=A0 998 24.=C2=A0=C3=BAno 19.07 metadata
-rw-rw-r-- 1 vdsm kvm=C2=A0 16M=C2=A0 7.=C2=A0lis 22.20 outbox

=C2=A0The ids files was generated by "touch" command after deleting them due "sanlock locking hang"=C2=A0 gluster crash & reboot
I expected that they will be filled automaticaly after gluster reboot ( the=C2=A0 shadow copy from=C2=A0=C2=A0 ".gluster " =C2=A0 dire= ctory=C2=A0 was deleted & created empty=C2=A0 too )

OK, it looks that sanlock=C2=A0 can't work with empty file or rewrite them .
Am I right ??

The last point - about "ids" workaround - this is offline version =3D VMs have to be moved out from for continual running with maintenance volume mode
But this is not acceptable in current situation, so the question again,=C2=A0 is it safe to do it online ??=C2=A0 ( YES / NO )

regs.
Pavel

On 1.3.2016 18:38, Nir Soffer wrote:

On Tue, Mar 1, 2016 at 5:07 PM, paf1= (a)email.cz <paf1(a)em= ail.cz> wrote:

Hello,=C2=A0 can anybody=C2=A0 explain this error no.13 ( open file ) in sanlock.log .

This is EACCES

Can you share the outoput of:

=C2=A0 =C2=A0 ls -lh /rhev/data-center/mnt/<server>:<_path>/<sd_uuid>/dom_md
=C2=A0

The size of=C2=A0 "ids" file is zero (0)

This is how we create the ids file when initializing it.

But then we use sanlock to initialize the ids file, and it should be 1MiB after that.

Is this ids files created by vdsm, or one you created yourself?

=C2=A0

2016-02-28 03:25:46+0100 269626 [1951]: open error -13 /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids
2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b6= 97-c199-4f58-89aa-102d44327124/dom_md/ids error -13
2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0

If the main problem is about zero file size, can I regenerate=C2=A0 this file online securely , with no VM dependence=C2=A0 ????

Yes, I think I already referred to the instructions how to do that in a previous mail.

dist =3D RHEL - 7 - 2.1511
kernel =3D 3.10.0 - 327.10.1.el7.x86_64
KVM =3D 2.3.0 - 29.1.el7
libvirt =3D libvirt-1.2.17-13.el7_2.3
vdsm =3D vdsm-4.16.30-0.el7
GlusterFS =3D glusterfs-3.7.8-1.el7

regs.
Pavel

_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org= /mailman/listinfo/users

--------------080108070503070703050100-- --===============4946150822962749594== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wODAxMDgwNzA1MDMwNzA3MDMwNTAxMDAKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK CkhJLApyZXF1ZXN0ZWQgb3V0cHV0OgoKIyBscyAtbGggL3JoZXYvZGF0YS1jZW50ZXIvbW50L2ds dXN0ZXJTRC9sb2NhbGhvc3Q6Ki8qL2RvbV9tZAoKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1CQ0svMGZjYWQ4ODgtZDU3My00N2JlLWJlZjMtMGJjMGI3 YTk5ZmI3L2RvbV9tZDoKdG90YWwgMiwxTQotcnctcnctLS0tIDEgdmRzbSBrdm0gMSwwTSAgMS4g YsWZZSAyMS4yOCBpZHMKLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAyMi4xNiBp bmJveAotcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDIyLjE3IGxlYXNlcwotcnct ci0tci0tIDEgdmRzbSBrdm0gIDMzNSAgNy4gbGlzIDIyLjE3IG1ldGFkYXRhCi1ydy1ydy0tLS0g MSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMjIuMTYgb3V0Ym94Cgovcmhldi9kYXRhLWNlbnRlci9t bnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVAxLzU1M2Q5YjkyLWU0YTAtNDA0Mi1hNTc5 LTRjYWJlYjU1ZGVkNC9kb21fbWQ6CnRvdGFsIDEsMU0KLXJ3LXItLXItLSAxIHZkc20ga3ZtICAg IDAgMjQuIMO6bm8gMDcuNDEgaWRzCi1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMg MDAuMTQgaW5ib3gKLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gIDcuIGxpcyAwMy41NiBsZWFz ZXMKLXJ3LXItLXItLSAxIHZkc20ga3ZtICAzMzMgIDcuIGxpcyAwMy41NiBtZXRhZGF0YQotcnct cnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE0IG91dGJveAoKL3JoZXYvZGF0YS1j ZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QMi84OGFkYmQ0OS02MmQ2LTQ1 YjEtOTk5Mi1iMDQ0NjRhMDQxMTIvZG9tX21kOgp0b3RhbCAxLDFNCi1ydy1yLS1yLS0gMSB2ZHNt IGt2bSAgICAwIDI0LiDDum5vIDA3LjQzIGlkcwotcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAg Ny4gbGlzIDAwLjE1IGluYm94Ci1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNICA3LiBsaXMgMjIu MTQgbGVhc2VzCi1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzMzICA3LiBsaXMgMjIuMTQgbWV0YWRh dGEKLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNSBvdXRib3gKCi9yaGV2 L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDMvM2MzNGFkNjMt NmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZDoKdG90YWwgMSwxTQotcnctci0tci0t IDEgdmRzbSBrdm0gICAgMCAyNC4gw7pubyAwNy40MyBpZHMKLXJ3LXJ3LS0tLSAxIHZkc20ga3Zt ICAxNk0gMjMuIMO6bm8gMjIuNTEgaW5ib3gKLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gMjMu IMO6bm8gMjMuMTIgbGVhc2VzCi1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgOTk4IDI1LiDDum5vIDAw LjM1IG1ldGFkYXRhCi1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTYgb3V0 Ym94Cgovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0 LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQ6CnRvdGFsIDEsMU0K LXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuNDQgaWRzCi1ydy1ydy0tLS0g MSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTcgaW5ib3gKLXJ3LXJ3LS0tLSAxIHZkc20ga3Zt IDIsME0gIDcuIGxpcyAwMC4xOCBsZWFzZXMKLXJ3LXItLXItLSAxIHZkc20ga3ZtICAzMzMgIDcu IGxpcyAwMC4xOCBtZXRhZGF0YQotcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAw LjE3IG91dGJveAoKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJL Vk0xMi1QMS80MmQ3MTBhOS1iODQ0LTQzZGMtYmU0MS03NzAwMmQxY2Q1NTMvZG9tX21kOgp0b3Rh bCAxLDFNCi1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjMyIGlkcwotcnct cnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDIyLjE4IGluYm94Ci1ydy1ydy0tLS0gMSB2 ZHNtIGt2bSAyLDBNICA3LiBsaXMgMjIuMTggbGVhc2VzCi1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAg MzMzICA3LiBsaXMgMjIuMTggbWV0YWRhdGEKLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcu IGxpcyAyMi4xOCBvdXRib3gKCi9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxo b3N0Ol8yS1ZNMTItUDIvZmY3MWI0N2ItMGY3Mi00NTI4LTliZmUtYzNkYTg4OGU0N2YwL2RvbV9t ZDoKdG90YWwgMywwTQotcnctcnctci0tIDEgdmRzbSBrdm0gMSwwTSAgMS4gYsWZZSAyMS4yOCBp ZHMKLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gMjUuIMO6bm8gMDAuNDIgaW5ib3gKLXJ3LXJ3 LS0tLSAxIHZkc20ga3ZtIDIsME0gMjUuIMO6bm8gMDAuNDQgbGVhc2VzCi1ydy1yLS1yLS0gMSB2 ZHNtIGt2bSAgOTk3IDI0LiDDum5vIDAyLjQ2IG1ldGFkYXRhCi1ydy1ydy0tLS0gMSB2ZHNtIGt2 bSAgMTZNIDI1LiDDum5vIDAwLjQ0IG91dGJveAoKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1QMy9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRl Y2I0YjQvZG9tX21kOgp0b3RhbCAyLDFNCi1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDD um5vIDA3LjM0IGlkcwotcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyMy4gw7pubyAyMi4zNSBp bmJveAotcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4gw7pubyAyMi4zOCBsZWFzZXMKLXJ3 LXItLXItLSAxIHZkc20ga3ZtIDEsMUsgMjQuIMO6bm8gMTkuMDcgbWV0YWRhdGEKLXJ3LXJ3LS0t LSAxIHZkc20ga3ZtICAxNk0gMjMuIMO6bm8gMjIuMjcgb3V0Ym94Cgovcmhldi9kYXRhLWNlbnRl ci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyX19QNC8zMDBlOWFjOC0zYzJmLTQ3MDMt OWJiMS0xZGYyMTMwYzdjOTcvZG9tX21kOgp0b3RhbCAzLDBNCi1ydy1ydy1yLS0gMSB2ZHNtIGt2 bSAxLDBNICAxLiBixZllIDIxLjI4IGlkcwotcnctcnctci0tIDEgdmRzbSBrdm0gIDE2TSAgNi4g bGlzIDIzLjUwIGluYm94Ci1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAyLDBNICA2LiBsaXMgMjMuNTEg bGVhc2VzCi1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgNzM0ICA3LiBsaXMgMDIuMTMgbWV0YWRhdGEK LXJ3LXJ3LXItLSAxIHZkc20ga3ZtICAxNk0gIDYuIGxpcyAxNi41NSBvdXRib3gKCi9yaGV2L2Rh dGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDUvMWNhNTZiNDUtNzAx ZS00YzIyLTlmNTktM2FlYmVhNGQ4NDc3L2RvbV9tZDoKdG90YWwgMSwxTQotcnctcnctci0tIDEg dmRzbSBrdm0gICAgMCAyNC4gw7pubyAwNy4zNSBpZHMKLXJ3LXJ3LXItLSAxIHZkc20ga3ZtICAx Nk0gMjQuIMO6bm8gMDEuMDYgaW5ib3gKLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gMjQuIMO6 bm8gMDIuNDQgbGVhc2VzCi1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgOTk4IDI0LiDDum5vIDE5LjA3 IG1ldGFkYXRhCi1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMjIuMjAgb3V0Ym94 CgogIFRoZSBpZHMgZmlsZXMgd2FzIGdlbmVyYXRlZCBieSAidG91Y2giIGNvbW1hbmQgYWZ0ZXIg ZGVsZXRpbmcgdGhlbSBkdWUgCiJzYW5sb2NrIGxvY2tpbmcgaGFuZyIgIGdsdXN0ZXIgY3Jhc2gg JiByZWJvb3QKSSBleHBlY3RlZCB0aGF0IHRoZXkgd2lsbCBiZSBmaWxsZWQgYXV0b21hdGljYWx5 IGFmdGVyIGdsdXN0ZXIgcmVib290ICggCnRoZSAgc2hhZG93IGNvcHkgZnJvbSAgICIuZ2x1c3Rl ciAiICAgZGlyZWN0b3J5ICB3YXMgZGVsZXRlZCAmIGNyZWF0ZWQgCmVtcHR5ICB0b28gKQoKT0ss IGl0IGxvb2tzIHRoYXQgc2FubG9jayAgY2FuJ3Qgd29yayB3aXRoIGVtcHR5IGZpbGUgb3IgcmV3 cml0ZSB0aGVtIC4KQW0gSSByaWdodCA/PwoKVGhlIGxhc3QgcG9pbnQgLSBhYm91dCAiaWRzIiB3 b3JrYXJvdW5kIC0gdGhpcyBpcyBvZmZsaW5lIHZlcnNpb24gPSBWTXMgCmhhdmUgdG8gYmUgbW92 ZWQgb3V0IGZyb20gZm9yIGNvbnRpbnVhbCBydW5uaW5nIHdpdGggbWFpbnRlbmFuY2Ugdm9sdW1l IG1vZGUKQnV0IHRoaXMgaXMgbm90IGFjY2VwdGFibGUgaW4gY3VycmVudCBzaXR1YXRpb24sIHNv IHRoZSBxdWVzdGlvbiBhZ2FpbiwgIAppcyBpdCBzYWZlIHRvIGRvIGl0IG9ubGluZSA/PyAgKCBZ RVMgLyBOTyApCgpyZWdzLgpQYXZlbAoKCk9uIDEuMy4yMDE2IDE4OjM4LCBOaXIgU29mZmVyIHdy b3RlOgo+IE9uIFR1ZSwgTWFyIDEsIDIwMTYgYXQgNTowNyBQTSwgcGFmMUBlbWFpbC5jeiA8bWFp bHRvOnBhZjFAZW1haWwuY3o+IAo+IDxwYWYxQGVtYWlsLmN6IDxtYWlsdG86cGFmMUBlbWFpbC5j ej4+IHdyb3RlOgo+Cj4gICAgIEhlbGxvLCAgY2FuIGFueWJvZHkgIGV4cGxhaW4gdGhpcyBlcnJv ciBuby4xMyAoIG9wZW4gZmlsZSApIGluCj4gICAgIHNhbmxvY2subG9nIC4KPgo+Cj4gVGhpcyBp cyBFQUNDRVMKPgo+IENhbiB5b3Ugc2hhcmUgdGhlIG91dG9wdXQgb2Y6Cj4KPiAgICAgbHMgLWxo IC9yaGV2L2RhdGEtY2VudGVyL21udC88c2VydmVyPjo8X3BhdGg+LzxzZF91dWlkPi9kb21fbWQK Pgo+Cj4gICAgIFRoZSBzaXplIG9mICAiaWRzIiBmaWxlIGlzIHplcm8gKDApCj4KPgo+IFRoaXMg aXMgaG93IHdlIGNyZWF0ZSB0aGUgaWRzIGZpbGUgd2hlbiBpbml0aWFsaXppbmcgaXQuCj4KPiBC dXQgdGhlbiB3ZSB1c2Ugc2FubG9jayB0byBpbml0aWFsaXplIHRoZSBpZHMgZmlsZSwgYW5kIGl0 IHNob3VsZCBiZSAKPiAxTWlCIGFmdGVyIHRoYXQuCj4KPiBJcyB0aGlzIGlkcyBmaWxlcyBjcmVh dGVkIGJ5IHZkc20sIG9yIG9uZSB5b3UgY3JlYXRlZCB5b3Vyc2VsZj8KPgo+ICAgICAyMDE2LTAy LTI4IDAzOjI1OjQ2KzAxMDAgMjY5NjI2IFsxOTUxXTogb3BlbiBlcnJvciAtMTMKPiAgICAgL3Jo ZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5 Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkcwo+ICAgICAyMDE2LTAyLTI4 IDAzOjI1OjQ2KzAxMDAgMjY5NjI2IFsxOTUxXTogczE4Nzk4NSBvcGVuX2Rpc2sKPiAgICAgL3Jo ZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5 Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkcwo+ICAgICBlcnJvciAtMTMK PiAgICAgMjAxNi0wMi0yOCAwMzoyNTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBzMTg3OTkyIGxv Y2tzcGFjZQo+ICAgICA3ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQ6MTovcmhl di9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3 LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzOjAKPgo+ICAgICBJZiB0aGUg bWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXplLCBjYW4gSSByZWdlbmVyYXRlIAo+ ICAgICB0aGlzIGZpbGUgb25saW5lIHNlY3VyZWx5ICwgd2l0aCBubyBWTSBkZXBlbmRlbmNlICA/ Pz8/Cj4KPgo+IFllcywgSSB0aGluayBJIGFscmVhZHkgcmVmZXJyZWQgdG8gdGhlIGluc3RydWN0 aW9ucyBob3cgdG8gZG8gdGhhdCBpbiAKPiBhIHByZXZpb3VzIG1haWwuCj4KPgo+Cj4gICAgIGRp c3QgPSBSSEVMIC0gNyAtIDIuMTUxMQo+ICAgICBrZXJuZWwgPSAzLjEwLjAgLSAzMjcuMTAuMS5l bDcueDg2XzY0Cj4gICAgIEtWTSA9IDIuMy4wIC0gMjkuMS5lbDcKPiAgICAgbGlidmlydCA9IGxp YnZpcnQtMS4yLjE3LTEzLmVsN18yLjMKPiAgICAgdmRzbSA9IHZkc20tNC4xNi4zMC0wLmVsNwo+ ICAgICBHbHVzdGVyRlMgPSBnbHVzdGVyZnMtMy43LjgtMS5lbDcKPgo+Cj4gICAgIHJlZ3MuCj4g ICAgIFBhdmVsCj4KPiAgICAgX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KPiAgICAgVXNlcnMgbWFpbGluZyBsaXN0Cj4gICAgIFVzZXJzQG92aXJ0Lm9yZyA8 bWFpbHRvOlVzZXJzQG92aXJ0Lm9yZz4KPiAgICAgaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWls bWFuL2xpc3RpbmZvL3VzZXJzCj4KPgoKCi0tLS0tLS0tLS0tLS0tMDgwMTA4MDcwNTAzMDcwNzAz MDUwMTAwCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04CkNvbnRlbnQtVHJh bnNmZXItRW5jb2Rpbmc6IDhiaXQKCjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04IiBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogIDwv aGVhZD4KICA8Ym9keSB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIEhJLDxi cj4KICAgIHJlcXVlc3RlZCBvdXRwdXQ6PGJyPgogICAgPGJyPgogICAgIyBscyAtbGggL3JoZXYv ZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6Ki8qL2RvbV9tZCA8YnI+CiAgICDC oCA8YnI+Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTIt QkNLLzBmY2FkODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21fbWQ6PGJyPgogICAg dG90YWwgMiwxTTxicj4KICAgIC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAxLDBNwqAgMS7CoGLFmWUg MjEuMjggaWRzPGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtwqAgMTZNwqAgNy7CoGxpcyAy Mi4xNiBpbmJveDxicj4KICAgIC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNwqAgNy7CoGxpcyAy Mi4xNyBsZWFzZXM8YnI+CiAgICAtcnctci0tci0tIDEgdmRzbSBrdm3CoCAzMzXCoCA3LsKgbGlz IDIyLjE3IG1ldGFkYXRhPGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtwqAgMTZNwqAgNy7C oGxpcyAyMi4xNiBvdXRib3g8YnI+CiAgICA8YnI+Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVz dGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDEvNTUzZDliOTItZTRhMC00MDQyLWE1NzktNGNhYmVi NTVkZWQ0L2RvbV9tZDo8YnI+CiAgICB0b3RhbCAxLDFNPGJyPgogICAgLXJ3LXItLXItLSAxIHZk c20ga3ZtwqDCoMKgIDAgMjQuwqDDum5vIDA3LjQxIGlkczxicj4KICAgIC1ydy1ydy0tLS0gMSB2 ZHNtIGt2bcKgIDE2TcKgIDcuwqBsaXMgMDAuMTQgaW5ib3g8YnI+CiAgICAtcnctcnctLS0tIDEg dmRzbSBrdm0gMiwwTcKgIDcuwqBsaXMgMDMuNTYgbGVhc2VzPGJyPgogICAgLXJ3LXItLXItLSAx IHZkc20ga3ZtwqAgMzMzwqAgNy7CoGxpcyAwMy41NiBtZXRhZGF0YTxicj4KICAgIC1ydy1ydy0t LS0gMSB2ZHNtIGt2bcKgIDE2TcKgIDcuwqBsaXMgMDAuMTQgb3V0Ym94PGJyPgogICAgPGJyPgov cmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVAyLzg4YWRi ZDQ5LTYyZDYtNDViMS05OTkyLWIwNDQ2NGEwNDExMi9kb21fbWQ6PGJyPgogICAgdG90YWwgMSwx TTxicj4KICAgIC1ydy1yLS1yLS0gMSB2ZHNtIGt2bcKgwqDCoCAwIDI0LsKgw7pubyAwNy40MyBp ZHM8YnI+CiAgICAtcnctcnctLS0tIDEgdmRzbSBrdm3CoCAxNk3CoCA3LsKgbGlzIDAwLjE1IGlu Ym94PGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME3CoCA3LsKgbGlzIDIyLjE0IGxl YXNlczxicj4KICAgIC1ydy1yLS1yLS0gMSB2ZHNtIGt2bcKgIDMzM8KgIDcuwqBsaXMgMjIuMTQg bWV0YWRhdGE8YnI+CiAgICAtcnctcnctLS0tIDEgdmRzbSBrdm3CoCAxNk3CoCA3LsKgbGlzIDAw LjE1IG91dGJveDxicj4KICAgIDxicj4KL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9s b2NhbGhvc3Q6XzFLVk0xMi1QMy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcv ZG9tX21kOjxicj4KICAgIHRvdGFsIDEsMU08YnI+CiAgICAtcnctci0tci0tIDEgdmRzbSBrdm3C oMKgwqAgMCAyNC7CoMO6bm8gMDcuNDMgaWRzPGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZkc20ga3Zt wqAgMTZNIDIzLsKgw7pubyAyMi41MSBpbmJveDxicj4KICAgIC1ydy1ydy0tLS0gMSB2ZHNtIGt2 bSAyLDBNIDIzLsKgw7pubyAyMy4xMiBsZWFzZXM8YnI+CiAgICAtcnctci0tci0tIDEgdmRzbSBr dm3CoCA5OTggMjUuwqDDum5vIDAwLjM1IG1ldGFkYXRhPGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZk c20ga3ZtwqAgMTZNwqAgNy7CoGxpcyAwMC4xNiBvdXRib3g8YnI+CiAgICA8YnI+Ci9yaGV2L2Rh dGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5 OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZDo8YnI+CiAgICB0b3RhbCAxLDFNPGJyPgog ICAgLXJ3LXItLXItLSAxIHZkc20ga3ZtwqDCoMKgIDAgMjQuwqDDum5vIDA3LjQ0IGlkczxicj4K ICAgIC1ydy1ydy0tLS0gMSB2ZHNtIGt2bcKgIDE2TcKgIDcuwqBsaXMgMDAuMTcgaW5ib3g8YnI+ CiAgICAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTcKgIDcuwqBsaXMgMDAuMTggbGVhc2VzPGJy PgogICAgLXJ3LXItLXItLSAxIHZkc20ga3ZtwqAgMzMzwqAgNy7CoGxpcyAwMC4xOCBtZXRhZGF0 YTxicj4KICAgIC1ydy1ydy0tLS0gMSB2ZHNtIGt2bcKgIDE2TcKgIDcuwqBsaXMgMDAuMTcgb3V0 Ym94PGJyPgogICAgPGJyPgovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9z dDpfMktWTTEyLVAxLzQyZDcxMGE5LWI4NDQtNDNkYy1iZTQxLTc3MDAyZDFjZDU1My9kb21fbWQ6 PGJyPgogICAgdG90YWwgMSwxTTxicj4KICAgIC1ydy1ydy1yLS0gMSB2ZHNtIGt2bcKgwqDCoCAw IDI0LsKgw7pubyAwNy4zMiBpZHM8YnI+CiAgICAtcnctcnctLS0tIDEgdmRzbSBrdm3CoCAxNk3C oCA3LsKgbGlzIDIyLjE4IGluYm94PGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME3C oCA3LsKgbGlzIDIyLjE4IGxlYXNlczxicj4KICAgIC1ydy1yLS1yLS0gMSB2ZHNtIGt2bcKgIDMz M8KgIDcuwqBsaXMgMjIuMTggbWV0YWRhdGE8YnI+CiAgICAtcnctcnctLS0tIDEgdmRzbSBrdm3C oCAxNk3CoCA3LsKgbGlzIDIyLjE4IG91dGJveDxicj4KICAgIDxicj4KL3JoZXYvZGF0YS1jZW50 ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1QMi9mZjcxYjQ3Yi0wZjcyLTQ1Mjgt OWJmZS1jM2RhODg4ZTQ3ZjAvZG9tX21kOjxicj4KICAgIHRvdGFsIDMsME08YnI+CiAgICAtcnct cnctci0tIDEgdmRzbSBrdm0gMSwwTcKgIDEuwqBixZllIDIxLjI4IGlkczxicj4KICAgIC1ydy1y dy0tLS0gMSB2ZHNtIGt2bcKgIDE2TSAyNS7CoMO6bm8gMDAuNDIgaW5ib3g8YnI+CiAgICAtcnct cnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyNS7CoMO6bm8gMDAuNDQgbGVhc2VzPGJyPgogICAgLXJ3 LXItLXItLSAxIHZkc20ga3ZtwqAgOTk3IDI0LsKgw7pubyAwMi40NiBtZXRhZGF0YTxicj4KICAg IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bcKgIDE2TSAyNS7CoMO6bm8gMDAuNDQgb3V0Ym94PGJyPgog ICAgPGJyPgovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEy LVAzL2VmMDEwZDA4LWFlZDEtNDFjNC1iYTlhLWU2ZDliZGVjYjRiNC9kb21fbWQ6PGJyPgogICAg dG90YWwgMiwxTTxicj4KICAgIC1ydy1yLS1yLS0gMSB2ZHNtIGt2bcKgwqDCoCAwIDI0LsKgw7pu byAwNy4zNCBpZHM8YnI+CiAgICAtcnctcnctLS0tIDEgdmRzbSBrdm3CoCAxNk0gMjMuwqDDum5v IDIyLjM1IGluYm94PGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gMjMuwqDDum5v IDIyLjM4IGxlYXNlczxicj4KICAgIC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAxLDFLIDI0LsKgw7pu byAxOS4wNyBtZXRhZGF0YTxicj4KICAgIC1ydy1ydy0tLS0gMSB2ZHNtIGt2bcKgIDE2TSAyMy7C oMO6bm8gMjIuMjcgb3V0Ym94PGJyPgogICAgPGJyPgovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1 c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyX19QNC8zMDBlOWFjOC0zYzJmLTQ3MDMtOWJiMS0xZGYy MTMwYzdjOTcvZG9tX21kOjxicj4KICAgIHRvdGFsIDMsME08YnI+CiAgICAtcnctcnctci0tIDEg dmRzbSBrdm0gMSwwTcKgIDEuwqBixZllIDIxLjI4IGlkczxicj4KICAgIC1ydy1ydy1yLS0gMSB2 ZHNtIGt2bcKgIDE2TcKgIDYuwqBsaXMgMjMuNTAgaW5ib3g8YnI+CiAgICAtcnctcnctci0tIDEg dmRzbSBrdm0gMiwwTcKgIDYuwqBsaXMgMjMuNTEgbGVhc2VzPGJyPgogICAgLXJ3LXJ3LXItLSAx IHZkc20ga3ZtwqAgNzM0wqAgNy7CoGxpcyAwMi4xMyBtZXRhZGF0YTxicj4KICAgIC1ydy1ydy1y LS0gMSB2ZHNtIGt2bcKgIDE2TcKgIDYuwqBsaXMgMTYuNTUgb3V0Ym94PGJyPgogICAgPGJyPgov cmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyLVA1LzFjYTU2 YjQ1LTcwMWUtNGMyMi05ZjU5LTNhZWJlYTRkODQ3Ny9kb21fbWQ6PGJyPgogICAgdG90YWwgMSwx TTxicj4KICAgIC1ydy1ydy1yLS0gMSB2ZHNtIGt2bcKgwqDCoCAwIDI0LsKgw7pubyAwNy4zNSBp ZHM8YnI+CiAgICAtcnctcnctci0tIDEgdmRzbSBrdm3CoCAxNk0gMjQuwqDDum5vIDAxLjA2IGlu Ym94PGJyPgogICAgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gMjQuwqDDum5vIDAyLjQ0IGxl YXNlczxicj4KICAgIC1ydy1yLS1yLS0gMSB2ZHNtIGt2bcKgIDk5OCAyNC7CoMO6bm8gMTkuMDcg bWV0YWRhdGE8YnI+CiAgICAtcnctcnctci0tIDEgdmRzbSBrdm3CoCAxNk3CoCA3LsKgbGlzIDIy LjIwIG91dGJveDxicj4KICAgIDxicj4KICAgIMKgVGhlIGlkcyBmaWxlcyB3YXMgZ2VuZXJhdGVk IGJ5ICJ0b3VjaCIgY29tbWFuZCBhZnRlciBkZWxldGluZyB0aGVtCiAgICBkdWUgInNhbmxvY2sg bG9ja2luZyBoYW5nIsKgIGdsdXN0ZXIgY3Jhc2ggJmFtcDsgcmVib290PGJyPgogICAgSSBleHBl Y3RlZCB0aGF0IHRoZXkgd2lsbCBiZSBmaWxsZWQgYXV0b21hdGljYWx5IGFmdGVyIGdsdXN0ZXIK ICAgIHJlYm9vdCAoIHRoZcKgIHNoYWRvdyBjb3B5IGZyb23CoMKgICIuZ2x1c3RlciAiIMKgIGRp cmVjdG9yecKgIHdhcwogICAgZGVsZXRlZCAmYW1wOyBjcmVhdGVkIGVtcHR5wqAgdG9vICk8YnI+ CiAgICA8YnI+CiAgICBPSywgaXQgbG9va3MgdGhhdCBzYW5sb2NrwqAgY2FuJ3Qgd29yayB3aXRo IGVtcHR5IGZpbGUgb3IgcmV3cml0ZQogICAgdGhlbSAuPGJyPgogICAgQW0gSSByaWdodCA/Pzxi cj4KICAgIDxicj4KICAgIFRoZSBsYXN0IHBvaW50IC0gYWJvdXQgImlkcyIgd29ya2Fyb3VuZCAt IHRoaXMgaXMgb2ZmbGluZSB2ZXJzaW9uID0KICAgIFZNcyBoYXZlIHRvIGJlIG1vdmVkIG91dCBm cm9tIGZvciBjb250aW51YWwgcnVubmluZyB3aXRoIG1haW50ZW5hbmNlCiAgICB2b2x1bWUgbW9k ZTxicj4KICAgIEJ1dCB0aGlzIGlzIG5vdCBhY2NlcHRhYmxlIGluIGN1cnJlbnQgc2l0dWF0aW9u LCBzbyB0aGUgcXVlc3Rpb24KICAgIGFnYWluLMKgIGlzIGl0IHNhZmUgdG8gZG8gaXQgb25saW5l ID8/wqAgKCBZRVMgLyBOTyApPGJyPgogICAgPGJyPgogICAgcmVncy48YnI+CiAgICBQYXZlbDxi cj4KICAgIDxicj4KICAgIDxicj4KICAgIDxkaXYgY2xhc3M9Im1vei1jaXRlLXByZWZpeCI+T24g MS4zLjIwMTYgMTg6MzgsIE5pciBTb2ZmZXIgd3JvdGU6PGJyPgogICAgPC9kaXY+CiAgICA8Ymxv Y2txdW90ZQpjaXRlPSJtaWQ6Q0FNUmJ5eXVzLXJTMFJ0T1d6ZVB3cldwN3hvT0g4eVJnVDAtX3d0 Y3VNOFdpV2RPNFZBQG1haWwuZ21haWwuY29tIgogICAgICB0eXBlPSJjaXRlIj4KICAgICAgPGRp diBkaXI9Imx0ciI+CiAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxfZXh0cmEiPgogICAgICAgICAg PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPk9uIFR1ZSwgTWFyIDEsIDIwMTYgYXQgNTowNyBQTSwg PGEKICAgICAgICAgICAgICBtb3otZG8tbm90LXNlbmQ9InRydWUiIGhyZWY9Im1haWx0bzpwYWYx QGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPgogICAgICAgICAgICA8c3BhbiBkaXI9Imx0ciI+ Jmx0OzxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAgICAgICAgICAgIGhyZWY9Im1haWx0 bzpwYWYxQGVtYWlsLmN6IiB0YXJnZXQ9Il9ibGFuayI+cGFmMUBlbWFpbC5jejwvYT4mZ3Q7PC9z cGFuPgogICAgICAgICAgICB3cm90ZTo8YnI+CiAgICAgICAgICAgIDxibG9ja3F1b3RlIGNsYXNz PSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweAowLjhleDtib3JkZXItbGVm dC13aWR0aDoxcHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVm dC1zdHlsZTpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4KICAgICAgICAgICAgICA8ZGl2IHRleHQ9 IiMwMDAwNjYiIGJnY29sb3I9IiNGRkZGRkYiPiBIZWxsbyzCoCBjYW4KICAgICAgICAgICAgICAg IGFueWJvZHnCoCBleHBsYWluIHRoaXMgZXJyb3Igbm8uMTMgKCBvcGVuIGZpbGUgKSBpbgogICAg ICAgICAgICAgICAgc2FubG9jay5sb2cgLjxicj4KICAgICAgICAgICAgICA8L2Rpdj4KICAgICAg ICAgICAgPC9ibG9ja3F1b3RlPgogICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgPC9k aXY+CiAgICAgICAgICAgIDxkaXY+VGhpcyBpcyBFQUNDRVM8L2Rpdj4KICAgICAgICAgICAgPGRp dj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2PkNhbiB5b3Ugc2hhcmUg dGhlIG91dG9wdXQgb2Y6PC9kaXY+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICA8 L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoCDCoCBscyAtbGgKL3JoZXYvZGF0YS1jZW50ZXIvbW50 LyZsdDtzZXJ2ZXImZ3Q7OiZsdDtfcGF0aCZndDsvJmx0O3NkX3V1aWQmZ3Q7L2RvbV9tZDwvZGl2 PgogICAgICAgICAgICA8ZGl2PsKgPC9kaXY+CiAgICAgICAgICAgIDxibG9ja3F1b3RlIGNsYXNz PSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweAowLjhleDtib3JkZXItbGVm dC13aWR0aDoxcHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVm dC1zdHlsZTpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4KICAgICAgICAgICAgICA8ZGl2IHRleHQ9 IiMwMDAwNjYiIGJnY29sb3I9IiNGRkZGRkYiPiA8YnI+CiAgICAgICAgICAgICAgICBUaGUgc2l6 ZSBvZsKgICJpZHMiIGZpbGUgaXMgemVybyAoMCk8YnI+CiAgICAgICAgICAgICAgPC9kaXY+CiAg ICAgICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAg IDwvZGl2PgogICAgICAgICAgICA8ZGl2PlRoaXMgaXMgaG93IHdlIGNyZWF0ZSB0aGUgaWRzIGZp bGUgd2hlbiBpbml0aWFsaXppbmcKICAgICAgICAgICAgICBpdC48L2Rpdj4KICAgICAgICAgICAg PGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2PkJ1dCB0aGVuIHdl IHVzZSBzYW5sb2NrIHRvIGluaXRpYWxpemUgdGhlIGlkcyBmaWxlLCBhbmQKICAgICAgICAgICAg ICBpdCBzaG91bGQgYmUgMU1pQiBhZnRlciB0aGF0LjwvZGl2PgogICAgICAgICAgICA8ZGl2Pjxi cj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+SXMgdGhpcyBpZHMgZmlsZXMg Y3JlYXRlZCBieSB2ZHNtLCBvciBvbmUgeW91IGNyZWF0ZWQKICAgICAgICAgICAgICB5b3Vyc2Vs Zj88L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoDwvZGl2PgogICAgICAgICAgICA8YmxvY2txdW90 ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHgKMC44ZXg7Ym9y ZGVyLWxlZnQtd2lkdGg6MXB4O2JvcmRlci1sZWZ0LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9y ZGVyLWxlZnQtc3R5bGU6c29saWQ7cGFkZGluZy1sZWZ0OjFleCI+CiAgICAgICAgICAgICAgPGRp diB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZGRkZGIj4gMjAxNi0wMi0yOAogICAgICAgICAg ICAgICAgMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBvcGVuIGVycm9yIC0xMwovcmhldi9k YXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMx OTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzPGJyPgogICAgICAgICAgICAgICAg MjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IHMxODc5ODUKICAgICAgICAg ICAgICAgIG9wZW5fZGlzawogICAgICAgICAgICAgICAgL3JoZXYvZGF0YS1jZW50ZXIvbW50L2ds dXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJk NDQzMjcxMjQvZG9tX21kL2lkcwoKICAgICAgICAgICAgICAgIGVycm9yIC0xMzxicj4KICAgICAg ICAgICAgICAgIDIwMTYtMDItMjggMDM6MjU6NTYrMDEwMCAyNjk2MzYgWzExMzA0XTogczE4Nzk5 MgogICAgICAgICAgICAgICAgbG9ja3NwYWNlCjdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0 NDMyNzEyNDoxOi9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZN MTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM6MDxi cj4KICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgIElmIHRoZSBtYWluIHByb2Js ZW0gaXMgYWJvdXQgemVybyBmaWxlIHNpemUsIGNhbiBJCiAgICAgICAgICAgICAgICByZWdlbmVy YXRlwqAgdGhpcyBmaWxlIG9ubGluZSBzZWN1cmVseSAsIHdpdGggbm8gVk0KICAgICAgICAgICAg ICAgIGRlcGVuZGVuY2XCoCA/Pz8/PGJyPgogICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAg ICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICA8L2Rpdj4K ICAgICAgICAgICAgPGRpdj5ZZXMsIEkgdGhpbmsgSSBhbHJlYWR5IHJlZmVycmVkIHRvIHRoZSBp bnN0cnVjdGlvbnMgaG93CiAgICAgICAgICAgICAgdG8gZG8gdGhhdCBpbiBhIHByZXZpb3VzIG1h aWwuPGJyPgogICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAg ICAgIDwvZGl2PgogICAgICAgICAgICA8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0 eWxlPSJtYXJnaW46MHB4IDBweCAwcHgKMC44ZXg7Ym9yZGVyLWxlZnQtd2lkdGg6MXB4O2JvcmRl ci1sZWZ0LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9yZGVyLWxlZnQtc3R5bGU6c29saWQ7cGFk ZGluZy1sZWZ0OjFleCI+CiAgICAgICAgICAgICAgPGRpdiB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9y PSIjRkZGRkZGIj4gPGJyPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgZGlz dCA9IFJIRUwgLSA3IC0gMi4xNTExPGJyPgogICAgICAgICAgICAgICAga2VybmVsID0gMy4xMC4w IC0gMzI3LjEwLjEuZWw3Lng4Nl82NDxicj4KICAgICAgICAgICAgICAgIEtWTSA9IDIuMy4wIC0g MjkuMS5lbDc8YnI+CiAgICAgICAgICAgICAgICBsaWJ2aXJ0ID0gbGlidmlydC0xLjIuMTctMTMu ZWw3XzIuMzxicj4KICAgICAgICAgICAgICAgIHZkc20gPSB2ZHNtLTQuMTYuMzAtMC5lbDc8YnI+ CiAgICAgICAgICAgICAgICBHbHVzdGVyRlMgPSBnbHVzdGVyZnMtMy43LjgtMS5lbDc8YnI+CiAg ICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICBy ZWdzLjxicj4KICAgICAgICAgICAgICAgIFBhdmVsPGJyPgogICAgICAgICAgICAgIDwvZGl2Pgog ICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICBfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXzxicj4KICAgICAgICAgICAgICBVc2VycyBtYWlsaW5nIGxp c3Q8YnI+CiAgICAgICAgICAgICAgPGEgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIiBocmVmPSJtYWls dG86VXNlcnNAb3ZpcnQub3JnIj5Vc2Vyc0BvdmlydC5vcmc8L2E+PGJyPgogICAgICAgICAgICAg IDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAgICAgICAgICAgIGhyZWY9Imh0dHA6Ly9s aXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyIKICAgICAgICAgICAgICAgIHJl bD0ibm9yZWZlcnJlciIgdGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFp bG1hbi9saXN0aW5mby91c2VyczwvYT48YnI+CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAg ICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICA8L2Rpdj4KICAgICAgICAgIDxicj4KICAgICAgICA8 L2Rpdj4KICAgICAgPC9kaXY+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+CiAgPC9ib2R5Pgo8 L2h0bWw+CgotLS0tLS0tLS0tLS0tLTA4MDEwODA3MDUwMzA3MDcwMzA1MDEwMC0tCg== --===============4946150822962749594==-- From nsoffer at redhat.com Tue Mar 1 17:15:19 2016 Content-Type: multipart/mixed; boundary="===============2299145876614063024==" MIME-Version: 1.0 From: Nir Soffer To: users at ovirt.org Subject: Re: [ovirt-users] open error -13 = sanlock Date: Wed, 02 Mar 2016 00:15:17 +0200 Message-ID: In-Reply-To: 56D600C3.8010405@email.cz --===============2299145876614063024== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz wrote: > > HI, > requested output: > > # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-be= f3-0bc0b7a99fb7/dom_md: > total 2,1M > -rw-rw---- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- good > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 inbox > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.17 leases > -rw-r--r-- 1 vdsm kvm 335 7. lis 22.17 metadata > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a57= 9-4cabeb55ded4/dom_md: > total 1,1M > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.41 ids <-- bad (sanlock cannot write, other can read) > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 inbox > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 03.56 leases > -rw-r--r-- 1 vdsm kvm 333 7. lis 03.56 metadata > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-999= 2-b04464a04112/dom_md: > total 1,1M > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock cannot write, other can read) > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 inbox > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.14 leases > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.14 metadata > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-ab4= 6-084f3d70b147/dom_md: > total 1,1M > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock cannot write, other can read) > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.51 inbox > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases > -rw-r--r-- 1 vdsm kvm 998 25. =C3=BAno 00.35 metadata > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.16 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md: > total 1,1M > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.44 ids <-- bad (sanlock cannot write, other can read) > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 inbox > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 00.18 leases > -rw-r--r-- 1 vdsm kvm 333 7. lis 00.18 metadata > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-be4= 1-77002d1cd553/dom_md: > total 1,1M > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.32 ids <-- bad (other c= an read) > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 inbox > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.18 leases > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.18 metadata > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9bf= e-c3da888e47f0/dom_md: > total 3,0M > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other c= an read) > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.42 inbox > -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases > -rw-r--r-- 1 vdsm kvm 997 24. =C3=BAno 02.46 metadata > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.44 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-ba9= a-e6d9bdecb4b4/dom_md: > total 2,1M > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.34 ids <-- bad (sanlock cannot write, other can read) > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.35 inbox > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases > -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.27 outbox > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-9b= b1-1df2130c7c97/dom_md: > total 3,0M > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other c= an read) > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 23.50 inbox <-- bad (other can read) > -rw-rw-r-- 1 vdsm kvm 2,0M 6. lis 23.51 leases <-- bad (other can read) > -rw-rw-r-- 1 vdsm kvm 734 7. lis 02.13 metadata <-- bad (group can write, other can read) > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 16.55 outbox <-- bad (other can read) > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9f5= 9-3aebea4d8477/dom_md: > total 1,1M > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.35 ids <-- bad (other c= an read) > -rw-rw-r-- 1 vdsm kvm 16M 24. =C3=BAno 01.06 inbox > -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases > -rw-r--r-- 1 vdsm kvm 998 24. =C3=BAno 19.07 metadata > -rw-rw-r-- 1 vdsm kvm 16M 7. lis 22.20 outbox It should look like this: -rw-rw----. 1 vdsm kvm 1.0M Mar 1 23:36 ids -rw-rw----. 1 vdsm kvm 2.0M Mar 1 23:35 leases -rw-r--r--. 1 vdsm kvm 353 Mar 1 23:35 metadata -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 outbox -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 inbox This explains the EACCES error. You can start by fixing the permissions manually, you can do this online. > The ids files was generated by "touch" command after deleting them due "sanlock locking hang" gluster crash & reboot > I expected that they will be filled automaticaly after gluster reboot ( the shadow copy from ".gluster " directory was deleted & created empty too ) I don't know about gluster shadow copy, I would not play with gluster internals. Adding Sahina for advice. > OK, it looks that sanlock can't work with empty file or rewrite them . > Am I right ?? Yes, the files must be initialized before sanlock can use them. You can initialize the file like this: sanlock direct init -s :0:repair//dom_md/ids:0 Taken from http://lists.ovirt.org/pipermail/users/2016-February/038046.html > The last point - about "ids" workaround - this is offline version =3D VMs have to be moved out from for continual running with maintenance volume mode > But this is not acceptable in current situation, so the question again, is it safe to do it online ?? ( YES / NO ) The ids file is accessed only by sanlock. I guess that you don't have a running SPM on this DC, since sanlock fails to acquire a host id, so you are pretty safe to fix the permissions and initialize the ids files. I would do this: 1. Stop engine, so it will not try to start vdsm 2. Stop vdsm on all hosts, so they do not try to acquire a host id with sanlock This does not affect running vms 3. Fix the permissions on the ids file, via glusterfs mount 4. Initialize the ids files from one of the hosts, via the glusterfs mount This should fix the ids files on all replicas 5. Start vdsm on all hosts 6. Start engine Engine will connect to all hosts, hosts will connect to storage and try to acquire a host id. Then Engine will start the SPM on one of the hosts, and your DC should become up. David, Sahina, can you confirm that this procedure is safe? Nir > > regs. > Pavel > > > > On 1.3.2016 18:38, Nir Soffer wrote: > > On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz wrote: >> >> Hello, can anybody explain this error no.13 ( open file ) in sanlock.log . > > > This is EACCES > > Can you share the outoput of: > > ls -lh /rhev/data-center/mnt/:<_path>//dom_md > >> >> >> The size of "ids" file is zero (0) > > > This is how we create the ids file when initializing it. > > But then we use sanlock to initialize the ids file, and it should be 1MiB after that. > > Is this ids files created by vdsm, or one you created yourself? > >> >> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids >> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids error -13 >> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 >> >> If the main problem is about zero file size, can I regenerate this file online securely , with no VM dependence ???? > > > Yes, I think I already referred to the instructions how to do that in a previous mail. > >> >> >> dist =3D RHEL - 7 - 2.1511 >> kernel =3D 3.10.0 - 327.10.1.el7.x86_64 >> KVM =3D 2.3.0 - 29.1.el7 >> libvirt =3D libvirt-1.2.17-13.el7_2.3 >> vdsm =3D vdsm-4.16.30-0.el7 >> GlusterFS =3D glusterfs-3.7.8-1.el7 >> >> >> regs. >> Pavel >> >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> > > --===============2299145876614063024== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+T24gVHVlLCBNYXIgMSwgMjAxNiBhdCAxMDo1MSBQTSwgPGEgaHJlZj0i bWFpbHRvOnBhZjFAZW1haWwuY3oiPnBhZjFAZW1haWwuY3o8L2E+ICZsdDs8YSBocmVmPSJtYWls dG86cGFmMUBlbWFpbC5jeiI+cGFmMUBlbWFpbC5jejwvYT4mZ3Q7IHdyb3RlOjxicj4mZ3Q7PGJy PiZndDsgSEksPGJyPiZndDsgcmVxdWVzdGVkIG91dHB1dDo8YnI+Jmd0Ozxicj4mZ3Q7ICMgbHMg LWxoIC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0OiovKi9kb21fbWQ8 YnI+Jmd0OyDCoDxicj4mZ3Q7IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxo b3N0Ol8xS1ZNMTItQkNLLzBmY2FkODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21f bWQ6PGJyPiZndDsgdG90YWwgMiwxTTxicj4mZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAxLDBN IMKgMS4gYsWZZSAyMS4yOCBpZHMgwqAgwqAgwqAgwqAmbHQ7LS0gZ29vZDxicj4mZ3Q7IC1ydy1y dy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4xNiBpbmJveDxicj4mZ3Q7IC1ydy1y dy0tLS0gMSB2ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDIyLjE3IGxlYXNlczxicj4mZ3Q7IC1ydy1y LS1yLS0gMSB2ZHNtIGt2bSDCoDMzNSDCoDcuIGxpcyAyMi4xNyBtZXRhZGF0YTxicj4mZ3Q7IC1y dy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4xNiBvdXRib3g8YnI+Jmd0Ozxi cj4mZ3Q7IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTIt UDEvNTUzZDliOTItZTRhMC00MDQyLWE1NzktNGNhYmViNTVkZWQ0L2RvbV9tZDo8YnI+Jmd0OyB0 b3RhbCAxLDFNPGJyPiZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgIMKgMCAyNC4gw7pubyAw Ny40MSBpZHMgwqAgwqAgwqAgwqAmbHQ7LS0gYmFkIChzYW5sb2NrIGNhbm5vdCB3cml0ZSwgb3Ro ZXIgY2FuIHJlYWQpPGJyPiZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlz IDAwLjE0IGluYm94PGJyPiZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMg MDMuNTYgbGVhc2VzPGJyPiZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgMzMzIMKgNy4gbGlz IDAzLjU2IG1ldGFkYXRhPGJyPiZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4g bGlzIDAwLjE0IG91dGJveDxicj4mZ3Q7PGJyPiZndDsgL3JoZXYvZGF0YS1jZW50ZXIvbW50L2ds dXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QMi84OGFkYmQ0OS02MmQ2LTQ1YjEtOTk5Mi1iMDQ0 NjRhMDQxMTIvZG9tX21kOjxicj4mZ3Q7IHRvdGFsIDEsMU08YnI+Jmd0OyAtcnctci0tci0tIDEg dmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjQzIGlkcyDCoCDCoCDCoCDCoCZsdDstLSBiYWQg KHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+Jmd0OyAtcnctcnctLS0t IDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTUgaW5ib3g8YnI+Jmd0OyAtcnctcnctLS0t IDEgdmRzbSBrdm0gMiwwTSDCoDcuIGxpcyAyMi4xNCBsZWFzZXM8YnI+Jmd0OyAtcnctci0tci0t IDEgdmRzbSBrdm0gwqAzMzMgwqA3LiBsaXMgMjIuMTQgbWV0YWRhdGE8YnI+Jmd0OyAtcnctcnct LS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTUgb3V0Ym94PGJyPiZndDs8YnI+Jmd0 OyAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVAzLzNj MzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQ6PGJyPiZndDsgdG90YWwg MSwxTTxicj4mZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuNDMg aWRzIMKgIMKgIMKgIMKgJmx0Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVyIGNh biByZWFkKTxicj4mZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAyMy4gw7pubyAyMi41 MSBpbmJveDxicj4mZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDIzLiDDum5vIDIzLjEy IGxlYXNlczxicj4mZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDk5OCAyNS4gw7pubyAwMC4z NSBtZXRhZGF0YTxicj4mZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAw MC4xNiBvdXRib3g8YnI+Jmd0Ozxicj4mZ3Q7IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVy U0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3 MTI0L2RvbV9tZDo8YnI+Jmd0OyB0b3RhbCAxLDFNPGJyPiZndDsgLXJ3LXItLXItLSAxIHZkc20g a3ZtIMKgIMKgMCAyNC4gw7pubyAwNy40NCBpZHMgwqAgwqAgwqAgwqAmbHQ7LS0gYmFkIChzYW5s b2NrIGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpPGJyPiZndDsgLXJ3LXJ3LS0tLSAxIHZk c20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE3IGluYm94PGJyPiZndDsgLXJ3LXJ3LS0tLSAxIHZk c20ga3ZtIDIsME0gwqA3LiBsaXMgMDAuMTggbGVhc2VzPGJyPiZndDsgLXJ3LXItLXItLSAxIHZk c20ga3ZtIMKgMzMzIMKgNy4gbGlzIDAwLjE4IG1ldGFkYXRhPGJyPiZndDsgLXJ3LXJ3LS0tLSAx IHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE3IG91dGJveDxicj4mZ3Q7PGJyPiZndDsgL3Jo ZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1QMS80MmQ3MTBh OS1iODQ0LTQzZGMtYmU0MS03NzAwMmQxY2Q1NTMvZG9tX21kOjxicj4mZ3Q7IHRvdGFsIDEsMU08 YnI+Jmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjMyIGlkcyDC oCDCoCDCoCDCoCZsdDstLSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4mZ3Q7IC1ydy1ydy0tLS0g MSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4xOCBpbmJveDxicj4mZ3Q7IC1ydy1ydy0tLS0g MSB2ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDIyLjE4IGxlYXNlczxicj4mZ3Q7IC1ydy1yLS1yLS0g MSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAyMi4xOCBtZXRhZGF0YTxicj4mZ3Q7IC1ydy1ydy0t LS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4xOCBvdXRib3g8YnI+Jmd0Ozxicj4mZ3Q7 IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDIvZmY3 MWI0N2ItMGY3Mi00NTI4LTliZmUtYzNkYTg4OGU0N2YwL2RvbV9tZDo8YnI+Jmd0OyB0b3RhbCAz LDBNPGJyPiZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDEsME0gwqAxLiBixZllIDIxLjI4IGlk c8KgwqAgwqAgwqAgwqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVhZCk8YnI+Jmd0OyAtcnctcnct LS0tIDEgdmRzbSBrdm0gwqAxNk0gMjUuIMO6bm8gMDAuNDIgaW5ib3jCoDxicj4mZ3Q7IC1ydy1y dy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDI1LiDDum5vIDAwLjQ0IGxlYXNlczxicj4mZ3Q7IC1ydy1y LS1yLS0gMSB2ZHNtIGt2bSDCoDk5NyAyNC4gw7pubyAwMi40NiBtZXRhZGF0YTxicj4mZ3Q7IC1y dy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAyNS4gw7pubyAwMC40NCBvdXRib3g8YnI+Jmd0Ozxi cj4mZ3Q7IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTIt UDMvZWYwMTBkMDgtYWVkMS00MWM0LWJhOWEtZTZkOWJkZWNiNGI0L2RvbV9tZDo8YnI+Jmd0OyB0 b3RhbCAyLDFNPGJyPiZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgIMKgMCAyNC4gw7pubyAw Ny4zNCBpZHPCoMKgIMKgIMKgIMKgJmx0Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90 aGVyIGNhbiByZWFkKTxicj4mZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAyMy4gw7pu byAyMi4zNSBpbmJveDxicj4mZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDIzLiDDum5v IDIyLjM4IGxlYXNlczxicj4mZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAxLDFLIDI0LiDDum5v IDE5LjA3IG1ldGFkYXRhPGJyPiZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIDIzLiDD um5vIDIyLjI3IG91dGJveDxicj4mZ3Q7PGJyPiZndDsgL3JoZXYvZGF0YS1jZW50ZXIvbW50L2ds dXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMl9fUDQvMzAwZTlhYzgtM2MyZi00NzAzLTliYjEtMWRm MjEzMGM3Yzk3L2RvbV9tZDo8YnI+Jmd0OyB0b3RhbCAzLDBNPGJyPiZndDsgLXJ3LXJ3LXItLSAx IHZkc20ga3ZtIDEsME0gwqAxLiBixZllIDIxLjI4IGlkc8KgwqAgwqAgwqAgwqAmbHQ7LS0gYmFk IChvdGhlciBjYW4gcmVhZCk8YnI+Jmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAxNk0gwqA2 LiBsaXMgMjMuNTAgaW5ib3jCoMKgIMKgIMKgIMKgJmx0Oy0tIGJhZCAob3RoZXIgY2FuIHJlYWQp PGRpdj4mZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAyLDBNIMKgNi4gbGlzIDIzLjUxIGxlYXNl c8KgwqAgwqAgwqAgwqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVhZCk8YnI+Jmd0OyAtcnctcnct ci0tIDEgdmRzbSBrdm0gwqA3MzQgwqA3LiBsaXMgMDIuMTMgbWV0YWRhdGHCoMKgIMKgIMKgIMKg Jmx0Oy0tIGJhZCAoZ3JvdXAgY2FuIHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+Jmd0OyAtcnct cnctci0tIDEgdmRzbSBrdm0gwqAxNk0gwqA2LiBsaXMgMTYuNTUgb3V0Ym94wqDCoCDCoCDCoCDC oCZsdDstLSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4mZ3Q7PGJyPiZndDsgL3JoZXYvZGF0YS1j ZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1QNS8xY2E1NmI0NS03MDFlLTRj MjItOWY1OS0zYWViZWE0ZDg0NzcvZG9tX21kOjxicj4mZ3Q7IHRvdGFsIDEsMU08YnI+Jmd0OyAt cnctcnctci0tIDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjM1IGlkc8KgwqAgwqAgwqAg wqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVhZCk8YnI+Jmd0OyAtcnctcnctci0tIDEgdmRzbSBr dm0gwqAxNk0gMjQuIMO6bm8gMDEuMDYgaW5ib3g8YnI+Jmd0OyAtcnctcnctci0tIDEgdmRzbSBr dm0gMiwwTSAyNC4gw7pubyAwMi40NCBsZWFzZXM8YnI+Jmd0OyAtcnctci0tci0tIDEgdmRzbSBr dm0gwqA5OTggMjQuIMO6bm8gMTkuMDcgbWV0YWRhdGE8YnI+Jmd0OyAtcnctcnctci0tIDEgdmRz bSBrdm0gwqAxNk0gwqA3LiBsaXMgMjIuMjAgb3V0Ym94PGJyPjxicj48YnI+SXQgc2hvdWxkIGxv b2sgbGlrZSB0aGlzOjxicj48YnI+LXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAxLjBNIE1hciDCoDEg MjM6MzYgaWRzPGJyPi1ydy1ydy0tLS0uIDEgdmRzbSBrdm0gMi4wTSBNYXIgwqAxIDIzOjM1IGxl YXNlczxicj4tcnctci0tci0tLiAxIHZkc20ga3ZtIMKgMzUzIE1hciDCoDEgMjM6MzUgbWV0YWRh dGE8YnI+LXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSDCoDE2TSBNYXIgwqAxIDIzOjM0IG91dGJveDxi cj4tcnctcnctLS0tLiAxIHZkc20ga3ZtIMKgMTZNIE1hciDCoDEgMjM6MzQgaW5ib3g8YnI+PGJy PlRoaXMgZXhwbGFpbnMgdGhlIEVBQ0NFUyBlcnJvci48YnI+PGJyPllvdSBjYW4gc3RhcnQgYnkg Zml4aW5nIHRoZSBwZXJtaXNzaW9ucyBtYW51YWxseSwgeW91IGNhbiBkbyB0aGlzIG9ubGluZS48 YnI+wqA8YnI+Jmd0OyDCoFRoZSBpZHMgZmlsZXMgd2FzIGdlbmVyYXRlZCBieSAmcXVvdDt0b3Vj aCZxdW90OyBjb21tYW5kIGFmdGVyIGRlbGV0aW5nIHRoZW0gZHVlICZxdW90O3NhbmxvY2sgbG9j a2luZyBoYW5nJnF1b3Q7IMKgZ2x1c3RlciBjcmFzaCAmYW1wOyByZWJvb3Q8YnI+Jmd0OyBJIGV4 cGVjdGVkIHRoYXQgdGhleSB3aWxsIGJlIGZpbGxlZCBhdXRvbWF0aWNhbHkgYWZ0ZXIgZ2x1c3Rl ciByZWJvb3QgKCB0aGUgwqBzaGFkb3cgY29weSBmcm9tIMKgICZxdW90Oy5nbHVzdGVyICZxdW90 OyDCoCBkaXJlY3RvcnkgwqB3YXMgZGVsZXRlZCAmYW1wOyBjcmVhdGVkIGVtcHR5IMKgdG9vICk8 YnI+PGJyPkkgZG9uJiMzOTt0IGtub3cgYWJvdXQgZ2x1c3RlciBzaGFkb3cgY29weSwgSSB3b3Vs ZCBub3QgcGxheSB3aXRoIGdsdXN0ZXIgaW50ZXJuYWxzLjwvZGl2PjxkaXY+QWRkaW5nIFNhaGlu YSBmb3IgYWR2aWNlLjxicj48YnI+Jmd0OyBPSywgaXQgbG9va3MgdGhhdCBzYW5sb2NrIMKgY2Fu JiMzOTt0IHdvcmsgd2l0aCBlbXB0eSBmaWxlIG9yIHJld3JpdGUgdGhlbSAuPGJyPiZndDsgQW0g SSByaWdodCA/Pzxicj48YnI+WWVzLCB0aGUgZmlsZXMgbXVzdCBiZSBpbml0aWFsaXplZCBiZWZv cmUgc2FubG9jayBjYW4gdXNlIHRoZW0uPGJyPjxicj5Zb3UgY2FuIGluaXRpYWxpemUgdGhlIGZp bGUgbGlrZSB0aGlzOjxicj48YnI+c2FubG9jayBkaXJlY3QgaW5pdCAtcyAmbHQ7c2RfdXVpZCZn dDs6MDpyZXBhaXIvJmx0O3NkX3V1aWQmZ3Q7L2RvbV9tZC9pZHM6MDxicj48YnI+VGFrZW4gZnJv bSA8YSBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL3BpcGVybWFpbC91c2Vycy8yMDE2LUZl YnJ1YXJ5LzAzODA0Ni5odG1sIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL3BpcGVybWFpbC91c2Vy cy8yMDE2LUZlYnJ1YXJ5LzAzODA0Ni5odG1sPC9hPjxicj48YnI+Jmd0OyBUaGUgbGFzdCBwb2lu dCAtIGFib3V0ICZxdW90O2lkcyZxdW90OyB3b3JrYXJvdW5kIC0gdGhpcyBpcyBvZmZsaW5lIHZl cnNpb24gPSBWTXMgaGF2ZSB0byBiZSBtb3ZlZCBvdXQgZnJvbSBmb3IgY29udGludWFsIHJ1bm5p bmcgd2l0aCBtYWludGVuYW5jZSB2b2x1bWUgbW9kZTxicj4mZ3Q7IEJ1dCB0aGlzIGlzIG5vdCBh Y2NlcHRhYmxlIGluIGN1cnJlbnQgc2l0dWF0aW9uLCBzbyB0aGUgcXVlc3Rpb24gYWdhaW4sIMKg aXMgaXQgc2FmZSB0byBkbyBpdCBvbmxpbmUgPz8gwqAoIFlFUyAvIE5PICk8L2Rpdj48ZGl2Pjxi cj48L2Rpdj48ZGl2PlRoZSBpZHMgZmlsZSBpcyBhY2Nlc3NlZCBvbmx5IGJ5IHNhbmxvY2suIEkg Z3Vlc3MgdGhhdCB5b3UgZG9uJiMzOTt0IGhhdmUgYSBydW5uaW5nPC9kaXY+PGRpdj5TUE0gb24g dGhpcyBEQywgc2luY2Ugc2FubG9jayBmYWlscyB0byBhY3F1aXJlIGEgaG9zdCBpZCwgc28geW91 IGFyZSBwcmV0dHkgc2FmZTwvZGl2PjxkaXY+dG8gZml4IHRoZSBwZXJtaXNzaW9ucyBhbmQgaW5p dGlhbGl6ZSB0aGUgaWRzIGZpbGVzLjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+SSB3b3VsZCBk byB0aGlzOjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+MS4gU3RvcCBlbmdpbmUsIMKgc28gaXQg d2lsbCBub3QgdHJ5IHRvIHN0YXJ0IHZkc208L2Rpdj48ZGl2PjIuIFN0b3AgdmRzbSBvbiBhbGwg aG9zdHMsIHNvIHRoZXkgZG8gbm90IHRyeSB0byBhY3F1aXJlIGEgaG9zdCBpZCB3aXRoIHNhbmxv Y2s8L2Rpdj48ZGl2PsKgIMKgIFRoaXMgZG9lcyBub3QgYWZmZWN0IHJ1bm5pbmcgdm1zPC9kaXY+ PGRpdj4zLiBGaXggdGhlIHBlcm1pc3Npb25zIG9uIHRoZSBpZHMgZmlsZSwgdmlhIGdsdXN0ZXJm cyBtb3VudDwvZGl2PjxkaXY+NC4gSW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGVzIGZyb20gb25lIG9m IHRoZSBob3N0cywgdmlhIHRoZSBnbHVzdGVyZnMgbW91bnQ8L2Rpdj48ZGl2PsKgIMKgIFRoaXMg c2hvdWxkIGZpeCB0aGUgaWRzIGZpbGVzIG9uIGFsbCByZXBsaWNhczwvZGl2PjxkaXY+NS4gU3Rh cnQgdmRzbSBvbiBhbGwgaG9zdHM8L2Rpdj48ZGl2PjYuIFN0YXJ0IGVuZ2luZTwvZGl2PjxkaXY+ PGJyPjwvZGl2PjxkaXY+RW5naW5lIHdpbGwgY29ubmVjdCB0byBhbGwgaG9zdHMsIGhvc3RzIHdp bGwgY29ubmVjdCB0byBzdG9yYWdlIGFuZCB0cnkgdG8gYWNxdWlyZSBhIGhvc3QgaWQuPC9kaXY+ PGRpdj5UaGVuIEVuZ2luZSB3aWxsIHN0YXJ0IHRoZSBTUE0gb24gb25lIG9mIHRoZSBob3N0cywg YW5kIHlvdXIgREMgc2hvdWxkIGJlY29tZSB1cC48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PkRh dmlkLCBTYWhpbmEsIGNhbiB5b3UgY29uZmlybSB0aGF0IHRoaXMgcHJvY2VkdXJlIGlzIHNhZmU/ PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5OaXI8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PiZn dDs8YnI+Jmd0OyByZWdzLjxicj4mZ3Q7IFBhdmVsPGJyPiZndDs8YnI+Jmd0Ozxicj4mZ3Q7PGJy PiZndDsgT24gMS4zLjIwMTYgMTg6MzgsIE5pciBTb2ZmZXIgd3JvdGU6PGJyPiZndDs8YnI+Jmd0 OyBPbiBUdWUsIE1hciAxLCAyMDE2IGF0IDU6MDcgUE0sIDxhIGhyZWY9Im1haWx0bzpwYWYxQGVt YWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiAmbHQ7PGEgaHJlZj0ibWFpbHRvOnBhZjFAZW1haWwu Y3oiPnBhZjFAZW1haWwuY3o8L2E+Jmd0OyB3cm90ZTo8YnI+Jmd0OyZndDs8YnI+Jmd0OyZndDsg SGVsbG8sIMKgY2FuIGFueWJvZHkgwqBleHBsYWluIHRoaXMgZXJyb3Igbm8uMTMgKCBvcGVuIGZp bGUgKSBpbiBzYW5sb2NrLmxvZyAuPGJyPiZndDs8YnI+Jmd0Ozxicj4mZ3Q7IFRoaXMgaXMgRUFD Q0VTPGJyPiZndDs8YnI+Jmd0OyBDYW4geW91IHNoYXJlIHRoZSBvdXRvcHV0IG9mOjxicj4mZ3Q7 PGJyPiZndDsgwqAgwqAgbHMgLWxoIC9yaGV2L2RhdGEtY2VudGVyL21udC8mbHQ7c2VydmVyJmd0 OzombHQ7X3BhdGgmZ3Q7LyZsdDtzZF91dWlkJmd0Oy9kb21fbWQ8YnI+Jmd0OyDCoDxicj4mZ3Q7 Jmd0Ozxicj4mZ3Q7Jmd0Ozxicj4mZ3Q7Jmd0OyBUaGUgc2l6ZSBvZiDCoCZxdW90O2lkcyZxdW90 OyBmaWxlIGlzIHplcm8gKDApPGJyPiZndDs8YnI+Jmd0Ozxicj4mZ3Q7IFRoaXMgaXMgaG93IHdl IGNyZWF0ZSB0aGUgaWRzIGZpbGUgd2hlbiBpbml0aWFsaXppbmcgaXQuPGJyPiZndDs8YnI+Jmd0 OyBCdXQgdGhlbiB3ZSB1c2Ugc2FubG9jayB0byBpbml0aWFsaXplIHRoZSBpZHMgZmlsZSwgYW5k IGl0IHNob3VsZCBiZSAxTWlCIGFmdGVyIHRoYXQuPGJyPiZndDs8YnI+Jmd0OyBJcyB0aGlzIGlk cyBmaWxlcyBjcmVhdGVkIGJ5IHZkc20sIG9yIG9uZSB5b3UgY3JlYXRlZCB5b3Vyc2VsZj88YnI+ Jmd0OyDCoDxicj4mZ3Q7Jmd0Ozxicj4mZ3Q7Jmd0OyAyMDE2LTAyLTI4IDAzOjI1OjQ2KzAxMDAg MjY5NjI2IFsxOTUxXTogb3BlbiBlcnJvciAtMTMgL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQz MjcxMjQvZG9tX21kL2lkczxicj4mZ3Q7Jmd0OyAyMDE2LTAyLTI4IDAzOjI1OjQ2KzAxMDAgMjY5 NjI2IFsxOTUxXTogczE4Nzk4NSBvcGVuX2Rpc2sgL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQz MjcxMjQvZG9tX21kL2lkcyBlcnJvciAtMTM8YnI+Jmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo1 NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBzMTg3OTkyIGxvY2tzcGFjZSA3ZjUyYjY5Ny1jMTk5LTRm NTgtODlhYS0xMDJkNDQzMjcxMjQ6MTovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xv Y2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9k b21fbWQvaWRzOjA8YnI+Jmd0OyZndDs8YnI+Jmd0OyZndDsgSWYgdGhlIG1haW4gcHJvYmxlbSBp cyBhYm91dCB6ZXJvIGZpbGUgc2l6ZSwgY2FuIEkgcmVnZW5lcmF0ZSDCoHRoaXMgZmlsZSBvbmxp bmUgc2VjdXJlbHkgLCB3aXRoIG5vIFZNIGRlcGVuZGVuY2UgwqA/Pz8/PGJyPiZndDs8YnI+Jmd0 Ozxicj4mZ3Q7IFllcywgSSB0aGluayBJIGFscmVhZHkgcmVmZXJyZWQgdG8gdGhlIGluc3RydWN0 aW9ucyBob3cgdG8gZG8gdGhhdCBpbiBhIHByZXZpb3VzIG1haWwuPGJyPiZndDs8YnI+Jmd0OyZn dDs8YnI+Jmd0OyZndDs8YnI+Jmd0OyZndDsgZGlzdCA9IFJIRUwgLSA3IC0gMi4xNTExPGJyPiZn dDsmZ3Q7IGtlcm5lbCA9IDMuMTAuMCAtIDMyNy4xMC4xLmVsNy54ODZfNjQ8YnI+Jmd0OyZndDsg S1ZNID0gMi4zLjAgLSAyOS4xLmVsNzxicj4mZ3Q7Jmd0OyBsaWJ2aXJ0ID0gbGlidmlydC0xLjIu MTctMTMuZWw3XzIuMzxicj4mZ3Q7Jmd0OyB2ZHNtID0gdmRzbS00LjE2LjMwLTAuZWw3PGJyPiZn dDsmZ3Q7IEdsdXN0ZXJGUyA9IGdsdXN0ZXJmcy0zLjcuOC0xLmVsNzxicj4mZ3Q7Jmd0Ozxicj4m Z3Q7Jmd0Ozxicj4mZ3Q7Jmd0OyByZWdzLjxicj4mZ3Q7Jmd0OyBQYXZlbDxicj4mZ3Q7Jmd0Ozxi cj4mZ3Q7Jmd0OyBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f Xzxicj4mZ3Q7Jmd0OyBVc2VycyBtYWlsaW5nIGxpc3Q8YnI+Jmd0OyZndDsgPGEgaHJlZj0ibWFp bHRvOlVzZXJzQG92aXJ0Lm9yZyI+VXNlcnNAb3ZpcnQub3JnPC9hPjxicj4mZ3Q7Jmd0OyA8YSBo cmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMiPmh0dHA6 Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VyczwvYT48YnI+Jmd0OyZndDs8 YnI+Jmd0Ozxicj4mZ3Q7PGJyPjwvZGl2PjwvZGl2Pgo= --===============2299145876614063024==-- From teigland at redhat.com Tue Mar 1 17:20:14 2016 Content-Type: multipart/mixed; boundary="===============2922328303880019099==" MIME-Version: 1.0 From: David Teigland To: users at ovirt.org Subject: Re: [ovirt-users] open error -13 = sanlock Date: Tue, 01 Mar 2016 16:20:13 -0600 Message-ID: <20160301222013.GB7457@redhat.com> In-Reply-To: CAMRbyyu9gwPfVpPxpDa4_gKWyXq1PavTm2V2rG2cU0AvE=JJPA@mail.gmail.com --===============2922328303880019099== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Wed, Mar 02, 2016 at 12:15:17AM +0200, Nir Soffer wrote: > 1. Stop engine, so it will not try to start vdsm > 2. Stop vdsm on all hosts, so they do not try to acquire a host id with > sanlock > This does not affect running vms > 3. Fix the permissions on the ids file, via glusterfs mount > 4. Initialize the ids files from one of the hosts, via the glusterfs mount > This should fix the ids files on all replicas > 5. Start vdsm on all hosts > 6. Start engine > = > Engine will connect to all hosts, hosts will connect to storage and try to > acquire a host id. > Then Engine will start the SPM on one of the hosts, and your DC should > become up. > = > David, Sahina, can you confirm that this procedure is safe? Looks right. --===============2922328303880019099==-- From sabose at redhat.com Wed Mar 2 01:32:53 2016 Content-Type: multipart/mixed; boundary="===============0844145120624418695==" MIME-Version: 1.0 From: Sahina Bose To: users at ovirt.org Subject: Re: [ovirt-users] open error -13 = sanlock Date: Wed, 02 Mar 2016 12:02:48 +0530 Message-ID: <56D68910.8040602@redhat.com> In-Reply-To: CAMRbyyu9gwPfVpPxpDa4_gKWyXq1PavTm2V2rG2cU0AvE=JJPA@mail.gmail.com --===============0844145120624418695== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------030409010206020806050907 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit On 03/02/2016 03:45 AM, Nir Soffer wrote: > On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz = > > wrote: > > > > HI, > > requested output: > > > > # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-= bef3-0bc0b7a99fb7/dom_md: > > total 2,1M > > -rw-rw---- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- good > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.17 leases > > -rw-r--r-- 1 vdsm kvm 335 7. lis 22.17 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a= 579-4cabeb55ded4/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.41 ids <-- bad (sanlo= ck = > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 03.56 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 03.56 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-9= 992-b04464a04112/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlo= ck = > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.14 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.14 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-a= b46-084f3d70b147/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlo= ck = > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.51 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases > > -rw-r--r-- 1 vdsm kvm 998 25. =C3=BAno 00.35 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.16 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.44 ids <-- bad (sanlo= ck = > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 00.18 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 00.18 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-b= e41-77002d1cd553/dom_md: > > total 1,1M > > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.32 ids <-- bad (other = > can read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.18 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.18 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9= bfe-c3da888e47f0/dom_md: > > total 3,0M > > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other = > can read) > > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.42 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases > > -rw-r--r-- 1 vdsm kvm 997 24. =C3=BAno 02.46 metadata > > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.44 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-b= a9a-e6d9bdecb4b4/dom_md: > > total 2,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.34 ids <-- bad (sanlo= ck = > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.35 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases > > -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata > > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.27 outbox > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-= 9bb1-1df2130c7c97/dom_md: > > total 3,0M > > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other = > can read) > > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 23.50 inbox <-- bad (other can = > read) > > -rw-rw-r-- 1 vdsm kvm 2,0M 6. lis 23.51 leases <-- bad (other can = > read) > > -rw-rw-r-- 1 vdsm kvm 734 7. lis 02.13 metadata <-- bad (group = > can write, other can read) > > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 16.55 outbox <-- bad (other can = > read) > > > > = > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9= f59-3aebea4d8477/dom_md: > > total 1,1M > > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.35 ids <-- bad (other can r= ead) > > -rw-rw-r-- 1 vdsm kvm 16M 24. =C3=BAno 01.06 inbox > > -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases > > -rw-r--r-- 1 vdsm kvm 998 24. =C3=BAno 19.07 metadata > > -rw-rw-r-- 1 vdsm kvm 16M 7. lis 22.20 outbox > > > It should look like this: > > -rw-rw----. 1 vdsm kvm 1.0M Mar 1 23:36 ids > -rw-rw----. 1 vdsm kvm 2.0M Mar 1 23:35 leases > -rw-r--r--. 1 vdsm kvm 353 Mar 1 23:35 metadata > -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 outbox > -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 inbox > > This explains the EACCES error. > > You can start by fixing the permissions manually, you can do this online. > > > The ids files was generated by "touch" command after deleting them = > due "sanlock locking hang" gluster crash & reboot > > I expected that they will be filled automaticaly after gluster = > reboot ( the shadow copy from ".gluster " directory was deleted & = > created empty too ) > > I don't know about gluster shadow copy, I would not play with gluster = > internals. > Adding Sahina for advice. Did you generate the ids file on the mount point. Ravi, can you help here? > > > OK, it looks that sanlock can't work with empty file or rewrite them . > > Am I right ?? > > Yes, the files must be initialized before sanlock can use them. > > You can initialize the file like this: > > sanlock direct init -s :0:repair//dom_md/ids:0 > > Taken from = > http://lists.ovirt.org/pipermail/users/2016-February/038046.html > > > The last point - about "ids" workaround - this is offline version =3D = > VMs have to be moved out from for continual running with maintenance = > volume mode > > But this is not acceptable in current situation, so the question = > again, is it safe to do it online ?? ( YES / NO ) > > The ids file is accessed only by sanlock. I guess that you don't have = > a running > SPM on this DC, since sanlock fails to acquire a host id, so you are = > pretty safe > to fix the permissions and initialize the ids files. > > I would do this: > > 1. Stop engine, so it will not try to start vdsm > 2. Stop vdsm on all hosts, so they do not try to acquire a host id = > with sanlock > This does not affect running vms > 3. Fix the permissions on the ids file, via glusterfs mount > 4. Initialize the ids files from one of the hosts, via the glusterfs mount > This should fix the ids files on all replicas > 5. Start vdsm on all hosts > 6. Start engine > > Engine will connect to all hosts, hosts will connect to storage and = > try to acquire a host id. > Then Engine will start the SPM on one of the hosts, and your DC should = > become up. > > David, Sahina, can you confirm that this procedure is safe? Yes, correcting from the mount point should fix it on all replicas > > Nir > > > > > regs. > > Pavel > > > > > > > > On 1.3.2016 18:38, Nir Soffer wrote: > > > > On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz = > > wrote: > >> > >> Hello, can anybody explain this error no.13 ( open file ) in = > sanlock.log . > > > > > > This is EACCES > > > > Can you share the outoput of: > > > > ls -lh /rhev/data-center/mnt/:<_path>//dom_md > > > >> > >> > >> The size of "ids" file is zero (0) > > > > > > This is how we create the ids file when initializing it. > > > > But then we use sanlock to initialize the ids file, and it should be = > 1MiB after that. > > > > Is this ids files created by vdsm, or one you created yourself? > > > >> > >> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 = > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids > >> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk = > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids = > error -13 > >> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace = > 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/lo= calhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 > >> > >> If the main problem is about zero file size, can I regenerate this = > file online securely , with no VM dependence ???? > > > > > > Yes, I think I already referred to the instructions how to do that = > in a previous mail. > > > >> > >> > >> dist =3D RHEL - 7 - 2.1511 > >> kernel =3D 3.10.0 - 327.10.1.el7.x86_64 > >> KVM =3D 2.3.0 - 29.1.el7 > >> libvirt =3D libvirt-1.2.17-13.el7_2.3 > >> vdsm =3D vdsm-4.16.30-0.el7 > >> GlusterFS =3D glusterfs-3.7.8-1.el7 > >> > >> > >> regs. > >> Pavel > >> > >> _______________________________________________ > >> Users mailing list > >> Users(a)ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/users > >> > > > > --------------030409010206020806050907 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit

On 03/02/2016 03:45 AM, Nir Soffer wrote:

On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.= cz <paf= 1(a)email.cz> wrote:
>
> HI,
> requested output:
>
> # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md
> =C2=A0
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-be= f3-0bc0b7a99fb7/dom_md:
> total 2,1M
> -rw-rw---- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids =C2=A0 = =C2=A0 =C2=A0 =C2=A0<-- good
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.17 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0335 =C2=A07. lis 22.17 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a57= 9-4cabeb55ded4/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.41 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 03.56 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 03.56 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-999= 2-b04464a04112/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.14 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.14 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-ab4= 6-084f3d70b147/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.51 inbox
> -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 25. =C3=BAno 00.35 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.16 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.44 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 00.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 00.18 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-be4= 1-77002d1cd553/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.32 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.18 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9bf= e-c3da888e47f0/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.42 inbox=C2=A0=
> -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0997 24. =C3=BAno 02.46 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.44 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-ba9= a-e6d9bdecb4b4/dom_md:
> total 2,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.34 ids=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.35 inbox
> -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases
> -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.27 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-9b= b1-1df2130c7c97/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 23.50 inbox=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm 2,0M =C2=A06. lis 23.51 leases=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A0734 =C2=A07. lis 02.13 metadata= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (group can write, other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 16.55 outbox=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9f5= 9-3aebea4d8477/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.35 ids= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M 24. =C3=BAno 01.06 inbox
> -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 24. =C3=BAno 19.07 metadata<= br> > -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.20 outbox

It should look like this:

-rw-rw----. 1 vdsm kvm 1.0M Mar =C2=A01 23:36 ids
-rw-rw----. 1 vdsm kvm 2.0M Mar =C2=A01 23:35 leases
-rw-r--r--. 1 vdsm kvm =C2=A0353 Mar =C2=A01 23:35 metadata
-rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 outbox
-rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 inbox

This explains the EACCES error.

You can start by fixing the permissions manually, you can do this online.
=C2=A0
> =C2=A0The ids files was generated by "touch" command after deleting them due "sanlock locking hang" =C2=A0gluster crash & reboot
> I expected that they will be filled automaticaly after gluster reboot ( the =C2=A0shadow copy from =C2=A0 ".gluster " = =C2=A0 directory =C2=A0was deleted & created empty =C2=A0too )

I don't know about gluster shadow copy, I would not play with gluster internals.

Adding Sahina for advice.

Did you generate the ids file on the mount point.

Ravi, can you help here?

> OK, it looks that sanlock =C2=A0can't work with empty file or rewrite them .
> Am I right ??

Yes, the files must be initialized before sanlock can use them.

You can initialize the file like this:

sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0

Taken from http://lists.ovirt.org/pipermail/users/2016-February/038046.html=

> The last point - about "ids" workaround - this is offline version =3D VMs have to be moved out from for continual running with maintenance volume mode
> But this is not acceptable in current situation, so the question again, =C2=A0is it safe to do it online ?? =C2=A0( YES /= NO )

The ids file is accessed only by sanlock. I guess that you don't have a running

SPM on this DC, since sanlock fails to acquire a host id, so you are pretty safe

to fix the permissions and initialize the ids files.

I would do this:

1. Stop engine, =C2=A0so it will not try to start vdsm

2. Stop vdsm on all hosts, so they do not try to acquire a host id with sanlock

=C2=A0 =C2=A0 This does not affect running vms

3. Fix the permissions on the ids file, via glusterfs mount
4. Initialize the ids files from one of the hosts, via the glusterfs mount

=C2=A0 =C2=A0 This should fix the ids files on all replicas
5. Start vdsm on all hosts

6. Start engine

Engine will connect to all hosts, hosts will connect to storage and try to acquire a host id.

Then Engine will start the SPM on one of the hosts, and your DC should become up.

David, Sahina, can you confirm that this procedure is safe?

Yes, correcting from the mount point should fix it on all replicas

Nir

>
> regs.
> Pavel
>
>
>
> On 1.3.2016 18:38, Nir Soffer wrote:
>
> On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)emai= l.cz <p= af1(a)email.cz> wrote:
>>
>> Hello, =C2=A0can anybody =C2=A0explain this error no.13 = ( open file ) in sanlock.log .
>
>
> This is EACCES
>
> Can you share the outoput of:
>
> =C2=A0 =C2=A0 ls -lh /rhev/data-center/mnt/<server>:<_path>/<sd_uuid>/dom_md > =C2=A0
>>
>>
>> The size of =C2=A0"ids" file is zero (0)
>
>
> This is how we create the ids file when initializing it.
>
> But then we use sanlock to initialize the ids file, and it should be 1MiB after that.
>
> Is this ids files created by vdsm, or one you created yourself?
> =C2=A0
>>
>> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids
>> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c19= 9-4f58-89aa-102d44327124/dom_md/ids error -13
>> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0
>>
>> If the main problem is about zero file size, can I regenerate =C2=A0this file online securely , with no VM dependence =C2=A0????
>
>
> Yes, I think I already referred to the instructions how to do that in a previous mail.
>
>>
>>
>> dist =3D RHEL - 7 - 2.1511
>> kernel =3D 3.10.0 - 327.10.1.el7.x86_64
>> KVM =3D 2.3.0 - 29.1.el7
>> libvirt =3D libvirt-1.2.17-13.el7_2.3
>> vdsm =3D vdsm-4.16.30-0.el7
>> GlusterFS =3D glusterfs-3.7.8-1.el7
>>
>>
>> regs.
>> Pavel
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://l= ists.ovirt.org/mailman/listinfo/users
>>
>
>

--------------030409010206020806050907-- --===============0844145120624418695== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMzA0MDkwMTAyMDYwMjA4MDYwNTA5MDcKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK CgoKT24gMDMvMDIvMjAxNiAwMzo0NSBBTSwgTmlyIFNvZmZlciB3cm90ZToKPiBPbiBUdWUsIE1h ciAxLCAyMDE2IGF0IDEwOjUxIFBNLCBwYWYxQGVtYWlsLmN6IDxtYWlsdG86cGFmMUBlbWFpbC5j ej4gCj4gPHBhZjFAZW1haWwuY3ogPG1haWx0bzpwYWYxQGVtYWlsLmN6Pj4gd3JvdGU6Cj4gPgo+ ID4gSEksCj4gPiByZXF1ZXN0ZWQgb3V0cHV0Ogo+ID4KPiA+ICMgbHMgLWxoIC9yaGV2L2RhdGEt Y2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0OiovKi9kb21fbWQKPiA+Cj4gPiAKPiAvcmhl di9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLUJDSy8wZmNhZDg4 OC1kNTczLTQ3YmUtYmVmMy0wYmMwYjdhOTlmYjcvZG9tX21kOgo+ID4gdG90YWwgMiwxTQo+ID4g LXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDEsME0gIDEuIGLFmWUgMjEuMjggaWRzICAgICAgICA8LS0g Z29vZAo+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAyMi4xNiBpbmJveAo+ ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gIDcuIGxpcyAyMi4xNyBsZWFzZXMKPiA+IC1y dy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzM1ICA3LiBsaXMgMjIuMTcgbWV0YWRhdGEKPiA+IC1ydy1y dy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMjIuMTYgb3V0Ym94Cj4gPgo+ID4gCj4gL3Jo ZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QMS81NTNkOWI5 Mi1lNGEwLTQwNDItYTU3OS00Y2FiZWI1NWRlZDQvZG9tX21kOgo+ID4gdG90YWwgMSwxTQo+ID4g LXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuNDEgaWRzICAgICAgICA8LS0g YmFkIChzYW5sb2NrIAo+IGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpCj4gPiAtcnctcnct LS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE0IGluYm94Cj4gPiAtcnctcnctLS0tIDEg dmRzbSBrdm0gMiwwTSAgNy4gbGlzIDAzLjU2IGxlYXNlcwo+ID4gLXJ3LXItLXItLSAxIHZkc20g a3ZtICAzMzMgIDcuIGxpcyAwMy41NiBtZXRhZGF0YQo+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3Zt ICAxNk0gIDcuIGxpcyAwMC4xNCBvdXRib3gKPiA+Cj4gPiAKPiAvcmhldi9kYXRhLWNlbnRlci9t bnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVAyLzg4YWRiZDQ5LTYyZDYtNDViMS05OTky LWIwNDQ2NGEwNDExMi9kb21fbWQ6Cj4gPiB0b3RhbCAxLDFNCj4gPiAtcnctci0tci0tIDEgdmRz bSBrdm0gICAgMCAyNC4gw7pubyAwNy40MyBpZHMgICAgICAgIDwtLSBiYWQgKHNhbmxvY2sgCj4g Y2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCkKPiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAg MTZNICA3LiBsaXMgMDAuMTUgaW5ib3gKPiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNICA3 LiBsaXMgMjIuMTQgbGVhc2VzCj4gPiAtcnctci0tci0tIDEgdmRzbSBrdm0gIDMzMyAgNy4gbGlz IDIyLjE0IG1ldGFkYXRhCj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAw LjE1IG91dGJveAo+ID4KPiA+IAo+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9j YWxob3N0Ol8xS1ZNMTItUDMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2Rv bV9tZDoKPiA+IHRvdGFsIDEsMU0KPiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDD um5vIDA3LjQzIGlkcyAgICAgICAgPC0tIGJhZCAoc2FubG9jayAKPiBjYW5ub3Qgd3JpdGUsIG90 aGVyIGNhbiByZWFkKQo+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gMjMuIMO6bm8gMjIu NTEgaW5ib3gKPiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDIzLiDDum5vIDIzLjEyIGxl YXNlcwo+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICA5OTggMjUuIMO6bm8gMDAuMzUgbWV0YWRh dGEKPiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTYgb3V0Ym94Cj4g Pgo+ID4gCj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0x Mi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kOgo+ID4gdG90 YWwgMSwxTQo+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuNDQgaWRz ICAgICAgICA8LS0gYmFkIChzYW5sb2NrIAo+IGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQp Cj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE3IGluYm94Cj4gPiAt cnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDAwLjE4IGxlYXNlcwo+ID4gLXJ3LXIt LXItLSAxIHZkc20ga3ZtICAzMzMgIDcuIGxpcyAwMC4xOCBtZXRhZGF0YQo+ID4gLXJ3LXJ3LS0t LSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNyBvdXRib3gKPiA+Cj4gPiAKPiAvcmhldi9k YXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyLVAxLzQyZDcxMGE5LWI4 NDQtNDNkYy1iZTQxLTc3MDAyZDFjZDU1My9kb21fbWQ6Cj4gPiB0b3RhbCAxLDFNCj4gPiAtcnct cnctci0tIDEgdmRzbSBrdm0gICAgMCAyNC4gw7pubyAwNy4zMiBpZHMgICAgICAgIDwtLSBiYWQg KG90aGVyIAo+IGNhbiByZWFkKQo+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxp cyAyMi4xOCBpbmJveAo+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gIDcuIGxpcyAyMi4x OCBsZWFzZXMKPiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzMzICA3LiBsaXMgMjIuMTggbWV0 YWRhdGEKPiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMjIuMTggb3V0Ym94 Cj4gPgo+ID4gCj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJL Vk0xMi1QMi9mZjcxYjQ3Yi0wZjcyLTQ1MjgtOWJmZS1jM2RhODg4ZTQ3ZjAvZG9tX21kOgo+ID4g dG90YWwgMywwTQo+ID4gLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDEsME0gIDEuIGLFmWUgMjEuMjgg aWRzICAgICAgICA8LS0gYmFkIChvdGhlciAKPiBjYW4gcmVhZCkKPiA+IC1ydy1ydy0tLS0gMSB2 ZHNtIGt2bSAgMTZNIDI1LiDDum5vIDAwLjQyIGluYm94Cj4gPiAtcnctcnctLS0tIDEgdmRzbSBr dm0gMiwwTSAyNS4gw7pubyAwMC40NCBsZWFzZXMKPiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAg OTk3IDI0LiDDum5vIDAyLjQ2IG1ldGFkYXRhCj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2 TSAyNS4gw7pubyAwMC40NCBvdXRib3gKPiA+Cj4gPiAKPiAvcmhldi9kYXRhLWNlbnRlci9tbnQv Z2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyLVAzL2VmMDEwZDA4LWFlZDEtNDFjNC1iYTlhLWU2 ZDliZGVjYjRiNC9kb21fbWQ6Cj4gPiB0b3RhbCAyLDFNCj4gPiAtcnctci0tci0tIDEgdmRzbSBr dm0gICAgMCAyNC4gw7pubyAwNy4zNCBpZHMgICAgICAgIDwtLSBiYWQgKHNhbmxvY2sgCj4gY2Fu bm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCkKPiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZN IDIzLiDDum5vIDIyLjM1IGluYm94Cj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4g w7pubyAyMi4zOCBsZWFzZXMKPiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAxLDFLIDI0LiDDum5v IDE5LjA3IG1ldGFkYXRhCj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyMy4gw7pubyAy Mi4yNyBvdXRib3gKPiA+Cj4gPiAKPiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xv Y2FsaG9zdDpfMktWTTEyX19QNC8zMDBlOWFjOC0zYzJmLTQ3MDMtOWJiMS0xZGYyMTMwYzdjOTcv ZG9tX21kOgo+ID4gdG90YWwgMywwTQo+ID4gLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDEsME0gIDEu IGLFmWUgMjEuMjggaWRzICAgICAgICA8LS0gYmFkIChvdGhlciAKPiBjYW4gcmVhZCkKPiA+IC1y dy1ydy1yLS0gMSB2ZHNtIGt2bSAgMTZNICA2LiBsaXMgMjMuNTAgaW5ib3ggIDwtLSBiYWQgKG90 aGVyIGNhbiAKPiByZWFkKQo+ID4gLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gIDYuIGxpcyAy My41MSBsZWFzZXMgIDwtLSBiYWQgKG90aGVyIGNhbiAKPiByZWFkKQo+ID4gLXJ3LXJ3LXItLSAx IHZkc20ga3ZtICA3MzQgIDcuIGxpcyAwMi4xMyBtZXRhZGF0YSAgPC0tIGJhZCAoZ3JvdXAgCj4g Y2FuIHdyaXRlLCBvdGhlciBjYW4gcmVhZCkKPiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgMTZN ICA2LiBsaXMgMTYuNTUgb3V0Ym94ICA8LS0gYmFkIChvdGhlciBjYW4gCj4gcmVhZCkKPiA+Cj4g PiAKPiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyLVA1 LzFjYTU2YjQ1LTcwMWUtNGMyMi05ZjU5LTNhZWJlYTRkODQ3Ny9kb21fbWQ6Cj4gPiB0b3RhbCAx LDFNCj4gPiAtcnctcnctci0tIDEgdmRzbSBrdm0gICAgMCAyNC4gw7pubyAwNy4zNSBpZHMgIDwt LSBiYWQgKG90aGVyIGNhbiByZWFkKQo+ID4gLXJ3LXJ3LXItLSAxIHZkc20ga3ZtICAxNk0gMjQu IMO6bm8gMDEuMDYgaW5ib3gKPiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAyLDBNIDI0LiDDum5v IDAyLjQ0IGxlYXNlcwo+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICA5OTggMjQuIMO6bm8gMTku MDcgbWV0YWRhdGEKPiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMjIuMjAg b3V0Ym94Cj4KPgo+IEl0IHNob3VsZCBsb29rIGxpa2UgdGhpczoKPgo+IC1ydy1ydy0tLS0uIDEg dmRzbSBrdm0gMS4wTSBNYXIgIDEgMjM6MzYgaWRzCj4gLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAy LjBNIE1hciAgMSAyMzozNSBsZWFzZXMKPiAtcnctci0tci0tLiAxIHZkc20ga3ZtICAzNTMgTWFy ICAxIDIzOjM1IG1ldGFkYXRhCj4gLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAgMTZNIE1hciAgMSAy MzozNCBvdXRib3gKPiAtcnctcnctLS0tLiAxIHZkc20ga3ZtICAxNk0gTWFyICAxIDIzOjM0IGlu Ym94Cj4KPiBUaGlzIGV4cGxhaW5zIHRoZSBFQUNDRVMgZXJyb3IuCj4KPiBZb3UgY2FuIHN0YXJ0 IGJ5IGZpeGluZyB0aGUgcGVybWlzc2lvbnMgbWFudWFsbHksIHlvdSBjYW4gZG8gdGhpcyBvbmxp bmUuCj4KPiA+ICBUaGUgaWRzIGZpbGVzIHdhcyBnZW5lcmF0ZWQgYnkgInRvdWNoIiBjb21tYW5k IGFmdGVyIGRlbGV0aW5nIHRoZW0gCj4gZHVlICJzYW5sb2NrIGxvY2tpbmcgaGFuZyIgIGdsdXN0 ZXIgY3Jhc2ggJiByZWJvb3QKPiA+IEkgZXhwZWN0ZWQgdGhhdCB0aGV5IHdpbGwgYmUgZmlsbGVk IGF1dG9tYXRpY2FseSBhZnRlciBnbHVzdGVyIAo+IHJlYm9vdCAoIHRoZSAgc2hhZG93IGNvcHkg ZnJvbSAgICIuZ2x1c3RlciAiIGRpcmVjdG9yeSAgd2FzIGRlbGV0ZWQgJiAKPiBjcmVhdGVkIGVt cHR5ICB0b28gKQo+Cj4gSSBkb24ndCBrbm93IGFib3V0IGdsdXN0ZXIgc2hhZG93IGNvcHksIEkg d291bGQgbm90IHBsYXkgd2l0aCBnbHVzdGVyIAo+IGludGVybmFscy4KPiBBZGRpbmcgU2FoaW5h IGZvciBhZHZpY2UuCgpEaWQgeW91IGdlbmVyYXRlIHRoZSBpZHMgZmlsZSBvbiB0aGUgbW91bnQg cG9pbnQuCgpSYXZpLCBjYW4geW91IGhlbHAgaGVyZT8KCj4KPiA+IE9LLCBpdCBsb29rcyB0aGF0 IHNhbmxvY2sgIGNhbid0IHdvcmsgd2l0aCBlbXB0eSBmaWxlIG9yIHJld3JpdGUgdGhlbSAuCj4g PiBBbSBJIHJpZ2h0ID8/Cj4KPiBZZXMsIHRoZSBmaWxlcyBtdXN0IGJlIGluaXRpYWxpemVkIGJl Zm9yZSBzYW5sb2NrIGNhbiB1c2UgdGhlbS4KPgo+IFlvdSBjYW4gaW5pdGlhbGl6ZSB0aGUgZmls ZSBsaWtlIHRoaXM6Cj4KPiBzYW5sb2NrIGRpcmVjdCBpbml0IC1zIDxzZF91dWlkPjowOnJlcGFp ci88c2RfdXVpZD4vZG9tX21kL2lkczowCj4KPiBUYWtlbiBmcm9tIAo+IGh0dHA6Ly9saXN0cy5v dmlydC5vcmcvcGlwZXJtYWlsL3VzZXJzLzIwMTYtRmVicnVhcnkvMDM4MDQ2Lmh0bWwKPgo+ID4g VGhlIGxhc3QgcG9pbnQgLSBhYm91dCAiaWRzIiB3b3JrYXJvdW5kIC0gdGhpcyBpcyBvZmZsaW5l IHZlcnNpb24gPSAKPiBWTXMgaGF2ZSB0byBiZSBtb3ZlZCBvdXQgZnJvbSBmb3IgY29udGludWFs IHJ1bm5pbmcgd2l0aCBtYWludGVuYW5jZSAKPiB2b2x1bWUgbW9kZQo+ID4gQnV0IHRoaXMgaXMg bm90IGFjY2VwdGFibGUgaW4gY3VycmVudCBzaXR1YXRpb24sIHNvIHRoZSBxdWVzdGlvbiAKPiBh Z2FpbiwgIGlzIGl0IHNhZmUgdG8gZG8gaXQgb25saW5lID8/ICAoIFlFUyAvIE5PICkKPgo+IFRo ZSBpZHMgZmlsZSBpcyBhY2Nlc3NlZCBvbmx5IGJ5IHNhbmxvY2suIEkgZ3Vlc3MgdGhhdCB5b3Ug ZG9uJ3QgaGF2ZSAKPiBhIHJ1bm5pbmcKPiBTUE0gb24gdGhpcyBEQywgc2luY2Ugc2FubG9jayBm YWlscyB0byBhY3F1aXJlIGEgaG9zdCBpZCwgc28geW91IGFyZSAKPiBwcmV0dHkgc2FmZQo+IHRv IGZpeCB0aGUgcGVybWlzc2lvbnMgYW5kIGluaXRpYWxpemUgdGhlIGlkcyBmaWxlcy4KPgo+IEkg d291bGQgZG8gdGhpczoKPgo+IDEuIFN0b3AgZW5naW5lLCAgc28gaXQgd2lsbCBub3QgdHJ5IHRv IHN0YXJ0IHZkc20KPiAyLiBTdG9wIHZkc20gb24gYWxsIGhvc3RzLCBzbyB0aGV5IGRvIG5vdCB0 cnkgdG8gYWNxdWlyZSBhIGhvc3QgaWQgCj4gd2l0aCBzYW5sb2NrCj4gICAgIFRoaXMgZG9lcyBu b3QgYWZmZWN0IHJ1bm5pbmcgdm1zCj4gMy4gRml4IHRoZSBwZXJtaXNzaW9ucyBvbiB0aGUgaWRz IGZpbGUsIHZpYSBnbHVzdGVyZnMgbW91bnQKPiA0LiBJbml0aWFsaXplIHRoZSBpZHMgZmlsZXMg ZnJvbSBvbmUgb2YgdGhlIGhvc3RzLCB2aWEgdGhlIGdsdXN0ZXJmcyBtb3VudAo+ICAgICBUaGlz IHNob3VsZCBmaXggdGhlIGlkcyBmaWxlcyBvbiBhbGwgcmVwbGljYXMKPiA1LiBTdGFydCB2ZHNt IG9uIGFsbCBob3N0cwo+IDYuIFN0YXJ0IGVuZ2luZQo+Cj4gRW5naW5lIHdpbGwgY29ubmVjdCB0 byBhbGwgaG9zdHMsIGhvc3RzIHdpbGwgY29ubmVjdCB0byBzdG9yYWdlIGFuZCAKPiB0cnkgdG8g YWNxdWlyZSBhIGhvc3QgaWQuCj4gVGhlbiBFbmdpbmUgd2lsbCBzdGFydCB0aGUgU1BNIG9uIG9u ZSBvZiB0aGUgaG9zdHMsIGFuZCB5b3VyIERDIHNob3VsZCAKPiBiZWNvbWUgdXAuCj4KPiBEYXZp ZCwgU2FoaW5hLCBjYW4geW91IGNvbmZpcm0gdGhhdCB0aGlzIHByb2NlZHVyZSBpcyBzYWZlPwoK WWVzLCBjb3JyZWN0aW5nIGZyb20gdGhlIG1vdW50IHBvaW50IHNob3VsZCBmaXggaXQgb24gYWxs IHJlcGxpY2FzCgoKPgo+IE5pcgo+Cj4gPgo+ID4gcmVncy4KPiA+IFBhdmVsCj4gPgo+ID4KPiA+ Cj4gPiBPbiAxLjMuMjAxNiAxODozOCwgTmlyIFNvZmZlciB3cm90ZToKPiA+Cj4gPiBPbiBUdWUs IE1hciAxLCAyMDE2IGF0IDU6MDcgUE0sIHBhZjFAZW1haWwuY3ogPG1haWx0bzpwYWYxQGVtYWls LmN6PiAKPiA8cGFmMUBlbWFpbC5jeiA8bWFpbHRvOnBhZjFAZW1haWwuY3o+PiB3cm90ZToKPiA+ Pgo+ID4+IEhlbGxvLCAgY2FuIGFueWJvZHkgIGV4cGxhaW4gdGhpcyBlcnJvciBuby4xMyAoIG9w ZW4gZmlsZSApIGluIAo+IHNhbmxvY2subG9nIC4KPiA+Cj4gPgo+ID4gVGhpcyBpcyBFQUNDRVMK PiA+Cj4gPiBDYW4geW91IHNoYXJlIHRoZSBvdXRvcHV0IG9mOgo+ID4KPiA+ICAgICBscyAtbGgg L3JoZXYvZGF0YS1jZW50ZXIvbW50LzxzZXJ2ZXI+OjxfcGF0aD4vPHNkX3V1aWQ+L2RvbV9tZAo+ ID4KPiA+Pgo+ID4+Cj4gPj4gVGhlIHNpemUgb2YgICJpZHMiIGZpbGUgaXMgemVybyAoMCkKPiA+ Cj4gPgo+ID4gVGhpcyBpcyBob3cgd2UgY3JlYXRlIHRoZSBpZHMgZmlsZSB3aGVuIGluaXRpYWxp emluZyBpdC4KPiA+Cj4gPiBCdXQgdGhlbiB3ZSB1c2Ugc2FubG9jayB0byBpbml0aWFsaXplIHRo ZSBpZHMgZmlsZSwgYW5kIGl0IHNob3VsZCBiZSAKPiAxTWlCIGFmdGVyIHRoYXQuCj4gPgo+ID4g SXMgdGhpcyBpZHMgZmlsZXMgY3JlYXRlZCBieSB2ZHNtLCBvciBvbmUgeW91IGNyZWF0ZWQgeW91 cnNlbGY/Cj4gPgo+ID4+Cj4gPj4gMjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1 MV06IG9wZW4gZXJyb3IgLTEzIAo+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9j YWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2Rv bV9tZC9pZHMKPiA+PiAyMDE2LTAyLTI4IDAzOjI1OjQ2KzAxMDAgMjY5NjI2IFsxOTUxXTogczE4 Nzk4NSBvcGVuX2Rpc2sgCj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhv c3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21k L2lkcyAKPiBlcnJvciAtMTMKPiA+PiAyMDE2LTAyLTI4IDAzOjI1OjU2KzAxMDAgMjY5NjM2IFsx MTMwNF06IHMxODc5OTIgbG9ja3NwYWNlIAo+IDdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0 NDMyNzEyNDoxOi9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZN MTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM6MAo+ ID4+Cj4gPj4gSWYgdGhlIG1haW4gcHJvYmxlbSBpcyBhYm91dCB6ZXJvIGZpbGUgc2l6ZSwgY2Fu IEkgcmVnZW5lcmF0ZSAgdGhpcyAKPiBmaWxlIG9ubGluZSBzZWN1cmVseSAsIHdpdGggbm8gVk0g ZGVwZW5kZW5jZSAgPz8/Pwo+ID4KPiA+Cj4gPiBZZXMsIEkgdGhpbmsgSSBhbHJlYWR5IHJlZmVy cmVkIHRvIHRoZSBpbnN0cnVjdGlvbnMgaG93IHRvIGRvIHRoYXQgCj4gaW4gYSBwcmV2aW91cyBt YWlsLgo+ID4KPiA+Pgo+ID4+Cj4gPj4gZGlzdCA9IFJIRUwgLSA3IC0gMi4xNTExCj4gPj4ga2Vy bmVsID0gMy4xMC4wIC0gMzI3LjEwLjEuZWw3Lng4Nl82NAo+ID4+IEtWTSA9IDIuMy4wIC0gMjku MS5lbDcKPiA+PiBsaWJ2aXJ0ID0gbGlidmlydC0xLjIuMTctMTMuZWw3XzIuMwo+ID4+IHZkc20g PSB2ZHNtLTQuMTYuMzAtMC5lbDcKPiA+PiBHbHVzdGVyRlMgPSBnbHVzdGVyZnMtMy43LjgtMS5l bDcKPiA+Pgo+ID4+Cj4gPj4gcmVncy4KPiA+PiBQYXZlbAo+ID4+Cj4gPj4gX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KPiA+PiBVc2VycyBtYWlsaW5nIGxp c3QKPiA+PiBVc2Vyc0BvdmlydC5vcmcgPG1haWx0bzpVc2Vyc0BvdmlydC5vcmc+Cj4gPj4gaHR0 cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzCj4gPj4KPiA+Cj4gPgoK Ci0tLS0tLS0tLS0tLS0tMDMwNDA5MDEwMjA2MDIwODA2MDUwOTA3CkNvbnRlbnQtVHlwZTogdGV4 dC9odG1sOyBjaGFyc2V0PXV0Zi04CkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQKCjxo dG1sPgogIDxoZWFkPgogICAgPG1ldGEgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04 IiBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogIDwvaGVhZD4KICA8Ym9keSB0ZXh0PSIjMDAw MDAwIiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIDxicj4KICAgIDxicj4KICAgIDxkaXYgY2xhc3M9 Im1vei1jaXRlLXByZWZpeCI+T24gMDMvMDIvMjAxNiAwMzo0NSBBTSwgTmlyIFNvZmZlcgogICAg ICB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQU1SYnl5 dTlnd1BmVnBQeHBEYTRfZ0tXeVhxMVBhdlRtMlYyckcyY1UwQXZFPUpKUEFAbWFpbC5nbWFpbC5j b20iCiAgICAgIHR5cGU9ImNpdGUiPgogICAgICA8ZGl2IGRpcj0ibHRyIj5PbiBUdWUsIE1hciAx LCAyMDE2IGF0IDEwOjUxIFBNLCA8YQogICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIiBo cmVmPSJtYWlsdG86cGFmMUBlbWFpbC5jeiI+PGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZp YXRlZCIgaHJlZj0ibWFpbHRvOnBhZjFAZW1haWwuY3oiPnBhZjFAZW1haWwuY3o8L2E+PC9hPgog ICAgICAgICZsdDs8YSBtb3otZG8tbm90LXNlbmQ9InRydWUiIGhyZWY9Im1haWx0bzpwYWYxQGVt YWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiZndDsKICAgICAgICB3cm90ZTo8YnI+CiAgICAgICAg Jmd0Ozxicj4KICAgICAgICAmZ3Q7IEhJLDxicj4KICAgICAgICAmZ3Q7IHJlcXVlc3RlZCBvdXRw dXQ6PGJyPgogICAgICAgICZndDs8YnI+CiAgICAgICAgJmd0OyAjIGxzIC1saAogICAgICAgIC9y aGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0OiovKi9kb21fbWQ8YnI+CiAg ICAgICAgJmd0OyDCoDxicj4KICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVz dGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItQkNLLzBmY2FkODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBi N2E5OWZiNy9kb21fbWQ6PGJyPgogICAgICAgICZndDsgdG90YWwgMiwxTTxicj4KICAgICAgICAm Z3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAxLDBNIMKgMS4gYsWZZSAyMS4yOCBpZHMgwqAgwqAg wqAgwqAmbHQ7LS0KICAgICAgICBnb29kPGJyPgogICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZk c20ga3ZtIMKgMTZNIMKgNy4gbGlzIDIyLjE2IGluYm94PGJyPgogICAgICAgICZndDsgLXJ3LXJ3 LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMjIuMTcgbGVhc2VzPGJyPgogICAgICAgICZn dDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgMzM1IMKgNy4gbGlzIDIyLjE3IG1ldGFkYXRhPGJy PgogICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDIyLjE2 IG91dGJveDxicj4KICAgICAgICAmZ3Q7PGJyPgogICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50 ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QMS81NTNkOWI5Mi1lNGEwLTQwNDIt YTU3OS00Y2FiZWI1NWRlZDQvZG9tX21kOjxicj4KICAgICAgICAmZ3Q7IHRvdGFsIDEsMU08YnI+ CiAgICAgICAgJmd0OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjQx IGlkcyDCoCDCoCDCoCDCoCZsdDstLQogICAgICAgIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUs IG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDC oDE2TSDCoDcuIGxpcyAwMC4xNCBpbmJveDxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2 ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDAzLjU2IGxlYXNlczxicj4KICAgICAgICAmZ3Q7IC1ydy1y LS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAwMy41NiBtZXRhZGF0YTxicj4KICAgICAg ICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAwMC4xNCBvdXRib3g8 YnI+CiAgICAgICAgJmd0Ozxicj4KICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9n bHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDIvODhhZGJkNDktNjJkNi00NWIxLTk5OTItYjA0 NDY0YTA0MTEyL2RvbV9tZDo8YnI+CiAgICAgICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAg ICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgIMKgMCAyNC4gw7pubyAwNy40MyBpZHMgwqAg wqAgwqAgwqAmbHQ7LS0KICAgICAgICBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBj YW4gcmVhZCk8YnI+CiAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3 LiBsaXMgMDAuMTUgaW5ib3g8YnI+CiAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0g MiwwTSDCoDcuIGxpcyAyMi4xNCBsZWFzZXM8YnI+CiAgICAgICAgJmd0OyAtcnctci0tci0tIDEg dmRzbSBrdm0gwqAzMzMgwqA3LiBsaXMgMjIuMTQgbWV0YWRhdGE8YnI+CiAgICAgICAgJmd0OyAt cnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTUgb3V0Ym94PGJyPgogICAg ICAgICZndDs8YnI+CiAgICAgICAgJmd0Owovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNE L2xvY2FsaG9zdDpfMUtWTTEyLVAzLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0 Ny9kb21fbWQ6PGJyPgogICAgICAgICZndDsgdG90YWwgMSwxTTxicj4KICAgICAgICAmZ3Q7IC1y dy1yLS1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuNDMgaWRzIMKgIMKgIMKgIMKg Jmx0Oy0tCiAgICAgICAgYmFkIChzYW5sb2NrIGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQp PGJyPgogICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIDIzLiDDum5vIDIy LjUxIGluYm94PGJyPgogICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gMjMu IMO6bm8gMjMuMTIgbGVhc2VzPGJyPgogICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3Zt IMKgOTk4IDI1LiDDum5vIDAwLjM1IG1ldGFkYXRhPGJyPgogICAgICAgICZndDsgLXJ3LXJ3LS0t LSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE2IG91dGJveDxicj4KICAgICAgICAmZ3Q7 PGJyPgogICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhv c3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21k Ojxicj4KICAgICAgICAmZ3Q7IHRvdGFsIDEsMU08YnI+CiAgICAgICAgJmd0OyAtcnctci0tci0t IDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjQ0IGlkcyDCoCDCoCDCoCDCoCZsdDstLQog ICAgICAgIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKTxicj4KICAg ICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAwMC4xNyBpbmJv eDxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDAw LjE4IGxlYXNlczxicj4KICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDC oDcuIGxpcyAwMC4xOCBtZXRhZGF0YTxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNt IGt2bSDCoDE2TSDCoDcuIGxpcyAwMC4xNyBvdXRib3g8YnI+CiAgICAgICAgJmd0Ozxicj4KICAg ICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZN MTItUDEvNDJkNzEwYTktYjg0NC00M2RjLWJlNDEtNzcwMDJkMWNkNTUzL2RvbV9tZDo8YnI+CiAg ICAgICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20g a3ZtIMKgIMKgMCAyNC4gw7pubyAwNy4zMiBpZHMgwqAgwqAgwqAgwqAmbHQ7LS0KICAgICAgICBi YWQgKG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2 bSDCoDE2TSDCoDcuIGxpcyAyMi4xOCBpbmJveDxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0g MSB2ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDIyLjE4IGxlYXNlczxicj4KICAgICAgICAmZ3Q7IC1y dy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAyMi4xOCBtZXRhZGF0YTxicj4KICAg ICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4xOCBvdXRi b3g8YnI+CiAgICAgICAgJmd0Ozxicj4KICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21u dC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDIvZmY3MWI0N2ItMGY3Mi00NTI4LTliZmUt YzNkYTg4OGU0N2YwL2RvbV9tZDo8YnI+CiAgICAgICAgJmd0OyB0b3RhbCAzLDBNPGJyPgogICAg ICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDEsME0gwqAxLiBixZllIDIxLjI4IGlkc8Kg wqAgwqAgwqAgwqAmbHQ7LS0KICAgICAgICBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4KICAgICAg ICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAyNS4gw7pubyAwMC40MiBpbmJveMKg PGJyPgogICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gMjUuIMO6bm8gMDAu NDQgbGVhc2VzPGJyPgogICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk3IDI0 LiDDum5vIDAyLjQ2IG1ldGFkYXRhPGJyPgogICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20g a3ZtIMKgMTZNIDI1LiDDum5vIDAwLjQ0IG91dGJveDxicj4KICAgICAgICAmZ3Q7PGJyPgogICAg ICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0x Mi1QMy9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRlY2I0YjQvZG9tX21kOjxicj4KICAg ICAgICAmZ3Q7IHRvdGFsIDIsMU08YnI+CiAgICAgICAgJmd0OyAtcnctci0tci0tIDEgdmRzbSBr dm0gwqAgwqAwIDI0LiDDum5vIDA3LjM0IGlkc8KgwqAgwqAgwqAgwqAmbHQ7LS0KICAgICAgICBi YWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+CiAgICAgICAgJmd0 OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gMjMuIMO6bm8gMjIuMzUgaW5ib3g8YnI+CiAg ICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4gw7pubyAyMi4zOCBsZWFz ZXM8YnI+CiAgICAgICAgJmd0OyAtcnctci0tci0tIDEgdmRzbSBrdm0gMSwxSyAyNC4gw7pubyAx OS4wNyBtZXRhZGF0YTxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2 TSAyMy4gw7pubyAyMi4yNyBvdXRib3g8YnI+CiAgICAgICAgJmd0Ozxicj4KICAgICAgICAmZ3Q7 Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTJfX1A0LzMw MGU5YWM4LTNjMmYtNDcwMy05YmIxLTFkZjIxMzBjN2M5Ny9kb21fbWQ6PGJyPgogICAgICAgICZn dDsgdG90YWwgMywwTTxicj4KICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAxLDBN IMKgMS4gYsWZZSAyMS4yOCBpZHPCoMKgIMKgIMKgIMKgJmx0Oy0tCiAgICAgICAgYmFkIChvdGhl ciBjYW4gcmVhZCk8YnI+CiAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAxNk0g wqA2LiBsaXMgMjMuNTAgaW5ib3jCoMKgIMKgIMKgCiAgICAgICAgwqAmbHQ7LS0gYmFkIChvdGhl ciBjYW4gcmVhZCkKICAgICAgICA8ZGl2PiZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0g wqA2LiBsaXMgMjMuNTEgbGVhc2VzwqDCoCDCoCDCoAogICAgICAgICAgwqAmbHQ7LS0gYmFkIChv dGhlciBjYW4gcmVhZCk8YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSDC oDczNCDCoDcuIGxpcyAwMi4xMyBtZXRhZGF0YcKgwqAgwqAgwqAKICAgICAgICAgIMKgJmx0Oy0t IGJhZCAoZ3JvdXAgY2FuIHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+CiAgICAgICAgICAmZ3Q7 IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDYuIGxpcyAxNi41NSBvdXRib3jCoMKgIMKg IMKgCiAgICAgICAgICDCoCZsdDstLSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAg ICZndDs8YnI+CiAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0Qv bG9jYWxob3N0Ol8yS1ZNMTItUDUvMWNhNTZiNDUtNzAxZS00YzIyLTlmNTktM2FlYmVhNGQ4NDc3 L2RvbV9tZDo8YnI+CiAgICAgICAgICAmZ3Q7IHRvdGFsIDEsMU08YnI+CiAgICAgICAgICAmZ3Q7 IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzUgaWRzwqDCoCDCoCDC oAogICAgICAgICAgwqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVhZCk8YnI+CiAgICAgICAgICAm Z3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSDCoDE2TSAyNC4gw7pubyAwMS4wNiBpbmJveDxicj4K ICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gMjQuIMO6bm8gMDIuNDQg bGVhc2VzPGJyPgogICAgICAgICAgJmd0OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqA5OTggMjQu IMO6bm8gMTkuMDcgbWV0YWRhdGE8YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNt IGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4yMCBvdXRib3g8YnI+CiAgICAgICAgICA8YnI+CiAgICAg ICAgICA8YnI+CiAgICAgICAgICBJdCBzaG91bGQgbG9vayBsaWtlIHRoaXM6PGJyPgogICAgICAg ICAgPGJyPgogICAgICAgICAgLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAxLjBNIE1hciDCoDEgMjM6 MzYgaWRzPGJyPgogICAgICAgICAgLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAyLjBNIE1hciDCoDEg MjM6MzUgbGVhc2VzPGJyPgogICAgICAgICAgLXJ3LXItLXItLS4gMSB2ZHNtIGt2bSDCoDM1MyBN YXIgwqAxIDIzOjM1IG1ldGFkYXRhPGJyPgogICAgICAgICAgLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2 bSDCoDE2TSBNYXIgwqAxIDIzOjM0IG91dGJveDxicj4KICAgICAgICAgIC1ydy1ydy0tLS0uIDEg dmRzbSBrdm0gwqAxNk0gTWFyIMKgMSAyMzozNCBpbmJveDxicj4KICAgICAgICAgIDxicj4KICAg ICAgICAgIFRoaXMgZXhwbGFpbnMgdGhlIEVBQ0NFUyBlcnJvci48YnI+CiAgICAgICAgICA8YnI+ CiAgICAgICAgICBZb3UgY2FuIHN0YXJ0IGJ5IGZpeGluZyB0aGUgcGVybWlzc2lvbnMgbWFudWFs bHksIHlvdSBjYW4gZG8KICAgICAgICAgIHRoaXMgb25saW5lLjxicj4KICAgICAgICAgIMKgPGJy PgogICAgICAgICAgJmd0OyDCoFRoZSBpZHMgZmlsZXMgd2FzIGdlbmVyYXRlZCBieSAidG91Y2gi IGNvbW1hbmQgYWZ0ZXIKICAgICAgICAgIGRlbGV0aW5nIHRoZW0gZHVlICJzYW5sb2NrIGxvY2tp bmcgaGFuZyIgwqBnbHVzdGVyIGNyYXNoICZhbXA7CiAgICAgICAgICByZWJvb3Q8YnI+CiAgICAg ICAgICAmZ3Q7IEkgZXhwZWN0ZWQgdGhhdCB0aGV5IHdpbGwgYmUgZmlsbGVkIGF1dG9tYXRpY2Fs eSBhZnRlcgogICAgICAgICAgZ2x1c3RlciByZWJvb3QgKCB0aGUgwqBzaGFkb3cgY29weSBmcm9t IMKgICIuZ2x1c3RlciAiIMKgCiAgICAgICAgICBkaXJlY3RvcnkgwqB3YXMgZGVsZXRlZCAmYW1w OyBjcmVhdGVkIGVtcHR5IMKgdG9vICk8YnI+CiAgICAgICAgICA8YnI+CiAgICAgICAgICBJIGRv bid0IGtub3cgYWJvdXQgZ2x1c3RlciBzaGFkb3cgY29weSwgSSB3b3VsZCBub3QgcGxheSB3aXRo CiAgICAgICAgICBnbHVzdGVyIGludGVybmFscy48L2Rpdj4KICAgICAgICA8ZGl2PkFkZGluZyBT YWhpbmEgZm9yIGFkdmljZS48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgIDwvZGl2PgogICAgPC9i bG9ja3F1b3RlPgogICAgPGJyPgogICAgRGlkIHlvdSBnZW5lcmF0ZSB0aGUgaWRzIGZpbGUgb24g dGhlIG1vdW50IHBvaW50Ljxicj4KICAgIDxicj4KICAgIFJhdmksIGNhbiB5b3UgaGVscCBoZXJl Pzxicj4KICAgIDxicj4KICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQU1SYnl5dTlnd1BmVnBQ eHBEYTRfZ0tXeVhxMVBhdlRtMlYyckcyY1UwQXZFPUpKUEFAbWFpbC5nbWFpbC5jb20iCiAgICAg IHR5cGU9ImNpdGUiPgogICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICA8ZGl2Pjxicj4KICAg ICAgICAgICZndDsgT0ssIGl0IGxvb2tzIHRoYXQgc2FubG9jayDCoGNhbid0IHdvcmsgd2l0aCBl bXB0eSBmaWxlIG9yCiAgICAgICAgICByZXdyaXRlIHRoZW0gLjxicj4KICAgICAgICAgICZndDsg QW0gSSByaWdodCA/Pzxicj4KICAgICAgICAgIDxicj4KICAgICAgICAgIFllcywgdGhlIGZpbGVz IG11c3QgYmUgaW5pdGlhbGl6ZWQgYmVmb3JlIHNhbmxvY2sgY2FuIHVzZQogICAgICAgICAgdGhl bS48YnI+CiAgICAgICAgICA8YnI+CiAgICAgICAgICBZb3UgY2FuIGluaXRpYWxpemUgdGhlIGZp bGUgbGlrZSB0aGlzOjxicj4KICAgICAgICAgIDxicj4KICAgICAgICAgIHNhbmxvY2sgZGlyZWN0 IGluaXQgLXMKICAgICAgICAgICZsdDtzZF91dWlkJmd0OzowOnJlcGFpci8mbHQ7c2RfdXVpZCZn dDsvZG9tX21kL2lkczowPGJyPgogICAgICAgICAgPGJyPgogICAgICAgICAgVGFrZW4gZnJvbSA8 YSBtb3otZG8tbm90LXNlbmQ9InRydWUiCiAgICAgICAgICAgIGhyZWY9Imh0dHA6Ly9saXN0cy5v dmlydC5vcmcvcGlwZXJtYWlsL3VzZXJzLzIwMTYtRmVicnVhcnkvMDM4MDQ2Lmh0bWwiPmh0dHA6 Ly9saXN0cy5vdmlydC5vcmcvcGlwZXJtYWlsL3VzZXJzLzIwMTYtRmVicnVhcnkvMDM4MDQ2Lmh0 bWw8L2E+PGJyPgogICAgICAgICAgPGJyPgogICAgICAgICAgJmd0OyBUaGUgbGFzdCBwb2ludCAt IGFib3V0ICJpZHMiIHdvcmthcm91bmQgLSB0aGlzIGlzIG9mZmxpbmUKICAgICAgICAgIHZlcnNp b24gPSBWTXMgaGF2ZSB0byBiZSBtb3ZlZCBvdXQgZnJvbSBmb3IgY29udGludWFsIHJ1bm5pbmcK ICAgICAgICAgIHdpdGggbWFpbnRlbmFuY2Ugdm9sdW1lIG1vZGU8YnI+CiAgICAgICAgICAmZ3Q7 IEJ1dCB0aGlzIGlzIG5vdCBhY2NlcHRhYmxlIGluIGN1cnJlbnQgc2l0dWF0aW9uLCBzbyB0aGUK ICAgICAgICAgIHF1ZXN0aW9uIGFnYWluLCDCoGlzIGl0IHNhZmUgdG8gZG8gaXQgb25saW5lID8/ IMKgKCBZRVMgLyBOTyApPC9kaXY+CiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAg ICAgICAgPGRpdj5UaGUgaWRzIGZpbGUgaXMgYWNjZXNzZWQgb25seSBieSBzYW5sb2NrLiBJIGd1 ZXNzIHRoYXQgeW91CiAgICAgICAgICBkb24ndCBoYXZlIGEgcnVubmluZzwvZGl2PgogICAgICAg IDxkaXY+U1BNIG9uIHRoaXMgREMsIHNpbmNlIHNhbmxvY2sgZmFpbHMgdG8gYWNxdWlyZSBhIGhv c3QgaWQsCiAgICAgICAgICBzbyB5b3UgYXJlIHByZXR0eSBzYWZlPC9kaXY+CiAgICAgICAgPGRp dj50byBmaXggdGhlIHBlcm1pc3Npb25zIGFuZCBpbml0aWFsaXplIHRoZSBpZHMgZmlsZXMuPC9k aXY+CiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRpdj5JIHdvdWxk IGRvIHRoaXM6PC9kaXY+CiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAg PGRpdj4xLiBTdG9wIGVuZ2luZSwgwqBzbyBpdCB3aWxsIG5vdCB0cnkgdG8gc3RhcnQgdmRzbTwv ZGl2PgogICAgICAgIDxkaXY+Mi4gU3RvcCB2ZHNtIG9uIGFsbCBob3N0cywgc28gdGhleSBkbyBu b3QgdHJ5IHRvIGFjcXVpcmUgYQogICAgICAgICAgaG9zdCBpZCB3aXRoIHNhbmxvY2s8L2Rpdj4K ICAgICAgICA8ZGl2PsKgIMKgIFRoaXMgZG9lcyBub3QgYWZmZWN0IHJ1bm5pbmcgdm1zPC9kaXY+ CiAgICAgICAgPGRpdj4zLiBGaXggdGhlIHBlcm1pc3Npb25zIG9uIHRoZSBpZHMgZmlsZSwgdmlh IGdsdXN0ZXJmcyBtb3VudDwvZGl2PgogICAgICAgIDxkaXY+NC4gSW5pdGlhbGl6ZSB0aGUgaWRz IGZpbGVzIGZyb20gb25lIG9mIHRoZSBob3N0cywgdmlhIHRoZQogICAgICAgICAgZ2x1c3RlcmZz IG1vdW50PC9kaXY+CiAgICAgICAgPGRpdj7CoCDCoCBUaGlzIHNob3VsZCBmaXggdGhlIGlkcyBm aWxlcyBvbiBhbGwgcmVwbGljYXM8L2Rpdj4KICAgICAgICA8ZGl2PjUuIFN0YXJ0IHZkc20gb24g YWxsIGhvc3RzPC9kaXY+CiAgICAgICAgPGRpdj42LiBTdGFydCBlbmdpbmU8L2Rpdj4KICAgICAg ICA8ZGl2Pjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2PkVuZ2luZSB3aWxsIGNvbm5l Y3QgdG8gYWxsIGhvc3RzLCBob3N0cyB3aWxsIGNvbm5lY3QgdG8KICAgICAgICAgIHN0b3JhZ2Ug YW5kIHRyeSB0byBhY3F1aXJlIGEgaG9zdCBpZC48L2Rpdj4KICAgICAgICA8ZGl2PlRoZW4gRW5n aW5lIHdpbGwgc3RhcnQgdGhlIFNQTSBvbiBvbmUgb2YgdGhlIGhvc3RzLCBhbmQKICAgICAgICAg IHlvdXIgREMgc2hvdWxkIGJlY29tZSB1cC48L2Rpdj4KICAgICAgICA8ZGl2Pjxicj4KICAgICAg ICA8L2Rpdj4KICAgICAgICA8ZGl2PkRhdmlkLCBTYWhpbmEsIGNhbiB5b3UgY29uZmlybSB0aGF0 IHRoaXMgcHJvY2VkdXJlIGlzIHNhZmU/PC9kaXY+CiAgICAgIDwvZGl2PgogICAgPC9ibG9ja3F1 b3RlPgogICAgPGJyPgogICAgWWVzLCBjb3JyZWN0aW5nIGZyb20gdGhlIG1vdW50IHBvaW50IHNo b3VsZCBmaXggaXQgb24gYWxsIHJlcGxpY2FzPGJyPgogICAgPGJyPgogICAgPGJyPgogICAgPGJs b2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVJieXl1OWd3UGZWcFB4cERhNF9nS1d5WHExUGF2VG0yVjJy RzJjVTBBdkU9SkpQQUBtYWlsLmdtYWlsLmNvbSIKICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgIDxk aXYgZGlyPSJsdHIiPgogICAgICAgIDxkaXY+PGJyPgogICAgICAgIDwvZGl2PgogICAgICAgIDxk aXY+TmlyPC9kaXY+CiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRp dj4mZ3Q7PGJyPgogICAgICAgICAgJmd0OyByZWdzLjxicj4KICAgICAgICAgICZndDsgUGF2ZWw8 YnI+CiAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICZndDs8 YnI+CiAgICAgICAgICAmZ3Q7IE9uIDEuMy4yMDE2IDE4OjM4LCBOaXIgU29mZmVyIHdyb3RlOjxi cj4KICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAmZ3Q7IE9uIFR1ZSwgTWFyIDEsIDIwMTYg YXQgNTowNyBQTSwgPGEKICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIiBocmVmPSJt YWlsdG86cGFmMUBlbWFpbC5jeiI+PGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIg aHJlZj0ibWFpbHRvOnBhZjFAZW1haWwuY3oiPnBhZjFAZW1haWwuY3o8L2E+PC9hPgogICAgICAg ICAgJmx0OzxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgaHJlZj0ibWFpbHRvOnBhZjFAZW1haWwu Y3oiPnBhZjFAZW1haWwuY3o8L2E+Jmd0OwogICAgICAgICAgd3JvdGU6PGJyPgogICAgICAgICAg Jmd0OyZndDs8YnI+CiAgICAgICAgICAmZ3Q7Jmd0OyBIZWxsbywgwqBjYW4gYW55Ym9keSDCoGV4 cGxhaW4gdGhpcyBlcnJvciBuby4xMyAoIG9wZW4KICAgICAgICAgIGZpbGUgKSBpbiBzYW5sb2Nr LmxvZyAuPGJyPgogICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICZndDs8YnI+CiAgICAgICAg ICAmZ3Q7IFRoaXMgaXMgRUFDQ0VTPGJyPgogICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICZn dDsgQ2FuIHlvdSBzaGFyZSB0aGUgb3V0b3B1dCBvZjo8YnI+CiAgICAgICAgICAmZ3Q7PGJyPgog ICAgICAgICAgJmd0OyDCoCDCoCBscyAtbGgKL3JoZXYvZGF0YS1jZW50ZXIvbW50LyZsdDtzZXJ2 ZXImZ3Q7OiZsdDtfcGF0aCZndDsvJmx0O3NkX3V1aWQmZ3Q7L2RvbV9tZDxicj4KICAgICAgICAg ICZndDsgwqA8YnI+CiAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICZndDsmZ3Q7PGJy PgogICAgICAgICAgJmd0OyZndDsgVGhlIHNpemUgb2YgwqAiaWRzIiBmaWxlIGlzIHplcm8gKDAp PGJyPgogICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAmZ3Q7 IFRoaXMgaXMgaG93IHdlIGNyZWF0ZSB0aGUgaWRzIGZpbGUgd2hlbiBpbml0aWFsaXppbmcgaXQu PGJyPgogICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICZndDsgQnV0IHRoZW4gd2UgdXNlIHNh bmxvY2sgdG8gaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGUsIGFuZAogICAgICAgICAgaXQgc2hvdWxk IGJlIDFNaUIgYWZ0ZXIgdGhhdC48YnI+CiAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgJmd0 OyBJcyB0aGlzIGlkcyBmaWxlcyBjcmVhdGVkIGJ5IHZkc20sIG9yIG9uZSB5b3UgY3JlYXRlZAog ICAgICAgICAgeW91cnNlbGY/PGJyPgogICAgICAgICAgJmd0OyDCoDxicj4KICAgICAgICAgICZn dDsmZ3Q7PGJyPgogICAgICAgICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2 OTYyNiBbMTk1MV06IG9wZW4gZXJyb3IKICAgICAgICAgIC0xMwovcmhldi9kYXRhLWNlbnRlci9t bnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFh LTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzPGJyPgogICAgICAgICAgJmd0OyZndDsgMjAxNi0wMi0y OCAwMzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IHMxODc5ODUKICAgICAgICAgIG9wZW5fZGlz awogICAgICAgICAgL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFL Vk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkcwog ICAgICAgICAgZXJyb3IgLTEzPGJyPgogICAgICAgICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoy NTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBzMTg3OTkyCiAgICAgICAgICBsb2Nrc3BhY2UKN2Y1 MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0OjE6L3JoZXYvZGF0YS1jZW50ZXIvbW50 L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0x MDJkNDQzMjcxMjQvZG9tX21kL2lkczowPGJyPgogICAgICAgICAgJmd0OyZndDs8YnI+CiAgICAg ICAgICAmZ3Q7Jmd0OyBJZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXpl LCBjYW4gSQogICAgICAgICAgcmVnZW5lcmF0ZSDCoHRoaXMgZmlsZSBvbmxpbmUgc2VjdXJlbHkg LCB3aXRoIG5vIFZNIGRlcGVuZGVuY2UKICAgICAgICAgIMKgPz8/Pzxicj4KICAgICAgICAgICZn dDs8YnI+CiAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgJmd0OyBZZXMsIEkgdGhpbmsgSSBh bHJlYWR5IHJlZmVycmVkIHRvIHRoZSBpbnN0cnVjdGlvbnMgaG93CiAgICAgICAgICB0byBkbyB0 aGF0IGluIGEgcHJldmlvdXMgbWFpbC48YnI+CiAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAg Jmd0OyZndDs8YnI+CiAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICZndDsmZ3Q7IGRp c3QgPSBSSEVMIC0gNyAtIDIuMTUxMTxicj4KICAgICAgICAgICZndDsmZ3Q7IGtlcm5lbCA9IDMu MTAuMCAtIDMyNy4xMC4xLmVsNy54ODZfNjQ8YnI+CiAgICAgICAgICAmZ3Q7Jmd0OyBLVk0gPSAy LjMuMCAtIDI5LjEuZWw3PGJyPgogICAgICAgICAgJmd0OyZndDsgbGlidmlydCA9IGxpYnZpcnQt MS4yLjE3LTEzLmVsN18yLjM8YnI+CiAgICAgICAgICAmZ3Q7Jmd0OyB2ZHNtID0gdmRzbS00LjE2 LjMwLTAuZWw3PGJyPgogICAgICAgICAgJmd0OyZndDsgR2x1c3RlckZTID0gZ2x1c3RlcmZzLTMu Ny44LTEuZWw3PGJyPgogICAgICAgICAgJmd0OyZndDs8YnI+CiAgICAgICAgICAmZ3Q7Jmd0Ozxi cj4KICAgICAgICAgICZndDsmZ3Q7IHJlZ3MuPGJyPgogICAgICAgICAgJmd0OyZndDsgUGF2ZWw8 YnI+CiAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICZndDsmZ3Q7IF9fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPGJyPgogICAgICAgICAgJmd0OyZn dDsgVXNlcnMgbWFpbGluZyBsaXN0PGJyPgogICAgICAgICAgJmd0OyZndDsgPGEgbW96LWRvLW5v dC1zZW5kPSJ0cnVlIgogICAgICAgICAgICBocmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIj5V c2Vyc0BvdmlydC5vcmc8L2E+PGJyPgogICAgICAgICAgJmd0OyZndDsgPGEgbW96LWRvLW5vdC1z ZW5kPSJ0cnVlIgogICAgICAgICAgICBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxt YW4vbGlzdGluZm8vdXNlcnMiPmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5m by91c2VyczwvYT48YnI+CiAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICZndDs8YnI+ CiAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgIDwvZGl2PgogICAgICA8L2Rpdj4KICAgIDwvYmxv Y2txdW90ZT4KICAgIDxicj4KICA8L2JvZHk+CjwvaHRtbD4KCi0tLS0tLS0tLS0tLS0tMDMwNDA5 MDEwMjA2MDIwODA2MDUwOTA3LS0K --===============0844145120624418695==-- From ravishankar at redhat.com Wed Mar 2 02:17:01 2016 Content-Type: multipart/mixed; boundary="===============4670495215605840215==" MIME-Version: 1.0 From: Ravishankar N To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Wed, 02 Mar 2016 12:46:53 +0530 Message-ID: <56D69365.4090303@redhat.com> In-Reply-To: 56D68910.8040602@redhat.com --===============4670495215605840215== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------050402010809060509080406 Content-Type: text/plain; charset=3DUTF-8; format=3Dflowed Content-Transfer-Encoding: 8bit On 03/02/2016 12:02 PM, Sahina Bose wrote: > > > On 03/02/2016 03:45 AM, Nir Soffer wrote: >> On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz > > wrote: >> > >> > HI, >> > requested output: >> > >> > # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be= -bef3-0bc0b7a99fb7/dom_md: >> > total 2,1M >> > -rw-rw---- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- good >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.17 leases >> > -rw-r--r-- 1 vdsm kvm 335 7. lis 22.17 metadata >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-= a579-4cabeb55ded4/dom_md: >> > total 1,1M >> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.41 ids <-- bad (sanlock = >> cannot write, other can read) >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 03.56 leases >> > -rw-r--r-- 1 vdsm kvm 333 7. lis 03.56 metadata >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-= 9992-b04464a04112/dom_md: >> > total 1,1M >> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock = >> cannot write, other can read) >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.14 leases >> > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.14 metadata >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-= ab46-084f3d70b147/dom_md: >> > total 1,1M >> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock = >> cannot write, other can read) >> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.51 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases >> > -rw-r--r-- 1 vdsm kvm 998 25. =C3=BAno 00.35 metadata >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.16 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-= 89aa-102d44327124/dom_md: >> > total 1,1M >> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.44 ids <-- bad (sanlock = >> cannot write, other can read) >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 00.18 leases >> > -rw-r--r-- 1 vdsm kvm 333 7. lis 00.18 metadata >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-= be41-77002d1cd553/dom_md: >> > total 1,1M >> > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.32 ids <-- bad (other can = read) >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.18 leases >> > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.18 metadata >> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-= 9bfe-c3da888e47f0/dom_md: >> > total 3,0M >> > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other can = read) >> > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.42 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases >> > -rw-r--r-- 1 vdsm kvm 997 24. =C3=BAno 02.46 metadata >> > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.44 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-= ba9a-e6d9bdecb4b4/dom_md: >> > total 2,1M >> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.34 ids <-- bad (sanlock = >> cannot write, other can read) >> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.35 inbox >> > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases >> > -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata >> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.27 outbox >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703= -9bb1-1df2130c7c97/dom_md: >> > total 3,0M >> > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other can = read) >> > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 23.50 inbox <-- bad (other can = >> read) >> > -rw-rw-r-- 1 vdsm kvm 2,0M 6. lis 23.51 leases <-- bad (other = >> can read) >> > -rw-rw-r-- 1 vdsm kvm 734 7. lis 02.13 metadata <-- bad (group = >> can write, other can read) >> > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 16.55 outbox <-- bad (other can = >> read) >> > >> > = >> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-= 9f59-3aebea4d8477/dom_md: >> > total 1,1M >> > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.35 ids <-- bad (other can = read) >> > -rw-rw-r-- 1 vdsm kvm 16M 24. =C3=BAno 01.06 inbox >> > -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases >> > -rw-r--r-- 1 vdsm kvm 998 24. =C3=BAno 19.07 metadata >> > -rw-rw-r-- 1 vdsm kvm 16M 7. lis 22.20 outbox >> >> >> It should look like this: >> >> -rw-rw----. 1 vdsm kvm 1.0M Mar 1 23:36 ids >> -rw-rw----. 1 vdsm kvm 2.0M Mar 1 23:35 leases >> -rw-r--r--. 1 vdsm kvm 353 Mar 1 23:35 metadata >> -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 outbox >> -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 inbox >> >> This explains the EACCES error. >> >> You can start by fixing the permissions manually, you can do this online. >> >> > The ids files was generated by "touch" command after deleting them = >> due "sanlock locking hang" gluster crash & reboot >> > I expected that they will be filled automaticaly after gluster = >> reboot ( the shadow copy from ".gluster " directory was deleted & = >> created empty too ) >> >> I don't know about gluster shadow copy, I would not play with gluster = >> internals. >> Adding Sahina for advice. > > Did you generate the ids file on the mount point. > > Ravi, can you help here? > Okay, so what I understand from the output above is you have different = gluster volumes mounted and some of them have incorrect permissions for = the 'ids' file. The way to fix it is to do it from the mount like Nir said. Why did you delete the file from the .glusterfs in the brick(s)? Was = there a gfid split brain? -Ravi >> >> > OK, it looks that sanlock can't work with empty file or rewrite them . >> > Am I right ?? >> >> Yes, the files must be initialized before sanlock can use them. >> >> You can initialize the file like this: >> >> sanlock direct init -s :0:repair//dom_md/ids:0 >> >> Taken from = >> http://lists.ovirt.org/pipermail/users/2016-February/038046.html >> >> > The last point - about "ids" workaround - this is offline version =3D = >> VMs have to be moved out from for continual running with maintenance = >> volume mode >> > But this is not acceptable in current situation, so the question = >> again, is it safe to do it online ?? ( YES / NO ) >> >> The ids file is accessed only by sanlock. I guess that you don't have = >> a running >> SPM on this DC, since sanlock fails to acquire a host id, so you are = >> pretty safe >> to fix the permissions and initialize the ids files. >> >> I would do this: >> >> 1. Stop engine, so it will not try to start vdsm >> 2. Stop vdsm on all hosts, so they do not try to acquire a host id = >> with sanlock >> This does not affect running vms >> 3. Fix the permissions on the ids file, via glusterfs mount >> 4. Initialize the ids files from one of the hosts, via the glusterfs = >> mount >> This should fix the ids files on all replicas >> 5. Start vdsm on all hosts >> 6. Start engine >> >> Engine will connect to all hosts, hosts will connect to storage and = >> try to acquire a host id. >> Then Engine will start the SPM on one of the hosts, and your DC = >> should become up. >> >> David, Sahina, can you confirm that this procedure is safe? > > Yes, correcting from the mount point should fix it on all replicas > > >> >> Nir >> >> > >> > regs. >> > Pavel >> > >> > >> > >> > On 1.3.2016 18:38, Nir Soffer wrote: >> > >> > On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz > > wrote: >> >> >> >> Hello, can anybody explain this error no.13 ( open file ) in = >> sanlock.log . >> > >> > >> > This is EACCES >> > >> > Can you share the outoput of: >> > >> > ls -lh /rhev/data-center/mnt/:<_path>//dom_md >> > >> >> >> >> >> >> The size of "ids" file is zero (0) >> > >> > >> > This is how we create the ids file when initializing it. >> > >> > But then we use sanlock to initialize the ids file, and it should = >> be 1MiB after that. >> > >> > Is this ids files created by vdsm, or one you created yourself? >> > >> >> >> >> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 = >> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-= 89aa-102d44327124/dom_md/ids >> >> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk = >> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-= 89aa-102d44327124/dom_md/ids = >> error -13 >> >> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace = >> 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/l= ocalhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 >> >> >> >> If the main problem is about zero file size, can I regenerate = >> this file online securely , with no VM dependence ???? >> > >> > >> > Yes, I think I already referred to the instructions how to do that = >> in a previous mail. >> > >> >> >> >> >> >> dist =3D RHEL - 7 - 2.1511 >> >> kernel =3D 3.10.0 - 327.10.1.el7.x86_64 >> >> KVM =3D 2.3.0 - 29.1.el7 >> >> libvirt =3D libvirt-1.2.17-13.el7_2.3 >> >> vdsm =3D vdsm-4.16.30-0.el7 >> >> GlusterFS =3D glusterfs-3.7.8-1.el7 >> >> >> >> >> >> regs. >> >> Pavel >> >> >> >> _______________________________________________ >> >> Users mailing list >> >> Users(a)ovirt.org >> >> http://lists.ovirt.org/mailman/listinfo/users >> >> >> > >> > > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users(a)gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users --------------050402010809060509080406 Content-Type: text/html; charset=3DUTF-8 Content-Transfer-Encoding: 8bit

On 03/02/2016 12:02 PM, Sahina Bose wrote:

On 03/02/2016 03:45 AM, Nir Soffer wrote:

On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz <paf1(a)emai= l.cz> wrote:
>
> HI,
> requested output:
>
> # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md
> =C2=A0
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-be= f3-0bc0b7a99fb7/dom_md:
> total 2,1M
> -rw-rw---- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids =C2= =A0 =C2=A0 =C2=A0 =C2=A0<-- good
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.17 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0335 =C2=A07. lis 22.17 metadata<= br> > -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a57= 9-4cabeb55ded4/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.41 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 03.56 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 03.56 metadata<= br> > -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-999= 2-b04464a04112/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.14 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.14 metadata<= br> > -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-ab4= 6-084f3d70b147/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.51 inbox
> -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 25. =C3=BAno 00.35 metadata<= br> > -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.16 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.44 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 00.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 00.18 metadata<= br> > -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-be4= 1-77002d1cd553/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.32 ids = =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 inbox
> -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.18 metadata<= br> > -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9bf= e-c3da888e47f0/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.42 inbox=C2= =A0
> -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0997 24. =C3=BAno 02.46 metadata<= br> > -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.44 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-ba9= a-e6d9bdecb4b4/dom_md:
> total 2,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.34 ids= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.35 inbox
> -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases
> -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.27 outbox
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-9b= b1-1df2130c7c97/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 23.50 inbox=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm 2,0M =C2=A06. lis 23.51 leases=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A0734 =C2=A07. lis 02.13 metadat= a=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (group can write, other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 16.55 outbox= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9f5= 9-3aebea4d8477/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.35 id= s=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M 24. =C3=BAno 01.06 inbox > -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 24. =C3=BAno 19.07 metadat= a
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.20 outbox<= br>

It should look like this:

-rw-rw----. 1 vdsm kvm 1.0M Mar =C2=A01 23:36 ids
-rw-rw----. 1 vdsm kvm 2.0M Mar =C2=A01 23:35 leases
-rw-r--r--. 1 vdsm kvm =C2=A0353 Mar =C2=A01 23:35 metadata
-rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 outbox
-rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 inbox

This explains the EACCES error.

You can start by fixing the permissions manually, you can do this online.
=C2=A0
> =C2=A0The ids files was generated by "touch" command after deleting them due "sanlock locking hang" =C2=A0gluster crash & reboot
> I expected that they will be filled automaticaly after gluster reboot ( the =C2=A0shadow copy from =C2=A0 ".gluster " = =C2=A0 directory =C2=A0was deleted & created empty =C2=A0too )

I don't know about gluster shadow copy, I would not play with gluster internals.

Adding Sahina for advice.

Did you generate the ids file on the mount point.

Ravi, can you help here?

Okay, so what I understand from the output above is you have different gluster volumes mounted and some of them have incorrect permissions for the 'ids' file. The way to fix it is to do it from the mount like Nir said.
Why did you delete the file from the .glusterfs in the brick(s)?=C2=A0 Was there a gfid split brain?

-Ravi

> OK, it looks that sanlock =C2=A0can't work with empty file or rewrite them .
> Am I right ??

Yes, the files must be initialized before sanlock can use them.

You can initialize the file like this:

sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0

Taken from http://lists.ovirt.org/pipermail/users/2016-February/038046.ht= ml

> The last point - about "ids" workaround - this is offline version =3D VMs have to be moved out from for continual running with maintenance volume mode
> But this is not acceptable in current situation, so the question again, =C2=A0is it safe to do it online ?? =C2=A0( YES= / NO )

The ids file is accessed only by sanlock. I guess that you don't have a running

SPM on this DC, since sanlock fails to acquire a host id, so you are pretty safe

to fix the permissions and initialize the ids files.

I would do this:

1. Stop engine, =C2=A0so it will not try to start vdsm

2. Stop vdsm on all hosts, so they do not try to acquire a host id with sanlock

=C2=A0 =C2=A0 This does not affect running vms

3. Fix the permissions on the ids file, via glusterfs mount

4. Initialize the ids files from one of the hosts, via the glusterfs mount

=C2=A0 =C2=A0 This should fix the ids files on all replicas<= /div>
5. Start vdsm on all hosts

6. Start engine

Engine will connect to all hosts, hosts will connect to storage and try to acquire a host id.

Then Engine will start the SPM on one of the hosts, and your DC should become up.

David, Sahina, can you confirm that this procedure is safe?

Yes, correcting from the mount point should fix it on all replicas

Nir

>
> regs.
> Pavel
>
>
>
> On 1.3.2016 18:38, Nir Soffer wrote:
>
> On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz <paf1(a)em= ail.cz> wrote:
>>
>> Hello, =C2=A0can anybody =C2=A0explain this error no.1= 3 ( open file ) in sanlock.log .
>
>
> This is EACCES
>
> Can you share the outoput of:
>
> =C2=A0 =C2=A0 ls -lh /rhev/data-center/mnt/<server>:<_path>/<sd_uuid>/dom_md > =C2=A0
>>
>>
>> The size of =C2=A0"ids" file is zero (0)
>
>
> This is how we create the ids file when initializing it.
>
> But then we use sanlock to initialize the ids file, and it should be 1MiB after that.
>
> Is this ids files created by vdsm, or one you created yourself?
> =C2=A0
>>
>> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids
>> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c= 199-4f58-89aa-102d44327124/dom_md/ids error -13
>> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0
>>
>> If the main problem is about zero file size, can I regenerate =C2=A0this file online securely , with no VM dependence =C2=A0????
>
>
> Yes, I think I already referred to the instructions how to do that in a previous mail.
>
>>
>>
>> dist =3D RHEL - 7 - 2.1511
>> kernel =3D 3.10.0 - 327.10.1.el7.x86_64
>> KVM =3D 2.3.0 - 29.1.el7
>> libvirt =3D libvirt-1.2.17-13.el7_2.3
>> vdsm =3D vdsm-4.16.30-0.el7
>> GlusterFS =3D glusterfs-3.7.8-1.el7
>>
>>
>> regs.
>> Pavel
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http:/= /lists.ovirt.org/mailman/listinfo/users
>>
>
>
_______________________________________________
Gluster-users mailing list
Gluster-users(a)gluster.org
http://www.gluster.org/mailman/listinfo/gluster-user=
s

--------------050402010809060509080406-- --===============4670495215605840215== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wNTA0MDIwMTA4MDkwNjA1MDkwODA0MDYKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PVVURi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK Ck9uIDAzLzAyLzIwMTYgMTI6MDIgUE0sIFNhaGluYSBCb3NlIHdyb3RlOgo+Cj4KPiBPbiAwMy8w Mi8yMDE2IDAzOjQ1IEFNLCBOaXIgU29mZmVyIHdyb3RlOgo+PiBPbiBUdWUsIE1hciAxLCAyMDE2 IGF0IDEwOjUxIFBNLCBwYWYxQGVtYWlsLmN6IDxwYWYxQGVtYWlsLmN6IAo+PiA8bWFpbHRvOnBh ZjFAZW1haWwuY3o+PiB3cm90ZToKPj4gPgo+PiA+IEhJLAo+PiA+IHJlcXVlc3RlZCBvdXRwdXQ6 Cj4+ID4KPj4gPiAjIGxzIC1saCAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2Fs aG9zdDoqLyovZG9tX21kCj4+ID4KPj4gPiAKPj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1CQ0svMGZjYWQ4ODgtZDU3My00N2JlLWJlZjMtMGJjMGI3 YTk5ZmI3L2RvbV9tZDoKPj4gPiB0b3RhbCAyLDFNCj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3Zt IDEsME0gIDEuIGLFmWUgMjEuMjggaWRzICA8LS0gZ29vZAo+PiA+IC1ydy1ydy0tLS0gMSB2ZHNt IGt2bSAgMTZNICA3LiBsaXMgMjIuMTYgaW5ib3gKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0g MiwwTSAgNy4gbGlzIDIyLjE3IGxlYXNlcwo+PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzM1 ICA3LiBsaXMgMjIuMTcgbWV0YWRhdGEKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAg Ny4gbGlzIDIyLjE2IG91dGJveAo+PiA+Cj4+ID4gCj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9n bHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDEvNTUzZDliOTItZTRhMC00MDQyLWE1NzktNGNh YmViNTVkZWQ0L2RvbV9tZDoKPj4gPiB0b3RhbCAxLDFNCj4+ID4gLXJ3LXItLXItLSAxIHZkc20g a3ZtICAgIDAgMjQuIMO6bm8gMDcuNDEgaWRzICA8LS0gYmFkIChzYW5sb2NrIAo+PiBjYW5ub3Qg d3JpdGUsIG90aGVyIGNhbiByZWFkKQo+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3 LiBsaXMgMDAuMTQgaW5ib3gKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlz IDAzLjU2IGxlYXNlcwo+PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzMzICA3LiBsaXMgMDMu NTYgbWV0YWRhdGEKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE0 IG91dGJveAo+PiA+Cj4+ID4gCj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9j YWxob3N0Ol8xS1ZNMTItUDIvODhhZGJkNDktNjJkNi00NWIxLTk5OTItYjA0NDY0YTA0MTEyL2Rv bV9tZDoKPj4gPiB0b3RhbCAxLDFNCj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQu IMO6bm8gMDcuNDMgaWRzICA8LS0gYmFkIChzYW5sb2NrIAo+PiBjYW5ub3Qgd3JpdGUsIG90aGVy IGNhbiByZWFkKQo+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTUg aW5ib3gKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDIyLjE0IGxlYXNl cwo+PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzMzICA3LiBsaXMgMjIuMTQgbWV0YWRhdGEK Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE1IG91dGJveAo+PiA+ Cj4+ID4gCj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZN MTItUDMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZDoKPj4gPiB0 b3RhbCAxLDFNCj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuNDMg aWRzICA8LS0gYmFkIChzYW5sb2NrIAo+PiBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKQo+ PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNIDIzLiDDum5vIDIyLjUxIGluYm94Cj4+ID4g LXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gMjMuIMO6bm8gMjMuMTIgbGVhc2VzCj4+ID4gLXJ3 LXItLXItLSAxIHZkc20ga3ZtICA5OTggMjUuIMO6bm8gMDAuMzUgbWV0YWRhdGEKPj4gPiAtcnct cnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE2IG91dGJveAo+PiA+Cj4+ID4gCj4+ IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1 MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZDoKPj4gPiB0b3RhbCAxLDFN Cj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuNDQgaWRzICA8LS0g YmFkIChzYW5sb2NrIAo+PiBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKQo+PiA+IC1ydy1y dy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTcgaW5ib3gKPj4gPiAtcnctcnctLS0t IDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDAwLjE4IGxlYXNlcwo+PiA+IC1ydy1yLS1yLS0gMSB2 ZHNtIGt2bSAgMzMzICA3LiBsaXMgMDAuMTggbWV0YWRhdGEKPj4gPiAtcnctcnctLS0tIDEgdmRz bSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE3IG91dGJveAo+PiA+Cj4+ID4gCj4+IC9yaGV2L2RhdGEt Y2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDEvNDJkNzEwYTktYjg0NC00 M2RjLWJlNDEtNzcwMDJkMWNkNTUzL2RvbV9tZDoKPj4gPiB0b3RhbCAxLDFNCj4+ID4gLXJ3LXJ3 LXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuMzIgaWRzICA8LS0gYmFkIChvdGhlciBj YW4gcmVhZCkKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDIyLjE4IGlu Ym94Cj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gIDcuIGxpcyAyMi4xOCBsZWFzZXMK Pj4gPiAtcnctci0tci0tIDEgdmRzbSBrdm0gIDMzMyAgNy4gbGlzIDIyLjE4IG1ldGFkYXRhCj4+ ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAyMi4xOCBvdXRib3gKPj4gPgo+ PiA+IAo+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEy LVAyL2ZmNzFiNDdiLTBmNzItNDUyOC05YmZlLWMzZGE4ODhlNDdmMC9kb21fbWQ6Cj4+ID4gdG90 YWwgMywwTQo+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAxLDBNICAxLiBixZllIDIxLjI4IGlk cyAgPC0tIGJhZCAob3RoZXIgY2FuIHJlYWQpCj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAx Nk0gMjUuIMO6bm8gMDAuNDIgaW5ib3gKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAy NS4gw7pubyAwMC40NCBsZWFzZXMKPj4gPiAtcnctci0tci0tIDEgdmRzbSBrdm0gIDk5NyAyNC4g w7pubyAwMi40NiBtZXRhZGF0YQo+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNIDI1LiDD um5vIDAwLjQ0IG91dGJveAo+PiA+Cj4+ID4gCj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVz dGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDMvZWYwMTBkMDgtYWVkMS00MWM0LWJhOWEtZTZkOWJk ZWNiNGI0L2RvbV9tZDoKPj4gPiB0b3RhbCAyLDFNCj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3Zt ICAgIDAgMjQuIMO6bm8gMDcuMzQgaWRzICA8LS0gYmFkIChzYW5sb2NrIAo+PiBjYW5ub3Qgd3Jp dGUsIG90aGVyIGNhbiByZWFkKQo+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNIDIzLiDD um5vIDIyLjM1IGluYm94Cj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gMjMuIMO6bm8g MjIuMzggbGVhc2VzCj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtIDEsMUsgMjQuIMO6bm8gMTku MDcgbWV0YWRhdGEKPj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyMy4gw7pubyAyMi4y NyBvdXRib3gKPj4gPgo+PiA+IAo+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xv Y2FsaG9zdDpfMktWTTEyX19QNC8zMDBlOWFjOC0zYzJmLTQ3MDMtOWJiMS0xZGYyMTMwYzdjOTcv ZG9tX21kOgo+PiA+IHRvdGFsIDMsME0KPj4gPiAtcnctcnctci0tIDEgdmRzbSBrdm0gMSwwTSAg MS4gYsWZZSAyMS4yOCBpZHMgIDwtLSBiYWQgKG90aGVyIGNhbiByZWFkKQo+PiA+IC1ydy1ydy1y LS0gMSB2ZHNtIGt2bSAgMTZNICA2LiBsaXMgMjMuNTAgaW5ib3ggIDwtLSBiYWQgKG90aGVyIGNh biAKPj4gcmVhZCkKPj4gPiAtcnctcnctci0tIDEgdmRzbSBrdm0gMiwwTSAgNi4gbGlzIDIzLjUx IGxlYXNlcyAgICA8LS0gYmFkIChvdGhlciAKPj4gY2FuIHJlYWQpCj4+ID4gLXJ3LXJ3LXItLSAx IHZkc20ga3ZtICA3MzQgIDcuIGxpcyAwMi4xMyBtZXRhZGF0YSAgPC0tIGJhZCAoZ3JvdXAgCj4+ IGNhbiB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpCj4+ID4gLXJ3LXJ3LXItLSAxIHZkc20ga3ZtICAx Nk0gIDYuIGxpcyAxNi41NSBvdXRib3ggIDwtLSBiYWQgKG90aGVyIGNhbiAKPj4gcmVhZCkKPj4g Pgo+PiA+IAo+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktW TTEyLVA1LzFjYTU2YjQ1LTcwMWUtNGMyMi05ZjU5LTNhZWJlYTRkODQ3Ny9kb21fbWQ6Cj4+ID4g dG90YWwgMSwxTQo+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjM1 IGlkcyAgPC0tIGJhZCAob3RoZXIgY2FuIHJlYWQpCj4+ID4gLXJ3LXJ3LXItLSAxIHZkc20ga3Zt ICAxNk0gMjQuIMO6bm8gMDEuMDYgaW5ib3gKPj4gPiAtcnctcnctci0tIDEgdmRzbSBrdm0gMiww TSAyNC4gw7pubyAwMi40NCBsZWFzZXMKPj4gPiAtcnctci0tci0tIDEgdmRzbSBrdm0gIDk5OCAy NC4gw7pubyAxOS4wNyBtZXRhZGF0YQo+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgMTZNICA3 LiBsaXMgMjIuMjAgb3V0Ym94Cj4+Cj4+Cj4+IEl0IHNob3VsZCBsb29rIGxpa2UgdGhpczoKPj4K Pj4gLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAxLjBNIE1hciAgMSAyMzozNiBpZHMKPj4gLXJ3LXJ3 LS0tLS4gMSB2ZHNtIGt2bSAyLjBNIE1hciAgMSAyMzozNSBsZWFzZXMKPj4gLXJ3LXItLXItLS4g MSB2ZHNtIGt2bSAgMzUzIE1hciAgMSAyMzozNSBtZXRhZGF0YQo+PiAtcnctcnctLS0tLiAxIHZk c20ga3ZtICAxNk0gTWFyICAxIDIzOjM0IG91dGJveAo+PiAtcnctcnctLS0tLiAxIHZkc20ga3Zt ICAxNk0gTWFyICAxIDIzOjM0IGluYm94Cj4+Cj4+IFRoaXMgZXhwbGFpbnMgdGhlIEVBQ0NFUyBl cnJvci4KPj4KPj4gWW91IGNhbiBzdGFydCBieSBmaXhpbmcgdGhlIHBlcm1pc3Npb25zIG1hbnVh bGx5LCB5b3UgY2FuIGRvIHRoaXMgb25saW5lLgo+Pgo+PiA+ICBUaGUgaWRzIGZpbGVzIHdhcyBn ZW5lcmF0ZWQgYnkgInRvdWNoIiBjb21tYW5kIGFmdGVyIGRlbGV0aW5nIHRoZW0gCj4+IGR1ZSAi c2FubG9jayBsb2NraW5nIGhhbmciICBnbHVzdGVyIGNyYXNoICYgcmVib290Cj4+ID4gSSBleHBl Y3RlZCB0aGF0IHRoZXkgd2lsbCBiZSBmaWxsZWQgYXV0b21hdGljYWx5IGFmdGVyIGdsdXN0ZXIg Cj4+IHJlYm9vdCAoIHRoZSAgc2hhZG93IGNvcHkgZnJvbSAgICIuZ2x1c3RlciAiIGRpcmVjdG9y eSAgd2FzIGRlbGV0ZWQgJiAKPj4gY3JlYXRlZCBlbXB0eSAgdG9vICkKPj4KPj4gSSBkb24ndCBr bm93IGFib3V0IGdsdXN0ZXIgc2hhZG93IGNvcHksIEkgd291bGQgbm90IHBsYXkgd2l0aCBnbHVz dGVyIAo+PiBpbnRlcm5hbHMuCj4+IEFkZGluZyBTYWhpbmEgZm9yIGFkdmljZS4KPgo+IERpZCB5 b3UgZ2VuZXJhdGUgdGhlIGlkcyBmaWxlIG9uIHRoZSBtb3VudCBwb2ludC4KPgo+IFJhdmksIGNh biB5b3UgaGVscCBoZXJlPwo+CgpPa2F5LCBzbyB3aGF0IEkgdW5kZXJzdGFuZCBmcm9tIHRoZSBv dXRwdXQgYWJvdmUgaXMgeW91IGhhdmUgZGlmZmVyZW50IApnbHVzdGVyIHZvbHVtZXMgbW91bnRl ZCBhbmQgc29tZSBvZiB0aGVtIGhhdmUgaW5jb3JyZWN0IHBlcm1pc3Npb25zIGZvciAKdGhlICdp ZHMnIGZpbGUuIFRoZSB3YXkgdG8gZml4IGl0IGlzIHRvIGRvIGl0IGZyb20gdGhlIG1vdW50IGxp a2UgTmlyIHNhaWQuCldoeSBkaWQgeW91IGRlbGV0ZSB0aGUgZmlsZSBmcm9tIHRoZSAuZ2x1c3Rl cmZzIGluIHRoZSBicmljayhzKT8gV2FzIAp0aGVyZSBhIGdmaWQgc3BsaXQgYnJhaW4/CgotUmF2 aQoKPj4KPj4gPiBPSywgaXQgbG9va3MgdGhhdCBzYW5sb2NrICBjYW4ndCB3b3JrIHdpdGggZW1w dHkgZmlsZSBvciByZXdyaXRlIHRoZW0gLgo+PiA+IEFtIEkgcmlnaHQgPz8KPj4KPj4gWWVzLCB0 aGUgZmlsZXMgbXVzdCBiZSBpbml0aWFsaXplZCBiZWZvcmUgc2FubG9jayBjYW4gdXNlIHRoZW0u Cj4+Cj4+IFlvdSBjYW4gaW5pdGlhbGl6ZSB0aGUgZmlsZSBsaWtlIHRoaXM6Cj4+Cj4+IHNhbmxv Y2sgZGlyZWN0IGluaXQgLXMgPHNkX3V1aWQ+OjA6cmVwYWlyLzxzZF91dWlkPi9kb21fbWQvaWRz OjAKPj4KPj4gVGFrZW4gZnJvbSAKPj4gaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9waXBlcm1haWwv dXNlcnMvMjAxNi1GZWJydWFyeS8wMzgwNDYuaHRtbAo+Pgo+PiA+IFRoZSBsYXN0IHBvaW50IC0g YWJvdXQgImlkcyIgd29ya2Fyb3VuZCAtIHRoaXMgaXMgb2ZmbGluZSB2ZXJzaW9uID0gCj4+IFZN cyBoYXZlIHRvIGJlIG1vdmVkIG91dCBmcm9tIGZvciBjb250aW51YWwgcnVubmluZyB3aXRoIG1h aW50ZW5hbmNlIAo+PiB2b2x1bWUgbW9kZQo+PiA+IEJ1dCB0aGlzIGlzIG5vdCBhY2NlcHRhYmxl IGluIGN1cnJlbnQgc2l0dWF0aW9uLCBzbyB0aGUgcXVlc3Rpb24gCj4+IGFnYWluLCAgaXMgaXQg c2FmZSB0byBkbyBpdCBvbmxpbmUgPz8gICggWUVTIC8gTk8gKQo+Pgo+PiBUaGUgaWRzIGZpbGUg aXMgYWNjZXNzZWQgb25seSBieSBzYW5sb2NrLiBJIGd1ZXNzIHRoYXQgeW91IGRvbid0IGhhdmUg Cj4+IGEgcnVubmluZwo+PiBTUE0gb24gdGhpcyBEQywgc2luY2Ugc2FubG9jayBmYWlscyB0byBh Y3F1aXJlIGEgaG9zdCBpZCwgc28geW91IGFyZSAKPj4gcHJldHR5IHNhZmUKPj4gdG8gZml4IHRo ZSBwZXJtaXNzaW9ucyBhbmQgaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGVzLgo+Pgo+PiBJIHdvdWxk IGRvIHRoaXM6Cj4+Cj4+IDEuIFN0b3AgZW5naW5lLCAgc28gaXQgd2lsbCBub3QgdHJ5IHRvIHN0 YXJ0IHZkc20KPj4gMi4gU3RvcCB2ZHNtIG9uIGFsbCBob3N0cywgc28gdGhleSBkbyBub3QgdHJ5 IHRvIGFjcXVpcmUgYSBob3N0IGlkIAo+PiB3aXRoIHNhbmxvY2sKPj4gICAgIFRoaXMgZG9lcyBu b3QgYWZmZWN0IHJ1bm5pbmcgdm1zCj4+IDMuIEZpeCB0aGUgcGVybWlzc2lvbnMgb24gdGhlIGlk cyBmaWxlLCB2aWEgZ2x1c3RlcmZzIG1vdW50Cj4+IDQuIEluaXRpYWxpemUgdGhlIGlkcyBmaWxl cyBmcm9tIG9uZSBvZiB0aGUgaG9zdHMsIHZpYSB0aGUgZ2x1c3RlcmZzIAo+PiBtb3VudAo+PiAg ICAgVGhpcyBzaG91bGQgZml4IHRoZSBpZHMgZmlsZXMgb24gYWxsIHJlcGxpY2FzCj4+IDUuIFN0 YXJ0IHZkc20gb24gYWxsIGhvc3RzCj4+IDYuIFN0YXJ0IGVuZ2luZQo+Pgo+PiBFbmdpbmUgd2ls bCBjb25uZWN0IHRvIGFsbCBob3N0cywgaG9zdHMgd2lsbCBjb25uZWN0IHRvIHN0b3JhZ2UgYW5k IAo+PiB0cnkgdG8gYWNxdWlyZSBhIGhvc3QgaWQuCj4+IFRoZW4gRW5naW5lIHdpbGwgc3RhcnQg dGhlIFNQTSBvbiBvbmUgb2YgdGhlIGhvc3RzLCBhbmQgeW91ciBEQyAKPj4gc2hvdWxkIGJlY29t ZSB1cC4KPj4KPj4gRGF2aWQsIFNhaGluYSwgY2FuIHlvdSBjb25maXJtIHRoYXQgdGhpcyBwcm9j ZWR1cmUgaXMgc2FmZT8KPgo+IFllcywgY29ycmVjdGluZyBmcm9tIHRoZSBtb3VudCBwb2ludCBz aG91bGQgZml4IGl0IG9uIGFsbCByZXBsaWNhcwo+Cj4KPj4KPj4gTmlyCj4+Cj4+ID4KPj4gPiBy ZWdzLgo+PiA+IFBhdmVsCj4+ID4KPj4gPgo+PiA+Cj4+ID4gT24gMS4zLjIwMTYgMTg6MzgsIE5p ciBTb2ZmZXIgd3JvdGU6Cj4+ID4KPj4gPiBPbiBUdWUsIE1hciAxLCAyMDE2IGF0IDU6MDcgUE0s IHBhZjFAZW1haWwuY3ogPHBhZjFAZW1haWwuY3ogCj4+IDxtYWlsdG86cGFmMUBlbWFpbC5jej4+ IHdyb3RlOgo+PiA+Pgo+PiA+PiBIZWxsbywgIGNhbiBhbnlib2R5ICBleHBsYWluIHRoaXMgZXJy b3Igbm8uMTMgKCBvcGVuIGZpbGUgKSBpbiAKPj4gc2FubG9jay5sb2cgLgo+PiA+Cj4+ID4KPj4g PiBUaGlzIGlzIEVBQ0NFUwo+PiA+Cj4+ID4gQ2FuIHlvdSBzaGFyZSB0aGUgb3V0b3B1dCBvZjoK Pj4gPgo+PiA+ICAgICBscyAtbGggL3JoZXYvZGF0YS1jZW50ZXIvbW50LzxzZXJ2ZXI+OjxfcGF0 aD4vPHNkX3V1aWQ+L2RvbV9tZAo+PiA+Cj4+ID4+Cj4+ID4+Cj4+ID4+IFRoZSBzaXplIG9mICAi aWRzIiBmaWxlIGlzIHplcm8gKDApCj4+ID4KPj4gPgo+PiA+IFRoaXMgaXMgaG93IHdlIGNyZWF0 ZSB0aGUgaWRzIGZpbGUgd2hlbiBpbml0aWFsaXppbmcgaXQuCj4+ID4KPj4gPiBCdXQgdGhlbiB3 ZSB1c2Ugc2FubG9jayB0byBpbml0aWFsaXplIHRoZSBpZHMgZmlsZSwgYW5kIGl0IHNob3VsZCAK Pj4gYmUgMU1pQiBhZnRlciB0aGF0Lgo+PiA+Cj4+ID4gSXMgdGhpcyBpZHMgZmlsZXMgY3JlYXRl ZCBieSB2ZHNtLCBvciBvbmUgeW91IGNyZWF0ZWQgeW91cnNlbGY/Cj4+ID4KPj4gPj4KPj4gPj4g MjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IG9wZW4gZXJyb3IgLTEzIAo+ PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0Lzdm NTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzCj4+ID4+IDIwMTYt MDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBzMTg3OTg1IG9wZW5fZGlzayAKPj4g L3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUy YjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkcyAKPj4gZXJyb3IgLTEz Cj4+ID4+IDIwMTYtMDItMjggMDM6MjU6NTYrMDEwMCAyNjk2MzYgWzExMzA0XTogczE4Nzk5MiBs b2Nrc3BhY2UgCj4+IDdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNDoxOi9yaGV2 L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTct YzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM6MAo+PiA+Pgo+PiA+PiBJZiB0 aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXplLCBjYW4gSSByZWdlbmVyYXRl IAo+PiAgdGhpcyBmaWxlIG9ubGluZSBzZWN1cmVseSAsIHdpdGggbm8gVk0gZGVwZW5kZW5jZSAg Pz8/Pwo+PiA+Cj4+ID4KPj4gPiBZZXMsIEkgdGhpbmsgSSBhbHJlYWR5IHJlZmVycmVkIHRvIHRo ZSBpbnN0cnVjdGlvbnMgaG93IHRvIGRvIHRoYXQgCj4+IGluIGEgcHJldmlvdXMgbWFpbC4KPj4g Pgo+PiA+Pgo+PiA+Pgo+PiA+PiBkaXN0ID0gUkhFTCAtIDcgLSAyLjE1MTEKPj4gPj4ga2VybmVs ID0gMy4xMC4wIC0gMzI3LjEwLjEuZWw3Lng4Nl82NAo+PiA+PiBLVk0gPSAyLjMuMCAtIDI5LjEu ZWw3Cj4+ID4+IGxpYnZpcnQgPSBsaWJ2aXJ0LTEuMi4xNy0xMy5lbDdfMi4zCj4+ID4+IHZkc20g PSB2ZHNtLTQuMTYuMzAtMC5lbDcKPj4gPj4gR2x1c3RlckZTID0gZ2x1c3RlcmZzLTMuNy44LTEu ZWw3Cj4+ID4+Cj4+ID4+Cj4+ID4+IHJlZ3MuCj4+ID4+IFBhdmVsCj4+ID4+Cj4+ID4+IF9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCj4+ID4+IFVzZXJzIG1h aWxpbmcgbGlzdAo+PiA+PiBVc2Vyc0BvdmlydC5vcmcgPG1haWx0bzpVc2Vyc0BvdmlydC5vcmc+ Cj4+ID4+IGh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2Vycwo+PiA+ Pgo+PiA+Cj4+ID4KPgo+Cj4KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwo+IEdsdXN0ZXItdXNlcnMgbWFpbGluZyBsaXN0Cj4gR2x1c3Rlci11c2Vyc0Bn bHVzdGVyLm9yZwo+IGh0dHA6Ly93d3cuZ2x1c3Rlci5vcmcvbWFpbG1hbi9saXN0aW5mby9nbHVz dGVyLXVzZXJzCgoKCi0tLS0tLS0tLS0tLS0tMDUwNDAyMDEwODA5MDYwNTA5MDgwNDA2CkNvbnRl bnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PVVURi04CkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rp bmc6IDhiaXQKCjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEgY29udGVudD0idGV4dC9odG1sOyBj aGFyc2V0PVVURi04IiBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogIDwvaGVhZD4KICA8Ym9k eSB0ZXh0PSIjMDAwMDAwIiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIDxkaXYgY2xhc3M9Im1vei1j aXRlLXByZWZpeCI+T24gMDMvMDIvMjAxNiAxMjowMiBQTSwgU2FoaW5hIEJvc2UKICAgICAgd3Jv dGU6PGJyPgogICAgPC9kaXY+CiAgICA8YmxvY2txdW90ZSBjaXRlPSJtaWQ6NTZENjg5MTAuODA0 MDYwMkByZWRoYXQuY29tIiB0eXBlPSJjaXRlIj4KICAgICAgPG1ldGEgY29udGVudD0idGV4dC9o dG1sOyBjaGFyc2V0PVVURi04IiBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogICAgICA8YnI+ CiAgICAgIDxicj4KICAgICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5PbiAwMy8wMi8y MDE2IDAzOjQ1IEFNLCBOaXIgU29mZmVyCiAgICAgICAgd3JvdGU6PGJyPgogICAgICA8L2Rpdj4K ICAgICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVJieXl1OWd3UGZWcFB4cERhNF9nS1d5WHEx UGF2VG0yVjJyRzJjVTBBdkU9SkpQQUBtYWlsLmdtYWlsLmNvbSIKICAgICAgICB0eXBlPSJjaXRl Ij4KICAgICAgICA8ZGl2IGRpcj0ibHRyIj5PbiBUdWUsIE1hciAxLCAyMDE2IGF0IDEwOjUxIFBN LCA8YQogICAgICAgICAgICBtb3otZG8tbm90LXNlbmQ9InRydWUiIGNsYXNzPSJtb3otdHh0LWxp bmstYWJicmV2aWF0ZWQiCiAgICAgICAgICAgIGhyZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6Ij48 YSBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJtYWlsdG86cGFmMUBlbWFp bC5jeiI+cGFmMUBlbWFpbC5jejwvYT48L2E+ICZsdDs8YQogICAgICAgICAgICBtb3otZG8tbm90 LXNlbmQ9InRydWUiIGhyZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6Ij48YSBjbGFzcz0ibW96LXR4 dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJtYWlsdG86cGFmMUBlbWFpbC5jeiI+cGFmMUBlbWFp bC5jejwvYT48L2E+Jmd0OwoKICAgICAgICAgIHdyb3RlOjxicj4KICAgICAgICAgICZndDs8YnI+ CiAgICAgICAgICAmZ3Q7IEhJLDxicj4KICAgICAgICAgICZndDsgcmVxdWVzdGVkIG91dHB1dDo8 YnI+CiAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgJmd0OyAjIGxzIC1saAogICAgICAgICAg L3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6Ki8qL2RvbV9tZDxicj4K ICAgICAgICAgICZndDsgwqA8YnI+CiAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21u dC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItQkNLLzBmY2FkODg4LWQ1NzMtNDdiZS1iZWYz LTBiYzBiN2E5OWZiNy9kb21fbWQ6PGJyPgogICAgICAgICAgJmd0OyB0b3RhbCAyLDFNPGJyPgog ICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMSwwTSDCoDEuIGLFmWUgMjEuMjgg aWRzIMKgIMKgIMKgCiAgICAgICAgICDCoCZsdDstLSBnb29kPGJyPgogICAgICAgICAgJmd0OyAt cnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMjIuMTYgaW5ib3g8YnI+CiAgICAg ICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDIyLjE3IGxlYXNl czxicj4KICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgMzM1IMKgNy4gbGlz IDIyLjE3IG1ldGFkYXRhPGJyPgogICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0g wqAxNk0gwqA3LiBsaXMgMjIuMTYgb3V0Ym94PGJyPgogICAgICAgICAgJmd0Ozxicj4KICAgICAg ICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0x Mi1QMS81NTNkOWI5Mi1lNGEwLTQwNDItYTU3OS00Y2FiZWI1NWRlZDQvZG9tX21kOjxicj4KICAg ICAgICAgICZndDsgdG90YWwgMSwxTTxicj4KICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZk c20ga3ZtIMKgIMKgMCAyNC4gw7pubyAwNy40MSBpZHMgwqAgwqAgwqAKICAgICAgICAgIMKgJmx0 Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKTxicj4KICAgICAg ICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE0IGluYm94 PGJyPgogICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSDCoDcuIGxpcyAw My41NiBsZWFzZXM8YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMz MyDCoDcuIGxpcyAwMy41NiBtZXRhZGF0YTxicj4KICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAx IHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE0IG91dGJveDxicj4KICAgICAgICAgICZndDs8 YnI+CiAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxo b3N0Ol8xS1ZNMTItUDIvODhhZGJkNDktNjJkNi00NWIxLTk5OTItYjA0NDY0YTA0MTEyL2RvbV9t ZDo8YnI+CiAgICAgICAgICAmZ3Q7IHRvdGFsIDEsMU08YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1y LS1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuNDMgaWRzIMKgIMKgIMKgCiAgICAg ICAgICDCoCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8 YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAw MC4xNSBpbmJveDxicj4KICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0g wqA3LiBsaXMgMjIuMTQgbGVhc2VzPGJyPgogICAgICAgICAgJmd0OyAtcnctci0tci0tIDEgdmRz bSBrdm0gwqAzMzMgwqA3LiBsaXMgMjIuMTQgbWV0YWRhdGE8YnI+CiAgICAgICAgICAmZ3Q7IC1y dy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAwMC4xNSBvdXRib3g8YnI+CiAgICAg ICAgICAmZ3Q7PGJyPgogICAgICAgICAgJmd0Owovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3Rl clNEL2xvY2FsaG9zdDpfMUtWTTEyLVAzLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcw YjE0Ny9kb21fbWQ6PGJyPgogICAgICAgICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAgICAg Jmd0OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjQzIGlkcyDCoCDC oCDCoAogICAgICAgICAgwqAmbHQ7LS0gYmFkIChzYW5sb2NrIGNhbm5vdCB3cml0ZSwgb3RoZXIg Y2FuIHJlYWQpPGJyPgogICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0g MjMuIMO6bm8gMjIuNTEgaW5ib3g8YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNt IGt2bSAyLDBNIDIzLiDDum5vIDIzLjEyIGxlYXNlczxicj4KICAgICAgICAgICZndDsgLXJ3LXIt LXItLSAxIHZkc20ga3ZtIMKgOTk4IDI1LiDDum5vIDAwLjM1IG1ldGFkYXRhPGJyPgogICAgICAg ICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTYgb3V0Ym94 PGJyPgogICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIv bW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlh YS0xMDJkNDQzMjcxMjQvZG9tX21kOjxicj4KICAgICAgICAgICZndDsgdG90YWwgMSwxTTxicj4K ICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgIMKgMCAyNC4gw7pubyAwNy40 NCBpZHMgwqAgwqAgwqAKICAgICAgICAgIMKgJmx0Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3Jp dGUsIG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20g a3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE3IGluYm94PGJyPgogICAgICAgICAgJmd0OyAtcnctcnct LS0tIDEgdmRzbSBrdm0gMiwwTSDCoDcuIGxpcyAwMC4xOCBsZWFzZXM8YnI+CiAgICAgICAgICAm Z3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAwMC4xOCBtZXRhZGF0YTxi cj4KICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAw LjE3IG91dGJveDxicj4KICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAmZ3Q7Ci9yaGV2L2Rh dGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDEvNDJkNzEwYTktYjg0 NC00M2RjLWJlNDEtNzcwMDJkMWNkNTUzL2RvbV9tZDo8YnI+CiAgICAgICAgICAmZ3Q7IHRvdGFs IDEsMU08YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQu IMO6bm8gMDcuMzIgaWRzIMKgIMKgIMKgCiAgICAgICAgICDCoCZsdDstLSBiYWQgKG90aGVyIGNh biByZWFkKTxicj4KICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKg Ny4gbGlzIDIyLjE4IGluYm94PGJyPgogICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBr dm0gMiwwTSDCoDcuIGxpcyAyMi4xOCBsZWFzZXM8YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1yLS1y LS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAyMi4xOCBtZXRhZGF0YTxicj4KICAgICAgICAg ICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDIyLjE4IG91dGJveDxi cj4KICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21u dC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDIvZmY3MWI0N2ItMGY3Mi00NTI4LTliZmUt YzNkYTg4OGU0N2YwL2RvbV9tZDo8YnI+CiAgICAgICAgICAmZ3Q7IHRvdGFsIDMsME08YnI+CiAg ICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAxLDBNIMKgMS4gYsWZZSAyMS4yOCBp ZHPCoMKgIMKgIMKgCiAgICAgICAgICDCoCZsdDstLSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4K ICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIDI1LiDDum5vIDAwLjQy IGluYm94wqA8YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDI1 LiDDum5vIDAwLjQ0IGxlYXNlczxicj4KICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20g a3ZtIMKgOTk3IDI0LiDDum5vIDAyLjQ2IG1ldGFkYXRhPGJyPgogICAgICAgICAgJmd0OyAtcnct cnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gMjUuIMO6bm8gMDAuNDQgb3V0Ym94PGJyPgogICAgICAg ICAgJmd0Ozxicj4KICAgICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJT RC9sb2NhbGhvc3Q6XzJLVk0xMi1QMy9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRlY2I0 YjQvZG9tX21kOjxicj4KICAgICAgICAgICZndDsgdG90YWwgMiwxTTxicj4KICAgICAgICAgICZn dDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgIMKgMCAyNC4gw7pubyAwNy4zNCBpZHPCoMKgIMKg IMKgCiAgICAgICAgICDCoCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBj YW4gcmVhZCk8YnI+CiAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAy My4gw7pubyAyMi4zNSBpbmJveDxicj4KICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20g a3ZtIDIsME0gMjMuIMO6bm8gMjIuMzggbGVhc2VzPGJyPgogICAgICAgICAgJmd0OyAtcnctci0t ci0tIDEgdmRzbSBrdm0gMSwxSyAyNC4gw7pubyAxOS4wNyBtZXRhZGF0YTxicj4KICAgICAgICAg ICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIDIzLiDDum5vIDIyLjI3IG91dGJveDxi cj4KICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21u dC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTJfX1A0LzMwMGU5YWM4LTNjMmYtNDcwMy05YmIx LTFkZjIxMzBjN2M5Ny9kb21fbWQ6PGJyPgogICAgICAgICAgJmd0OyB0b3RhbCAzLDBNPGJyPgog ICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gMSwwTSDCoDEuIGLFmWUgMjEuMjgg aWRzwqDCoCDCoCDCoAogICAgICAgICAgwqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVhZCk8YnI+ CiAgICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDYuIGxpcyAyMy41 MCBpbmJveMKgwqAgwqAgwqAKICAgICAgICAgIMKgJmx0Oy0tIGJhZCAob3RoZXIgY2FuIHJlYWQp CiAgICAgICAgICA8ZGl2PiZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gwqA2LiBsaXMg MjMuNTEgbGVhc2VzwqDCoCDCoAogICAgICAgICAgICDCoCDCoCZsdDstLSBiYWQgKG90aGVyIGNh biByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqA3MzQg wqA3LiBsaXMgMDIuMTMgbWV0YWRhdGHCoMKgIMKgIMKgCiAgICAgICAgICAgIMKgJmx0Oy0tIGJh ZCAoZ3JvdXAgY2FuIHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+CiAgICAgICAgICAgICZndDsg LXJ3LXJ3LXItLSAxIHZkc20ga3ZtIMKgMTZNIMKgNi4gbGlzIDE2LjU1IG91dGJveMKgwqAgwqAg wqAKICAgICAgICAgICAgwqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVhZCk8YnI+CiAgICAgICAg ICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1QNS8xY2E1NmI0NS03MDFlLTRjMjItOWY1OS0zYWViZWE0 ZDg0NzcvZG9tX21kOjxicj4KICAgICAgICAgICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAg ICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzUgaWRz wqDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4K ICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAxNk0gMjQuIMO6bm8gMDEu MDYgaW5ib3g8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0g MjQuIMO6bm8gMDIuNDQgbGVhc2VzPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2 ZHNtIGt2bSDCoDk5OCAyNC4gw7pubyAxOS4wNyBtZXRhZGF0YTxicj4KICAgICAgICAgICAgJmd0 OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMjIuMjAgb3V0Ym94PGJyPgog ICAgICAgICAgICA8YnI+CiAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgSXQgc2hvdWxkIGxv b2sgbGlrZSB0aGlzOjxicj4KICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAtcnctcnctLS0t LiAxIHZkc20ga3ZtIDEuME0gTWFyIMKgMSAyMzozNiBpZHM8YnI+CiAgICAgICAgICAgIC1ydy1y dy0tLS0uIDEgdmRzbSBrdm0gMi4wTSBNYXIgwqAxIDIzOjM1IGxlYXNlczxicj4KICAgICAgICAg ICAgLXJ3LXItLXItLS4gMSB2ZHNtIGt2bSDCoDM1MyBNYXIgwqAxIDIzOjM1IG1ldGFkYXRhPGJy PgogICAgICAgICAgICAtcnctcnctLS0tLiAxIHZkc20ga3ZtIMKgMTZNIE1hciDCoDEgMjM6MzQg b3V0Ym94PGJyPgogICAgICAgICAgICAtcnctcnctLS0tLiAxIHZkc20ga3ZtIMKgMTZNIE1hciDC oDEgMjM6MzQgaW5ib3g8YnI+CiAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgVGhpcyBleHBs YWlucyB0aGUgRUFDQ0VTIGVycm9yLjxicj4KICAgICAgICAgICAgPGJyPgogICAgICAgICAgICBZ b3UgY2FuIHN0YXJ0IGJ5IGZpeGluZyB0aGUgcGVybWlzc2lvbnMgbWFudWFsbHksIHlvdSBjYW4g ZG8KICAgICAgICAgICAgdGhpcyBvbmxpbmUuPGJyPgogICAgICAgICAgICDCoDxicj4KICAgICAg ICAgICAgJmd0OyDCoFRoZSBpZHMgZmlsZXMgd2FzIGdlbmVyYXRlZCBieSAidG91Y2giIGNvbW1h bmQgYWZ0ZXIKICAgICAgICAgICAgZGVsZXRpbmcgdGhlbSBkdWUgInNhbmxvY2sgbG9ja2luZyBo YW5nIiDCoGdsdXN0ZXIgY3Jhc2gKICAgICAgICAgICAgJmFtcDsgcmVib290PGJyPgogICAgICAg ICAgICAmZ3Q7IEkgZXhwZWN0ZWQgdGhhdCB0aGV5IHdpbGwgYmUgZmlsbGVkIGF1dG9tYXRpY2Fs eSBhZnRlcgogICAgICAgICAgICBnbHVzdGVyIHJlYm9vdCAoIHRoZSDCoHNoYWRvdyBjb3B5IGZy b20gwqAgIi5nbHVzdGVyICIgwqAKICAgICAgICAgICAgZGlyZWN0b3J5IMKgd2FzIGRlbGV0ZWQg JmFtcDsgY3JlYXRlZCBlbXB0eSDCoHRvbyApPGJyPgogICAgICAgICAgICA8YnI+CiAgICAgICAg ICAgIEkgZG9uJ3Qga25vdyBhYm91dCBnbHVzdGVyIHNoYWRvdyBjb3B5LCBJIHdvdWxkIG5vdCBw bGF5CiAgICAgICAgICAgIHdpdGggZ2x1c3RlciBpbnRlcm5hbHMuPC9kaXY+CiAgICAgICAgICA8 ZGl2PkFkZGluZyBTYWhpbmEgZm9yIGFkdmljZS48YnI+CiAgICAgICAgICA8L2Rpdj4KICAgICAg ICA8L2Rpdj4KICAgICAgPC9ibG9ja3F1b3RlPgogICAgICA8YnI+CiAgICAgIERpZCB5b3UgZ2Vu ZXJhdGUgdGhlIGlkcyBmaWxlIG9uIHRoZSBtb3VudCBwb2ludC48YnI+CiAgICAgIDxicj4KICAg ICAgUmF2aSwgY2FuIHlvdSBoZWxwIGhlcmU/PGJyPgogICAgICA8YnI+CiAgICA8L2Jsb2NrcXVv dGU+CiAgICA8YnI+CiAgICBPa2F5LCBzbyB3aGF0IEkgdW5kZXJzdGFuZCBmcm9tIHRoZSBvdXRw dXQgYWJvdmUgaXMgeW91IGhhdmUKICAgIGRpZmZlcmVudCBnbHVzdGVyIHZvbHVtZXMgbW91bnRl ZCBhbmQgc29tZSBvZiB0aGVtIGhhdmUgaW5jb3JyZWN0CiAgICBwZXJtaXNzaW9ucyBmb3IgdGhl ICdpZHMnIGZpbGUuIFRoZSB3YXkgdG8gZml4IGl0IGlzIHRvIGRvIGl0IGZyb20KICAgIHRoZSBt b3VudCBsaWtlIE5pciBzYWlkLjxicj4KICAgIFdoeSBkaWQgeW91IGRlbGV0ZSB0aGUgZmlsZSBm cm9tIHRoZSAuZ2x1c3RlcmZzIGluIHRoZSBicmljayhzKT/CoAogICAgV2FzIHRoZXJlIGEgZ2Zp ZCBzcGxpdCBicmFpbj8gPGJyPgogICAgPGJyPgogICAgLVJhdmk8YnI+CiAgICA8YnI+CiAgICA8 YmxvY2txdW90ZSBjaXRlPSJtaWQ6NTZENjg5MTAuODA0MDYwMkByZWRoYXQuY29tIiB0eXBlPSJj aXRlIj4KICAgICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVJieXl1OWd3UGZWcFB4cERhNF9n S1d5WHExUGF2VG0yVjJyRzJjVTBBdkU9SkpQQUBtYWlsLmdtYWlsLmNvbSIKICAgICAgICB0eXBl PSJjaXRlIj4KICAgICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICAgIDxkaXY+PGJyPgogICAg ICAgICAgICAmZ3Q7IE9LLCBpdCBsb29rcyB0aGF0IHNhbmxvY2sgwqBjYW4ndCB3b3JrIHdpdGgg ZW1wdHkgZmlsZQogICAgICAgICAgICBvciByZXdyaXRlIHRoZW0gLjxicj4KICAgICAgICAgICAg Jmd0OyBBbSBJIHJpZ2h0ID8/PGJyPgogICAgICAgICAgICA8YnI+CiAgICAgICAgICAgIFllcywg dGhlIGZpbGVzIG11c3QgYmUgaW5pdGlhbGl6ZWQgYmVmb3JlIHNhbmxvY2sgY2FuIHVzZQogICAg ICAgICAgICB0aGVtLjxicj4KICAgICAgICAgICAgPGJyPgogICAgICAgICAgICBZb3UgY2FuIGlu aXRpYWxpemUgdGhlIGZpbGUgbGlrZSB0aGlzOjxicj4KICAgICAgICAgICAgPGJyPgogICAgICAg ICAgICBzYW5sb2NrIGRpcmVjdCBpbml0IC1zCiAgICAgICAgICAgICZsdDtzZF91dWlkJmd0Ozow OnJlcGFpci8mbHQ7c2RfdXVpZCZndDsvZG9tX21kL2lkczowPGJyPgogICAgICAgICAgICA8YnI+ CiAgICAgICAgICAgIFRha2VuIGZyb20gPGEgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIgogICAgICAg ICAgICAgIGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvcGlwZXJtYWlsL3VzZXJzLzIwMTYt RmVicnVhcnkvMDM4MDQ2Lmh0bWwiPmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvcGlwZXJtYWlsL3Vz ZXJzLzIwMTYtRmVicnVhcnkvMDM4MDQ2Lmh0bWw8L2E+PGJyPgogICAgICAgICAgICA8YnI+CiAg ICAgICAgICAgICZndDsgVGhlIGxhc3QgcG9pbnQgLSBhYm91dCAiaWRzIiB3b3JrYXJvdW5kIC0g dGhpcyBpcwogICAgICAgICAgICBvZmZsaW5lIHZlcnNpb24gPSBWTXMgaGF2ZSB0byBiZSBtb3Zl ZCBvdXQgZnJvbSBmb3IKICAgICAgICAgICAgY29udGludWFsIHJ1bm5pbmcgd2l0aCBtYWludGVu YW5jZSB2b2x1bWUgbW9kZTxicj4KICAgICAgICAgICAgJmd0OyBCdXQgdGhpcyBpcyBub3QgYWNj ZXB0YWJsZSBpbiBjdXJyZW50IHNpdHVhdGlvbiwgc28gdGhlCiAgICAgICAgICAgIHF1ZXN0aW9u IGFnYWluLCDCoGlzIGl0IHNhZmUgdG8gZG8gaXQgb25saW5lID8/IMKgKCBZRVMgLyBOTyApPC9k aXY+CiAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPGRpdj5U aGUgaWRzIGZpbGUgaXMgYWNjZXNzZWQgb25seSBieSBzYW5sb2NrLiBJIGd1ZXNzIHRoYXQKICAg ICAgICAgICAgeW91IGRvbid0IGhhdmUgYSBydW5uaW5nPC9kaXY+CiAgICAgICAgICA8ZGl2PlNQ TSBvbiB0aGlzIERDLCBzaW5jZSBzYW5sb2NrIGZhaWxzIHRvIGFjcXVpcmUgYSBob3N0IGlkLAog ICAgICAgICAgICBzbyB5b3UgYXJlIHByZXR0eSBzYWZlPC9kaXY+CiAgICAgICAgICA8ZGl2PnRv IGZpeCB0aGUgcGVybWlzc2lvbnMgYW5kIGluaXRpYWxpemUgdGhlIGlkcyBmaWxlcy48L2Rpdj4K ICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgPC9kaXY+CiAgICAgICAgICA8ZGl2Pkkgd291 bGQgZG8gdGhpczo8L2Rpdj4KICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgPC9kaXY+CiAg ICAgICAgICA8ZGl2PjEuIFN0b3AgZW5naW5lLCDCoHNvIGl0IHdpbGwgbm90IHRyeSB0byBzdGFy dCB2ZHNtPC9kaXY+CiAgICAgICAgICA8ZGl2PjIuIFN0b3AgdmRzbSBvbiBhbGwgaG9zdHMsIHNv IHRoZXkgZG8gbm90IHRyeSB0byBhY3F1aXJlCiAgICAgICAgICAgIGEgaG9zdCBpZCB3aXRoIHNh bmxvY2s8L2Rpdj4KICAgICAgICAgIDxkaXY+wqAgwqAgVGhpcyBkb2VzIG5vdCBhZmZlY3QgcnVu bmluZyB2bXM8L2Rpdj4KICAgICAgICAgIDxkaXY+My4gRml4IHRoZSBwZXJtaXNzaW9ucyBvbiB0 aGUgaWRzIGZpbGUsIHZpYSBnbHVzdGVyZnMKICAgICAgICAgICAgbW91bnQ8L2Rpdj4KICAgICAg ICAgIDxkaXY+NC4gSW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGVzIGZyb20gb25lIG9mIHRoZSBob3N0 cywgdmlhCiAgICAgICAgICAgIHRoZSBnbHVzdGVyZnMgbW91bnQ8L2Rpdj4KICAgICAgICAgIDxk aXY+wqAgwqAgVGhpcyBzaG91bGQgZml4IHRoZSBpZHMgZmlsZXMgb24gYWxsIHJlcGxpY2FzPC9k aXY+CiAgICAgICAgICA8ZGl2PjUuIFN0YXJ0IHZkc20gb24gYWxsIGhvc3RzPC9kaXY+CiAgICAg ICAgICA8ZGl2PjYuIFN0YXJ0IGVuZ2luZTwvZGl2PgogICAgICAgICAgPGRpdj48YnI+CiAgICAg ICAgICA8L2Rpdj4KICAgICAgICAgIDxkaXY+RW5naW5lIHdpbGwgY29ubmVjdCB0byBhbGwgaG9z dHMsIGhvc3RzIHdpbGwgY29ubmVjdCB0bwogICAgICAgICAgICBzdG9yYWdlIGFuZCB0cnkgdG8g YWNxdWlyZSBhIGhvc3QgaWQuPC9kaXY+CiAgICAgICAgICA8ZGl2PlRoZW4gRW5naW5lIHdpbGwg c3RhcnQgdGhlIFNQTSBvbiBvbmUgb2YgdGhlIGhvc3RzLCBhbmQKICAgICAgICAgICAgeW91ciBE QyBzaG91bGQgYmVjb21lIHVwLjwvZGl2PgogICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICA8 L2Rpdj4KICAgICAgICAgIDxkaXY+RGF2aWQsIFNhaGluYSwgY2FuIHlvdSBjb25maXJtIHRoYXQg dGhpcyBwcm9jZWR1cmUgaXMKICAgICAgICAgICAgc2FmZT88L2Rpdj4KICAgICAgICA8L2Rpdj4K ICAgICAgPC9ibG9ja3F1b3RlPgogICAgICA8YnI+CiAgICAgIFllcywgY29ycmVjdGluZyBmcm9t IHRoZSBtb3VudCBwb2ludCBzaG91bGQgZml4IGl0IG9uIGFsbCByZXBsaWNhczxicj4KICAgICAg PGJyPgogICAgICA8YnI+CiAgICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQU1SYnl5dTlnd1Bm VnBQeHBEYTRfZ0tXeVhxMVBhdlRtMlYyckcyY1UwQXZFPUpKUEFAbWFpbC5nbWFpbC5jb20iCiAg ICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgICAgPGRpdiBkaXI9Imx0ciI+CiAgICAgICAgICA8ZGl2 Pjxicj4KICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPGRpdj5OaXI8L2Rpdj4KICAgICAgICAg IDxkaXY+PGJyPgogICAgICAgICAgPC9kaXY+CiAgICAgICAgICA8ZGl2PiZndDs8YnI+CiAgICAg ICAgICAgICZndDsgcmVncy48YnI+CiAgICAgICAgICAgICZndDsgUGF2ZWw8YnI+CiAgICAgICAg ICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAg ICAgICAgICAgICZndDsgT24gMS4zLjIwMTYgMTg6MzgsIE5pciBTb2ZmZXIgd3JvdGU6PGJyPgog ICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7IE9uIFR1ZSwgTWFyIDEsIDIwMTYg YXQgNTowNyBQTSwgPGEKICAgICAgICAgICAgICBtb3otZG8tbm90LXNlbmQ9InRydWUiIGNsYXNz PSJtb3otdHh0LWxpbmstYWJicmV2aWF0ZWQiCiAgICAgICAgICAgICAgaHJlZj0ibWFpbHRvOnBh ZjFAZW1haWwuY3oiPjxhIGNsYXNzPSJtb3otdHh0LWxpbmstYWJicmV2aWF0ZWQiIGhyZWY9Im1h aWx0bzpwYWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPjwvYT4gJmx0OzxhCiAgICAgICAg ICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIiBocmVmPSJtYWlsdG86cGFmMUBlbWFpbC5jeiI+ PGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIgaHJlZj0ibWFpbHRvOnBhZjFAZW1h aWwuY3oiPnBhZjFAZW1haWwuY3o8L2E+PC9hPiZndDsKCiAgICAgICAgICAgIHdyb3RlOjxicj4K ICAgICAgICAgICAgJmd0OyZndDs8YnI+CiAgICAgICAgICAgICZndDsmZ3Q7IEhlbGxvLCDCoGNh biBhbnlib2R5IMKgZXhwbGFpbiB0aGlzIGVycm9yIG5vLjEzICgKICAgICAgICAgICAgb3BlbiBm aWxlICkgaW4gc2FubG9jay5sb2cgLjxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAg ICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0OyBUaGlzIGlzIEVBQ0NFUzxicj4KICAgICAgICAg ICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0OyBDYW4geW91IHNoYXJlIHRoZSBvdXRvcHV0IG9m Ojxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0OyDCoCDCoCBscyAtbGgK L3JoZXYvZGF0YS1jZW50ZXIvbW50LyZsdDtzZXJ2ZXImZ3Q7OiZsdDtfcGF0aCZndDsvJmx0O3Nk X3V1aWQmZ3Q7L2RvbV9tZDxicj4KICAgICAgICAgICAgJmd0OyDCoDxicj4KICAgICAgICAgICAg Jmd0OyZndDs8YnI+CiAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0 OyBUaGUgc2l6ZSBvZiDCoCJpZHMiIGZpbGUgaXMgemVybyAoMCk8YnI+CiAgICAgICAgICAgICZn dDs8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsgVGhpcyBpcyBob3cg d2UgY3JlYXRlIHRoZSBpZHMgZmlsZSB3aGVuIGluaXRpYWxpemluZwogICAgICAgICAgICBpdC48 YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsgQnV0IHRoZW4gd2UgdXNl IHNhbmxvY2sgdG8gaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGUsIGFuZAogICAgICAgICAgICBpdCBz aG91bGQgYmUgMU1pQiBhZnRlciB0aGF0Ljxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAg ICAgICAgJmd0OyBJcyB0aGlzIGlkcyBmaWxlcyBjcmVhdGVkIGJ5IHZkc20sIG9yIG9uZSB5b3Ug Y3JlYXRlZAogICAgICAgICAgICB5b3Vyc2VsZj88YnI+CiAgICAgICAgICAgICZndDsgwqA8YnI+ CiAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0OyAyMDE2LTAyLTI4 IDAzOjI1OjQ2KzAxMDAgMjY5NjI2IFsxOTUxXTogb3BlbiBlcnJvcgogICAgICAgICAgICAtMTMK L3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUy YjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkczxicj4KICAgICAgICAg ICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IHMxODc5 ODUKICAgICAgICAgICAgb3Blbl9kaXNrCiAgICAgICAgICAgIC9yaGV2L2RhdGEtY2VudGVyL21u dC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEt MTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHMKCiAgICAgICAgICAgIGVycm9yIC0xMzxicj4KICAgICAg ICAgICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBz MTg3OTkyCiAgICAgICAgICAgIGxvY2tzcGFjZQo3ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJk NDQzMjcxMjQ6MTovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtW TTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzOjA8 YnI+CiAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0OyBJZiB0aGUg bWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXplLCBjYW4gSQogICAgICAgICAgICBy ZWdlbmVyYXRlIMKgdGhpcyBmaWxlIG9ubGluZSBzZWN1cmVseSAsIHdpdGggbm8gVk0KICAgICAg ICAgICAgZGVwZW5kZW5jZSDCoD8/Pz88YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAg ICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsgWWVzLCBJIHRoaW5rIEkgYWxyZWFkeSByZWZl cnJlZCB0byB0aGUgaW5zdHJ1Y3Rpb25zIGhvdwogICAgICAgICAgICB0byBkbyB0aGF0IGluIGEg cHJldmlvdXMgbWFpbC48YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsm Z3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgJmd0OyZndDsgZGlz dCA9IFJIRUwgLSA3IC0gMi4xNTExPGJyPgogICAgICAgICAgICAmZ3Q7Jmd0OyBrZXJuZWwgPSAz LjEwLjAgLSAzMjcuMTAuMS5lbDcueDg2XzY0PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0OyBLVk0g PSAyLjMuMCAtIDI5LjEuZWw3PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0OyBsaWJ2aXJ0ID0gbGli dmlydC0xLjIuMTctMTMuZWw3XzIuMzxicj4KICAgICAgICAgICAgJmd0OyZndDsgdmRzbSA9IHZk c20tNC4xNi4zMC0wLmVsNzxicj4KICAgICAgICAgICAgJmd0OyZndDsgR2x1c3RlckZTID0gZ2x1 c3RlcmZzLTMuNy44LTEuZWw3PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAg ICAgJmd0OyZndDs8YnI+CiAgICAgICAgICAgICZndDsmZ3Q7IHJlZ3MuPGJyPgogICAgICAgICAg ICAmZ3Q7Jmd0OyBQYXZlbDxicj4KICAgICAgICAgICAgJmd0OyZndDs8YnI+CiAgICAgICAgICAg ICZndDsmZ3Q7IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f PGJyPgogICAgICAgICAgICAmZ3Q7Jmd0OyBVc2VycyBtYWlsaW5nIGxpc3Q8YnI+CiAgICAgICAg ICAgICZndDsmZ3Q7IDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAgICAgICAgICBocmVm PSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIj5Vc2Vyc0BvdmlydC5vcmc8L2E+PGJyPgogICAgICAg ICAgICAmZ3Q7Jmd0OyA8YSBtb3otZG8tbm90LXNlbmQ9InRydWUiCiAgICAgICAgICAgICAgaHJl Zj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzIj5odHRwOi8v bGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnM8L2E+PGJyPgogICAgICAgICAg ICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0Ozxicj4K ICAgICAgICAgIDwvZGl2PgogICAgICAgIDwvZGl2PgogICAgICA8L2Jsb2NrcXVvdGU+CiAgICAg IDxicj4KICAgICAgPGJyPgogICAgICA8ZmllbGRzZXQgY2xhc3M9Im1pbWVBdHRhY2htZW50SGVh ZGVyIj48L2ZpZWxkc2V0PgogICAgICA8YnI+CiAgICAgIDxwcmUgd3JhcD0iIj5fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpHbHVzdGVyLXVzZXJzIG1haWxp bmcgbGlzdAo8YSBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIiBocmVmPSJtYWlsdG86 R2x1c3Rlci11c2Vyc0BnbHVzdGVyLm9yZyI+R2x1c3Rlci11c2Vyc0BnbHVzdGVyLm9yZzwvYT4K PGEgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIgaHJlZj0iaHR0cDovL3d3dy5nbHVzdGVy Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2dsdXN0ZXItdXNlcnMiPmh0dHA6Ly93d3cuZ2x1c3Rlci5v cmcvbWFpbG1hbi9saXN0aW5mby9nbHVzdGVyLXVzZXJzPC9hPjwvcHJlPgogICAgPC9ibG9ja3F1 b3RlPgogICAgPGJyPgogICAgPGJyPgogIDwvYm9keT4KPC9odG1sPgoKLS0tLS0tLS0tLS0tLS0w NTA0MDIwMTA4MDkwNjA1MDkwODA0MDYtLQo= --===============4670495215605840215==-- From paf1 at email.cz Wed Mar 2 05:36:26 2016 Content-Type: multipart/mixed; boundary="===============2054102223816475046==" MIME-Version: 1.0 From: paf1 at email.cz To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Wed, 02 Mar 2016 11:36:19 +0100 Message-ID: <56D6C223.4080602@email.cz> In-Reply-To: 56D69365.4090303@redhat.com --===============2054102223816475046== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------020603060204030604090206 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit Hi guys, thx a lot for your support .......at first. Because we had been under huge time pressure, we found "google = workaround" which delete both files . It helped, probabbly at first = steps of recover . eg: " # find /STORAGES/g1r5p5/GFS/ -samefile = /STORAGES/g1r5p5/GFS/3da46e07-d1ea-4f10-9250-6cbbb7b94d80/dom_md/ids = -print -delete " ----------------------> Well at first I'll fix permittions from mount points to 660 . If "ids" file will be writeable , can't became gluster colaps ?? regs.Pavel On 2.3.2016 08:16, Ravishankar N wrote: > On 03/02/2016 12:02 PM, Sahina Bose wrote: >> >> >> On 03/02/2016 03:45 AM, Nir Soffer wrote: >>> On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz wrot= e: >>> > >>> > HI, >>> > requested output: >>> > >>> > # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47b= e-bef3-0bc0b7a99fb7/dom_md: >>> > total 2,1M >>> > -rw-rw---- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- good >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.17 leases >>> > -rw-r--r-- 1 vdsm kvm 335 7. lis 22.17 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042= -a579-4cabeb55ded4/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.41 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 03.56 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 03.56 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1= -9992-b04464a04112/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.14 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.14 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23= -ab46-084f3d70b147/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.51 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases >>> > -rw-r--r-- 1 vdsm kvm 998 25. =C3=BAno 00.35 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.16 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58= -89aa-102d44327124/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.44 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 00.18 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 00.18 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc= -be41-77002d1cd553/dom_md: >>> > total 1,1M >>> > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.32 ids <-- bad (other can= read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.18 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.18 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528= -9bfe-c3da888e47f0/dom_md: >>> > total 3,0M >>> > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other can= read) >>> > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.42 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases >>> > -rw-r--r-- 1 vdsm kvm 997 24. =C3=BAno 02.46 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.44 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4= -ba9a-e6d9bdecb4b4/dom_md: >>> > total 2,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.34 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.35 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases >>> > -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.27 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-470= 3-9bb1-1df2130c7c97/dom_md: >>> > total 3,0M >>> > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other can= read) >>> > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 23.50 inbox <-- bad (other can = >>> read) >>> > -rw-rw-r-- 1 vdsm kvm 2,0M 6. lis 23.51 leases <-- bad = >>> (other can read) >>> > -rw-rw-r-- 1 vdsm kvm 734 7. lis 02.13 metadata <-- bad = >>> (group can write, other can read) >>> > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 16.55 outbox <-- bad (other = >>> can read) >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22= -9f59-3aebea4d8477/dom_md: >>> > total 1,1M >>> > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.35 ids <-- bad (other can= read) >>> > -rw-rw-r-- 1 vdsm kvm 16M 24. =C3=BAno 01.06 inbox >>> > -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases >>> > -rw-r--r-- 1 vdsm kvm 998 24. =C3=BAno 19.07 metadata >>> > -rw-rw-r-- 1 vdsm kvm 16M 7. lis 22.20 outbox >>> >>> >>> It should look like this: >>> >>> -rw-rw----. 1 vdsm kvm 1.0M Mar 1 23:36 ids >>> -rw-rw----. 1 vdsm kvm 2.0M Mar 1 23:35 leases >>> -rw-r--r--. 1 vdsm kvm 353 Mar 1 23:35 metadata >>> -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 outbox >>> -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 inbox >>> >>> This explains the EACCES error. >>> >>> You can start by fixing the permissions manually, you can do this = >>> online. >>> >>> > The ids files was generated by "touch" command after deleting = >>> them due "sanlock locking hang" gluster crash & reboot >>> > I expected that they will be filled automaticaly after gluster = >>> reboot ( the shadow copy from ".gluster " directory was = >>> deleted & created empty too ) >>> >>> I don't know about gluster shadow copy, I would not play with = >>> gluster internals. >>> Adding Sahina for advice. >> >> Did you generate the ids file on the mount point. >> >> Ravi, can you help here? >> > > Okay, so what I understand from the output above is you have different = > gluster volumes mounted and some of them have incorrect permissions = > for the 'ids' file. The way to fix it is to do it from the mount like = > Nir said. > Why did you delete the file from the .glusterfs in the brick(s)? Was = > there a gfid split brain? > > -Ravi > >>> >>> > OK, it looks that sanlock can't work with empty file or rewrite = >>> them . >>> > Am I right ?? >>> >>> Yes, the files must be initialized before sanlock can use them. >>> >>> You can initialize the file like this: >>> >>> sanlock direct init -s :0:repair//dom_md/ids:0 >>> >>> Taken from = >>> http://lists.ovirt.org/pipermail/users/2016-February/038046.html >>> >>> > The last point - about "ids" workaround - this is offline version = >>> =3D VMs have to be moved out from for continual running with = >>> maintenance volume mode >>> > But this is not acceptable in current situation, so the question = >>> again, is it safe to do it online ?? ( YES / NO ) >>> >>> The ids file is accessed only by sanlock. I guess that you don't = >>> have a running >>> SPM on this DC, since sanlock fails to acquire a host id, so you are = >>> pretty safe >>> to fix the permissions and initialize the ids files. >>> >>> I would do this: >>> >>> 1. Stop engine, so it will not try to start vdsm >>> 2. Stop vdsm on all hosts, so they do not try to acquire a host id = >>> with sanlock >>> This does not affect running vms >>> 3. Fix the permissions on the ids file, via glusterfs mount >>> 4. Initialize the ids files from one of the hosts, via the glusterfs = >>> mount >>> This should fix the ids files on all replicas >>> 5. Start vdsm on all hosts >>> 6. Start engine >>> >>> Engine will connect to all hosts, hosts will connect to storage and = >>> try to acquire a host id. >>> Then Engine will start the SPM on one of the hosts, and your DC = >>> should become up. >>> >>> David, Sahina, can you confirm that this procedure is safe? >> >> Yes, correcting from the mount point should fix it on all replicas >> >> >>> >>> Nir >>> >>> > >>> > regs. >>> > Pavel >>> > >>> > >>> > >>> > On 1.3.2016 18:38, Nir Soffer wrote: >>> > >>> > On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz wro= te: >>> >> >>> >> Hello, can anybody explain this error no.13 ( open file ) in = >>> sanlock.log . >>> > >>> > >>> > This is EACCES >>> > >>> > Can you share the outoput of: >>> > >>> > ls -lh /rhev/data-center/mnt/:<_path>//dom_md >>> > >>> >> >>> >> >>> >> The size of "ids" file is zero (0) >>> > >>> > >>> > This is how we create the ids file when initializing it. >>> > >>> > But then we use sanlock to initialize the ids file, and it should = >>> be 1MiB after that. >>> > >>> > Is this ids files created by vdsm, or one you created yourself? >>> > >>> >> >>> >> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58= -89aa-102d44327124/dom_md/ids >>> >> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58= -89aa-102d44327124/dom_md/ids = >>> error -13 >>> >> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace = >>> 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/= localhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 >>> >> >>> >> If the main problem is about zero file size, can I regenerate = >>> this file online securely , with no VM dependence ???? >>> > >>> > >>> > Yes, I think I already referred to the instructions how to do that = >>> in a previous mail. >>> > >>> >> >>> >> >>> >> dist =3D RHEL - 7 - 2.1511 >>> >> kernel =3D 3.10.0 - 327.10.1.el7.x86_64 >>> >> KVM =3D 2.3.0 - 29.1.el7 >>> >> libvirt =3D libvirt-1.2.17-13.el7_2.3 >>> >> vdsm =3D vdsm-4.16.30-0.el7 >>> >> GlusterFS =3D glusterfs-3.7.8-1.el7 >>> >> >>> >> >>> >> regs. >>> >> Pavel >>> >> >>> >> _______________________________________________ >>> >> Users mailing list >>> >> Users(a)ovirt.org >>> >> http://lists.ovirt.org/mailman/listinfo/users >>> >> >>> > >>> > >> >> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users(a)gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-users > > --------------020603060204030604090206 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit Hi guys,
thx a lot for your support .......at first.

Because we had been under huge time pressure, we found "google workaround"=C2=A0 which delete both files . It helped, probabbly at fir= st steps of recover .
eg: " #=C2=A0 find /STORAGES/g1r5p5/GFS/ -samefile /STORAGES/g1r5p5/GFS/3da46e07-d1ea-4f10-9250-6cbbb7b94d80/dom_md/ids -print -delete "

---------------------->
Well at first I'll=C2=A0 fix permittions from mount points=C2=A0 to 660= .
If "ids"=C2=A0 file will be writeable , can't=C2=A0 became gluster cola= ps ??

regs.Pavel

On 2.3.2016 08:16, Ravishankar N wrote:<= br>

On 03/02/2016 12:02 PM, Sahina Bose wrote:

On 03/02/2016 03:45 AM, Nir Soffer wrote:

On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz <paf1(a)email.cz> wrote= :
>
> HI,
> requested output:
>
> # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md
> =C2=A0
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-be= f3-0bc0b7a99fb7/dom_md:
> total 2,1M
> -rw-rw---- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids =C2= =A0 =C2=A0 =C2=A0 =C2=A0<-- good
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.17 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0335 =C2=A07. lis 22.17 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a57= 9-4cabeb55ded4/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.41 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 03.56 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 03.56 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-999= 2-b04464a04112/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.14 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.14 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-ab4= 6-084f3d70b147/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.51 inbox > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 25. =C3=BAno 00.35 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.16 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.44 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 00.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 00.18 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-be4= 1-77002d1cd553/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.32 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.18 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9bf= e-c3da888e47f0/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.42 inbox= =C2=A0
> -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0997 24. =C3=BAno 02.46 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.44 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-ba9= a-e6d9bdecb4b4/dom_md:
> total 2,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.34 id= s=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.35 inbox > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases
> -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.27 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-9b= b1-1df2130c7c97/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 23.50 inbox= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm 2,0M =C2=A06. lis 23.51 leases= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A0734 =C2=A07. lis 02.13 metad= ata=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (group can write, other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 16.55 outbo= x=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9f5= 9-3aebea4d8477/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.35 = ids=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M 24. =C3=BAno 01.06 inbox=
> -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 24. =C3=BAno 19.07 metad= ata
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.20 outbo= x

It should look like this:

-rw-rw----. 1 vdsm kvm 1.0M Mar =C2=A01 23:36 ids
-rw-rw----. 1 vdsm kvm 2.0M Mar =C2=A01 23:35 leases
-rw-r--r--. 1 vdsm kvm =C2=A0353 Mar =C2=A01 23:35 metadata -rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 outbox
-rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 inbox

This explains the EACCES error.

You can start by fixing the permissions manually, you can do this online.
=C2=A0
> =C2=A0The ids files was generated by "touch" command aft= er deleting them due "sanlock locking hang" =C2=A0gluster crash & reboot
> I expected that they will be filled automaticaly after gluster reboot ( the =C2=A0shadow copy from =C2=A0 ".gl= uster " =C2=A0 directory =C2=A0was deleted & created empty =C2=A0= too )

I don't know about gluster shadow copy, I would not play with gluster internals.

Adding Sahina for advice.

Did you generate the ids file on the mount point.

Ravi, can you help here?

Okay, so what I understand from the output above is you have different gluster volumes mounted and some of them have incorrect permissions for the 'ids' file. The way to fix it is to do it from the mount like Nir said.
Why did you delete the file from the .glusterfs in the brick(s)?=C2= =A0 Was there a gfid split brain?

-Ravi
> OK, it looks that sanlock =C2=A0can't work with empty fi= le or rewrite them .
> Am I right ??

Yes, the files must be initialized before sanlock can use them.

You can initialize the file like this:

sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0

Taken from http://lists.ovirt.org/pipermail/users/2016-February/038046.= html

> The last point - about "ids" workaround - this is offline version =3D VMs have to be moved out from for continual running with maintenance volume mode
> But this is not acceptable in current situation, so the question again, =C2=A0is it safe to do it online ?? =C2= =A0( YES / NO )

The ids file is accessed only by sanlock. I guess that you don't have a running

SPM on this DC, since sanlock fails to acquire a host id, so you are pretty safe

to fix the permissions and initialize the ids files.

I would do this:

1. Stop engine, =C2=A0so it will not try to start vdsm
2. Stop vdsm on all hosts, so they do not try to acquire a host id with sanlock

=C2=A0 =C2=A0 This does not affect running vms

3. Fix the permissions on the ids file, via glusterfs mount

4. Initialize the ids files from one of the hosts, via the glusterfs mount

=C2=A0 =C2=A0 This should fix the ids files on all replica= s

5. Start vdsm on all hosts

6. Start engine

Engine will connect to all hosts, hosts will connect to storage and try to acquire a host id.

Then Engine will start the SPM on one of the hosts, and your DC should become up.

David, Sahina, can you confirm that this procedure is safe?

Yes, correcting from the mount point should fix it on all replicas

Nir

>
> regs.
> Pavel
>
>
>
> On 1.3.2016 18:38, Nir Soffer wrote:
>
> On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz <paf1(a)email.cz> wro= te:
>>
>> Hello, =C2=A0can anybody =C2=A0explain this error no= .13 ( open file ) in sanlock.log .
>
>
> This is EACCES
>
> Can you share the outoput of:
>
> =C2=A0 =C2=A0 ls -lh /rhev/data-center/mnt/<server>:<_path>/<sd_uuid>/dom_md > =C2=A0
>>
>>
>> The size of =C2=A0"ids" file is zero (0)
>
>
> This is how we create the ids file when initializing it.
>
> But then we use sanlock to initialize the ids file, and it should be 1MiB after that.
>
> Is this ids files created by vdsm, or one you created yourself?
> =C2=A0
>>
>> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids
>> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697= -c199-4f58-89aa-102d44327124/dom_md/ids error -13
>> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0
>>
>> If the main problem is about zero file size, can I regenerate =C2=A0this file online securely , with no VM dependence =C2=A0????
>
>
> Yes, I think I already referred to the instructions how to do that in a previous mail.
>
>>
>>
>> dist =3D RHEL - 7 - 2.1511
>> kernel =3D 3.10.0 - 327.10.1.el7.x86_64
>> KVM =3D 2.3.0 - 29.1.el7
>> libvirt =3D libvirt-1.2.17-13.el7_2.3
>> vdsm =3D vdsm-4.16.30-0.el7
>> GlusterFS =3D glusterfs-3.7.8-1.el7
>>
>>
>> regs.
>> Pavel
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http= ://lists.ovirt.org/mailman/listinfo/users
>>
>
>
_______________________________________________
Gluster-users mailing list
Gluster-users(a)gluster.org
http://www.gluster.org/mail=
man/listinfo/gluster-users

--------------020603060204030604090206-- --===============2054102223816475046== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMjA2MDMwNjAyMDQwMzA2MDQwOTAyMDYKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK CkhpIGd1eXMsCnRoeCBhIGxvdCBmb3IgeW91ciBzdXBwb3J0IC4uLi4uLi5hdCBmaXJzdC4KCkJl Y2F1c2Ugd2UgaGFkIGJlZW4gdW5kZXIgaHVnZSB0aW1lIHByZXNzdXJlLCB3ZSBmb3VuZCAiZ29v Z2xlIAp3b3JrYXJvdW5kIiAgd2hpY2ggZGVsZXRlIGJvdGggZmlsZXMgLiBJdCBoZWxwZWQsIHBy b2JhYmJseSBhdCBmaXJzdCAKc3RlcHMgb2YgcmVjb3ZlciAuCmVnOiAiICMgIGZpbmQgL1NUT1JB R0VTL2cxcjVwNS9HRlMvIC1zYW1lZmlsZSAKL1NUT1JBR0VTL2cxcjVwNS9HRlMvM2RhNDZlMDct ZDFlYS00ZjEwLTkyNTAtNmNiYmI3Yjk0ZDgwL2RvbV9tZC9pZHMgCi1wcmludCAtZGVsZXRlICIK Ci0tLS0tLS0tLS0tLS0tLS0tLS0tLS0+CldlbGwgYXQgZmlyc3QgSSdsbCAgZml4IHBlcm1pdHRp b25zIGZyb20gbW91bnQgcG9pbnRzICB0byA2NjAgLgpJZiAiaWRzIiAgZmlsZSB3aWxsIGJlIHdy aXRlYWJsZSAsIGNhbid0ICBiZWNhbWUgZ2x1c3RlciBjb2xhcHMgPz8KCnJlZ3MuUGF2ZWwKCgpP biAyLjMuMjAxNiAwODoxNiwgUmF2aXNoYW5rYXIgTiB3cm90ZToKPiBPbiAwMy8wMi8yMDE2IDEy OjAyIFBNLCBTYWhpbmEgQm9zZSB3cm90ZToKPj4KPj4KPj4gT24gMDMvMDIvMjAxNiAwMzo0NSBB TSwgTmlyIFNvZmZlciB3cm90ZToKPj4+IE9uIFR1ZSwgTWFyIDEsIDIwMTYgYXQgMTA6NTEgUE0s IHBhZjFAZW1haWwuY3ogPHBhZjFAZW1haWwuY3o+IHdyb3RlOgo+Pj4gPgo+Pj4gPiBISSwKPj4+ ID4gcmVxdWVzdGVkIG91dHB1dDoKPj4+ID4KPj4+ID4gIyBscyAtbGggL3JoZXYvZGF0YS1jZW50 ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6Ki8qL2RvbV9tZAo+Pj4gPgo+Pj4gPiAKPj4+IC9y aGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItQkNLLzBmY2Fk ODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21fbWQ6Cj4+PiA+IHRvdGFsIDIsMU0K Pj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDEsME0gIDEuIGLFmWUgMjEuMjggaWRzICA8LS0g Z29vZAo+Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDIyLjE2IGluYm94 Cj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNICA3LiBsaXMgMjIuMTcgbGVhc2VzCj4+ PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzM1ICA3LiBsaXMgMjIuMTcgbWV0YWRhdGEKPj4+ ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAyMi4xNiBvdXRib3gKPj4+ID4K Pj4+ID4gCj4+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtW TTEyLVAxLzU1M2Q5YjkyLWU0YTAtNDA0Mi1hNTc5LTRjYWJlYjU1ZGVkNC9kb21fbWQ6Cj4+PiA+ IHRvdGFsIDEsMU0KPj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcu NDEgaWRzICA8LS0gYmFkIChzYW5sb2NrIAo+Pj4gY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVh ZCkKPj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNCBpbmJveAo+ Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDAzLjU2IGxlYXNlcwo+Pj4g PiAtcnctci0tci0tIDEgdmRzbSBrdm0gIDMzMyAgNy4gbGlzIDAzLjU2IG1ldGFkYXRhCj4+PiA+ IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTQgb3V0Ym94Cj4+PiA+Cj4+ PiA+IAo+Pj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0x Mi1QMi84OGFkYmQ0OS02MmQ2LTQ1YjEtOTk5Mi1iMDQ0NjRhMDQxMTIvZG9tX21kOgo+Pj4gPiB0 b3RhbCAxLDFNCj4+PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjQz IGlkcyAgPC0tIGJhZCAoc2FubG9jayAKPj4+IGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQp Cj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTUgaW5ib3gKPj4+ ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gIDcuIGxpcyAyMi4xNCBsZWFzZXMKPj4+ID4g LXJ3LXItLXItLSAxIHZkc20ga3ZtICAzMzMgIDcuIGxpcyAyMi4xNCBtZXRhZGF0YQo+Pj4gPiAt cnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE1IG91dGJveAo+Pj4gPgo+Pj4g PiAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTIt UDMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZDoKPj4+ID4gdG90 YWwgMSwxTQo+Pj4gPiAtcnctci0tci0tIDEgdmRzbSBrdm0gICAgMCAyNC4gw7pubyAwNy40MyBp ZHMgIDwtLSBiYWQgKHNhbmxvY2sgCj4+PiBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKQo+ Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyMy4gw7pubyAyMi41MSBpbmJveAo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4gw7pubyAyMy4xMiBsZWFzZXMKPj4+ID4g LXJ3LXItLXItLSAxIHZkc20ga3ZtICA5OTggMjUuIMO6bm8gMDAuMzUgbWV0YWRhdGEKPj4+ID4g LXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNiBvdXRib3gKPj4+ID4KPj4+ ID4gCj4+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEy LVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQ6Cj4+PiA+IHRv dGFsIDEsMU0KPj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuNDQg aWRzICA8LS0gYmFkIChzYW5sb2NrIAo+Pj4gY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCkK Pj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNyBpbmJveAo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDAwLjE4IGxlYXNlcwo+Pj4gPiAt cnctci0tci0tIDEgdmRzbSBrdm0gIDMzMyAgNy4gbGlzIDAwLjE4IG1ldGFkYXRhCj4+PiA+IC1y dy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTcgb3V0Ym94Cj4+PiA+Cj4+PiA+ IAo+Pj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1Q MS80MmQ3MTBhOS1iODQ0LTQzZGMtYmU0MS03NzAwMmQxY2Q1NTMvZG9tX21kOgo+Pj4gPiB0b3Rh bCAxLDFNCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjMyIGlk cyAgPC0tIGJhZCAob3RoZXIgY2FuIHJlYWQpCj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAg MTZNICA3LiBsaXMgMjIuMTggaW5ib3gKPj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0g IDcuIGxpcyAyMi4xOCBsZWFzZXMKPj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAzMzMgIDcu IGxpcyAyMi4xOCBtZXRhZGF0YQo+Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4g bGlzIDIyLjE4IG91dGJveAo+Pj4gPgo+Pj4gPiAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9n bHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDIvZmY3MWI0N2ItMGY3Mi00NTI4LTliZmUtYzNk YTg4OGU0N2YwL2RvbV9tZDoKPj4+ID4gdG90YWwgMywwTQo+Pj4gPiAtcnctcnctci0tIDEgdmRz bSBrdm0gMSwwTSAgMS4gYsWZZSAyMS4yOCBpZHMgIDwtLSBiYWQgKG90aGVyIGNhbiByZWFkKQo+ Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyNS4gw7pubyAwMC40MiBpbmJveAo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyNS4gw7pubyAwMC40NCBsZWFzZXMKPj4+ID4g LXJ3LXItLXItLSAxIHZkc20ga3ZtICA5OTcgMjQuIMO6bm8gMDIuNDYgbWV0YWRhdGEKPj4+ID4g LXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gMjUuIMO6bm8gMDAuNDQgb3V0Ym94Cj4+PiA+Cj4+ PiA+IAo+Pj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0x Mi1QMy9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRlY2I0YjQvZG9tX21kOgo+Pj4gPiB0 b3RhbCAyLDFNCj4+PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjM0 IGlkcyAgPC0tIGJhZCAoc2FubG9jayAKPj4+IGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQp Cj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNIDIzLiDDum5vIDIyLjM1IGluYm94Cj4+ PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDIzLiDDum5vIDIyLjM4IGxlYXNlcwo+Pj4g PiAtcnctci0tci0tIDEgdmRzbSBrdm0gMSwxSyAyNC4gw7pubyAxOS4wNyBtZXRhZGF0YQo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyMy4gw7pubyAyMi4yNyBvdXRib3gKPj4+ID4K Pj4+ID4gCj4+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktW TTEyX19QNC8zMDBlOWFjOC0zYzJmLTQ3MDMtOWJiMS0xZGYyMTMwYzdjOTcvZG9tX21kOgo+Pj4g PiB0b3RhbCAzLDBNCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAxLDBNICAxLiBixZllIDIx LjI4IGlkcyAgPC0tIGJhZCAob3RoZXIgY2FuIHJlYWQpCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNt IGt2bSAgMTZNICA2LiBsaXMgMjMuNTAgaW5ib3ggIDwtLSBiYWQgKG90aGVyIGNhbiAKPj4+IHJl YWQpCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAyLDBNICA2LiBsaXMgMjMuNTEgbGVhc2Vz ICAgICAgPC0tIGJhZCAKPj4+IChvdGhlciBjYW4gcmVhZCkKPj4+ID4gLXJ3LXJ3LXItLSAxIHZk c20ga3ZtICA3MzQgIDcuIGxpcyAwMi4xMyBtZXRhZGF0YSAgICA8LS0gYmFkIAo+Pj4gKGdyb3Vw IGNhbiB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAg MTZNICA2LiBsaXMgMTYuNTUgb3V0Ym94ICA8LS0gYmFkIChvdGhlciAKPj4+IGNhbiByZWFkKQo+ Pj4gPgo+Pj4gPiAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0 Ol8yS1ZNMTItUDUvMWNhNTZiNDUtNzAxZS00YzIyLTlmNTktM2FlYmVhNGQ4NDc3L2RvbV9tZDoK Pj4+ID4gdG90YWwgMSwxTQo+Pj4gPiAtcnctcnctci0tIDEgdmRzbSBrdm0gICAgMCAyNC4gw7pu byAwNy4zNSBpZHMgIDwtLSBiYWQgKG90aGVyIGNhbiByZWFkKQo+Pj4gPiAtcnctcnctci0tIDEg dmRzbSBrdm0gIDE2TSAyNC4gw7pubyAwMS4wNiBpbmJveAo+Pj4gPiAtcnctcnctci0tIDEgdmRz bSBrdm0gMiwwTSAyNC4gw7pubyAwMi40NCBsZWFzZXMKPj4+ID4gLXJ3LXItLXItLSAxIHZkc20g a3ZtICA5OTggMjQuIMO6bm8gMTkuMDcgbWV0YWRhdGEKPj4+ID4gLXJ3LXJ3LXItLSAxIHZkc20g a3ZtICAxNk0gIDcuIGxpcyAyMi4yMCBvdXRib3gKPj4+Cj4+Pgo+Pj4gSXQgc2hvdWxkIGxvb2sg bGlrZSB0aGlzOgo+Pj4KPj4+IC1ydy1ydy0tLS0uIDEgdmRzbSBrdm0gMS4wTSBNYXIgIDEgMjM6 MzYgaWRzCj4+PiAtcnctcnctLS0tLiAxIHZkc20ga3ZtIDIuME0gTWFyICAxIDIzOjM1IGxlYXNl cwo+Pj4gLXJ3LXItLXItLS4gMSB2ZHNtIGt2bSAgMzUzIE1hciAgMSAyMzozNSBtZXRhZGF0YQo+ Pj4gLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAgMTZNIE1hciAgMSAyMzozNCBvdXRib3gKPj4+IC1y dy1ydy0tLS0uIDEgdmRzbSBrdm0gIDE2TSBNYXIgIDEgMjM6MzQgaW5ib3gKPj4+Cj4+PiBUaGlz IGV4cGxhaW5zIHRoZSBFQUNDRVMgZXJyb3IuCj4+Pgo+Pj4gWW91IGNhbiBzdGFydCBieSBmaXhp bmcgdGhlIHBlcm1pc3Npb25zIG1hbnVhbGx5LCB5b3UgY2FuIGRvIHRoaXMgCj4+PiBvbmxpbmUu Cj4+Pgo+Pj4gPiAgVGhlIGlkcyBmaWxlcyB3YXMgZ2VuZXJhdGVkIGJ5ICJ0b3VjaCIgY29tbWFu ZCBhZnRlciBkZWxldGluZyAKPj4+IHRoZW0gZHVlICJzYW5sb2NrIGxvY2tpbmcgaGFuZyIgIGds dXN0ZXIgY3Jhc2ggJiByZWJvb3QKPj4+ID4gSSBleHBlY3RlZCB0aGF0IHRoZXkgd2lsbCBiZSBm aWxsZWQgYXV0b21hdGljYWx5IGFmdGVyIGdsdXN0ZXIgCj4+PiByZWJvb3QgKCB0aGUgIHNoYWRv dyBjb3B5IGZyb20gICAiLmdsdXN0ZXIgIiAgIGRpcmVjdG9yeSAgd2FzIAo+Pj4gZGVsZXRlZCAm IGNyZWF0ZWQgZW1wdHkgIHRvbyApCj4+Pgo+Pj4gSSBkb24ndCBrbm93IGFib3V0IGdsdXN0ZXIg c2hhZG93IGNvcHksIEkgd291bGQgbm90IHBsYXkgd2l0aCAKPj4+IGdsdXN0ZXIgaW50ZXJuYWxz Lgo+Pj4gQWRkaW5nIFNhaGluYSBmb3IgYWR2aWNlLgo+Pgo+PiBEaWQgeW91IGdlbmVyYXRlIHRo ZSBpZHMgZmlsZSBvbiB0aGUgbW91bnQgcG9pbnQuCj4+Cj4+IFJhdmksIGNhbiB5b3UgaGVscCBo ZXJlPwo+Pgo+Cj4gT2theSwgc28gd2hhdCBJIHVuZGVyc3RhbmQgZnJvbSB0aGUgb3V0cHV0IGFi b3ZlIGlzIHlvdSBoYXZlIGRpZmZlcmVudCAKPiBnbHVzdGVyIHZvbHVtZXMgbW91bnRlZCBhbmQg c29tZSBvZiB0aGVtIGhhdmUgaW5jb3JyZWN0IHBlcm1pc3Npb25zIAo+IGZvciB0aGUgJ2lkcycg ZmlsZS4gVGhlIHdheSB0byBmaXggaXQgaXMgdG8gZG8gaXQgZnJvbSB0aGUgbW91bnQgbGlrZSAK PiBOaXIgc2FpZC4KPiBXaHkgZGlkIHlvdSBkZWxldGUgdGhlIGZpbGUgZnJvbSB0aGUgLmdsdXN0 ZXJmcyBpbiB0aGUgYnJpY2socyk/IFdhcyAKPiB0aGVyZSBhIGdmaWQgc3BsaXQgYnJhaW4/Cj4K PiAtUmF2aQo+Cj4+Pgo+Pj4gPiBPSywgaXQgbG9va3MgdGhhdCBzYW5sb2NrICBjYW4ndCB3b3Jr IHdpdGggZW1wdHkgZmlsZSBvciByZXdyaXRlIAo+Pj4gdGhlbSAuCj4+PiA+IEFtIEkgcmlnaHQg Pz8KPj4+Cj4+PiBZZXMsIHRoZSBmaWxlcyBtdXN0IGJlIGluaXRpYWxpemVkIGJlZm9yZSBzYW5s b2NrIGNhbiB1c2UgdGhlbS4KPj4+Cj4+PiBZb3UgY2FuIGluaXRpYWxpemUgdGhlIGZpbGUgbGlr ZSB0aGlzOgo+Pj4KPj4+IHNhbmxvY2sgZGlyZWN0IGluaXQgLXMgPHNkX3V1aWQ+OjA6cmVwYWly LzxzZF91dWlkPi9kb21fbWQvaWRzOjAKPj4+Cj4+PiBUYWtlbiBmcm9tIAo+Pj4gaHR0cDovL2xp c3RzLm92aXJ0Lm9yZy9waXBlcm1haWwvdXNlcnMvMjAxNi1GZWJydWFyeS8wMzgwNDYuaHRtbAo+ Pj4KPj4+ID4gVGhlIGxhc3QgcG9pbnQgLSBhYm91dCAiaWRzIiB3b3JrYXJvdW5kIC0gdGhpcyBp cyBvZmZsaW5lIHZlcnNpb24gCj4+PiA9IFZNcyBoYXZlIHRvIGJlIG1vdmVkIG91dCBmcm9tIGZv ciBjb250aW51YWwgcnVubmluZyB3aXRoIAo+Pj4gbWFpbnRlbmFuY2Ugdm9sdW1lIG1vZGUKPj4+ ID4gQnV0IHRoaXMgaXMgbm90IGFjY2VwdGFibGUgaW4gY3VycmVudCBzaXR1YXRpb24sIHNvIHRo ZSBxdWVzdGlvbiAKPj4+IGFnYWluLCAgaXMgaXQgc2FmZSB0byBkbyBpdCBvbmxpbmUgPz8gICgg WUVTIC8gTk8gKQo+Pj4KPj4+IFRoZSBpZHMgZmlsZSBpcyBhY2Nlc3NlZCBvbmx5IGJ5IHNhbmxv Y2suIEkgZ3Vlc3MgdGhhdCB5b3UgZG9uJ3QgCj4+PiBoYXZlIGEgcnVubmluZwo+Pj4gU1BNIG9u IHRoaXMgREMsIHNpbmNlIHNhbmxvY2sgZmFpbHMgdG8gYWNxdWlyZSBhIGhvc3QgaWQsIHNvIHlv dSBhcmUgCj4+PiBwcmV0dHkgc2FmZQo+Pj4gdG8gZml4IHRoZSBwZXJtaXNzaW9ucyBhbmQgaW5p dGlhbGl6ZSB0aGUgaWRzIGZpbGVzLgo+Pj4KPj4+IEkgd291bGQgZG8gdGhpczoKPj4+Cj4+PiAx LiBTdG9wIGVuZ2luZSwgIHNvIGl0IHdpbGwgbm90IHRyeSB0byBzdGFydCB2ZHNtCj4+PiAyLiBT dG9wIHZkc20gb24gYWxsIGhvc3RzLCBzbyB0aGV5IGRvIG5vdCB0cnkgdG8gYWNxdWlyZSBhIGhv c3QgaWQgCj4+PiB3aXRoIHNhbmxvY2sKPj4+ICAgICBUaGlzIGRvZXMgbm90IGFmZmVjdCBydW5u aW5nIHZtcwo+Pj4gMy4gRml4IHRoZSBwZXJtaXNzaW9ucyBvbiB0aGUgaWRzIGZpbGUsIHZpYSBn bHVzdGVyZnMgbW91bnQKPj4+IDQuIEluaXRpYWxpemUgdGhlIGlkcyBmaWxlcyBmcm9tIG9uZSBv ZiB0aGUgaG9zdHMsIHZpYSB0aGUgZ2x1c3RlcmZzIAo+Pj4gbW91bnQKPj4+ICAgICBUaGlzIHNo b3VsZCBmaXggdGhlIGlkcyBmaWxlcyBvbiBhbGwgcmVwbGljYXMKPj4+IDUuIFN0YXJ0IHZkc20g b24gYWxsIGhvc3RzCj4+PiA2LiBTdGFydCBlbmdpbmUKPj4+Cj4+PiBFbmdpbmUgd2lsbCBjb25u ZWN0IHRvIGFsbCBob3N0cywgaG9zdHMgd2lsbCBjb25uZWN0IHRvIHN0b3JhZ2UgYW5kIAo+Pj4g dHJ5IHRvIGFjcXVpcmUgYSBob3N0IGlkLgo+Pj4gVGhlbiBFbmdpbmUgd2lsbCBzdGFydCB0aGUg U1BNIG9uIG9uZSBvZiB0aGUgaG9zdHMsIGFuZCB5b3VyIERDIAo+Pj4gc2hvdWxkIGJlY29tZSB1 cC4KPj4+Cj4+PiBEYXZpZCwgU2FoaW5hLCBjYW4geW91IGNvbmZpcm0gdGhhdCB0aGlzIHByb2Nl ZHVyZSBpcyBzYWZlPwo+Pgo+PiBZZXMsIGNvcnJlY3RpbmcgZnJvbSB0aGUgbW91bnQgcG9pbnQg c2hvdWxkIGZpeCBpdCBvbiBhbGwgcmVwbGljYXMKPj4KPj4KPj4+Cj4+PiBOaXIKPj4+Cj4+PiA+ Cj4+PiA+IHJlZ3MuCj4+PiA+IFBhdmVsCj4+PiA+Cj4+PiA+Cj4+PiA+Cj4+PiA+IE9uIDEuMy4y MDE2IDE4OjM4LCBOaXIgU29mZmVyIHdyb3RlOgo+Pj4gPgo+Pj4gPiBPbiBUdWUsIE1hciAxLCAy MDE2IGF0IDU6MDcgUE0sIHBhZjFAZW1haWwuY3ogPHBhZjFAZW1haWwuY3o+IHdyb3RlOgo+Pj4g Pj4KPj4+ID4+IEhlbGxvLCAgY2FuIGFueWJvZHkgIGV4cGxhaW4gdGhpcyBlcnJvciBuby4xMyAo IG9wZW4gZmlsZSApIGluIAo+Pj4gc2FubG9jay5sb2cgLgo+Pj4gPgo+Pj4gPgo+Pj4gPiBUaGlz IGlzIEVBQ0NFUwo+Pj4gPgo+Pj4gPiBDYW4geW91IHNoYXJlIHRoZSBvdXRvcHV0IG9mOgo+Pj4g Pgo+Pj4gPiAgICAgbHMgLWxoIC9yaGV2L2RhdGEtY2VudGVyL21udC88c2VydmVyPjo8X3BhdGg+ LzxzZF91dWlkPi9kb21fbWQKPj4+ID4KPj4+ID4+Cj4+PiA+Pgo+Pj4gPj4gVGhlIHNpemUgb2Yg ICJpZHMiIGZpbGUgaXMgemVybyAoMCkKPj4+ID4KPj4+ID4KPj4+ID4gVGhpcyBpcyBob3cgd2Ug Y3JlYXRlIHRoZSBpZHMgZmlsZSB3aGVuIGluaXRpYWxpemluZyBpdC4KPj4+ID4KPj4+ID4gQnV0 IHRoZW4gd2UgdXNlIHNhbmxvY2sgdG8gaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGUsIGFuZCBpdCBz aG91bGQgCj4+PiBiZSAxTWlCIGFmdGVyIHRoYXQuCj4+PiA+Cj4+PiA+IElzIHRoaXMgaWRzIGZp bGVzIGNyZWF0ZWQgYnkgdmRzbSwgb3Igb25lIHlvdSBjcmVhdGVkIHlvdXJzZWxmPwo+Pj4gPgo+ Pj4gPj4KPj4+ID4+IDIwMTYtMDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBvcGVu IGVycm9yIC0xMyAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0 Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9p ZHMKPj4+ID4+IDIwMTYtMDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBzMTg3OTg1 IG9wZW5fZGlzayAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0 Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9p ZHMgCj4+PiBlcnJvciAtMTMKPj4+ID4+IDIwMTYtMDItMjggMDM6MjU6NTYrMDEwMCAyNjk2MzYg WzExMzA0XTogczE4Nzk5MiBsb2Nrc3BhY2UgCj4+PiA3ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0x MDJkNDQzMjcxMjQ6MTovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpf MUtWTTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRz OjAKPj4+ID4+Cj4+PiA+PiBJZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBz aXplLCBjYW4gSSByZWdlbmVyYXRlIAo+Pj4gIHRoaXMgZmlsZSBvbmxpbmUgc2VjdXJlbHkgLCB3 aXRoIG5vIFZNIGRlcGVuZGVuY2UgID8/Pz8KPj4+ID4KPj4+ID4KPj4+ID4gWWVzLCBJIHRoaW5r IEkgYWxyZWFkeSByZWZlcnJlZCB0byB0aGUgaW5zdHJ1Y3Rpb25zIGhvdyB0byBkbyB0aGF0IAo+ Pj4gaW4gYSBwcmV2aW91cyBtYWlsLgo+Pj4gPgo+Pj4gPj4KPj4+ID4+Cj4+PiA+PiBkaXN0ID0g UkhFTCAtIDcgLSAyLjE1MTEKPj4+ID4+IGtlcm5lbCA9IDMuMTAuMCAtIDMyNy4xMC4xLmVsNy54 ODZfNjQKPj4+ID4+IEtWTSA9IDIuMy4wIC0gMjkuMS5lbDcKPj4+ID4+IGxpYnZpcnQgPSBsaWJ2 aXJ0LTEuMi4xNy0xMy5lbDdfMi4zCj4+PiA+PiB2ZHNtID0gdmRzbS00LjE2LjMwLTAuZWw3Cj4+ PiA+PiBHbHVzdGVyRlMgPSBnbHVzdGVyZnMtMy43LjgtMS5lbDcKPj4+ID4+Cj4+PiA+Pgo+Pj4g Pj4gcmVncy4KPj4+ID4+IFBhdmVsCj4+PiA+Pgo+Pj4gPj4gX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX18KPj4+ID4+IFVzZXJzIG1haWxpbmcgbGlzdAo+Pj4g Pj4gVXNlcnNAb3ZpcnQub3JnIDxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPgo+Pj4gPj4gaHR0cDov L2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzCj4+PiA+Pgo+Pj4gPgo+Pj4g Pgo+Pgo+Pgo+Pgo+PiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwo+PiBHbHVzdGVyLXVzZXJzIG1haWxpbmcgbGlzdAo+PiBHbHVzdGVyLXVzZXJzQGdsdXN0 ZXIub3JnCj4+IGh0dHA6Ly93d3cuZ2x1c3Rlci5vcmcvbWFpbG1hbi9saXN0aW5mby9nbHVzdGVy LXVzZXJzCj4KPgoKCi0tLS0tLS0tLS0tLS0tMDIwNjAzMDYwMjA0MDMwNjA0MDkwMjA2CkNvbnRl bnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04CkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rp bmc6IDhiaXQKCjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEgY29udGVudD0idGV4dC9odG1sOyBj aGFyc2V0PXV0Zi04IiBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogIDwvaGVhZD4KICA8Ym9k eSB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIEhpIGd1eXMsIDxicj4KICAg IHRoeCBhIGxvdCBmb3IgeW91ciBzdXBwb3J0IC4uLi4uLi5hdCBmaXJzdC48YnI+CiAgICA8YnI+ CiAgICBCZWNhdXNlIHdlIGhhZCBiZWVuIHVuZGVyIGh1Z2UgdGltZSBwcmVzc3VyZSwgd2UgZm91 bmQgImdvb2dsZQogICAgd29ya2Fyb3VuZCLCoCB3aGljaCBkZWxldGUgYm90aCBmaWxlcyAuIEl0 IGhlbHBlZCwgcHJvYmFiYmx5IGF0IGZpcnN0CiAgICBzdGVwcyBvZiByZWNvdmVyIC48YnI+CiAg ICBlZzogIiAjwqAgZmluZCAvU1RPUkFHRVMvZzFyNXA1L0dGUy8gLXNhbWVmaWxlCiAgICAvU1RP UkFHRVMvZzFyNXA1L0dGUy8zZGE0NmUwNy1kMWVhLTRmMTAtOTI1MC02Y2JiYjdiOTRkODAvZG9t X21kL2lkcwogICAgLXByaW50IC1kZWxldGUgIjxicj4KICAgIDxicj4KICAgIC0tLS0tLS0tLS0t LS0tLS0tLS0tLS0mZ3Q7PGJyPgogICAgV2VsbCBhdCBmaXJzdCBJJ2xswqAgZml4IHBlcm1pdHRp b25zIGZyb20gbW91bnQgcG9pbnRzwqAgdG8gNjYwIC48YnI+CiAgICBJZiAiaWRzIsKgIGZpbGUg d2lsbCBiZSB3cml0ZWFibGUgLCBjYW4ndMKgIGJlY2FtZSBnbHVzdGVyIGNvbGFwcyA/Pzxicj4K ICAgIDxicj4KICAgIHJlZ3MuUGF2ZWw8YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8ZGl2IGNs YXNzPSJtb3otY2l0ZS1wcmVmaXgiPk9uIDIuMy4yMDE2IDA4OjE2LCBSYXZpc2hhbmthciBOIHdy b3RlOjxicj4KICAgIDwvZGl2PgogICAgPGJsb2NrcXVvdGUgY2l0ZT0ibWlkOjU2RDY5MzY1LjQw OTAzMDNAcmVkaGF0LmNvbSIgdHlwZT0iY2l0ZSI+CiAgICAgIDxtZXRhIGNvbnRlbnQ9InRleHQv aHRtbDsgY2hhcnNldD11dGYtOCIgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICAgICAgPGRp diBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5PbiAwMy8wMi8yMDE2IDEyOjAyIFBNLCBTYWhpbmEg Qm9zZQogICAgICAgIHdyb3RlOjxicj4KICAgICAgPC9kaXY+CiAgICAgIDxibG9ja3F1b3RlIGNp dGU9Im1pZDo1NkQ2ODkxMC44MDQwNjAyQHJlZGhhdC5jb20iIHR5cGU9ImNpdGUiPgogICAgICAg IDxtZXRhIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCIKICAgICAgICAgIGh0dHAt ZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAgICAgICAgPGJyPgogICAgICAgIDxicj4KICAgICAgICA8 ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPk9uIDAzLzAyLzIwMTYgMDM6NDUgQU0sIE5pciBT b2ZmZXIKICAgICAgICAgIHdyb3RlOjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8YmxvY2tx dW90ZQpjaXRlPSJtaWQ6Q0FNUmJ5eXU5Z3dQZlZwUHhwRGE0X2dLV3lYcTFQYXZUbTJWMnJHMmNV MEF2RT1KSlBBQG1haWwuZ21haWwuY29tIgogICAgICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgICAg ICA8ZGl2IGRpcj0ibHRyIj5PbiBUdWUsIE1hciAxLCAyMDE2IGF0IDEwOjUxIFBNLCA8YQogICAg ICAgICAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgY2xhc3M9Im1vei10eHQtbGluay1hYmJy ZXZpYXRlZCIKICAgICAgICAgICAgICBocmVmPSJtYWlsdG86cGFmMUBlbWFpbC5jeiI+cGFmMUBl bWFpbC5jejwvYT4gJmx0OzxhCiAgICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIiBj bGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIgogICAgICAgICAgICAgIGhyZWY9Im1haWx0 bzpwYWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiZndDsgd3JvdGU6PGJyPgogICAgICAg ICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7IEhJLDxicj4KICAgICAgICAgICAgJmd0OyBy ZXF1ZXN0ZWQgb3V0cHV0Ojxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0 OyAjIGxzIC1saAogICAgICAgICAgICAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xv Y2FsaG9zdDoqLyovZG9tX21kPGJyPgogICAgICAgICAgICAmZ3Q7IMKgPGJyPgogICAgICAgICAg ICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTIt QkNLLzBmY2FkODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21fbWQ6PGJyPgogICAg ICAgICAgICAmZ3Q7IHRvdGFsIDIsMU08YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAx IHZkc20ga3ZtIDEsME0gwqAxLiBixZllIDIxLjI4IGlkcyDCoCDCoCDCoAogICAgICAgICAgICDC oCZsdDstLSBnb29kPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDC oDE2TSDCoDcuIGxpcyAyMi4xNiBpbmJveDxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0t IDEgdmRzbSBrdm0gMiwwTSDCoDcuIGxpcyAyMi4xNyBsZWFzZXM8YnI+CiAgICAgICAgICAgICZn dDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgMzM1IMKgNy4gbGlzIDIyLjE3IG1ldGFkYXRhPGJy PgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAy Mi4xNiBvdXRib3g8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsKL3Jo ZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QMS81NTNkOWI5 Mi1lNGEwLTQwNDItYTU3OS00Y2FiZWI1NWRlZDQvZG9tX21kOjxicj4KICAgICAgICAgICAgJmd0 OyB0b3RhbCAxLDFNPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDC oCDCoDAgMjQuIMO6bm8gMDcuNDEgaWRzIMKgIMKgIMKgCiAgICAgICAgICAgIMKgJmx0Oy0tIGJh ZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAg Jmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTQgaW5ib3g8YnI+ CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMDMu NTYgbGVhc2VzPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMz MyDCoDcuIGxpcyAwMy41NiBtZXRhZGF0YTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0t IDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTQgb3V0Ym94PGJyPgogICAgICAgICAgICAm Z3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0Qv bG9jYWxob3N0Ol8xS1ZNMTItUDIvODhhZGJkNDktNjJkNi00NWIxLTk5OTItYjA0NDY0YTA0MTEy L2RvbV9tZDo8YnI+CiAgICAgICAgICAgICZndDsgdG90YWwgMSwxTTxicj4KICAgICAgICAgICAg Jmd0OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjQzIGlkcyDCoCDC oCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhl ciBjYW4gcmVhZCk8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKg MTZNIMKgNy4gbGlzIDAwLjE1IGluYm94PGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0g MSB2ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDIyLjE0IGxlYXNlczxicj4KICAgICAgICAgICAgJmd0 OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqAzMzMgwqA3LiBsaXMgMjIuMTQgbWV0YWRhdGE8YnI+ CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAw LjE1IG91dGJveDxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0Owovcmhl di9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVAzLzNjMzRhZDYz LTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQ6PGJyPgogICAgICAgICAgICAmZ3Q7 IHRvdGFsIDEsMU08YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKg IMKgMCAyNC4gw7pubyAwNy40MyBpZHMgwqAgwqAgwqAKICAgICAgICAgICAgwqAmbHQ7LS0gYmFk IChzYW5sb2NrIGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAm Z3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAyMy4gw7pubyAyMi41MSBpbmJveDxicj4K ICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4gw7pubyAyMy4x MiBsZWFzZXM8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk4 IDI1LiDDum5vIDAwLjM1IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0g MSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAwMC4xNiBvdXRib3g8YnI+CiAgICAgICAgICAgICZn dDs8YnI+CiAgICAgICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9s b2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQv ZG9tX21kOjxicj4KICAgICAgICAgICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAgICAgICAm Z3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuNDQgaWRzIMKgIMKg IMKgCiAgICAgICAgICAgIMKgJmx0Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVy IGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAx Nk0gwqA3LiBsaXMgMDAuMTcgaW5ib3g8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAx IHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMDAuMTggbGVhc2VzPGJyPgogICAgICAgICAgICAmZ3Q7 IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAwMC4xOCBtZXRhZGF0YTxicj4K ICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAu MTcgb3V0Ym94PGJyPgogICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2 L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDEvNDJkNzEwYTkt Yjg0NC00M2RjLWJlNDEtNzcwMDJkMWNkNTUzL2RvbV9tZDo8YnI+CiAgICAgICAgICAgICZndDsg dG90YWwgMSwxTTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAg wqAwIDI0LiDDum5vIDA3LjMyIGlkcyDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQg KG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBr dm0gwqAxNk0gwqA3LiBsaXMgMjIuMTggaW5ib3g8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3 LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMjIuMTggbGVhc2VzPGJyPgogICAgICAgICAg ICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAyMi4xOCBtZXRhZGF0 YTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBs aXMgMjIuMTggb3V0Ym94PGJyPgogICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7 Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDIvZmY3 MWI0N2ItMGY3Mi00NTI4LTliZmUtYzNkYTg4OGU0N2YwL2RvbV9tZDo8YnI+CiAgICAgICAgICAg ICZndDsgdG90YWwgMywwTTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBr dm0gMSwwTSDCoDEuIGLFmWUgMjEuMjggaWRzwqDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDst LSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEg dmRzbSBrdm0gwqAxNk0gMjUuIMO6bm8gMDAuNDIgaW5ib3jCoDxicj4KICAgICAgICAgICAgJmd0 OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyNS4gw7pubyAwMC40NCBsZWFzZXM8YnI+CiAg ICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk3IDI0LiDDum5vIDAyLjQ2 IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2 TSAyNS4gw7pubyAwMC40NCBvdXRib3g8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAg ICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0x Mi1QMy9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRlY2I0YjQvZG9tX21kOjxicj4KICAg ICAgICAgICAgJmd0OyB0b3RhbCAyLDFNPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0g MSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzQgaWRzwqDCoCDCoCDCoAogICAgICAgICAg ICDCoCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+ CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIDIzLiDDum5vIDIy LjM1IGluYm94PGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBN IDIzLiDDum5vIDIyLjM4IGxlYXNlczxicj4KICAgICAgICAgICAgJmd0OyAtcnctci0tci0tIDEg dmRzbSBrdm0gMSwxSyAyNC4gw7pubyAxOS4wNyBtZXRhZGF0YTxicj4KICAgICAgICAgICAgJmd0 OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gMjMuIMO6bm8gMjIuMjcgb3V0Ym94PGJyPgog ICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21u dC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTJfX1A0LzMwMGU5YWM4LTNjMmYtNDcwMy05YmIx LTFkZjIxMzBjN2M5Ny9kb21fbWQ6PGJyPgogICAgICAgICAgICAmZ3Q7IHRvdGFsIDMsME08YnI+ CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDEsME0gwqAxLiBixZllIDIx LjI4IGlkc8KgwqAgwqAgwqAKICAgICAgICAgICAgwqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVh ZCk8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIMKgMTZNIMKgNi4g bGlzIDIzLjUwIGluYm94wqDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQgKG90aGVy IGNhbiByZWFkKQogICAgICAgICAgICA8ZGl2PiZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIs ME0gwqA2LiBsaXMgMjMuNTEgbGVhc2VzwqDCoAogICAgICAgICAgICAgIMKgIMKgIMKgJmx0Oy0t IGJhZCAob3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAx IHZkc20ga3ZtIMKgNzM0IMKgNy4gbGlzIDAyLjEzIG1ldGFkYXRhwqDCoCDCoAogICAgICAgICAg ICAgIMKgIMKgJmx0Oy0tIGJhZCAoZ3JvdXAgY2FuIHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+ CiAgICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAxNk0gwqA2LiBsaXMg MTYuNTUgb3V0Ym94wqDCoCDCoCDCoAogICAgICAgICAgICAgIMKgJmx0Oy0tIGJhZCAob3RoZXIg Y2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0Owov cmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyLVA1LzFjYTU2 YjQ1LTcwMWUtNGMyMi05ZjU5LTNhZWJlYTRkODQ3Ny9kb21fbWQ6PGJyPgogICAgICAgICAgICAg ICZndDsgdG90YWwgMSwxTTxicj4KICAgICAgICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNt IGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzUgaWRzwqDCoCDCoCDCoAogICAgICAgICAgICAgIMKg Jmx0Oy0tIGJhZCAob3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3 LXItLSAxIHZkc20ga3ZtIMKgMTZNIDI0LiDDum5vIDAxLjA2IGluYm94PGJyPgogICAgICAgICAg ICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gMjQuIMO6bm8gMDIuNDQgbGVhc2Vz PGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk4IDI0LiDD um5vIDE5LjA3IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZk c20ga3ZtIMKgMTZNIMKgNy4gbGlzIDIyLjIwIG91dGJveDxicj4KICAgICAgICAgICAgICA8YnI+ CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIEl0IHNob3VsZCBsb29rIGxpa2UgdGhp czo8YnI+CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIC1ydy1ydy0tLS0uIDEgdmRz bSBrdm0gMS4wTSBNYXIgwqAxIDIzOjM2IGlkczxicj4KICAgICAgICAgICAgICAtcnctcnctLS0t LiAxIHZkc20ga3ZtIDIuME0gTWFyIMKgMSAyMzozNSBsZWFzZXM8YnI+CiAgICAgICAgICAgICAg LXJ3LXItLXItLS4gMSB2ZHNtIGt2bSDCoDM1MyBNYXIgwqAxIDIzOjM1IG1ldGFkYXRhPGJyPgog ICAgICAgICAgICAgIC1ydy1ydy0tLS0uIDEgdmRzbSBrdm0gwqAxNk0gTWFyIMKgMSAyMzozNCBv dXRib3g8YnI+CiAgICAgICAgICAgICAgLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSDCoDE2TSBNYXIg wqAxIDIzOjM0IGluYm94PGJyPgogICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICBUaGlz IGV4cGxhaW5zIHRoZSBFQUNDRVMgZXJyb3IuPGJyPgogICAgICAgICAgICAgIDxicj4KICAgICAg ICAgICAgICBZb3UgY2FuIHN0YXJ0IGJ5IGZpeGluZyB0aGUgcGVybWlzc2lvbnMgbWFudWFsbHks IHlvdSBjYW4KICAgICAgICAgICAgICBkbyB0aGlzIG9ubGluZS48YnI+CiAgICAgICAgICAgICAg wqA8YnI+CiAgICAgICAgICAgICAgJmd0OyDCoFRoZSBpZHMgZmlsZXMgd2FzIGdlbmVyYXRlZCBi eSAidG91Y2giIGNvbW1hbmQgYWZ0ZXIKICAgICAgICAgICAgICBkZWxldGluZyB0aGVtIGR1ZSAi c2FubG9jayBsb2NraW5nIGhhbmciIMKgZ2x1c3RlciBjcmFzaAogICAgICAgICAgICAgICZhbXA7 IHJlYm9vdDxicj4KICAgICAgICAgICAgICAmZ3Q7IEkgZXhwZWN0ZWQgdGhhdCB0aGV5IHdpbGwg YmUgZmlsbGVkIGF1dG9tYXRpY2FseQogICAgICAgICAgICAgIGFmdGVyIGdsdXN0ZXIgcmVib290 ICggdGhlIMKgc2hhZG93IGNvcHkgZnJvbSDCoCAiLmdsdXN0ZXIgIgogICAgICAgICAgICAgIMKg IGRpcmVjdG9yeSDCoHdhcyBkZWxldGVkICZhbXA7IGNyZWF0ZWQgZW1wdHkgwqB0b28gKTxicj4K ICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgSSBkb24ndCBrbm93IGFib3V0IGdsdXN0 ZXIgc2hhZG93IGNvcHksIEkgd291bGQgbm90IHBsYXkKICAgICAgICAgICAgICB3aXRoIGdsdXN0 ZXIgaW50ZXJuYWxzLjwvZGl2PgogICAgICAgICAgICA8ZGl2PkFkZGluZyBTYWhpbmEgZm9yIGFk dmljZS48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAgICAgPC9i bG9ja3F1b3RlPgogICAgICAgIDxicj4KICAgICAgICBEaWQgeW91IGdlbmVyYXRlIHRoZSBpZHMg ZmlsZSBvbiB0aGUgbW91bnQgcG9pbnQuPGJyPgogICAgICAgIDxicj4KICAgICAgICBSYXZpLCBj YW4geW91IGhlbHAgaGVyZT88YnI+CiAgICAgICAgPGJyPgogICAgICA8L2Jsb2NrcXVvdGU+CiAg ICAgIDxicj4KICAgICAgT2theSwgc28gd2hhdCBJIHVuZGVyc3RhbmQgZnJvbSB0aGUgb3V0cHV0 IGFib3ZlIGlzIHlvdSBoYXZlCiAgICAgIGRpZmZlcmVudCBnbHVzdGVyIHZvbHVtZXMgbW91bnRl ZCBhbmQgc29tZSBvZiB0aGVtIGhhdmUgaW5jb3JyZWN0CiAgICAgIHBlcm1pc3Npb25zIGZvciB0 aGUgJ2lkcycgZmlsZS4gVGhlIHdheSB0byBmaXggaXQgaXMgdG8gZG8gaXQgZnJvbQogICAgICB0 aGUgbW91bnQgbGlrZSBOaXIgc2FpZC48YnI+CiAgICAgIFdoeSBkaWQgeW91IGRlbGV0ZSB0aGUg ZmlsZSBmcm9tIHRoZSAuZ2x1c3RlcmZzIGluIHRoZSBicmljayhzKT/CoAogICAgICBXYXMgdGhl cmUgYSBnZmlkIHNwbGl0IGJyYWluPyA8YnI+CiAgICAgIDxicj4KICAgICAgLVJhdmk8YnI+CiAg ICAgIDxicj4KICAgICAgPGJsb2NrcXVvdGUgY2l0ZT0ibWlkOjU2RDY4OTEwLjgwNDA2MDJAcmVk aGF0LmNvbSIgdHlwZT0iY2l0ZSI+CiAgICAgICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVJi eXl1OWd3UGZWcFB4cERhNF9nS1d5WHExUGF2VG0yVjJyRzJjVTBBdkU9SkpQQUBtYWlsLmdtYWls LmNvbSIKICAgICAgICAgIHR5cGU9ImNpdGUiPgogICAgICAgICAgPGRpdiBkaXI9Imx0ciI+CiAg ICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICAgICZndDsgT0ssIGl0IGxvb2tzIHRoYXQg c2FubG9jayDCoGNhbid0IHdvcmsgd2l0aCBlbXB0eSBmaWxlCiAgICAgICAgICAgICAgb3IgcmV3 cml0ZSB0aGVtIC48YnI+CiAgICAgICAgICAgICAgJmd0OyBBbSBJIHJpZ2h0ID8/PGJyPgogICAg ICAgICAgICAgIDxicj4KICAgICAgICAgICAgICBZZXMsIHRoZSBmaWxlcyBtdXN0IGJlIGluaXRp YWxpemVkIGJlZm9yZSBzYW5sb2NrIGNhbiB1c2UKICAgICAgICAgICAgICB0aGVtLjxicj4KICAg ICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgWW91IGNhbiBpbml0aWFsaXplIHRoZSBmaWxl IGxpa2UgdGhpczo8YnI+CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIHNhbmxvY2sg ZGlyZWN0IGluaXQgLXMKICAgICAgICAgICAgICAmbHQ7c2RfdXVpZCZndDs6MDpyZXBhaXIvJmx0 O3NkX3V1aWQmZ3Q7L2RvbV9tZC9pZHM6MDxicj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAg ICAgICAgVGFrZW4gZnJvbSA8YSBtb3otZG8tbm90LXNlbmQ9InRydWUiCiAgICAgICAgICAgICAg ICBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL3BpcGVybWFpbC91c2Vycy8yMDE2LUZlYnJ1 YXJ5LzAzODA0Ni5odG1sIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL3BpcGVybWFpbC91c2Vycy8y MDE2LUZlYnJ1YXJ5LzAzODA0Ni5odG1sPC9hPjxicj4KICAgICAgICAgICAgICA8YnI+CiAgICAg ICAgICAgICAgJmd0OyBUaGUgbGFzdCBwb2ludCAtIGFib3V0ICJpZHMiIHdvcmthcm91bmQgLSB0 aGlzIGlzCiAgICAgICAgICAgICAgb2ZmbGluZSB2ZXJzaW9uID0gVk1zIGhhdmUgdG8gYmUgbW92 ZWQgb3V0IGZyb20gZm9yCiAgICAgICAgICAgICAgY29udGludWFsIHJ1bm5pbmcgd2l0aCBtYWlu dGVuYW5jZSB2b2x1bWUgbW9kZTxicj4KICAgICAgICAgICAgICAmZ3Q7IEJ1dCB0aGlzIGlzIG5v dCBhY2NlcHRhYmxlIGluIGN1cnJlbnQgc2l0dWF0aW9uLCBzbwogICAgICAgICAgICAgIHRoZSBx dWVzdGlvbiBhZ2FpbiwgwqBpcyBpdCBzYWZlIHRvIGRvIGl0IG9ubGluZSA/PyDCoCggWUVTCiAg ICAgICAgICAgICAgLyBOTyApPC9kaXY+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAg ICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj5UaGUgaWRzIGZpbGUgaXMgYWNjZXNzZWQgb25seSBi eSBzYW5sb2NrLiBJIGd1ZXNzIHRoYXQKICAgICAgICAgICAgICB5b3UgZG9uJ3QgaGF2ZSBhIHJ1 bm5pbmc8L2Rpdj4KICAgICAgICAgICAgPGRpdj5TUE0gb24gdGhpcyBEQywgc2luY2Ugc2FubG9j ayBmYWlscyB0byBhY3F1aXJlIGEgaG9zdAogICAgICAgICAgICAgIGlkLCBzbyB5b3UgYXJlIHBy ZXR0eSBzYWZlPC9kaXY+CiAgICAgICAgICAgIDxkaXY+dG8gZml4IHRoZSBwZXJtaXNzaW9ucyBh bmQgaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGVzLjwvZGl2PgogICAgICAgICAgICA8ZGl2Pjxicj4K ICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+SSB3b3VsZCBkbyB0aGlzOjwvZGl2 PgogICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxk aXY+MS4gU3RvcCBlbmdpbmUsIMKgc28gaXQgd2lsbCBub3QgdHJ5IHRvIHN0YXJ0IHZkc208L2Rp dj4KICAgICAgICAgICAgPGRpdj4yLiBTdG9wIHZkc20gb24gYWxsIGhvc3RzLCBzbyB0aGV5IGRv IG5vdCB0cnkgdG8KICAgICAgICAgICAgICBhY3F1aXJlIGEgaG9zdCBpZCB3aXRoIHNhbmxvY2s8 L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoCDCoCBUaGlzIGRvZXMgbm90IGFmZmVjdCBydW5uaW5n IHZtczwvZGl2PgogICAgICAgICAgICA8ZGl2PjMuIEZpeCB0aGUgcGVybWlzc2lvbnMgb24gdGhl IGlkcyBmaWxlLCB2aWEgZ2x1c3RlcmZzCiAgICAgICAgICAgICAgbW91bnQ8L2Rpdj4KICAgICAg ICAgICAgPGRpdj40LiBJbml0aWFsaXplIHRoZSBpZHMgZmlsZXMgZnJvbSBvbmUgb2YgdGhlIGhv c3RzLCB2aWEKICAgICAgICAgICAgICB0aGUgZ2x1c3RlcmZzIG1vdW50PC9kaXY+CiAgICAgICAg ICAgIDxkaXY+wqAgwqAgVGhpcyBzaG91bGQgZml4IHRoZSBpZHMgZmlsZXMgb24gYWxsIHJlcGxp Y2FzPC9kaXY+CiAgICAgICAgICAgIDxkaXY+NS4gU3RhcnQgdmRzbSBvbiBhbGwgaG9zdHM8L2Rp dj4KICAgICAgICAgICAgPGRpdj42LiBTdGFydCBlbmdpbmU8L2Rpdj4KICAgICAgICAgICAgPGRp dj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2PkVuZ2luZSB3aWxsIGNv bm5lY3QgdG8gYWxsIGhvc3RzLCBob3N0cyB3aWxsIGNvbm5lY3QgdG8KICAgICAgICAgICAgICBz dG9yYWdlIGFuZCB0cnkgdG8gYWNxdWlyZSBhIGhvc3QgaWQuPC9kaXY+CiAgICAgICAgICAgIDxk aXY+VGhlbiBFbmdpbmUgd2lsbCBzdGFydCB0aGUgU1BNIG9uIG9uZSBvZiB0aGUgaG9zdHMsIGFu ZAogICAgICAgICAgICAgIHlvdXIgREMgc2hvdWxkIGJlY29tZSB1cC48L2Rpdj4KICAgICAgICAg ICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2PkRhdmlkLCBT YWhpbmEsIGNhbiB5b3UgY29uZmlybSB0aGF0IHRoaXMgcHJvY2VkdXJlIGlzCiAgICAgICAgICAg ICAgc2FmZT88L2Rpdj4KICAgICAgICAgIDwvZGl2PgogICAgICAgIDwvYmxvY2txdW90ZT4KICAg ICAgICA8YnI+CiAgICAgICAgWWVzLCBjb3JyZWN0aW5nIGZyb20gdGhlIG1vdW50IHBvaW50IHNo b3VsZCBmaXggaXQgb24gYWxsCiAgICAgICAgcmVwbGljYXM8YnI+CiAgICAgICAgPGJyPgogICAg ICAgIDxicj4KICAgICAgICA8YmxvY2txdW90ZQpjaXRlPSJtaWQ6Q0FNUmJ5eXU5Z3dQZlZwUHhw RGE0X2dLV3lYcTFQYXZUbTJWMnJHMmNVMEF2RT1KSlBBQG1haWwuZ21haWwuY29tIgogICAgICAg ICAgdHlwZT0iY2l0ZSI+CiAgICAgICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICAgICAgPGRp dj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2Pk5pcjwvZGl2PgogICAg ICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+Jmd0 Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IHJlZ3MuPGJyPgogICAgICAgICAgICAgICZndDsgUGF2 ZWw8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAg ICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0OyBPbiAxLjMuMjAxNiAxODozOCwg TmlyIFNvZmZlciB3cm90ZTo8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAg ICAmZ3Q7IE9uIFR1ZSwgTWFyIDEsIDIwMTYgYXQgNTowNyBQTSwgPGEKICAgICAgICAgICAgICAg IG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIK ICAgICAgICAgICAgICAgIGhyZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6 PC9hPiAmbHQ7PGEKICAgICAgICAgICAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgY2xhc3M9 Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIKICAgICAgICAgICAgICAgIGhyZWY9Im1haWx0bzpw YWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiZndDsgd3JvdGU6PGJyPgogICAgICAgICAg ICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IEhlbGxvLCDCoGNhbiBhbnli b2R5IMKgZXhwbGFpbiB0aGlzIGVycm9yIG5vLjEzICgKICAgICAgICAgICAgICBvcGVuIGZpbGUg KSBpbiBzYW5sb2NrLmxvZyAuPGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAg ICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IFRoaXMgaXMgRUFDQ0VTPGJyPgogICAgICAg ICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0OyBDYW4geW91IHNoYXJlIHRoZSBvdXRv cHV0IG9mOjxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsgwqAg wqAgbHMgLWxoCi9yaGV2L2RhdGEtY2VudGVyL21udC8mbHQ7c2VydmVyJmd0OzombHQ7X3BhdGgm Z3Q7LyZsdDtzZF91dWlkJmd0Oy9kb21fbWQ8YnI+CiAgICAgICAgICAgICAgJmd0OyDCoDxicj4K ICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAg ICAgICAgICAgICAmZ3Q7Jmd0OyBUaGUgc2l6ZSBvZiDCoCJpZHMiIGZpbGUgaXMgemVybyAoMCk8 YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAg ICAgICAgICZndDsgVGhpcyBpcyBob3cgd2UgY3JlYXRlIHRoZSBpZHMgZmlsZSB3aGVuIGluaXRp YWxpemluZwogICAgICAgICAgICAgIGl0Ljxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAg ICAgICAgICAgICZndDsgQnV0IHRoZW4gd2UgdXNlIHNhbmxvY2sgdG8gaW5pdGlhbGl6ZSB0aGUg aWRzIGZpbGUsCiAgICAgICAgICAgICAgYW5kIGl0IHNob3VsZCBiZSAxTWlCIGFmdGVyIHRoYXQu PGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0OyBJcyB0aGlzIGlk cyBmaWxlcyBjcmVhdGVkIGJ5IHZkc20sIG9yIG9uZSB5b3UgY3JlYXRlZAogICAgICAgICAgICAg IHlvdXJzZWxmPzxicj4KICAgICAgICAgICAgICAmZ3Q7IMKgPGJyPgogICAgICAgICAgICAgICZn dDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IDIwMTYtMDItMjggMDM6MjU6NDYrMDEw MCAyNjk2MjYgWzE5NTFdOiBvcGVuCiAgICAgICAgICAgICAgZXJyb3IgLTEzCi9yaGV2L2RhdGEt Y2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00 ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM8YnI+CiAgICAgICAgICAgICAgJmd0OyZn dDsgMjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IHMxODc5ODUKICAgICAg ICAgICAgICBvcGVuX2Rpc2sKICAgICAgICAgICAgICAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1 c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0 NDMyNzEyNC9kb21fbWQvaWRzCgoKICAgICAgICAgICAgICBlcnJvciAtMTM8YnI+CiAgICAgICAg ICAgICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBz MTg3OTkyCiAgICAgICAgICAgICAgbG9ja3NwYWNlCjdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEw MmQ0NDMyNzEyNDoxOi9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8x S1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM6 MDxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBJ ZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXplLCBjYW4KICAgICAgICAg ICAgICBJIHJlZ2VuZXJhdGUgwqB0aGlzIGZpbGUgb25saW5lIHNlY3VyZWx5ICwgd2l0aCBubyBW TQogICAgICAgICAgICAgIGRlcGVuZGVuY2UgwqA/Pz8/PGJyPgogICAgICAgICAgICAgICZndDs8 YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IFllcywgSSB0aGlu ayBJIGFscmVhZHkgcmVmZXJyZWQgdG8gdGhlIGluc3RydWN0aW9ucwogICAgICAgICAgICAgIGhv dyB0byBkbyB0aGF0IGluIGEgcHJldmlvdXMgbWFpbC48YnI+CiAgICAgICAgICAgICAgJmd0Ozxi cj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4K ICAgICAgICAgICAgICAmZ3Q7Jmd0OyBkaXN0ID0gUkhFTCAtIDcgLSAyLjE1MTE8YnI+CiAgICAg ICAgICAgICAgJmd0OyZndDsga2VybmVsID0gMy4xMC4wIC0gMzI3LjEwLjEuZWw3Lng4Nl82NDxi cj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBLVk0gPSAyLjMuMCAtIDI5LjEuZWw3PGJyPgogICAg ICAgICAgICAgICZndDsmZ3Q7IGxpYnZpcnQgPSBsaWJ2aXJ0LTEuMi4xNy0xMy5lbDdfMi4zPGJy PgogICAgICAgICAgICAgICZndDsmZ3Q7IHZkc20gPSB2ZHNtLTQuMTYuMzAtMC5lbDc8YnI+CiAg ICAgICAgICAgICAgJmd0OyZndDsgR2x1c3RlckZTID0gZ2x1c3RlcmZzLTMuNy44LTEuZWw3PGJy PgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgog ICAgICAgICAgICAgICZndDsmZ3Q7IHJlZ3MuPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IFBh dmVsPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7 IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPGJyPgogICAg ICAgICAgICAgICZndDsmZ3Q7IFVzZXJzIG1haWxpbmcgbGlzdDxicj4KICAgICAgICAgICAgICAm Z3Q7Jmd0OyA8YSBtb3otZG8tbm90LXNlbmQ9InRydWUiCiAgICAgICAgICAgICAgICBocmVmPSJt YWlsdG86VXNlcnNAb3ZpcnQub3JnIj5Vc2Vyc0BvdmlydC5vcmc8L2E+PGJyPgogICAgICAgICAg ICAgICZndDsmZ3Q7IDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAgICAgICAgICAgIGhy ZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyI+aHR0cDov L2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzPC9hPjxicj4KICAgICAgICAg ICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZn dDs8YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAgICAgPC9ibG9j a3F1b3RlPgogICAgICAgIDxicj4KICAgICAgICA8YnI+CiAgICAgICAgPGZpZWxkc2V0IGNsYXNz PSJtaW1lQXR0YWNobWVudEhlYWRlciI+PC9maWVsZHNldD4KICAgICAgICA8YnI+CiAgICAgICAg PHByZSB3cmFwPSIiPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fCkdsdXN0ZXItdXNlcnMgbWFpbGluZyBsaXN0CjxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIg Y2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIgaHJlZj0ibWFpbHRvOkdsdXN0ZXItdXNl cnNAZ2x1c3Rlci5vcmciPkdsdXN0ZXItdXNlcnNAZ2x1c3Rlci5vcmc8L2E+CjxhIG1vei1kby1u b3Qtc2VuZD0idHJ1ZSIgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIgaHJlZj0iaHR0cDov L3d3dy5nbHVzdGVyLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2dsdXN0ZXItdXNlcnMiPmh0dHA6Ly93 d3cuZ2x1c3Rlci5vcmcvbWFpbG1hbi9saXN0aW5mby9nbHVzdGVyLXVzZXJzPC9hPjwvcHJlPgog ICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgIDxicj4KICAgICAgPGJyPgogICAgPC9ibG9ja3F1b3Rl PgogICAgPGJyPgogIDwvYm9keT4KPC9odG1sPgoKLS0tLS0tLS0tLS0tLS0wMjA2MDMwNjAyMDQw MzA2MDQwOTAyMDYtLQo= --===============2054102223816475046==-- From paf1 at email.cz Wed Mar 2 05:43:40 2016 Content-Type: multipart/mixed; boundary="===============0575629602994281979==" MIME-Version: 1.0 From: paf1 at email.cz To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Wed, 02 Mar 2016 11:43:35 +0100 Message-ID: <56D6C3D7.9090608@email.cz> In-Reply-To: 56D69365.4090303@redhat.com --===============0575629602994281979== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------080701090204000803060504 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 7bit Yes we have had "ids" split brains + some other VM's files Split brains was fixed by healing with preffered ( source ) brick. eg: " # gluster volume heal 1KVM12-P1 split-brain source-brick = 16.0.0.161:/STORAGES/g1r5p1/GFS " Pavel > Okay, so what I understand from the output above is you have different = > gluster volumes mounted and some of them have incorrect permissions = > for the 'ids' file. The way to fix it is to do it from the mount like = > Nir said. > Why did you delete the file from the .glusterfs in the brick(s)? Was = > there a gfid split brain? > > -Ravi > --------------080701090204000803060504 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit Yes we have had "ids" split brains + some other VM's files
Split brains was fixed by healing with preffered ( source ) brick.

eg: " # gluster volume heal 1KVM12-P1 split-brain source-brick 16.0.0.161:/STORAGES/g1r5p1/GFS "

Pavel

Okay, so what I understand from the output above is you have different gluster volumes mounted and some of them have incorrect permissions for the 'ids' file. The way to fix it is to do it from the mount like Nir said.
Why did you delete the file from the .glusterfs in the brick(s)?=C2= =A0 Was there a gfid split brain?

-Ravi

--------------080701090204000803060504-- --===============0575629602994281979== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wODA3MDEwOTAyMDQwMDA4MDMwNjA1MDQKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQK ClllcyB3ZSBoYXZlIGhhZCAiaWRzIiBzcGxpdCBicmFpbnMgKyBzb21lIG90aGVyIFZNJ3MgZmls ZXMKU3BsaXQgYnJhaW5zIHdhcyBmaXhlZCBieSBoZWFsaW5nIHdpdGggcHJlZmZlcmVkICggc291 cmNlICkgYnJpY2suCgplZzogIiAjIGdsdXN0ZXIgdm9sdW1lIGhlYWwgMUtWTTEyLVAxIHNwbGl0 LWJyYWluIHNvdXJjZS1icmljayAKMTYuMC4wLjE2MTovU1RPUkFHRVMvZzFyNXAxL0dGUyAiCgpQ YXZlbAoKCj4gT2theSwgc28gd2hhdCBJIHVuZGVyc3RhbmQgZnJvbSB0aGUgb3V0cHV0IGFib3Zl IGlzIHlvdSBoYXZlIGRpZmZlcmVudCAKPiBnbHVzdGVyIHZvbHVtZXMgbW91bnRlZCBhbmQgc29t ZSBvZiB0aGVtIGhhdmUgaW5jb3JyZWN0IHBlcm1pc3Npb25zIAo+IGZvciB0aGUgJ2lkcycgZmls ZS4gVGhlIHdheSB0byBmaXggaXQgaXMgdG8gZG8gaXQgZnJvbSB0aGUgbW91bnQgbGlrZSAKPiBO aXIgc2FpZC4KPiBXaHkgZGlkIHlvdSBkZWxldGUgdGhlIGZpbGUgZnJvbSB0aGUgLmdsdXN0ZXJm cyBpbiB0aGUgYnJpY2socyk/IFdhcyAKPiB0aGVyZSBhIGdmaWQgc3BsaXQgYnJhaW4/Cj4KPiAt UmF2aQo+CgoKLS0tLS0tLS0tLS0tLS0wODA3MDEwOTAyMDQwMDA4MDMwNjA1MDQKQ29udGVudC1U eXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzog OGJpdAoKPGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJz ZXQ9dXRmLTgiIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAgPC9oZWFkPgogIDxib2R5IHRl eHQ9IiMwMDAwNjYiIGJnY29sb3I9IiNGRkZGRkYiPgogICAgWWVzIHdlIGhhdmUgaGFkICJpZHMi IHNwbGl0IGJyYWlucyArIHNvbWUgb3RoZXIgVk0ncyBmaWxlczxicj4KICAgIFNwbGl0IGJyYWlu cyB3YXMgZml4ZWQgYnkgaGVhbGluZyB3aXRoIHByZWZmZXJlZCAoIHNvdXJjZSApIGJyaWNrLjxi cj4KICAgIDxicj4KICAgIGVnOiAiICMgZ2x1c3RlciB2b2x1bWUgaGVhbCAxS1ZNMTItUDEgc3Bs aXQtYnJhaW4gc291cmNlLWJyaWNrCiAgICAxNi4wLjAuMTYxOi9TVE9SQUdFUy9nMXI1cDEvR0ZT ICI8YnI+CiAgICA8YnI+CiAgICBQYXZlbDxicj4KICAgIDxicj4KICAgIDxicj4KICAgIDxibG9j a3F1b3RlIGNpdGU9Im1pZDo1NkQ2OTM2NS40MDkwMzAzQHJlZGhhdC5jb20iIHR5cGU9ImNpdGUi PgogICAgICBPa2F5LCBzbyB3aGF0IEkgdW5kZXJzdGFuZCBmcm9tIHRoZSBvdXRwdXQgYWJvdmUg aXMgeW91IGhhdmUKICAgICAgZGlmZmVyZW50IGdsdXN0ZXIgdm9sdW1lcyBtb3VudGVkIGFuZCBz b21lIG9mIHRoZW0gaGF2ZSBpbmNvcnJlY3QKICAgICAgcGVybWlzc2lvbnMgZm9yIHRoZSAnaWRz JyBmaWxlLiBUaGUgd2F5IHRvIGZpeCBpdCBpcyB0byBkbyBpdCBmcm9tCiAgICAgIHRoZSBtb3Vu dCBsaWtlIE5pciBzYWlkLjxicj4KICAgICAgV2h5IGRpZCB5b3UgZGVsZXRlIHRoZSBmaWxlIGZy b20gdGhlIC5nbHVzdGVyZnMgaW4gdGhlIGJyaWNrKHMpP8KgCiAgICAgIFdhcyB0aGVyZSBhIGdm aWQgc3BsaXQgYnJhaW4/IDxicj4KICAgICAgPGJyPgogICAgICAtUmF2aTxicj4KICAgICAgPGJy PgogICAgPC9ibG9ja3F1b3RlPgogICAgPGJyPgogIDwvYm9keT4KPC9odG1sPgoKLS0tLS0tLS0t LS0tLS0wODA3MDEwOTAyMDQwMDA4MDMwNjA1MDQtLQo= --===============0575629602994281979==-- From paf1 at email.cz Wed Mar 2 12:48:30 2016 Content-Type: multipart/mixed; boundary="===============4162557774279759291==" MIME-Version: 1.0 From: paf1 at email.cz To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Wed, 02 Mar 2016 18:48:23 +0100 Message-ID: <56D72767.7020004@email.cz> In-Reply-To: 56D69365.4090303@redhat.com --===============4162557774279759291== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------000700000508080601030108 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit UPDATE: all "ids" file have permittion fixed to 660 now # find /STORAGES -name ids -exec ls -l {} \; -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.41 = /STORAGES/g1r5p1/GFS/553d9b92-e4a0-4042-a579-4cabeb55ded4/dom_md/ids -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.43 = /STORAGES/g1r5p2/GFS/88adbd49-62d6-45b1-9992-b04464a04112/dom_md/ids -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.43 = /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.44 = /STORAGES/g1r5p4/GFS/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids -rw-rw---- 2 vdsm kvm 1048576 24. =C3=BAno 13.03 = /STORAGES/g1r5p5/GFS/3b24d023-fd35-4666-af2f-f5e1d19531ad/dom_md/ids -rw-rw---- 2 vdsm kvm 1048576 2. b=C5=99e 17.47 = /STORAGES/g2r5p1/GFS/0fcad888-d573-47be-bef3-0bc0b7a99fb7/dom_md/ids SPM is and was running continually ....... I tried to update "ids" file - ONLINE ( offline not possible yet ) # sanlock direct init -s = 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4= e23-ab46-084f3d70b147/dom_md/ids:0 # find /STORAGES -name ids -exec ls -l {} \; | grep p3 -rw-rw---- 1 vdsm kvm 1048576 2. b=C5=99e 18.32 = /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids The storage ids file has correct permittions, size, owners , but is not = checking by sanlock =3D the same access time What's wrong ?? regs. Pa. PS: # find /STORAGES -samefile = /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids -print /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids =3D missing "shadowfile" in " .gluster " dir. How can I fix it ?? - online ! On 2.3.2016 08:16, Ravishankar N wrote: > On 03/02/2016 12:02 PM, Sahina Bose wrote: >> >> >> On 03/02/2016 03:45 AM, Nir Soffer wrote: >>> On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz wrot= e: >>> > >>> > HI, >>> > requested output: >>> > >>> > # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47b= e-bef3-0bc0b7a99fb7/dom_md: >>> > total 2,1M >>> > -rw-rw---- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- good >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.17 leases >>> > -rw-r--r-- 1 vdsm kvm 335 7. lis 22.17 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042= -a579-4cabeb55ded4/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.41 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 03.56 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 03.56 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1= -9992-b04464a04112/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.14 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.14 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23= -ab46-084f3d70b147/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.51 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases >>> > -rw-r--r-- 1 vdsm kvm 998 25. =C3=BAno 00.35 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.16 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58= -89aa-102d44327124/dom_md: >>> > total 1,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.44 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 00.18 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 00.18 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc= -be41-77002d1cd553/dom_md: >>> > total 1,1M >>> > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.32 ids <-- bad (other can= read) >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.18 leases >>> > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.18 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528= -9bfe-c3da888e47f0/dom_md: >>> > total 3,0M >>> > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other can= read) >>> > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.42 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases >>> > -rw-r--r-- 1 vdsm kvm 997 24. =C3=BAno 02.46 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.44 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4= -ba9a-e6d9bdecb4b4/dom_md: >>> > total 2,1M >>> > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.34 ids <-- bad (sanlock = >>> cannot write, other can read) >>> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.35 inbox >>> > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases >>> > -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata >>> > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.27 outbox >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-470= 3-9bb1-1df2130c7c97/dom_md: >>> > total 3,0M >>> > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other can= read) >>> > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 23.50 inbox <-- bad (other can = >>> read) >>> > -rw-rw-r-- 1 vdsm kvm 2,0M 6. lis 23.51 leases <-- bad = >>> (other can read) >>> > -rw-rw-r-- 1 vdsm kvm 734 7. lis 02.13 metadata <-- bad = >>> (group can write, other can read) >>> > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 16.55 outbox <-- bad (other = >>> can read) >>> > >>> > = >>> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22= -9f59-3aebea4d8477/dom_md: >>> > total 1,1M >>> > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.35 ids <-- bad (other can= read) >>> > -rw-rw-r-- 1 vdsm kvm 16M 24. =C3=BAno 01.06 inbox >>> > -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases >>> > -rw-r--r-- 1 vdsm kvm 998 24. =C3=BAno 19.07 metadata >>> > -rw-rw-r-- 1 vdsm kvm 16M 7. lis 22.20 outbox >>> >>> >>> It should look like this: >>> >>> -rw-rw----. 1 vdsm kvm 1.0M Mar 1 23:36 ids >>> -rw-rw----. 1 vdsm kvm 2.0M Mar 1 23:35 leases >>> -rw-r--r--. 1 vdsm kvm 353 Mar 1 23:35 metadata >>> -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 outbox >>> -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 inbox >>> >>> This explains the EACCES error. >>> >>> You can start by fixing the permissions manually, you can do this = >>> online. >>> >>> > The ids files was generated by "touch" command after deleting = >>> them due "sanlock locking hang" gluster crash & reboot >>> > I expected that they will be filled automaticaly after gluster = >>> reboot ( the shadow copy from ".gluster " directory was = >>> deleted & created empty too ) >>> >>> I don't know about gluster shadow copy, I would not play with = >>> gluster internals. >>> Adding Sahina for advice. >> >> Did you generate the ids file on the mount point. >> >> Ravi, can you help here? >> > > Okay, so what I understand from the output above is you have different = > gluster volumes mounted and some of them have incorrect permissions = > for the 'ids' file. The way to fix it is to do it from the mount like = > Nir said. > Why did you delete the file from the .glusterfs in the brick(s)? Was = > there a gfid split brain? > > -Ravi > >>> >>> > OK, it looks that sanlock can't work with empty file or rewrite = >>> them . >>> > Am I right ?? >>> >>> Yes, the files must be initialized before sanlock can use them. >>> >>> You can initialize the file like this: >>> >>> sanlock direct init -s :0:repair//dom_md/ids:0 >>> >>> Taken from = >>> http://lists.ovirt.org/pipermail/users/2016-February/038046.html >>> >>> > The last point - about "ids" workaround - this is offline version = >>> =3D VMs have to be moved out from for continual running with = >>> maintenance volume mode >>> > But this is not acceptable in current situation, so the question = >>> again, is it safe to do it online ?? ( YES / NO ) >>> >>> The ids file is accessed only by sanlock. I guess that you don't = >>> have a running >>> SPM on this DC, since sanlock fails to acquire a host id, so you are = >>> pretty safe >>> to fix the permissions and initialize the ids files. >>> >>> I would do this: >>> >>> 1. Stop engine, so it will not try to start vdsm >>> 2. Stop vdsm on all hosts, so they do not try to acquire a host id = >>> with sanlock >>> This does not affect running vms >>> 3. Fix the permissions on the ids file, via glusterfs mount >>> 4. Initialize the ids files from one of the hosts, via the glusterfs = >>> mount >>> This should fix the ids files on all replicas >>> 5. Start vdsm on all hosts >>> 6. Start engine >>> >>> Engine will connect to all hosts, hosts will connect to storage and = >>> try to acquire a host id. >>> Then Engine will start the SPM on one of the hosts, and your DC = >>> should become up. >>> >>> David, Sahina, can you confirm that this procedure is safe? >> >> Yes, correcting from the mount point should fix it on all replicas >> >> >>> >>> Nir >>> >>> > >>> > regs. >>> > Pavel >>> > >>> > >>> > >>> > On 1.3.2016 18:38, Nir Soffer wrote: >>> > >>> > On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz wro= te: >>> >> >>> >> Hello, can anybody explain this error no.13 ( open file ) in = >>> sanlock.log . >>> > >>> > >>> > This is EACCES >>> > >>> > Can you share the outoput of: >>> > >>> > ls -lh /rhev/data-center/mnt/:<_path>//dom_md >>> > >>> >> >>> >> >>> >> The size of "ids" file is zero (0) >>> > >>> > >>> > This is how we create the ids file when initializing it. >>> > >>> > But then we use sanlock to initialize the ids file, and it should = >>> be 1MiB after that. >>> > >>> > Is this ids files created by vdsm, or one you created yourself? >>> > >>> >> >>> >> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58= -89aa-102d44327124/dom_md/ids >>> >> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk = >>> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58= -89aa-102d44327124/dom_md/ids = >>> error -13 >>> >> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace = >>> 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/= localhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 >>> >> >>> >> If the main problem is about zero file size, can I regenerate = >>> this file online securely , with no VM dependence ???? >>> > >>> > >>> > Yes, I think I already referred to the instructions how to do that = >>> in a previous mail. >>> > >>> >> >>> >> >>> >> dist =3D RHEL - 7 - 2.1511 >>> >> kernel =3D 3.10.0 - 327.10.1.el7.x86_64 >>> >> KVM =3D 2.3.0 - 29.1.el7 >>> >> libvirt =3D libvirt-1.2.17-13.el7_2.3 >>> >> vdsm =3D vdsm-4.16.30-0.el7 >>> >> GlusterFS =3D glusterfs-3.7.8-1.el7 >>> >> >>> >> >>> >> regs. >>> >> Pavel >>> >> >>> >> _______________________________________________ >>> >> Users mailing list >>> >> Users(a)ovirt.org >>> >> http://lists.ovirt.org/mailman/listinfo/users >>> >> >>> > >>> > >> >> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users(a)gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-users > > --------------000700000508080601030108 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit UPDATE:

all "ids"=C2=A0 file have permittion fixed to 660 now

#=C2=A0 find /STORAGES -name ids -exec ls -l {} \;
-rw-rw---- 2 vdsm kvm 0 24.=C2=A0=C3=BAno 07.41 /STORAGES/g1r5p1/GFS/553d9b92-e4a0-4042-a579-4cabeb55ded4/dom_md/ids
-rw-rw---- 2 vdsm kvm 0 24.=C2=A0=C3=BAno 07.43 /STORAGES/g1r5p2/GFS/88adbd49-62d6-45b1-9992-b04464a04112/dom_md/ids
-rw-rw---- 2 vdsm kvm 0 24.=C2=A0=C3=BAno 07.43 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids
-rw-rw---- 2 vdsm kvm 0 24.=C2=A0=C3=BAno 07.44 /STORAGES/g1r5p4/GFS/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids
-rw-rw---- 2 vdsm kvm 1048576 24.=C2=A0=C3=BAno 13.03 /STORAGES/g1r5p5/GFS/3b24d023-fd35-4666-af2f-f5e1d19531ad/dom_md/ids
-rw-rw---- 2 vdsm kvm 1048576=C2=A0 2.=C2=A0b=C5=99e 17.47 /STORAGES/g2r5p1/GFS/0fcad888-d573-47be-bef3-0bc0b7a99fb7/dom_md/ids

SPM is and=C2=A0 was=C2=A0 running continually .......

I tried to update "ids" file - ONLINE=C2=A0 ( offline not possible yet = )
# sanlock direct init -s 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4= e23-ab46-084f3d70b147/dom_md/ids:0

#=C2=A0 find /STORAGES -name ids -exec ls -l {} \; | grep p3
-rw-rw---- 1 vdsm kvm 1048576=C2=A0 2.=C2=A0b=C5=99e 18.32 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids

The storage ids file has correct permittions, size, owners , but is not checking by sanlock =3D the same access time
What's wrong ??

regs.
Pa.
PS:=C2=A0 # find /STORAGES -samefile /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids -print
/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids
=3D missing "shadowfile" in " .gluster " dir.
How can I fix it ?? - online !

On 2.3.2016 08:16, Ravishankar N wrote:<= br>

On 03/02/2016 12:02 PM, Sahina Bose wrote:

On 03/02/2016 03:45 AM, Nir Soffer wrote:

On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz <paf1(a)email.cz> wrote= :
>
> HI,
> requested output:
>
> # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md
> =C2=A0
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-be= f3-0bc0b7a99fb7/dom_md:
> total 2,1M
> -rw-rw---- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids =C2= =A0 =C2=A0 =C2=A0 =C2=A0<-- good
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.17 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0335 =C2=A07. lis 22.17 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.16 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a57= 9-4cabeb55ded4/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.41 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 03.56 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 03.56 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.14 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-999= 2-b04464a04112/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.14 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.14 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.15 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-ab4= 6-084f3d70b147/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.43 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.51 inbox > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 25. =C3=BAno 00.35 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.16 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md:
> total 1,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.44 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 00.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 00.18 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 00.17 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-be4= 1-77002d1cd553/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.32 id= s =C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 inbox > -rw-rw---- 1 vdsm kvm 2,0M =C2=A07. lis 22.18 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0333 =C2=A07. lis 22.18 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.18 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9bf= e-c3da888e47f0/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.42 inbox= =C2=A0
> -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0997 24. =C3=BAno 02.46 metadat= a
> -rw-rw---- 1 vdsm kvm =C2=A016M 25. =C3=BAno 00.44 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-ba9= a-e6d9bdecb4b4/dom_md:
> total 2,1M
> -rw-r--r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.34 id= s=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (sanlock cannot write, other can read)
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.35 inbox > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases
> -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata
> -rw-rw---- 1 vdsm kvm =C2=A016M 23. =C3=BAno 22.27 outbox<= br> >
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-9b= b1-1df2130c7c97/dom_md:
> total 3,0M
> -rw-rw-r-- 1 vdsm kvm 1,0M =C2=A01. b=C5=99e 21.28 ids=C2= =A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 23.50 inbox= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm 2,0M =C2=A06. lis 23.51 leases= =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A0734 =C2=A07. lis 02.13 metad= ata=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (group can write, other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A06. lis 16.55 outbo= x=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
>
> /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9f5= 9-3aebea4d8477/dom_md:
> total 1,1M
> -rw-rw-r-- 1 vdsm kvm =C2=A0 =C2=A00 24. =C3=BAno 07.35 = ids=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0<-- bad (other can read)
> -rw-rw-r-- 1 vdsm kvm =C2=A016M 24. =C3=BAno 01.06 inbox=
> -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases
> -rw-r--r-- 1 vdsm kvm =C2=A0998 24. =C3=BAno 19.07 metad= ata
> -rw-rw-r-- 1 vdsm kvm =C2=A016M =C2=A07. lis 22.20 outbo= x

It should look like this:

-rw-rw----. 1 vdsm kvm 1.0M Mar =C2=A01 23:36 ids
-rw-rw----. 1 vdsm kvm 2.0M Mar =C2=A01 23:35 leases
-rw-r--r--. 1 vdsm kvm =C2=A0353 Mar =C2=A01 23:35 metadata -rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 outbox
-rw-rw----. 1 vdsm kvm =C2=A016M Mar =C2=A01 23:34 inbox

This explains the EACCES error.

You can start by fixing the permissions manually, you can do this online.
=C2=A0
> =C2=A0The ids files was generated by "touch" command aft= er deleting them due "sanlock locking hang" =C2=A0gluster crash & reboot
> I expected that they will be filled automaticaly after gluster reboot ( the =C2=A0shadow copy from =C2=A0 ".gl= uster " =C2=A0 directory =C2=A0was deleted & created empty =C2=A0= too )

I don't know about gluster shadow copy, I would not play with gluster internals.

Adding Sahina for advice.

Did you generate the ids file on the mount point.

Ravi, can you help here?

Okay, so what I understand from the output above is you have different gluster volumes mounted and some of them have incorrect permissions for the 'ids' file. The way to fix it is to do it from the mount like Nir said.
Why did you delete the file from the .glusterfs in the brick(s)?=C2= =A0 Was there a gfid split brain?

-Ravi
> OK, it looks that sanlock =C2=A0can't work with empty fi= le or rewrite them .
> Am I right ??

Yes, the files must be initialized before sanlock can use them.

You can initialize the file like this:

sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0

Taken from http://lists.ovirt.org/pipermail/users/2016-February/038046.= html

> The last point - about "ids" workaround - this is offline version =3D VMs have to be moved out from for continual running with maintenance volume mode
> But this is not acceptable in current situation, so the question again, =C2=A0is it safe to do it online ?? =C2= =A0( YES / NO )

The ids file is accessed only by sanlock. I guess that you don't have a running

SPM on this DC, since sanlock fails to acquire a host id, so you are pretty safe

to fix the permissions and initialize the ids files.

I would do this:

1. Stop engine, =C2=A0so it will not try to start vdsm
2. Stop vdsm on all hosts, so they do not try to acquire a host id with sanlock

=C2=A0 =C2=A0 This does not affect running vms

3. Fix the permissions on the ids file, via glusterfs mount

4. Initialize the ids files from one of the hosts, via the glusterfs mount

=C2=A0 =C2=A0 This should fix the ids files on all replica= s

5. Start vdsm on all hosts

6. Start engine

Engine will connect to all hosts, hosts will connect to storage and try to acquire a host id.

Then Engine will start the SPM on one of the hosts, and your DC should become up.

David, Sahina, can you confirm that this procedure is safe?

Yes, correcting from the mount point should fix it on all replicas

Nir

>
> regs.
> Pavel
>
>
>
> On 1.3.2016 18:38, Nir Soffer wrote:
>
> On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz <paf1(a)email.cz> wro= te:
>>
>> Hello, =C2=A0can anybody =C2=A0explain this error no= .13 ( open file ) in sanlock.log .
>
>
> This is EACCES
>
> Can you share the outoput of:
>
> =C2=A0 =C2=A0 ls -lh /rhev/data-center/mnt/<server>:<_path>/<sd_uuid>/dom_md > =C2=A0
>>
>>
>> The size of =C2=A0"ids" file is zero (0)
>
>
> This is how we create the ids file when initializing it.
>
> But then we use sanlock to initialize the ids file, and it should be 1MiB after that.
>
> Is this ids files created by vdsm, or one you created yourself?
> =C2=A0
>>
>> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-89a= a-102d44327124/dom_md/ids
>> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697= -c199-4f58-89aa-102d44327124/dom_md/ids error -13
>> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/loca= lhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0
>>
>> If the main problem is about zero file size, can I regenerate =C2=A0this file online securely , with no VM dependence =C2=A0????
>
>
> Yes, I think I already referred to the instructions how to do that in a previous mail.
>
>>
>>
>> dist =3D RHEL - 7 - 2.1511
>> kernel =3D 3.10.0 - 327.10.1.el7.x86_64
>> KVM =3D 2.3.0 - 29.1.el7
>> libvirt =3D libvirt-1.2.17-13.el7_2.3
>> vdsm =3D vdsm-4.16.30-0.el7
>> GlusterFS =3D glusterfs-3.7.8-1.el7
>>
>>
>> regs.
>> Pavel
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http= ://lists.ovirt.org/mailman/listinfo/users
>>
>
>
_______________________________________________
Gluster-users mailing list
Gluster-users(a)gluster.org
http://www.gluster.org/mail=
man/listinfo/gluster-users

--------------000700000508080601030108-- --===============4162557774279759291== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMDA3MDAwMDA1MDgwODA2MDEwMzAxMDgKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK ClVQREFURToKCmFsbCAiaWRzIiAgZmlsZSBoYXZlIHBlcm1pdHRpb24gZml4ZWQgdG8gNjYwIG5v dwoKIyAgZmluZCAvU1RPUkFHRVMgLW5hbWUgaWRzIC1leGVjIGxzIC1sIHt9IFw7Ci1ydy1ydy0t LS0gMiB2ZHNtIGt2bSAwIDI0LiDDum5vIDA3LjQxIAovU1RPUkFHRVMvZzFyNXAxL0dGUy81NTNk OWI5Mi1lNGEwLTQwNDItYTU3OS00Y2FiZWI1NWRlZDQvZG9tX21kL2lkcwotcnctcnctLS0tIDIg dmRzbSBrdm0gMCAyNC4gw7pubyAwNy40MyAKL1NUT1JBR0VTL2cxcjVwMi9HRlMvODhhZGJkNDkt NjJkNi00NWIxLTk5OTItYjA0NDY0YTA0MTEyL2RvbV9tZC9pZHMKLXJ3LXJ3LS0tLSAyIHZkc20g a3ZtIDAgMjQuIMO6bm8gMDcuNDMgCi9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYt NGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCi1ydy1ydy0tLS0gMiB2ZHNtIGt2bSAw IDI0LiDDum5vIDA3LjQ0IAovU1RPUkFHRVMvZzFyNXA0L0dGUy83ZjUyYjY5Ny1jMTk5LTRmNTgt ODlhYS0xMDJkNDQzMjcxMjQvZG9tX21kL2lkcwotcnctcnctLS0tIDIgdmRzbSBrdm0gMTA0ODU3 NiAyNC4gw7pubyAxMy4wMyAKL1NUT1JBR0VTL2cxcjVwNS9HRlMvM2IyNGQwMjMtZmQzNS00NjY2 LWFmMmYtZjVlMWQxOTUzMWFkL2RvbV9tZC9pZHMKLXJ3LXJ3LS0tLSAyIHZkc20ga3ZtIDEwNDg1 NzYgIDIuIGLFmWUgMTcuNDcgCi9TVE9SQUdFUy9nMnI1cDEvR0ZTLzBmY2FkODg4LWQ1NzMtNDdi ZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21fbWQvaWRzCgpTUE0gaXMgYW5kICB3YXMgIHJ1bm5pbmcg Y29udGludWFsbHkgLi4uLi4uLgoKSSB0cmllZCB0byB1cGRhdGUgImlkcyIgZmlsZSAtIE9OTElO RSAgKCBvZmZsaW5lIG5vdCBwb3NzaWJsZSB5ZXQgKQojIHNhbmxvY2sgZGlyZWN0IGluaXQgLXMg CjNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0NzowOi9TVE9SQUdFUy9nMXI1cDMv R0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzOjAKCiMg IGZpbmQgL1NUT1JBR0VTIC1uYW1lIGlkcyAtZXhlYyBscyAtbCB7fSBcOyB8IGdyZXAgcDMKLXJ3 LXJ3LS0tLSAxIHZkc20ga3ZtIDEwNDg1NzYgIDIuIGLFmWUgMTguMzIgCi9TVE9SQUdFUy9nMXI1 cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCgpU aGUgc3RvcmFnZSBpZHMgZmlsZSBoYXMgY29ycmVjdCBwZXJtaXR0aW9ucywgc2l6ZSwgb3duZXJz ICwgYnV0IGlzIG5vdCAKY2hlY2tpbmcgYnkgc2FubG9jayA9IHRoZSBzYW1lIGFjY2VzcyB0aW1l CldoYXQncyB3cm9uZyA/PwoKcmVncy4KUGEuClBTOiAgIyBmaW5kIC9TVE9SQUdFUyAtc2FtZWZp bGUgCi9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcw YjE0Ny9kb21fbWQvaWRzIC1wcmludAovU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2 LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcwo9IG1pc3NpbmcgInNoYWRvd2ZpbGUi IGluICIgLmdsdXN0ZXIgIiBkaXIuCkhvdyBjYW4gSSBmaXggaXQgPz8gLSBvbmxpbmUgIQoKCgpP biAyLjMuMjAxNiAwODoxNiwgUmF2aXNoYW5rYXIgTiB3cm90ZToKPiBPbiAwMy8wMi8yMDE2IDEy OjAyIFBNLCBTYWhpbmEgQm9zZSB3cm90ZToKPj4KPj4KPj4gT24gMDMvMDIvMjAxNiAwMzo0NSBB TSwgTmlyIFNvZmZlciB3cm90ZToKPj4+IE9uIFR1ZSwgTWFyIDEsIDIwMTYgYXQgMTA6NTEgUE0s IHBhZjFAZW1haWwuY3ogPHBhZjFAZW1haWwuY3o+IHdyb3RlOgo+Pj4gPgo+Pj4gPiBISSwKPj4+ ID4gcmVxdWVzdGVkIG91dHB1dDoKPj4+ID4KPj4+ID4gIyBscyAtbGggL3JoZXYvZGF0YS1jZW50 ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6Ki8qL2RvbV9tZAo+Pj4gPgo+Pj4gPiAKPj4+IC9y aGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItQkNLLzBmY2Fk ODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21fbWQ6Cj4+PiA+IHRvdGFsIDIsMU0K Pj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDEsME0gIDEuIGLFmWUgMjEuMjggaWRzICA8LS0g Z29vZAo+Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDIyLjE2IGluYm94 Cj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNICA3LiBsaXMgMjIuMTcgbGVhc2VzCj4+ PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgMzM1ICA3LiBsaXMgMjIuMTcgbWV0YWRhdGEKPj4+ ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAyMi4xNiBvdXRib3gKPj4+ID4K Pj4+ID4gCj4+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtW TTEyLVAxLzU1M2Q5YjkyLWU0YTAtNDA0Mi1hNTc5LTRjYWJlYjU1ZGVkNC9kb21fbWQ6Cj4+PiA+ IHRvdGFsIDEsMU0KPj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcu NDEgaWRzICA8LS0gYmFkIChzYW5sb2NrIAo+Pj4gY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVh ZCkKPj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNCBpbmJveAo+ Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDAzLjU2IGxlYXNlcwo+Pj4g PiAtcnctci0tci0tIDEgdmRzbSBrdm0gIDMzMyAgNy4gbGlzIDAzLjU2IG1ldGFkYXRhCj4+PiA+ IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTQgb3V0Ym94Cj4+PiA+Cj4+ PiA+IAo+Pj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0x Mi1QMi84OGFkYmQ0OS02MmQ2LTQ1YjEtOTk5Mi1iMDQ0NjRhMDQxMTIvZG9tX21kOgo+Pj4gPiB0 b3RhbCAxLDFNCj4+PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjQz IGlkcyAgPC0tIGJhZCAoc2FubG9jayAKPj4+IGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQp Cj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTUgaW5ib3gKPj4+ ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gIDcuIGxpcyAyMi4xNCBsZWFzZXMKPj4+ID4g LXJ3LXItLXItLSAxIHZkc20ga3ZtICAzMzMgIDcuIGxpcyAyMi4xNCBtZXRhZGF0YQo+Pj4gPiAt cnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4gbGlzIDAwLjE1IG91dGJveAo+Pj4gPgo+Pj4g PiAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTIt UDMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZDoKPj4+ID4gdG90 YWwgMSwxTQo+Pj4gPiAtcnctci0tci0tIDEgdmRzbSBrdm0gICAgMCAyNC4gw7pubyAwNy40MyBp ZHMgIDwtLSBiYWQgKHNhbmxvY2sgCj4+PiBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKQo+ Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyMy4gw7pubyAyMi41MSBpbmJveAo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4gw7pubyAyMy4xMiBsZWFzZXMKPj4+ID4g LXJ3LXItLXItLSAxIHZkc20ga3ZtICA5OTggMjUuIMO6bm8gMDAuMzUgbWV0YWRhdGEKPj4+ID4g LXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNiBvdXRib3gKPj4+ID4KPj4+ ID4gCj4+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEy LVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQ6Cj4+PiA+IHRv dGFsIDEsMU0KPj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAgIDAgMjQuIMO6bm8gMDcuNDQg aWRzICA8LS0gYmFkIChzYW5sb2NrIAo+Pj4gY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCkK Pj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gIDcuIGxpcyAwMC4xNyBpbmJveAo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAgNy4gbGlzIDAwLjE4IGxlYXNlcwo+Pj4gPiAt cnctci0tci0tIDEgdmRzbSBrdm0gIDMzMyAgNy4gbGlzIDAwLjE4IG1ldGFkYXRhCj4+PiA+IC1y dy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNICA3LiBsaXMgMDAuMTcgb3V0Ym94Cj4+PiA+Cj4+PiA+ IAo+Pj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1Q MS80MmQ3MTBhOS1iODQ0LTQzZGMtYmU0MS03NzAwMmQxY2Q1NTMvZG9tX21kOgo+Pj4gPiB0b3Rh bCAxLDFNCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjMyIGlk cyAgPC0tIGJhZCAob3RoZXIgY2FuIHJlYWQpCj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAg MTZNICA3LiBsaXMgMjIuMTggaW5ib3gKPj4+ID4gLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0g IDcuIGxpcyAyMi4xOCBsZWFzZXMKPj4+ID4gLXJ3LXItLXItLSAxIHZkc20ga3ZtICAzMzMgIDcu IGxpcyAyMi4xOCBtZXRhZGF0YQo+Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAgNy4g bGlzIDIyLjE4IG91dGJveAo+Pj4gPgo+Pj4gPiAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9n bHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDIvZmY3MWI0N2ItMGY3Mi00NTI4LTliZmUtYzNk YTg4OGU0N2YwL2RvbV9tZDoKPj4+ID4gdG90YWwgMywwTQo+Pj4gPiAtcnctcnctci0tIDEgdmRz bSBrdm0gMSwwTSAgMS4gYsWZZSAyMS4yOCBpZHMgIDwtLSBiYWQgKG90aGVyIGNhbiByZWFkKQo+ Pj4gPiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyNS4gw7pubyAwMC40MiBpbmJveAo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyNS4gw7pubyAwMC40NCBsZWFzZXMKPj4+ID4g LXJ3LXItLXItLSAxIHZkc20ga3ZtICA5OTcgMjQuIMO6bm8gMDIuNDYgbWV0YWRhdGEKPj4+ID4g LXJ3LXJ3LS0tLSAxIHZkc20ga3ZtICAxNk0gMjUuIMO6bm8gMDAuNDQgb3V0Ym94Cj4+PiA+Cj4+ PiA+IAo+Pj4gL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0x Mi1QMy9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRlY2I0YjQvZG9tX21kOgo+Pj4gPiB0 b3RhbCAyLDFNCj4+PiA+IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSAgICAwIDI0LiDDum5vIDA3LjM0 IGlkcyAgPC0tIGJhZCAoc2FubG9jayAKPj4+IGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQp Cj4+PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAgMTZNIDIzLiDDum5vIDIyLjM1IGluYm94Cj4+ PiA+IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDIzLiDDum5vIDIyLjM4IGxlYXNlcwo+Pj4g PiAtcnctci0tci0tIDEgdmRzbSBrdm0gMSwxSyAyNC4gw7pubyAxOS4wNyBtZXRhZGF0YQo+Pj4g PiAtcnctcnctLS0tIDEgdmRzbSBrdm0gIDE2TSAyMy4gw7pubyAyMi4yNyBvdXRib3gKPj4+ID4K Pj4+ID4gCj4+PiAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktW TTEyX19QNC8zMDBlOWFjOC0zYzJmLTQ3MDMtOWJiMS0xZGYyMTMwYzdjOTcvZG9tX21kOgo+Pj4g PiB0b3RhbCAzLDBNCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAxLDBNICAxLiBixZllIDIx LjI4IGlkcyAgPC0tIGJhZCAob3RoZXIgY2FuIHJlYWQpCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNt IGt2bSAgMTZNICA2LiBsaXMgMjMuNTAgaW5ib3ggIDwtLSBiYWQgKG90aGVyIGNhbiAKPj4+IHJl YWQpCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAyLDBNICA2LiBsaXMgMjMuNTEgbGVhc2Vz ICAgICAgPC0tIGJhZCAKPj4+IChvdGhlciBjYW4gcmVhZCkKPj4+ID4gLXJ3LXJ3LXItLSAxIHZk c20ga3ZtICA3MzQgIDcuIGxpcyAwMi4xMyBtZXRhZGF0YSAgICA8LS0gYmFkIAo+Pj4gKGdyb3Vw IGNhbiB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpCj4+PiA+IC1ydy1ydy1yLS0gMSB2ZHNtIGt2bSAg MTZNICA2LiBsaXMgMTYuNTUgb3V0Ym94ICA8LS0gYmFkIChvdGhlciAKPj4+IGNhbiByZWFkKQo+ Pj4gPgo+Pj4gPiAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0 Ol8yS1ZNMTItUDUvMWNhNTZiNDUtNzAxZS00YzIyLTlmNTktM2FlYmVhNGQ4NDc3L2RvbV9tZDoK Pj4+ID4gdG90YWwgMSwxTQo+Pj4gPiAtcnctcnctci0tIDEgdmRzbSBrdm0gICAgMCAyNC4gw7pu byAwNy4zNSBpZHMgIDwtLSBiYWQgKG90aGVyIGNhbiByZWFkKQo+Pj4gPiAtcnctcnctci0tIDEg dmRzbSBrdm0gIDE2TSAyNC4gw7pubyAwMS4wNiBpbmJveAo+Pj4gPiAtcnctcnctci0tIDEgdmRz bSBrdm0gMiwwTSAyNC4gw7pubyAwMi40NCBsZWFzZXMKPj4+ID4gLXJ3LXItLXItLSAxIHZkc20g a3ZtICA5OTggMjQuIMO6bm8gMTkuMDcgbWV0YWRhdGEKPj4+ID4gLXJ3LXJ3LXItLSAxIHZkc20g a3ZtICAxNk0gIDcuIGxpcyAyMi4yMCBvdXRib3gKPj4+Cj4+Pgo+Pj4gSXQgc2hvdWxkIGxvb2sg bGlrZSB0aGlzOgo+Pj4KPj4+IC1ydy1ydy0tLS0uIDEgdmRzbSBrdm0gMS4wTSBNYXIgIDEgMjM6 MzYgaWRzCj4+PiAtcnctcnctLS0tLiAxIHZkc20ga3ZtIDIuME0gTWFyICAxIDIzOjM1IGxlYXNl cwo+Pj4gLXJ3LXItLXItLS4gMSB2ZHNtIGt2bSAgMzUzIE1hciAgMSAyMzozNSBtZXRhZGF0YQo+ Pj4gLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSAgMTZNIE1hciAgMSAyMzozNCBvdXRib3gKPj4+IC1y dy1ydy0tLS0uIDEgdmRzbSBrdm0gIDE2TSBNYXIgIDEgMjM6MzQgaW5ib3gKPj4+Cj4+PiBUaGlz IGV4cGxhaW5zIHRoZSBFQUNDRVMgZXJyb3IuCj4+Pgo+Pj4gWW91IGNhbiBzdGFydCBieSBmaXhp bmcgdGhlIHBlcm1pc3Npb25zIG1hbnVhbGx5LCB5b3UgY2FuIGRvIHRoaXMgCj4+PiBvbmxpbmUu Cj4+Pgo+Pj4gPiAgVGhlIGlkcyBmaWxlcyB3YXMgZ2VuZXJhdGVkIGJ5ICJ0b3VjaCIgY29tbWFu ZCBhZnRlciBkZWxldGluZyAKPj4+IHRoZW0gZHVlICJzYW5sb2NrIGxvY2tpbmcgaGFuZyIgIGds dXN0ZXIgY3Jhc2ggJiByZWJvb3QKPj4+ID4gSSBleHBlY3RlZCB0aGF0IHRoZXkgd2lsbCBiZSBm aWxsZWQgYXV0b21hdGljYWx5IGFmdGVyIGdsdXN0ZXIgCj4+PiByZWJvb3QgKCB0aGUgIHNoYWRv dyBjb3B5IGZyb20gICAiLmdsdXN0ZXIgIiAgIGRpcmVjdG9yeSAgd2FzIAo+Pj4gZGVsZXRlZCAm IGNyZWF0ZWQgZW1wdHkgIHRvbyApCj4+Pgo+Pj4gSSBkb24ndCBrbm93IGFib3V0IGdsdXN0ZXIg c2hhZG93IGNvcHksIEkgd291bGQgbm90IHBsYXkgd2l0aCAKPj4+IGdsdXN0ZXIgaW50ZXJuYWxz Lgo+Pj4gQWRkaW5nIFNhaGluYSBmb3IgYWR2aWNlLgo+Pgo+PiBEaWQgeW91IGdlbmVyYXRlIHRo ZSBpZHMgZmlsZSBvbiB0aGUgbW91bnQgcG9pbnQuCj4+Cj4+IFJhdmksIGNhbiB5b3UgaGVscCBo ZXJlPwo+Pgo+Cj4gT2theSwgc28gd2hhdCBJIHVuZGVyc3RhbmQgZnJvbSB0aGUgb3V0cHV0IGFi b3ZlIGlzIHlvdSBoYXZlIGRpZmZlcmVudCAKPiBnbHVzdGVyIHZvbHVtZXMgbW91bnRlZCBhbmQg c29tZSBvZiB0aGVtIGhhdmUgaW5jb3JyZWN0IHBlcm1pc3Npb25zIAo+IGZvciB0aGUgJ2lkcycg ZmlsZS4gVGhlIHdheSB0byBmaXggaXQgaXMgdG8gZG8gaXQgZnJvbSB0aGUgbW91bnQgbGlrZSAK PiBOaXIgc2FpZC4KPiBXaHkgZGlkIHlvdSBkZWxldGUgdGhlIGZpbGUgZnJvbSB0aGUgLmdsdXN0 ZXJmcyBpbiB0aGUgYnJpY2socyk/IFdhcyAKPiB0aGVyZSBhIGdmaWQgc3BsaXQgYnJhaW4/Cj4K PiAtUmF2aQo+Cj4+Pgo+Pj4gPiBPSywgaXQgbG9va3MgdGhhdCBzYW5sb2NrICBjYW4ndCB3b3Jr IHdpdGggZW1wdHkgZmlsZSBvciByZXdyaXRlIAo+Pj4gdGhlbSAuCj4+PiA+IEFtIEkgcmlnaHQg Pz8KPj4+Cj4+PiBZZXMsIHRoZSBmaWxlcyBtdXN0IGJlIGluaXRpYWxpemVkIGJlZm9yZSBzYW5s b2NrIGNhbiB1c2UgdGhlbS4KPj4+Cj4+PiBZb3UgY2FuIGluaXRpYWxpemUgdGhlIGZpbGUgbGlr ZSB0aGlzOgo+Pj4KPj4+IHNhbmxvY2sgZGlyZWN0IGluaXQgLXMgPHNkX3V1aWQ+OjA6cmVwYWly LzxzZF91dWlkPi9kb21fbWQvaWRzOjAKPj4+Cj4+PiBUYWtlbiBmcm9tIAo+Pj4gaHR0cDovL2xp c3RzLm92aXJ0Lm9yZy9waXBlcm1haWwvdXNlcnMvMjAxNi1GZWJydWFyeS8wMzgwNDYuaHRtbAo+ Pj4KPj4+ID4gVGhlIGxhc3QgcG9pbnQgLSBhYm91dCAiaWRzIiB3b3JrYXJvdW5kIC0gdGhpcyBp cyBvZmZsaW5lIHZlcnNpb24gCj4+PiA9IFZNcyBoYXZlIHRvIGJlIG1vdmVkIG91dCBmcm9tIGZv ciBjb250aW51YWwgcnVubmluZyB3aXRoIAo+Pj4gbWFpbnRlbmFuY2Ugdm9sdW1lIG1vZGUKPj4+ ID4gQnV0IHRoaXMgaXMgbm90IGFjY2VwdGFibGUgaW4gY3VycmVudCBzaXR1YXRpb24sIHNvIHRo ZSBxdWVzdGlvbiAKPj4+IGFnYWluLCAgaXMgaXQgc2FmZSB0byBkbyBpdCBvbmxpbmUgPz8gICgg WUVTIC8gTk8gKQo+Pj4KPj4+IFRoZSBpZHMgZmlsZSBpcyBhY2Nlc3NlZCBvbmx5IGJ5IHNhbmxv Y2suIEkgZ3Vlc3MgdGhhdCB5b3UgZG9uJ3QgCj4+PiBoYXZlIGEgcnVubmluZwo+Pj4gU1BNIG9u IHRoaXMgREMsIHNpbmNlIHNhbmxvY2sgZmFpbHMgdG8gYWNxdWlyZSBhIGhvc3QgaWQsIHNvIHlv dSBhcmUgCj4+PiBwcmV0dHkgc2FmZQo+Pj4gdG8gZml4IHRoZSBwZXJtaXNzaW9ucyBhbmQgaW5p dGlhbGl6ZSB0aGUgaWRzIGZpbGVzLgo+Pj4KPj4+IEkgd291bGQgZG8gdGhpczoKPj4+Cj4+PiAx LiBTdG9wIGVuZ2luZSwgIHNvIGl0IHdpbGwgbm90IHRyeSB0byBzdGFydCB2ZHNtCj4+PiAyLiBT dG9wIHZkc20gb24gYWxsIGhvc3RzLCBzbyB0aGV5IGRvIG5vdCB0cnkgdG8gYWNxdWlyZSBhIGhv c3QgaWQgCj4+PiB3aXRoIHNhbmxvY2sKPj4+ICAgICBUaGlzIGRvZXMgbm90IGFmZmVjdCBydW5u aW5nIHZtcwo+Pj4gMy4gRml4IHRoZSBwZXJtaXNzaW9ucyBvbiB0aGUgaWRzIGZpbGUsIHZpYSBn bHVzdGVyZnMgbW91bnQKPj4+IDQuIEluaXRpYWxpemUgdGhlIGlkcyBmaWxlcyBmcm9tIG9uZSBv ZiB0aGUgaG9zdHMsIHZpYSB0aGUgZ2x1c3RlcmZzIAo+Pj4gbW91bnQKPj4+ICAgICBUaGlzIHNo b3VsZCBmaXggdGhlIGlkcyBmaWxlcyBvbiBhbGwgcmVwbGljYXMKPj4+IDUuIFN0YXJ0IHZkc20g b24gYWxsIGhvc3RzCj4+PiA2LiBTdGFydCBlbmdpbmUKPj4+Cj4+PiBFbmdpbmUgd2lsbCBjb25u ZWN0IHRvIGFsbCBob3N0cywgaG9zdHMgd2lsbCBjb25uZWN0IHRvIHN0b3JhZ2UgYW5kIAo+Pj4g dHJ5IHRvIGFjcXVpcmUgYSBob3N0IGlkLgo+Pj4gVGhlbiBFbmdpbmUgd2lsbCBzdGFydCB0aGUg U1BNIG9uIG9uZSBvZiB0aGUgaG9zdHMsIGFuZCB5b3VyIERDIAo+Pj4gc2hvdWxkIGJlY29tZSB1 cC4KPj4+Cj4+PiBEYXZpZCwgU2FoaW5hLCBjYW4geW91IGNvbmZpcm0gdGhhdCB0aGlzIHByb2Nl ZHVyZSBpcyBzYWZlPwo+Pgo+PiBZZXMsIGNvcnJlY3RpbmcgZnJvbSB0aGUgbW91bnQgcG9pbnQg c2hvdWxkIGZpeCBpdCBvbiBhbGwgcmVwbGljYXMKPj4KPj4KPj4+Cj4+PiBOaXIKPj4+Cj4+PiA+ Cj4+PiA+IHJlZ3MuCj4+PiA+IFBhdmVsCj4+PiA+Cj4+PiA+Cj4+PiA+Cj4+PiA+IE9uIDEuMy4y MDE2IDE4OjM4LCBOaXIgU29mZmVyIHdyb3RlOgo+Pj4gPgo+Pj4gPiBPbiBUdWUsIE1hciAxLCAy MDE2IGF0IDU6MDcgUE0sIHBhZjFAZW1haWwuY3ogPHBhZjFAZW1haWwuY3o+IHdyb3RlOgo+Pj4g Pj4KPj4+ID4+IEhlbGxvLCAgY2FuIGFueWJvZHkgIGV4cGxhaW4gdGhpcyBlcnJvciBuby4xMyAo IG9wZW4gZmlsZSApIGluIAo+Pj4gc2FubG9jay5sb2cgLgo+Pj4gPgo+Pj4gPgo+Pj4gPiBUaGlz IGlzIEVBQ0NFUwo+Pj4gPgo+Pj4gPiBDYW4geW91IHNoYXJlIHRoZSBvdXRvcHV0IG9mOgo+Pj4g Pgo+Pj4gPiAgICAgbHMgLWxoIC9yaGV2L2RhdGEtY2VudGVyL21udC88c2VydmVyPjo8X3BhdGg+ LzxzZF91dWlkPi9kb21fbWQKPj4+ID4KPj4+ID4+Cj4+PiA+Pgo+Pj4gPj4gVGhlIHNpemUgb2Yg ICJpZHMiIGZpbGUgaXMgemVybyAoMCkKPj4+ID4KPj4+ID4KPj4+ID4gVGhpcyBpcyBob3cgd2Ug Y3JlYXRlIHRoZSBpZHMgZmlsZSB3aGVuIGluaXRpYWxpemluZyBpdC4KPj4+ID4KPj4+ID4gQnV0 IHRoZW4gd2UgdXNlIHNhbmxvY2sgdG8gaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGUsIGFuZCBpdCBz aG91bGQgCj4+PiBiZSAxTWlCIGFmdGVyIHRoYXQuCj4+PiA+Cj4+PiA+IElzIHRoaXMgaWRzIGZp bGVzIGNyZWF0ZWQgYnkgdmRzbSwgb3Igb25lIHlvdSBjcmVhdGVkIHlvdXJzZWxmPwo+Pj4gPgo+ Pj4gPj4KPj4+ID4+IDIwMTYtMDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBvcGVu IGVycm9yIC0xMyAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0 Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9p ZHMKPj4+ID4+IDIwMTYtMDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBzMTg3OTg1 IG9wZW5fZGlzayAKPj4+IC9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0 Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9p ZHMgCj4+PiBlcnJvciAtMTMKPj4+ID4+IDIwMTYtMDItMjggMDM6MjU6NTYrMDEwMCAyNjk2MzYg WzExMzA0XTogczE4Nzk5MiBsb2Nrc3BhY2UgCj4+PiA3ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0x MDJkNDQzMjcxMjQ6MTovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpf MUtWTTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRz OjAKPj4+ID4+Cj4+PiA+PiBJZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBz aXplLCBjYW4gSSByZWdlbmVyYXRlIAo+Pj4gIHRoaXMgZmlsZSBvbmxpbmUgc2VjdXJlbHkgLCB3 aXRoIG5vIFZNIGRlcGVuZGVuY2UgID8/Pz8KPj4+ID4KPj4+ID4KPj4+ID4gWWVzLCBJIHRoaW5r IEkgYWxyZWFkeSByZWZlcnJlZCB0byB0aGUgaW5zdHJ1Y3Rpb25zIGhvdyB0byBkbyB0aGF0IAo+ Pj4gaW4gYSBwcmV2aW91cyBtYWlsLgo+Pj4gPgo+Pj4gPj4KPj4+ID4+Cj4+PiA+PiBkaXN0ID0g UkhFTCAtIDcgLSAyLjE1MTEKPj4+ID4+IGtlcm5lbCA9IDMuMTAuMCAtIDMyNy4xMC4xLmVsNy54 ODZfNjQKPj4+ID4+IEtWTSA9IDIuMy4wIC0gMjkuMS5lbDcKPj4+ID4+IGxpYnZpcnQgPSBsaWJ2 aXJ0LTEuMi4xNy0xMy5lbDdfMi4zCj4+PiA+PiB2ZHNtID0gdmRzbS00LjE2LjMwLTAuZWw3Cj4+ PiA+PiBHbHVzdGVyRlMgPSBnbHVzdGVyZnMtMy43LjgtMS5lbDcKPj4+ID4+Cj4+PiA+Pgo+Pj4g Pj4gcmVncy4KPj4+ID4+IFBhdmVsCj4+PiA+Pgo+Pj4gPj4gX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX18KPj4+ID4+IFVzZXJzIG1haWxpbmcgbGlzdAo+Pj4g Pj4gVXNlcnNAb3ZpcnQub3JnIDxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPgo+Pj4gPj4gaHR0cDov L2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzCj4+PiA+Pgo+Pj4gPgo+Pj4g Pgo+Pgo+Pgo+Pgo+PiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwo+PiBHbHVzdGVyLXVzZXJzIG1haWxpbmcgbGlzdAo+PiBHbHVzdGVyLXVzZXJzQGdsdXN0 ZXIub3JnCj4+IGh0dHA6Ly93d3cuZ2x1c3Rlci5vcmcvbWFpbG1hbi9saXN0aW5mby9nbHVzdGVy LXVzZXJzCj4KPgoKCi0tLS0tLS0tLS0tLS0tMDAwNzAwMDAwNTA4MDgwNjAxMDMwMTA4CkNvbnRl bnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04CkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rp bmc6IDhiaXQKCjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEgY29udGVudD0idGV4dC9odG1sOyBj aGFyc2V0PXV0Zi04IiBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogIDwvaGVhZD4KICA8Ym9k eSB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIFVQREFURTo8YnI+CiAgICA8 YnI+CiAgICBhbGwgImlkcyLCoCBmaWxlIGhhdmUgcGVybWl0dGlvbiBmaXhlZCB0byA2NjAgbm93 PGJyPgogICAgPGJyPgogICAgI8KgIGZpbmQgL1NUT1JBR0VTIC1uYW1lIGlkcyAtZXhlYyBscyAt bCB7fSBcOzxicj4KICAgIC1ydy1ydy0tLS0gMiB2ZHNtIGt2bSAwIDI0LsKgw7pubyAwNy40MQog ICAgL1NUT1JBR0VTL2cxcjVwMS9HRlMvNTUzZDliOTItZTRhMC00MDQyLWE1NzktNGNhYmViNTVk ZWQ0L2RvbV9tZC9pZHM8YnI+CiAgICAtcnctcnctLS0tIDIgdmRzbSBrdm0gMCAyNC7CoMO6bm8g MDcuNDMKICAgIC9TVE9SQUdFUy9nMXI1cDIvR0ZTLzg4YWRiZDQ5LTYyZDYtNDViMS05OTkyLWIw NDQ2NGEwNDExMi9kb21fbWQvaWRzPGJyPgogICAgLXJ3LXJ3LS0tLSAyIHZkc20ga3ZtIDAgMjQu wqDDum5vIDA3LjQzCiAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMt YWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczxicj4KICAgIC1ydy1ydy0tLS0gMiB2ZHNtIGt2 bSAwIDI0LsKgw7pubyAwNy40NAogICAgL1NUT1JBR0VTL2cxcjVwNC9HRlMvN2Y1MmI2OTctYzE5 OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM8YnI+CiAgICAtcnctcnctLS0tIDIg dmRzbSBrdm0gMTA0ODU3NiAyNC7CoMO6bm8gMTMuMDMKICAgIC9TVE9SQUdFUy9nMXI1cDUvR0ZT LzNiMjRkMDIzLWZkMzUtNDY2Ni1hZjJmLWY1ZTFkMTk1MzFhZC9kb21fbWQvaWRzPGJyPgogICAg LXJ3LXJ3LS0tLSAyIHZkc20ga3ZtIDEwNDg1NzbCoCAyLsKgYsWZZSAxNy40NwogICAgL1NUT1JB R0VTL2cycjVwMS9HRlMvMGZjYWQ4ODgtZDU3My00N2JlLWJlZjMtMGJjMGI3YTk5ZmI3L2RvbV9t ZC9pZHM8YnI+CiAgICA8YnI+CiAgICBTUE0gaXMgYW5kwqAgd2FzwqAgcnVubmluZyBjb250aW51 YWxseSAuLi4uLi4uPGJyPgogICAgPGJyPgogICAgSSB0cmllZCB0byB1cGRhdGUgImlkcyIgZmls ZSAtIE9OTElORcKgICggb2ZmbGluZSBub3QgcG9zc2libGUgeWV0ICk8YnI+CiAgICAjIHNhbmxv Y2sgZGlyZWN0IGluaXQgLXMKM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3OjA6 L1NUT1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3 L2RvbV9tZC9pZHM6MDxicj4KICAgIDxicj4KICAgICPCoCBmaW5kIC9TVE9SQUdFUyAtbmFtZSBp ZHMgLWV4ZWMgbHMgLWwge30gXDsgfCBncmVwIHAzPGJyPgogICAgLXJ3LXJ3LS0tLSAxIHZkc20g a3ZtIDEwNDg1NzbCoCAyLsKgYsWZZSAxOC4zMgogICAgL1NUT1JBR0VTL2cxcjVwMy9HRlMvM2Mz NGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZC9pZHM8YnI+CiAgICA8YnI+ CiAgICBUaGUgc3RvcmFnZSBpZHMgZmlsZSBoYXMgY29ycmVjdCBwZXJtaXR0aW9ucywgc2l6ZSwg b3duZXJzICwgYnV0IGlzCiAgICBub3QgY2hlY2tpbmcgYnkgc2FubG9jayA9IHRoZSBzYW1lIGFj Y2VzcyB0aW1lIDxicj4KICAgIFdoYXQncyB3cm9uZyA/Pzxicj4KICAgIDxicj4KICAgIHJlZ3Mu PGJyPgogICAgUGEuPGJyPgogICAgUFM6wqAgIyBmaW5kIC9TVE9SQUdFUyAtc2FtZWZpbGUKICAg IC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0 Ny9kb21fbWQvaWRzCiAgICAtcHJpbnQ8YnI+CiAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0 YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczxicj4KICAgID0gbWlz c2luZyAic2hhZG93ZmlsZSIgaW4gIiAuZ2x1c3RlciAiIGRpci48YnI+CiAgICBIb3cgY2FuIEkg Zml4IGl0ID8/IC0gb25saW5lICE8YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8 ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPk9uIDIuMy4yMDE2IDA4OjE2LCBSYXZpc2hhbmth ciBOIHdyb3RlOjxicj4KICAgIDwvZGl2PgogICAgPGJsb2NrcXVvdGUgY2l0ZT0ibWlkOjU2RDY5 MzY1LjQwOTAzMDNAcmVkaGF0LmNvbSIgdHlwZT0iY2l0ZSI+CiAgICAgIDxtZXRhIGNvbnRlbnQ9 InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCIgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICAg ICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5PbiAwMy8wMi8yMDE2IDEyOjAyIFBNLCBT YWhpbmEgQm9zZQogICAgICAgIHdyb3RlOjxicj4KICAgICAgPC9kaXY+CiAgICAgIDxibG9ja3F1 b3RlIGNpdGU9Im1pZDo1NkQ2ODkxMC44MDQwNjAyQHJlZGhhdC5jb20iIHR5cGU9ImNpdGUiPgog ICAgICAgIDxtZXRhIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCIKICAgICAgICAg IGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAgICAgICAgPGJyPgogICAgICAgIDxicj4KICAg ICAgICA8ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPk9uIDAzLzAyLzIwMTYgMDM6NDUgQU0s IE5pciBTb2ZmZXIKICAgICAgICAgIHdyb3RlOjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8 YmxvY2txdW90ZQpjaXRlPSJtaWQ6Q0FNUmJ5eXU5Z3dQZlZwUHhwRGE0X2dLV3lYcTFQYXZUbTJW MnJHMmNVMEF2RT1KSlBBQG1haWwuZ21haWwuY29tIgogICAgICAgICAgdHlwZT0iY2l0ZSI+CiAg ICAgICAgICA8ZGl2IGRpcj0ibHRyIj5PbiBUdWUsIE1hciAxLCAyMDE2IGF0IDEwOjUxIFBNLCA8 YQogICAgICAgICAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgY2xhc3M9Im1vei10eHQtbGlu ay1hYmJyZXZpYXRlZCIKICAgICAgICAgICAgICBocmVmPSJtYWlsdG86cGFmMUBlbWFpbC5jeiI+ cGFmMUBlbWFpbC5jejwvYT4gJmx0OzxhCiAgICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0 cnVlIiBjbGFzcz0ibW96LXR4dC1saW5rLWFiYnJldmlhdGVkIgogICAgICAgICAgICAgIGhyZWY9 Im1haWx0bzpwYWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiZndDsgd3JvdGU6PGJyPgog ICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7IEhJLDxicj4KICAgICAgICAgICAg Jmd0OyByZXF1ZXN0ZWQgb3V0cHV0Ojxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAg ICAgJmd0OyAjIGxzIC1saAogICAgICAgICAgICAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3Rl clNEL2xvY2FsaG9zdDoqLyovZG9tX21kPGJyPgogICAgICAgICAgICAmZ3Q7IMKgPGJyPgogICAg ICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8x S1ZNMTItQkNLLzBmY2FkODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21fbWQ6PGJy PgogICAgICAgICAgICAmZ3Q7IHRvdGFsIDIsMU08YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3 LS0tLSAxIHZkc20ga3ZtIDEsME0gwqAxLiBixZllIDIxLjI4IGlkcyDCoCDCoCDCoAogICAgICAg ICAgICDCoCZsdDstLSBnb29kPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNt IGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4xNiBpbmJveDxicj4KICAgICAgICAgICAgJmd0OyAtcnct cnctLS0tIDEgdmRzbSBrdm0gMiwwTSDCoDcuIGxpcyAyMi4xNyBsZWFzZXM8YnI+CiAgICAgICAg ICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgMzM1IMKgNy4gbGlzIDIyLjE3IG1ldGFk YXRhPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcu IGxpcyAyMi4xNiBvdXRib3g8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICZn dDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QMS81 NTNkOWI5Mi1lNGEwLTQwNDItYTU3OS00Y2FiZWI1NWRlZDQvZG9tX21kOjxicj4KICAgICAgICAg ICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNt IGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuNDEgaWRzIMKgIMKgIMKgCiAgICAgICAgICAgIMKgJmx0 Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKTxicj4KICAgICAg ICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTQgaW5i b3g8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBs aXMgMDMuNTYgbGVhc2VzPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2 bSDCoDMzMyDCoDcuIGxpcyAwMy41NiBtZXRhZGF0YTxicj4KICAgICAgICAgICAgJmd0OyAtcnct cnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTQgb3V0Ym94PGJyPgogICAgICAg ICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVz dGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDIvODhhZGJkNDktNjJkNi00NWIxLTk5OTItYjA0NDY0 YTA0MTEyL2RvbV9tZDo8YnI+CiAgICAgICAgICAgICZndDsgdG90YWwgMSwxTTxicj4KICAgICAg ICAgICAgJmd0OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjQzIGlk cyDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRl LCBvdGhlciBjYW4gcmVhZCk8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20g a3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE1IGluYm94PGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1y dy0tLS0gMSB2ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDIyLjE0IGxlYXNlczxicj4KICAgICAgICAg ICAgJmd0OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqAzMzMgwqA3LiBsaXMgMjIuMTQgbWV0YWRh dGE8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4g bGlzIDAwLjE1IG91dGJveDxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0 Owovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVAzLzNj MzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQ6PGJyPgogICAgICAgICAg ICAmZ3Q7IHRvdGFsIDEsMU08YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20g a3ZtIMKgIMKgMCAyNC4gw7pubyAwNy40MyBpZHMgwqAgwqAgwqAKICAgICAgICAgICAgwqAmbHQ7 LS0gYmFkIChzYW5sb2NrIGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAg ICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAyMy4gw7pubyAyMi41MSBpbmJv eDxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4gw7pu byAyMy4xMiBsZWFzZXM8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3Zt IMKgOTk4IDI1LiDDum5vIDAwLjM1IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1y dy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAwMC4xNiBvdXRib3g8YnI+CiAgICAgICAg ICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0 ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQz MjcxMjQvZG9tX21kOjxicj4KICAgICAgICAgICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAg ICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuNDQgaWRz IMKgIMKgIMKgCiAgICAgICAgICAgIMKgJmx0Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUs IG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBr dm0gwqAxNk0gwqA3LiBsaXMgMDAuMTcgaW5ib3g8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3 LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMDAuMTggbGVhc2VzPGJyPgogICAgICAgICAg ICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAwMC4xOCBtZXRhZGF0 YTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBs aXMgMDAuMTcgb3V0Ym94PGJyPgogICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7 Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDEvNDJk NzEwYTktYjg0NC00M2RjLWJlNDEtNzcwMDJkMWNkNTUzL2RvbV9tZDo8YnI+CiAgICAgICAgICAg ICZndDsgdG90YWwgMSwxTTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBr dm0gwqAgwqAwIDI0LiDDum5vIDA3LjMyIGlkcyDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDst LSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEg dmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMjIuMTggaW5ib3g8YnI+CiAgICAgICAgICAgICZndDsg LXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMjIuMTggbGVhc2VzPGJyPgogICAg ICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAyMi4xOCBt ZXRhZGF0YTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0g wqA3LiBsaXMgMjIuMTggb3V0Ym94PGJyPgogICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAg ICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTIt UDIvZmY3MWI0N2ItMGY3Mi00NTI4LTliZmUtYzNkYTg4OGU0N2YwL2RvbV9tZDo8YnI+CiAgICAg ICAgICAgICZndDsgdG90YWwgMywwTTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEg dmRzbSBrdm0gMSwwTSDCoDEuIGLFmWUgMjEuMjggaWRzwqDCoCDCoCDCoAogICAgICAgICAgICDC oCZsdDstLSBiYWQgKG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnct LS0tIDEgdmRzbSBrdm0gwqAxNk0gMjUuIMO6bm8gMDAuNDIgaW5ib3jCoDxicj4KICAgICAgICAg ICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyNS4gw7pubyAwMC40NCBsZWFzZXM8 YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk3IDI0LiDDum5v IDAyLjQ2IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2 bSDCoDE2TSAyNS4gw7pubyAwMC40NCBvdXRib3g8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAg ICAgICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6 XzJLVk0xMi1QMy9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRlY2I0YjQvZG9tX21kOjxi cj4KICAgICAgICAgICAgJmd0OyB0b3RhbCAyLDFNPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1y LS1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzQgaWRzwqDCoCDCoCDCoAogICAg ICAgICAgICDCoCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVh ZCk8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIDIzLiDD um5vIDIyLjM1IGluYm94PGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2 bSAyLDBNIDIzLiDDum5vIDIyLjM4IGxlYXNlczxicj4KICAgICAgICAgICAgJmd0OyAtcnctci0t ci0tIDEgdmRzbSBrdm0gMSwxSyAyNC4gw7pubyAxOS4wNyBtZXRhZGF0YTxicj4KICAgICAgICAg ICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gMjMuIMO6bm8gMjIuMjcgb3V0Ym94 PGJyPgogICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2Vu dGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTJfX1A0LzMwMGU5YWM4LTNjMmYtNDcw My05YmIxLTFkZjIxMzBjN2M5Ny9kb21fbWQ6PGJyPgogICAgICAgICAgICAmZ3Q7IHRvdGFsIDMs ME08YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDEsME0gwqAxLiBi xZllIDIxLjI4IGlkc8KgwqAgwqAgwqAKICAgICAgICAgICAgwqAmbHQ7LS0gYmFkIChvdGhlciBj YW4gcmVhZCk8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIMKgMTZN IMKgNi4gbGlzIDIzLjUwIGluYm94wqDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQg KG90aGVyIGNhbiByZWFkKQogICAgICAgICAgICA8ZGl2PiZndDsgLXJ3LXJ3LXItLSAxIHZkc20g a3ZtIDIsME0gwqA2LiBsaXMgMjMuNTEgbGVhc2VzwqDCoAogICAgICAgICAgICAgIMKgIMKgIMKg Jmx0Oy0tIGJhZCAob3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3 LXItLSAxIHZkc20ga3ZtIMKgNzM0IMKgNy4gbGlzIDAyLjEzIG1ldGFkYXRhwqDCoCDCoAogICAg ICAgICAgICAgIMKgIMKgJmx0Oy0tIGJhZCAoZ3JvdXAgY2FuIHdyaXRlLCBvdGhlciBjYW4gcmVh ZCk8YnI+CiAgICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAxNk0gwqA2 LiBsaXMgMTYuNTUgb3V0Ym94wqDCoCDCoCDCoAogICAgICAgICAgICAgIMKgJmx0Oy0tIGJhZCAo b3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAg Jmd0Owovcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyLVA1 LzFjYTU2YjQ1LTcwMWUtNGMyMi05ZjU5LTNhZWJlYTRkODQ3Ny9kb21fbWQ6PGJyPgogICAgICAg ICAgICAgICZndDsgdG90YWwgMSwxTTxicj4KICAgICAgICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0g MSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzUgaWRzwqDCoCDCoCDCoAogICAgICAgICAg ICAgIMKgJmx0Oy0tIGJhZCAob3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDsg LXJ3LXJ3LXItLSAxIHZkc20ga3ZtIMKgMTZNIDI0LiDDum5vIDAxLjA2IGluYm94PGJyPgogICAg ICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gMjQuIMO6bm8gMDIuNDQg bGVhc2VzPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk4 IDI0LiDDum5vIDE5LjA3IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3LXIt LSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDIyLjIwIG91dGJveDxicj4KICAgICAgICAgICAg ICA8YnI+CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIEl0IHNob3VsZCBsb29rIGxp a2UgdGhpczo8YnI+CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIC1ydy1ydy0tLS0u IDEgdmRzbSBrdm0gMS4wTSBNYXIgwqAxIDIzOjM2IGlkczxicj4KICAgICAgICAgICAgICAtcnct cnctLS0tLiAxIHZkc20ga3ZtIDIuME0gTWFyIMKgMSAyMzozNSBsZWFzZXM8YnI+CiAgICAgICAg ICAgICAgLXJ3LXItLXItLS4gMSB2ZHNtIGt2bSDCoDM1MyBNYXIgwqAxIDIzOjM1IG1ldGFkYXRh PGJyPgogICAgICAgICAgICAgIC1ydy1ydy0tLS0uIDEgdmRzbSBrdm0gwqAxNk0gTWFyIMKgMSAy MzozNCBvdXRib3g8YnI+CiAgICAgICAgICAgICAgLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSDCoDE2 TSBNYXIgwqAxIDIzOjM0IGluYm94PGJyPgogICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAg ICBUaGlzIGV4cGxhaW5zIHRoZSBFQUNDRVMgZXJyb3IuPGJyPgogICAgICAgICAgICAgIDxicj4K ICAgICAgICAgICAgICBZb3UgY2FuIHN0YXJ0IGJ5IGZpeGluZyB0aGUgcGVybWlzc2lvbnMgbWFu dWFsbHksIHlvdSBjYW4KICAgICAgICAgICAgICBkbyB0aGlzIG9ubGluZS48YnI+CiAgICAgICAg ICAgICAgwqA8YnI+CiAgICAgICAgICAgICAgJmd0OyDCoFRoZSBpZHMgZmlsZXMgd2FzIGdlbmVy YXRlZCBieSAidG91Y2giIGNvbW1hbmQgYWZ0ZXIKICAgICAgICAgICAgICBkZWxldGluZyB0aGVt IGR1ZSAic2FubG9jayBsb2NraW5nIGhhbmciIMKgZ2x1c3RlciBjcmFzaAogICAgICAgICAgICAg ICZhbXA7IHJlYm9vdDxicj4KICAgICAgICAgICAgICAmZ3Q7IEkgZXhwZWN0ZWQgdGhhdCB0aGV5 IHdpbGwgYmUgZmlsbGVkIGF1dG9tYXRpY2FseQogICAgICAgICAgICAgIGFmdGVyIGdsdXN0ZXIg cmVib290ICggdGhlIMKgc2hhZG93IGNvcHkgZnJvbSDCoCAiLmdsdXN0ZXIgIgogICAgICAgICAg ICAgIMKgIGRpcmVjdG9yeSDCoHdhcyBkZWxldGVkICZhbXA7IGNyZWF0ZWQgZW1wdHkgwqB0b28g KTxicj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgSSBkb24ndCBrbm93IGFib3V0 IGdsdXN0ZXIgc2hhZG93IGNvcHksIEkgd291bGQgbm90IHBsYXkKICAgICAgICAgICAgICB3aXRo IGdsdXN0ZXIgaW50ZXJuYWxzLjwvZGl2PgogICAgICAgICAgICA8ZGl2PkFkZGluZyBTYWhpbmEg Zm9yIGFkdmljZS48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAg ICAgPC9ibG9ja3F1b3RlPgogICAgICAgIDxicj4KICAgICAgICBEaWQgeW91IGdlbmVyYXRlIHRo ZSBpZHMgZmlsZSBvbiB0aGUgbW91bnQgcG9pbnQuPGJyPgogICAgICAgIDxicj4KICAgICAgICBS YXZpLCBjYW4geW91IGhlbHAgaGVyZT88YnI+CiAgICAgICAgPGJyPgogICAgICA8L2Jsb2NrcXVv dGU+CiAgICAgIDxicj4KICAgICAgT2theSwgc28gd2hhdCBJIHVuZGVyc3RhbmQgZnJvbSB0aGUg b3V0cHV0IGFib3ZlIGlzIHlvdSBoYXZlCiAgICAgIGRpZmZlcmVudCBnbHVzdGVyIHZvbHVtZXMg bW91bnRlZCBhbmQgc29tZSBvZiB0aGVtIGhhdmUgaW5jb3JyZWN0CiAgICAgIHBlcm1pc3Npb25z IGZvciB0aGUgJ2lkcycgZmlsZS4gVGhlIHdheSB0byBmaXggaXQgaXMgdG8gZG8gaXQgZnJvbQog ICAgICB0aGUgbW91bnQgbGlrZSBOaXIgc2FpZC48YnI+CiAgICAgIFdoeSBkaWQgeW91IGRlbGV0 ZSB0aGUgZmlsZSBmcm9tIHRoZSAuZ2x1c3RlcmZzIGluIHRoZSBicmljayhzKT/CoAogICAgICBX YXMgdGhlcmUgYSBnZmlkIHNwbGl0IGJyYWluPyA8YnI+CiAgICAgIDxicj4KICAgICAgLVJhdmk8 YnI+CiAgICAgIDxicj4KICAgICAgPGJsb2NrcXVvdGUgY2l0ZT0ibWlkOjU2RDY4OTEwLjgwNDA2 MDJAcmVkaGF0LmNvbSIgdHlwZT0iY2l0ZSI+CiAgICAgICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlk OkNBTVJieXl1OWd3UGZWcFB4cERhNF9nS1d5WHExUGF2VG0yVjJyRzJjVTBBdkU9SkpQQUBtYWls LmdtYWlsLmNvbSIKICAgICAgICAgIHR5cGU9ImNpdGUiPgogICAgICAgICAgPGRpdiBkaXI9Imx0 ciI+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICAgICZndDsgT0ssIGl0IGxvb2tz IHRoYXQgc2FubG9jayDCoGNhbid0IHdvcmsgd2l0aCBlbXB0eSBmaWxlCiAgICAgICAgICAgICAg b3IgcmV3cml0ZSB0aGVtIC48YnI+CiAgICAgICAgICAgICAgJmd0OyBBbSBJIHJpZ2h0ID8/PGJy PgogICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICBZZXMsIHRoZSBmaWxlcyBtdXN0IGJl IGluaXRpYWxpemVkIGJlZm9yZSBzYW5sb2NrIGNhbiB1c2UKICAgICAgICAgICAgICB0aGVtLjxi cj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgWW91IGNhbiBpbml0aWFsaXplIHRo ZSBmaWxlIGxpa2UgdGhpczo8YnI+CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIHNh bmxvY2sgZGlyZWN0IGluaXQgLXMKICAgICAgICAgICAgICAmbHQ7c2RfdXVpZCZndDs6MDpyZXBh aXIvJmx0O3NkX3V1aWQmZ3Q7L2RvbV9tZC9pZHM6MDxicj4KICAgICAgICAgICAgICA8YnI+CiAg ICAgICAgICAgICAgVGFrZW4gZnJvbSA8YSBtb3otZG8tbm90LXNlbmQ9InRydWUiCiAgICAgICAg ICAgICAgICBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL3BpcGVybWFpbC91c2Vycy8yMDE2 LUZlYnJ1YXJ5LzAzODA0Ni5odG1sIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL3BpcGVybWFpbC91 c2Vycy8yMDE2LUZlYnJ1YXJ5LzAzODA0Ni5odG1sPC9hPjxicj4KICAgICAgICAgICAgICA8YnI+ CiAgICAgICAgICAgICAgJmd0OyBUaGUgbGFzdCBwb2ludCAtIGFib3V0ICJpZHMiIHdvcmthcm91 bmQgLSB0aGlzIGlzCiAgICAgICAgICAgICAgb2ZmbGluZSB2ZXJzaW9uID0gVk1zIGhhdmUgdG8g YmUgbW92ZWQgb3V0IGZyb20gZm9yCiAgICAgICAgICAgICAgY29udGludWFsIHJ1bm5pbmcgd2l0 aCBtYWludGVuYW5jZSB2b2x1bWUgbW9kZTxicj4KICAgICAgICAgICAgICAmZ3Q7IEJ1dCB0aGlz IGlzIG5vdCBhY2NlcHRhYmxlIGluIGN1cnJlbnQgc2l0dWF0aW9uLCBzbwogICAgICAgICAgICAg IHRoZSBxdWVzdGlvbiBhZ2FpbiwgwqBpcyBpdCBzYWZlIHRvIGRvIGl0IG9ubGluZSA/PyDCoCgg WUVTCiAgICAgICAgICAgICAgLyBOTyApPC9kaXY+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAg ICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj5UaGUgaWRzIGZpbGUgaXMgYWNjZXNzZWQg b25seSBieSBzYW5sb2NrLiBJIGd1ZXNzIHRoYXQKICAgICAgICAgICAgICB5b3UgZG9uJ3QgaGF2 ZSBhIHJ1bm5pbmc8L2Rpdj4KICAgICAgICAgICAgPGRpdj5TUE0gb24gdGhpcyBEQywgc2luY2Ug c2FubG9jayBmYWlscyB0byBhY3F1aXJlIGEgaG9zdAogICAgICAgICAgICAgIGlkLCBzbyB5b3Ug YXJlIHByZXR0eSBzYWZlPC9kaXY+CiAgICAgICAgICAgIDxkaXY+dG8gZml4IHRoZSBwZXJtaXNz aW9ucyBhbmQgaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGVzLjwvZGl2PgogICAgICAgICAgICA8ZGl2 Pjxicj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+SSB3b3VsZCBkbyB0aGlz OjwvZGl2PgogICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAg ICAgIDxkaXY+MS4gU3RvcCBlbmdpbmUsIMKgc28gaXQgd2lsbCBub3QgdHJ5IHRvIHN0YXJ0IHZk c208L2Rpdj4KICAgICAgICAgICAgPGRpdj4yLiBTdG9wIHZkc20gb24gYWxsIGhvc3RzLCBzbyB0 aGV5IGRvIG5vdCB0cnkgdG8KICAgICAgICAgICAgICBhY3F1aXJlIGEgaG9zdCBpZCB3aXRoIHNh bmxvY2s8L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoCDCoCBUaGlzIGRvZXMgbm90IGFmZmVjdCBy dW5uaW5nIHZtczwvZGl2PgogICAgICAgICAgICA8ZGl2PjMuIEZpeCB0aGUgcGVybWlzc2lvbnMg b24gdGhlIGlkcyBmaWxlLCB2aWEgZ2x1c3RlcmZzCiAgICAgICAgICAgICAgbW91bnQ8L2Rpdj4K ICAgICAgICAgICAgPGRpdj40LiBJbml0aWFsaXplIHRoZSBpZHMgZmlsZXMgZnJvbSBvbmUgb2Yg dGhlIGhvc3RzLCB2aWEKICAgICAgICAgICAgICB0aGUgZ2x1c3RlcmZzIG1vdW50PC9kaXY+CiAg ICAgICAgICAgIDxkaXY+wqAgwqAgVGhpcyBzaG91bGQgZml4IHRoZSBpZHMgZmlsZXMgb24gYWxs IHJlcGxpY2FzPC9kaXY+CiAgICAgICAgICAgIDxkaXY+NS4gU3RhcnQgdmRzbSBvbiBhbGwgaG9z dHM8L2Rpdj4KICAgICAgICAgICAgPGRpdj42LiBTdGFydCBlbmdpbmU8L2Rpdj4KICAgICAgICAg ICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2PkVuZ2luZSB3 aWxsIGNvbm5lY3QgdG8gYWxsIGhvc3RzLCBob3N0cyB3aWxsIGNvbm5lY3QgdG8KICAgICAgICAg ICAgICBzdG9yYWdlIGFuZCB0cnkgdG8gYWNxdWlyZSBhIGhvc3QgaWQuPC9kaXY+CiAgICAgICAg ICAgIDxkaXY+VGhlbiBFbmdpbmUgd2lsbCBzdGFydCB0aGUgU1BNIG9uIG9uZSBvZiB0aGUgaG9z dHMsIGFuZAogICAgICAgICAgICAgIHlvdXIgREMgc2hvdWxkIGJlY29tZSB1cC48L2Rpdj4KICAg ICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2PkRh dmlkLCBTYWhpbmEsIGNhbiB5b3UgY29uZmlybSB0aGF0IHRoaXMgcHJvY2VkdXJlIGlzCiAgICAg ICAgICAgICAgc2FmZT88L2Rpdj4KICAgICAgICAgIDwvZGl2PgogICAgICAgIDwvYmxvY2txdW90 ZT4KICAgICAgICA8YnI+CiAgICAgICAgWWVzLCBjb3JyZWN0aW5nIGZyb20gdGhlIG1vdW50IHBv aW50IHNob3VsZCBmaXggaXQgb24gYWxsCiAgICAgICAgcmVwbGljYXM8YnI+CiAgICAgICAgPGJy PgogICAgICAgIDxicj4KICAgICAgICA8YmxvY2txdW90ZQpjaXRlPSJtaWQ6Q0FNUmJ5eXU5Z3dQ ZlZwUHhwRGE0X2dLV3lYcTFQYXZUbTJWMnJHMmNVMEF2RT1KSlBBQG1haWwuZ21haWwuY29tIgog ICAgICAgICAgdHlwZT0iY2l0ZSI+CiAgICAgICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICAg ICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2Pk5pcjwvZGl2 PgogICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxk aXY+Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IHJlZ3MuPGJyPgogICAgICAgICAgICAgICZn dDsgUGF2ZWw8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7PGJy PgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0OyBPbiAxLjMuMjAxNiAx ODozOCwgTmlyIFNvZmZlciB3cm90ZTo8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAg ICAgICAgICAmZ3Q7IE9uIFR1ZSwgTWFyIDEsIDIwMTYgYXQgNTowNyBQTSwgPGEKICAgICAgICAg ICAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZp YXRlZCIKICAgICAgICAgICAgICAgIGhyZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6Ij5wYWYxQGVt YWlsLmN6PC9hPiAmbHQ7PGEKICAgICAgICAgICAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIg Y2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIKICAgICAgICAgICAgICAgIGhyZWY9Im1h aWx0bzpwYWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiZndDsgd3JvdGU6PGJyPgogICAg ICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IEhlbGxvLCDCoGNh biBhbnlib2R5IMKgZXhwbGFpbiB0aGlzIGVycm9yIG5vLjEzICgKICAgICAgICAgICAgICBvcGVu IGZpbGUgKSBpbiBzYW5sb2NrLmxvZyAuPGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAg ICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IFRoaXMgaXMgRUFDQ0VTPGJyPgog ICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0OyBDYW4geW91IHNoYXJlIHRo ZSBvdXRvcHV0IG9mOjxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZn dDsgwqAgwqAgbHMgLWxoCi9yaGV2L2RhdGEtY2VudGVyL21udC8mbHQ7c2VydmVyJmd0OzombHQ7 X3BhdGgmZ3Q7LyZsdDtzZF91dWlkJmd0Oy9kb21fbWQ8YnI+CiAgICAgICAgICAgICAgJmd0OyDC oDxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxi cj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBUaGUgc2l6ZSBvZiDCoCJpZHMiIGZpbGUgaXMgemVy byAoMCk8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgog ICAgICAgICAgICAgICZndDsgVGhpcyBpcyBob3cgd2UgY3JlYXRlIHRoZSBpZHMgZmlsZSB3aGVu IGluaXRpYWxpemluZwogICAgICAgICAgICAgIGl0Ljxicj4KICAgICAgICAgICAgICAmZ3Q7PGJy PgogICAgICAgICAgICAgICZndDsgQnV0IHRoZW4gd2UgdXNlIHNhbmxvY2sgdG8gaW5pdGlhbGl6 ZSB0aGUgaWRzIGZpbGUsCiAgICAgICAgICAgICAgYW5kIGl0IHNob3VsZCBiZSAxTWlCIGFmdGVy IHRoYXQuPGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0OyBJcyB0 aGlzIGlkcyBmaWxlcyBjcmVhdGVkIGJ5IHZkc20sIG9yIG9uZSB5b3UgY3JlYXRlZAogICAgICAg ICAgICAgIHlvdXJzZWxmPzxicj4KICAgICAgICAgICAgICAmZ3Q7IMKgPGJyPgogICAgICAgICAg ICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IDIwMTYtMDItMjggMDM6MjU6 NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBvcGVuCiAgICAgICAgICAgICAgZXJyb3IgLTEzCi9yaGV2 L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTct YzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM8YnI+CiAgICAgICAgICAgICAg Jmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2OTYyNiBbMTk1MV06IHMxODc5ODUK ICAgICAgICAgICAgICBvcGVuX2Rpc2sKICAgICAgICAgICAgICAvcmhldi9kYXRhLWNlbnRlci9t bnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0LzdmNTJiNjk3LWMxOTktNGY1OC04OWFh LTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzCgoKICAgICAgICAgICAgICBlcnJvciAtMTM8YnI+CiAg ICAgICAgICAgICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo1NiswMTAwIDI2OTYzNiBbMTEz MDRdOiBzMTg3OTkyCiAgICAgICAgICAgICAgbG9ja3NwYWNlCjdmNTJiNjk3LWMxOTktNGY1OC04 OWFhLTEwMmQ0NDMyNzEyNDoxOi9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxo b3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9t ZC9pZHM6MDxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7 Jmd0OyBJZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHplcm8gZmlsZSBzaXplLCBjYW4KICAg ICAgICAgICAgICBJIHJlZ2VuZXJhdGUgwqB0aGlzIGZpbGUgb25saW5lIHNlY3VyZWx5ICwgd2l0 aCBubyBWTQogICAgICAgICAgICAgIGRlcGVuZGVuY2UgwqA/Pz8/PGJyPgogICAgICAgICAgICAg ICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IFllcywg SSB0aGluayBJIGFscmVhZHkgcmVmZXJyZWQgdG8gdGhlIGluc3RydWN0aW9ucwogICAgICAgICAg ICAgIGhvdyB0byBkbyB0aGF0IGluIGEgcHJldmlvdXMgbWFpbC48YnI+CiAgICAgICAgICAgICAg Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0 Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBkaXN0ID0gUkhFTCAtIDcgLSAyLjE1MTE8YnI+ CiAgICAgICAgICAgICAgJmd0OyZndDsga2VybmVsID0gMy4xMC4wIC0gMzI3LjEwLjEuZWw3Lng4 Nl82NDxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBLVk0gPSAyLjMuMCAtIDI5LjEuZWw3PGJy PgogICAgICAgICAgICAgICZndDsmZ3Q7IGxpYnZpcnQgPSBsaWJ2aXJ0LTEuMi4xNy0xMy5lbDdf Mi4zPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IHZkc20gPSB2ZHNtLTQuMTYuMzAtMC5lbDc8 YnI+CiAgICAgICAgICAgICAgJmd0OyZndDsgR2x1c3RlckZTID0gZ2x1c3RlcmZzLTMuNy44LTEu ZWw3PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7 PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IHJlZ3MuPGJyPgogICAgICAgICAgICAgICZndDsm Z3Q7IFBhdmVsPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZn dDsmZ3Q7IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPGJy PgogICAgICAgICAgICAgICZndDsmZ3Q7IFVzZXJzIG1haWxpbmcgbGlzdDxicj4KICAgICAgICAg ICAgICAmZ3Q7Jmd0OyA8YSBtb3otZG8tbm90LXNlbmQ9InRydWUiCiAgICAgICAgICAgICAgICBo cmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIj5Vc2Vyc0BvdmlydC5vcmc8L2E+PGJyPgogICAg ICAgICAgICAgICZndDsmZ3Q7IDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAgICAgICAg ICAgIGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyI+ aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzPC9hPjxicj4KICAg ICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAg ICAgICZndDs8YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAgICAg PC9ibG9ja3F1b3RlPgogICAgICAgIDxicj4KICAgICAgICA8YnI+CiAgICAgICAgPGZpZWxkc2V0 IGNsYXNzPSJtaW1lQXR0YWNobWVudEhlYWRlciI+PC9maWVsZHNldD4KICAgICAgICA8YnI+CiAg ICAgICAgPHByZSB3cmFwPSIiPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fCkdsdXN0ZXItdXNlcnMgbWFpbGluZyBsaXN0CjxhIG1vei1kby1ub3Qtc2VuZD0i dHJ1ZSIgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIgaHJlZj0ibWFpbHRvOkdsdXN0 ZXItdXNlcnNAZ2x1c3Rlci5vcmciPkdsdXN0ZXItdXNlcnNAZ2x1c3Rlci5vcmc8L2E+CjxhIG1v ei1kby1ub3Qtc2VuZD0idHJ1ZSIgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIgaHJlZj0i aHR0cDovL3d3dy5nbHVzdGVyLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2dsdXN0ZXItdXNlcnMiPmh0 dHA6Ly93d3cuZ2x1c3Rlci5vcmcvbWFpbG1hbi9saXN0aW5mby9nbHVzdGVyLXVzZXJzPC9hPjwv cHJlPgogICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgIDxicj4KICAgICAgPGJyPgogICAgPC9ibG9j a3F1b3RlPgogICAgPGJyPgogIDwvYm9keT4KPC9odG1sPgoKLS0tLS0tLS0tLS0tLS0wMDA3MDAw MDA1MDgwODA2MDEwMzAxMDgtLQo= --===============4162557774279759291==-- From nsoffer at redhat.com Wed Mar 2 14:13:51 2016 Content-Type: multipart/mixed; boundary="===============7415982641314222456==" MIME-Version: 1.0 From: Nir Soffer To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Wed, 02 Mar 2016 21:13:49 +0200 Message-ID: In-Reply-To: 56D72767.7020004@email.cz --===============7415982641314222456== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Wed, Mar 2, 2016 at 7:48 PM, paf1(a)email.cz wrote: > UPDATE: > > all "ids" file have permittion fixed to 660 now > > # find /STORAGES -name ids -exec ls -l {} \; > -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.41 > /STORAGES/g1r5p1/GFS/553d9b92-e4a0-4042-a579-4cabeb55ded4/dom_md/ids > -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.43 > /STORAGES/g1r5p2/GFS/88adbd49-62d6-45b1-9992-b04464a04112/dom_md/ids > -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.43 > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids > -rw-rw---- 2 vdsm kvm 0 24. =C3=BAno 07.44 > /STORAGES/g1r5p4/GFS/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids > -rw-rw---- 2 vdsm kvm 1048576 24. =C3=BAno 13.03 > /STORAGES/g1r5p5/GFS/3b24d023-fd35-4666-af2f-f5e1d19531ad/dom_md/ids > -rw-rw---- 2 vdsm kvm 1048576 2. b=C5=99e 17.47 > /STORAGES/g2r5p1/GFS/0fcad888-d573-47be-bef3-0bc0b7a99fb7/dom_md/ids > > SPM is and was running continually ....... > You must stop vdsm on all hosts, please follow the instructions in the previous mail. > > I tried to update "ids" file - ONLINE ( offline not possible yet ) > # sanlock direct init -s > 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-6c66= -4e23-ab46-084f3d70b147/dom_md/ids:0 > > # find /STORAGES -name ids -exec ls -l {} \; | grep p3 > -rw-rw---- 1 vdsm kvm 1048576 2. b=C5=99e 18.32 > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids > > The storage ids file has correct permittions, size, owners , but is not > checking by sanlock =3D the same access time > What's wrong ?? > sanlock will access the files when vdsm when vdsm will start the domain monitors when connecting to the pool. > regs. > Pa. > PS: # find /STORAGES -samefile > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids -pri= nt > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids > =3D missing "shadowfile" in " .gluster " dir. > How can I fix it ?? - online ! > Ravi? > > > > > On 2.3.2016 08:16, Ravishankar N wrote: > > On 03/02/2016 12:02 PM, Sahina Bose wrote: > > > > On 03/02/2016 03:45 AM, Nir Soffer wrote: > > On Tue, Mar 1, 2016 at 10:51 PM, paf1(a)email.cz wrote: > > > > HI, > > requested output: > > > > # ls -lh /rhev/data-center/mnt/glusterSD/localhost:*/*/dom_md > > > > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-BCK/0fcad888-d573-47be-= bef3-0bc0b7a99fb7/dom_md: > > total 2,1M > > -rw-rw---- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- good > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.17 leases > > -rw-r--r-- 1 vdsm kvm 335 7. lis 22.17 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.16 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P1/553d9b92-e4a0-4042-a= 579-4cabeb55ded4/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.41 ids <-- bad (sanlo= ck > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 03.56 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 03.56 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.14 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P2/88adbd49-62d6-45b1-9= 992-b04464a04112/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlo= ck > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.14 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.14 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.15 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P3/3c34ad63-6c66-4e23-a= b46-084f3d70b147/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.43 ids <-- bad (sanlo= ck > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.51 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 23.12 leases > > -rw-r--r-- 1 vdsm kvm 998 25. =C3=BAno 00.35 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.16 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md: > > total 1,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.44 ids <-- bad (sanlo= ck > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 00.18 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 00.18 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 00.17 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P1/42d710a9-b844-43dc-b= e41-77002d1cd553/dom_md: > > total 1,1M > > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.32 ids <-- bad (other= can > read) > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 7. lis 22.18 leases > > -rw-r--r-- 1 vdsm kvm 333 7. lis 22.18 metadata > > -rw-rw---- 1 vdsm kvm 16M 7. lis 22.18 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P2/ff71b47b-0f72-4528-9= bfe-c3da888e47f0/dom_md: > > total 3,0M > > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other= can > read) > > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.42 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 25. =C3=BAno 00.44 leases > > -rw-r--r-- 1 vdsm kvm 997 24. =C3=BAno 02.46 metadata > > -rw-rw---- 1 vdsm kvm 16M 25. =C3=BAno 00.44 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P3/ef010d08-aed1-41c4-b= a9a-e6d9bdecb4b4/dom_md: > > total 2,1M > > -rw-r--r-- 1 vdsm kvm 0 24. =C3=BAno 07.34 ids <-- bad (sanlo= ck > cannot write, other can read) > > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.35 inbox > > -rw-rw---- 1 vdsm kvm 2,0M 23. =C3=BAno 22.38 leases > > -rw-r--r-- 1 vdsm kvm 1,1K 24. =C3=BAno 19.07 metadata > > -rw-rw---- 1 vdsm kvm 16M 23. =C3=BAno 22.27 outbox > > > > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12__P4/300e9ac8-3c2f-4703-= 9bb1-1df2130c7c97/dom_md: > > total 3,0M > > -rw-rw-r-- 1 vdsm kvm 1,0M 1. b=C5=99e 21.28 ids <-- bad (other= can > read) > > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 23.50 inbox <-- bad (other can > read) > > -rw-rw-r-- 1 vdsm kvm 2,0M 6. lis 23.51 leases <-- bad (other > can read) > > -rw-rw-r-- 1 vdsm kvm 734 7. lis 02.13 metadata <-- bad (group > can write, other can read) > > -rw-rw-r-- 1 vdsm kvm 16M 6. lis 16.55 outbox <-- bad (other > can read) > > > > > /rhev/data-center/mnt/glusterSD/localhost:_2KVM12-P5/1ca56b45-701e-4c22-9= f59-3aebea4d8477/dom_md: > > total 1,1M > > -rw-rw-r-- 1 vdsm kvm 0 24. =C3=BAno 07.35 ids <-- bad (other= can > read) > > -rw-rw-r-- 1 vdsm kvm 16M 24. =C3=BAno 01.06 inbox > > -rw-rw-r-- 1 vdsm kvm 2,0M 24. =C3=BAno 02.44 leases > > -rw-r--r-- 1 vdsm kvm 998 24. =C3=BAno 19.07 metadata > > -rw-rw-r-- 1 vdsm kvm 16M 7. lis 22.20 outbox > > > It should look like this: > > -rw-rw----. 1 vdsm kvm 1.0M Mar 1 23:36 ids > -rw-rw----. 1 vdsm kvm 2.0M Mar 1 23:35 leases > -rw-r--r--. 1 vdsm kvm 353 Mar 1 23:35 metadata > -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 outbox > -rw-rw----. 1 vdsm kvm 16M Mar 1 23:34 inbox > > This explains the EACCES error. > > You can start by fixing the permissions manually, you can do this online. > > > The ids files was generated by "touch" command after deleting them due > "sanlock locking hang" gluster crash & reboot > > I expected that they will be filled automaticaly after gluster reboot ( > the shadow copy from ".gluster " directory was deleted & created > empty too ) > > I don't know about gluster shadow copy, I would not play with gluster > internals. > Adding Sahina for advice. > > > Did you generate the ids file on the mount point. > > Ravi, can you help here? > > > Okay, so what I understand from the output above is you have different > gluster volumes mounted and some of them have incorrect permissions for t= he > 'ids' file. The way to fix it is to do it from the mount like Nir said. > Why did you delete the file from the .glusterfs in the brick(s)? Was > there a gfid split brain? > > -Ravi > > > > OK, it looks that sanlock can't work with empty file or rewrite them . > > Am I right ?? > > Yes, the files must be initialized before sanlock can use them. > > You can initialize the file like this: > > sanlock direct init -s :0:repair//dom_md/ids:0 > > Taken from > http://lists.ovirt.org/pipermail/users/2016-February/038046.html > > > The last point - about "ids" workaround - this is offline version =3D V= Ms > have to be moved out from for continual running with maintenance volume m= ode > > But this is not acceptable in current situation, so the question again, > is it safe to do it online ?? ( YES / NO ) > > The ids file is accessed only by sanlock. I guess that you don't have a > running > SPM on this DC, since sanlock fails to acquire a host id, so you are > pretty safe > to fix the permissions and initialize the ids files. > > I would do this: > > 1. Stop engine, so it will not try to start vdsm > 2. Stop vdsm on all hosts, so they do not try to acquire a host id with > sanlock > This does not affect running vms > 3. Fix the permissions on the ids file, via glusterfs mount > 4. Initialize the ids files from one of the hosts, via the glusterfs mount > This should fix the ids files on all replicas > 5. Start vdsm on all hosts > 6. Start engine > > Engine will connect to all hosts, hosts will connect to storage and try to > acquire a host id. > Then Engine will start the SPM on one of the hosts, and your DC should > become up. > > David, Sahina, can you confirm that this procedure is safe? > > > Yes, correcting from the mount point should fix it on all replicas > > > > Nir > > > > > regs. > > Pavel > > > > > > > > On 1.3.2016 18:38, Nir Soffer wrote: > > > > On Tue, Mar 1, 2016 at 5:07 PM, paf1(a)email.cz wrote: > >> > >> Hello, can anybody explain this error no.13 ( open file ) in > sanlock.log . > > > > > > This is EACCES > > > > Can you share the outoput of: > > > > ls -lh /rhev/data-center/mnt/:<_path>//dom_md > > > >> > >> > >> The size of "ids" file is zero (0) > > > > > > This is how we create the ids file when initializing it. > > > > But then we use sanlock to initialize the ids file, and it should be > 1MiB after that. > > > > Is this ids files created by vdsm, or one you created yourself? > > > >> > >> 2016-02-28 03:25:46+0100 269626 [1951]: open error -13 > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids > >> 2016-02-28 03:25:46+0100 269626 [1951]: s187985 open_disk > /rhev/data-center/mnt/glusterSD/localhost:_1KVM12-P4/7f52b697-c199-4f58-8= 9aa-102d44327124/dom_md/ids > error -13 > >> 2016-02-28 03:25:56+0100 269636 [11304]: s187992 lockspace > 7f52b697-c199-4f58-89aa-102d44327124:1:/rhev/data-center/mnt/glusterSD/lo= calhost:_1KVM12-P4/7f52b697-c199-4f58-89aa-102d44327124/dom_md/ids:0 > >> > >> If the main problem is about zero file size, can I regenerate this > file online securely , with no VM dependence ???? > > > > > > Yes, I think I already referred to the instructions how to do that in a > previous mail. > > > >> > >> > >> dist =3D RHEL - 7 - 2.1511 > >> kernel =3D 3.10.0 - 327.10.1.el7.x86_64 > >> KVM =3D 2.3.0 - 29.1.el7 > >> libvirt =3D libvirt-1.2.17-13.el7_2.3 > >> vdsm =3D vdsm-4.16.30-0.el7 > >> GlusterFS =3D glusterfs-3.7.8-1.el7 > >> > >> > >> regs. > >> Pavel > >> > >> _______________________________________________ > >> Users mailing list > >> Users(a)ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/users > >> > > > > > > > > > _______________________________________________ > Gluster-users mailing listGluster-users(a)gluster.orghttp://www.gluster.o= rg/mailman/listinfo/gluster-users > > > > > --===============7415982641314222456== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+PGRpdiBjbGFzcz0iZ21haWxfZXh0cmEiPjxkaXYgY2xhc3M9ImdtYWls X3F1b3RlIj5PbiBXZWQsIE1hciAyLCAyMDE2IGF0IDc6NDggUE0sIDxhIGhyZWY9Im1haWx0bzpw YWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPiA8c3BhbiBkaXI9Imx0ciI+Jmx0OzxhIGhy ZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6IiB0YXJnZXQ9Il9ibGFuayI+cGFmMUBlbWFpbC5jejwv YT4mZ3Q7PC9zcGFuPiB3cm90ZTo8YnI+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBz dHlsZT0ibWFyZ2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGlu Zy1sZWZ0OjFleCI+CiAgCiAgICAKICAKICA8ZGl2IHRleHQ9IiMwMDAwNjYiIGJnY29sb3I9IiNG RkZGRkYiPgogICAgVVBEQVRFOjxicj4KICAgIDxicj4KICAgIGFsbCAmcXVvdDtpZHMmcXVvdDvC oCBmaWxlIGhhdmUgcGVybWl0dGlvbiBmaXhlZCB0byA2NjAgbm93PGJyPgogICAgPGJyPgogICAg I8KgIGZpbmQgL1NUT1JBR0VTIC1uYW1lIGlkcyAtZXhlYyBscyAtbCB7fSBcOzxicj4KICAgIC1y dy1ydy0tLS0gMiB2ZHNtIGt2bSAwIDI0LsKgw7pubyAwNy40MQogICAgL1NUT1JBR0VTL2cxcjVw MS9HRlMvNTUzZDliOTItZTRhMC00MDQyLWE1NzktNGNhYmViNTVkZWQ0L2RvbV9tZC9pZHM8YnI+ CiAgICAtcnctcnctLS0tIDIgdmRzbSBrdm0gMCAyNC7CoMO6bm8gMDcuNDMKICAgIC9TVE9SQUdF Uy9nMXI1cDIvR0ZTLzg4YWRiZDQ5LTYyZDYtNDViMS05OTkyLWIwNDQ2NGEwNDExMi9kb21fbWQv aWRzPGJyPgogICAgLXJ3LXJ3LS0tLSAyIHZkc20ga3ZtIDAgMjQuwqDDum5vIDA3LjQzCiAgICAv U1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcv ZG9tX21kL2lkczxicj4KICAgIC1ydy1ydy0tLS0gMiB2ZHNtIGt2bSAwIDI0LsKgw7pubyAwNy40 NAogICAgL1NUT1JBR0VTL2cxcjVwNC9HRlMvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0 MzI3MTI0L2RvbV9tZC9pZHM8YnI+CiAgICAtcnctcnctLS0tIDIgdmRzbSBrdm0gMTA0ODU3NiAy NC7CoMO6bm8gMTMuMDMKICAgIC9TVE9SQUdFUy9nMXI1cDUvR0ZTLzNiMjRkMDIzLWZkMzUtNDY2 Ni1hZjJmLWY1ZTFkMTk1MzFhZC9kb21fbWQvaWRzPGJyPgogICAgLXJ3LXJ3LS0tLSAyIHZkc20g a3ZtIDEwNDg1NzbCoCAyLsKgYsWZZSAxNy40NwogICAgL1NUT1JBR0VTL2cycjVwMS9HRlMvMGZj YWQ4ODgtZDU3My00N2JlLWJlZjMtMGJjMGI3YTk5ZmI3L2RvbV9tZC9pZHM8YnI+CiAgICA8YnI+ CiAgICBTUE0gaXMgYW5kwqAgd2FzwqAgcnVubmluZyBjb250aW51YWxseSAuLi4uLi4uPGJyPjwv ZGl2PjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PllvdSBtdXN0IHN0b3AgdmRzbSBv biBhbGwgaG9zdHMsIHBsZWFzZSBmb2xsb3cgdGhlIGluc3RydWN0aW9ucyBpbiB0aGUgcHJldmlv dXMgbWFpbC48L2Rpdj48ZGl2PsKgPC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3Rl IiBzdHlsZT0ibWFyZ2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFk ZGluZy1sZWZ0OjFleCI+PGRpdiB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAg IDxicj4KICAgIEkgdHJpZWQgdG8gdXBkYXRlICZxdW90O2lkcyZxdW90OyBmaWxlIC0gT05MSU5F wqAgKCBvZmZsaW5lIG5vdCBwb3NzaWJsZSB5ZXQgKTxicj4KICAgICMgc2FubG9jayBkaXJlY3Qg aW5pdCAtcwozYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDc6MDovU1RPUkFHRVMv ZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lk czowPGJyPgogICAgPGJyPgogICAgI8KgIGZpbmQgL1NUT1JBR0VTIC1uYW1lIGlkcyAtZXhlYyBs cyAtbCB7fSBcOyB8IGdyZXAgcDM8YnI+CiAgICAtcnctcnctLS0tIDEgdmRzbSBrdm0gMTA0ODU3 NsKgIDIuwqBixZllIDE4LjMyCiAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2 LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczxicj4KICAgIDxicj4KICAgIFRoZSBz dG9yYWdlIGlkcyBmaWxlIGhhcyBjb3JyZWN0IHBlcm1pdHRpb25zLCBzaXplLCBvd25lcnMgLCBi dXQgaXMKICAgIG5vdCBjaGVja2luZyBieSBzYW5sb2NrID0gdGhlIHNhbWUgYWNjZXNzIHRpbWUg PGJyPgogICAgV2hhdCYjMzk7cyB3cm9uZyA/PzwvZGl2PjwvYmxvY2txdW90ZT48ZGl2Pjxicj48 L2Rpdj48ZGl2PnNhbmxvY2sgd2lsbCBhY2Nlc3MgdGhlIGZpbGVzIHdoZW4gdmRzbSB3aGVuIHZk c20gd2lsbCBzdGFydCB0aGUgZG9tYWluIG1vbml0b3JzwqA8L2Rpdj48ZGl2PndoZW4gY29ubmVj dGluZyB0byB0aGUgcG9vbC48L2Rpdj48ZGl2PsKgPC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9Imdt YWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mg c29saWQ7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZG RkZGIj4KICAgIHJlZ3MuPGJyPgogICAgUGEuPGJyPgogICAgUFM6wqAgIyBmaW5kIC9TVE9SQUdF UyAtc2FtZWZpbGUKICAgIC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1h YjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCiAgICAtcHJpbnQ8YnI+CiAgICAvU1RPUkFHRVMv ZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lk czxicj4KICAgID0gbWlzc2luZyAmcXVvdDtzaGFkb3dmaWxlJnF1b3Q7IGluICZxdW90OyAuZ2x1 c3RlciAmcXVvdDsgZGlyLjxicj4KICAgIEhvdyBjYW4gSSBmaXggaXQgPz8gLSBvbmxpbmUgITwv ZGl2PjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PlJhdmk/PC9kaXY+PGRpdj7CoDwv ZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAgMCAu OGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPjxkaXYgdGV4 dD0iIzAwMDA2NiIgYmdjb2xvcj0iI0ZGRkZGRiI+PHNwYW4gY2xhc3M9IiI+PGJyPgogICAgPGJy PgogICAgPGJyPgogICAgPGJyPgogICAgPGRpdj5PbiAyLjMuMjAxNiAwODoxNiwgUmF2aXNoYW5r YXIgTiB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAgIDwvc3Bhbj48ZGl2PjxkaXYgY2xhc3M9Img1 Ij48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4KICAgICAgCiAgICAgIDxkaXY+T24gMDMvMDIvMjAx NiAxMjowMiBQTSwgU2FoaW5hIEJvc2UKICAgICAgICB3cm90ZTo8YnI+CiAgICAgIDwvZGl2Pgog ICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4KICAgICAgICAKICAgICAgICA8YnI+CiAgICAg ICAgPGJyPgogICAgICAgIDxkaXY+T24gMDMvMDIvMjAxNiAwMzo0NSBBTSwgTmlyIFNvZmZlcgog ICAgICAgICAgd3JvdGU6PGJyPgogICAgICAgIDwvZGl2PgogICAgICAgIDxibG9ja3F1b3RlIHR5 cGU9ImNpdGUiPgogICAgICAgICAgPGRpdiBkaXI9Imx0ciI+T24gVHVlLCBNYXIgMSwgMjAxNiBh dCAxMDo1MSBQTSwgPGEgaHJlZj0ibWFpbHRvOnBhZjFAZW1haWwuY3oiIHRhcmdldD0iX2JsYW5r Ij5wYWYxQGVtYWlsLmN6PC9hPiAmbHQ7PGEgaHJlZj0ibWFpbHRvOnBhZjFAZW1haWwuY3oiIHRh cmdldD0iX2JsYW5rIj5wYWYxQGVtYWlsLmN6PC9hPiZndDsgd3JvdGU6PGJyPgogICAgICAgICAg ICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7IEhJLDxicj4KICAgICAgICAgICAgJmd0OyByZXF1 ZXN0ZWQgb3V0cHV0Ojxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0OyAj IGxzIC1saAogICAgICAgICAgICAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2Fs aG9zdDoqLyovZG9tX21kPGJyPgogICAgICAgICAgICAmZ3Q7IMKgPGJyPgogICAgICAgICAgICAm Z3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItQkNL LzBmY2FkODg4LWQ1NzMtNDdiZS1iZWYzLTBiYzBiN2E5OWZiNy9kb21fbWQ6PGJyPgogICAgICAg ICAgICAmZ3Q7IHRvdGFsIDIsMU08YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZk c20ga3ZtIDEsME0gwqAxLiBixZllIDIxLjI4IGlkcyDCoCDCoCDCoAogICAgICAgICAgICDCoCZs dDstLSBnb29kPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2 TSDCoDcuIGxpcyAyMi4xNiBpbmJveDxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEg dmRzbSBrdm0gMiwwTSDCoDcuIGxpcyAyMi4xNyBsZWFzZXM8YnI+CiAgICAgICAgICAgICZndDsg LXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgMzM1IMKgNy4gbGlzIDIyLjE3IG1ldGFkYXRhPGJyPgog ICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAyMi4x NiBvdXRib3g8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICZndDsKL3JoZXYv ZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzFLVk0xMi1QMS81NTNkOWI5Mi1l NGEwLTQwNDItYTU3OS00Y2FiZWI1NWRlZDQvZG9tX21kOjxicj4KICAgICAgICAgICAgJmd0OyB0 b3RhbCAxLDFNPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoCDC oDAgMjQuIMO6bm8gMDcuNDEgaWRzIMKgIMKgIMKgCiAgICAgICAgICAgIMKgJmx0Oy0tIGJhZCAo c2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0 OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTQgaW5ib3g8YnI+CiAg ICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMDMuNTYg bGVhc2VzPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDC oDcuIGxpcyAwMy41NiBtZXRhZGF0YTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEg dmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTQgb3V0Ym94PGJyPgogICAgICAgICAgICAmZ3Q7 PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9j YWxob3N0Ol8xS1ZNMTItUDIvODhhZGJkNDktNjJkNi00NWIxLTk5OTItYjA0NDY0YTA0MTEyL2Rv bV9tZDo8YnI+CiAgICAgICAgICAgICZndDsgdG90YWwgMSwxTTxicj4KICAgICAgICAgICAgJmd0 OyAtcnctci0tci0tIDEgdmRzbSBrdm0gwqAgwqAwIDI0LiDDum5vIDA3LjQzIGlkcyDCoCDCoCDC oAogICAgICAgICAgICDCoCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBj YW4gcmVhZCk8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZN IMKgNy4gbGlzIDAwLjE1IGluYm94PGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2 ZHNtIGt2bSAyLDBNIMKgNy4gbGlzIDIyLjE0IGxlYXNlczxicj4KICAgICAgICAgICAgJmd0OyAt cnctci0tci0tIDEgdmRzbSBrdm0gwqAzMzMgwqA3LiBsaXMgMjIuMTQgbWV0YWRhdGE8YnI+CiAg ICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIMKgNy4gbGlzIDAwLjE1 IG91dGJveDxicj4KICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgJmd0Owovcmhldi9k YXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVAzLzNjMzRhZDYzLTZj NjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQ6PGJyPgogICAgICAgICAgICAmZ3Q7IHRv dGFsIDEsMU08YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgIMKg MCAyNC4gw7pubyAwNy40MyBpZHMgwqAgwqAgwqAKICAgICAgICAgICAgwqAmbHQ7LS0gYmFkIChz YW5sb2NrIGNhbm5vdCB3cml0ZSwgb3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAmZ3Q7 IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAyMy4gw7pubyAyMi41MSBpbmJveDxicj4KICAg ICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyMy4gw7pubyAyMy4xMiBs ZWFzZXM8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk4IDI1 LiDDum5vIDAwLjM1IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2 ZHNtIGt2bSDCoDE2TSDCoDcuIGxpcyAwMC4xNiBvdXRib3g8YnI+CiAgICAgICAgICAgICZndDs8 YnI+CiAgICAgICAgICAgICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2Nh bGhvc3Q6XzFLVk0xMi1QNC83ZjUyYjY5Ny1jMTk5LTRmNTgtODlhYS0xMDJkNDQzMjcxMjQvZG9t X21kOjxicj4KICAgICAgICAgICAgJmd0OyB0b3RhbCAxLDFNPGJyPgogICAgICAgICAgICAmZ3Q7 IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuNDQgaWRzIMKgIMKgIMKg CiAgICAgICAgICAgIMKgJmx0Oy0tIGJhZCAoc2FubG9jayBjYW5ub3Qgd3JpdGUsIG90aGVyIGNh biByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0g wqA3LiBsaXMgMDAuMTcgaW5ib3g8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZk c20ga3ZtIDIsME0gwqA3LiBsaXMgMDAuMTggbGVhc2VzPGJyPgogICAgICAgICAgICAmZ3Q7IC1y dy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAwMC4xOCBtZXRhZGF0YTxicj4KICAg ICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMgMDAuMTcg b3V0Ym94PGJyPgogICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2L2Rh dGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDEvNDJkNzEwYTktYjg0 NC00M2RjLWJlNDEtNzcwMDJkMWNkNTUzL2RvbV9tZDo8YnI+CiAgICAgICAgICAgICZndDsgdG90 YWwgMSwxTTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAgwqAw IDI0LiDDum5vIDA3LjMyIGlkcyDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQgKG90 aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0g wqAxNk0gwqA3LiBsaXMgMjIuMTggaW5ib3g8YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LS0t LSAxIHZkc20ga3ZtIDIsME0gwqA3LiBsaXMgMjIuMTggbGVhc2VzPGJyPgogICAgICAgICAgICAm Z3Q7IC1ydy1yLS1yLS0gMSB2ZHNtIGt2bSDCoDMzMyDCoDcuIGxpcyAyMi4xOCBtZXRhZGF0YTxi cj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gwqA3LiBsaXMg MjIuMTggb3V0Ym94PGJyPgogICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9y aGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTItUDIvZmY3MWI0 N2ItMGY3Mi00NTI4LTliZmUtYzNkYTg4OGU0N2YwL2RvbV9tZDo8YnI+CiAgICAgICAgICAgICZn dDsgdG90YWwgMywwTTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0g MSwwTSDCoDEuIGLFmWUgMjEuMjggaWRzwqDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBi YWQgKG90aGVyIGNhbiByZWFkKTxicj4KICAgICAgICAgICAgJmd0OyAtcnctcnctLS0tIDEgdmRz bSBrdm0gwqAxNk0gMjUuIMO6bm8gMDAuNDIgaW5ib3jCoDxicj4KICAgICAgICAgICAgJmd0OyAt cnctcnctLS0tIDEgdmRzbSBrdm0gMiwwTSAyNS4gw7pubyAwMC40NCBsZWFzZXM8YnI+CiAgICAg ICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk3IDI0LiDDum5vIDAyLjQ2IG1l dGFkYXRhPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSDCoDE2TSAy NS4gw7pubyAwMC40NCBvdXRib3g8YnI+CiAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAg ICZndDsKL3JoZXYvZGF0YS1jZW50ZXIvbW50L2dsdXN0ZXJTRC9sb2NhbGhvc3Q6XzJLVk0xMi1Q My9lZjAxMGQwOC1hZWQxLTQxYzQtYmE5YS1lNmQ5YmRlY2I0YjQvZG9tX21kOjxicj4KICAgICAg ICAgICAgJmd0OyB0b3RhbCAyLDFNPGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1yLS1yLS0gMSB2 ZHNtIGt2bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzQgaWRzwqDCoCDCoCDCoAogICAgICAgICAgICDC oCZsdDstLSBiYWQgKHNhbmxvY2sgY2Fubm90IHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+CiAg ICAgICAgICAgICZndDsgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIMKgMTZNIDIzLiDDum5vIDIyLjM1 IGluYm94PGJyPgogICAgICAgICAgICAmZ3Q7IC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAyLDBNIDIz LiDDum5vIDIyLjM4IGxlYXNlczxicj4KICAgICAgICAgICAgJmd0OyAtcnctci0tci0tIDEgdmRz bSBrdm0gMSwxSyAyNC4gw7pubyAxOS4wNyBtZXRhZGF0YTxicj4KICAgICAgICAgICAgJmd0OyAt cnctcnctLS0tIDEgdmRzbSBrdm0gwqAxNk0gMjMuIMO6bm8gMjIuMjcgb3V0Ym94PGJyPgogICAg ICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAmZ3Q7Ci9yaGV2L2RhdGEtY2VudGVyL21udC9n bHVzdGVyU0QvbG9jYWxob3N0Ol8yS1ZNMTJfX1A0LzMwMGU5YWM4LTNjMmYtNDcwMy05YmIxLTFk ZjIxMzBjN2M5Ny9kb21fbWQ6PGJyPgogICAgICAgICAgICAmZ3Q7IHRvdGFsIDMsME08YnI+CiAg ICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDEsME0gwqAxLiBixZllIDIxLjI4 IGlkc8KgwqAgwqAgwqAKICAgICAgICAgICAgwqAmbHQ7LS0gYmFkIChvdGhlciBjYW4gcmVhZCk8 YnI+CiAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIMKgMTZNIMKgNi4gbGlz IDIzLjUwIGluYm94wqDCoCDCoCDCoAogICAgICAgICAgICDCoCZsdDstLSBiYWQgKG90aGVyIGNh biByZWFkKQogICAgICAgICAgICA8ZGl2PiZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0g wqA2LiBsaXMgMjMuNTEgbGVhc2VzwqDCoAogICAgICAgICAgICAgIMKgIMKgIMKgJmx0Oy0tIGJh ZCAob3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZk c20ga3ZtIMKgNzM0IMKgNy4gbGlzIDAyLjEzIG1ldGFkYXRhwqDCoCDCoAogICAgICAgICAgICAg IMKgIMKgJmx0Oy0tIGJhZCAoZ3JvdXAgY2FuIHdyaXRlLCBvdGhlciBjYW4gcmVhZCk8YnI+CiAg ICAgICAgICAgICAgJmd0OyAtcnctcnctci0tIDEgdmRzbSBrdm0gwqAxNk0gwqA2LiBsaXMgMTYu NTUgb3V0Ym94wqDCoCDCoCDCoAogICAgICAgICAgICAgIMKgJmx0Oy0tIGJhZCAob3RoZXIgY2Fu IHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0Owovcmhl di9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMktWTTEyLVA1LzFjYTU2YjQ1 LTcwMWUtNGMyMi05ZjU5LTNhZWJlYTRkODQ3Ny9kb21fbWQ6PGJyPgogICAgICAgICAgICAgICZn dDsgdG90YWwgMSwxTTxicj4KICAgICAgICAgICAgICAmZ3Q7IC1ydy1ydy1yLS0gMSB2ZHNtIGt2 bSDCoCDCoDAgMjQuIMO6bm8gMDcuMzUgaWRzwqDCoCDCoCDCoAogICAgICAgICAgICAgIMKgJmx0 Oy0tIGJhZCAob3RoZXIgY2FuIHJlYWQpPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3LXIt LSAxIHZkc20ga3ZtIMKgMTZNIDI0LiDDum5vIDAxLjA2IGluYm94PGJyPgogICAgICAgICAgICAg ICZndDsgLXJ3LXJ3LXItLSAxIHZkc20ga3ZtIDIsME0gMjQuIMO6bm8gMDIuNDQgbGVhc2VzPGJy PgogICAgICAgICAgICAgICZndDsgLXJ3LXItLXItLSAxIHZkc20ga3ZtIMKgOTk4IDI0LiDDum5v IDE5LjA3IG1ldGFkYXRhPGJyPgogICAgICAgICAgICAgICZndDsgLXJ3LXJ3LXItLSAxIHZkc20g a3ZtIMKgMTZNIMKgNy4gbGlzIDIyLjIwIG91dGJveDxicj4KICAgICAgICAgICAgICA8YnI+CiAg ICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIEl0IHNob3VsZCBsb29rIGxpa2UgdGhpczo8 YnI+CiAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgIC1ydy1ydy0tLS0uIDEgdmRzbSBr dm0gMS4wTSBNYXIgwqAxIDIzOjM2IGlkczxicj4KICAgICAgICAgICAgICAtcnctcnctLS0tLiAx IHZkc20ga3ZtIDIuME0gTWFyIMKgMSAyMzozNSBsZWFzZXM8YnI+CiAgICAgICAgICAgICAgLXJ3 LXItLXItLS4gMSB2ZHNtIGt2bSDCoDM1MyBNYXIgwqAxIDIzOjM1IG1ldGFkYXRhPGJyPgogICAg ICAgICAgICAgIC1ydy1ydy0tLS0uIDEgdmRzbSBrdm0gwqAxNk0gTWFyIMKgMSAyMzozNCBvdXRi b3g8YnI+CiAgICAgICAgICAgICAgLXJ3LXJ3LS0tLS4gMSB2ZHNtIGt2bSDCoDE2TSBNYXIgwqAx IDIzOjM0IGluYm94PGJyPgogICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICBUaGlzIGV4 cGxhaW5zIHRoZSBFQUNDRVMgZXJyb3IuPGJyPgogICAgICAgICAgICAgIDxicj4KICAgICAgICAg ICAgICBZb3UgY2FuIHN0YXJ0IGJ5IGZpeGluZyB0aGUgcGVybWlzc2lvbnMgbWFudWFsbHksIHlv dSBjYW4KICAgICAgICAgICAgICBkbyB0aGlzIG9ubGluZS48YnI+CiAgICAgICAgICAgICAgwqA8 YnI+CiAgICAgICAgICAgICAgJmd0OyDCoFRoZSBpZHMgZmlsZXMgd2FzIGdlbmVyYXRlZCBieSAm cXVvdDt0b3VjaCZxdW90OyBjb21tYW5kIGFmdGVyCiAgICAgICAgICAgICAgZGVsZXRpbmcgdGhl bSBkdWUgJnF1b3Q7c2FubG9jayBsb2NraW5nIGhhbmcmcXVvdDsgwqBnbHVzdGVyIGNyYXNoCiAg ICAgICAgICAgICAgJmFtcDsgcmVib290PGJyPgogICAgICAgICAgICAgICZndDsgSSBleHBlY3Rl ZCB0aGF0IHRoZXkgd2lsbCBiZSBmaWxsZWQgYXV0b21hdGljYWx5CiAgICAgICAgICAgICAgYWZ0 ZXIgZ2x1c3RlciByZWJvb3QgKCB0aGUgwqBzaGFkb3cgY29weSBmcm9tIMKgICZxdW90Oy5nbHVz dGVyICZxdW90OwogICAgICAgICAgICAgIMKgIGRpcmVjdG9yeSDCoHdhcyBkZWxldGVkICZhbXA7 IGNyZWF0ZWQgZW1wdHkgwqB0b28gKTxicj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgSSBkb24mIzM5O3Qga25vdyBhYm91dCBnbHVzdGVyIHNoYWRvdyBjb3B5LCBJIHdvdWxkIG5v dCBwbGF5CiAgICAgICAgICAgICAgd2l0aCBnbHVzdGVyIGludGVybmFscy48L2Rpdj4KICAgICAg ICAgICAgPGRpdj5BZGRpbmcgU2FoaW5hIGZvciBhZHZpY2UuPGJyPgogICAgICAgICAgICA8L2Rp dj4KICAgICAgICAgIDwvZGl2PgogICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICA8YnI+CiAg ICAgICAgRGlkIHlvdSBnZW5lcmF0ZSB0aGUgaWRzIGZpbGUgb24gdGhlIG1vdW50IHBvaW50Ljxi cj4KICAgICAgICA8YnI+CiAgICAgICAgUmF2aSwgY2FuIHlvdSBoZWxwIGhlcmU/PGJyPgogICAg ICAgIDxicj4KICAgICAgPC9ibG9ja3F1b3RlPgogICAgICA8YnI+CiAgICAgIE9rYXksIHNvIHdo YXQgSSB1bmRlcnN0YW5kIGZyb20gdGhlIG91dHB1dCBhYm92ZSBpcyB5b3UgaGF2ZQogICAgICBk aWZmZXJlbnQgZ2x1c3RlciB2b2x1bWVzIG1vdW50ZWQgYW5kIHNvbWUgb2YgdGhlbSBoYXZlIGlu Y29ycmVjdAogICAgICBwZXJtaXNzaW9ucyBmb3IgdGhlICYjMzk7aWRzJiMzOTsgZmlsZS4gVGhl IHdheSB0byBmaXggaXQgaXMgdG8gZG8gaXQgZnJvbQogICAgICB0aGUgbW91bnQgbGlrZSBOaXIg c2FpZC48YnI+CiAgICAgIFdoeSBkaWQgeW91IGRlbGV0ZSB0aGUgZmlsZSBmcm9tIHRoZSAuZ2x1 c3RlcmZzIGluIHRoZSBicmljayhzKT/CoAogICAgICBXYXMgdGhlcmUgYSBnZmlkIHNwbGl0IGJy YWluPyA8YnI+CiAgICAgIDxicj4KICAgICAgLVJhdmk8YnI+CiAgICAgIDxicj4KICAgICAgPGJs b2NrcXVvdGUgdHlwZT0iY2l0ZSI+CiAgICAgICAgPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+CiAg ICAgICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAg ICAgJmd0OyBPSywgaXQgbG9va3MgdGhhdCBzYW5sb2NrIMKgY2FuJiMzOTt0IHdvcmsgd2l0aCBl bXB0eSBmaWxlCiAgICAgICAgICAgICAgb3IgcmV3cml0ZSB0aGVtIC48YnI+CiAgICAgICAgICAg ICAgJmd0OyBBbSBJIHJpZ2h0ID8/PGJyPgogICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAg ICBZZXMsIHRoZSBmaWxlcyBtdXN0IGJlIGluaXRpYWxpemVkIGJlZm9yZSBzYW5sb2NrIGNhbiB1 c2UKICAgICAgICAgICAgICB0aGVtLjxicj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgWW91IGNhbiBpbml0aWFsaXplIHRoZSBmaWxlIGxpa2UgdGhpczo8YnI+CiAgICAgICAgICAg ICAgPGJyPgogICAgICAgICAgICAgIHNhbmxvY2sgZGlyZWN0IGluaXQgLXMKICAgICAgICAgICAg ICAmbHQ7c2RfdXVpZCZndDs6MDpyZXBhaXIvJmx0O3NkX3V1aWQmZ3Q7L2RvbV9tZC9pZHM6MDxi cj4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgVGFrZW4gZnJvbSA8YSBocmVmPSJo dHRwOi8vbGlzdHMub3ZpcnQub3JnL3BpcGVybWFpbC91c2Vycy8yMDE2LUZlYnJ1YXJ5LzAzODA0 Ni5odG1sIiB0YXJnZXQ9Il9ibGFuayI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9waXBlcm1haWwv dXNlcnMvMjAxNi1GZWJydWFyeS8wMzgwNDYuaHRtbDwvYT48YnI+CiAgICAgICAgICAgICAgPGJy PgogICAgICAgICAgICAgICZndDsgVGhlIGxhc3QgcG9pbnQgLSBhYm91dCAmcXVvdDtpZHMmcXVv dDsgd29ya2Fyb3VuZCAtIHRoaXMgaXMKICAgICAgICAgICAgICBvZmZsaW5lIHZlcnNpb24gPSBW TXMgaGF2ZSB0byBiZSBtb3ZlZCBvdXQgZnJvbSBmb3IKICAgICAgICAgICAgICBjb250aW51YWwg cnVubmluZyB3aXRoIG1haW50ZW5hbmNlIHZvbHVtZSBtb2RlPGJyPgogICAgICAgICAgICAgICZn dDsgQnV0IHRoaXMgaXMgbm90IGFjY2VwdGFibGUgaW4gY3VycmVudCBzaXR1YXRpb24sIHNvCiAg ICAgICAgICAgICAgdGhlIHF1ZXN0aW9uIGFnYWluLCDCoGlzIGl0IHNhZmUgdG8gZG8gaXQgb25s aW5lID8/IMKgKCBZRVMKICAgICAgICAgICAgICAvIE5PICk8L2Rpdj4KICAgICAgICAgICAgPGRp dj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICA8ZGl2PlRoZSBpZHMgZmlsZSBp cyBhY2Nlc3NlZCBvbmx5IGJ5IHNhbmxvY2suIEkgZ3Vlc3MgdGhhdAogICAgICAgICAgICAgIHlv dSBkb24mIzM5O3QgaGF2ZSBhIHJ1bm5pbmc8L2Rpdj4KICAgICAgICAgICAgPGRpdj5TUE0gb24g dGhpcyBEQywgc2luY2Ugc2FubG9jayBmYWlscyB0byBhY3F1aXJlIGEgaG9zdAogICAgICAgICAg ICAgIGlkLCBzbyB5b3UgYXJlIHByZXR0eSBzYWZlPC9kaXY+CiAgICAgICAgICAgIDxkaXY+dG8g Zml4IHRoZSBwZXJtaXNzaW9ucyBhbmQgaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGVzLjwvZGl2Pgog ICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+ SSB3b3VsZCBkbyB0aGlzOjwvZGl2PgogICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAg PC9kaXY+CiAgICAgICAgICAgIDxkaXY+MS4gU3RvcCBlbmdpbmUsIMKgc28gaXQgd2lsbCBub3Qg dHJ5IHRvIHN0YXJ0IHZkc208L2Rpdj4KICAgICAgICAgICAgPGRpdj4yLiBTdG9wIHZkc20gb24g YWxsIGhvc3RzLCBzbyB0aGV5IGRvIG5vdCB0cnkgdG8KICAgICAgICAgICAgICBhY3F1aXJlIGEg aG9zdCBpZCB3aXRoIHNhbmxvY2s8L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoCDCoCBUaGlzIGRv ZXMgbm90IGFmZmVjdCBydW5uaW5nIHZtczwvZGl2PgogICAgICAgICAgICA8ZGl2PjMuIEZpeCB0 aGUgcGVybWlzc2lvbnMgb24gdGhlIGlkcyBmaWxlLCB2aWEgZ2x1c3RlcmZzCiAgICAgICAgICAg ICAgbW91bnQ8L2Rpdj4KICAgICAgICAgICAgPGRpdj40LiBJbml0aWFsaXplIHRoZSBpZHMgZmls ZXMgZnJvbSBvbmUgb2YgdGhlIGhvc3RzLCB2aWEKICAgICAgICAgICAgICB0aGUgZ2x1c3RlcmZz IG1vdW50PC9kaXY+CiAgICAgICAgICAgIDxkaXY+wqAgwqAgVGhpcyBzaG91bGQgZml4IHRoZSBp ZHMgZmlsZXMgb24gYWxsIHJlcGxpY2FzPC9kaXY+CiAgICAgICAgICAgIDxkaXY+NS4gU3RhcnQg dmRzbSBvbiBhbGwgaG9zdHM8L2Rpdj4KICAgICAgICAgICAgPGRpdj42LiBTdGFydCBlbmdpbmU8 L2Rpdj4KICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAg ICA8ZGl2PkVuZ2luZSB3aWxsIGNvbm5lY3QgdG8gYWxsIGhvc3RzLCBob3N0cyB3aWxsIGNvbm5l Y3QgdG8KICAgICAgICAgICAgICBzdG9yYWdlIGFuZCB0cnkgdG8gYWNxdWlyZSBhIGhvc3QgaWQu PC9kaXY+CiAgICAgICAgICAgIDxkaXY+VGhlbiBFbmdpbmUgd2lsbCBzdGFydCB0aGUgU1BNIG9u IG9uZSBvZiB0aGUgaG9zdHMsIGFuZAogICAgICAgICAgICAgIHlvdXIgREMgc2hvdWxkIGJlY29t ZSB1cC48L2Rpdj4KICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAg ICAgICAgICA8ZGl2PkRhdmlkLCBTYWhpbmEsIGNhbiB5b3UgY29uZmlybSB0aGF0IHRoaXMgcHJv Y2VkdXJlIGlzCiAgICAgICAgICAgICAgc2FmZT88L2Rpdj4KICAgICAgICAgIDwvZGl2PgogICAg ICAgIDwvYmxvY2txdW90ZT4KICAgICAgICA8YnI+CiAgICAgICAgWWVzLCBjb3JyZWN0aW5nIGZy b20gdGhlIG1vdW50IHBvaW50IHNob3VsZCBmaXggaXQgb24gYWxsCiAgICAgICAgcmVwbGljYXM8 YnI+CiAgICAgICAgPGJyPgogICAgICAgIDxicj4KICAgICAgICA8YmxvY2txdW90ZSB0eXBlPSJj aXRlIj4KICAgICAgICAgIDxkaXYgZGlyPSJsdHIiPgogICAgICAgICAgICA8ZGl2Pjxicj4KICAg ICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+TmlyPC9kaXY+CiAgICAgICAgICAgIDxk aXY+PGJyPgogICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj4mZ3Q7PGJyPgogICAg ICAgICAgICAgICZndDsgcmVncy48YnI+CiAgICAgICAgICAgICAgJmd0OyBQYXZlbDxicj4KICAg ICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAg Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IE9uIDEuMy4yMDE2IDE4OjM4LCBOaXIgU29mZmVy IHdyb3RlOjxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsgT24g VHVlLCBNYXIgMSwgMjAxNiBhdCA1OjA3IFBNLCA8YSBocmVmPSJtYWlsdG86cGFmMUBlbWFpbC5j eiIgdGFyZ2V0PSJfYmxhbmsiPnBhZjFAZW1haWwuY3o8L2E+ICZsdDs8YSBocmVmPSJtYWlsdG86 cGFmMUBlbWFpbC5jeiIgdGFyZ2V0PSJfYmxhbmsiPnBhZjFAZW1haWwuY3o8L2E+Jmd0OyB3cm90 ZTo8YnI+CiAgICAgICAgICAgICAgJmd0OyZndDs8YnI+CiAgICAgICAgICAgICAgJmd0OyZndDsg SGVsbG8sIMKgY2FuIGFueWJvZHkgwqBleHBsYWluIHRoaXMgZXJyb3Igbm8uMTMgKAogICAgICAg ICAgICAgIG9wZW4gZmlsZSApIGluIHNhbmxvY2subG9nIC48YnI+CiAgICAgICAgICAgICAgJmd0 Ozxicj4KICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsgVGhpcyBpcyBF QUNDRVM8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7IENhbiB5 b3Ugc2hhcmUgdGhlIG91dG9wdXQgb2Y6PGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAg ICAgICAgICAgJmd0OyDCoCDCoCBscyAtbGgKL3JoZXYvZGF0YS1jZW50ZXIvbW50LyZsdDtzZXJ2 ZXImZ3Q7OiZsdDtfcGF0aCZndDsvJmx0O3NkX3V1aWQmZ3Q7L2RvbV9tZDxicj4KICAgICAgICAg ICAgICAmZ3Q7IMKgPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAg ICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IFRoZSBzaXplIG9mIMKgJnF1b3Q7 aWRzJnF1b3Q7IGZpbGUgaXMgemVybyAoMCk8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAg ICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsgVGhpcyBpcyBob3cgd2UgY3Jl YXRlIHRoZSBpZHMgZmlsZSB3aGVuIGluaXRpYWxpemluZwogICAgICAgICAgICAgIGl0Ljxicj4K ICAgICAgICAgICAgICAmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsgQnV0IHRoZW4gd2UgdXNl IHNhbmxvY2sgdG8gaW5pdGlhbGl6ZSB0aGUgaWRzIGZpbGUsCiAgICAgICAgICAgICAgYW5kIGl0 IHNob3VsZCBiZSAxTWlCIGFmdGVyIHRoYXQuPGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAg ICAgICAgICAgICAgJmd0OyBJcyB0aGlzIGlkcyBmaWxlcyBjcmVhdGVkIGJ5IHZkc20sIG9yIG9u ZSB5b3UgY3JlYXRlZAogICAgICAgICAgICAgIHlvdXJzZWxmPzxicj4KICAgICAgICAgICAgICAm Z3Q7IMKgPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsm Z3Q7IDIwMTYtMDItMjggMDM6MjU6NDYrMDEwMCAyNjk2MjYgWzE5NTFdOiBvcGVuCiAgICAgICAg ICAgICAgZXJyb3IgLTEzCi9yaGV2L2RhdGEtY2VudGVyL21udC9nbHVzdGVyU0QvbG9jYWxob3N0 Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9p ZHM8YnI+CiAgICAgICAgICAgICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoyNTo0NiswMTAwIDI2 OTYyNiBbMTk1MV06IHMxODc5ODUKICAgICAgICAgICAgICBvcGVuX2Rpc2sKICAgICAgICAgICAg ICAvcmhldi9kYXRhLWNlbnRlci9tbnQvZ2x1c3RlclNEL2xvY2FsaG9zdDpfMUtWTTEyLVA0Lzdm NTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNC9kb21fbWQvaWRzCgoKICAgICAgICAg ICAgICBlcnJvciAtMTM8YnI+CiAgICAgICAgICAgICAgJmd0OyZndDsgMjAxNi0wMi0yOCAwMzoy NTo1NiswMTAwIDI2OTYzNiBbMTEzMDRdOiBzMTg3OTkyCiAgICAgICAgICAgICAgbG9ja3NwYWNl CjdmNTJiNjk3LWMxOTktNGY1OC04OWFhLTEwMmQ0NDMyNzEyNDoxOi9yaGV2L2RhdGEtY2VudGVy L21udC9nbHVzdGVyU0QvbG9jYWxob3N0Ol8xS1ZNMTItUDQvN2Y1MmI2OTctYzE5OS00ZjU4LTg5 YWEtMTAyZDQ0MzI3MTI0L2RvbV9tZC9pZHM6MDxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxi cj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBJZiB0aGUgbWFpbiBwcm9ibGVtIGlzIGFib3V0IHpl cm8gZmlsZSBzaXplLCBjYW4KICAgICAgICAgICAgICBJIHJlZ2VuZXJhdGUgwqB0aGlzIGZpbGUg b25saW5lIHNlY3VyZWx5ICwgd2l0aCBubyBWTQogICAgICAgICAgICAgIGRlcGVuZGVuY2UgwqA/ Pz8/PGJyPgogICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAg ICAgICAgICAgICAmZ3Q7IFllcywgSSB0aGluayBJIGFscmVhZHkgcmVmZXJyZWQgdG8gdGhlIGlu c3RydWN0aW9ucwogICAgICAgICAgICAgIGhvdyB0byBkbyB0aGF0IGluIGEgcHJldmlvdXMgbWFp bC48YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4K ICAgICAgICAgICAgICAmZ3Q7Jmd0Ozxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBkaXN0ID0g UkhFTCAtIDcgLSAyLjE1MTE8YnI+CiAgICAgICAgICAgICAgJmd0OyZndDsga2VybmVsID0gMy4x MC4wIC0gMzI3LjEwLjEuZWw3Lng4Nl82NDxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyBLVk0g PSAyLjMuMCAtIDI5LjEuZWw3PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IGxpYnZpcnQgPSBs aWJ2aXJ0LTEuMi4xNy0xMy5lbDdfMi4zPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IHZkc20g PSB2ZHNtLTQuMTYuMzAtMC5lbDc8YnI+CiAgICAgICAgICAgICAgJmd0OyZndDsgR2x1c3RlckZT ID0gZ2x1c3RlcmZzLTMuNy44LTEuZWw3PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgog ICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IHJlZ3MuPGJy PgogICAgICAgICAgICAgICZndDsmZ3Q7IFBhdmVsPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7 PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fPGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7IFVzZXJzIG1haWxp bmcgbGlzdDxicj4KICAgICAgICAgICAgICAmZ3Q7Jmd0OyA8YSBocmVmPSJtYWlsdG86VXNlcnNA b3ZpcnQub3JnIiB0YXJnZXQ9Il9ibGFuayI+VXNlcnNAb3ZpcnQub3JnPC9hPjxicj4KICAgICAg ICAgICAgICAmZ3Q7Jmd0OyA8YSBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4v bGlzdGluZm8vdXNlcnMiIHRhcmdldD0iX2JsYW5rIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21h aWxtYW4vbGlzdGluZm8vdXNlcnM8L2E+PGJyPgogICAgICAgICAgICAgICZndDsmZ3Q7PGJyPgog ICAgICAgICAgICAgICZndDs8YnI+CiAgICAgICAgICAgICAgJmd0Ozxicj4KICAgICAgICAgICAg PC9kaXY+CiAgICAgICAgICA8L2Rpdj4KICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgPGJy PgogICAgICAgIDxicj4KICAgICAgICA8ZmllbGRzZXQ+PC9maWVsZHNldD4KICAgICAgICA8YnI+ CiAgICAgICAgPHByZT5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwpHbHVzdGVyLXVzZXJzIG1haWxpbmcgbGlzdAo8YSBocmVmPSJtYWlsdG86R2x1c3Rlci11 c2Vyc0BnbHVzdGVyLm9yZyIgdGFyZ2V0PSJfYmxhbmsiPkdsdXN0ZXItdXNlcnNAZ2x1c3Rlci5v cmc8L2E+CjxhIGhyZWY9Imh0dHA6Ly93d3cuZ2x1c3Rlci5vcmcvbWFpbG1hbi9saXN0aW5mby9n bHVzdGVyLXVzZXJzIiB0YXJnZXQ9Il9ibGFuayI+aHR0cDovL3d3dy5nbHVzdGVyLm9yZy9tYWls bWFuL2xpc3RpbmZvL2dsdXN0ZXItdXNlcnM8L2E+PC9wcmU+CiAgICAgIDwvYmxvY2txdW90ZT4K ICAgICAgPGJyPgogICAgICA8YnI+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+CiAgPC9kaXY+ PC9kaXY+PC9kaXY+Cgo8L2Jsb2NrcXVvdGU+PC9kaXY+PGJyPjwvZGl2PjwvZGl2Pgo= --===============7415982641314222456==-- From ravishankar at redhat.com Wed Mar 2 20:02:59 2016 Content-Type: multipart/mixed; boundary="===============0555692295953529289==" MIME-Version: 1.0 From: Ravishankar N To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Thu, 03 Mar 2016 06:32:51 +0530 Message-ID: <56D78D3B.9060101@redhat.com> In-Reply-To: CAMRbyysFqJwke8vdmYL7NGUuzpFNJppZC3riLR8jC7pd33DVdQ@mail.gmail.com --===============0555692295953529289== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------050101000701070505090304 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 7bit On 03/03/2016 12:43 AM, Nir Soffer wrote: > > PS: # find /STORAGES -samefile > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids > -print > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids > =3D missing "shadowfile" in " .gluster " dir. > How can I fix it ?? - online ! > > > Ravi? Is this the case in all 3 bricks of the replica? BTW, you can just stat the file on the brick and see the link count (it = must be 2) instead of running the more expensive find command. --------------050101000701070505090304 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit

On 03/03/2016 12:43 AM, Nir Soffer wrote:

PS:=C2=A0 # find /STORAGES -samefile /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/= ids -print
/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids
=3D missing "shadowfile" in " .gluster " dir.
How can I fix it ?? - online !

Ravi?

Is this the case in all 3 bricks of the replica?
BTW, you can just stat the file on the brick and see the link count (it must be 2) instead of running the more expensive find command.

--------------050101000701070505090304-- --===============0555692295953529289== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wNTAxMDEwMDA3MDEwNzA1MDUwOTAzMDQKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQK Ck9uIDAzLzAzLzIwMTYgMTI6NDMgQU0sIE5pciBTb2ZmZXIgd3JvdGU6Cj4KPiAgICAgUFM6ICAj IGZpbmQgL1NUT1JBR0VTIC1zYW1lZmlsZQo+ICAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0 YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcwo+ICAgICAtcHJpbnQK PiAgICAgL1NUT1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNk NzBiMTQ3L2RvbV9tZC9pZHMKPiAgICAgPSBtaXNzaW5nICJzaGFkb3dmaWxlIiBpbiAiIC5nbHVz dGVyICIgZGlyLgo+ICAgICBIb3cgY2FuIEkgZml4IGl0ID8/IC0gb25saW5lICEKPgo+Cj4gUmF2 aT8KSXMgdGhpcyB0aGUgY2FzZSBpbiBhbGwgMyBicmlja3Mgb2YgdGhlIHJlcGxpY2E/CkJUVywg eW91IGNhbiBqdXN0IHN0YXQgdGhlIGZpbGUgb24gdGhlIGJyaWNrIGFuZCBzZWUgdGhlIGxpbmsg Y291bnQgKGl0IAptdXN0IGJlIDIpIGluc3RlYWQgb2YgcnVubmluZyB0aGUgbW9yZSBleHBlbnNp dmUgZmluZCBjb21tYW5kLgoKCi0tLS0tLS0tLS0tLS0tMDUwMTAxMDAwNzAxMDcwNTA1MDkwMzA0 CkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04CkNvbnRlbnQtVHJhbnNmZXIt RW5jb2Rpbmc6IDhiaXQKCjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEgY29udGVudD0idGV4dC9o dG1sOyBjaGFyc2V0PXV0Zi04IiBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogIDwvaGVhZD4K ICA8Ym9keSB0ZXh0PSIjMDAwMDAwIiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIDxkaXYgY2xhc3M9 Im1vei1jaXRlLXByZWZpeCI+T24gMDMvMDMvMjAxNiAxMjo0MyBBTSwgTmlyIFNvZmZlcgogICAg ICB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQU1SYnl5 c0ZxSndrZTh2ZG1ZTDdOR1V1enBGTkpwcFpDM3JpTFI4akM3cGQzM0RWZFFAbWFpbC5nbWFpbC5j b20iCiAgICAgIHR5cGU9ImNpdGUiPgogICAgICA8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVv dGUiIHN0eWxlPSJtYXJnaW46MCAwIDAKICAgICAgICAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2Nj IHNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPgogICAgICAgIDxkaXYgdGV4dD0iIzAwMDA2NiIgYmdj b2xvcj0iI0ZGRkZGRiI+UFM6wqAgIyBmaW5kIC9TVE9SQUdFUwogICAgICAgICAgLXNhbWVmaWxl CiAgICAgICAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0w ODRmM2Q3MGIxNDcvZG9tX21kL2lkcwogICAgICAgICAgLXByaW50PGJyPgovU1RPUkFHRVMvZzFy NXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczxi cj4KICAgICAgICAgID0gbWlzc2luZyAic2hhZG93ZmlsZSIgaW4gIiAuZ2x1c3RlciAiIGRpci48 YnI+CiAgICAgICAgICBIb3cgY2FuIEkgZml4IGl0ID8/IC0gb25saW5lICE8L2Rpdj4KICAgICAg PC9ibG9ja3F1b3RlPgogICAgICA8ZGl2Pjxicj4KICAgICAgPC9kaXY+CiAgICAgIDxkaXY+UmF2 aT88L2Rpdj4KICAgIDwvYmxvY2txdW90ZT4KICAgIElzIHRoaXMgdGhlIGNhc2UgaW4gYWxsIDMg YnJpY2tzIG9mIHRoZSByZXBsaWNhPyA8YnI+CiAgICBCVFcsIHlvdSBjYW4ganVzdCBzdGF0IHRo ZSBmaWxlIG9uIHRoZSBicmljayBhbmQgc2VlIHRoZSBsaW5rIGNvdW50CiAgICAoaXQgbXVzdCBi ZSAyKSBpbnN0ZWFkIG9mIHJ1bm5pbmcgdGhlIG1vcmUgZXhwZW5zaXZlIGZpbmQgY29tbWFuZC48 YnI+CiAgICA8YnI+CiAgPC9ib2R5Pgo8L2h0bWw+CgotLS0tLS0tLS0tLS0tLTA1MDEwMTAwMDcw MTA3MDUwNTA5MDMwNC0tCg== --===============0555692295953529289==-- From paf1 at email.cz Thu Mar 3 04:23:56 2016 Content-Type: multipart/mixed; boundary="===============4228431815239851844==" MIME-Version: 1.0 From: paf1 at email.cz To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Thu, 03 Mar 2016 10:23:35 +0100 Message-ID: <56D80297.6080407@email.cz> In-Reply-To: 56D78D3B.9060101@redhat.com --===============4228431815239851844== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------090101060200080100090200 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit This is replica 2, only , with following settings Options Reconfigured: performance.quick-read: off performance.read-ahead: off performance.io-cache: off performance.stat-prefetch: off cluster.eager-lock: enable network.remote-dio: enable cluster.quorum-type: fixed cluster.server-quorum-type: none storage.owner-uid: 36 storage.owner-gid: 36 cluster.quorum-count: 1 cluster.self-heal-daemon: enable If I'll create "ids" file manually ( eg. " sanlock direct init -s = 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4= e23-ab46-084f3d70b147/dom_md/ids:0 = " ) on both bricks, vdsm is writing only to half of them ( that with 2 links =3D correct ) "ids" file has correct permittions, owner, size on both bricks. brick 1: -rw-rw---- 1 vdsm kvm 1048576 2. b=C5=99e 18.56 = /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - = not updated brick 2: -rw-rw---- 2 vdsm kvm 1048576 3. b=C5=99e 10.16 = /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - = is continually updated What happens when I'll restart vdsm ? Will oVirt storages go to "disable = " state ??? =3D disconnect VMs storages ? regs.Pa. On 3.3.2016 02:02, Ravishankar N wrote: > On 03/03/2016 12:43 AM, Nir Soffer wrote: >> >> PS: # find /STORAGES -samefile >> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids >> -print >> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids >> =3D missing "shadowfile" in " .gluster " dir. >> How can I fix it ?? - online ! >> >> >> Ravi? > Is this the case in all 3 bricks of the replica? > BTW, you can just stat the file on the brick and see the link count = > (it must be 2) instead of running the more expensive find command. > --------------090101060200080100090200 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit This is replica 2, only , with following settings

Options Reconfigured:
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.quorum-type: fixed
cluster.server-quorum-type: none
storage.owner-uid: 36
storage.owner-gid: 36
cluster.quorum-count: 1
cluster.self-heal-daemon: enable

If I'll create "ids" file manually (=C2=A0 eg. " sanlock direct init -s 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4= e23-ab46-084f3d70b147/dom_md/ids:0 " ) on both bricks,
vdsm is writing only to half of them ( that with 2 links =3D correct )<= br> "ids" file has correct permittions, owner, size=C2=A0 on both bricks. brick 1:=C2=A0 -rw-rw---- 1 vdsm kvm 1048576=C2=A0 2.=C2=A0b=C5=99e 18.= 56 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - not updated
brick 2:=C2=A0 -rw-rw---- 2 vdsm kvm 1048576=C2=A0 3.=C2=A0b=C5=99e 10.= 16 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - is continually updated

What happens when I'll restart vdsm ? Will oVirt storages go to "disable " state ??? =3D disconnect VMs storages ?

regs.Pa.

On 3.3.2016 02:02, Ravishankar N wrote:<= br>

On 03/03/2016 12:43 AM, Nir Soffer wrote:

PS:=C2=A0 # find /STORA= GES -samefile /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_m= d/ids -print
/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids
=3D missing "shadowfile" in " .gluster " dir.
How can I fix it ?? - online !

Ravi?

Is this the case in all 3 bricks of the replica?
BTW, you can just stat the file on the brick and see the link count (it must be 2) instead of running the more expensive find command.

--------------090101060200080100090200-- --===============4228431815239851844== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wOTAxMDEwNjAyMDAwODAxMDAwOTAyMDAKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK ClRoaXMgaXMgcmVwbGljYSAyLCBvbmx5ICwgd2l0aCBmb2xsb3dpbmcgc2V0dGluZ3MKCk9wdGlv bnMgUmVjb25maWd1cmVkOgpwZXJmb3JtYW5jZS5xdWljay1yZWFkOiBvZmYKcGVyZm9ybWFuY2Uu cmVhZC1haGVhZDogb2ZmCnBlcmZvcm1hbmNlLmlvLWNhY2hlOiBvZmYKcGVyZm9ybWFuY2Uuc3Rh dC1wcmVmZXRjaDogb2ZmCmNsdXN0ZXIuZWFnZXItbG9jazogZW5hYmxlCm5ldHdvcmsucmVtb3Rl LWRpbzogZW5hYmxlCmNsdXN0ZXIucXVvcnVtLXR5cGU6IGZpeGVkCmNsdXN0ZXIuc2VydmVyLXF1 b3J1bS10eXBlOiBub25lCnN0b3JhZ2Uub3duZXItdWlkOiAzNgpzdG9yYWdlLm93bmVyLWdpZDog MzYKY2x1c3Rlci5xdW9ydW0tY291bnQ6IDEKY2x1c3Rlci5zZWxmLWhlYWwtZGFlbW9uOiBlbmFi bGUKCklmIEknbGwgY3JlYXRlICJpZHMiIGZpbGUgbWFudWFsbHkgKCAgZWcuICIgc2FubG9jayBk aXJlY3QgaW5pdCAtcyAKM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3OjA6L1NU T1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2Rv bV9tZC9pZHM6MCAKIiApIG9uIGJvdGggYnJpY2tzLAp2ZHNtIGlzIHdyaXRpbmcgb25seSB0byBo YWxmIG9mIHRoZW0gKCB0aGF0IHdpdGggMiBsaW5rcyA9IGNvcnJlY3QgKQoiaWRzIiBmaWxlIGhh cyBjb3JyZWN0IHBlcm1pdHRpb25zLCBvd25lciwgc2l6ZSAgb24gYm90aCBicmlja3MuCmJyaWNr IDE6ICAtcnctcnctLS0tIDEgdmRzbSBrdm0gMTA0ODU3NiAgMi4gYsWZZSAxOC41NiAKL1NUT1JB R0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9t ZC9pZHMgLSAKbm90IHVwZGF0ZWQKYnJpY2sgMjogIC1ydy1ydy0tLS0gMiB2ZHNtIGt2bSAxMDQ4 NTc2ICAzLiBixZllIDEwLjE2IAovU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRl MjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcyAtIAppcyBjb250aW51YWxseSB1cGRhdGVk CgpXaGF0IGhhcHBlbnMgd2hlbiBJJ2xsIHJlc3RhcnQgdmRzbSA/IFdpbGwgb1ZpcnQgc3RvcmFn ZXMgZ28gdG8gImRpc2FibGUgCiIgc3RhdGUgPz8/ID0gZGlzY29ubmVjdCBWTXMgc3RvcmFnZXMg PwoKcmVncy5QYS4KCk9uIDMuMy4yMDE2IDAyOjAyLCBSYXZpc2hhbmthciBOIHdyb3RlOgo+IE9u IDAzLzAzLzIwMTYgMTI6NDMgQU0sIE5pciBTb2ZmZXIgd3JvdGU6Cj4+Cj4+ICAgICBQUzogICMg ZmluZCAvU1RPUkFHRVMgLXNhbWVmaWxlCj4+ICAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0 YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcwo+PiAgICAgLXByaW50 Cj4+ICAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRm M2Q3MGIxNDcvZG9tX21kL2lkcwo+PiAgICAgPSBtaXNzaW5nICJzaGFkb3dmaWxlIiBpbiAiIC5n bHVzdGVyICIgZGlyLgo+PiAgICAgSG93IGNhbiBJIGZpeCBpdCA/PyAtIG9ubGluZSAhCj4+Cj4+ Cj4+IFJhdmk/Cj4gSXMgdGhpcyB0aGUgY2FzZSBpbiBhbGwgMyBicmlja3Mgb2YgdGhlIHJlcGxp Y2E/Cj4gQlRXLCB5b3UgY2FuIGp1c3Qgc3RhdCB0aGUgZmlsZSBvbiB0aGUgYnJpY2sgYW5kIHNl ZSB0aGUgbGluayBjb3VudCAKPiAoaXQgbXVzdCBiZSAyKSBpbnN0ZWFkIG9mIHJ1bm5pbmcgdGhl IG1vcmUgZXhwZW5zaXZlIGZpbmQgY29tbWFuZC4KPgoKCi0tLS0tLS0tLS0tLS0tMDkwMTAxMDYw MjAwMDgwMTAwMDkwMjAwCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04CkNv bnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQKCjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04IiBodHRwLWVxdWl2PSJDb250ZW50LVR5 cGUiPgogIDwvaGVhZD4KICA8Ym9keSB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZGRkZGIj4K ICAgIFRoaXMgaXMgcmVwbGljYSAyLCBvbmx5ICwgd2l0aCBmb2xsb3dpbmcgc2V0dGluZ3M8YnI+ CiAgICA8YnI+CiAgICBPcHRpb25zIFJlY29uZmlndXJlZDo8YnI+CiAgICBwZXJmb3JtYW5jZS5x dWljay1yZWFkOiBvZmY8YnI+CiAgICBwZXJmb3JtYW5jZS5yZWFkLWFoZWFkOiBvZmY8YnI+CiAg ICBwZXJmb3JtYW5jZS5pby1jYWNoZTogb2ZmPGJyPgogICAgcGVyZm9ybWFuY2Uuc3RhdC1wcmVm ZXRjaDogb2ZmPGJyPgogICAgY2x1c3Rlci5lYWdlci1sb2NrOiBlbmFibGU8YnI+CiAgICBuZXR3 b3JrLnJlbW90ZS1kaW86IGVuYWJsZTxicj4KICAgIGNsdXN0ZXIucXVvcnVtLXR5cGU6IGZpeGVk PGJyPgogICAgY2x1c3Rlci5zZXJ2ZXItcXVvcnVtLXR5cGU6IG5vbmU8YnI+CiAgICBzdG9yYWdl Lm93bmVyLXVpZDogMzY8YnI+CiAgICBzdG9yYWdlLm93bmVyLWdpZDogMzY8YnI+CiAgICBjbHVz dGVyLnF1b3J1bS1jb3VudDogMTxicj4KICAgIGNsdXN0ZXIuc2VsZi1oZWFsLWRhZW1vbjogZW5h YmxlPGJyPgogICAgPGJyPgogICAgSWYgSSdsbCBjcmVhdGUgImlkcyIgZmlsZSBtYW51YWxseSAo wqAgZWcuICIgc2FubG9jayBkaXJlY3QgaW5pdCAtcwozYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0w ODRmM2Q3MGIxNDc6MDovU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0 Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczowCiAgICAiICkgb24gYm90aCBicmlja3MsPGJyPgog ICAgdmRzbSBpcyB3cml0aW5nIG9ubHkgdG8gaGFsZiBvZiB0aGVtICggdGhhdCB3aXRoIDIgbGlu a3MgPSBjb3JyZWN0ICk8YnI+CiAgICAiaWRzIiBmaWxlIGhhcyBjb3JyZWN0IHBlcm1pdHRpb25z LCBvd25lciwgc2l6ZcKgIG9uIGJvdGggYnJpY2tzLjxicj4KICAgIGJyaWNrIDE6wqAgLXJ3LXJ3 LS0tLSAxIHZkc20ga3ZtIDEwNDg1NzbCoCAyLsKgYsWZZSAxOC41NgogICAgL1NUT1JBR0VTL2cx cjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZC9pZHMK ICAgIC0gbm90IHVwZGF0ZWQ8YnI+CiAgICBicmljayAyOsKgIC1ydy1ydy0tLS0gMiB2ZHNtIGt2 bSAxMDQ4NTc2wqAgMy7CoGLFmWUgMTAuMTYKICAgIC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRh ZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCiAgICAtIGlzIGNvbnRp bnVhbGx5IHVwZGF0ZWQ8YnI+CiAgICA8YnI+CiAgICBXaGF0IGhhcHBlbnMgd2hlbiBJJ2xsIHJl c3RhcnQgdmRzbSA/IFdpbGwgb1ZpcnQgc3RvcmFnZXMgZ28gdG8KICAgICJkaXNhYmxlICIgc3Rh dGUgPz8/ID0gZGlzY29ubmVjdCBWTXMgc3RvcmFnZXMgPzxicj4KICAgIDxicj4KICAgIHJlZ3Mu UGEuPGJyPgogICAgPGJyPgogICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5PbiAzLjMu MjAxNiAwMjowMiwgUmF2aXNoYW5rYXIgTiB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAgIDxibG9j a3F1b3RlIGNpdGU9Im1pZDo1NkQ3OEQzQi45MDYwMTAxQHJlZGhhdC5jb20iIHR5cGU9ImNpdGUi PgogICAgICA8bWV0YSBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiIGh0dHAtZXF1 aXY9IkNvbnRlbnQtVHlwZSI+CiAgICAgIDxkaXYgY2xhc3M9Im1vei1jaXRlLXByZWZpeCI+T24g MDMvMDMvMjAxNiAxMjo0MyBBTSwgTmlyIFNvZmZlcgogICAgICAgIHdyb3RlOjxicj4KICAgICAg PC9kaXY+CiAgICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQU1SYnl5c0ZxSndrZTh2ZG1ZTDdO R1V1enBGTkpwcFpDM3JpTFI4akM3cGQzM0RWZFFAbWFpbC5nbWFpbC5jb20iCiAgICAgICAgdHlw ZT0iY2l0ZSI+CiAgICAgICAgPGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0i bWFyZ2luOjAgMCAwCiAgICAgICAgICAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3Bh ZGRpbmctbGVmdDoxZXgiPgogICAgICAgICAgPGRpdiB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIj RkZGRkZGIj5QUzrCoCAjIGZpbmQgL1NUT1JBR0VTCiAgICAgICAgICAgIC1zYW1lZmlsZQogICAg ICAgICAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRm M2Q3MGIxNDcvZG9tX21kL2lkcwogICAgICAgICAgICAtcHJpbnQ8YnI+Ci9TVE9SQUdFUy9nMXI1 cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzPGJy PgogICAgICAgICAgICA9IG1pc3NpbmcgInNoYWRvd2ZpbGUiIGluICIgLmdsdXN0ZXIgIiBkaXIu PGJyPgogICAgICAgICAgICBIb3cgY2FuIEkgZml4IGl0ID8/IC0gb25saW5lICE8L2Rpdj4KICAg ICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAg ICAgPGRpdj5SYXZpPzwvZGl2PgogICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgIElzIHRoaXMgdGhl IGNhc2UgaW4gYWxsIDMgYnJpY2tzIG9mIHRoZSByZXBsaWNhPyA8YnI+CiAgICAgIEJUVywgeW91 IGNhbiBqdXN0IHN0YXQgdGhlIGZpbGUgb24gdGhlIGJyaWNrIGFuZCBzZWUgdGhlIGxpbmsKICAg ICAgY291bnQgKGl0IG11c3QgYmUgMikgaW5zdGVhZCBvZiBydW5uaW5nIHRoZSBtb3JlIGV4cGVu c2l2ZSBmaW5kCiAgICAgIGNvbW1hbmQuPGJyPgogICAgICA8YnI+CiAgICA8L2Jsb2NrcXVvdGU+ CiAgICA8YnI+CiAgPC9ib2R5Pgo8L2h0bWw+CgotLS0tLS0tLS0tLS0tLTA5MDEwMTA2MDIwMDA4 MDEwMDA5MDIwMC0tCg== --===============4228431815239851844==-- From ravishankar at redhat.com Thu Mar 3 05:50:38 2016 Content-Type: multipart/mixed; boundary="===============5045971417505254478==" MIME-Version: 1.0 From: Ravishankar N To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Thu, 03 Mar 2016 16:20:30 +0530 Message-ID: <56D816F6.6060103@redhat.com> In-Reply-To: 56D80297.6080407@email.cz --===============5045971417505254478== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------010103050402020505080306 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit On 03/03/2016 02:53 PM, paf1(a)email.cz wrote: > This is replica 2, only , with following settings > > Options Reconfigured: > performance.quick-read: off > performance.read-ahead: off > performance.io-cache: off > performance.stat-prefetch: off > cluster.eager-lock: enable > network.remote-dio: enable > cluster.quorum-type: fixed Not sure why you have set this option. Ideally replica 3 or arbiter volumes are recommended for gluster+ovirt = use. (client) quorum does not make sense for a 2 node setup. I have a = detailed write up here which explains things = http://gluster.readthedocs.org/en/latest/Administrator%20Guide/arbiter-volu= mes-and-quorum/ = which explains things. > cluster.server-quorum-type: none > storage.owner-uid: 36 > storage.owner-gid: 36 > cluster.quorum-count: 1 > cluster.self-heal-daemon: enable > > If I'll create "ids" file manually ( eg. " sanlock direct init -s = > 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-6c66= -4e23-ab46-084f3d70b147/dom_md/ids:0 = > " ) on both bricks, > vdsm is writing only to half of them ( that with 2 links =3D correct ) > "ids" file has correct permittions, owner, size on both bricks. > brick 1: -rw-rw---- 1 vdsm kvm 1048576 2. b=C5=99e 18.56 = > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - = > not updated Okay, so this one has link count =3D1 which means the .glusterfs hardlink = is missing. Can you try deleting this file from the brick and perform a = stat on the file from the mount? That should heal (i.e recreate it ) on = this brick from the other brick with the appropriate .glusterfs hard link. > brick 2: -rw-rw---- 2 vdsm kvm 1048576 3. b=C5=99e 10.16 = > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - = > is continually updated > > What happens when I'll restart vdsm ? Will oVirt storages go to = > "disable " state ??? =3D disconnect VMs storages ? No idea on this one... -Ravi > > regs.Pa. > > On 3.3.2016 02:02, Ravishankar N wrote: >> On 03/03/2016 12:43 AM, Nir Soffer wrote: >>> >>> PS: # find /STORAGES -samefile >>> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids >>> -print >>> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids >>> =3D missing "shadowfile" in " .gluster " dir. >>> How can I fix it ?? - online ! >>> >>> >>> Ravi? >> Is this the case in all 3 bricks of the replica? >> BTW, you can just stat the file on the brick and see the link count = >> (it must be 2) instead of running the more expensive find command. >> > --------------010103050402020505080306 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit

On 03/03/2016 02:53 PM, paf1(a)email.cz wrote:

This is replica 2, only , with following settings

Options Reconfigured:
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.quorum-type: fixed

Not sure why you have set this option.
Ideally replica 3 or arbiter volumes are recommended for gluster+ovirt use.=C2=A0 (client) quorum does not make sense for a 2 no= de setup. I have a detailed write up here which explains things http://glus= ter.readthedocs.org/en/latest/Administrator%20Guide/arbiter-volumes-and-quo= rum/ which explains things.

cluster.server-quorum-type: none
storage.owner-uid: 36
storage.owner-gid: 36
cluster.quorum-count: 1
cluster.self-heal-daemon: enable

If I'll create "ids" file manually (=C2=A0 eg. " sanlock direct init = -s 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-= 6c66-4e23-ab46-084f3d70b147/dom_md/ids:0 " ) on both bricks,
vdsm is writing only to half of them ( that with 2 links =3D correct )
"ids" file has correct permittions, owner, size=C2=A0 on both bricks.=
brick 1:=C2=A0 -rw-rw---- 1 vdsm kvm 1048576=C2=A0 2.=C2=A0b=C5=99e 1= 8.56 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - not updated

Okay, so this one has link count =3D1 which means the .glusterfs hardlink is missing.=C2=A0 Can you try deleting this file from the brick and perform a stat on the file from the mount? That should heal (i.e recreate it ) on this brick from the other brick with the appropriate .glusterfs hard link.

bri= ck 2:=C2=A0 -rw-rw---- 2 vdsm kvm 1048576=C2=A0 3.=C2=A0b=C5=99e 10.16 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - is continually updated

What happens when I'll restart vdsm ? Will oVirt storages go to "disable " state ??? =3D disconnect VMs storages ?

No idea on this one...
-Ravi

regs.Pa.

On 3.3.2016 02:02, Ravishankar N wrote:

On 03/03/2016 12:43 AM, Nir Soffer wrote:

PS:=C2=A0 # find /STO= RAGES -samefile /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom= _md/ids -print
/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids
=3D missing "shadowfile" in " .gluster " dir.
How can I fix it ?? - online !

Ravi?

Is this the case in all 3 bricks of the replica?
BTW, you can just stat the file on the brick and see the link count (it must be 2) instead of running the more expensive find command.

--------------010103050402020505080306-- --===============5045971417505254478== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wMTAxMDMwNTA0MDIwMjA1MDUwODAzMDYKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK Ck9uIDAzLzAzLzIwMTYgMDI6NTMgUE0sIHBhZjFAZW1haWwuY3ogd3JvdGU6Cj4gVGhpcyBpcyBy ZXBsaWNhIDIsIG9ubHkgLCB3aXRoIGZvbGxvd2luZyBzZXR0aW5ncwo+Cj4gT3B0aW9ucyBSZWNv bmZpZ3VyZWQ6Cj4gcGVyZm9ybWFuY2UucXVpY2stcmVhZDogb2ZmCj4gcGVyZm9ybWFuY2UucmVh ZC1haGVhZDogb2ZmCj4gcGVyZm9ybWFuY2UuaW8tY2FjaGU6IG9mZgo+IHBlcmZvcm1hbmNlLnN0 YXQtcHJlZmV0Y2g6IG9mZgo+IGNsdXN0ZXIuZWFnZXItbG9jazogZW5hYmxlCj4gbmV0d29yay5y ZW1vdGUtZGlvOiBlbmFibGUKPiBjbHVzdGVyLnF1b3J1bS10eXBlOiBmaXhlZApOb3Qgc3VyZSB3 aHkgeW91IGhhdmUgc2V0IHRoaXMgb3B0aW9uLgpJZGVhbGx5IHJlcGxpY2EgMyBvciBhcmJpdGVy IHZvbHVtZXMgYXJlIHJlY29tbWVuZGVkIGZvciBnbHVzdGVyK292aXJ0IAp1c2UuICAoY2xpZW50 KSBxdW9ydW0gZG9lcyBub3QgbWFrZSBzZW5zZSBmb3IgYSAyIG5vZGUgc2V0dXAuIEkgaGF2ZSBh IApkZXRhaWxlZCB3cml0ZSB1cCBoZXJlIHdoaWNoIGV4cGxhaW5zIHRoaW5ncyAKaHR0cDovL2ds dXN0ZXIucmVhZHRoZWRvY3Mub3JnL2VuL2xhdGVzdC9BZG1pbmlzdHJhdG9yJTIwR3VpZGUvYXJi aXRlci12b2x1bWVzLWFuZC1xdW9ydW0vIAp3aGljaCBleHBsYWlucyB0aGluZ3MuCgo+IGNsdXN0 ZXIuc2VydmVyLXF1b3J1bS10eXBlOiBub25lCj4gc3RvcmFnZS5vd25lci11aWQ6IDM2Cj4gc3Rv cmFnZS5vd25lci1naWQ6IDM2Cj4gY2x1c3Rlci5xdW9ydW0tY291bnQ6IDEKPiBjbHVzdGVyLnNl bGYtaGVhbC1kYWVtb246IGVuYWJsZQo+Cj4gSWYgSSdsbCBjcmVhdGUgImlkcyIgZmlsZSBtYW51 YWxseSAoICBlZy4gIiBzYW5sb2NrIGRpcmVjdCBpbml0IC1zIAo+IDNjMzRhZDYzLTZjNjYtNGUy My1hYjQ2LTA4NGYzZDcwYjE0NzowOi9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYt NGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzOjAgCj4gIiApIG9uIGJvdGggYnJpY2tz LAo+IHZkc20gaXMgd3JpdGluZyBvbmx5IHRvIGhhbGYgb2YgdGhlbSAoIHRoYXQgd2l0aCAyIGxp bmtzID0gY29ycmVjdCApCj4gImlkcyIgZmlsZSBoYXMgY29ycmVjdCBwZXJtaXR0aW9ucywgb3du ZXIsIHNpemUgIG9uIGJvdGggYnJpY2tzLgo+IGJyaWNrIDE6ICAtcnctcnctLS0tIDEgdmRzbSBr dm0gMTA0ODU3NiAgMi4gYsWZZSAxOC41NiAKPiAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2 My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcyAtIAo+IG5vdCB1cGRhdGVk CgpPa2F5LCBzbyB0aGlzIG9uZSBoYXMgbGluayBjb3VudCA9MSB3aGljaCBtZWFucyB0aGUgLmds dXN0ZXJmcyBoYXJkbGluayAKaXMgbWlzc2luZy4gIENhbiB5b3UgdHJ5IGRlbGV0aW5nIHRoaXMg ZmlsZSBmcm9tIHRoZSBicmljayBhbmQgcGVyZm9ybSBhIApzdGF0IG9uIHRoZSBmaWxlIGZyb20g dGhlIG1vdW50PyBUaGF0IHNob3VsZCBoZWFsIChpLmUgcmVjcmVhdGUgaXQgKSBvbiAKdGhpcyBi cmljayBmcm9tIHRoZSBvdGhlciBicmljayB3aXRoIHRoZSBhcHByb3ByaWF0ZSAuZ2x1c3RlcmZz IGhhcmQgbGluay4KCgo+IGJyaWNrIDI6ICAtcnctcnctLS0tIDIgdmRzbSBrdm0gMTA0ODU3NiAg My4gYsWZZSAxMC4xNiAKPiAvU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMt YWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcyAtIAo+IGlzIGNvbnRpbnVhbGx5IHVwZGF0ZWQK Pgo+IFdoYXQgaGFwcGVucyB3aGVuIEknbGwgcmVzdGFydCB2ZHNtID8gV2lsbCBvVmlydCBzdG9y YWdlcyBnbyB0byAKPiAiZGlzYWJsZSAiIHN0YXRlID8/PyA9IGRpc2Nvbm5lY3QgVk1zIHN0b3Jh Z2VzID8KCk5vIGlkZWEgb24gdGhpcyBvbmUuLi4KLVJhdmkKPgo+IHJlZ3MuUGEuCj4KPiBPbiAz LjMuMjAxNiAwMjowMiwgUmF2aXNoYW5rYXIgTiB3cm90ZToKPj4gT24gMDMvMDMvMjAxNiAxMjo0 MyBBTSwgTmlyIFNvZmZlciB3cm90ZToKPj4+Cj4+PiAgICAgUFM6ICAjIGZpbmQgL1NUT1JBR0VT IC1zYW1lZmlsZQo+Pj4gICAgIC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUy My1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCj4+PiAgICAgLXByaW50Cj4+PiAgICAgL1NU T1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2Rv bV9tZC9pZHMKPj4+ICAgICA9IG1pc3NpbmcgInNoYWRvd2ZpbGUiIGluICIgLmdsdXN0ZXIgIiBk aXIuCj4+PiAgICAgSG93IGNhbiBJIGZpeCBpdCA/PyAtIG9ubGluZSAhCj4+Pgo+Pj4KPj4+IFJh dmk/Cj4+IElzIHRoaXMgdGhlIGNhc2UgaW4gYWxsIDMgYnJpY2tzIG9mIHRoZSByZXBsaWNhPwo+ PiBCVFcsIHlvdSBjYW4ganVzdCBzdGF0IHRoZSBmaWxlIG9uIHRoZSBicmljayBhbmQgc2VlIHRo ZSBsaW5rIGNvdW50IAo+PiAoaXQgbXVzdCBiZSAyKSBpbnN0ZWFkIG9mIHJ1bm5pbmcgdGhlIG1v cmUgZXhwZW5zaXZlIGZpbmQgY29tbWFuZC4KPj4KPgoKCgotLS0tLS0tLS0tLS0tLTAxMDEwMzA1 MDQwMjAyMDUwNTA4MDMwNgpDb250ZW50LVR5cGU6IHRleHQvaHRtbDsgY2hhcnNldD11dGYtOApD b250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0Cgo8aHRtbD4KICA8aGVhZD4KICAgIDxtZXRh IGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCIgaHR0cC1lcXVpdj0iQ29udGVudC1U eXBlIj4KICA8L2hlYWQ+CiAgPGJvZHkgdGV4dD0iIzAwMDAwMCIgYmdjb2xvcj0iI0ZGRkZGRiI+ CiAgICA8ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPk9uIDAzLzAzLzIwMTYgMDI6NTMgUE0s IDxhIGNsYXNzPSJtb3otdHh0LWxpbmstYWJicmV2aWF0ZWQiIGhyZWY9Im1haWx0bzpwYWYxQGVt YWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPgogICAgICB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAg IDxibG9ja3F1b3RlIGNpdGU9Im1pZDo1NkQ4MDI5Ny42MDgwNDA3QGVtYWlsLmN6IiB0eXBlPSJj aXRlIj4KICAgICAgPG1ldGEgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04IiBodHRw LWVxdWl2PSJDb250ZW50LVR5cGUiPgogICAgICBUaGlzIGlzIHJlcGxpY2EgMiwgb25seSAsIHdp dGggZm9sbG93aW5nIHNldHRpbmdzPGJyPgogICAgICA8YnI+CiAgICAgIE9wdGlvbnMgUmVjb25m aWd1cmVkOjxicj4KICAgICAgcGVyZm9ybWFuY2UucXVpY2stcmVhZDogb2ZmPGJyPgogICAgICBw ZXJmb3JtYW5jZS5yZWFkLWFoZWFkOiBvZmY8YnI+CiAgICAgIHBlcmZvcm1hbmNlLmlvLWNhY2hl OiBvZmY8YnI+CiAgICAgIHBlcmZvcm1hbmNlLnN0YXQtcHJlZmV0Y2g6IG9mZjxicj4KICAgICAg Y2x1c3Rlci5lYWdlci1sb2NrOiBlbmFibGU8YnI+CiAgICAgIG5ldHdvcmsucmVtb3RlLWRpbzog ZW5hYmxlPGJyPgogICAgICBjbHVzdGVyLnF1b3J1bS10eXBlOiBmaXhlZDxicj4KICAgIDwvYmxv Y2txdW90ZT4KICAgIE5vdCBzdXJlIHdoeSB5b3UgaGF2ZSBzZXQgdGhpcyBvcHRpb24uPGJyPgog ICAgSWRlYWxseSByZXBsaWNhIDMgb3IgYXJiaXRlciB2b2x1bWVzIGFyZSByZWNvbW1lbmRlZCBm b3IKICAgIGdsdXN0ZXIrb3ZpcnQgdXNlLsKgIChjbGllbnQpIHF1b3J1bSBkb2VzIG5vdCBtYWtl IHNlbnNlIGZvciBhIDIgbm9kZQogICAgc2V0dXAuIEkgaGF2ZSBhIGRldGFpbGVkIHdyaXRlIHVw IGhlcmUgd2hpY2ggZXhwbGFpbnMgdGhpbmdzCiAgICA8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZy ZWV0ZXh0IiBocmVmPSJodHRwOi8vZ2x1c3Rlci5yZWFkdGhlZG9jcy5vcmcvZW4vbGF0ZXN0L0Fk bWluaXN0cmF0b3IlMjBHdWlkZS9hcmJpdGVyLXZvbHVtZXMtYW5kLXF1b3J1bS8iPmh0dHA6Ly9n bHVzdGVyLnJlYWR0aGVkb2NzLm9yZy9lbi9sYXRlc3QvQWRtaW5pc3RyYXRvciUyMEd1aWRlL2Fy Yml0ZXItdm9sdW1lcy1hbmQtcXVvcnVtLzwvYT4KICAgIHdoaWNoIGV4cGxhaW5zIHRoaW5ncy48 YnI+CiAgICA8YnI+CiAgICA8YmxvY2txdW90ZSBjaXRlPSJtaWQ6NTZEODAyOTcuNjA4MDQwN0Bl bWFpbC5jeiIgdHlwZT0iY2l0ZSI+CiAgICAgIGNsdXN0ZXIuc2VydmVyLXF1b3J1bS10eXBlOiBu b25lPGJyPgogICAgICBzdG9yYWdlLm93bmVyLXVpZDogMzY8YnI+CiAgICAgIHN0b3JhZ2Uub3du ZXItZ2lkOiAzNjxicj4KICAgICAgY2x1c3Rlci5xdW9ydW0tY291bnQ6IDE8YnI+CiAgICAgIGNs dXN0ZXIuc2VsZi1oZWFsLWRhZW1vbjogZW5hYmxlPGJyPgogICAgICA8YnI+CiAgICAgIElmIEkn bGwgY3JlYXRlICJpZHMiIGZpbGUgbWFudWFsbHkgKMKgIGVnLiAiIHNhbmxvY2sgZGlyZWN0IGlu aXQgLXMKICAgICAgM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3OjA6L1NUT1JB R0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9t ZC9pZHM6MAoKICAgICAgIiApIG9uIGJvdGggYnJpY2tzLDxicj4KICAgICAgdmRzbSBpcyB3cml0 aW5nIG9ubHkgdG8gaGFsZiBvZiB0aGVtICggdGhhdCB3aXRoIDIgbGlua3MgPSBjb3JyZWN0CiAg ICAgICk8YnI+CiAgICAgICJpZHMiIGZpbGUgaGFzIGNvcnJlY3QgcGVybWl0dGlvbnMsIG93bmVy LCBzaXplwqAgb24gYm90aCBicmlja3MuPGJyPgogICAgICBicmljayAxOsKgIC1ydy1ydy0tLS0g MSB2ZHNtIGt2bSAxMDQ4NTc2wqAgMi7CoGLFmWUgMTguNTYKICAgICAgL1NUT1JBR0VTL2cxcjVw My9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZC9pZHMKICAg ICAgLSBub3QgdXBkYXRlZDxicj4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxicj4KICAgIE9rYXks IHNvIHRoaXMgb25lIGhhcyBsaW5rIGNvdW50ID0xIHdoaWNoIG1lYW5zIHRoZSAuZ2x1c3RlcmZz CiAgICBoYXJkbGluayBpcyBtaXNzaW5nLsKgIENhbiB5b3UgdHJ5IGRlbGV0aW5nIHRoaXMgZmls ZSBmcm9tIHRoZSBicmljawogICAgYW5kIHBlcmZvcm0gYSBzdGF0IG9uIHRoZSBmaWxlIGZyb20g dGhlIG1vdW50PyBUaGF0IHNob3VsZCBoZWFsIChpLmUKICAgIHJlY3JlYXRlIGl0ICkgb24gdGhp cyBicmljayBmcm9tIHRoZSBvdGhlciBicmljayB3aXRoIHRoZQogICAgYXBwcm9wcmlhdGUgLmds dXN0ZXJmcyBoYXJkIGxpbmsuPGJyPgogICAgPGJyPgogICAgPGJyPgogICAgPGJsb2NrcXVvdGUg Y2l0ZT0ibWlkOjU2RDgwMjk3LjYwODA0MDdAZW1haWwuY3oiIHR5cGU9ImNpdGUiPiBicmljawog ICAgICAyOsKgIC1ydy1ydy0tLS0gMiB2ZHNtIGt2bSAxMDQ4NTc2wqAgMy7CoGLFmWUgMTAuMTYK ICAgICAgL1NUT1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNk NzBiMTQ3L2RvbV9tZC9pZHMKICAgICAgLSBpcyBjb250aW51YWxseSB1cGRhdGVkPGJyPgogICAg ICA8YnI+CiAgICAgIFdoYXQgaGFwcGVucyB3aGVuIEknbGwgcmVzdGFydCB2ZHNtID8gV2lsbCBv VmlydCBzdG9yYWdlcyBnbyB0bwogICAgICAiZGlzYWJsZSAiIHN0YXRlID8/PyA9IGRpc2Nvbm5l Y3QgVk1zIHN0b3JhZ2VzID88YnI+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+CiAgICBObyBp ZGVhIG9uIHRoaXMgb25lLi4uPGJyPgogICAgLVJhdmk8YnI+CiAgICA8YmxvY2txdW90ZSBjaXRl PSJtaWQ6NTZEODAyOTcuNjA4MDQwN0BlbWFpbC5jeiIgdHlwZT0iY2l0ZSI+IDxicj4KICAgICAg cmVncy5QYS48YnI+CiAgICAgIDxicj4KICAgICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4 Ij5PbiAzLjMuMjAxNiAwMjowMiwgUmF2aXNoYW5rYXIgTgogICAgICAgIHdyb3RlOjxicj4KICAg ICAgPC9kaXY+CiAgICAgIDxibG9ja3F1b3RlIGNpdGU9Im1pZDo1NkQ3OEQzQi45MDYwMTAxQHJl ZGhhdC5jb20iIHR5cGU9ImNpdGUiPgogICAgICAgIDxtZXRhIGNvbnRlbnQ9InRleHQvaHRtbDsg Y2hhcnNldD11dGYtOCIKICAgICAgICAgIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAgICAg ICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5PbiAwMy8wMy8yMDE2IDEyOjQzIEFNLCBO aXIgU29mZmVyCiAgICAgICAgICB3cm90ZTo8YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGJs b2NrcXVvdGUKY2l0ZT0ibWlkOkNBTVJieXlzRnFKd2tlOHZkbVlMN05HVXV6cEZOSnBwWkMzcmlM UjhqQzdwZDMzRFZkUUBtYWlsLmdtYWlsLmNvbSIKICAgICAgICAgIHR5cGU9ImNpdGUiPgogICAg ICAgICAgPGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAw CiAgICAgICAgICAgIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0 OjFleCI+CiAgICAgICAgICAgIDxkaXYgdGV4dD0iIzAwMDA2NiIgYmdjb2xvcj0iI0ZGRkZGRiI+ UFM6wqAgIyBmaW5kIC9TVE9SQUdFUwogICAgICAgICAgICAgIC1zYW1lZmlsZQogICAgICAgICAg ICAgIC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcw YjE0Ny9kb21fbWQvaWRzCiAgICAgICAgICAgICAgLXByaW50PGJyPgovU1RPUkFHRVMvZzFyNXAz L0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczxicj4K ICAgICAgICAgICAgICA9IG1pc3NpbmcgInNoYWRvd2ZpbGUiIGluICIgLmdsdXN0ZXIgIiBkaXIu PGJyPgogICAgICAgICAgICAgIEhvdyBjYW4gSSBmaXggaXQgPz8gLSBvbmxpbmUgITwvZGl2Pgog ICAgICAgICAgPC9ibG9ja3F1b3RlPgogICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICA8L2Rp dj4KICAgICAgICAgIDxkaXY+UmF2aT88L2Rpdj4KICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAg ICAgSXMgdGhpcyB0aGUgY2FzZSBpbiBhbGwgMyBicmlja3Mgb2YgdGhlIHJlcGxpY2E/IDxicj4K ICAgICAgICBCVFcsIHlvdSBjYW4ganVzdCBzdGF0IHRoZSBmaWxlIG9uIHRoZSBicmljayBhbmQg c2VlIHRoZSBsaW5rCiAgICAgICAgY291bnQgKGl0IG11c3QgYmUgMikgaW5zdGVhZCBvZiBydW5u aW5nIHRoZSBtb3JlIGV4cGVuc2l2ZSBmaW5kCiAgICAgICAgY29tbWFuZC48YnI+CiAgICAgICAg PGJyPgogICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgIDxicj4KICAgIDwvYmxvY2txdW90ZT4KICAg IDxicj4KICAgIDxicj4KICA8L2JvZHk+CjwvaHRtbD4KCi0tLS0tLS0tLS0tLS0tMDEwMTAzMDUw NDAyMDIwNTA1MDgwMzA2LS0K --===============5045971417505254478==-- From nsoffer at redhat.com Thu Mar 3 06:19:11 2016 Content-Type: multipart/mixed; boundary="===============8173604615685052876==" MIME-Version: 1.0 From: Nir Soffer To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Thu, 03 Mar 2016 13:19:09 +0200 Message-ID: In-Reply-To: 56D80297.6080407@email.cz --===============8173604615685052876== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Thu, Mar 3, 2016 at 11:23 AM, paf1(a)email.cz wrote: > This is replica 2, only , with following settings > Replica 2 is not supported. Even if you "fix" this now, you will have the same issue soon. > > Options Reconfigured: > performance.quick-read: off > performance.read-ahead: off > performance.io-cache: off > performance.stat-prefetch: off > cluster.eager-lock: enable > network.remote-dio: enable > cluster.quorum-type: fixed > cluster.server-quorum-type: none > storage.owner-uid: 36 > storage.owner-gid: 36 > cluster.quorum-count: 1 > cluster.self-heal-daemon: enable > > If I'll create "ids" file manually ( eg. " sanlock direct init -s > 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-6c66= -4e23-ab46-084f3d70b147/dom_md/ids:0 > " ) on both bricks, > vdsm is writing only to half of them ( that with 2 links =3D correct ) > "ids" file has correct permittions, owner, size on both bricks. > brick 1: -rw-rw---- 1 vdsm kvm 1048576 2. b=C5=99e 18.56 > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - not > updated > brick 2: -rw-rw---- 2 vdsm kvm 1048576 3. b=C5=99e 10.16 > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids - is > continually updated > > What happens when I'll restart vdsm ? Will oVirt storages go to "disable " > state ??? =3D disconnect VMs storages ? > Nothing will happen, the vms will continue to run normally. On block storage, stopping vdsm will prevent automatic extending of vm disks when the disk become too full, but on file based storage (like gluster) there is no issue. > > regs.Pa. > > > On 3.3.2016 02:02, Ravishankar N wrote: > > On 03/03/2016 12:43 AM, Nir Soffer wrote: > > PS: # find /STORAGES -samefile >> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids -pr= int >> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids >> =3D missing "shadowfile" in " .gluster " dir. >> How can I fix it ?? - online ! >> > > Ravi? > > Is this the case in all 3 bricks of the replica? > BTW, you can just stat the file on the brick and see the link count (it > must be 2) instead of running the more expensive find command. > > > --===============8173604615685052876== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9Imx0ciI+PGRpdiBjbGFzcz0iZ21haWxfZXh0cmEiPjxkaXYgY2xhc3M9ImdtYWls X3F1b3RlIj5PbiBUaHUsIE1hciAzLCAyMDE2IGF0IDExOjIzIEFNLCA8YSBocmVmPSJtYWlsdG86 cGFmMUBlbWFpbC5jeiI+cGFmMUBlbWFpbC5jejwvYT4gPHNwYW4gZGlyPSJsdHIiPiZsdDs8YSBo cmVmPSJtYWlsdG86cGFmMUBlbWFpbC5jeiIgdGFyZ2V0PSJfYmxhbmsiPnBhZjFAZW1haWwuY3o8 L2E+Jmd0Ozwvc3Bhbj4gd3JvdGU6PGJyPjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIg c3R5bGU9Im1hcmdpbjowIDAgMCAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRp bmctbGVmdDoxZXgiPgogIAogICAgCiAgCiAgPGRpdiB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIj RkZGRkZGIj4KICAgIFRoaXMgaXMgcmVwbGljYSAyLCBvbmx5ICwgd2l0aCBmb2xsb3dpbmcgc2V0 dGluZ3M8YnI+PC9kaXY+PC9ibG9ja3F1b3RlPjxkaXY+PGJyPjwvZGl2PjxkaXY+UmVwbGljYSAy IGlzIG5vdCBzdXBwb3J0ZWQuIEV2ZW4gaWYgeW91ICZxdW90O2ZpeCZxdW90OyB0aGlzIG5vdywg eW91IHdpbGwgaGF2ZSB0aGUgc2FtZSBpc3N1ZTwvZGl2PjxkaXY+c29vbi48L2Rpdj48ZGl2PsKg PC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAw IC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiB0 ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIDxicj4KICAgIE9wdGlvbnMgUmVj b25maWd1cmVkOjxicj4KICAgIHBlcmZvcm1hbmNlLnF1aWNrLXJlYWQ6IG9mZjxicj4KICAgIHBl cmZvcm1hbmNlLnJlYWQtYWhlYWQ6IG9mZjxicj4KICAgIHBlcmZvcm1hbmNlLmlvLWNhY2hlOiBv ZmY8YnI+CiAgICBwZXJmb3JtYW5jZS5zdGF0LXByZWZldGNoOiBvZmY8YnI+CiAgICBjbHVzdGVy LmVhZ2VyLWxvY2s6IGVuYWJsZTxicj4KICAgIG5ldHdvcmsucmVtb3RlLWRpbzogZW5hYmxlPGJy PgogICAgY2x1c3Rlci5xdW9ydW0tdHlwZTogZml4ZWQ8YnI+CiAgICBjbHVzdGVyLnNlcnZlci1x dW9ydW0tdHlwZTogbm9uZTxicj4KICAgIHN0b3JhZ2Uub3duZXItdWlkOiAzNjxicj4KICAgIHN0 b3JhZ2Uub3duZXItZ2lkOiAzNjxicj4KICAgIGNsdXN0ZXIucXVvcnVtLWNvdW50OiAxPGJyPgog ICAgY2x1c3Rlci5zZWxmLWhlYWwtZGFlbW9uOiBlbmFibGU8YnI+CiAgICA8YnI+CiAgICBJZiBJ JiMzOTtsbCBjcmVhdGUgJnF1b3Q7aWRzJnF1b3Q7IGZpbGUgbWFudWFsbHkgKMKgIGVnLiAmcXVv dDsgc2FubG9jayBkaXJlY3QgaW5pdCAtcwozYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3 MGIxNDc6MDovU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRm M2Q3MGIxNDcvZG9tX21kL2lkczowCiAgICAmcXVvdDsgKSBvbiBib3RoIGJyaWNrcyw8YnI+CiAg ICB2ZHNtIGlzIHdyaXRpbmcgb25seSB0byBoYWxmIG9mIHRoZW0gKCB0aGF0IHdpdGggMiBsaW5r cyA9IGNvcnJlY3QgKTxicj4KICAgICZxdW90O2lkcyZxdW90OyBmaWxlIGhhcyBjb3JyZWN0IHBl cm1pdHRpb25zLCBvd25lciwgc2l6ZcKgIG9uIGJvdGggYnJpY2tzLjxicj4KICAgIGJyaWNrIDE6 wqAgLXJ3LXJ3LS0tLSAxIHZkc20ga3ZtIDEwNDg1NzbCoCAyLsKgYsWZZSAxOC41NgogICAgL1NU T1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2Rv bV9tZC9pZHMKICAgIC0gbm90IHVwZGF0ZWQ8YnI+CiAgICBicmljayAyOsKgIC1ydy1ydy0tLS0g MiB2ZHNtIGt2bSAxMDQ4NTc2wqAgMy7CoGLFmWUgMTAuMTYKICAgIC9TVE9SQUdFUy9nMXI1cDMv R0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCiAgICAt IGlzIGNvbnRpbnVhbGx5IHVwZGF0ZWQ8YnI+CiAgICA8YnI+CiAgICBXaGF0IGhhcHBlbnMgd2hl biBJJiMzOTtsbCByZXN0YXJ0IHZkc20gPyBXaWxsIG9WaXJ0IHN0b3JhZ2VzIGdvIHRvCiAgICAm cXVvdDtkaXNhYmxlICZxdW90OyBzdGF0ZSA/Pz8gPSBkaXNjb25uZWN0IFZNcyBzdG9yYWdlcyA/ PGJyPjwvZGl2PjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PsKgTm90aGluZyB3aWxs IGhhcHBlbiwgdGhlIHZtcyB3aWxsIGNvbnRpbnVlIHRvIHJ1biBub3JtYWxseS48L2Rpdj48ZGl2 Pjxicj48L2Rpdj48ZGl2Pk9uIGJsb2NrIHN0b3JhZ2UsIHN0b3BwaW5nIHZkc20gd2lsbCBwcmV2 ZW50IGF1dG9tYXRpYyBleHRlbmRpbmcgb2Ygdm0gZGlza3M8L2Rpdj48ZGl2PndoZW4gdGhlIGRp c2sgYmVjb21lIHRvbyBmdWxsLCBidXQgb24gZmlsZSBiYXNlZCBzdG9yYWdlIChsaWtlIGdsdXN0 ZXIpIHRoZXJlIGlzIG5vIGlzc3VlLjwvZGl2PjxkaXY+wqA8L2Rpdj48YmxvY2txdW90ZSBjbGFz cz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAgLjhleDtib3JkZXItbGVmdDoxcHgg I2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IHRleHQ9IiMwMDAwNjYiIGJnY29sb3I9 IiNGRkZGRkYiPgogICAgPGJyPgogICAgcmVncy5QYS48ZGl2PjxkaXYgY2xhc3M9Img1Ij48YnI+ CiAgICA8YnI+CiAgICA8ZGl2Pk9uIDMuMy4yMDE2IDAyOjAyLCBSYXZpc2hhbmthciBOIHdyb3Rl Ojxicj4KICAgIDwvZGl2PgogICAgPGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+CiAgICAgIAogICAg ICA8ZGl2Pk9uIDAzLzAzLzIwMTYgMTI6NDMgQU0sIE5pciBTb2ZmZXIKICAgICAgICB3cm90ZTo8 YnI+CiAgICAgIDwvZGl2PgogICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4KICAgICAgICA8 YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAgLjhleDti b3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4KICAgICAgICAgIDxk aXYgdGV4dD0iIzAwMDA2NiIgYmdjb2xvcj0iI0ZGRkZGRiI+UFM6wqAgIyBmaW5kIC9TVE9SQUdF UwogICAgICAgICAgICAtc2FtZWZpbGUKICAgICAgICAgICAgL1NUT1JBR0VTL2cxcjVwMy9HRlMv M2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZC9pZHMKICAgICAgICAg ICAgLXByaW50PGJyPgovU1RPUkFHRVMvZzFyNXAzL0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0 Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczxicj4KICAgICAgICAgICAgPSBtaXNzaW5nICZxdW90 O3NoYWRvd2ZpbGUmcXVvdDsgaW4gJnF1b3Q7IC5nbHVzdGVyICZxdW90OyBkaXIuPGJyPgogICAg ICAgICAgICBIb3cgY2FuIEkgZml4IGl0ID8/IC0gb25saW5lICE8L2Rpdj4KICAgICAgICA8L2Js b2NrcXVvdGU+CiAgICAgICAgPGRpdj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRpdj5S YXZpPzwvZGl2PgogICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgIElzIHRoaXMgdGhlIGNhc2UgaW4g YWxsIDMgYnJpY2tzIG9mIHRoZSByZXBsaWNhPyA8YnI+CiAgICAgIEJUVywgeW91IGNhbiBqdXN0 IHN0YXQgdGhlIGZpbGUgb24gdGhlIGJyaWNrIGFuZCBzZWUgdGhlIGxpbmsKICAgICAgY291bnQg KGl0IG11c3QgYmUgMikgaW5zdGVhZCBvZiBydW5uaW5nIHRoZSBtb3JlIGV4cGVuc2l2ZSBmaW5k CiAgICAgIGNvbW1hbmQuPGJyPgogICAgICA8YnI+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8YnI+ CiAgPC9kaXY+PC9kaXY+PC9kaXY+Cgo8L2Jsb2NrcXVvdGU+PC9kaXY+PGJyPjwvZGl2PjwvZGl2 Pgo= --===============8173604615685052876==-- From paf1 at email.cz Thu Mar 3 10:12:10 2016 Content-Type: multipart/mixed; boundary="===============2163385157278793131==" MIME-Version: 1.0 From: paf1 at email.cz To: users at ovirt.org Subject: Re: [ovirt-users] [Gluster-users] open error -13 = sanlock Date: Thu, 03 Mar 2016 16:12:03 +0100 Message-ID: <56D85443.1050405@email.cz> In-Reply-To: CAMRbyyth7DEGXAGnbpxtNX73RBK73e7_Yb_z-wpbnkZHX=yMSQ@mail.gmail.com --===============2163385157278793131== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------050603070709080002010605 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit OK, will extend replica 2 to replica 3 ( arbiter ) ASAP . If is deleted "untouching" ids file on brick , healing of this file = doesn't work . regs.Pa. On 3.3.2016 12:19, Nir Soffer wrote: > On Thu, Mar 3, 2016 at 11:23 AM, paf1(a)email.cz = > > wrote: > > This is replica 2, only , with following settings > > > Replica 2 is not supported. Even if you "fix" this now, you will have = > the same issue > soon. > > > Options Reconfigured: > performance.quick-read: off > performance.read-ahead: off > performance.io-cache: off > performance.stat-prefetch: off > cluster.eager-lock: enable > network.remote-dio: enable > cluster.quorum-type: fixed > cluster.server-quorum-type: none > storage.owner-uid: 36 > storage.owner-gid: 36 > cluster.quorum-count: 1 > cluster.self-heal-daemon: enable > > If I'll create "ids" file manually ( eg. " sanlock direct init -s > 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS/3c34ad63-= 6c66-4e23-ab46-084f3d70b147/dom_md/ids:0 > " ) on both bricks, > vdsm is writing only to half of them ( that with 2 links =3D correct ) > "ids" file has correct permittions, owner, size on both bricks. > brick 1: -rw-rw---- 1 vdsm kvm 1048576 2. b=C5=99e 18.56 > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids > - not updated > brick 2: -rw-rw---- 2 vdsm kvm 1048576 3. b=C5=99e 10.16 > /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids > - is continually updated > > What happens when I'll restart vdsm ? Will oVirt storages go to > "disable " state ??? =3D disconnect VMs storages ? > > > Nothing will happen, the vms will continue to run normally. > > On block storage, stopping vdsm will prevent automatic extending of vm = > disks > when the disk become too full, but on file based storage (like = > gluster) there is no issue. > > > regs.Pa. > > > On 3.3.2016 02:02, Ravishankar N wrote: >> On 03/03/2016 12:43 AM, Nir Soffer wrote: >>> >>> PS: # find /STORAGES -samefile >>> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_m= d/ids >>> -print >>> /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_m= d/ids >>> =3D missing "shadowfile" in " .gluster " dir. >>> How can I fix it ?? - online ! >>> >>> >>> Ravi? >> Is this the case in all 3 bricks of the replica? >> BTW, you can just stat the file on the brick and see the link >> count (it must be 2) instead of running the more expensive find >> command. >> > > --------------050603070709080002010605 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit OK,
will extend replica 2 to replica 3 ( arbiter )=C2=A0 ASAP .

If is deleted "untouching" ids file on brick , healing of this file doesn't work .

regs.Pa.

On 3.3.2016 12:19, Nir Soffer wrote:

On Thu, Mar 3, 2016 at 11:23 AM, paf1= (a)email.cz <paf1(a)em= ail.cz> wrote:

This is replica 2, only , with following settings

Replica 2 is not supported. Even if you "fix" this now, you will have the same issue

soon.

=C2=A0

Options Reconfigured:
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.quorum-type: fixed
cluster.server-quorum-type: none
storage.owner-uid: 36
storage.owner-gid: 36
cluster.quorum-count: 1
cluster.self-heal-daemon: enable

If I'll create "ids" file manually (=C2=A0 eg. " sanlock direct init -s 3c34ad63-6c66-4e23-ab46-084f3d70b147:0:/STORAGES/g1r5p3/GFS= /3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids:0 " ) on both bricks,
vdsm is writing only to half of them ( that with 2 links =3D correct )
"ids" file has correct permittions, owner, size=C2=A0 on bo= th bricks.
brick 1:=C2=A0 -rw-rw---- 1 vdsm kvm 1048576=C2=A0 2.=C2=A0= b=C5=99e 18.56 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/d= om_md/ids - not updated
brick 2:=C2=A0 -rw-rw---- 2 vdsm kvm 1048576=C2=A0 3.=C2=A0= b=C5=99e 10.16 /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/d= om_md/ids - is continually updated

What happens when I'll restart vdsm ? Will oVirt storages go to "disable " state ??? =3D disconnect VMs storages ?

=C2=A0Nothing will happen, the vms will continue to run normally.

On block storage, stopping vdsm will prevent automatic extending of vm disks

when the disk become too full, but on file based storage (like gluster) there is no issue.

=C2=A0

regs.Pa.

On 3.3.2016 02:02, Ravishankar N wrote:

On 03/03/2016 12:43 AM, Nir Soffer wrote:

PS:=C2= =A0 # find /STORAGES -samefile /STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-08= 4f3d70b147/dom_md/ids -print
/STORAGES/g1r5p3/GFS/3c34ad63-6c66-4e23-ab46-084f3d70b147/dom_md/ids
=3D missing "shadowfile" in " .gluster " dir. How can I fix it ?? - online !

Ravi?

Is this the case in all 3 bricks of the replica?
BTW, you can just stat the file on the brick and see the link count (it must be 2) instead of running the more expensive find command.

--------------050603070709080002010605-- --===============2163385157278793131== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wNTA2MDMwNzA3MDkwODAwMDIwMTA2MDUKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK Ck9LLAp3aWxsIGV4dGVuZCByZXBsaWNhIDIgdG8gcmVwbGljYSAzICggYXJiaXRlciApICBBU0FQ IC4KCklmIGlzIGRlbGV0ZWQgInVudG91Y2hpbmciIGlkcyBmaWxlIG9uIGJyaWNrICwgaGVhbGlu ZyBvZiB0aGlzIGZpbGUgCmRvZXNuJ3Qgd29yayAuCgpyZWdzLlBhLgoKT24gMy4zLjIwMTYgMTI6 MTksIE5pciBTb2ZmZXIgd3JvdGU6Cj4gT24gVGh1LCBNYXIgMywgMjAxNiBhdCAxMToyMyBBTSwg cGFmMUBlbWFpbC5jeiA8bWFpbHRvOnBhZjFAZW1haWwuY3o+IAo+IDxwYWYxQGVtYWlsLmN6IDxt YWlsdG86cGFmMUBlbWFpbC5jej4+IHdyb3RlOgo+Cj4gICAgIFRoaXMgaXMgcmVwbGljYSAyLCBv bmx5ICwgd2l0aCBmb2xsb3dpbmcgc2V0dGluZ3MKPgo+Cj4gUmVwbGljYSAyIGlzIG5vdCBzdXBw b3J0ZWQuIEV2ZW4gaWYgeW91ICJmaXgiIHRoaXMgbm93LCB5b3Ugd2lsbCBoYXZlIAo+IHRoZSBz YW1lIGlzc3VlCj4gc29vbi4KPgo+Cj4gICAgIE9wdGlvbnMgUmVjb25maWd1cmVkOgo+ICAgICBw ZXJmb3JtYW5jZS5xdWljay1yZWFkOiBvZmYKPiAgICAgcGVyZm9ybWFuY2UucmVhZC1haGVhZDog b2ZmCj4gICAgIHBlcmZvcm1hbmNlLmlvLWNhY2hlOiBvZmYKPiAgICAgcGVyZm9ybWFuY2Uuc3Rh dC1wcmVmZXRjaDogb2ZmCj4gICAgIGNsdXN0ZXIuZWFnZXItbG9jazogZW5hYmxlCj4gICAgIG5l dHdvcmsucmVtb3RlLWRpbzogZW5hYmxlCj4gICAgIGNsdXN0ZXIucXVvcnVtLXR5cGU6IGZpeGVk Cj4gICAgIGNsdXN0ZXIuc2VydmVyLXF1b3J1bS10eXBlOiBub25lCj4gICAgIHN0b3JhZ2Uub3du ZXItdWlkOiAzNgo+ICAgICBzdG9yYWdlLm93bmVyLWdpZDogMzYKPiAgICAgY2x1c3Rlci5xdW9y dW0tY291bnQ6IDEKPiAgICAgY2x1c3Rlci5zZWxmLWhlYWwtZGFlbW9uOiBlbmFibGUKPgo+ICAg ICBJZiBJJ2xsIGNyZWF0ZSAiaWRzIiBmaWxlIG1hbnVhbGx5ICggIGVnLiAiIHNhbmxvY2sgZGly ZWN0IGluaXQgLXMKPiAgICAgM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3OjA6 L1NUT1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3 L2RvbV9tZC9pZHM6MAo+ICAgICAiICkgb24gYm90aCBicmlja3MsCj4gICAgIHZkc20gaXMgd3Jp dGluZyBvbmx5IHRvIGhhbGYgb2YgdGhlbSAoIHRoYXQgd2l0aCAyIGxpbmtzID0gY29ycmVjdCAp Cj4gICAgICJpZHMiIGZpbGUgaGFzIGNvcnJlY3QgcGVybWl0dGlvbnMsIG93bmVyLCBzaXplICBv biBib3RoIGJyaWNrcy4KPiAgICAgYnJpY2sgMTogIC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAxMDQ4 NTc2ICAyLiBixZllIDE4LjU2Cj4gICAgIC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZj NjYtNGUyMy1hYjQ2LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCj4gICAgIC0gbm90IHVwZGF0ZWQK PiAgICAgYnJpY2sgMjogIC1ydy1ydy0tLS0gMiB2ZHNtIGt2bSAxMDQ4NTc2ICAzLiBixZllIDEw LjE2Cj4gICAgIC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2LTA4 NGYzZDcwYjE0Ny9kb21fbWQvaWRzCj4gICAgIC0gaXMgY29udGludWFsbHkgdXBkYXRlZAo+Cj4g ICAgIFdoYXQgaGFwcGVucyB3aGVuIEknbGwgcmVzdGFydCB2ZHNtID8gV2lsbCBvVmlydCBzdG9y YWdlcyBnbyB0bwo+ICAgICAiZGlzYWJsZSAiIHN0YXRlID8/PyA9IGRpc2Nvbm5lY3QgVk1zIHN0 b3JhZ2VzID8KPgo+Cj4gIE5vdGhpbmcgd2lsbCBoYXBwZW4sIHRoZSB2bXMgd2lsbCBjb250aW51 ZSB0byBydW4gbm9ybWFsbHkuCj4KPiBPbiBibG9jayBzdG9yYWdlLCBzdG9wcGluZyB2ZHNtIHdp bGwgcHJldmVudCBhdXRvbWF0aWMgZXh0ZW5kaW5nIG9mIHZtIAo+IGRpc2tzCj4gd2hlbiB0aGUg ZGlzayBiZWNvbWUgdG9vIGZ1bGwsIGJ1dCBvbiBmaWxlIGJhc2VkIHN0b3JhZ2UgKGxpa2UgCj4g Z2x1c3RlcikgdGhlcmUgaXMgbm8gaXNzdWUuCj4KPgo+ICAgICByZWdzLlBhLgo+Cj4KPiAgICAg T24gMy4zLjIwMTYgMDI6MDIsIFJhdmlzaGFua2FyIE4gd3JvdGU6Cj4+ICAgICBPbiAwMy8wMy8y MDE2IDEyOjQzIEFNLCBOaXIgU29mZmVyIHdyb3RlOgo+Pj4KPj4+ICAgICAgICAgUFM6ICAjIGZp bmQgL1NUT1JBR0VTIC1zYW1lZmlsZQo+Pj4gICAgICAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8z YzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcwo+Pj4gICAgICAg ICAtcHJpbnQKPj4+ICAgICAgICAgL1NUT1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00 ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3L2RvbV9tZC9pZHMKPj4+ICAgICAgICAgPSBtaXNzaW5nICJz aGFkb3dmaWxlIiBpbiAiIC5nbHVzdGVyICIgZGlyLgo+Pj4gICAgICAgICBIb3cgY2FuIEkgZml4 IGl0ID8/IC0gb25saW5lICEKPj4+Cj4+Pgo+Pj4gICAgIFJhdmk/Cj4+ICAgICBJcyB0aGlzIHRo ZSBjYXNlIGluIGFsbCAzIGJyaWNrcyBvZiB0aGUgcmVwbGljYT8KPj4gICAgIEJUVywgeW91IGNh biBqdXN0IHN0YXQgdGhlIGZpbGUgb24gdGhlIGJyaWNrIGFuZCBzZWUgdGhlIGxpbmsKPj4gICAg IGNvdW50IChpdCBtdXN0IGJlIDIpIGluc3RlYWQgb2YgcnVubmluZyB0aGUgbW9yZSBleHBlbnNp dmUgZmluZAo+PiAgICAgY29tbWFuZC4KPj4KPgo+CgoKLS0tLS0tLS0tLS0tLS0wNTA2MDMwNzA3 MDkwODAwMDIwMTA2MDUKQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgKQ29u dGVudC1UcmFuc2Zlci1FbmNvZGluZzogOGJpdAoKPGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBj b250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlw ZSI+CiAgPC9oZWFkPgogIDxib2R5IHRleHQ9IiMwMDAwNjYiIGJnY29sb3I9IiNGRkZGRkYiPgog ICAgT0ssIDxicj4KICAgIHdpbGwgZXh0ZW5kIHJlcGxpY2EgMiB0byByZXBsaWNhIDMgKCBhcmJp dGVyICnCoCBBU0FQIC48YnI+CiAgICA8YnI+CiAgICBJZiBpcyBkZWxldGVkICJ1bnRvdWNoaW5n IiBpZHMgZmlsZSBvbiBicmljayAsIGhlYWxpbmcgb2YgdGhpcyBmaWxlCiAgICBkb2Vzbid0IHdv cmsgLjxicj4KICAgIDxicj4KICAgIHJlZ3MuUGEuPGJyPgogICAgPGJyPgogICAgPGRpdiBjbGFz cz0ibW96LWNpdGUtcHJlZml4Ij5PbiAzLjMuMjAxNiAxMjoxOSwgTmlyIFNvZmZlciB3cm90ZTo8 YnI+CiAgICA8L2Rpdj4KICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQU1SYnl5dGg3REVHWEFH bmJweHROWDczUkJLNzNlN19ZYl96LXdwYm5rWkhYPXlNU1FAbWFpbC5nbWFpbC5jb20iCiAgICAg IHR5cGU9ImNpdGUiPgogICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICA8ZGl2IGNsYXNzPSJn bWFpbF9leHRyYSI+CiAgICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gVGh1LCBN YXIgMywgMjAxNiBhdCAxMToyMyBBTSwgPGEKICAgICAgICAgICAgICBtb3otZG8tbm90LXNlbmQ9 InRydWUiIGhyZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6Ij5wYWYxQGVtYWlsLmN6PC9hPgogICAg ICAgICAgICA8c3BhbiBkaXI9Imx0ciI+Jmx0OzxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAg ICAgICAgICAgICAgIGhyZWY9Im1haWx0bzpwYWYxQGVtYWlsLmN6IiB0YXJnZXQ9Il9ibGFuayI+ cGFmMUBlbWFpbC5jejwvYT4mZ3Q7PC9zcGFuPgogICAgICAgICAgICB3cm90ZTo8YnI+CiAgICAg ICAgICAgIDxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAg MAogICAgICAgICAgICAgIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1s ZWZ0OjFleCI+CiAgICAgICAgICAgICAgPGRpdiB0ZXh0PSIjMDAwMDY2IiBiZ2NvbG9yPSIjRkZG RkZGIj4gVGhpcyBpcyByZXBsaWNhIDIsCiAgICAgICAgICAgICAgICBvbmx5ICwgd2l0aCBmb2xs b3dpbmcgc2V0dGluZ3M8YnI+CiAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDwvYmxv Y2txdW90ZT4KICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAg ICAgICA8ZGl2PlJlcGxpY2EgMiBpcyBub3Qgc3VwcG9ydGVkLiBFdmVuIGlmIHlvdSAiZml4IiB0 aGlzIG5vdywKICAgICAgICAgICAgICB5b3Ugd2lsbCBoYXZlIHRoZSBzYW1lIGlzc3VlPC9kaXY+ CiAgICAgICAgICAgIDxkaXY+c29vbi48L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoDwvZGl2Pgog ICAgICAgICAgICA8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46 MCAwIDAKICAgICAgICAgICAgICAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRp bmctbGVmdDoxZXgiPgogICAgICAgICAgICAgIDxkaXYgdGV4dD0iIzAwMDA2NiIgYmdjb2xvcj0i I0ZGRkZGRiI+IDxicj4KICAgICAgICAgICAgICAgIE9wdGlvbnMgUmVjb25maWd1cmVkOjxicj4K ICAgICAgICAgICAgICAgIHBlcmZvcm1hbmNlLnF1aWNrLXJlYWQ6IG9mZjxicj4KICAgICAgICAg ICAgICAgIHBlcmZvcm1hbmNlLnJlYWQtYWhlYWQ6IG9mZjxicj4KICAgICAgICAgICAgICAgIHBl cmZvcm1hbmNlLmlvLWNhY2hlOiBvZmY8YnI+CiAgICAgICAgICAgICAgICBwZXJmb3JtYW5jZS5z dGF0LXByZWZldGNoOiBvZmY8YnI+CiAgICAgICAgICAgICAgICBjbHVzdGVyLmVhZ2VyLWxvY2s6 IGVuYWJsZTxicj4KICAgICAgICAgICAgICAgIG5ldHdvcmsucmVtb3RlLWRpbzogZW5hYmxlPGJy PgogICAgICAgICAgICAgICAgY2x1c3Rlci5xdW9ydW0tdHlwZTogZml4ZWQ8YnI+CiAgICAgICAg ICAgICAgICBjbHVzdGVyLnNlcnZlci1xdW9ydW0tdHlwZTogbm9uZTxicj4KICAgICAgICAgICAg ICAgIHN0b3JhZ2Uub3duZXItdWlkOiAzNjxicj4KICAgICAgICAgICAgICAgIHN0b3JhZ2Uub3du ZXItZ2lkOiAzNjxicj4KICAgICAgICAgICAgICAgIGNsdXN0ZXIucXVvcnVtLWNvdW50OiAxPGJy PgogICAgICAgICAgICAgICAgY2x1c3Rlci5zZWxmLWhlYWwtZGFlbW9uOiBlbmFibGU8YnI+CiAg ICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICBJZiBJJ2xsIGNyZWF0ZSAiaWRzIiBm aWxlIG1hbnVhbGx5ICjCoCBlZy4gIiBzYW5sb2NrCiAgICAgICAgICAgICAgICBkaXJlY3QgaW5p dCAtcwogICAgICAgICAgICAgICAgM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBiMTQ3 OjA6L1NUT1JBR0VTL2cxcjVwMy9HRlMvM2MzNGFkNjMtNmM2Ni00ZTIzLWFiNDYtMDg0ZjNkNzBi MTQ3L2RvbV9tZC9pZHM6MAoKICAgICAgICAgICAgICAgICIgKSBvbiBib3RoIGJyaWNrcyw8YnI+ CiAgICAgICAgICAgICAgICB2ZHNtIGlzIHdyaXRpbmcgb25seSB0byBoYWxmIG9mIHRoZW0gKCB0 aGF0IHdpdGggMiBsaW5rcwogICAgICAgICAgICAgICAgPSBjb3JyZWN0ICk8YnI+CiAgICAgICAg ICAgICAgICAiaWRzIiBmaWxlIGhhcyBjb3JyZWN0IHBlcm1pdHRpb25zLCBvd25lciwgc2l6ZcKg IG9uIGJvdGgKICAgICAgICAgICAgICAgIGJyaWNrcy48YnI+CiAgICAgICAgICAgICAgICBicmlj ayAxOsKgIC1ydy1ydy0tLS0gMSB2ZHNtIGt2bSAxMDQ4NTc2wqAgMi7CoGLFmWUgMTguNTYKICAg ICAgICAgICAgICAgIC9TVE9SQUdFUy9nMXI1cDMvR0ZTLzNjMzRhZDYzLTZjNjYtNGUyMy1hYjQ2 LTA4NGYzZDcwYjE0Ny9kb21fbWQvaWRzCiAgICAgICAgICAgICAgICAtIG5vdCB1cGRhdGVkPGJy PgogICAgICAgICAgICAgICAgYnJpY2sgMjrCoCAtcnctcnctLS0tIDIgdmRzbSBrdm0gMTA0ODU3 NsKgIDMuwqBixZllIDEwLjE2CiAgICAgICAgICAgICAgICAvU1RPUkFHRVMvZzFyNXAzL0dGUy8z YzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcwogICAgICAgICAg ICAgICAgLSBpcyBjb250aW51YWxseSB1cGRhdGVkPGJyPgogICAgICAgICAgICAgICAgPGJyPgog ICAgICAgICAgICAgICAgV2hhdCBoYXBwZW5zIHdoZW4gSSdsbCByZXN0YXJ0IHZkc20gPyBXaWxs IG9WaXJ0CiAgICAgICAgICAgICAgICBzdG9yYWdlcyBnbyB0byAiZGlzYWJsZSAiIHN0YXRlID8/ PyA9IGRpc2Nvbm5lY3QgVk1zCiAgICAgICAgICAgICAgICBzdG9yYWdlcyA/PGJyPgogICAgICAg ICAgICAgIDwvZGl2PgogICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgIDxkaXY+ PGJyPgogICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoE5vdGhpbmcgd2lsbCBo YXBwZW4sIHRoZSB2bXMgd2lsbCBjb250aW51ZSB0byBydW4KICAgICAgICAgICAgICBub3JtYWxs eS48L2Rpdj4KICAgICAgICAgICAgPGRpdj48YnI+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAg ICAgICA8ZGl2Pk9uIGJsb2NrIHN0b3JhZ2UsIHN0b3BwaW5nIHZkc20gd2lsbCBwcmV2ZW50IGF1 dG9tYXRpYwogICAgICAgICAgICAgIGV4dGVuZGluZyBvZiB2bSBkaXNrczwvZGl2PgogICAgICAg ICAgICA8ZGl2PndoZW4gdGhlIGRpc2sgYmVjb21lIHRvbyBmdWxsLCBidXQgb24gZmlsZSBiYXNl ZAogICAgICAgICAgICAgIHN0b3JhZ2UgKGxpa2UgZ2x1c3RlcikgdGhlcmUgaXMgbm8gaXNzdWUu PC9kaXY+CiAgICAgICAgICAgIDxkaXY+wqA8L2Rpdj4KICAgICAgICAgICAgPGJsb2NrcXVvdGUg Y2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwCiAgICAgICAgICAgICAgLjhl eDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4KICAgICAgICAg ICAgICA8ZGl2IHRleHQ9IiMwMDAwNjYiIGJnY29sb3I9IiNGRkZGRkYiPiA8YnI+CiAgICAgICAg ICAgICAgICByZWdzLlBhLgogICAgICAgICAgICAgICAgPGRpdj4KICAgICAgICAgICAgICAgICAg PGRpdiBjbGFzcz0iaDUiPjxicj4KICAgICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgICAgICAgPGRpdj5PbiAzLjMuMjAxNiAwMjowMiwgUmF2aXNoYW5rYXIgTiB3cm90ZTo8YnI+ CiAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgPGJsb2NrcXVv dGUgdHlwZT0iY2l0ZSI+CiAgICAgICAgICAgICAgICAgICAgICA8ZGl2Pk9uIDAzLzAzLzIwMTYg MTI6NDMgQU0sIE5pciBTb2ZmZXIgd3JvdGU6PGJyPgogICAgICAgICAgICAgICAgICAgICAgPC9k aXY+CiAgICAgICAgICAgICAgICAgICAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIj4KICAgICAg ICAgICAgICAgICAgICAgICAgPGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0i bWFyZ2luOjAKICAgICAgICAgICAgICAgICAgICAgICAgICAwIDAgLjhleDtib3JkZXItbGVmdDox cHggI2NjYwogICAgICAgICAgICAgICAgICAgICAgICAgIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgi PgogICAgICAgICAgICAgICAgICAgICAgICAgIDxkaXYgdGV4dD0iIzAwMDA2NiIgYmdjb2xvcj0i I0ZGRkZGRiI+UFM6wqAgIwogICAgICAgICAgICAgICAgICAgICAgICAgICAgZmluZCAvU1RPUkFH RVMgLXNhbWVmaWxlCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAvU1RPUkFHRVMvZzFyNXAz L0dGUy8zYzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkcwogICAg ICAgICAgICAgICAgICAgICAgICAgICAgLXByaW50PGJyPgovU1RPUkFHRVMvZzFyNXAzL0dGUy8z YzM0YWQ2My02YzY2LTRlMjMtYWI0Ni0wODRmM2Q3MGIxNDcvZG9tX21kL2lkczxicj4KICAgICAg ICAgICAgICAgICAgICAgICAgICAgID0gbWlzc2luZyAic2hhZG93ZmlsZSIgaW4gIiAuZ2x1c3Rl ciAiIGRpci48YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICBIb3cgY2FuIEkgZml4IGl0 ID8/IC0gb25saW5lICE8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgPC9ibG9ja3F1b3Rl PgogICAgICAgICAgICAgICAgICAgICAgICA8ZGl2Pjxicj4KICAgICAgICAgICAgICAgICAgICAg ICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgICAgIDxkaXY+UmF2aT88L2Rpdj4KICAgICAg ICAgICAgICAgICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICAgICAgICAgICAgICAgIElzIHRo aXMgdGhlIGNhc2UgaW4gYWxsIDMgYnJpY2tzIG9mIHRoZSByZXBsaWNhPyA8YnI+CiAgICAgICAg ICAgICAgICAgICAgICBCVFcsIHlvdSBjYW4ganVzdCBzdGF0IHRoZSBmaWxlIG9uIHRoZSBicmlj ayBhbmQKICAgICAgICAgICAgICAgICAgICAgIHNlZSB0aGUgbGluayBjb3VudCAoaXQgbXVzdCBi ZSAyKSBpbnN0ZWFkIG9mCiAgICAgICAgICAgICAgICAgICAgICBydW5uaW5nIHRoZSBtb3JlIGV4 cGVuc2l2ZSBmaW5kIGNvbW1hbmQuPGJyPgogICAgICAgICAgICAgICAgICAgICAgPGJyPgogICAg ICAgICAgICAgICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICAgICAgICAgICAgICA8YnI+CiAg ICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAg ICAgPC9kaXY+CiAgICAgICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICAgIDwvZGl2PgogICAg ICAgICAgPGJyPgogICAgICAgIDwvZGl2PgogICAgICA8L2Rpdj4KICAgIDwvYmxvY2txdW90ZT4K ICAgIDxicj4KICA8L2JvZHk+CjwvaHRtbD4KCi0tLS0tLS0tLS0tLS0tMDUwNjAzMDcwNzA5MDgw MDAyMDEwNjA1LS0K --===============2163385157278793131==--