From knarra at redhat.com Mon Jul 3 13:28:31 2017 Content-Type: multipart/mixed; boundary="===============2861013256419917111==" MIME-Version: 1.0 From: knarra To: users at ovirt.org Subject: Re: [ovirt-users] add fourth full gluster node and remove arbiter: ovirt 4.1 with hosted engine Date: Mon, 03 Jul 2017 18:58:28 +0530 Message-ID: <12869870-7d3a-1b80-4199-d336dfb8570c@redhat.com> In-Reply-To: CAGK=3kwU7VbyQjLyvG5TS0LtozOkBpGidOsBYeoyp_eOapTB6g@mail.gmail.com --===============2861013256419917111== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------A57F4257A1EE9A2CA7556540 Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 7bit On 07/03/2017 06:53 PM, yayo (j) wrote: > Hi, > > And sorry for delay > > 2017-06-30 14:09 GMT+02:00 knarra >: > > To add a fully replicated node you need to reduce the replica > count to 2 and add new brick to the volume so that it becomes > replica 3. Reducing replica count by removing a brick from replica > / arbiter cannot be done from UI currently and this has to be done > using gluster CLI. > AFAIR, there was an issue where vm's were going to paused state > when reducing the replica count and increasing it to 3. Not sure > if this still holds good with the latest release. > > Any specific reason why you want to move to full replication > instead of using an arbiter node ? > > > We have a new server with the same hard disk size of other two node, = > so, why not? Why join the cluster as an arbiter when we can have the = > same disk capacity to add extra replication? > > >> and remove the arbiter node (Also a way to move the arbiter role >> to the new node, If needed) > To move arbiter role to a new node you can move the node to > maintenance , add new node and replace old brick with new brick. > You can follow the steps below to do that. > > * Move the node to be replaced into Maintenance mode > * Prepare the replacement node > * Prepare bricks on that node. > * Create replacement brick directories > * Ensure the new directories are owned by the vdsm user and the > kvm group. > * # mkdir /rhgs/bricks/engine > * # chmod vdsm:kvm /rhgs/bricks/engine > * # mkdir /rhgs/bricks/data > * # chmod vdsm:kvm /rhgs/bricks/data > * Run the following command from one of the healthy cluster members: > * # gluster peer probe > * add the new host to the cluster. > * Add new host address to gluster network > * Click Network Interfaces sub-tab. > * Click Set up Host Networks. > * Drag and drop the glusternw network onto the IP address of the > new host. > * Click OK > * Replace the old brick with the brick on the new host > * Click the Bricks sub-tab. > * Verify that brick heal completes successfully. > * In the Hosts tab, right-click on the old host and click Remove. > * Clean old host metadata > * # hosted-engine --clean-metadata --host-id=3D > --force-clean > > > > I need this (reads: I need the arbiter role) if I reduce replica count = > then I add the new node as full replica and increasing replica count = > again to 3? (As you expained above) > Above steps hold good if you want to move the arbiter role to a new node. If you want to move to full replica, reducing the replica count will = work fine but increasing it again back to 3 might cause vm pause issues. > Thank you --------------A57F4257A1EE9A2CA7556540 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit
On 07/03/2017 06:53 PM, yayo (j) wrote:<= br>
Hi,

And sorry for delay

2017-06-30 14:09 GMT+02:00 knarra <knarr= a(a)redhat.com>:
To add a fully replicated node=C2=A0 you need to reduce the replica count to 2 and add new brick to the volume so that it becomes replica 3. Reducing replica count by removing a brick from replica / arbiter cannot be done from UI currently and this has to be done using gluster CLI.
=C2=A0AFAIR, there was an issue where vm's were going to paused state when reducing the replica count and increasing it to 3. Not sure if this still holds good with the latest release.

Any specific reason why you want to move to full replication instead of using an arbiter node ?

We have a new server with the same hard disk size of other two node, so, why not? Why join the cluster as an arbiter when we can have the same disk capacity to add extra replication?=C2=A0

=C2=A0
<= br>
and remove the arbiter node (Also a way to move the arbiter role to the new node, If needed)
To move arbiter role to a new no= de you can move the node to maintenance , add=C2=A0 new node and replace=C2=A0 old brick with= new brick. You can follow the steps below to do that.

  • Move the node to be replaced into Maintenance mode
  • Prepare the replacement node
  • Prepare bricks on that node= .
  • Create replacement brick directories
  • Ensure the new directories are owned by the vdsm user and the kvm group.
  • # mkdir /rhgs/bricks/engine
  • # chmod vdsm:kvm /rhgs/bricks/engine
  • # mkdir /rhgs/bricks/data
  • # chmod vdsm:kvm /rhgs/bricks/data
  • Run the following command from one of the healthy cluster members:
  • # gluster peer probe <new_node>
  • =C2=A0add the new host to t= he cluster.
  • Add new host address to gluster network
  • Click Network Interfaces sub-tab.
  • Click Set up Host Networks.
  • Drag and drop the glusternw network onto the IP address of the new host.
  • Click OK
  • Replace the old brick with the brick on the new host
  • Click the Bricks sub-tab.
  • Verify that brick heal completes successfully.
  • In the Hosts tab, right-click on the old host and click Remove.
  • Clean old host metadata
  • # hosted-engine --clean-metadata --host-id=3D<old_host_id> --force-clean


I need this (reads: I need the arbiter role) if I reduce replica count then I add the new node as full replica and increasing replica count again to 3? (As you expained above)

Above steps hold good if you want to move the arbiter role to a new node.

If you want to move to full replica, reducing the replica count will work fine but increasing it again back to 3 might cause vm pause issues.




Thank you


--------------A57F4257A1EE9A2CA7556540-- --===============2861013256419917111== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS1BNTdGNDI1N0ExRUU5QTJDQTc1NTY1NDAKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQK Ck9uIDA3LzAzLzIwMTcgMDY6NTMgUE0sIHlheW8gKGopIHdyb3RlOgo+IEhpLAo+Cj4gQW5kIHNv cnJ5IGZvciBkZWxheQo+Cj4gMjAxNy0wNi0zMCAxNDowOSBHTVQrMDI6MDAga25hcnJhIDxrbmFy cmFAcmVkaGF0LmNvbSAKPiA8bWFpbHRvOmtuYXJyYUByZWRoYXQuY29tPj46Cj4KPiAgICAgVG8g YWRkIGEgZnVsbHkgcmVwbGljYXRlZCBub2RlICB5b3UgbmVlZCB0byByZWR1Y2UgdGhlIHJlcGxp Y2EKPiAgICAgY291bnQgdG8gMiBhbmQgYWRkIG5ldyBicmljayB0byB0aGUgdm9sdW1lIHNvIHRo YXQgaXQgYmVjb21lcwo+ICAgICByZXBsaWNhIDMuIFJlZHVjaW5nIHJlcGxpY2EgY291bnQgYnkg cmVtb3ZpbmcgYSBicmljayBmcm9tIHJlcGxpY2EKPiAgICAgLyBhcmJpdGVyIGNhbm5vdCBiZSBk b25lIGZyb20gVUkgY3VycmVudGx5IGFuZCB0aGlzIGhhcyB0byBiZSBkb25lCj4gICAgIHVzaW5n IGdsdXN0ZXIgQ0xJLgo+ICAgICAgQUZBSVIsIHRoZXJlIHdhcyBhbiBpc3N1ZSB3aGVyZSB2bSdz IHdlcmUgZ29pbmcgdG8gcGF1c2VkIHN0YXRlCj4gICAgIHdoZW4gcmVkdWNpbmcgdGhlIHJlcGxp Y2EgY291bnQgYW5kIGluY3JlYXNpbmcgaXQgdG8gMy4gTm90IHN1cmUKPiAgICAgaWYgdGhpcyBz dGlsbCBob2xkcyBnb29kIHdpdGggdGhlIGxhdGVzdCByZWxlYXNlLgo+Cj4gICAgIEFueSBzcGVj aWZpYyByZWFzb24gd2h5IHlvdSB3YW50IHRvIG1vdmUgdG8gZnVsbCByZXBsaWNhdGlvbgo+ICAg ICBpbnN0ZWFkIG9mIHVzaW5nIGFuIGFyYml0ZXIgbm9kZSA/Cj4KPgo+IFdlIGhhdmUgYSBuZXcg c2VydmVyIHdpdGggdGhlIHNhbWUgaGFyZCBkaXNrIHNpemUgb2Ygb3RoZXIgdHdvIG5vZGUsIAo+ IHNvLCB3aHkgbm90PyBXaHkgam9pbiB0aGUgY2x1c3RlciBhcyBhbiBhcmJpdGVyIHdoZW4gd2Ug Y2FuIGhhdmUgdGhlIAo+IHNhbWUgZGlzayBjYXBhY2l0eSB0byBhZGQgZXh0cmEgcmVwbGljYXRp b24/Cj4KPgo+PiAgICAgYW5kIHJlbW92ZSB0aGUgYXJiaXRlciBub2RlIChBbHNvIGEgd2F5IHRv IG1vdmUgdGhlIGFyYml0ZXIgcm9sZQo+PiAgICAgdG8gdGhlIG5ldyBub2RlLCBJZiBuZWVkZWQp Cj4gICAgIFRvIG1vdmUgYXJiaXRlciByb2xlIHRvIGEgbmV3IG5vZGUgeW91IGNhbiBtb3ZlIHRo ZSBub2RlIHRvCj4gICAgIG1haW50ZW5hbmNlICwgYWRkICBuZXcgbm9kZSBhbmQgcmVwbGFjZSAg b2xkIGJyaWNrIHdpdGggbmV3IGJyaWNrLgo+ICAgICBZb3UgY2FuIGZvbGxvdyB0aGUgc3RlcHMg YmVsb3cgdG8gZG8gdGhhdC4KPgo+ICAgICAgICogTW92ZSB0aGUgbm9kZSB0byBiZSByZXBsYWNl ZCBpbnRvIE1haW50ZW5hbmNlIG1vZGUKPiAgICAgICAqIFByZXBhcmUgdGhlIHJlcGxhY2VtZW50 IG5vZGUKPiAgICAgICAqIFByZXBhcmUgYnJpY2tzIG9uIHRoYXQgbm9kZS4KPiAgICAgICAqIENy ZWF0ZSByZXBsYWNlbWVudCBicmljayBkaXJlY3Rvcmllcwo+ICAgICAgICogRW5zdXJlIHRoZSBu ZXcgZGlyZWN0b3JpZXMgYXJlIG93bmVkIGJ5IHRoZSB2ZHNtIHVzZXIgYW5kIHRoZQo+ICAgICAg ICAga3ZtIGdyb3VwLgo+ICAgICAgICogIyBta2RpciAvcmhncy9icmlja3MvZW5naW5lCj4gICAg ICAgKiAjIGNobW9kIHZkc206a3ZtIC9yaGdzL2JyaWNrcy9lbmdpbmUKPiAgICAgICAqICMgbWtk aXIgL3JoZ3MvYnJpY2tzL2RhdGEKPiAgICAgICAqICMgY2htb2QgdmRzbTprdm0gL3JoZ3MvYnJp Y2tzL2RhdGEKPiAgICAgICAqIFJ1biB0aGUgZm9sbG93aW5nIGNvbW1hbmQgZnJvbSBvbmUgb2Yg dGhlIGhlYWx0aHkgY2x1c3RlciBtZW1iZXJzOgo+ICAgICAgICogIyBnbHVzdGVyIHBlZXIgcHJv YmUgPG5ld19ub2RlPgo+ICAgICAgICogYWRkIHRoZSBuZXcgaG9zdCB0byB0aGUgY2x1c3Rlci4K PiAgICAgICAqIEFkZCBuZXcgaG9zdCBhZGRyZXNzIHRvIGdsdXN0ZXIgbmV0d29yawo+ICAgICAg ICogQ2xpY2sgTmV0d29yayBJbnRlcmZhY2VzIHN1Yi10YWIuCj4gICAgICAgKiBDbGljayBTZXQg dXAgSG9zdCBOZXR3b3Jrcy4KPiAgICAgICAqIERyYWcgYW5kIGRyb3AgdGhlIGdsdXN0ZXJudyBu ZXR3b3JrIG9udG8gdGhlIElQIGFkZHJlc3Mgb2YgdGhlCj4gICAgICAgICBuZXcgaG9zdC4KPiAg ICAgICAqIENsaWNrIE9LCj4gICAgICAgKiBSZXBsYWNlIHRoZSBvbGQgYnJpY2sgd2l0aCB0aGUg YnJpY2sgb24gdGhlIG5ldyBob3N0Cj4gICAgICAgKiBDbGljayB0aGUgQnJpY2tzIHN1Yi10YWIu Cj4gICAgICAgKiBWZXJpZnkgdGhhdCBicmljayBoZWFsIGNvbXBsZXRlcyBzdWNjZXNzZnVsbHku Cj4gICAgICAgKiBJbiB0aGUgSG9zdHMgdGFiLCByaWdodC1jbGljayBvbiB0aGUgb2xkIGhvc3Qg YW5kIGNsaWNrIFJlbW92ZS4KPiAgICAgICAqIENsZWFuIG9sZCBob3N0IG1ldGFkYXRhCj4gICAg ICAgKiAjIGhvc3RlZC1lbmdpbmUgLS1jbGVhbi1tZXRhZGF0YSAtLWhvc3QtaWQ9PG9sZF9ob3N0 X2lkPgo+ICAgICAgICAgLS1mb3JjZS1jbGVhbgo+Cj4KPgo+IEkgbmVlZCB0aGlzIChyZWFkczog SSBuZWVkIHRoZSBhcmJpdGVyIHJvbGUpIGlmIEkgcmVkdWNlIHJlcGxpY2EgY291bnQgCj4gdGhl biBJIGFkZCB0aGUgbmV3IG5vZGUgYXMgZnVsbCByZXBsaWNhIGFuZCBpbmNyZWFzaW5nIHJlcGxp Y2EgY291bnQgCj4gYWdhaW4gdG8gMz8gKEFzIHlvdSBleHBhaW5lZCBhYm92ZSkKPgpBYm92ZSBz dGVwcyBob2xkIGdvb2QgaWYgeW91IHdhbnQgdG8gbW92ZSB0aGUgYXJiaXRlciByb2xlIHRvIGEg bmV3IG5vZGUuCgpJZiB5b3Ugd2FudCB0byBtb3ZlIHRvIGZ1bGwgcmVwbGljYSwgcmVkdWNpbmcg dGhlIHJlcGxpY2EgY291bnQgd2lsbCAKd29yayBmaW5lIGJ1dCBpbmNyZWFzaW5nIGl0IGFnYWlu IGJhY2sgdG8gMyBtaWdodCBjYXVzZSB2bSBwYXVzZSBpc3N1ZXMuCgoKCgo+IFRoYW5rIHlvdQoK CgotLS0tLS0tLS0tLS0tLUE1N0Y0MjU3QTFFRTlBMkNBNzU1NjU0MApDb250ZW50LVR5cGU6IHRl eHQvaHRtbDsgY2hhcnNldD11dGYtOApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0Cgo8 aHRtbD4KICA8aGVhZD4KICAgIDxtZXRhIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYt OCIgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICA8L2hlYWQ+CiAgPGJvZHkgYmdjb2xvcj0i I0ZGRkZGRiIgdGV4dD0iIzAwMDAwMCI+CiAgICA8ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgi Pk9uIDA3LzAzLzIwMTcgMDY6NTMgUE0sIHlheW8gKGopIHdyb3RlOjxicj4KICAgIDwvZGl2Pgog ICAgPGJsb2NrcXVvdGUKY2l0ZT0ibWlkOkNBR0s9M2t3VTdWYnlRakx5dkc1VFMwTHRvek9rQnBH aWRPc0JZZW95cF9lT2FwVEI2Z0BtYWlsLmdtYWlsLmNvbSIKICAgICAgdHlwZT0iY2l0ZSI+CiAg ICAgIDxkaXYgZGlyPSJsdHIiPgogICAgICAgIDxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj5IaSw8 L2Rpdj4KICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGJyPgogICAgICAgIDwvZGl2 PgogICAgICAgIDxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj5BbmQgc29ycnkgZm9yIGRlbGF5PC9k aXY+CiAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxfZXh0cmEiPjxicj4KICAgICAgICAgIDxkaXYg Y2xhc3M9ImdtYWlsX3F1b3RlIj4yMDE3LTA2LTMwIDE0OjA5IEdNVCswMjowMCBrbmFycmEgPHNw YW4KICAgICAgICAgICAgICBkaXI9Imx0ciI+Jmx0OzxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIK ICAgICAgICAgICAgICAgIGhyZWY9Im1haWx0bzprbmFycmFAcmVkaGF0LmNvbSIgdGFyZ2V0PSJf YmxhbmsiPmtuYXJyYUByZWRoYXQuY29tPC9hPiZndDs8L3NwYW4+Ojxicj4KICAgICAgICAgICAg PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwCiAgICAg ICAgICAgICAgLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4 Ij4KICAgICAgICAgICAgICA8ZGl2IGJnY29sb3I9IiNGRkZGRkYiIHRleHQ9IiMwMDAwMDAiPlRv IDxzcGFuIGNsYXNzPSJpbCI+YWRkPC9zcGFuPgogICAgICAgICAgICAgICAgYSBmdWxseSByZXBs aWNhdGVkIDxzcGFuIGNsYXNzPSJpbCI+bm9kZTwvc3Bhbj7CoCB5b3UKICAgICAgICAgICAgICAg IG5lZWQgdG8gcmVkdWNlIHRoZSByZXBsaWNhIGNvdW50IHRvIDIgYW5kIDxzcGFuCiAgICAgICAg ICAgICAgICAgIGNsYXNzPSJpbCI+YWRkPC9zcGFuPiBuZXcgYnJpY2sgdG8gdGhlIHZvbHVtZSBz byB0aGF0CiAgICAgICAgICAgICAgICBpdCBiZWNvbWVzIHJlcGxpY2EgMy4gUmVkdWNpbmcgcmVw bGljYSBjb3VudCBieSByZW1vdmluZwogICAgICAgICAgICAgICAgYSBicmljayBmcm9tIHJlcGxp Y2EgLyBhcmJpdGVyIGNhbm5vdCBiZSBkb25lIGZyb20gVUkKICAgICAgICAgICAgICAgIGN1cnJl bnRseSBhbmQgdGhpcyBoYXMgdG8gYmUgZG9uZSB1c2luZyA8c3BhbgogICAgICAgICAgICAgICAg ICBjbGFzcz0iaWwiPmdsdXN0ZXI8L3NwYW4+IENMSS4gPGJyPgogICAgICAgICAgICAgICAgwqBB RkFJUiwgdGhlcmUgd2FzIGFuIGlzc3VlIHdoZXJlIHZtJ3Mgd2VyZSBnb2luZyB0bwogICAgICAg ICAgICAgICAgcGF1c2VkIHN0YXRlIHdoZW4gcmVkdWNpbmcgdGhlIHJlcGxpY2EgY291bnQgYW5k CiAgICAgICAgICAgICAgICBpbmNyZWFzaW5nIGl0IHRvIDMuIE5vdCBzdXJlIGlmIHRoaXMgc3Rp bGwgaG9sZHMgZ29vZAogICAgICAgICAgICAgICAgd2l0aCB0aGUgbGF0ZXN0IHJlbGVhc2UuIDxi cj4KICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgIEFueSBzcGVjaWZpYyByZWFz b24gd2h5IHlvdSB3YW50IHRvIG1vdmUgdG8gPHNwYW4KICAgICAgICAgICAgICAgICAgY2xhc3M9 ImlsIj5mdWxsPC9zcGFuPiByZXBsaWNhdGlvbiBpbnN0ZWFkIG9mIHVzaW5nIGFuCiAgICAgICAg ICAgICAgICBhcmJpdGVyIDxzcGFuIGNsYXNzPSJpbCI+bm9kZTwvc3Bhbj4gPzxzcGFuIGNsYXNz PSIiPjxicj4KICAgICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgPC9zcGFuPjwv ZGl2PgogICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAg ICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj5XZSBoYXZlIGEgbmV3IHNlcnZlciB3aXRo IHRoZSBzYW1lIGhhcmQgZGlzayBzaXplIG9mCiAgICAgICAgICAgICAgb3RoZXIgdHdvIG5vZGUs IHNvLCB3aHkgbm90PyBXaHkgam9pbiB0aGUgY2x1c3RlciBhcyBhbgogICAgICAgICAgICAgIGFy Yml0ZXIgd2hlbiB3ZSBjYW4gaGF2ZSB0aGUgc2FtZSBkaXNrIGNhcGFjaXR5IHRvIGFkZAogICAg ICAgICAgICAgIGV4dHJhIHJlcGxpY2F0aW9uP8KgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+PGJy PgogICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj7CoDwvZGl2PgogICAgICAgICAg ICA8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAKICAg ICAgICAgICAgICAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDox ZXgiPgogICAgICAgICAgICAgIDxkaXYgYmdjb2xvcj0iI0ZGRkZGRiIgdGV4dD0iIzAwMDAwMCI+ PHNwYW4gY2xhc3M9IiI+IDxicj4KICAgICAgICAgICAgICAgICAgPGJsb2NrcXVvdGUgdHlwZT0i Y2l0ZSI+CiAgICAgICAgICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+CiAgICAgICAgICAgICAg ICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+CiAgICAgICAgICAgICAgICAgICAgICAg IDxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj5hbmQgcmVtb3ZlIHRoZSBhcmJpdGVyCiAgICAgICAg ICAgICAgICAgICAgICAgICAgPHNwYW4gY2xhc3M9ImlsIj5ub2RlPC9zcGFuPiAoQWxzbyBhIHdh eSB0bwogICAgICAgICAgICAgICAgICAgICAgICAgIG1vdmUgdGhlIGFyYml0ZXIgcm9sZSB0byB0 aGUgbmV3IDxzcGFuCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBjbGFzcz0iaWwiPm5vZGU8 L3NwYW4+LCBJZiBuZWVkZWQpIDwvZGl2PgogICAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAg ICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgIDwvYmxvY2txdW90ZT4K ICAgICAgICAgICAgICAgIDwvc3Bhbj4gVG8gbW92ZSBhcmJpdGVyIHJvbGUgdG8gYSBuZXcgPHNw YW4gY2xhc3M9ImlsIj5ub2RlPC9zcGFuPgogICAgICAgICAgICAgICAgeW91IGNhbiBtb3ZlIHRo ZSA8c3BhbiBjbGFzcz0iaWwiPm5vZGU8L3NwYW4+IHRvCiAgICAgICAgICAgICAgICBtYWludGVu YW5jZSAsIDxzcGFuIGNsYXNzPSJpbCI+YWRkPC9zcGFuPsKgIG5ldyA8c3BhbgogICAgICAgICAg ICAgICAgICBjbGFzcz0iaWwiPm5vZGU8L3NwYW4+IGFuZCByZXBsYWNlwqAgb2xkIGJyaWNrIHdp dGggbmV3CiAgICAgICAgICAgICAgICBicmljay4gWW91IGNhbiBmb2xsb3cgdGhlIHN0ZXBzIGJl bG93IHRvIGRvIHRoYXQuPGJyPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAg PHVsPgogICAgICAgICAgICAgICAgICA8bGk+TW92ZSB0aGUgPHNwYW4gY2xhc3M9ImlsIj5ub2Rl PC9zcGFuPiB0byBiZQogICAgICAgICAgICAgICAgICAgIHJlcGxhY2VkIGludG8gTWFpbnRlbmFu Y2UgbW9kZTwvbGk+CiAgICAgICAgICAgICAgICAgIDxsaT5QcmVwYXJlIHRoZSByZXBsYWNlbWVu dCA8c3BhbiBjbGFzcz0iaWwiPm5vZGU8L3NwYW4+PC9saT4KICAgICAgICAgICAgICAgICAgPGxp PlByZXBhcmUgYnJpY2tzIG9uIHRoYXQgPHNwYW4gY2xhc3M9ImlsIj5ub2RlPC9zcGFuPi48L2xp PgogICAgICAgICAgICAgICAgICA8bGk+Q3JlYXRlIHJlcGxhY2VtZW50IGJyaWNrIGRpcmVjdG9y aWVzPC9saT4KICAgICAgICAgICAgICAgICAgPGxpPkVuc3VyZSB0aGUgbmV3IGRpcmVjdG9yaWVz IGFyZSBvd25lZCBieSB0aGUgdmRzbQogICAgICAgICAgICAgICAgICAgIHVzZXIgYW5kIHRoZSBr dm0gZ3JvdXAuPC9saT4KICAgICAgICAgICAgICAgICAgPGxpPiMgbWtkaXIgL3JoZ3MvYnJpY2tz L2VuZ2luZTwvbGk+CiAgICAgICAgICAgICAgICAgIDxsaT4jIGNobW9kIHZkc206a3ZtIC9yaGdz L2JyaWNrcy9lbmdpbmU8L2xpPgogICAgICAgICAgICAgICAgICA8bGk+IyBta2RpciAvcmhncy9i cmlja3MvZGF0YTwvbGk+CiAgICAgICAgICAgICAgICAgIDxsaT4jIGNobW9kIHZkc206a3ZtIC9y aGdzL2JyaWNrcy9kYXRhPC9saT4KICAgICAgICAgICAgICAgICAgPGxpPlJ1biB0aGUgZm9sbG93 aW5nIGNvbW1hbmQgZnJvbSBvbmUgb2YgdGhlIGhlYWx0aHkKICAgICAgICAgICAgICAgICAgICBj bHVzdGVyIG1lbWJlcnM6PC9saT4KICAgICAgICAgICAgICAgICAgPGxpPiMgPHNwYW4gY2xhc3M9 ImlsIj5nbHVzdGVyPC9zcGFuPiBwZWVyIHByb2JlCiAgICAgICAgICAgICAgICAgICAgJmx0O25l d19ub2RlJmd0OzwvbGk+CiAgICAgICAgICAgICAgICAgIDxsaT7CoDxzcGFuIGNsYXNzPSJpbCI+ YWRkPC9zcGFuPiB0aGUgbmV3IGhvc3QgdG8gdGhlCiAgICAgICAgICAgICAgICAgICAgY2x1c3Rl ci48L2xpPgogICAgICAgICAgICAgICAgICA8bGk+PHNwYW4gY2xhc3M9ImlsIj5BZGQ8L3NwYW4+ IG5ldyBob3N0IGFkZHJlc3MgdG8gPHNwYW4KICAgICAgICAgICAgICAgICAgICAgIGNsYXNzPSJp bCI+Z2x1c3Rlcjwvc3Bhbj4gbmV0d29yazwvbGk+CiAgICAgICAgICAgICAgICAgIDxsaT5DbGlj ayBOZXR3b3JrIEludGVyZmFjZXMgc3ViLXRhYi48L2xpPgogICAgICAgICAgICAgICAgICA8bGk+ Q2xpY2sgU2V0IHVwIEhvc3QgTmV0d29ya3MuPC9saT4KICAgICAgICAgICAgICAgICAgPGxpPkRy YWcgYW5kIGRyb3AgdGhlIGdsdXN0ZXJudyBuZXR3b3JrIG9udG8gdGhlIElQCiAgICAgICAgICAg ICAgICAgICAgYWRkcmVzcyBvZiB0aGUgbmV3IGhvc3QuPC9saT4KICAgICAgICAgICAgICAgICAg PGxpPkNsaWNrIE9LPC9saT4KICAgICAgICAgICAgICAgICAgPGxpPlJlcGxhY2UgdGhlIG9sZCBi cmljayB3aXRoIHRoZSBicmljayBvbiB0aGUgbmV3CiAgICAgICAgICAgICAgICAgICAgaG9zdDwv bGk+CiAgICAgICAgICAgICAgICAgIDxsaT5DbGljayB0aGUgQnJpY2tzIHN1Yi10YWIuPC9saT4K ICAgICAgICAgICAgICAgICAgPGxpPlZlcmlmeSB0aGF0IGJyaWNrIGhlYWwgY29tcGxldGVzIHN1 Y2Nlc3NmdWxseS48L2xpPgogICAgICAgICAgICAgICAgICA8bGk+SW4gdGhlIEhvc3RzIHRhYiwg cmlnaHQtY2xpY2sgb24gdGhlIG9sZCBob3N0IGFuZAogICAgICAgICAgICAgICAgICAgIGNsaWNr IFJlbW92ZS48L2xpPgogICAgICAgICAgICAgICAgICA8bGk+Q2xlYW4gb2xkIGhvc3QgbWV0YWRh dGE8L2xpPgogICAgICAgICAgICAgICAgICA8bGk+IyBob3N0ZWQtZW5naW5lIC0tY2xlYW4tbWV0 YWRhdGEKICAgICAgICAgICAgICAgICAgICAtLWhvc3QtaWQ9Jmx0O29sZF9ob3N0X2lkJmd0OyAt LWZvcmNlLWNsZWFuPC9saT4KICAgICAgICAgICAgICAgIDwvdWw+CiAgICAgICAgICAgICAgICA8 c3BhbiBjbGFzcz0iIj4gPGJyPgogICAgICAgICAgICAgICAgPC9zcGFuPjwvZGl2PgogICAgICAg ICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICA8L2Rp dj4KICAgICAgICAgICAgPGRpdj5JIG5lZWQgdGhpcyAocmVhZHM6IEkgbmVlZCB0aGUgYXJiaXRl ciByb2xlKSBpZiBJCiAgICAgICAgICAgICAgcmVkdWNlIHJlcGxpY2EgY291bnQgdGhlbiBJIGFk ZCB0aGUgbmV3IG5vZGUgYXMgZnVsbAogICAgICAgICAgICAgIHJlcGxpY2EgYW5kIGluY3JlYXNp bmcgcmVwbGljYSBjb3VudCBhZ2FpbiB0byAzPyAoQXMgeW91CiAgICAgICAgICAgICAgZXhwYWlu ZWQgYWJvdmUpPC9kaXY+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICA8L2Rpdj4K ICAgICAgICAgIDwvZGl2PgogICAgICAgIDwvZGl2PgogICAgICA8L2Rpdj4KICAgIDwvYmxvY2tx dW90ZT4KICAgIEFib3ZlIHN0ZXBzIGhvbGQgZ29vZCBpZiB5b3Ugd2FudCB0byBtb3ZlIHRoZSBh cmJpdGVyIHJvbGUgdG8gYSBuZXcKICAgIG5vZGUuPGJyPgogICAgPGJyPgogICAgSWYgeW91IHdh bnQgdG8gbW92ZSB0byBmdWxsIHJlcGxpY2EsIHJlZHVjaW5nIHRoZSByZXBsaWNhIGNvdW50IHdp bGwKICAgIHdvcmsgZmluZSBidXQgaW5jcmVhc2luZyBpdCBhZ2FpbiBiYWNrIHRvIDMgbWlnaHQg Y2F1c2Ugdm0gcGF1c2UKICAgIGlzc3Vlcy48YnI+CiAgICA8YnI+CiAgICA8YnI+CiAgICA8YnI+ CiAgICA8YnI+CiAgICA8YmxvY2txdW90ZQpjaXRlPSJtaWQ6Q0FHSz0za3dVN1ZieVFqTHl2RzVU UzBMdG96T2tCcEdpZE9zQlllb3lwX2VPYXBUQjZnQG1haWwuZ21haWwuY29tIgogICAgICB0eXBl PSJjaXRlIj4KICAgICAgPGRpdiBkaXI9Imx0ciI+CiAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxf ZXh0cmEiPgogICAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPgogICAgICAgICAgICA8 ZGl2PlRoYW5rIHlvdTwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAgICAgPC9kaXY+CiAgICAg IDwvZGl2PgogICAgPC9ibG9ja3F1b3RlPgogICAgPHA+PGJyPgogICAgPC9wPgogIDwvYm9keT4K PC9odG1sPgoKLS0tLS0tLS0tLS0tLS1BNTdGNDI1N0ExRUU5QTJDQTc1NTY1NDAtLQo= --===============2861013256419917111==--