From donny at cloudspin.me Tue Dec 16 14:45:41 2014 Content-Type: multipart/mixed; boundary="===============6396720613524251367==" MIME-Version: 1.0 From: Donny Davis To: users at ovirt.org Subject: Re: [ovirt-users] ERROR 'no free file handlers in pool' while creating VM from template Date: Tue, 16 Dec 2014 12:45:29 -0700 Message-ID: <014e01d01968$da75c9b0$8f615d10$@cloudspin.me> In-Reply-To: CAAegNz0A0y26+_1JRg+HY=sPjNT06DvMCX7ituyd+--9PnvjeQ@mail.gmail.com --===============6396720613524251367== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multipart message in MIME format. ------=3D_NextPart_000_014F_01D0192E.2E17DC10 Content-Type: text/plain; charset=3D"UTF-8" Content-Transfer-Encoding: quoted-printable The only thing I can think of would be file hard and soft limits, but I =3D am no oVirt pro.=3D20 =3D20 'no free file handlers in pool' that would make sense to me=3DE2=3D80=3DA6= =3D20 Donny =3D20 =3D20 From: Tiemen Ruiten [mailto:t.ruiten(a)rdmedia.com]=3D20 Sent: Tuesday, December 16, 2014 12:40 PM To: Donny Davis Cc: Users(a)ovirt.org Subject: Re: [ovirt-users] ERROR 'no free file handlers in pool' while =3D creating VM from template =3D20 About 25-30. The nodes are Xeon(R) CPU E5-2650 0 @ 2.00GHz with 16 =3D hyperthreaded cores and 64 GB of RAM each. At the moment I created the =3D VM, processor load on both nodes was less than 1.=3D20 =3D20 On 16 December 2014 at 20:29, Donny Davis wrote: What is the VM load you are running on your servers? -----Original Message----- From: users-bounces(a)ovirt.org [mailto:users-bounces(a)ovirt.org] On Behal= f =3D Of Tiemen Ruiten Sent: Tuesday, December 16, 2014 12:27 PM To: users(a)ovirt.org Subject: [ovirt-users] ERROR 'no free file handlers in pool' while =3D creating VM from template Hello, I ran into a nasty problem today when creating a new, cloned VM from a =3D template (one virtual 20 GBdisk) on our two-node oVirt cluster: on the =3D node where I started a VM creation job, load skyrocketed and some VMs =3D stopped responding until and after the job failed. Everything recovered =3D without intervention, but this obviously shouldn't happen. I have =3D attached the relevant vdsm log file. The button to create the VM was =3D pressed around 11:17, the first error in the vdsm log is at 11:23:58. The ISO domain is a gluster volume exposed via NFS, the storage domain =3D for the VM's is also a gluster volume. The underlying filesystem is ZFS. The hypervisor nodes are full CentOS 6 installs. I'm guessing the 'no free file handlers in pool' in the vdsm log file is = =3D key here. What can I do to prevent this from happening again? Apart from = =3D not creating new VMs of course :) Tiemen =3D20 --=3D20 Tiemen Ruiten Systems Engineer R&D Media ------=3D_NextPart_000_014F_01D0192E.2E17DC10 Content-Type: text/html; charset=3D"UTF-8" Content-Transfer-Encoding: quoted-printable

The only thing I can think of would be file hard and soft limits, but = =3D I am no oVirt pro.

'no free file =3D handlers in pool' that would make sense to me=3DE2=3D80=3DA6

Donny

From:= =3D = =3D Tiemen Ruiten [mailto:t.ruiten(a)rdmedia.com]
Sent: Tuesday, =3D December 16, 2014 12:40 PM
To: Donny Davis
Cc: =3D Users(a)ovirt.org
Subject: Re: [ovirt-users] ERROR 'no free file = =3D handlers in pool' while creating VM from =3D template

About = =3D 25-30. The nodes are Xeon(R) CPU E5-2650 0 @ 2.00GHz with 16 =3D hyperthreaded cores and 64 GB of RAM each. At the moment I created the =3D VM, processor load on both nodes was less than =3D 1.

On 16 = =3D December 2014 at 20:29, Donny Davis <donny(a)cloudspin.me> wrote:

What is the VM load you are running on your =3D servers?

-----Original Message-----
From: = =3D users-bounces(a)ovirt.org =3D [mailto:users-bounces(a)ovirt.org] = On =3D Behalf Of Tiemen Ruiten
Sent: Tuesday, December 16, 2014 12:27 =3D PM
To: users(a)ovirt.org
Subject: =3D [ovirt-users] ERROR 'no free file handlers in pool' while creating VM =3D from template

Hello,

I ran into a nasty problem today when = =3D creating a new, cloned VM from a template (one virtual 20 GBdisk) on our = =3D two-node oVirt cluster: on the node where I started a VM creation job, =3D load skyrocketed and some VMs stopped responding until and after the job = =3D failed. Everything recovered without intervention, but this obviously =3D shouldn't happen. I have attached the relevant vdsm log file. The button = =3D to create the VM was pressed around 11:17, the first error in the vdsm =3D log is at 11:23:58.

The ISO domain is a gluster volume exposed =3D via NFS, the storage domain for the VM's is also a gluster volume. The =3D underlying filesystem is ZFS.
The hypervisor nodes are full CentOS 6 =3D installs.

I'm guessing the 'no free file handlers in pool' in the = =3D vdsm log file is key here. What can I do to prevent this from happening =3D again? Apart from not creating new VMs of course =3D :)

Tiemen

-- = =3D

Tiemen Ruiten
Systems = =3D Engineer
R&D =3D Media

------=3D_NextPart_000_014F_01D0192E.2E17DC10-- --===============6396720613524251367== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpcGFydCBtZXNzYWdlIGluIE1JTUUgZm9ybWF0LgoKLS0tLS0tPV9OZXh0 UGFydF8wMDBfMDE0Rl8wMUQwMTkyRS4yRTE3REMxMApDb250ZW50LVR5cGU6IHRleHQvcGxhaW47 CgljaGFyc2V0PSJVVEYtOCIKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogcXVvdGVkLXByaW50 YWJsZQoKVGhlIG9ubHkgdGhpbmcgSSBjYW4gdGhpbmsgb2Ygd291bGQgYmUgZmlsZSBoYXJkIGFu ZCBzb2Z0IGxpbWl0cywgYnV0IEkgPQphbSBubyBvVmlydCBwcm8uPTIwCgo9MjAKCidubyBmcmVl IGZpbGUgaGFuZGxlcnMgaW4gcG9vbCcgdGhhdCB3b3VsZCBtYWtlIHNlbnNlIHRvIG1lPUUyPTgw PUE2PTIwCgpEb25ueQoKPTIwCgo9MjAKCkZyb206IFRpZW1lbiBSdWl0ZW4gW21haWx0bzp0LnJ1 aXRlbkByZG1lZGlhLmNvbV09MjAKU2VudDogVHVlc2RheSwgRGVjZW1iZXIgMTYsIDIwMTQgMTI6 NDAgUE0KVG86IERvbm55IERhdmlzCkNjOiBVc2Vyc0BvdmlydC5vcmcKU3ViamVjdDogUmU6IFtv dmlydC11c2Vyc10gRVJST1IgJ25vIGZyZWUgZmlsZSBoYW5kbGVycyBpbiBwb29sJyB3aGlsZSA9 CmNyZWF0aW5nIFZNIGZyb20gdGVtcGxhdGUKCj0yMAoKQWJvdXQgMjUtMzAuIFRoZSBub2RlcyBh cmUgWGVvbihSKSBDUFUgRTUtMjY1MCAwIEAgMi4wMEdIeiB3aXRoIDE2ID0KaHlwZXJ0aHJlYWRl ZCBjb3JlcyBhbmQgNjQgR0Igb2YgUkFNIGVhY2guIEF0IHRoZSBtb21lbnQgSSBjcmVhdGVkIHRo ZSA9ClZNLCBwcm9jZXNzb3IgbG9hZCBvbiBib3RoIG5vZGVzIHdhcyBsZXNzIHRoYW4gMS49MjAK Cj0yMAoKT24gMTYgRGVjZW1iZXIgMjAxNCBhdCAyMDoyOSwgRG9ubnkgRGF2aXMgPGRvbm55QGNs b3Vkc3Bpbi5tZT4gd3JvdGU6CgpXaGF0IGlzIHRoZSBWTSBsb2FkIHlvdSBhcmUgcnVubmluZyBv biB5b3VyIHNlcnZlcnM/CgoKLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0KRnJvbTogdXNlcnMt Ym91bmNlc0BvdmlydC5vcmcgW21haWx0bzp1c2Vycy1ib3VuY2VzQG92aXJ0Lm9yZ10gT24gQmVo YWxmID0KT2YgVGllbWVuIFJ1aXRlbgpTZW50OiBUdWVzZGF5LCBEZWNlbWJlciAxNiwgMjAxNCAx MjoyNyBQTQpUbzogdXNlcnNAb3ZpcnQub3JnClN1YmplY3Q6IFtvdmlydC11c2Vyc10gRVJST1Ig J25vIGZyZWUgZmlsZSBoYW5kbGVycyBpbiBwb29sJyB3aGlsZSA9CmNyZWF0aW5nIFZNIGZyb20g dGVtcGxhdGUKCkhlbGxvLAoKSSByYW4gaW50byBhIG5hc3R5IHByb2JsZW0gdG9kYXkgd2hlbiBj cmVhdGluZyBhIG5ldywgY2xvbmVkIFZNIGZyb20gYSA9CnRlbXBsYXRlIChvbmUgdmlydHVhbCAy MCBHQmRpc2spIG9uIG91ciB0d28tbm9kZSBvVmlydCBjbHVzdGVyOiBvbiB0aGUgPQpub2RlIHdo ZXJlIEkgc3RhcnRlZCBhIFZNIGNyZWF0aW9uIGpvYiwgbG9hZCBza3lyb2NrZXRlZCBhbmQgc29t ZSBWTXMgPQpzdG9wcGVkIHJlc3BvbmRpbmcgdW50aWwgYW5kIGFmdGVyIHRoZSBqb2IgZmFpbGVk LiBFdmVyeXRoaW5nIHJlY292ZXJlZCA9CndpdGhvdXQgaW50ZXJ2ZW50aW9uLCBidXQgdGhpcyBv YnZpb3VzbHkgc2hvdWxkbid0IGhhcHBlbi4gSSBoYXZlID0KYXR0YWNoZWQgdGhlIHJlbGV2YW50 IHZkc20gbG9nIGZpbGUuIFRoZSBidXR0b24gdG8gY3JlYXRlIHRoZSBWTSB3YXMgPQpwcmVzc2Vk IGFyb3VuZCAxMToxNywgdGhlIGZpcnN0IGVycm9yIGluIHRoZSB2ZHNtIGxvZyBpcyBhdCAxMToy Mzo1OC4KClRoZSBJU08gZG9tYWluIGlzIGEgZ2x1c3RlciB2b2x1bWUgZXhwb3NlZCB2aWEgTkZT LCB0aGUgc3RvcmFnZSBkb21haW4gPQpmb3IgdGhlIFZNJ3MgaXMgYWxzbyBhIGdsdXN0ZXIgdm9s dW1lLiBUaGUgdW5kZXJseWluZyBmaWxlc3lzdGVtIGlzIFpGUy4KVGhlIGh5cGVydmlzb3Igbm9k ZXMgYXJlIGZ1bGwgQ2VudE9TIDYgaW5zdGFsbHMuCgpJJ20gZ3Vlc3NpbmcgdGhlICdubyBmcmVl IGZpbGUgaGFuZGxlcnMgaW4gcG9vbCcgaW4gdGhlIHZkc20gbG9nIGZpbGUgaXMgPQprZXkgaGVy ZS4gV2hhdCBjYW4gSSBkbyB0byBwcmV2ZW50IHRoaXMgZnJvbSBoYXBwZW5pbmcgYWdhaW4/IEFw YXJ0IGZyb20gPQpub3QgY3JlYXRpbmcgbmV3IFZNcyBvZiBjb3Vyc2UgOikKClRpZW1lbgoKCgoK Cgo9MjAKCi0tPTIwCgpUaWVtZW4gUnVpdGVuClN5c3RlbXMgRW5naW5lZXIKUiZEIE1lZGlhCgoK LS0tLS0tPV9OZXh0UGFydF8wMDBfMDE0Rl8wMUQwMTkyRS4yRTE3REMxMApDb250ZW50LVR5cGU6 IHRleHQvaHRtbDsKCWNoYXJzZXQ9IlVURi04IgpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiBx dW90ZWQtcHJpbnRhYmxlCgo8aHRtbCB4bWxuczp2PTNEInVybjpzY2hlbWFzLW1pY3Jvc29mdC1j b206dm1sIiA9CnhtbG5zOm89M0QidXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2Zm aWNlIiA9CnhtbG5zOnc9M0QidXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIg PQp4bWxuczptPTNEImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vb2ZmaWNlLzIwMDQvMTIv b21tbCIgPQp4bWxucz0zRCJodHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj48aGVhZD48 bWV0YSA9Cmh0dHAtZXF1aXY9M0RDb250ZW50LVR5cGUgY29udGVudD0zRCJ0ZXh0L2h0bWw7IGNo YXJzZXQ9M0R1dGYtOCI+PG1ldGEgPQpuYW1lPTNER2VuZXJhdG9yIGNvbnRlbnQ9M0QiTWljcm9z b2Z0IFdvcmQgMTQgKGZpbHRlcmVkID0KbWVkaXVtKSI+PHN0eWxlPjwhLS0KLyogRm9udCBEZWZp bml0aW9ucyAqLwpAZm9udC1mYWNlCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsKCXBhbm9zZS0xOjIg MTUgNSAyIDIgMiA0IDMgMiA0O30KQGZvbnQtZmFjZQoJe2ZvbnQtZmFtaWx5OlRhaG9tYTsKCXBh bm9zZS0xOjIgMTEgNiA0IDMgNSA0IDQgMiA0O30KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8KcC5N c29Ob3JtYWwsIGxpLk1zb05vcm1hbCwgZGl2Lk1zb05vcm1hbAoJe21hcmdpbjowaW47CgltYXJn aW4tYm90dG9tOi4wMDAxcHQ7Cglmb250LXNpemU6MTIuMHB0OwoJZm9udC1mYW1pbHk6IlRpbWVz IE5ldyBSb21hbiIsInNlcmlmIjt9CmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsKCXttc28tc3R5 bGUtcHJpb3JpdHk6OTk7Cgljb2xvcjpibHVlOwoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9 CmE6dmlzaXRlZCwgc3Bhbi5Nc29IeXBlcmxpbmtGb2xsb3dlZAoJe21zby1zdHlsZS1wcmlvcml0 eTo5OTsKCWNvbG9yOnB1cnBsZTsKCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQpwLk1zb0Fj ZXRhdGUsIGxpLk1zb0FjZXRhdGUsIGRpdi5Nc29BY2V0YXRlCgl7bXNvLXN0eWxlLXByaW9yaXR5 Ojk5OwoJbXNvLXN0eWxlLWxpbms6IkJhbGxvb24gVGV4dCBDaGFyIjsKCW1hcmdpbjowaW47Cglt YXJnaW4tYm90dG9tOi4wMDAxcHQ7Cglmb250LXNpemU6OC4wcHQ7Cglmb250LWZhbWlseToiVGFo b21hIiwic2Fucy1zZXJpZiI7fQpzcGFuLkVtYWlsU3R5bGUxNwoJe21zby1zdHlsZS10eXBlOnBl cnNvbmFsLXJlcGx5OwoJZm9udC1mYW1pbHk6IkNhbGlicmkiLCJzYW5zLXNlcmlmIjsKCWNvbG9y OiMxRjQ5N0Q7fQpzcGFuLkJhbGxvb25UZXh0Q2hhcgoJe21zby1zdHlsZS1uYW1lOiJCYWxsb29u IFRleHQgQ2hhciI7Cgltc28tc3R5bGUtcHJpb3JpdHk6OTk7Cgltc28tc3R5bGUtbGluazoiQmFs bG9vbiBUZXh0IjsKCWZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIjt9Ci5Nc29DaHBE ZWZhdWx0Cgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7Cglmb250LWZhbWlseToiQ2FsaWJy aSIsInNhbnMtc2VyaWYiO30KQHBhZ2UgV29yZFNlY3Rpb24xCgl7c2l6ZTo4LjVpbiAxMS4waW47 CgltYXJnaW46MS4waW4gMS4waW4gMS4waW4gMS4waW47fQpkaXYuV29yZFNlY3Rpb24xCgl7cGFn ZTpXb3JkU2VjdGlvbjE7fQotLT48L3N0eWxlPjwhLS1baWYgZ3RlIG1zbyA5XT48eG1sPgo8bzpz aGFwZWRlZmF1bHRzIHY6ZXh0PTNEImVkaXQiIHNwaWRtYXg9M0QiMTAyNiIgLz4KPC94bWw+PCFb ZW5kaWZdLS0+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+CjxvOnNoYXBlbGF5b3V0IHY6ZXh0PTNE ImVkaXQiPgo8bzppZG1hcCB2OmV4dD0zRCJlZGl0IiBkYXRhPTNEIjEiIC8+CjwvbzpzaGFwZWxh eW91dD48L3htbD48IVtlbmRpZl0tLT48L2hlYWQ+PGJvZHkgbGFuZz0zREVOLVVTIGxpbms9M0Ri bHVlID0Kdmxpbms9M0RwdXJwbGU+PGRpdiBjbGFzcz0zRFdvcmRTZWN0aW9uMT48cCBjbGFzcz0z RE1zb05vcm1hbD48c3BhbiA9CnN0eWxlPTNEJ2ZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6 IkNhbGlicmkiLCJzYW5zLXNlcmlmIjtjb2xvcjojMUY0OTc9CkQnPlRoZSBvbmx5IHRoaW5nIEkg Y2FuIHRoaW5rIG9mIHdvdWxkIGJlIGZpbGUgaGFyZCBhbmQgc29mdCBsaW1pdHMsIGJ1dCA9Ckkg YW0gbm8gb1ZpcnQgcHJvLiA8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9M0RNc29Ob3Jt YWw+PHNwYW4gPQpzdHlsZT0zRCdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3PQpEJz48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48 L3A+PHAgY2xhc3M9M0RNc29Ob3JtYWw+J25vIGZyZWUgZmlsZSA9CmhhbmRsZXJzIGluIHBvb2wn IHRoYXQgd291bGQgbWFrZSBzZW5zZSB0byBtZT1FMj04MD1BNiA8bzpwPjwvbzpwPjwvcD48cCA9 CmNsYXNzPTNETXNvTm9ybWFsPkRvbm55PHNwYW4gPQpzdHlsZT0zRCdmb250LXNpemU6MTEuMHB0 O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3PQpEJz48bzpw PjwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9M0RNc29Ob3JtYWw+PHNwYW4gPQpzdHlsZT0zRCdm b250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJpIiwic2Fucy1zZXJpZiI7Y29sb3I6 IzFGNDk3PQpEJz48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+PHAgY2xhc3M9M0RNc29Ob3Jt YWw+PHNwYW4gPQpzdHlsZT0zRCdmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiJDYWxpYnJp Iiwic2Fucy1zZXJpZiI7Y29sb3I6IzFGNDk3PQpEJz48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48 L3A+PHAgY2xhc3M9M0RNc29Ob3JtYWw+PGI+PHNwYW4gPQpzdHlsZT0zRCdmb250LXNpemU6MTAu MHB0O2ZvbnQtZmFtaWx5OiJUYWhvbWEiLCJzYW5zLXNlcmlmIic+RnJvbTo8L3NwYW4+PQo8L2I+ PHNwYW4gc3R5bGU9M0QnZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseToiVGFob21hIiwic2Fu cy1zZXJpZiInPiA9ClRpZW1lbiBSdWl0ZW4gW21haWx0bzp0LnJ1aXRlbkByZG1lZGlhLmNvbV0g PGJyPjxiPlNlbnQ6PC9iPiBUdWVzZGF5LCA9CkRlY2VtYmVyIDE2LCAyMDE0IDEyOjQwIFBNPGJy PjxiPlRvOjwvYj4gRG9ubnkgRGF2aXM8YnI+PGI+Q2M6PC9iPiA9ClVzZXJzQG92aXJ0Lm9yZzxi cj48Yj5TdWJqZWN0OjwvYj4gUmU6IFtvdmlydC11c2Vyc10gRVJST1IgJ25vIGZyZWUgZmlsZSA9 CmhhbmRsZXJzIGluIHBvb2wnIHdoaWxlIGNyZWF0aW5nIFZNIGZyb20gPQp0ZW1wbGF0ZTxvOnA+ PC9vOnA+PC9zcGFuPjwvcD48cCA9CmNsYXNzPTNETXNvTm9ybWFsPjxvOnA+Jm5ic3A7PC9vOnA+ PC9wPjxkaXY+PHAgY2xhc3M9M0RNc29Ob3JtYWw+QWJvdXQgPQoyNS0zMC4gVGhlIG5vZGVzIGFy ZSBYZW9uKFIpIENQVSBFNS0yNjUwIDAgQCAyLjAwR0h6IHdpdGggMTYgPQpoeXBlcnRocmVhZGVk IGNvcmVzIGFuZCA2NCBHQiBvZiBSQU0gZWFjaC4gQXQgdGhlIG1vbWVudCBJIGNyZWF0ZWQgdGhl ID0KVk0sIHByb2Nlc3NvciBsb2FkIG9uIGJvdGggbm9kZXMgd2FzIGxlc3MgdGhhbiA9CjEuJm5i c3A7PG86cD48L286cD48L3A+PC9kaXY+PGRpdj48cCA9CmNsYXNzPTNETXNvTm9ybWFsPjxvOnA+ Jm5ic3A7PC9vOnA+PC9wPjxkaXY+PHAgY2xhc3M9M0RNc29Ob3JtYWw+T24gMTYgPQpEZWNlbWJl ciAyMDE0IGF0IDIwOjI5LCBEb25ueSBEYXZpcyAmbHQ7PGEgPQpocmVmPTNEIm1haWx0bzpkb25u eUBjbG91ZHNwaW4ubWUiID0KdGFyZ2V0PTNEIl9ibGFuayI+ZG9ubnlAY2xvdWRzcGluLm1lPC9h PiZndDsgd3JvdGU6PG86cD48L286cD48L3A+PHAgPQpjbGFzcz0zRE1zb05vcm1hbD5XaGF0IGlz IHRoZSBWTSBsb2FkIHlvdSBhcmUgcnVubmluZyBvbiB5b3VyID0Kc2VydmVycz88bzpwPjwvbzpw PjwvcD48ZGl2PjxkaXY+PHAgY2xhc3M9M0RNc29Ob3JtYWwgPQpzdHlsZT0zRCdtYXJnaW4tYm90 dG9tOjEyLjBwdCc+PGJyPi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tPGJyPkZyb206ID0KPGEg aHJlZj0zRCJtYWlsdG86dXNlcnMtYm91bmNlc0BvdmlydC5vcmciPnVzZXJzLWJvdW5jZXNAb3Zp cnQub3JnPC9hPiA9ClttYWlsdG86PGEgPQpocmVmPTNEIm1haWx0bzp1c2Vycy1ib3VuY2VzQG92 aXJ0Lm9yZyI+dXNlcnMtYm91bmNlc0BvdmlydC5vcmc8L2E+XSBPbiA9CkJlaGFsZiBPZiBUaWVt ZW4gUnVpdGVuPGJyPlNlbnQ6IFR1ZXNkYXksIERlY2VtYmVyIDE2LCAyMDE0IDEyOjI3ID0KUE08 YnI+VG86IDxhID0KaHJlZj0zRCJtYWlsdG86dXNlcnNAb3ZpcnQub3JnIj51c2Vyc0BvdmlydC5v cmc8L2E+PGJyPlN1YmplY3Q6ID0KW292aXJ0LXVzZXJzXSBFUlJPUiAnbm8gZnJlZSBmaWxlIGhh bmRsZXJzIGluIHBvb2wnIHdoaWxlIGNyZWF0aW5nIFZNID0KZnJvbSB0ZW1wbGF0ZTxicj48YnI+ SGVsbG8sPGJyPjxicj5JIHJhbiBpbnRvIGEgbmFzdHkgcHJvYmxlbSB0b2RheSB3aGVuID0KY3Jl YXRpbmcgYSBuZXcsIGNsb25lZCBWTSBmcm9tIGEgdGVtcGxhdGUgKG9uZSB2aXJ0dWFsIDIwIEdC ZGlzaykgb24gb3VyID0KdHdvLW5vZGUgb1ZpcnQgY2x1c3Rlcjogb24gdGhlIG5vZGUgd2hlcmUg SSBzdGFydGVkIGEgVk0gY3JlYXRpb24gam9iLCA9CmxvYWQgc2t5cm9ja2V0ZWQgYW5kIHNvbWUg Vk1zIHN0b3BwZWQgcmVzcG9uZGluZyB1bnRpbCBhbmQgYWZ0ZXIgdGhlIGpvYiA9CmZhaWxlZC4g RXZlcnl0aGluZyByZWNvdmVyZWQgd2l0aG91dCBpbnRlcnZlbnRpb24sIGJ1dCB0aGlzIG9idmlv dXNseSA9CnNob3VsZG4ndCBoYXBwZW4uIEkgaGF2ZSBhdHRhY2hlZCB0aGUgcmVsZXZhbnQgdmRz bSBsb2cgZmlsZS4gVGhlIGJ1dHRvbiA9CnRvIGNyZWF0ZSB0aGUgVk0gd2FzIHByZXNzZWQgYXJv dW5kIDExOjE3LCB0aGUgZmlyc3QgZXJyb3IgaW4gdGhlIHZkc20gPQpsb2cgaXMgYXQgMTE6MjM6 NTguPGJyPjxicj5UaGUgSVNPIGRvbWFpbiBpcyBhIGdsdXN0ZXIgdm9sdW1lIGV4cG9zZWQgPQp2 aWEgTkZTLCB0aGUgc3RvcmFnZSBkb21haW4gZm9yIHRoZSBWTSdzIGlzIGFsc28gYSBnbHVzdGVy IHZvbHVtZS4gVGhlID0KdW5kZXJseWluZyBmaWxlc3lzdGVtIGlzIFpGUy48YnI+VGhlIGh5cGVy dmlzb3Igbm9kZXMgYXJlIGZ1bGwgQ2VudE9TIDYgPQppbnN0YWxscy48YnI+PGJyPkknbSBndWVz c2luZyB0aGUgJ25vIGZyZWUgZmlsZSBoYW5kbGVycyBpbiBwb29sJyBpbiB0aGUgPQp2ZHNtIGxv ZyBmaWxlIGlzIGtleSBoZXJlLiBXaGF0IGNhbiBJIGRvIHRvIHByZXZlbnQgdGhpcyBmcm9tIGhh cHBlbmluZyA9CmFnYWluPyBBcGFydCBmcm9tIG5vdCBjcmVhdGluZyBuZXcgVk1zIG9mIGNvdXJz ZSA9CjopPGJyPjxicj5UaWVtZW48YnI+PGJyPjxvOnA+PC9vOnA+PC9wPjwvZGl2PjwvZGl2Pjwv ZGl2PjxwID0KY2xhc3M9M0RNc29Ob3JtYWw+PGJyIGNsZWFyPTNEYWxsPjxvOnA+PC9vOnA+PC9w PjxkaXY+PHAgPQpjbGFzcz0zRE1zb05vcm1hbD48bzpwPiZuYnNwOzwvbzpwPjwvcD48L2Rpdj48 cCBjbGFzcz0zRE1zb05vcm1hbD4tLSA9CjxvOnA+PC9vOnA+PC9wPjxkaXY+PGRpdj48cCBjbGFz cz0zRE1zb05vcm1hbD5UaWVtZW4gUnVpdGVuPGJyPlN5c3RlbXMgPQpFbmdpbmVlcjxicj5SJmFt cDtEID0KTWVkaWE8bzpwPjwvbzpwPjwvcD48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2JvZHk+ PC9odG1sPgotLS0tLS09X05leHRQYXJ0XzAwMF8wMTRGXzAxRDAxOTJFLjJFMTdEQzEwLS0KCg== --===============6396720613524251367==--