From russ at sonicbx.com Tue Jan 7 22:57:03 2014 Content-Type: multipart/mixed; boundary="===============6575675445336831978==" MIME-Version: 1.0 From: Russell Purinton To: users at ovirt.org Subject: [Users] SSD Caching Date: Wed, 08 Jan 2014 03:56:45 +0000 Message-ID: <86a9b704e7fa4477871d73afa3359748@mx.sonic.local> --===============6575675445336831978== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable --_000_86a9b704e7fa4477871d73afa3359748mxsoniclocal_ Content-Type: text/plain; charset=3D"us-ascii" Content-Transfer-Encoding: quoted-printable [20:42] is anybody out there using a good RAM+SSD caching syste= =3D m ahead of gluster storage? [20:42] sorry if that came through twice [20:44] im thinking about making the SSD one giant swap file th= =3D en creating a very large ramdisk in virtual memory and using that as a bloc= =3D k level cache for parts and pieces of virtual machine disk images [20:44] then i think the memory managers would inherently play = =3D the role of storage tiering ie: keeping the hottest data in memory and the = =3D coldest data on swap [20:45] everything i have seen today has been setup as "consu= =3D mer" =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D=3D3D> SSD cache =3D3D=3D3D= =3D3D=3D3D> real disks [20:45] but i'd like to actually do "consumer" =3D3D=3D3D=3D3D>= RAM+S=3D SD cache =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D> real disks [20:46] i realize doing a virtual memory disk means the cache w= =3D ill be cleared on every reboot, and I'm ok with that [20:47] i know this can be done with NFS and cachefilesd(fscach= =3D e), but how could something be integrated into the native gluster clients? [20:47] i'd prefer not to have to access gluster via NFS [20:49] any feedback from this room is greatly appreciated, get= =3D ting someone started to build managed HA cloud hosting --_000_86a9b704e7fa4477871d73afa3359748mxsoniclocal_ Content-Type: text/html; charset=3D"us-ascii" Content-Transfer-Encoding: quoted-printable

[20:42] <soni= =3D crose> is anybody out there using a good RAM+SSD caching system ahea= =3D d of gluster storage?

[20:42] <soni= =3D crose> sorry if that came through twice

[20:44] <soni= =3D crose> im thinking about making the SSD one giant swap file then creatin= =3D g a very large ramdisk in virtual memory and using that as a block level cache for parts and pieces of virtual machine disk images= =3D

[20:44] <soni= =3D crose> then i think the memory managers would inherently play the role o= =3D f storage tiering ie: keeping the hottest data in memory and the coldest data on swap

[20:45] <soni= =3D crose> everything i have seen today has been setup as "= =3D consumer" =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D=3D3D= > SSD cache=3D =3D3D=3D3D=3D3D=3D3D> real disks

[20:45] <soni= =3D crose> but i'd like to actually do "consumer" =3D3D=3D3D=3D3D&= gt; RA=3D M+SSD cache =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D> real = disks

[20:46] <soni= =3D crose> i realize doing a virtual memory disk means the cache will be cle= =3D ared on every reboot, and I'm ok with that

[20:47] <soni= =3D crose> i know this can be done with NFS and cachefilesd(fscache), but ho= =3D w could something be integrated into the native gluster clients?= =3D

[20:47] <soni= =3D crose> i'd prefer not to have to access gluster via NFS

[20:49] <soni= =3D crose> any feedback from this room is greatly appreciated, getting someo= =3D ne started to build managed HA cloud hosting

--_000_86a9b704e7fa4477871d73afa3359748mxsoniclocal_-- --===============6575675445336831978== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" LS1fMDAwXzg2YTliNzA0ZTdmYTQ0Nzc4NzFkNzNhZmEzMzU5NzQ4bXhzb25pY2xvY2FsXwpDb250 ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9InVzLWFzY2lpIgpDb250ZW50LVRyYW5zZmVy LUVuY29kaW5nOiBxdW90ZWQtcHJpbnRhYmxlCgpbMjA6NDJdIDxzb25pY3Jvc2U+IGlzIGFueWJv ZHkgb3V0IHRoZXJlIHVzaW5nIGEgZ29vZCBSQU0rU1NEIGNhY2hpbmcgc3lzdGU9Cm0gYWhlYWQg b2YgZ2x1c3RlciBzdG9yYWdlPwpbMjA6NDJdIDxzb25pY3Jvc2U+IHNvcnJ5IGlmIHRoYXQgY2Ft ZSB0aHJvdWdoIHR3aWNlClsyMDo0NF0gPHNvbmljcm9zZT4gaW0gdGhpbmtpbmcgYWJvdXQgbWFr aW5nIHRoZSBTU0Qgb25lIGdpYW50IHN3YXAgZmlsZSB0aD0KZW4gY3JlYXRpbmcgYSB2ZXJ5IGxh cmdlIHJhbWRpc2sgaW4gdmlydHVhbCBtZW1vcnkgYW5kIHVzaW5nIHRoYXQgYXMgYSBibG9jPQpr IGxldmVsIGNhY2hlIGZvciBwYXJ0cyBhbmQgcGllY2VzIG9mIHZpcnR1YWwgbWFjaGluZSBkaXNr IGltYWdlcwpbMjA6NDRdIDxzb25pY3Jvc2U+IHRoZW4gaSB0aGluayB0aGUgbWVtb3J5IG1hbmFn ZXJzIHdvdWxkIGluaGVyZW50bHkgcGxheSA9CnRoZSByb2xlIG9mIHN0b3JhZ2UgdGllcmluZyBp ZToga2VlcGluZyB0aGUgaG90dGVzdCBkYXRhIGluIG1lbW9yeSBhbmQgdGhlID0KY29sZGVzdCBk YXRhIG9uIHN3YXAKWzIwOjQ1XSA8c29uaWNyb3NlPiBldmVyeXRoaW5nIGkgaGF2ZSBzZWVuIHRv ZGF5IGhhcyBiZWVuIHNldHVwIGFzICAgImNvbnN1PQptZXIiICA9M0Q9M0Q9M0Q+ICBuZXR3b3Jr ID0zRD0zRD0zRD0zRD4gU1NEIGNhY2hlID0zRD0zRD0zRD0zRD4gcmVhbCBkaXNrcwpbMjA6NDVd IDxzb25pY3Jvc2U+IGJ1dCBpJ2QgbGlrZSB0byBhY3R1YWxseSBkbyAiY29uc3VtZXIiID0zRD0z RD0zRD4gUkFNK1M9ClNEIGNhY2hlID0zRD0zRD0zRD4gIG5ldHdvcmsgPTNEPTNEPTNEPiByZWFs IGRpc2tzClsyMDo0Nl0gPHNvbmljcm9zZT4gaSByZWFsaXplIGRvaW5nIGEgdmlydHVhbCBtZW1v cnkgZGlzayBtZWFucyB0aGUgY2FjaGUgdz0KaWxsIGJlIGNsZWFyZWQgb24gZXZlcnkgcmVib290 LCBhbmQgSSdtIG9rIHdpdGggdGhhdApbMjA6NDddIDxzb25pY3Jvc2U+IGkga25vdyB0aGlzIGNh biBiZSBkb25lIHdpdGggTkZTIGFuZCBjYWNoZWZpbGVzZChmc2NhY2g9CmUpLCBidXQgaG93IGNv dWxkIHNvbWV0aGluZyBiZSBpbnRlZ3JhdGVkIGludG8gdGhlIG5hdGl2ZSBnbHVzdGVyIGNsaWVu dHM/ClsyMDo0N10gPHNvbmljcm9zZT4gaSdkIHByZWZlciBub3QgdG8gaGF2ZSB0byBhY2Nlc3Mg Z2x1c3RlciB2aWEgTkZTClsyMDo0OV0gPHNvbmljcm9zZT4gYW55IGZlZWRiYWNrIGZyb20gdGhp cyByb29tIGlzIGdyZWF0bHkgYXBwcmVjaWF0ZWQsIGdldD0KdGluZyBzb21lb25lIHN0YXJ0ZWQg dG8gYnVpbGQgbWFuYWdlZCBIQSBjbG91ZCBob3N0aW5nCgotLV8wMDBfODZhOWI3MDRlN2ZhNDQ3 Nzg3MWQ3M2FmYTMzNTk3NDhteHNvbmljbG9jYWxfCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBj aGFyc2V0PSJ1cy1hc2NpaSIKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogcXVvdGVkLXByaW50 YWJsZQoKPGh0bWwgeG1sbnM6dj0zRCJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOnZtbCIgeG1s bnM6bz0zRCJ1cm46c2NoZW1hcy1taWNyPQpvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6 dz0zRCJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiA9CnhtbG5zOm09M0Qi aHR0cDovL3NjaGVtYXMubWljcm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0z RCJodHRwOj0KLy93d3cudzMub3JnL1RSL1JFQy1odG1sNDAiPgo8aGVhZD4KPG1ldGEgaHR0cC1l cXVpdj0zRCJDb250ZW50LVR5cGUiIGNvbnRlbnQ9M0QidGV4dC9odG1sOyBjaGFyc2V0PTNEdXMt YXNjaWkiPQo+CjxtZXRhIG5hbWU9M0QiR2VuZXJhdG9yIiBjb250ZW50PTNEIk1pY3Jvc29mdCBX b3JkIDE1IChmaWx0ZXJlZCBtZWRpdW0pIj4KPHN0eWxlPjwhLS0KLyogRm9udCBEZWZpbml0aW9u cyAqLwpAZm9udC1mYWNlCgl7Zm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCI7CglwYW5vc2UtMToy IDQgNSAzIDUgNCA2IDMgMiA0O30KQGZvbnQtZmFjZQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7Cglw YW5vc2UtMToyIDE1IDUgMiAyIDIgNCAzIDIgNDt9CkBmb250LWZhY2UKCXtmb250LWZhbWlseTpW ZXJkYW5hOwoJcGFub3NlLTE6MiAxMSA2IDQgMyA1IDQgNCAyIDQ7fQovKiBTdHlsZSBEZWZpbml0 aW9ucyAqLwpwLk1zb05vcm1hbCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsCgl7bWFyZ2lu OjBpbjsKCW1hcmdpbi1ib3R0b206LjAwMDFwdDsKCWZvbnQtc2l6ZToxMS4wcHQ7Cglmb250LWZh bWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO30KYTpsaW5rLCBzcGFuLk1zb0h5cGVybGluawoJ e21zby1zdHlsZS1wcmlvcml0eTo5OTsKCWNvbG9yOiMwNTYzQzE7Cgl0ZXh0LWRlY29yYXRpb246 dW5kZXJsaW5lO30KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkCgl7bXNvLXN0 eWxlLXByaW9yaXR5Ojk5OwoJY29sb3I6Izk1NEY3MjsKCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxp bmU7fQpzcGFuLkVtYWlsU3R5bGUxNwoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLWNvbXBvc2U7 Cglmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiOwoJY29sb3I6d2luZG93dGV4dDt9 Ci5Nc29DaHBEZWZhdWx0Cgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7Cglmb250LWZhbWls eToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO30KQHBhZ2UgV29yZFNlY3Rpb24xCgl7c2l6ZTo4LjVp biAxMS4waW47CgltYXJnaW46MS4waW4gMS4waW4gMS4waW4gMS4waW47fQpkaXYuV29yZFNlY3Rp b24xCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQotLT48L3N0eWxlPjwhLS1baWYgZ3RlIG1zbyA5XT48 eG1sPgo8bzpzaGFwZWRlZmF1bHRzIHY6ZXh0PTNEImVkaXQiIHNwaWRtYXg9M0QiMTAyNiIgLz4K PC94bWw+PCFbZW5kaWZdLS0+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+CjxvOnNoYXBlbGF5b3V0 IHY6ZXh0PTNEImVkaXQiPgo8bzppZG1hcCB2OmV4dD0zRCJlZGl0IiBkYXRhPTNEIjEiIC8+Cjwv bzpzaGFwZWxheW91dD48L3htbD48IVtlbmRpZl0tLT4KPC9oZWFkPgo8Ym9keSBsYW5nPTNEIkVO LVVTIiBsaW5rPTNEIiMwNTYzQzEiIHZsaW5rPTNEIiM5NTRGNzIiPgo8ZGl2IGNsYXNzPTNEIldv cmRTZWN0aW9uMSI+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCIgc3R5bGU9M0QibWFyZ2luLWxlZnQ6 MTAuMHB0O3RleHQtaW5kZW50Oi0xMC4wcHQ7dGV4dD0KLWF1dG9zcGFjZTpub25lIj4KPHNwYW4g c3R5bGU9M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDss JnF1b3Q7c2Fucy1zPQplcmlmJnF1b3Q7O2NvbG9yOmdyYXk7YmFja2dyb3VuZDp3aGl0ZTttc28t aGlnaGxpZ2h0OndoaXRlIj5bMjA6NDJdCjwvc3Bhbj48c3BhbiBzdHlsZT0zRCJmb250LXNpemU6 OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVvdD0KO3NhbnMtc2VyaWYm cXVvdDs7Y29sb3I6bmF2eTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdobGlnaHQ6d2hpdGUiPiZs dDtzb25pPQpjcm9zZSZndDsgaXMgYW55Ym9keSBvdXQgdGhlcmUgdXNpbmcgYSBnb29kIFJBTSYj NDM7U1NEIGNhY2hpbmcgc3lzdGVtIGFoZWE9CmQgb2YgZ2x1c3RlciBzdG9yYWdlPzxvOnA+PC9v OnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIiBzdHlsZT0zRCJtYXJnaW4tbGVm dDoxMC4wcHQ7dGV4dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0PQotYXV0b3NwYWNlOm5vbmUiPgo8c3Bh biBzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90 OywmcXVvdDtzYW5zLXM9CmVyaWYmcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRlO21z by1oaWdobGlnaHQ6d2hpdGUiPlsyMDo0Ml0KPC9zcGFuPjxzcGFuIHN0eWxlPTNEImZvbnQtc2l6 ZTo5LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90PQo7c2Fucy1zZXJp ZiZxdW90Oztjb2xvcjpuYXZ5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0ZSI+ Jmx0O3Nvbmk9CmNyb3NlJmd0OyBzb3JyeSBpZiB0aGF0IGNhbWUgdGhyb3VnaCB0d2ljZTxvOnA+ PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIiBzdHlsZT0zRCJtYXJnaW4t bGVmdDoxMC4wcHQ7dGV4dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0PQotYXV0b3NwYWNlOm5vbmUiPgo8 c3BhbiBzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZx dW90OywmcXVvdDtzYW5zLXM9CmVyaWYmcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRl O21zby1oaWdobGlnaHQ6d2hpdGUiPlsyMDo0NF0KPC9zcGFuPjxzcGFuIHN0eWxlPTNEImZvbnQt c2l6ZTo5LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90PQo7c2Fucy1z ZXJpZiZxdW90Oztjb2xvcjpuYXZ5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0 ZSI+Jmx0O3Nvbmk9CmNyb3NlJmd0OyBpbSB0aGlua2luZyBhYm91dCBtYWtpbmcgdGhlIFNTRCBv bmUgZ2lhbnQgc3dhcCBmaWxlIHRoZW4gY3JlYXRpbj0KZyBhIHZlcnkgbGFyZ2UgcmFtZGlzayBp biB2aXJ0dWFsIG1lbW9yeSBhbmQgdXNpbmcgdGhhdCBhcyBhCiBibG9jayBsZXZlbCBjYWNoZSBm b3IgcGFydHMgYW5kIHBpZWNlcyBvZiB2aXJ0dWFsIG1hY2hpbmUgZGlzayBpbWFnZXM8bzpwPj0K PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9M0QiTXNvTm9ybWFsIiBzdHlsZT0zRCJtYXJnaW4t bGVmdDoxMC4wcHQ7dGV4dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0PQotYXV0b3NwYWNlOm5vbmUiPgo8 c3BhbiBzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZx dW90OywmcXVvdDtzYW5zLXM9CmVyaWYmcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRl O21zby1oaWdobGlnaHQ6d2hpdGUiPlsyMDo0NF0KPC9zcGFuPjxzcGFuIHN0eWxlPTNEImZvbnQt c2l6ZTo5LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90PQo7c2Fucy1z ZXJpZiZxdW90Oztjb2xvcjpuYXZ5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0 ZSI+Jmx0O3Nvbmk9CmNyb3NlJmd0OyB0aGVuIGkgdGhpbmsgdGhlIG1lbW9yeSBtYW5hZ2VycyB3 b3VsZCBpbmhlcmVudGx5IHBsYXkgdGhlIHJvbGUgbz0KZiBzdG9yYWdlIHRpZXJpbmcgaWU6IGtl ZXBpbmcgdGhlIGhvdHRlc3QgZGF0YSBpbiBtZW1vcnkgYW5kCiB0aGUgY29sZGVzdCBkYXRhIG9u IHN3YXA8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCIgc3R5bGU9 M0QibWFyZ2luLWxlZnQ6MTAuMHB0O3RleHQtaW5kZW50Oi0xMC4wcHQ7dGV4dD0KLWF1dG9zcGFj ZTpub25lIj4KPHNwYW4gc3R5bGU9M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90 O1ZlcmRhbmEmcXVvdDssJnF1b3Q7c2Fucy1zPQplcmlmJnF1b3Q7O2NvbG9yOmdyYXk7YmFja2dy b3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0OndoaXRlIj5bMjA6NDVdCjwvc3Bhbj48c3BhbiBzdHls ZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVv dD0KO3NhbnMtc2VyaWYmcXVvdDs7Y29sb3I6bmF2eTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdo bGlnaHQ6d2hpdGUiPiZsdDtzb25pPQpjcm9zZSZndDsgZXZlcnl0aGluZyBpIGhhdmUgc2VlbiB0 b2RheSBoYXMgYmVlbiBzZXR1cCBhcyZuYnNwOyZuYnNwOyAmcXVvdDs9CmNvbnN1bWVyJnF1b3Q7 Jm5ic3A7ID0zRD0zRD0zRCZndDsmbmJzcDsgbmV0d29yayA9M0Q9M0Q9M0Q9M0QmZ3Q7IFNTRCBj YWNoZT0KID0zRD0zRD0zRD0zRCZndDsgcmVhbCBkaXNrczxvOnA+PC9vOnA+PC9zcGFuPjwvcD4K PHAgY2xhc3M9M0QiTXNvTm9ybWFsIiBzdHlsZT0zRCJtYXJnaW4tbGVmdDoxMC4wcHQ7dGV4dC1p bmRlbnQ6LTEwLjBwdDt0ZXh0PQotYXV0b3NwYWNlOm5vbmUiPgo8c3BhbiBzdHlsZT0zRCJmb250 LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVvdDtzYW5zLXM9 CmVyaWYmcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdobGlnaHQ6d2hp dGUiPlsyMDo0NV0KPC9zcGFuPjxzcGFuIHN0eWxlPTNEImZvbnQtc2l6ZTo5LjBwdDtmb250LWZh bWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90PQo7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjpu YXZ5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0ZSI+Jmx0O3Nvbmk9CmNyb3Nl Jmd0OyBidXQgaSdkIGxpa2UgdG8gYWN0dWFsbHkgZG8gJnF1b3Q7Y29uc3VtZXImcXVvdDsgPTNE PTNEPTNEJmd0OyBSQT0KTSYjNDM7U1NEIGNhY2hlID0zRD0zRD0zRCZndDsmbmJzcDsgbmV0d29y ayA9M0Q9M0Q9M0QmZ3Q7IHJlYWwgZGlza3M8bzpwPjwvPQpvOnA+PC9zcGFuPjwvcD4KPHAgY2xh c3M9M0QiTXNvTm9ybWFsIiBzdHlsZT0zRCJtYXJnaW4tbGVmdDoxMC4wcHQ7dGV4dC1pbmRlbnQ6 LTEwLjBwdDt0ZXh0PQotYXV0b3NwYWNlOm5vbmUiPgo8c3BhbiBzdHlsZT0zRCJmb250LXNpemU6 OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVvdDtzYW5zLXM9CmVyaWYm cXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdobGlnaHQ6d2hpdGUiPlsy MDo0Nl0KPC9zcGFuPjxzcGFuIHN0eWxlPTNEImZvbnQtc2l6ZTo5LjBwdDtmb250LWZhbWlseTom cXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90PQo7c2Fucy1zZXJpZiZxdW90Oztjb2xvcjpuYXZ5O2Jh Y2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0ZSI+Jmx0O3Nvbmk9CmNyb3NlJmd0OyBp IHJlYWxpemUgZG9pbmcgYSB2aXJ0dWFsIG1lbW9yeSBkaXNrIG1lYW5zIHRoZSBjYWNoZSB3aWxs IGJlIGNsZT0KYXJlZCBvbiBldmVyeSByZWJvb3QsIGFuZCBJJ20gb2sgd2l0aCB0aGF0PG86cD48 L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0zRCJNc29Ob3JtYWwiIHN0eWxlPTNEIm1hcmdpbi1s ZWZ0OjEwLjBwdDt0ZXh0LWluZGVudDotMTAuMHB0O3RleHQ9Ci1hdXRvc3BhY2U6bm9uZSI+Cjxz cGFuIHN0eWxlPTNEImZvbnQtc2l6ZTo5LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1 b3Q7LCZxdW90O3NhbnMtcz0KZXJpZiZxdW90Oztjb2xvcjpncmF5O2JhY2tncm91bmQ6d2hpdGU7 bXNvLWhpZ2hsaWdodDp3aGl0ZSI+WzIwOjQ3XQo8L3NwYW4+PHNwYW4gc3R5bGU9M0QiZm9udC1z aXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q9CjtzYW5zLXNl cmlmJnF1b3Q7O2NvbG9yOm5hdnk7YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0OndoaXRl Ij4mbHQ7c29uaT0KY3Jvc2UmZ3Q7IGkga25vdyB0aGlzIGNhbiBiZSBkb25lIHdpdGggTkZTIGFu ZCBjYWNoZWZpbGVzZChmc2NhY2hlKSwgYnV0IGhvPQp3IGNvdWxkIHNvbWV0aGluZyBiZSBpbnRl Z3JhdGVkIGludG8gdGhlIG5hdGl2ZSBnbHVzdGVyIGNsaWVudHM/PG86cD48L286cD49Cjwvc3Bh bj48L3A+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCIgc3R5bGU9M0QibWFyZ2luLWxlZnQ6MTAuMHB0 O3RleHQtaW5kZW50Oi0xMC4wcHQ7dGV4dD0KLWF1dG9zcGFjZTpub25lIj4KPHNwYW4gc3R5bGU9 M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q7 c2Fucy1zPQplcmlmJnF1b3Q7O2NvbG9yOmdyYXk7YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxp Z2h0OndoaXRlIj5bMjA6NDddCjwvc3Bhbj48c3BhbiBzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7 Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVvdD0KO3NhbnMtc2VyaWYmcXVvdDs7 Y29sb3I6bmF2eTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdobGlnaHQ6d2hpdGUiPiZsdDtzb25p PQpjcm9zZSZndDsgaSdkIHByZWZlciBub3QgdG8gaGF2ZSB0byBhY2Nlc3MgZ2x1c3RlciB2aWEg TkZTPG86cD48L286cD48L3NwYW49Cj48L3A+CjxwIGNsYXNzPTNEIk1zb05vcm1hbCI+PHNwYW4g c3R5bGU9M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1Zlcj0KZGFuYSZxdW90 OywmcXVvdDtzYW5zLXNlcmlmJnF1b3Q7O2NvbG9yOmdyYXk7YmFja2dyb3VuZDp3aGl0ZTttc28t aGlnaGxpZ2h0PQo6d2hpdGUiPlsyMDo0OV0KPC9zcGFuPjxzcGFuIHN0eWxlPTNEImZvbnQtc2l6 ZTo5LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90PQo7c2Fucy1zZXJp ZiZxdW90Oztjb2xvcjpuYXZ5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0ZSI+ Jmx0O3Nvbmk9CmNyb3NlJmd0OyBhbnkgZmVlZGJhY2sgZnJvbSB0aGlzIHJvb20gaXMgZ3JlYXRs eSBhcHByZWNpYXRlZCwgZ2V0dGluZyBzb21lbz0KbmUgc3RhcnRlZCB0byBidWlsZCBtYW5hZ2Vk IEhBIGNsb3VkIGhvc3Rpbmc8L3NwYW4+PG86cD48L286cD48L3A+CjwvZGl2Pgo8L2JvZHk+Cjwv aHRtbD4KCi0tXzAwMF84NmE5YjcwNGU3ZmE0NDc3ODcxZDczYWZhMzM1OTc0OG14c29uaWNsb2Nh bF8tLQo= --===============6575675445336831978==-- From darrell.budic at zenfire.com Wed Jan 8 22:44:38 2014 Content-Type: multipart/mixed; boundary="===============3775077665713835670==" MIME-Version: 1.0 From: Darrell Budic To: users at ovirt.org Subject: Re: [Users] SSD Caching Date: Wed, 08 Jan 2014 21:44:35 -0600 Message-ID: In-Reply-To: 86a9b704e7fa4477871d73afa3359748@mx.sonic.local --===============3775077665713835670== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable --Apple-Mail=3D_0A9B4D64-6C36-4EE3-AE89-2D7BABBBC310 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=3Dwindows-1252 Stick your bricks on ZFS and let it do it for you. Works well, although =3D I haven=3D92t done much benchmarking of it. My test setup is described in = =3D the thread under [Users] Creation of preallocated disk with Gluster =3D replication. I=3D92ve seen some blog posts here and there about gluster on = =3D ZFS for this reason too. -Darrell On Jan 7, 2014, at 9:56 PM, Russell Purinton wrote: > [20:42] is anybody out there using a good RAM+SSD caching =3D system ahead of gluster storage? > [20:42] sorry if that came through twice > [20:44] im thinking about making the SSD one giant swap =3D file then creating a very large ramdisk in virtual memory and using that = =3D as a block level cache for parts and pieces of virtual machine disk =3D images > [20:44] then i think the memory managers would inherently =3D play the role of storage tiering ie: keeping the hottest data in memory =3D and the coldest data on swap > [20:45] everything i have seen today has been setup as =3D "consumer" =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D=3D3D> SSD cache =3D3D= =3D3D=3D3D=3D3D> =3D real disks > [20:45] but i'd like to actually do "consumer" =3D3D=3D3D=3D3= D> =3D RAM+SSD cache =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D> real disks > [20:46] i realize doing a virtual memory disk means the =3D cache will be cleared on every reboot, and I'm ok with that > [20:47] i know this can be done with NFS and =3D cachefilesd(fscache), but how could something be integrated into the =3D native gluster clients? > [20:47] i'd prefer not to have to access gluster via NFS > [20:49] any feedback from this room is greatly =3D appreciated, getting someone started to build managed HA cloud hosting > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users --Apple-Mail=3D_0A9B4D64-6C36-4EE3-AE89-2D7BABBBC310 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=3Dwindows-1252 Stick =3D your bricks on ZFS and let it do it for you. Works well, although I =3D haven=3D92t done much benchmarking of it. My test setup is described in =3D the thread under [Users] Creation of preallocated disk with Gluster =3D replication. I=3D92ve seen some blog posts here and there about gluster on = =3D ZFS for this reason =3D too.

-Darrell

On Jan 7, =3D 2014, at 9:56 PM, Russell Purinton <russ(a)sonicbx.com> =3D wrote:

[20:42] <sonicrose> = =3D is anybody out there using a good RAM+SSD caching system ahead of =3D gluster storage?
[20:42] <sonicrose> = =3D sorry if that came through twice
[20:44] <sonicrose> = =3D im thinking about making the SSD one giant swap file then creating a =3D very large ramdisk in virtual memory and using that as a block level cache for parts and pieces of virtual machine disk =3D images
[20:44] <sonicrose> = =3D then i think the memory managers would inherently play the role of =3D storage tiering ie: keeping the hottest data in memory and the coldest data on swap
[20:45] <sonicrose> = =3D everything i have seen today has been setup as =3D "consumer" =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D=3D3D>= SSD cache =3D =3D3D=3D3D=3D3D=3D3D> real disks
[20:45] <sonicrose> = =3D but i'd like to actually do "consumer" =3D3D=3D3D=3D3D> RAM+SSD cache = =3D =3D3D=3D3D=3D3D> network =3D3D=3D3D=3D3D> real =3D disks
[20:46] <sonicrose> = =3D i realize doing a virtual memory disk means the cache will be cleared on = =3D every reboot, and I'm ok with that
[20:47] <sonicrose> = =3D i know this can be done with NFS and cachefilesd(fscache), but how could = =3D something be integrated into the native gluster =3D clients?
[20:47] <sonicrose> = =3D i'd prefer not to have to access gluster via NFS
[20:49] <sonicrose> = =3D any feedback from this room is greatly appreciated, getting someone =3D started to build managed HA cloud hosting

_______________________________________________
Users mailing =3D list
Users(a)ovirt.org
http://lists.o= virt.=3D org/mailman/listinfo/users

=3D --Apple-Mail=3D_0A9B4D64-6C36-4EE3-AE89-2D7BABBBC310-- --===============3775077665713835670== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" Ci0tQXBwbGUtTWFpbD1fMEE5QjRENjQtNkMzNi00RUUzLUFFODktMkQ3QkFCQkJDMzEwCkNvbnRl bnQtVHJhbnNmZXItRW5jb2Rpbmc6IHF1b3RlZC1wcmludGFibGUKQ29udGVudC1UeXBlOiB0ZXh0 L3BsYWluOwoJY2hhcnNldD13aW5kb3dzLTEyNTIKClN0aWNrIHlvdXIgYnJpY2tzIG9uIFpGUyBh bmQgbGV0IGl0IGRvIGl0IGZvciB5b3UuIFdvcmtzIHdlbGwsIGFsdGhvdWdoID0KSSBoYXZlbj05 MnQgZG9uZSBtdWNoIGJlbmNobWFya2luZyBvZiBpdC4gTXkgdGVzdCBzZXR1cCBpcyBkZXNjcmli ZWQgaW4gPQp0aGUgdGhyZWFkIHVuZGVyIFtVc2Vyc10gQ3JlYXRpb24gb2YgcHJlYWxsb2NhdGVk IGRpc2sgd2l0aCBHbHVzdGVyID0KcmVwbGljYXRpb24uIEk9OTJ2ZSBzZWVuIHNvbWUgYmxvZyBw b3N0cyBoZXJlIGFuZCB0aGVyZSBhYm91dCBnbHVzdGVyIG9uID0KWkZTIGZvciB0aGlzIHJlYXNv biB0b28uCgogLURhcnJlbGwKCk9uIEphbiA3LCAyMDE0LCBhdCA5OjU2IFBNLCBSdXNzZWxsIFB1 cmludG9uIDxydXNzQHNvbmljYnguY29tPiB3cm90ZToKCj4gWzIwOjQyXSA8c29uaWNyb3NlPiBp cyBhbnlib2R5IG91dCB0aGVyZSB1c2luZyBhIGdvb2QgUkFNK1NTRCBjYWNoaW5nID0Kc3lzdGVt IGFoZWFkIG9mIGdsdXN0ZXIgc3RvcmFnZT8KPiBbMjA6NDJdIDxzb25pY3Jvc2U+IHNvcnJ5IGlm IHRoYXQgY2FtZSB0aHJvdWdoIHR3aWNlCj4gWzIwOjQ0XSA8c29uaWNyb3NlPiBpbSB0aGlua2lu ZyBhYm91dCBtYWtpbmcgdGhlIFNTRCBvbmUgZ2lhbnQgc3dhcCA9CmZpbGUgdGhlbiBjcmVhdGlu ZyBhIHZlcnkgbGFyZ2UgcmFtZGlzayBpbiB2aXJ0dWFsIG1lbW9yeSBhbmQgdXNpbmcgdGhhdCA9 CmFzIGEgYmxvY2sgbGV2ZWwgY2FjaGUgZm9yIHBhcnRzIGFuZCBwaWVjZXMgb2YgdmlydHVhbCBt YWNoaW5lIGRpc2sgPQppbWFnZXMKPiBbMjA6NDRdIDxzb25pY3Jvc2U+IHRoZW4gaSB0aGluayB0 aGUgbWVtb3J5IG1hbmFnZXJzIHdvdWxkIGluaGVyZW50bHkgPQpwbGF5IHRoZSByb2xlIG9mIHN0 b3JhZ2UgdGllcmluZyBpZToga2VlcGluZyB0aGUgaG90dGVzdCBkYXRhIGluIG1lbW9yeSA9CmFu ZCB0aGUgY29sZGVzdCBkYXRhIG9uIHN3YXAKPiBbMjA6NDVdIDxzb25pY3Jvc2U+IGV2ZXJ5dGhp bmcgaSBoYXZlIHNlZW4gdG9kYXkgaGFzIGJlZW4gc2V0dXAgYXMgICA9CiJjb25zdW1lciIgID0z RD0zRD0zRD4gIG5ldHdvcmsgPTNEPTNEPTNEPTNEPiBTU0QgY2FjaGUgPTNEPTNEPTNEPTNEPiA9 CnJlYWwgZGlza3MKPiBbMjA6NDVdIDxzb25pY3Jvc2U+IGJ1dCBpJ2QgbGlrZSB0byBhY3R1YWxs eSBkbyAiY29uc3VtZXIiID0zRD0zRD0zRD4gPQpSQU0rU1NEIGNhY2hlID0zRD0zRD0zRD4gIG5l dHdvcmsgPTNEPTNEPTNEPiByZWFsIGRpc2tzCj4gWzIwOjQ2XSA8c29uaWNyb3NlPiBpIHJlYWxp emUgZG9pbmcgYSB2aXJ0dWFsIG1lbW9yeSBkaXNrIG1lYW5zIHRoZSA9CmNhY2hlIHdpbGwgYmUg Y2xlYXJlZCBvbiBldmVyeSByZWJvb3QsIGFuZCBJJ20gb2sgd2l0aCB0aGF0Cj4gWzIwOjQ3XSA8 c29uaWNyb3NlPiBpIGtub3cgdGhpcyBjYW4gYmUgZG9uZSB3aXRoIE5GUyBhbmQgPQpjYWNoZWZp bGVzZChmc2NhY2hlKSwgYnV0IGhvdyBjb3VsZCBzb21ldGhpbmcgYmUgaW50ZWdyYXRlZCBpbnRv IHRoZSA9Cm5hdGl2ZSBnbHVzdGVyIGNsaWVudHM/Cj4gWzIwOjQ3XSA8c29uaWNyb3NlPiBpJ2Qg cHJlZmVyIG5vdCB0byBoYXZlIHRvIGFjY2VzcyBnbHVzdGVyIHZpYSBORlMKPiBbMjA6NDldIDxz b25pY3Jvc2U+IGFueSBmZWVkYmFjayBmcm9tIHRoaXMgcm9vbSBpcyBncmVhdGx5ID0KYXBwcmVj aWF0ZWQsIGdldHRpbmcgc29tZW9uZSBzdGFydGVkIHRvIGJ1aWxkIG1hbmFnZWQgSEEgY2xvdWQg aG9zdGluZwo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f Cj4gVXNlcnMgbWFpbGluZyBsaXN0Cj4gVXNlcnNAb3ZpcnQub3JnCj4gaHR0cDovL2xpc3RzLm92 aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzCgoKLS1BcHBsZS1NYWlsPV8wQTlCNEQ2NC02 QzM2LTRFRTMtQUU4OS0yRDdCQUJCQkMzMTAKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogcXVv dGVkLXByaW50YWJsZQpDb250ZW50LVR5cGU6IHRleHQvaHRtbDsKCWNoYXJzZXQ9d2luZG93cy0x MjUyCgo8aHRtbD48aGVhZD48bWV0YSBodHRwLWVxdWl2PTNEIkNvbnRlbnQtVHlwZSIgY29udGVu dD0zRCJ0ZXh0L2h0bWwgPQpjaGFyc2V0PTNEd2luZG93cy0xMjUyIj48L2hlYWQ+PGJvZHkgc3R5 bGU9M0Qid29yZC13cmFwOiBicmVhay13b3JkOyA9Ci13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsg LXdlYmtpdC1saW5lLWJyZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsiPlN0aWNrID0KeW91ciBicmlj a3Mgb24gWkZTIGFuZCBsZXQgaXQgZG8gaXQgZm9yIHlvdS4gV29ya3Mgd2VsbCwgYWx0aG91Z2gg SSA9CmhhdmVuPTkydCBkb25lIG11Y2ggYmVuY2htYXJraW5nIG9mIGl0LiBNeSB0ZXN0IHNldHVw IGlzIGRlc2NyaWJlZCBpbiA9CnRoZSB0aHJlYWQgdW5kZXIgW1VzZXJzXSBDcmVhdGlvbiBvZiBw cmVhbGxvY2F0ZWQgZGlzayB3aXRoIEdsdXN0ZXIgPQpyZXBsaWNhdGlvbi4gST05MnZlIHNlZW4g c29tZSBibG9nIHBvc3RzIGhlcmUgYW5kIHRoZXJlIGFib3V0IGdsdXN0ZXIgb24gPQpaRlMgZm9y IHRoaXMgcmVhc29uID0KdG9vLjxkaXY+PGJyPjwvZGl2PjxkaXY+Jm5ic3A7LURhcnJlbGw8YnI+ PGRpdj48YnI+PGRpdj48ZGl2Pk9uIEphbiA3LCA9CjIwMTQsIGF0IDk6NTYgUE0sIFJ1c3NlbGwg UHVyaW50b24gJmx0OzxhID0KaHJlZj0zRCJtYWlsdG86cnVzc0Bzb25pY2J4LmNvbSI+cnVzc0Bz b25pY2J4LmNvbTwvYT4mZ3Q7ID0Kd3JvdGU6PC9kaXY+PGJyIGNsYXNzPTNEIkFwcGxlLWludGVy Y2hhbmdlLW5ld2xpbmUiPjxibG9ja3F1b3RlID0KdHlwZT0zRCJjaXRlIj4KCjxtZXRhIGh0dHAt ZXF1aXY9M0QiQ29udGVudC1UeXBlIiBjb250ZW50PTNEInRleHQvaHRtbDsgPQpjaGFyc2V0PTNE dXMtYXNjaWkiPgo8bWV0YSBuYW1lPTNEIkdlbmVyYXRvciIgY29udGVudD0zRCJNaWNyb3NvZnQg V29yZCAxNSAoZmlsdGVyZWQgPQptZWRpdW0pIj4KPHN0eWxlPjwhLS0KLyogRm9udCBEZWZpbml0 aW9ucyAqLwpAZm9udC1mYWNlCgl7Zm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCI7CglwYW5vc2Ut MToyIDQgNSAzIDUgNCA2IDMgMiA0O30KQGZvbnQtZmFjZQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7 CglwYW5vc2UtMToyIDE1IDUgMiAyIDIgNCAzIDIgNDt9CkBmb250LWZhY2UKCXtmb250LWZhbWls eTpWZXJkYW5hOwoJcGFub3NlLTE6MiAxMSA2IDQgMyA1IDQgNCAyIDQ7fQovKiBTdHlsZSBEZWZp bml0aW9ucyAqLwpwLk1zb05vcm1hbCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsCgl7bWFy Z2luOjBpbjsKCW1hcmdpbi1ib3R0b206LjAwMDFwdDsKCWZvbnQtc2l6ZToxMS4wcHQ7Cglmb250 LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO30KYTpsaW5rLCBzcGFuLk1zb0h5cGVybGlu awoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsKCWNvbG9yOiMwNTYzQzE7Cgl0ZXh0LWRlY29yYXRp b246dW5kZXJsaW5lO30KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkCgl7bXNv LXN0eWxlLXByaW9yaXR5Ojk5OwoJY29sb3I6Izk1NEY3MjsKCXRleHQtZGVjb3JhdGlvbjp1bmRl cmxpbmU7fQpzcGFuLkVtYWlsU3R5bGUxNwoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLWNvbXBv c2U7Cglmb250LWZhbWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiOwoJY29sb3I6d2luZG93dGV4 dDt9Ci5Nc29DaHBEZWZhdWx0Cgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7Cglmb250LWZh bWlseToiQ2FsaWJyaSIsInNhbnMtc2VyaWYiO30KQHBhZ2UgV29yZFNlY3Rpb24xCgl7c2l6ZTo4 LjVpbiAxMS4waW47CgltYXJnaW46MS4waW4gMS4waW4gMS4waW4gMS4waW47fQpkaXYuV29yZFNl Y3Rpb24xCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQotLT48L3N0eWxlPjwhLS1baWYgZ3RlIG1zbyA5 XT48eG1sPgo8bzpzaGFwZWRlZmF1bHRzIHY6ZXh0PTNEImVkaXQiIHNwaWRtYXg9M0QiMTAyNiIg Lz4KPC94bWw+PCFbZW5kaWZdLS0+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+CjxvOnNoYXBlbGF5 b3V0IHY6ZXh0PTNEImVkaXQiPgo8bzppZG1hcCB2OmV4dD0zRCJlZGl0IiBkYXRhPTNEIjEiIC8+ CjwvbzpzaGFwZWxheW91dD48L3htbD48IVtlbmRpZl0tLT4KCjxkaXYgbGFuZz0zRCJFTi1VUyIg bGluaz0zRCIjMDU2M0MxIiB2bGluaz0zRCIjOTU0RjcyIj4KPGRpdiBjbGFzcz0zRCJXb3JkU2Vj dGlvbjEiPjxwIGNsYXNzPTNEIk1zb05vcm1hbCIgPQpzdHlsZT0zRCJtYXJnaW4tbGVmdDoxMC4w cHQ7dGV4dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0LWF1dG9zcGFjZTpub25lIj4KPHNwYW4gPQpzdHls ZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVv dDtzYW5zLXNlcmlmJj0KcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdo bGlnaHQ6d2hpdGUiPlsyMDo0Ml0KPC9zcGFuPjxzcGFuID0Kc3R5bGU9M0QiZm9udC1zaXplOjku MHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiY9CnF1 b3Q7O2NvbG9yOm5hdnk7YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0OndoaXRlIj4mbHQ7 c29uaWNyb3NlJmd0OyA9CmlzIGFueWJvZHkgb3V0IHRoZXJlIHVzaW5nIGEgZ29vZCBSQU0rU1NE IGNhY2hpbmcgc3lzdGVtIGFoZWFkIG9mID0KZ2x1c3RlciBzdG9yYWdlPzxvOnA+PC9vOnA+PC9z cGFuPjwvcD48cCBjbGFzcz0zRCJNc29Ob3JtYWwiID0Kc3R5bGU9M0QibWFyZ2luLWxlZnQ6MTAu MHB0O3RleHQtaW5kZW50Oi0xMC4wcHQ7dGV4dC1hdXRvc3BhY2U6bm9uZSI+CjxzcGFuID0Kc3R5 bGU9M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1 b3Q7c2Fucy1zZXJpZiY9CnF1b3Q7O2NvbG9yOmdyYXk7YmFja2dyb3VuZDp3aGl0ZTttc28taGln aGxpZ2h0OndoaXRlIj5bMjA6NDJdCjwvc3Bhbj48c3BhbiA9CnN0eWxlPTNEImZvbnQtc2l6ZTo5 LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmPQpx dW90Oztjb2xvcjpuYXZ5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0ZSI+Jmx0 O3Nvbmljcm9zZSZndDsgPQpzb3JyeSBpZiB0aGF0IGNhbWUgdGhyb3VnaCB0d2ljZTxvOnA+PC9v OnA+PC9zcGFuPjwvcD48cCA9CmNsYXNzPTNEIk1zb05vcm1hbCIgPQpzdHlsZT0zRCJtYXJnaW4t bGVmdDoxMC4wcHQ7dGV4dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0LWF1dG9zcGFjZTpub25lIj4KPHNw YW4gPQpzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZx dW90OywmcXVvdDtzYW5zLXNlcmlmJj0KcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRl O21zby1oaWdobGlnaHQ6d2hpdGUiPlsyMDo0NF0KPC9zcGFuPjxzcGFuID0Kc3R5bGU9M0QiZm9u dC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q7c2Fucy1z ZXJpZiY9CnF1b3Q7O2NvbG9yOm5hdnk7YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0Ondo aXRlIj4mbHQ7c29uaWNyb3NlJmd0OyA9CmltIHRoaW5raW5nIGFib3V0IG1ha2luZyB0aGUgU1NE IG9uZSBnaWFudCBzd2FwIGZpbGUgdGhlbiBjcmVhdGluZyBhID0KdmVyeSBsYXJnZSByYW1kaXNr IGluIHZpcnR1YWwgbWVtb3J5IGFuZCB1c2luZyB0aGF0IGFzIGEKIGJsb2NrIGxldmVsIGNhY2hl IGZvciBwYXJ0cyBhbmQgcGllY2VzIG9mIHZpcnR1YWwgbWFjaGluZSBkaXNrID0KaW1hZ2VzPG86 cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPTNEIk1zb05vcm1hbCIgPQpzdHlsZT0zRCJtYXJn aW4tbGVmdDoxMC4wcHQ7dGV4dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0LWF1dG9zcGFjZTpub25lIj4K PHNwYW4gPQpzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFu YSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJj0KcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndo aXRlO21zby1oaWdobGlnaHQ6d2hpdGUiPlsyMDo0NF0KPC9zcGFuPjxzcGFuID0Kc3R5bGU9M0Qi Zm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q7c2Fu cy1zZXJpZiY9CnF1b3Q7O2NvbG9yOm5hdnk7YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0 OndoaXRlIj4mbHQ7c29uaWNyb3NlJmd0OyA9CnRoZW4gaSB0aGluayB0aGUgbWVtb3J5IG1hbmFn ZXJzIHdvdWxkIGluaGVyZW50bHkgcGxheSB0aGUgcm9sZSBvZiA9CnN0b3JhZ2UgdGllcmluZyBp ZToga2VlcGluZyB0aGUgaG90dGVzdCBkYXRhIGluIG1lbW9yeSBhbmQKIHRoZSBjb2xkZXN0IGRh dGEgb24gc3dhcDxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCBjbGFzcz0zRCJNc29Ob3JtYWwiID0K c3R5bGU9M0QibWFyZ2luLWxlZnQ6MTAuMHB0O3RleHQtaW5kZW50Oi0xMC4wcHQ7dGV4dC1hdXRv c3BhY2U6bm9uZSI+CjxzcGFuID0Kc3R5bGU9M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5 OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiY9CnF1b3Q7O2NvbG9yOmdyYXk7 YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0OndoaXRlIj5bMjA6NDVdCjwvc3Bhbj48c3Bh biA9CnN0eWxlPTNEImZvbnQtc2l6ZTo5LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1 b3Q7LCZxdW90O3NhbnMtc2VyaWYmPQpxdW90Oztjb2xvcjpuYXZ5O2JhY2tncm91bmQ6d2hpdGU7 bXNvLWhpZ2hsaWdodDp3aGl0ZSI+Jmx0O3Nvbmljcm9zZSZndDsgPQpldmVyeXRoaW5nIGkgaGF2 ZSBzZWVuIHRvZGF5IGhhcyBiZWVuIHNldHVwIGFzJm5ic3A7Jm5ic3A7ID0KImNvbnN1bWVyIiZu YnNwOyA9M0Q9M0Q9M0QmZ3Q7Jm5ic3A7IG5ldHdvcmsgPTNEPTNEPTNEPTNEJmd0OyBTU0QgY2Fj aGUgPQo9M0Q9M0Q9M0Q9M0QmZ3Q7IHJlYWwgZGlza3M8bzpwPjwvbzpwPjwvc3Bhbj48L3A+PHAg Y2xhc3M9M0QiTXNvTm9ybWFsIiA9CnN0eWxlPTNEIm1hcmdpbi1sZWZ0OjEwLjBwdDt0ZXh0LWlu ZGVudDotMTAuMHB0O3RleHQtYXV0b3NwYWNlOm5vbmUiPgo8c3BhbiA9CnN0eWxlPTNEImZvbnQt c2l6ZTo5LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90O3NhbnMtc2Vy aWYmPQpxdW90Oztjb2xvcjpncmF5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0 ZSI+WzIwOjQ1XQo8L3NwYW4+PHNwYW4gPQpzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJj0KcXVvdDs7Y29sb3I6 bmF2eTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdobGlnaHQ6d2hpdGUiPiZsdDtzb25pY3Jvc2Um Z3Q7ID0KYnV0IGknZCBsaWtlIHRvIGFjdHVhbGx5IGRvICJjb25zdW1lciIgPTNEPTNEPTNEJmd0 OyBSQU0rU1NEIGNhY2hlID0KPTNEPTNEPTNEJmd0OyZuYnNwOyBuZXR3b3JrID0zRD0zRD0zRCZn dDsgcmVhbCA9CmRpc2tzPG86cD48L286cD48L3NwYW4+PC9wPjxwIGNsYXNzPTNEIk1zb05vcm1h bCIgPQpzdHlsZT0zRCJtYXJnaW4tbGVmdDoxMC4wcHQ7dGV4dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0 LWF1dG9zcGFjZTpub25lIj4KPHNwYW4gPQpzdHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1m YW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVvdDtzYW5zLXNlcmlmJj0KcXVvdDs7Y29sb3I6 Z3JheTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdobGlnaHQ6d2hpdGUiPlsyMDo0Nl0KPC9zcGFu PjxzcGFuID0Kc3R5bGU9M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRh bmEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiY9CnF1b3Q7O2NvbG9yOm5hdnk7YmFja2dyb3VuZDp3 aGl0ZTttc28taGlnaGxpZ2h0OndoaXRlIj4mbHQ7c29uaWNyb3NlJmd0OyA9CmkgcmVhbGl6ZSBk b2luZyBhIHZpcnR1YWwgbWVtb3J5IGRpc2sgbWVhbnMgdGhlIGNhY2hlIHdpbGwgYmUgY2xlYXJl ZCBvbiA9CmV2ZXJ5IHJlYm9vdCwgYW5kIEknbSBvayB3aXRoIHRoYXQ8bzpwPjwvbzpwPjwvc3Bh bj48L3A+PHAgPQpjbGFzcz0zRCJNc29Ob3JtYWwiID0Kc3R5bGU9M0QibWFyZ2luLWxlZnQ6MTAu MHB0O3RleHQtaW5kZW50Oi0xMC4wcHQ7dGV4dC1hdXRvc3BhY2U6bm9uZSI+CjxzcGFuID0Kc3R5 bGU9M0QiZm9udC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1 b3Q7c2Fucy1zZXJpZiY9CnF1b3Q7O2NvbG9yOmdyYXk7YmFja2dyb3VuZDp3aGl0ZTttc28taGln aGxpZ2h0OndoaXRlIj5bMjA6NDddCjwvc3Bhbj48c3BhbiA9CnN0eWxlPTNEImZvbnQtc2l6ZTo5 LjBwdDtmb250LWZhbWlseTomcXVvdDtWZXJkYW5hJnF1b3Q7LCZxdW90O3NhbnMtc2VyaWYmPQpx dW90Oztjb2xvcjpuYXZ5O2JhY2tncm91bmQ6d2hpdGU7bXNvLWhpZ2hsaWdodDp3aGl0ZSI+Jmx0 O3Nvbmljcm9zZSZndDsgPQppIGtub3cgdGhpcyBjYW4gYmUgZG9uZSB3aXRoIE5GUyBhbmQgY2Fj aGVmaWxlc2QoZnNjYWNoZSksIGJ1dCBob3cgY291bGQgPQpzb21ldGhpbmcgYmUgaW50ZWdyYXRl ZCBpbnRvIHRoZSBuYXRpdmUgZ2x1c3RlciA9CmNsaWVudHM/PG86cD48L286cD48L3NwYW4+PC9w PjxwIGNsYXNzPTNEIk1zb05vcm1hbCIgPQpzdHlsZT0zRCJtYXJnaW4tbGVmdDoxMC4wcHQ7dGV4 dC1pbmRlbnQ6LTEwLjBwdDt0ZXh0LWF1dG9zcGFjZTpub25lIj4KPHNwYW4gPQpzdHlsZT0zRCJm b250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90OywmcXVvdDtzYW5z LXNlcmlmJj0KcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRlO21zby1oaWdobGlnaHQ6 d2hpdGUiPlsyMDo0N10KPC9zcGFuPjxzcGFuID0Kc3R5bGU9M0QiZm9udC1zaXplOjkuMHB0O2Zv bnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiY9CnF1b3Q7O2Nv bG9yOm5hdnk7YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0OndoaXRlIj4mbHQ7c29uaWNy b3NlJmd0OyA9CmknZCBwcmVmZXIgbm90IHRvIGhhdmUgdG8gYWNjZXNzIGdsdXN0ZXIgdmlhIE5G UzxvOnA+PC9vOnA+PC9zcGFuPjwvcD48cCA9CmNsYXNzPTNEIk1zb05vcm1hbCI+PHNwYW4gPQpz dHlsZT0zRCJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VmVyZGFuYSZxdW90Oywm cXVvdDtzYW5zLXNlcmlmJj0KcXVvdDs7Y29sb3I6Z3JheTtiYWNrZ3JvdW5kOndoaXRlO21zby1o aWdobGlnaHQ6d2hpdGUiPlsyMDo0OV0KPC9zcGFuPjxzcGFuID0Kc3R5bGU9M0QiZm9udC1zaXpl OjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1ZlcmRhbmEmcXVvdDssJnF1b3Q7c2Fucy1zZXJpZiY9 CnF1b3Q7O2NvbG9yOm5hdnk7YmFja2dyb3VuZDp3aGl0ZTttc28taGlnaGxpZ2h0OndoaXRlIj4m bHQ7c29uaWNyb3NlJmd0OyA9CmFueSBmZWVkYmFjayBmcm9tIHRoaXMgcm9vbSBpcyBncmVhdGx5 IGFwcHJlY2lhdGVkLCBnZXR0aW5nIHNvbWVvbmUgPQpzdGFydGVkIHRvIGJ1aWxkIG1hbmFnZWQg SEEgY2xvdWQgaG9zdGluZzwvc3Bhbj48bzpwPjwvbzpwPjwvcD4KPC9kaXY+CjwvZGl2PgoKX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188YnI+VXNlcnMgbWFp bGluZyA9Cmxpc3Q8YnI+PGEgPQpocmVmPTNEIm1haWx0bzpVc2Vyc0BvdmlydC5vcmciPlVzZXJz QG92aXJ0Lm9yZzwvYT48YnI+aHR0cDovL2xpc3RzLm92aXJ0Lj0Kb3JnL21haWxtYW4vbGlzdGlu Zm8vdXNlcnM8YnI+PC9ibG9ja3F1b3RlPjwvZGl2Pjxicj48L2Rpdj48L2Rpdj48L2JvZHk+PC89 Cmh0bWw+PQoKLS1BcHBsZS1NYWlsPV8wQTlCNEQ2NC02QzM2LTRFRTMtQUU4OS0yRDdCQUJCQkMz MTAtLQo= --===============3775077665713835670==--