From fernando.frediani at upx.com Thu Mar 2 12:47:52 2017 Content-Type: multipart/mixed; boundary="===============5791117965635509698==" MIME-Version: 1.0 From: FERNANDO FREDIANI To: users at ovirt.org Subject: Re: [ovirt-users] Replicated Glusterfs on top of ZFS Date: Thu, 02 Mar 2017 09:47:44 -0300 Message-ID: <59969230-290d-aca4-be6a-36567407088e@upx.com> In-Reply-To: CAAqDm6bgqQLjh8rb=KG-DqGtcz4ghbGb2XS614nRbVJo2WmW7Q@mail.gmail.com --===============5791117965635509698== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------D5468FBE72B4A6F3BF994ECE Content-Type: text/plain; charset=3Dwindows-1252; format=3Dflowed Content-Transfer-Encoding: 7bit Am I understanding correctly, but you have Gluster on the top of ZFS = which is on the top of LVM ? If so, why the usage of LVM was necessary ? = I have ZFS with any need of LVM. Fernando On 02/03/2017 06:19, Arman Khalatyan wrote: > Hi, > I use 3 nodes with zfs and glusterfs. > Are there any suggestions to optimize it? > > host zfs config 4TB-HDD+250GB-SSD: > [root(a)clei22 ~]# zpool status > pool: zclei22 > state: ONLINE > scan: scrub repaired 0 in 0h0m with 0 errors on Tue Feb 28 14:16:07 2017 > config: > > NAME STATE READ WRITE CKSUM > zclei22 ONLINE 0 0 0 > HGST_HUS724040ALA640_PN2334PBJ4SV6T1 ONLINE 0 0 0 > logs > lv_slog ONLINE 0 0 0 > cache > lv_cache ONLINE 0 0 0 > > errors: No known data errors > > Name: > GluReplica > Volume ID: > ee686dfe-203a-4caa-a691-26353460cc48 > Volume Type: > Replicate (Arbiter) > Replica Count: > 2 + 1 > Number of Bricks: > 3 > Transport Types: > TCP, RDMA > Maximum no of snapshots: > 256 > Capacity: > 3.51 TiB total, 190.56 GiB used, 3.33 TiB free > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users --------------D5468FBE72B4A6F3BF994ECE Content-Type: text/html; charset=3Dwindows-1252 Content-Transfer-Encoding: 8bit

Am I understanding correctly, but you have Gluster on the top of ZFS which is on the top of LVM ? If so, why the usage of LVM was necessary ? I have ZFS with any need of LVM.

Fernando

On 02/03/2017 06:19, Arman Khalatyan wrote:

Hi,

I use 3 nodes with zfs and glusterfs.

Are there any suggestions to optimize it?

host zfs config 4TB-HDD+250GB-SSD:
[root(a)clei22 ~]# zpool status
=C2=A0 pool: zclei22
=C2=A0state: ONLINE
=C2=A0 scan: scrub repaired 0 in 0h0m with 0 errors on Tue Feb 28 14:16:07 2017
config:

=C2=A0=C2=A0=C2=A0 NAME=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 STATE=C2=A0=C2=A0=C2=A0=C2=A0 READ WRITE CKSUM
=C2=A0=C2=A0=C2=A0 zclei22=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0
=C2=A0=C2=A0=C2=A0 =C2=A0 HGST_HUS724040ALA640_PN2334PBJ4SV6T1=C2= =A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0
=C2=A0=C2=A0=C2=A0 logs
=C2=A0=C2=A0=C2=A0 =C2=A0 lv_slog=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0
=C2=A0=C2=A0=C2=A0 cache
=C2=A0=C2=A0=C2=A0 =C2=A0 lv_cache=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ONLIN= E=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0

errors: No known data errors

Name:
GluReplica
Volume ID:
ee686dfe-203a-4caa-a691-26353460cc48
Volume Type:
Replicate (Arbiter)
Replica Count:
2 + 1
Number of Bricks:
3
Transport Types:
TCP, RDMA
Maximum no of snapshots:
256
Capacity:
3.51 TiB total, 190.56 GiB used, 3.33 TiB free
_______________________________________________
Users mailing list
Use=
rs(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

--------------D5468FBE72B4A6F3BF994ECE-- --===============5791117965635509698== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS1ENTQ2OEZCRTcyQjRBNkYzQkY5OTRFQ0UKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXdpbmRvd3MtMTI1MjsgZm9ybWF0PWZsb3dlZApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5n OiA3Yml0CgpBbSBJIHVuZGVyc3RhbmRpbmcgY29ycmVjdGx5LCBidXQgeW91IGhhdmUgR2x1c3Rl ciBvbiB0aGUgdG9wIG9mIFpGUyAKd2hpY2ggaXMgb24gdGhlIHRvcCBvZiBMVk0gPyBJZiBzbywg d2h5IHRoZSB1c2FnZSBvZiBMVk0gd2FzIG5lY2Vzc2FyeSA/IApJIGhhdmUgWkZTIHdpdGggYW55 IG5lZWQgb2YgTFZNLgoKRmVybmFuZG8KCgpPbiAwMi8wMy8yMDE3IDA2OjE5LCBBcm1hbiBLaGFs YXR5YW4gd3JvdGU6Cj4gSGksCj4gSSB1c2UgMyBub2RlcyB3aXRoIHpmcyBhbmQgZ2x1c3RlcmZz Lgo+IEFyZSB0aGVyZSBhbnkgc3VnZ2VzdGlvbnMgdG8gb3B0aW1pemUgaXQ/Cj4KPiBob3N0IHpm cyBjb25maWcgNFRCLUhERCsyNTBHQi1TU0Q6Cj4gW3Jvb3RAY2xlaTIyIH5dIyB6cG9vbCBzdGF0 dXMKPiAgIHBvb2w6IHpjbGVpMjIKPiAgc3RhdGU6IE9OTElORQo+ICAgc2Nhbjogc2NydWIgcmVw YWlyZWQgMCBpbiAwaDBtIHdpdGggMCBlcnJvcnMgb24gVHVlIEZlYiAyOCAxNDoxNjowNyAyMDE3 Cj4gY29uZmlnOgo+Cj4gICAgIE5BTUUgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBTVEFURSAgICAgUkVBRCBXUklURSBDS1NVTQo+ICAgICB6Y2xlaTIyICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgT05MSU5FICAgICAgIDAgMCAgICAgMAo+ICAgICAgIEhHU1RfSFVT NzI0MDQwQUxBNjQwX1BOMjMzNFBCSjRTVjZUMSAgT05MSU5FICAgICAgIDAgMCAgICAgMAo+ICAg ICBsb2dzCj4gICAgICAgbHZfc2xvZyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBPTkxJ TkUgICAgICAgMCAwICAgICAwCj4gICAgIGNhY2hlCj4gICAgICAgbHZfY2FjaGUgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICBPTkxJTkUgICAgICAgMCAwICAgICAwCj4KPiBlcnJvcnM6IE5v IGtub3duIGRhdGEgZXJyb3JzCj4KPiBOYW1lOgo+IEdsdVJlcGxpY2EKPiBWb2x1bWUgSUQ6Cj4g ZWU2ODZkZmUtMjAzYS00Y2FhLWE2OTEtMjYzNTM0NjBjYzQ4Cj4gVm9sdW1lIFR5cGU6Cj4gUmVw bGljYXRlIChBcmJpdGVyKQo+IFJlcGxpY2EgQ291bnQ6Cj4gMiArIDEKPiBOdW1iZXIgb2YgQnJp Y2tzOgo+IDMKPiBUcmFuc3BvcnQgVHlwZXM6Cj4gVENQLCBSRE1BCj4gTWF4aW11bSBubyBvZiBz bmFwc2hvdHM6Cj4gMjU2Cj4gQ2FwYWNpdHk6Cj4gMy41MSBUaUIgdG90YWwsIDE5MC41NiBHaUIg dXNlZCwgMy4zMyBUaUIgZnJlZQo+Cj4KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fXwo+IFVzZXJzIG1haWxpbmcgbGlzdAo+IFVzZXJzQG92aXJ0Lm9yZwo+ IGh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycwoKCi0tLS0tLS0t LS0tLS0tRDU0NjhGQkU3MkI0QTZGM0JGOTk0RUNFCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBj aGFyc2V0PXdpbmRvd3MtMTI1MgpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0Cgo8aHRt bD4KICA8aGVhZD4KICAgIDxtZXRhIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD13aW5kb3dz LTEyNTIiCiAgICAgIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAgPC9oZWFkPgogIDxib2R5 IGJnY29sb3I9IiNGRkZGRkYiIHRleHQ9IiMwMDAwMDAiPgogICAgPHA+QW0gSSB1bmRlcnN0YW5k aW5nIGNvcnJlY3RseSwgYnV0IHlvdSBoYXZlIEdsdXN0ZXIgb24gdGhlIHRvcCBvZgogICAgICBa RlMgd2hpY2ggaXMgb24gdGhlIHRvcCBvZiBMVk0gPyBJZiBzbywgd2h5IHRoZSB1c2FnZSBvZiBM Vk0gd2FzCiAgICAgIG5lY2Vzc2FyeSA/IEkgaGF2ZSBaRlMgd2l0aCBhbnkgbmVlZCBvZiBMVk0u PC9wPgogICAgPHA+RmVybmFuZG88YnI+CiAgICA8L3A+CiAgICA8YnI+CiAgICA8ZGl2IGNsYXNz PSJtb3otY2l0ZS1wcmVmaXgiPk9uIDAyLzAzLzIwMTcgMDY6MTksIEFybWFuIEtoYWxhdHlhbgog ICAgICB3cm90ZTo8YnI+CiAgICA8L2Rpdj4KICAgIDxibG9ja3F1b3RlCmNpdGU9Im1pZDpDQUFx RG02YmdxUUxqaDhyYj1LRy1EcUd0Y3o0Z2hiR2IyWFM2MTRuUmJWSm8yV21XN1FAbWFpbC5nbWFp bC5jb20iCiAgICAgIHR5cGU9ImNpdGUiPgogICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICA8 ZGl2PgogICAgICAgICAgPGRpdj5IaSwgPGJyPgogICAgICAgICAgPC9kaXY+CiAgICAgICAgICBJ IHVzZSAzIG5vZGVzIHdpdGggemZzIGFuZCBnbHVzdGVyZnMuPGJyPgogICAgICAgIDwvZGl2Pgog ICAgICAgIEFyZSB0aGVyZSBhbnkgc3VnZ2VzdGlvbnMgdG8gb3B0aW1pemUgaXQ/PGJyPgogICAg ICAgIDxkaXY+PGJyPgogICAgICAgICAgaG9zdCB6ZnMgY29uZmlnIDRUQi1IREQrMjUwR0ItU1NE Ojxicj4KICAgICAgICAgIFtyb290QGNsZWkyMiB+XSMgenBvb2wgc3RhdHVzIDxicj4KICAgICAg ICAgIKAgcG9vbDogemNsZWkyMjxicj4KICAgICAgICAgIKBzdGF0ZTogT05MSU5FPGJyPgogICAg ICAgICAgoCBzY2FuOiBzY3J1YiByZXBhaXJlZCAwIGluIDBoMG0gd2l0aCAwIGVycm9ycyBvbiBU dWUgRmViIDI4CiAgICAgICAgICAxNDoxNjowNyAyMDE3PGJyPgogICAgICAgICAgY29uZmlnOjxi cj4KICAgICAgICAgIDxicj4KICAgICAgICAgIKCgoCBOQU1FoKCgoKCgoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKCgoKCgoKAgU1RBVEWgoKCgIFJFQUQKICAgICAgICAgIFdSSVRFIENLU1VNPGJyPgog ICAgICAgICAgoKCgIHpjbGVpMjKgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCBPTkxJ TkWgoKCgoKAgMKCgoKAKICAgICAgICAgIDCgoKCgIDA8YnI+CiAgICAgICAgICCgoKAgoCBIR1NU X0hVUzcyNDA0MEFMQTY0MF9QTjIzMzRQQko0U1Y2VDGgIE9OTElORaCgoKCgoCAwoKCgoAogICAg ICAgICAgMKCgoKAgMDxicj4KICAgICAgICAgIKCgoCBsb2dzPGJyPgogICAgICAgICAgoKCgIKAg bHZfc2xvZ6CgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCBPTkxJTkWgoKCgoKAgMKCgoKAK ICAgICAgICAgIDCgoKCgIDA8YnI+CiAgICAgICAgICCgoKAgY2FjaGU8YnI+CiAgICAgICAgICCg oKAgoCBsdl9jYWNoZaCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgIE9OTElORaCgoKCgoCAw oKCgoAogICAgICAgICAgMKCgoKAgMDxicj4KICAgICAgICAgIDxicj4KICAgICAgICAgIGVycm9y czogTm8ga25vd24gZGF0YSBlcnJvcnM8YnI+CiAgICAgICAgICA8YnI+CiAgICAgICAgICBOYW1l Ojxicj4KICAgICAgICAgIEdsdVJlcGxpY2E8YnI+CiAgICAgICAgICBWb2x1bWUgSUQ6PGJyPgog ICAgICAgICAgZWU2ODZkZmUtMjAzYS00Y2FhLWE2OTEtMjYzNTM0NjBjYzQ4PGJyPgogICAgICAg ICAgVm9sdW1lIFR5cGU6PGJyPgogICAgICAgICAgUmVwbGljYXRlIChBcmJpdGVyKTxicj4KICAg ICAgICAgIFJlcGxpY2EgQ291bnQ6PGJyPgogICAgICAgICAgMiArIDE8YnI+CiAgICAgICAgICBO dW1iZXIgb2YgQnJpY2tzOjxicj4KICAgICAgICAgIDM8YnI+CiAgICAgICAgICBUcmFuc3BvcnQg VHlwZXM6PGJyPgogICAgICAgICAgVENQLCBSRE1BPGJyPgogICAgICAgICAgTWF4aW11bSBubyBv ZiBzbmFwc2hvdHM6PGJyPgogICAgICAgICAgMjU2PGJyPgogICAgICAgICAgQ2FwYWNpdHk6PGJy PgogICAgICAgICAgMy41MSBUaUIgdG90YWwsIDE5MC41NiBHaUIgdXNlZCwgMy4zMyBUaUIgZnJl ZTxicj4KICAgICAgICA8L2Rpdj4KICAgICAgPC9kaXY+CiAgICAgIDxicj4KICAgICAgPGZpZWxk c2V0IGNsYXNzPSJtaW1lQXR0YWNobWVudEhlYWRlciI+PC9maWVsZHNldD4KICAgICAgPGJyPgog ICAgICA8cHJlIHdyYXA9IiI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KVXNlcnMgbWFpbGluZyBsaXN0CjxhIGNsYXNzPSJtb3otdHh0LWxpbmstYWJicmV2 aWF0ZWQiIGhyZWY9Im1haWx0bzpVc2Vyc0BvdmlydC5vcmciPlVzZXJzQG92aXJ0Lm9yZzwvYT4K PGEgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIgaHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0 Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxt YW4vbGlzdGluZm8vdXNlcnM8L2E+CjwvcHJlPgogICAgPC9ibG9ja3F1b3RlPgogICAgPGJyPgog IDwvYm9keT4KPC9odG1sPgoKLS0tLS0tLS0tLS0tLS1ENTQ2OEZCRTcyQjRBNkYzQkY5OTRFQ0Ut LQo= --===============5791117965635509698==--