From suporte at logicworks.pt Sat May 25 10:02:52 2013 Content-Type: multipart/mixed; boundary="===============6904935269043906649==" MIME-Version: 1.0 From: suporte at logicworks.pt To: users at ovirt.org Subject: [Users] deduplication Date: Sat, 25 May 2013 15:02:40 +0100 Message-ID: <1691474377.35303.1369490560789.JavaMail.root@logicworks.pt> In-Reply-To: 716378002.35299.1369490552394.JavaMail.root@logicworks.pt --===============6904935269043906649== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable ------=3D_Part_35302_1064706683.1369490560788 Content-Type: text/plain; charset=3Dutf-8 Content-Transfer-Encoding: 7bit is deduplication possible? = Regards = Jose = -- = Jose Ferradeira = http://www.logicworks.pt = ------=3D_Part_35302_1064706683.1369490560788 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 7bit
Jose=3D20
From: "Jiri Belka"=3D20 To: suporte(a)logicworks.pt=3D20 Cc: users(a)ovirt.org=3D20 Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=3D20 Subject: Re: [Users] deduplication=3D20 On Tue, 28 May 2013 14:29:05 +0100 (WEST)=3D20 suporte(a)logicworks.pt wrote:=3D20 > That's why I'm making this questions, to demystify some buzzwords around = =3D here.=3D20 > But if you have a strong and good technology why not create buzzwords to = =3D get into as many people as possible? without trapped them.=3D20 > Share a disk containing "static" data is a good idea, do you know from wh= =3D ere I can start?=3D20 Everything depends on your needs, design planning. Maybe then sharing=3D20 disk would be better to share via NFS/iscsi. Of course if you have many=3D20 VMs each of them is different you will fail. But if you have mostly=3D20 homogeneous environment you can think about this approach. Sure you have=3D= 20 to have plan for upgrading "base" "static" shared OS data, you have to=3D20 have plan how to install additional software (different destination=3D20 than /usr or /usr/local)... If you already have your own build host=3D20 which builds for you OS packages and you have already your own plan for=3D20 deployment, you have done first steps. If you depend on upgrading each=3D20 machine separately from Internet, then first you should plan your=3D20 environment, configuration management etc.=3D20 Well, in many times people do not do any planning, they just think some=3D20 good technology would save their "poor" design.=3D20 j.=3D20
=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20 ------=3D_Part_850_29656067.1369990209067 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: quoted-printable <=3D div style=3D3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; co= lo=3D r: #000000'>So, we can say that dedup has more disadvantages than advantage= =3D s.
Absolutely agree with you, planning i= s =3D the best thing to do, but normally people want a plug'n'play system with al= =3D l included, because there is not much time to think and planning, and there= =3D are many companies that know how to take advantage of this people characteristics.
Any way, I think another solution for dedup is Fr= ee=3D NAS using ZFS.
Jose
From: "Jiri Belka" <jbelka(a)redhat.com>
To: supor= te(a)l=3D ogicworks.pt
Cc: users= (a)ovi=3D rt.org
Sent: Qua= rta-=3D feira, 29 de Maio de 2013 7:33:10
Subject: = Re: =3D [Users] deduplication
On Tue, 28 May 2013 14:29:05 +0100 (WEST)<= br=3D > suporte(a)logicworks.pt wrote:
> That's why I'm making this questions, to dem= ys=3D tify some buzzwords around here.
> But if you have a strong and good technology= w=3D hy not create buzzwords to get into as many people as possible? without tra= =3D pped them.
> Share a disk containing "static" data is a g= oo=3D d idea, do you know from where I can start?
Everything depends on your needs, design planning= . =3D Maybe then sharing
disk would be better to share via NFS/iscsi. Of c= ou=3D rse if you have many
VMs each of them is different you will fail. But = if=3D you have mostly
homogeneous environment you can think about this = ap=3D proach. Sure you have
to have plan for upgrading "base" "static" shared= O=3D S data, you have to
have plan how to install additional software (dif= fe=3D rent destination
than /usr or /usr/local)... If you already have y= ou=3D r own build host
which builds for you OS packages and you have alr= ea=3D dy your own plan for
deployment, you have done first steps. If you dep= en=3D d on upgrading each
machine separately from Internet, then first you = sh=3D ould plan your
environment, configuration management etc.=
Well, in many times people do not do any planning= , =3D they just think some
good technology would save their "poor" design.= fo=3D nt>
j.
-- Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar ---------------------------------------------------------------------------= =3D ---- Karli Sj=3DC3=3DB6berg Swedish University of Agricultural Sciences Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8) S-750 07 Uppsala, Sweden Phone: +46-(0)18-67 15 66 karli.s= joberg=3D @slu.se |
And what about dedup of Netapp?=3D20Much better implementation, in my opinion. You are able schedule dedup-runs= =3D to go at night so your user=3DC2=3DB4s performance isn=3DC2=3DB4t impacted= , and yo=3D u get the savings. The question is if you value the savings enough to take = =3D on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with= =3D compression enabled and buy in standard HDD's from anywhere... We did;)=3D= 20 /Karli=3D20
Jose=3D20
From: "Karli Sj=3DC3=3DB6berg"=3D20 To: suporte(a)logicworks.pt=3D20 Cc: "Jiri Belka" , users(a)ovirt.org=3D20 Sent: Quinta-feira, 30 de Maio de 2013 8:33:19=3D20 Subject: Re: [Users] deduplication=3D20 ons 2013-05-29 klockan 09:59 +0100 skrev suporte(a)logicworks.pt:=3D20 Absolutely agree with you, planning is the best thing to do, but normally p= =3D eople want a plug'n'play system with all included, because there is not muc= =3D h time to think and planning, and there are many companies that know how to= =3D take advantage of this people characteristics.=3D20 Any way, I think another solution for dedup is FreeNAS using ZFS.=3D20FreeNAS is just FreeBSD with a fancy web-ui ontop, so it=3DC2=3DB4s neither= mor=3D e or less of ZFS than you would have otherwise, And regarding dedup in ZFS;= =3D Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! It=3DC2=3DB4s said that i= t may increa=3D se performance when you have a very suitable usecase, e.g. everything exact= =3D ly the same over and over. What=3DC2=3DB4s not said is that scrubbing and r= esil=3D vering slows down to a snail (from hundreds of MB/s, or GB if your system i= =3D s large enough, down to less than 10), just from dedup. Also deleting snaps= =3D hots of datasets that have(or have had) dedup on can kill the entire system= =3D , and when I say kill, I mean really fubar. Been there, regretted that... N= =3D ow, compression on the other hand, you get basically for free and gives dec= =3D ent savings, I highly recommend that.=3D20 /Karli=3D20Jose=3D20 From: "Jiri Belka"=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20=3D20 To: suporte(a)logicworks.pt=3D20 Cc: users(a)ovirt.org=3D20 Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=3D20 Subject: Re: [Users] deduplication=3D20 On Tue, 28 May 2013 14:29:05 +0100 (WEST)=3D20 suporte(a)logicworks.pt wrote:=3D20 > That's why I'm making this questions, to demystify some buzzwords around = =3D here.=3D20 > But if you have a strong and good technology why not create buzzwords to = =3D get into as many people as possible? without trapped them.=3D20 > Share a disk containing "static" data is a good idea, do you know from wh= =3D ere I can start?=3D20 Everything depends on your needs, design planning. Maybe then sharing=3D20 disk would be better to share via NFS/iscsi. Of course if you have many=3D20 VMs each of them is different you will fail. But if you have mostly=3D20 homogeneous environment you can think about this approach. Sure you have=3D= 20 to have plan for upgrading "base" "static" shared OS data, you have to=3D20 have plan how to install additional software (different destination=3D20 than /usr or /usr/local)... If you already have your own build host=3D20 which builds for you OS packages and you have already your own plan for=3D20 deployment, you have done first steps. If you depend on upgrading each=3D20 machine separately from Internet, then first you should plan your=3D20 environment, configuration management etc.=3D20 Well, in many times people do not do any planning, they just think some=3D20 good technology would save their "poor" design.=3D20 j.=3D20
=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20 ------=3D_Part_1134_11907452.1369996394775 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: quoted-printable <=3D div style=3D3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; co= lo=3D r: #000000'>Thanks a lot Karli, you make my mind clear about deduplication,= =3D once again we cannot have the best of both worlds.
So, we can say that dedup has more di= sa=3D dvantages than advantages.
And what about dedup of Netapp?
Jose
From: "Karli Sj=3DC3=3DB6berg" <Karli.Sjoberg(a)slu.se>
To: supor= te(a)l=3D ogicworks.pt
Cc: "Jiri= Bel=3D ka" <jbelka(a)redhat.com>, users(a)ovirt.org
Sent: Qui= nta-=3D feira, 30 de Maio de 2013 8:33:19
Subject: = Re: =3D [Users] deduplication
ons 2013-05-29 klockan 09:59 +0100 skrev suporte(= a)lo=3D gicworks.pt:
Absolutely agree with you, planning i= s =3D the best thing to do, but normally people want a plug'n'play system with al= =3D l included, because there is not much time to think and planning, and there= =3D are many companies that know how to take advantage of this people characteristics.
Any way, I think another solution for dedup is Fr= ee=3D NAS using ZFS.
FreeNAS is just FreeBSD with a fancy web-ui ontop= , =3D so it=3DC2=3DB4s neither more or less of ZFS than you would have otherwise,= And=3D regarding dedup in ZFS; Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! I= t=3DC2=3DB4s=3D said that it may in=3D crease performance when you have a very suitable usecase, e.g. everything exactly the same over and over. What=3DC2=3DB4s not said is that scrubbing and re= silv=3D ering slows down to a snail (from hundreds of MB/s, or GB if your system is= =3D large enough, down to less than 10), just from dedup. Also deleting snapshots of datasets that have(or have had) ded= =3D up on can kill the entire system, and when I say kill, I mean really fubar.= =3D Been there, regretted that... Now, compression on the other hand, you get = =3D basically for free and gives decent savings, I highly recommend that.
/Karli
Jose
From: "Ji= ri B=3D elka" <jbelka(a)redhat.com>
To: supor= te(a)l=3D ogicworks.pt
Cc: users= (a)ovi=3D rt.org
Sent: Qua= rta-=3D feira, 29 de Maio de 2013 7:33:10
Subject: = Re: =3D [Users] deduplication
On Tue, 28 May 2013 14:29:05 +0100 (WEST)<= br=3D > suporte(a)logicworks.pt wrote:
> That's why I'm making this questions, to dem= ys=3D tify some buzzwords around here.
> But if you have a strong and good technology= w=3D hy not create buzzwords to get into as many people as possible? without tra= =3D pped them.
> Share a disk containing "static" data is a g= oo=3D d idea, do you know from where I can start?
Everything depends on your needs, design planning= . =3D Maybe then sharing
disk would be better to share via NFS/iscsi. Of c= ou=3D rse if you have many
VMs each of them is different you will fail. But = if=3D you have mostly
homogeneous environment you can think about this = ap=3D proach. Sure you have
to have plan for upgrading "base" "static" shared= O=3D S data, you have to
have plan how to install additional software (dif= fe=3D rent destination
than /usr or /usr/local)... If you already have y= ou=3D r own build host
which builds for you OS packages and you have alr= ea=3D dy your own plan for
deployment, you have done first steps. If you dep= en=3D d on upgrading each
machine separately from Internet, then first you = sh=3D ould plan your
environment, configuration management etc.=
Well, in many times people do not do any planning= , =3D they just think some
good technology would save their "poor" design.= fo=3D nt>
j.
--
Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar
---------------------------------------------------------------------------= =3D ----
Karli Sj=3DC3=3DB6berg
Swedish University of Agricultural Sciences
Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)
S-750 07 Uppsala, Sweden
Phone: +46-(0)18-67 15 66
karli.s= joberg=3D @slu.se
-- Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar ---------------------------------------------------------------------------= =3D ---- Karli Sj=3DC3=3DB6berg Swedish University of Agricultural Sciences Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8) S-750 07 Uppsala, Sweden Phone: +46-(0)18-67 15 66 karli.s= joberg=3D @slu.se |
So, we can say that dedup has more disadvantages than advantages.=3D20 For a primary system; most definitely, yes.=3D20 But for a backup system, that has tons of RAM and SSD's for cache, and you = =3D have lots of virtual machines that are based off of the template, or are ve= =3D ry much the same, then you have a real use-case. I=3DC2=3DB4m active at the= Fre=3D eBSD forums where one person reports storing 150TB of data in only 30TB of = =3D physical disk. The best practice of scrubbing is once a week on "enterprise= =3D " systems, though he is only able to do it once a month, because that=3DC2= =3DB4=3D s how long it takes for a scrub to complete in that system. So you=3DC2=3DB= 4ve =3D got to choose performance or savings, you can=3DC2=3DB4t have both.=3D20------=3D_Part_1742_408268.1370275951382 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: quoted-printable <=3D div style=3D3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; co= lo=3D r: #000000'>Hi Juan,And what about dedup of Netapp?=3D20Much better implementation, in my opinion. You are able schedule dedup-runs= =3D to go at night so your user=3DC2=3DB4s performance isn=3DC2=3DB4t impacted= , and yo=3D u get the savings. The question is if you value the savings enough to take = =3D on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with= =3D compression enabled and buy in standard HDD's from anywhere... We did;)=3D= 20 /Karli=3D20Jose=3D20From: "Karli Sj=3DC3=3DB6berg" < Karli.Sjoberg(a)slu.se >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: "Jiri Belka" < jbelka(a)redhat.com >, users(a)ovirt.org=3D20 Sent: Quinta-feira, 30 de Maio de 2013 8:33:19=3D20 Subject: Re: [Users] deduplication=3D20 ons 2013-05-29 klockan 09:59 +0100 skrev suporte(a)logicworks.pt :=3D20Absolutely agree with you, planning is the best thing to do, but normally p= =3D eople want a plug'n'play system with all included, because there is not muc= =3D h time to think and planning, and there are many companies that know how to= =3D take advantage of this people characteristics.=3D20 Any way, I think another solution for dedup is FreeNAS using ZFS.=3D20FreeNAS is just FreeBSD with a fancy web-ui ontop, so it=3DC2=3DB4s neither= mor=3D e or less of ZFS than you would have otherwise, And regarding dedup in ZFS;= =3D Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! It=3DC2=3DB4s said that i= t may increa=3D se performance when you have a very suitable usecase, e.g. everything exact= =3D ly the same over and over. What=3DC2=3DB4s not said is that scrubbing and r= esil=3D vering slows down to a snail (from hundreds of MB/s, or GB if your system i= =3D s large enough, down to less than 10), just from dedup. Also deleting snaps= =3D hots of datasets that have(or have had) dedup on can kill the entire system= =3D , and when I say kill, I mean really fubar. Been there, regretted that... N= =3D ow, compression on the other hand, you get basically for free and gives dec= =3D ent savings, I highly recommend that.=3D20 /Karli=3D20Jose=3D20 From: "Jiri Belka" < jbelka(a)redhat.com >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: users(a)ovirt.org=3D20 Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=3D20 Subject: Re: [Users] deduplication=3D20 On Tue, 28 May 2013 14:29:05 +0100 (WEST)=3D20 suporte(a)logicworks.pt wrote:=3D20 > That's why I'm making this questions, to demystify some buzzwords around = =3D here.=3D20 > But if you have a strong and good technology why not create buzzwords to = =3D get into as many people as possible? without trapped them.=3D20 > Share a disk containing "static" data is a good idea, do you know from wh= =3D ere I can start?=3D20 Everything depends on your needs, design planning. Maybe then sharing=3D20 disk would be better to share via NFS/iscsi. Of course if you have many=3D20 VMs each of them is different you will fail. But if you have mostly=3D20 homogeneous environment you can think about this approach. Sure you have=3D= 20 to have plan for upgrading "base" "static" shared OS data, you have to=3D20 have plan how to install additional software (different destination=3D20 than /usr or /usr/local)... If you already have your own build host=3D20 which builds for you OS packages and you have already your own plan for=3D20 deployment, you have done first steps. If you depend on upgrading each=3D20 machine separately from Internet, then first you should plan your=3D20 environment, configuration management etc.=3D20 Well, in many times people do not do any planning, they just think some=3D20 good technology would save their "poor" design.=3D20 j.=3D20=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20 _______________________________________________=3D20 Users mailing list=3D20 Users(a)ovirt.org=3D20 http://lists.ovirt.org/mailman/listinfo/users=3D20
Thanks a l= ot=3D Karli, you make my mind clear about deduplication, once again we cannot ha= =3D ve the best of both worlds.
I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, runni= =3D ng on Linux, has no better performance but supports DRDB.
JoseFrom: "Karli Sj=3DC3=3DB6berg" <Karli.Sjoberg(a)slu.se>T=3D o: sup= orte(a)l=3D ogicworks.ptSent: Sexta-feira, 31 = de M=3D aio de 2013 10:45:41
Cc: "Jiri Belka" <jbelka(a)redhat.com>, users(a)ovirt.org
Subject: Re: [Users] deduplication
fre 2013-05-31 klockan 09:50 +0100 skrev suporte(a)logicworks.pt:So, we can say that dedup has more di= sa=3D dvantages than advantages.
For a primary system; most definitely, yes.
But for a backup system, that has tons of RAM and SSD's for cache, and you = =3D have lots of virtual machines that are based off of the template, or are ve= =3D ry much the same, then you have a real use-case. I=3DC2=3DB4m active at the= Fre=3D eBSD forums where one person reports storing 150TB of data in only 30TB of physical disk. The best practice of = =3D scrubbing is once a week on "enterprise" systems, though he is only able to= =3D do it once a month, because that=3DC2=3DB4s how long it takes for a scrub = to c=3D omplete in that system. So you=3DC2=3DB4ve got to choose performance or savings, you can=3DC2=3DB4t have both.
And what about dedup of Netapp?
Much better implementation, in my opinion. You are able schedule dedup-runs= =3D to go at night so your user=3DC2=3DB4s performance isn=3DC2=3DB4t impacted= , and yo=3D u get the savings. The question is if you value the savings enough to take = =3D on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with compression enabled and buy in standard H= =3D DD's from anywhere... We did;)
/Karli
Jose
From: "Karli Sj=3DC3=3DB6berg" <Karli.Sjoberg(a)slu.se>
To: suporte(a)logicwo= rks.pt=3D a>
Cc: "Jiri= Bel=3D ka" <jbelka= (a)redha=3D t.com>, u= sers(a)o=3D virt.org
Sent: Qui= nta-=3D feira, 30 de Maio de 2013 8:33:19
Subject: = Re: =3D [Users] deduplication
ons 2013-05-29 klockan 09:59 +0100 skrev suporte(a)logicworks.p= t:
Absolutely agree with you, planning i= s =3D the best thing to do, but normally people want a plug'n'play system with al= =3D l included, because there is not much time to think and planning, and there= =3D are many companies that know how to take advantage of this people characteristics.
Any way, I think another solution for dedup is Fr= ee=3D NAS using ZFS.
FreeNAS is just FreeBSD with a fancy web-ui ontop= , =3D so it=3DC2=3DB4s neither more or less of ZFS than you would have otherwise,= And=3D regarding dedup in ZFS; Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! I= t=3DC2=3DB4s=3D said that it may in=3D crease performance when you have a very suitable usecase, e.g. everything exactly the same over and over. What=3DC2=3DB4s not said is that scrubbing and re= silv=3D ering slows down to a snail (from hundreds of MB/s, or GB if your system is= =3D large enough, down to less than 10), just from dedup. Also deleting snapshots of datasets that have(or have had) ded= =3D up on can kill the entire system, and when I say kill, I mean really fubar.= =3D Been there, regretted that... Now, compression on the other hand, you get = =3D basically for free and gives decent savings, I highly recommend that.
/Karli
Jose
From: "Ji= ri B=3D elka" <jbel= ka(a)red=3D hat.com>
To: suporte(a)logicwo= rks.pt=3D a>
Cc: users(a)ovirt.org= font>
Sent: Qua= rta-=3D feira, 29 de Maio de 2013 7:33:10
Subject: = Re: =3D [Users] deduplication
On Tue, 28 May 2013 14:29:05 +0100 (WEST)<= br=3D > suporte(a)logicworks.pt wrote:
> That's why I'm making this questions, to dem= ys=3D tify some buzzwords around here.
> But if you have a strong and good technology= w=3D hy not create buzzwords to get into as many people as possible? without tra= =3D pped them.
> Share a disk containing "static" data is a g= oo=3D d idea, do you know from where I can start?
Everything depends on your needs, design planning= . =3D Maybe then sharing
disk would be better to share via NFS/iscsi. Of c= ou=3D rse if you have many
VMs each of them is different you will fail. But = if=3D you have mostly
homogeneous environment you can think about this = ap=3D proach. Sure you have
to have plan for upgrading "base" "static" shared= O=3D S data, you have to
have plan how to install additional software (dif= fe=3D rent destination
than /usr or /usr/local)... If you already have y= ou=3D r own build host
which builds for you OS packages and you have alr= ea=3D dy your own plan for
deployment, you have done first steps. If you dep= en=3D d on upgrading each
machine separately from Internet, then first you = sh=3D ould plan your
environment, configuration management etc.=
Well, in many times people do not do any planning= , =3D they just think some
good technology would save their "poor" design.= fo=3D nt>
j.
--
Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar
---------------------------------------------------------------------------= =3D ----
Karli Sj=3DC3=3DB6berg
Swedish University of Agricultural Sciences
Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)
S-750 07 Uppsala, Sweden
Phone: +46-(0)18-67 15 66
karli.s= joberg=3D @slu.se
--
Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar
---------------------------------------------------------------------------= =3D ----
Karli Sj=3DC3=3DB6berg
Swedish University of Agricultural Sciences
Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)
S-750 07 Uppsala, Sweden
Phone: +46-(0)18-67 15 66
karli.s= joberg=3D @slu.se
_____________________________________= =3D __________
Users mailing list
Users(a)ovirt.or= g
http://lists.ovirt.org/mailman/listinfo/users
Thanks a lot Karli, you make my mind clear about deduplication, once again = =3D we cannot have the best of both worlds.=3D20 I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, running o= =3D n Linux, has no better performance but supports DRDB.=3D20 Jose=3D20 From: "Karli Sj=3DC3=3DB6berg" < Karli.Sjoberg(a)slu.se >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: "Jiri Belka" < jbelka(a)redhat.com >, users(a)ovirt.org=3D20 Sent: Sexta-feira, 31 de Maio de 2013 10:45:41=3D20 Subject: Re: [Users] deduplication=3D20 fre 2013-05-31 klockan 09:50 +0100 skrev suporte(a)logicworks.pt :=3D20--=3D20 Chris Noffsinger=3D20 ------=3D_Part_1980_16023178.1370280384707 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: quoted-printable <=3D div style=3D3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; co= lo=3D r: #000000'>If we have a hardware RAID controller will we need RAID-Z ?So, we can say that dedup has more disadvantages than advantages.=3D20 For a primary system; most definitely, yes.=3D20 But for a backup system, that has tons of RAM and SSD's for cache, and you = =3D have lots of virtual machines that are based off of the template, or are ve= =3D ry much the same, then you have a real use-case. I=3DC2=3DB4m active at the= Fre=3D eBSD forums where one person reports storing 150TB of data in only 30TB of = =3D physical disk. The best practice of scrubbing is once a week on "enterprise= =3D " systems, though he is only able to do it once a month, because that=3DC2= =3DB4=3D s how long it takes for a scrub to complete in that system. So you=3DC2=3DB= 4ve =3D got to choose performance or savings, you can=3DC2=3DB4t have both.=3D20_______________________________________________=3D20 Users mailing list=3D20 Users(a)ovirt.org=3D20 http://lists.ovirt.org/mailman/listinfo/users=3D20And what about dedup of Netapp?=3D20Much better implementation, in my opinion. You are able schedule dedup-runs= =3D to go at night so your user=3DC2=3DB4s performance isn=3DC2=3DB4t impacted= , and yo=3D u get the savings. The question is if you value the savings enough to take = =3D on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with= =3D compression enabled and buy in standard HDD's from anywhere... We did;)=3D= 20 /Karli=3D20Jose=3D20From: "Karli Sj=3DC3=3DB6berg" < Karli.Sjoberg(a)slu.se >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: "Jiri Belka" < jbelka(a)redhat.com >, users(a)ovirt.org=3D20 Sent: Quinta-feira, 30 de Maio de 2013 8:33:19=3D20 Subject: Re: [Users] deduplication=3D20 ons 2013-05-29 klockan 09:59 +0100 skrev suporte(a)logicworks.pt :=3D20Absolutely agree with you, planning is the best thing to do, but normally p= =3D eople want a plug'n'play system with all included, because there is not muc= =3D h time to think and planning, and there are many companies that know how to= =3D take advantage of this people characteristics.=3D20 Any way, I think another solution for dedup is FreeNAS using ZFS.=3D20FreeNAS is just FreeBSD with a fancy web-ui ontop, so it=3DC2=3DB4s neither= mor=3D e or less of ZFS than you would have otherwise, And regarding dedup in ZFS;= =3D Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! It=3DC2=3DB4s said that i= t may increa=3D se performance when you have a very suitable usecase, e.g. everything exact= =3D ly the same over and over. What=3DC2=3DB4s not said is that scrubbing and r= esil=3D vering slows down to a snail (from hundreds of MB/s, or GB if your system i= =3D s large enough, down to less than 10), just from dedup. Also deleting snaps= =3D hots of datasets that have(or have had) dedup on can kill the entire system= =3D , and when I say kill, I mean really fubar. Been there, regretted that... N= =3D ow, compression on the other hand, you get basically for free and gives dec= =3D ent savings, I highly recommend that.=3D20 /Karli=3D20Jose=3D20 From: "Jiri Belka" < jbelka(a)redhat.com >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: users(a)ovirt.org=3D20 Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=3D20 Subject: Re: [Users] deduplication=3D20 On Tue, 28 May 2013 14:29:05 +0100 (WEST)=3D20 suporte(a)logicworks.pt wrote:=3D20 > That's why I'm making this questions, to demystify some buzzwords around = =3D here.=3D20 > But if you have a strong and good technology why not create buzzwords to = =3D get into as many people as possible? without trapped them.=3D20 > Share a disk containing "static" data is a good idea, do you know from wh= =3D ere I can start?=3D20 Everything depends on your needs, design planning. Maybe then sharing=3D20 disk would be better to share via NFS/iscsi. Of course if you have many=3D20 VMs each of them is different you will fail. But if you have mostly=3D20 homogeneous environment you can think about this approach. Sure you have=3D= 20 to have plan for upgrading "base" "static" shared OS data, you have to=3D20 have plan how to install additional software (different destination=3D20 than /usr or /usr/local)... If you already have your own build host=3D20 which builds for you OS packages and you have already your own plan for=3D20 deployment, you have done first steps. If you depend on upgrading each=3D20 machine separately from Internet, then first you should plan your=3D20 environment, configuration management etc.=3D20 Well, in many times people do not do any planning, they just think some=3D20 good technology would save their "poor" design.=3D20 j.=3D20=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20 _______________________________________________=3D20 Users mailing list=3D20 Users(a)ovirt.org=3D20 http://lists.ovirt.org/mailman/listinfo/users=3D20
Hi Juan,
thanks for your info, I'll try to test FreeNAS with comp= =3D ression. Do you use it with iSCSI or NFS?
JoseFrom: "Juan Jose" <jj197005(a)gmail.com>
To: suporte(a)logicworks.pt, users(a)ovirt.org
Sent: Segunda-feira, 3 de Junho de 2013 13:37:21
Subject: = =3D Re: [Users] deduplicationHello Jose,We also have FreeNAS working in our infraestructure, w= =3D ith about 3 TB and ZFS. Some of the pools has compression enabled and you c= =3D an save space with it. We have this FreeNAS connected to a hypervisor Xen a= =3D nd it works very well and it's stable and sure. We have nine virtual server= =3D s some wirtualized and other paravirtualized, and some Windows Server machi= =3D ne all about 2 years in production without any problem. My idea is connect = =3D this infrastructure with oVirt wo be able to have some resources for test V= =3D Ms in that. Only wanted to share as another FreeNas success experience.Juanjo.
<=3D div class=3D3D"gmail_quote">On Fri, May 31, 2013 at 12:33 PM, <su= porte@=3D logicworks.pt> wrote:
Thanks a l= ot=3D Karli, you make my mind clear about deduplication, once again we cannot ha= =3D ve the best of both worlds.
I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, runni= =3D ng on Linux, has no better performance but supports DRDB.
JoseFrom: "Karli Sj=3DC3=3DB6berg" <Karli.Sjoberg(a)slu.se>Sent: Sexta-feira, 31 = de M=3D aio de 2013 10:45:41
To: suporte(a)logicw= orks.pt<=3D /a>
Cc: "Jiri Belka" <jbelka(a)redhat.com>, users(a)ovirt.org
Subject: Re: [Users] deduplication
fre 2013-05-31 klockan 09:50 +0100 skrev suporte(a)logicworks.pt:So, we can say that dedup has more di= sa=3D dvantages than advantages.
For a primary system; most definitely, yes.
But for a backup system, that has tons of RAM and SSD's for cache, and you = =3D have lots of virtual machines that are based off of the template, or are ve= =3D ry much the same, then you have a real use-case. I=3DC2=3DB4m active at the= Fre=3D eBSD forums where one person reports storing 150TB of data in only 30TB of physical disk. The best practice of = =3D scrubbing is once a week on "enterprise" systems, though he is only able to= =3D do it once a month, because that=3DC2=3DB4s how long it takes for a scrub = to c=3D omplete in that system. So you=3DC2=3DB4ve got to choose performance or savings, you can=3DC2=3DB4t have both.
And what about dedup of Netapp?
Much better implementation, in my opinion. You are able schedule dedup-runs= =3D to go at night so your user=3DC2=3DB4s performance isn=3DC2=3DB4t impacted= , and yo=3D u get the savings. The question is if you value the savings enough to take = =3D on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with compression enabled and buy in standard H= =3D DD's from anywhere... We did;)
/Karli
Jose
From: "Karli Sj=3DC3=3DB6berg" <Karli.Sjoberg(a)slu.se>
To: suporte(a)logicwo= rks.pt=3D a>
Cc: "Jiri= Bel=3D ka" <jbelka= (a)redha=3D t.com>, u= sers(a)o=3D virt.org
Sent: Qui= nta-=3D feira, 30 de Maio de 2013 8:33:19
Subject: = Re: =3D [Users] deduplication
ons 2013-05-29 klockan 09:59 +0100 skrev suporte(a)logicworks.p= t:
Absolutely agree with you, planning i= s =3D the best thing to do, but normally people want a plug'n'play system with al= =3D l included, because there is not much time to think and planning, and there= =3D are many companies that know how to take advantage of this people characteristics.
Any way, I think another solution for dedup is Fr= ee=3D NAS using ZFS.
FreeNAS is just FreeBSD with a fancy web-ui ontop= , =3D so it=3DC2=3DB4s neither more or less of ZFS than you would have otherwise,= And=3D regarding dedup in ZFS; Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! I= t=3DC2=3DB4s=3D said that it may in=3D crease performance when you have a very suitable usecase, e.g. everything exactly the same over and over. What=3DC2=3DB4s not said is that scrubbing and re= silv=3D ering slows down to a snail (from hundreds of MB/s, or GB if your system is= =3D large enough, down to less than 10), just from dedup. Also deleting snapshots of datasets that have(or have had) ded= =3D up on can kill the entire system, and when I say kill, I mean really fubar.= =3D Been there, regretted that... Now, compression on the other hand, you get = =3D basically for free and gives decent savings, I highly recommend that.
/Karli
Jose
From: "Ji= ri B=3D elka" <jbel= ka(a)red=3D hat.com>
To: suporte(a)logicwo= rks.pt=3D a>
Cc: users(a)ovirt.org= font>
Sent: Qua= rta-=3D feira, 29 de Maio de 2013 7:33:10
Subject: = Re: =3D [Users] deduplication
On Tue, 28 May 2013 14:29:05 +0100 (WEST)<= br=3D > suporte(a)logicworks.pt wrote:
> That's why I'm making this questions, to dem= ys=3D tify some buzzwords around here.
> But if you have a strong and good technology= w=3D hy not create buzzwords to get into as many people as possible? without tra= =3D pped them.
> Share a disk containing "static" data is a g= oo=3D d idea, do you know from where I can start?
Everything depends on your needs, design planning= . =3D Maybe then sharing
disk would be better to share via NFS/iscsi. Of c= ou=3D rse if you have many
VMs each of them is different you will fail. But = if=3D you have mostly
homogeneous environment you can think about this = ap=3D proach. Sure you have
to have plan for upgrading "base" "static" shared= O=3D S data, you have to
have plan how to install additional software (dif= fe=3D rent destination
than /usr or /usr/local)... If you already have y= ou=3D r own build host
which builds for you OS packages and you have alr= ea=3D dy your own plan for
deployment, you have done first steps. If you dep= en=3D d on upgrading each
machine separately from Internet, then first you = sh=3D ould plan your
environment, configuration management etc.=
Well, in many times people do not do any planning= , =3D they just think some
good technology would save their "poor" design.= fo=3D nt>
j.
--
Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar
---------------------------------------------------------------------------= =3D ----
Karli Sj=3DC3=3DB6berg
Swedish University of Agricultural Sciences
Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)
S-750 07 Uppsala, Sweden
Phone: +46-(0)18-67 15 66
karli.s= joberg=3D @slu.se
--
Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar
---------------------------------------------------------------------------= =3D ----
Karli Sj=3DC3=3DB6berg
Swedish University of Agricultural Sciences
Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)
S-750 07 Uppsala, Sweden
Phone: +46-(0)18-67 15 66
karli.s= joberg=3D @slu.se
_____________________________________= =3D __________
Users mailing list
Users(a)ovirt.or= g
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list
Users(a)ovirt.or= g
http://lists.ovirt.org/mailman/listinfo/users
--
Chris = No=3D ffsinger
Thanks a lot Karli, you make my mind clear about deduplication, once again = =3D we cannot have the best of both worlds.=3D20 I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, running o= =3D n Linux, has no better performance but supports DRDB.=3D20 Jose=3D20 From: "Karli Sj=3DC3=3DB6berg" < Karli.Sjoberg(a)slu.se >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: "Jiri Belka" < jbelka(a)redhat.com >, users(a)ovirt.org=3D20 Sent: Sexta-feira, 31 de Maio de 2013 10:45:41=3D20 Subject: Re: [Users] deduplication=3D20 fre 2013-05-31 klockan 09:50 +0100 skrev suporte(a)logicworks.pt :=3D20------=3D_Part_237_16733285.1370360396749 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: quoted-printable <=3D div style=3D3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; co= lo=3D r: #000000'>Hi Juanjo,So, we can say that dedup has more disadvantages than advantages.=3D20 For a primary system; most definitely, yes.=3D20 But for a backup system, that has tons of RAM and SSD's for cache, and you = =3D have lots of virtual machines that are based off of the template, or are ve= =3D ry much the same, then you have a real use-case. I=3DC2=3DB4m active at the= Fre=3D eBSD forums where one person reports storing 150TB of data in only 30TB of = =3D physical disk. The best practice of scrubbing is once a week on "enterprise= =3D " systems, though he is only able to do it once a month, because that=3DC2= =3DB4=3D s how long it takes for a scrub to complete in that system. So you=3DC2=3DB= 4ve =3D got to choose performance or savings, you can=3DC2=3DB4t have both.=3D20And what about dedup of Netapp?=3D20Much better implementation, in my opinion. You are able schedule dedup-runs= =3D to go at night so your user=3DC2=3DB4s performance isn=3DC2=3DB4t impacted= , and yo=3D u get the savings. The question is if you value the savings enough to take = =3D on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with= =3D compression enabled and buy in standard HDD's from anywhere... We did;)=3D= 20 /Karli=3D20Jose=3D20From: "Karli Sj=3DC3=3DB6berg" < Karli.Sjoberg(a)slu.se >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: "Jiri Belka" < jbelka(a)redhat.com >, users(a)ovirt.org=3D20 Sent: Quinta-feira, 30 de Maio de 2013 8:33:19=3D20 Subject: Re: [Users] deduplication=3D20 ons 2013-05-29 klockan 09:59 +0100 skrev suporte(a)logicworks.pt :=3D20Absolutely agree with you, planning is the best thing to do, but normally p= =3D eople want a plug'n'play system with all included, because there is not muc= =3D h time to think and planning, and there are many companies that know how to= =3D take advantage of this people characteristics.=3D20 Any way, I think another solution for dedup is FreeNAS using ZFS.=3D20FreeNAS is just FreeBSD with a fancy web-ui ontop, so it=3DC2=3DB4s neither= mor=3D e or less of ZFS than you would have otherwise, And regarding dedup in ZFS;= =3D Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! It=3DC2=3DB4s said that i= t may increa=3D se performance when you have a very suitable usecase, e.g. everything exact= =3D ly the same over and over. What=3DC2=3DB4s not said is that scrubbing and r= esil=3D vering slows down to a snail (from hundreds of MB/s, or GB if your system i= =3D s large enough, down to less than 10), just from dedup. Also deleting snaps= =3D hots of datasets that have(or have had) dedup on can kill the entire system= =3D , and when I say kill, I mean really fubar. Been there, regretted that... N= =3D ow, compression on the other hand, you get basically for free and gives dec= =3D ent savings, I highly recommend that.=3D20 /Karli=3D20Jose=3D20 From: "Jiri Belka" < jbelka(a)redhat.com >=3D20 To: suporte(a)logicworks.pt=3D20 Cc: users(a)ovirt.org=3D20 Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=3D20 Subject: Re: [Users] deduplication=3D20 On Tue, 28 May 2013 14:29:05 +0100 (WEST)=3D20 suporte(a)logicworks.pt wrote:=3D20 > That's why I'm making this questions, to demystify some buzzwords around = =3D here.=3D20 > But if you have a strong and good technology why not create buzzwords to = =3D get into as many people as possible? without trapped them.=3D20 > Share a disk containing "static" data is a good idea, do you know from wh= =3D ere I can start?=3D20 Everything depends on your needs, design planning. Maybe then sharing=3D20 disk would be better to share via NFS/iscsi. Of course if you have many=3D20 VMs each of them is different you will fail. But if you have mostly=3D20 homogeneous environment you can think about this approach. Sure you have=3D= 20 to have plan for upgrading "base" "static" shared OS data, you have to=3D20 have plan how to install additional software (different destination=3D20 than /usr or /usr/local)... If you already have your own build host=3D20 which builds for you OS packages and you have already your own plan for=3D20 deployment, you have done first steps. If you depend on upgrading each=3D20 machine separately from Internet, then first you should plan your=3D20 environment, configuration management etc.=3D20 Well, in many times people do not do any planning, they just think some=3D20 good technology would save their "poor" design.=3D20 j.=3D20=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20=3D09--=3D20 Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar=3D20 ---------------------------------------------------------------------------= =3D ----=3D20 Karli Sj=3DC3=3DB6berg=3D20 Swedish University of Agricultural Sciences=3D20 Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)=3D20 S-750 07 Uppsala, Sweden=3D20 Phone: +46-(0)18-67 15 66=3D20 karli.sjoberg(a)slu.se=3D20 _______________________________________________=3D20 Users mailing list=3D20 Users(a)ovirt.org=3D20 http://lists.ovirt.org/mailman/listinfo/users=3D20
Hi Juan,=3D
thanks for your info, I'll try to test FreeNAS with compression. Do you= =3D use it with iSCSI or NFS?
JoseFrom: "Juan Jose" <jj197005(a)gmail.com>
To: supor=3D te(a)logicworks.pt, =3D users(a)ovirt.org
Sent: Segunda-feira, 3 de Junho de 2013 13:= 37=3D :21
Subject: Re: [Users] deduplication
<= di=3D v dir=3D3D"ltr">Hello Jose,We also have = Fr=3D eeNAS working in our infraestructure, with about 3 TB and ZFS. Some of the = =3D pools has compression enabled and you can save space with it. We have this = =3D FreeNAS connected to a hypervisor Xen and it works very well and it's stabl= =3D e and sure. We have nine virtual servers some wirtualized and other paravir= =3D tualized, and some Windows Server machine all about 2 years in production w= =3D ithout any problem. My idea is connect this infrastructure with oVirt wo be= =3D able to have some resources for test VMs in that. Only wanted to share as = =3D another FreeNas success experience.Juanjo.
<=3D div class=3D3D"gmail_quote">On Fri, May 31, 2013 at 12:33 PM, <su= porte@=3D logicworks.pt> wrote:
Thanks a l= ot=3D Karli, you make my mind clear about deduplication, once again we cannot ha= =3D ve the best of both worlds.
I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, runni= =3D ng on Linux, has no better performance but supports DRDB.
JoseFrom: "Karli Sj=3DC3=3DB6berg" <Karli.Sjoberg(a)slu.se>Sent: Sexta-feira, 31 = de M=3D aio de 2013 10:45:41
To: suporte(a)logicw= orks.pt<=3D /a>
Cc: "Jiri Belka" <jbelka(a)redhat.com>, users(a)ovirt.org
Subject: Re: [Users] deduplication
fre 2013-05-31 klockan 09:50 +0100 skrev suporte(a)logicworks.pt:So, we can say that dedup has more di= sa=3D dvantages than advantages.
For a primary system; most definitely, yes.
But for a backup system, that has tons of RAM and SSD's for cache, and you = =3D have lots of virtual machines that are based off of the template, or are ve= =3D ry much the same, then you have a real use-case. I=3DC2=3DB4m active at the= Fre=3D eBSD forums where one person reports storing 150TB of data in only 30TB of physical disk. The best practice of = =3D scrubbing is once a week on "enterprise" systems, though he is only able to= =3D do it once a month, because that=3DC2=3DB4s how long it takes for a scrub = to c=3D omplete in that system. So you=3DC2=3DB4ve got to choose performance or savings, you can=3DC2=3DB4t have both.
And what about dedup of Netapp?
Much better implementation, in my opinion. You are able schedule dedup-runs= =3D to go at night so your user=3DC2=3DB4s performance isn=3DC2=3DB4t impacted= , and yo=3D u get the savings. The question is if you value the savings enough to take = =3D on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with compression enabled and buy in standard H= =3D DD's from anywhere... We did;)
/Karli
Jose
From: "Karli Sj=3DC3=3DB6berg" <Karli.Sjoberg(a)slu.se>
To: suporte(a)logicwo= rks.pt=3D a>
Cc: "Jiri= Bel=3D ka" <jbelka= (a)redha=3D t.com>, u= sers(a)o=3D virt.org
Sent: Qui= nta-=3D feira, 30 de Maio de 2013 8:33:19
Subject: = Re: =3D [Users] deduplication
ons 2013-05-29 klockan 09:59 +0100 skrev suporte(a)logicworks.p= t:
Absolutely agree with you, planning i= s =3D the best thing to do, but normally people want a plug'n'play system with al= =3D l included, because there is not much time to think and planning, and there= =3D are many companies that know how to take advantage of this people characteristics.
Any way, I think another solution for dedup is Fr= ee=3D NAS using ZFS.
FreeNAS is just FreeBSD with a fancy web-ui ontop= , =3D so it=3DC2=3DB4s neither more or less of ZFS than you would have otherwise,= And=3D regarding dedup in ZFS; Just don=3DC2=3DB4t, it=3DC2=3DB4s not worth it! I= t=3DC2=3DB4s=3D said that it may in=3D crease performance when you have a very suitable usecase, e.g. everything exactly the same over and over. What=3DC2=3DB4s not said is that scrubbing and re= silv=3D ering slows down to a snail (from hundreds of MB/s, or GB if your system is= =3D large enough, down to less than 10), just from dedup. Also deleting snapshots of datasets that have(or have had) ded= =3D up on can kill the entire system, and when I say kill, I mean really fubar.= =3D Been there, regretted that... Now, compression on the other hand, you get = =3D basically for free and gives decent savings, I highly recommend that.
/Karli
Jose
From: "Ji= ri B=3D elka" <jbel= ka(a)red=3D hat.com>
To: suporte(a)logicwo= rks.pt=3D a>
Cc: users(a)ovirt.org= font>
Sent: Qua= rta-=3D feira, 29 de Maio de 2013 7:33:10
Subject: = Re: =3D [Users] deduplication
On Tue, 28 May 2013 14:29:05 +0100 (WEST)<= br=3D > suporte(a)logicworks.pt wrote:
> That's why I'm making this questions, to dem= ys=3D tify some buzzwords around here.
> But if you have a strong and good technology= w=3D hy not create buzzwords to get into as many people as possible? without tra= =3D pped them.
> Share a disk containing "static" data is a g= oo=3D d idea, do you know from where I can start?
Everything depends on your needs, design planning= . =3D Maybe then sharing
disk would be better to share via NFS/iscsi. Of c= ou=3D rse if you have many
VMs each of them is different you will fail. But = if=3D you have mostly
homogeneous environment you can think about this = ap=3D proach. Sure you have
to have plan for upgrading "base" "static" shared= O=3D S data, you have to
have plan how to install additional software (dif= fe=3D rent destination
than /usr or /usr/local)... If you already have y= ou=3D r own build host
which builds for you OS packages and you have alr= ea=3D dy your own plan for
deployment, you have done first steps. If you dep= en=3D d on upgrading each
machine separately from Internet, then first you = sh=3D ould plan your
environment, configuration management etc.=
Well, in many times people do not do any planning= , =3D they just think some
good technology would save their "poor" design.= fo=3D nt>
j.
--
Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar
---------------------------------------------------------------------------= =3D ----
Karli Sj=3DC3=3DB6berg
Swedish University of Agricultural Sciences
Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)
S-750 07 Uppsala, Sweden
Phone: +46-(0)18-67 15 66
karli.s= joberg=3D @slu.se
--
Med V=3DC3=3DA4nliga H=3DC3=3DA4lsningar
---------------------------------------------------------------------------= =3D ----
Karli Sj=3DC3=3DB6berg
Swedish University of Agricultural Sciences
Box 7079 (Visiting Address Kron=3DC3=3DA5sv=3DC3=3DA4gen 8)
S-750 07 Uppsala, Sweden
Phone: +46-(0)18-67 15 66
karli.s= joberg=3D @slu.se
_____________________________________= =3D __________
Users mailing list
Users(a)ovirt.or= g
http://lists.ovirt.org/mailman/listinfo/users