Re: [Users] deduplication

------=_Part_1134_11907452.1369996394775 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Thanks a lot Karli, you make my mind clear about deduplication, once again = we cannot have the best of both worlds.=20 I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, running o= n Linux, has no better performance but supports DRDB.=20 Jose=20 ----- Original Message ----- From: "Karli Sj=C3=B6berg" <Karli.Sjoberg(a)slu.se&gt;=20 To: suporte(a)logicworks.pt=20 Cc: "Jiri Belka" <jbelka(a)redhat.com&gt;, users(a)ovirt.org=20 Sent: Sexta-feira, 31 de Maio de 2013 10:45:41=20 Subject: Re: [Users] deduplication=20 fre 2013-05-31 klockan 09:50 +0100 skrev suporte@logicworks.pt:=20 So, we can say that dedup has more disadvantages than advantages.=20 For a primary system; most definitely, yes.=20 But for a backup system, that has tons of RAM and SSD's for cache, and you = have lots of virtual machines that are based off of the template, or are ve= ry much the same, then you have a real use-case. I=C2=B4m active at the Fre= eBSD forums where one person reports storing 150TB of data in only 30TB of = physical disk. The best practice of scrubbing is once a week on "enterprise= " systems, though he is only able to do it once a month, because that=C2=B4= s how long it takes for a scrub to complete in that system. So you=C2=B4ve = got to choose performance or savings, you can=C2=B4t have both.=20 <blockquote And what about dedup of Netapp?=20 </blockquote Much better implementation, in my opinion. You are able schedule dedup-runs= to go at night so your user=C2=B4s performance isn=C2=B4t impacted, and yo= u get the savings. The question is if you value the savings enough to take = on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with= compression enabled and buy in standard HDD's from anywhere... We did;)=20 /Karli=20 <blockquote Jose=20 </blockquote <blockquoteFrom: "Karli Sj=C3=B6berg" <Karli.Sjoberg(a)slu.se&gt;=20 To: suporte(a)logicworks.pt=20 Cc: "Jiri Belka" <jbelka(a)redhat.com&gt;, users(a)ovirt.org=20 Sent: Quinta-feira, 30 de Maio de 2013 8:33:19=20 Subject: Re: [Users] deduplication=20 ons 2013-05-29 klockan 09:59 +0100 skrev suporte@logicworks.pt:=20 <blockquoteAbsolutely agree with you, planning is the best thing to do, but normally p= eople want a plug'n'play system with all included, because there is not muc= h time to think and planning, and there are many companies that know how to= take advantage of this people characteristics.=20 Any way, I think another solution for dedup is FreeNAS using ZFS.=20 </blockquote FreeNAS is just FreeBSD with a fancy web-ui ontop, so it=C2=B4s neither mor= e or less of ZFS than you would have otherwise, And regarding dedup in ZFS;= Just don=C2=B4t, it=C2=B4s not worth it! It=C2=B4s said that it may increa= se performance when you have a very suitable usecase, e.g. everything exact= ly the same over and over. What=C2=B4s not said is that scrubbing and resil= vering slows down to a snail (from hundreds of MB/s, or GB if your system i= s large enough, down to less than 10), just from dedup. Also deleting snaps= hots of datasets that have(or have had) dedup on can kill the entire system= , and when I say kill, I mean really fubar. Been there, regretted that... N= ow, compression on the other hand, you get basically for free and gives dec= ent savings, I highly recommend that.=20 /Karli=20 <blockquote Jose=20 From: "Jiri Belka" <jbelka(a)redhat.com&gt;=20 To: suporte(a)logicworks.pt=20 Cc: users(a)ovirt.org=20 Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=20 Subject: Re: [Users] deduplication=20 On Tue, 28 May 2013 14:29:05 +0100 (WEST)=20 suporte(a)logicworks.pt wrote:=20 here.=20 get into as many people as possible? without trapped them.=20 ere I can start?=20 Everything depends on your needs, design planning. Maybe then sharing=20 disk would be better to share via NFS/iscsi. Of course if you have many=20 VMs each of them is different you will fail. But if you have mostly=20 homogeneous environment you can think about this approach. Sure you have=20 to have plan for upgrading "base" "static" shared OS data, you have to=20 have plan how to install additional software (different destination=20 than /usr or /usr/local)... If you already have your own build host=20 which builds for you OS packages and you have already your own plan for=20 deployment, you have done first steps. If you depend on upgrading each=20 machine separately from Internet, then first you should plan your=20 environment, configuration management etc.=20 Well, in many times people do not do any planning, they just think some=20 good technology would save their "poor" design.=20 j.=20 </blockquote =09--=20 Med V=C3=A4nliga H=C3=A4lsningar=20 ---------------------------------------------------------------------------= ----=20 Karli Sj=C3=B6berg=20 Swedish University of Agricultural Sciences=20 Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)=20 S-750 07 Uppsala, Sweden=20 Phone: +46-(0)18-67 15 66=20 karli.sjoberg(a)slu.se=20 </blockquote <blockquote </blockquote =09--=20 Med V=C3=A4nliga H=C3=A4lsningar=20 ---------------------------------------------------------------------------= ----=20 Karli Sj=C3=B6berg=20 Swedish University of Agricultural Sciences=20 Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)=20 S-750 07 Uppsala, Sweden=20 Phone: +46-(0)18-67 15 66=20 karli.sjoberg(a)slu.se=20 ------=_Part_1134_11907452.1369996394775 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><head><style type=3D'text/css'>p { margin: 0; }</style></head><body><= div style=3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; colo= r: #000000'>Thanks a lot Karli, you make my mind clear about deduplication,= once again we cannot have the best of both worlds.<br><br>I'll try FreeNAS= despite my poor knowledge on FreeBSD. Openfiler, running on Linux, has no = better performance but supports DRDB.<br><br>Jose<br><br><hr id=3D"zwchr"><= div style=3D"color: rgb(0, 0, 0); font-weight: normal; font-style: normal; = text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: = 12pt;"><b>From: </b>"Karli Sj=C3=B6berg" &amp;lt;Karli.Sjoberg(a)slu.se&amp;gt;&lt;br&gt;&lt;b= .com&gt;, users(a)ovirt.org&lt;br&gt;&lt;b&gt;Sent: </b>Sexta-feira, 31 de Maio de 2013 1= 0:45:41<br><b>Subject: </b>Re: [Users] deduplication<br><br fre 2013-05-31 klockan 09:50 +0100 skrev suporte(a)logicworks.pt: <blockquote><font color=3D"#000000">So, we can say that dedup has more disa= dvantages than advantages.</font><br</blockquote<brFor a primary system; most definitely, yes.<br<brBut for a backup system, that has tons of RAM and SSD's for cache, and you = have lots of virtual machines that are based off of the template, or are ve= ry much the same, then you have a real use-case. I=C2=B4m active at the Fre= eBSD forums where one person reports storing 150TB of data in only 30TB of physical disk. The best practice of = scrubbing is once a week on "enterprise" systems, though he is only able to= do it once a month, because that=C2=B4s how long it takes for a scrub to c= omplete in that system. So you=C2=B4ve got to choose performance or savings, you can=C2=B4t have both.<br<br<blockquote><br<font color=3D"#000000">And what about dedup of Netapp?</font><br</blockquote<brMuch better implementation, in my opinion. You are able schedule dedup-runs= to go at night so your user=C2=B4s performance isn=C2=B4t impacted, and yo= u get the savings. The question is if you value the savings enough to take = on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with compression enabled and buy in standard H= DD's from anywhere... We did;)<br<br/Karli<br<br<blockquote><br<font color=3D"#000000">Jose</font><br<br<hr align=3D"center"</blockquote<blockquote><b><font color=3D"#000000">From: </font></b><font color=3D"#000= 000">"Karli Sj=C3=B6berg" &amp;lt;Karli.Sjoberg(a)slu.se&amp;gt;&lt;/font&gt;&lt;br<b><font color=3D"#000000">To: </font></b><font color=3D"#000000">suporte@l= ogicworks.pt</font><br<b><font color=3D"#000000">Cc: </font></b><font color=3D"#000000">"Jiri Bel= ka" &amp;lt;jbelka(a)redhat.com&amp;gt;, users(a)ovirt.org&lt;/font&gt;&lt;br<b><font color=3D"#000000">Sent: </font></b><font color=3D"#000000">Quinta-= feira, 30 de Maio de 2013 8:33:19</font><br<b><font color=3D"#000000">Subject: </font></b><font color=3D"#000000">Re: = [Users] deduplication</font><br<br<font color=3D"#000000">ons 2013-05-29 klockan 09:59 +0100 skrev suporte@lo= gicworks.pt: </font><br<blockquote><font color=3D"#000000">Absolutely agree with you, planning is = the best thing to do, but normally people want a plug'n'play system with al= l included, because there is not much time to think and planning, and there= are many companies that know how to take advantage of this people characteristics.</font><br<font color=3D"#000000">Any way, I think another solution for dedup is Free= NAS using ZFS.</font><br</blockquote<br<font color=3D"#000000">FreeNAS is just FreeBSD with a fancy web-ui ontop, = so it=C2=B4s neither more or less of ZFS than you would have otherwise, And= regarding dedup in ZFS; Just don=C2=B4t, it=C2=B4s not worth it! It=C2=B4s= said that it </font><font color=3D"#000000"><b>may</b></font><font color=3D"#000000"> in= crease performance when you have a very suitable usecase, e.g. everything </font><font color=3D"#000000"><b>exactly</b></font><font color=3D"#000000"= ering slows down to a snail (from hundreds of MB/s, or GB if your system is= large enough, down to less than 10), just from dedup. Also deleting snapshots of datasets that have(or have had) ded= up on can kill the entire system, and when I say kill, I mean really fubar.= Been there, regretted that... Now, compression on the other hand, you get = basically for free and gives decent savings, I highly recommend that.</font><br<br<font color=3D"#000000">/Karli</font><br<br<blockquote><br<font color=3D"#000000">Jose</font><br<br<br<hr align=3D"center"<br<b><font color=3D"#000000">From: </font></b><font color=3D"#000000">"Jiri B= elka" &amp;lt;jbelka(a)redhat.com&amp;gt;&lt;/font&gt;&lt;br<b><font color=3D"#000000">To: </font></b><font color=3D"#000000">suporte@l= ogicworks.pt</font><br<b><font color=3D"#000000">Cc: </font></b><font color=3D"#000000">users@ovi= rt.org</font><br<b><font color=3D"#000000">Sent: </font></b><font color=3D"#000000">Quarta-= feira, 29 de Maio de 2013 7:33:10</font><br<b><font color=3D"#000000">Subject: </font></b><font color=3D"#000000">Re: = [Users] deduplication</font><br<br<font color=3D"#000000">On Tue, 28 May 2013 14:29:05 +0100 (WEST)</font><br= <font color=3D&quot;#000000&quot;&gt;suporte(a)logicworks.pt wrote:</font><br<br<font color=3D"#000000">&gt; That's why I'm making this questions, to demys= tify some buzzwords around here.</font><br<font color=3D"#000000">&gt; But if you have a strong and good technology w= hy not create buzzwords to get into as many people as possible? without tra= pped them.</font><br<font color=3D"#000000">&gt; Share a disk containing "static" data is a goo= d idea, do you know from where I can start?</font><br<br<font color=3D"#000000">Everything depends on your needs, design planning. = Maybe then sharing</font><br<font color=3D"#000000">disk would be better to share via NFS/iscsi. Of cou= rse if you have many</font><br<font color=3D"#000000">VMs each of them is different you will fail. But if= you have mostly</font><br<font color=3D"#000000">homogeneous environment you can think about this ap= proach. Sure you have</font><br<font color=3D"#000000">to have plan for upgrading "base" "static" shared O= S data, you have to</font><br<font color=3D"#000000">have plan how to install additional software (diffe= rent destination</font><br<font color=3D"#000000">than /usr or /usr/local)... If you already have you= r own build host</font><br<font color=3D"#000000">which builds for you OS packages and you have alrea= dy your own plan for</font><br<font color=3D"#000000">deployment, you have done first steps. If you depen= d on upgrading each</font><br<font color=3D"#000000">machine separately from Internet, then first you sh= ould plan your</font><br<font color=3D"#000000">environment, configuration management etc.</font><b= r<br<font color=3D"#000000">Well, in many times people do not do any planning, = they just think some</font><br<font color=3D"#000000">good technology would save their "poor" design.</fo= nt><br<br<font color=3D"#000000">j.</font><br<br<br<br</blockquote<br<table cellpadding=3D"0" cellspacing=3D"0" width=3D"100%"<tbody<tr<td>-- <br<brMed V=C3=A4nliga H=C3=A4lsningar<br---------------------------------------------------------------------------= ----<brKarli Sj=C3=B6berg<brSwedish University of Agricultural Sciences<brBox 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)<brS-750 07 Uppsala, Sweden<brPhone: &nbsp;+46-(0)18-67 15 66<br<a href=3D"mailto:karli.sjoberg@adm.slu.se" target=3D"_blank">karli.sjoberg= @slu.se</a> </td</tr</tbody</table</blockquote<blockquote><br<br</blockquote<br<table cellpadding=3D"0" cellspacing=3D"0" width=3D"100%"<tbody<tr<td>-- <br<brMed V=C3=A4nliga H=C3=A4lsningar<br---------------------------------------------------------------------------= ----<brKarli Sj=C3=B6berg<brSwedish University of Agricultural Sciences<brBox 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)<brS-750 07 Uppsala, Sweden<brPhone: &nbsp;+46-(0)18-67 15 66<br<a href=3D"mailto:karli.sjoberg@adm.slu.se" target=3D"_blank">karli.sjoberg= @slu.se</a> </td</tr</tbody</table</div><br></div></body></html------=_Part_1134_11907452.1369996394775--