Re: [Users] deduplication

3 Jun 2013

      ------=_Part_1980_16023178.1370280384707
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

If we have a hardware RAID controller will we need RAID-Z ?=20

Jose=20

----- Original Message -----

From: "Chris Noffsinger" <cnoffsin@gmail.com>=20
To: suporte@logicworks.pt=20
Cc: "Juan Jose" <jj197005@gmail.com>, users@ovirt.org=20
Sent: Segunda-feira, 3 de Junho de 2013 17:16:55=20
Subject: Re: [Users] deduplication=20

Just wanted to add that Freenas is great. I use it with NFS and ISCSI and i=
t works well. What I will say, on the HP DNS-320 I have in it I have had to=
 go to the command prompt to fix some multipathing issues when I first add =
a disk but I beleive that is just a product of the cciss controller driver =
in that server.=20

On Mon, Jun 3, 2013 at 12:12 PM, < suporte@logicworks.pt > wrote:=20

Hi Juan,=20

thanks for your info, I'll try to test FreeNAS with compression. Do you use=
 it with iSCSI or NFS?=20

Jose=20

From: "Juan Jose" < jj197005@gmail.com >=20
To: suporte@logicworks.pt , users@ovirt.org=20
Sent: Segunda-feira, 3 de Junho de 2013 13:37:21=20
Subject: Re: [Users] deduplication=20

Hello Jose,=20

We also have FreeNAS working in our infraestructure, with about 3 TB and ZF=
S. Some of the pools has compression enabled and you can save space with it=
. We have this FreeNAS connected to a hypervisor Xen and it works very well=
 and it's stable and sure. We have nine virtual servers some wirtualized an=
d other paravirtualized, and some Windows Server machine all about 2 years =
in production without any problem. My idea is connect this infrastructure w=
ith oVirt wo be able to have some resources for test VMs in that. Only want=
ed to share as another FreeNas success experience.=20

Juanjo.=20

On Fri, May 31, 2013 at 12:33 PM, < suporte@logicworks.pt > wrote:=20

<blockquote>

Thanks a lot Karli, you make my mind clear about deduplication, once again =
we cannot have the best of both worlds.=20

I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, running o=
n Linux, has no better performance but supports DRDB.=20

Jose=20

From: "Karli Sj=C3=B6berg" < Karli.Sjoberg@slu.se >=20
To: suporte@logicworks.pt=20
Cc: "Jiri Belka" < jbelka@redhat.com >, users@ovirt.org=20
Sent: Sexta-feira, 31 de Maio de 2013 10:45:41=20
Subject: Re: [Users] deduplication=20

fre 2013-05-31 klockan 09:50 +0100 skrev suporte@logicworks.pt :=20
<blockquote>
So, we can say that dedup has more disadvantages than advantages.=20

For a primary system; most definitely, yes.=20

But for a backup system, that has tons of RAM and SSD's for cache, and you =
have lots of virtual machines that are based off of the template, or are ve=
ry much the same, then you have a real use-case. I=C2=B4m active at the Fre=
eBSD forums where one person reports storing 150TB of data in only 30TB of =
physical disk. The best practice of scrubbing is once a week on "enterprise=
" systems, though he is only able to do it once a month, because that=C2=B4=
s how long it takes for a scrub to complete in that system. So you=C2=B4ve =
got to choose performance or savings, you can=C2=B4t have both.=20

<blockquote>

And what about dedup of Netapp?=20

</blockquote>

Much better implementation, in my opinion. You are able schedule dedup-runs=
 to go at night so your user=C2=B4s performance isn=C2=B4t impacted, and yo=
u get the savings. The question is if you value the savings enough to take =
on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with=
 compression enabled and buy in standard HDD's from anywhere... We did;)=20

/Karli=20

<blockquote>

Jose=20

</blockquote>

<blockquote>
From: "Karli Sj=C3=B6berg" < Karli.Sjoberg@slu.se >=20
To: suporte@logicworks.pt=20
Cc: "Jiri Belka" < jbelka@redhat.com >, users@ovirt.org=20
Sent: Quinta-feira, 30 de Maio de 2013 8:33:19=20
Subject: Re: [Users] deduplication=20

ons 2013-05-29 klockan 09:59 +0100 skrev suporte@logicworks.pt :=20

<blockquote>
Absolutely agree with you, planning is the best thing to do, but normally p=
eople want a plug'n'play system with all included, because there is not muc=
h time to think and planning, and there are many companies that know how to=
 take advantage of this people characteristics.=20
Any way, I think another solution for dedup is FreeNAS using ZFS.=20

</blockquote>

FreeNAS is just FreeBSD with a fancy web-ui ontop, so it=C2=B4s neither mor=
e or less of ZFS than you would have otherwise, And regarding dedup in ZFS;=
 Just don=C2=B4t, it=C2=B4s not worth it! It=C2=B4s said that it may increa=
se performance when you have a very suitable usecase, e.g. everything exact=
ly the same over and over. What=C2=B4s not said is that scrubbing and resil=
vering slows down to a snail (from hundreds of MB/s, or GB if your system i=
s large enough, down to less than 10), just from dedup. Also deleting snaps=
hots of datasets that have(or have had) dedup on can kill the entire system=
, and when I say kill, I mean really fubar. Been there, regretted that... N=
ow, compression on the other hand, you get basically for free and gives dec=
ent savings, I highly recommend that.=20

/Karli=20

<blockquote>

Jose=20

From: "Jiri Belka" < jbelka@redhat.com >=20
To: suporte@logicworks.pt=20
Cc: users@ovirt.org=20
Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=20
Subject: Re: [Users] deduplication=20

On Tue, 28 May 2013 14:29:05 +0100 (WEST)=20
suporte@logicworks.pt wrote:=20
...
That's why I'm making this questions, to demystify some buzzwords around =
here.=20
But if you have a strong and good technology why not create buzzwords to =
get into as many people as possible? without trapped them.=20
Share a disk containing "static" data is a good idea, do you know from wh=
ere I can start?=20
...
<div> </div><div>We also have FreeNAS working in our infraestructure, w=
ith about 3 TB and ZFS. Some of the pools has compression enabled and you c=
an save space with it. We have this FreeNAS connected to a hypervisor Xen a=
nd it works very well and it's stable and sure. We have nine virtual server=
s some wirtualized and other paravirtualized, and some Windows Server machi=
ne all about 2 years in production without any problem. My idea is connect =
Everything depends on your needs, design planning. Maybe then sharing=20
disk would be better to share via NFS/iscsi. Of course if you have many=20
VMs each of them is different you will fail. But if you have mostly=20
homogeneous environment you can think about this approach. Sure you have=20
to have plan for upgrading "base" "static" shared OS data, you have to=20
have plan how to install additional software (different destination=20
than /usr or /usr/local)... If you already have your own build host=20
which builds for you OS packages and you have already your own plan for=20
deployment, you have done first steps. If you depend on upgrading each=20
machine separately from Internet, then first you should plan your=20
environment, configuration management etc.=20

Well, in many times people do not do any planning, they just think some=20
good technology would save their "poor" design.=20

j.=20

</blockquote>

=09--=20

Med V=C3=A4nliga H=C3=A4lsningar=20
---------------------------------------------------------------------------=
----=20
Karli Sj=C3=B6berg=20
Swedish University of Agricultural Sciences=20
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)=20
S-750 07 Uppsala, Sweden=20
Phone: +46-(0)18-67 15 66=20
karli.sjoberg@slu.se=20
</blockquote>

<blockquote>

</blockquote>

=09--=20

Med V=C3=A4nliga H=C3=A4lsningar=20
---------------------------------------------------------------------------=
----=20
Karli Sj=C3=B6berg=20
Swedish University of Agricultural Sciences=20
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)=20
S-750 07 Uppsala, Sweden=20
Phone: +46-(0)18-67 15 66=20
karli.sjoberg@slu.se=20

_______________________________________________=20
Users mailing list=20
Users@ovirt.org=20
http://lists.ovirt.org/mailman/listinfo/users=20

</blockquote>

_______________________________________________=20
Users mailing list=20
Users@ovirt.org=20
http://lists.ovirt.org/mailman/listinfo/users=20

</blockquote>

--=20
Chris Noffsinger=20

------=_Part_1980_16023178.1370280384707
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><style type=3D'text/css'>p { margin: 0; }</style></head><body><=
div style=3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; colo=
r: #000000'>If we have a hardware RAID controller will we need RAID-Z ? =
 Jose <hr id=3D"zwchr"><div style=3D"color: rgb(0, 0, 0); font-we=
ight: normal; font-style: normal; text-decoration: none; font-family: Helve=
tica,Arial,sans-serif; font-size: 12pt;">From: "Chris Noffsinger" &l=
t;cnoffsin@gmail.com> To: suporte@logicworks.pt Cc: =
"Juan Jose" <jj197005@gmail.com>, users@ovirt.org Sent: Seg=
unda-feira, 3 de Junho de 2013 17:16:55 Subject: Re: [Users] dedu=
plication Just wanted to add that Freenas is great.  I use it w=
ith NFS and ISCSI and it works well.  What I will say, on the HP DNS-3=
20 I have in it I have had to go to the command prompt to fix some multipat=
hing issues when I first add a disk but I beleive that is just a product of=
 the cciss controller driver in that server. 
 <div class=3D"gmail_quote">On Mon, Jun 3, 2013 at 12:12 PM, <<a href=3D"mailto:suporte@logicworks.pt" target=3D"_blank">sup=
orte@logicworks.pt</a>> wrote: <blockquote class=3D"gmail_quot=
e" style=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204,=
 204); padding-left: 1ex;">
<div><div style=3D"font-size: 10pt; font-family: arial,helvetica,sans-serif=
;">Hi Juan, thanks for your info, I'll try to test FreeNAS with comp=
ression. Do you use it with iSCSI or NFS? Jose <hr><div style=
=3D"font-size: 12pt; font-style: normal; font-family: Helvetica,Arial,sans-=
serif; text-decoration: none; font-weight: normal;">
From: "Juan Jose" <<a href=3D"mailto:jj197005@gmail.com" target=
=3D"_blank">jj197005@gmail.com</a>> To: <a href=3D"mailto:supo=
rte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt</a>, <a href=3D"=
mailto:users@ovirt.org" target=3D"_blank">users@ovirt.org</a> 
Sent: Segunda-feira, 3 de Junho de 2013 13:37:21 Subject: =
Re: [Users] deduplication <div dir=3D"ltr"> <div>Hello Jose,</div=
this infrastructure with oVirt wo be able to have some resources for test V=
Ms in that. Only wanted to share as another FreeNas success experience.</di=
v>

<div><br></div><div>Juanjo.</div></div><div class=3D"gmail_extra"><br><br><=
div class=3D"gmail_quote">On Fri, May 31, 2013 at 12:33 PM,  <span dir=3D"l=
tr"><<a href=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@=
logicworks.pt</a>></span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; borde=
r-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div><div style=
=3D"font-size: 10pt; font-family: arial,helvetica,sans-serif;">Thanks a lot=
 Karli, you make my mind clear about deduplication, once again we cannot ha=
ve the best of both worlds.<br>

I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, runni=
ng on Linux, has no better performance but supports DRDB. Jose <b=
r><hr><div style=3D"font-size: 12pt; font-style: normal; font-family: Helve=
tica,Arial,sans-serif; text-decoration: none; font-weight: normal;">

<div><b>From: </b>"Karli Sj=C3=B6berg" <<a href=3D"mailto:Karli.Sjoberg@=
slu.se" target=3D"_blank">Karli.Sjoberg@slu.se</a>><br><b>To: </b><a hre=
f=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt<=
/a><br>

<b>Cc: </b>"Jiri Belka" <<a href=3D"mailto:jbelka@redhat.com" target=3D"=
_blank">jbelka@redhat.com</a>>, <a href=3D"mailto:users@ovirt.org" targe=
t=3D"_blank">users@ovirt.org</a><br></div><b>Sent: </b>Sexta-feira, 31 de M=
aio de 2013 10:45:41<br>

<b>Subject: </b>Re: [Users] deduplication<div><div><br><br>

fre 2013-05-31 klockan 09:50 +0100 skrev <a href=3D"mailto:suporte@logicwor=
ks.pt" target=3D"_blank">suporte@logicworks.pt</a>:
<blockquote>So, we can say that dedup has more disa=
dvantages than advantages. 
</blockquote>
 
For a primary system; most definitely, yes. 
 
But for a backup system, that has tons of RAM and SSD's for cache, and you =
have lots of virtual machines that are based off of the template, or are ve=
ry much the same, then you have a real use-case. I=C2=B4m active at the Fre=
eBSD forums where one person reports
 storing 150TB of data in only 30TB of physical disk. The best practice of =
scrubbing is once a week on "enterprise" systems, though he is only able to=
 do it once a month, because that=C2=B4s how long it takes for a scrub to c=
omplete in that system. So you=C2=B4ve got
 to choose performance or savings, you can=C2=B4t have both. 
 
<blockquote> 
And what about dedup of Netapp? 
</blockquote>
 
Much better implementation, in my opinion. You are able schedule dedup-runs=
 to go at night so your user=C2=B4s performance isn=C2=B4t impacted, and yo=
u get the savings. The question is if you value the savings enough to take =
on price-tag that is NetApp. Or just build
 your own FreeBSD/ZFS server with compression enabled and buy in standard H=
DD's from anywhere... We did;) 
 
/Karli 
 
<blockquote> 
Jose 
 
<hr align=3D"center">
</blockquote>
<blockquote>From: "Karli Sj=C3=B6berg" <<a href=3D"mailto:Karli.Sjoberg@slu.se" targe=
t=3D"_blank">Karli.Sjoberg@slu.se</a>> 
To: <a href=
=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt</=
a> 
Cc: "Jiri Bel=
ka" <<a href=3D"mailto:jbelka@redhat.com" target=3D"_blank">jbelka@redha=
t.com</a>>, <a href=3D"mailto:users@ovirt.org" target=3D"_blank">users@o=
virt.org</a>

<b><font color=3D"#000000">Sent: </font></b><font color=3D"#000000">Quinta-=
feira, 30 de Maio de 2013 8:33:19</font><br>
<b><font color=3D"#000000">Subject: </font></b><font color=3D"#000000">Re: =
[Users] deduplication</font><br>
<br>
<font color=3D"#000000">ons 2013-05-29 klockan 09:59 +0100 skrev <a href=3D=
"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt</a>:
</font><br>
<blockquote><font color=3D"#000000">Absolutely agree with you, planning is =
the best thing to do, but normally people want a plug'n'play system with al=
l included, because there is not much time to think and planning, and there=
 are many companies that know how
 to take advantage of this people characteristics.</font><br>
<font color=3D"#000000">Any way, I think another solution for dedup is Free=
NAS using ZFS.</font><br>
</blockquote>
<br>
<font color=3D"#000000">FreeNAS is just FreeBSD with a fancy web-ui ontop, =
so it=C2=B4s neither more or less of ZFS than you would have otherwise, And=
 regarding dedup in ZFS; Just don=C2=B4t, it=C2=B4s not worth it! It=C2=B4s=
 said that it
</font><font color=3D"#000000"><b>may</b></font><font color=3D"#000000"> in=
crease performance when you have a very suitable usecase, e.g. everything
</font><font color=3D"#000000"><b>exactly</b></font><font color=3D"#000000"=
...
the same over and over. What=C2=B4s not said is that scrubbing and resilv=
ering slows down to a snail (from hundreds of MB/s, or GB if your system is=
 large enough, down to less than 10), just
 from dedup. Also deleting snapshots of datasets that have(or have had) ded=
up on can kill the entire system, and when I say kill, I mean really fubar.=
 Been there, regretted that... Now, compression on the other hand, you get =
basically for free and gives decent
 savings, I highly recommend that.</font><br>
<br>
<font color=3D"#000000">/Karli</font><br>
<br>
<blockquote><br>
<font color=3D"#000000">Jose</font><br>
<br>
<br>
<hr align=3D"center">
<br>
<b><font color=3D"#000000">From: </font></b><font color=3D"#000000">"Jiri B=
elka" <<a href=3D"mailto:jbelka@redhat.com" target=3D"_blank">jbelka@red=
hat.com</a>></font><br>
<b><font color=3D"#000000">To: </font></b><font color=3D"#000000"><a href=
=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt</=
a></font><br>
<b><font color=3D"#000000">Cc: </font></b><font color=3D"#000000"><a href=
=3D"mailto:users@ovirt.org" target=3D"_blank">users@ovirt.org</a></font><br=
<b><font color=3D"#000000">Sent: </font></b><font color=3D"#000000">Quarta-=
feira, 29 de Maio de 2013 7:33:10</font><br>
<b><font color=3D"#000000">Subject: </font></b><font color=3D"#000000">Re: =
[Users] deduplication</font><br>
<br>
<font color=3D"#000000">On Tue, 28 May 2013 14:29:05 +0100 (WEST)</font><br=
...
<font color=3D"#000000"><a href=3D"mailto:suporte@logicworks.pt" target=3D"=
_blank">suporte@logicworks.pt</a> wrote:</font><br>
<br>
<font color=3D"#000000">> That's why I'm making this questions, to demys=
tify some buzzwords around here.</font><br>
<font color=3D"#000000">> But if you have a strong and good technology w=
hy not create buzzwords to get into as many people as possible? without tra=
pped them.</font><br>
<font color=3D"#000000">> Share a disk containing "static" data is a goo=
d idea, do you know from where I can start?</font><br>
<br>
<font color=3D"#000000">Everything depends on your needs, design planning. =
Maybe then sharing</font><br>
<font color=3D"#000000">disk would be better to share via NFS/iscsi. Of cou=
rse if you have many</font><br>
<font color=3D"#000000">VMs each of them is different you will fail. But if=
 you have mostly</font><br>
<font color=3D"#000000">homogeneous environment you can think about this ap=
proach. Sure you have</font><br>
<font color=3D"#000000">to have plan for upgrading "base" "static" shared O=
S data, you have to</font><br>
<font color=3D"#000000">have plan how to install additional software (diffe=
rent destination</font><br>
<font color=3D"#000000">than /usr or /usr/local)... If you already have you=
r own build host</font><br>
<font color=3D"#000000">which builds for you OS packages and you have alrea=
dy your own plan for</font><br>
<font color=3D"#000000">deployment, you have done first steps. If you depen=
d on upgrading each</font><br>
<font color=3D"#000000">machine separately from Internet, then first you sh=
ould plan your</font><br>
<font color=3D"#000000">environment, configuration management etc.</font><b=
r>
<br>
<font color=3D"#000000">Well, in many times people do not do any planning, =
they just think some</font><br>
<font color=3D"#000000">good technology would save their "poor" design.</fo=
nt><br>
<br>
<font color=3D"#000000">j.</font><br>
<br>
<br>
<br>
</blockquote>
<br>
<table cellpadding=3D"0" cellspacing=3D"0" width=3D"100%">
<tbody>
<tr>
<td>-- <br>
<br>
Med V=C3=A4nliga H=C3=A4lsningar<br>
---------------------------------------------------------------------------=
----<br>
Karli Sj=C3=B6berg<br>
Swedish University of Agricultural Sciences<br>
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)<br>
S-750 07 Uppsala, Sweden<br>
Phone:  <a href=3D"tel:%2B46-%280%2918-67%2015%2066" target=3D"_blank"=
...
+46-(0)18-67 15 66</a><br>
<a href=3D"mailto:karli.sjoberg@adm.slu.se" target=3D"_blank">karli.sjoberg=
@slu.se</a> </td>
</tr>
</tbody>
</table>
</blockquote>
<blockquote><br>
<br>
</blockquote>
<br>
<table cellpadding=3D"0" cellspacing=3D"0" width=3D"100%">
<tbody>
<tr>
<td>-- <br>
<br>
Med V=C3=A4nliga H=C3=A4lsningar<br>
---------------------------------------------------------------------------=
----<br>
Karli Sj=C3=B6berg<br>
Swedish University of Agricultural Sciences<br>
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)<br>
S-750 07 Uppsala, Sweden<br>
Phone:  <a href=3D"tel:%2B46-%280%2918-67%2015%2066" target=3D"_blank"=
+46-(0)18-67 15 66</a><br>
<a href=3D"mailto:karli.sjoberg@adm.slu.se" target=3D"_blank">karli.sjoberg=
@slu.se</a> </td>
</tr>
</tbody>
</table>
</div></div></div><br></div></div><br>_____________________________________=
__________<br>
Users mailing list<br>
<a href=3D"mailto:Users@ovirt.org" target=3D"_blank">Users@ovirt.org</a><br=
<a href=3D"http://lists.ovirt.org/mailman/listinfo/users" target=3D"_blank"=
...
http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>
</div><br></div></div><br>_______________________________________________<b=
r>
Users mailing list<br>
<a href=3D"mailto:Users@ovirt.org" target=3D"_blank">Users@ovirt.org</a><br=
<a href=3D"http://lists.ovirt.org/mailman/listinfo/users" target=3D"_blank"=
...
http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>Chris No=
ffsinger<br>
</div><br></div></body></html>
------=_Part_1980_16023178.1370280384707--

Re: [Users] deduplication

suporte＠logicworks.pt