Re: [Users] deduplication

4 Jun 2013

      ------=_Part_237_16733285.1370360396749
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi Juanjo,=20

Very useful, thanks a lot.You save a medium of 60%.=20

Thanks=20

----- Original Message -----

From: "Juan Jose" <jj197005@gmail.com>=20
To: suporte@logicworks.pt, users@ovirt.org=20
Sent: Ter=C3=A7a-feira, 4 de Junho de 2013 12:39:02=20
Subject: Re: [Users] deduplication=20

Hello Jose,=20

We are using only with iSCSI and as I have said it's working very well. We =
have three of this virtual servers that have the compression "lzjb" activat=
ed. The ratios are:=20

Little server with Dovecot mail server has 21 GB and it has 8,6 GB real, sa=
ve 60% space=20
Little server with many log files has 11GB and it's using 2,2 GB real, save=
 80% space=20
Little Windows Server OS for FlexLM and other software license control with=
 15 GB and it's using 9,2 GB real, save 40% space=20

I hope this data are useful to you.=20

Thanks,=20

Juanjo.=20

On Mon, Jun 3, 2013 at 6:12 PM, < suporte@logicworks.pt > wrote:=20

Hi Juan,=20

thanks for your info, I'll try to test FreeNAS with compression. Do you use=
 it with iSCSI or NFS?=20

Jose=20

From: "Juan Jose" < jj197005@gmail.com >=20
To: suporte@logicworks.pt , users@ovirt.org=20
Sent: Segunda-feira, 3 de Junho de 2013 13:37:21=20
Subject: Re: [Users] deduplication=20

Hello Jose,=20

We also have FreeNAS working in our infraestructure, with about 3 TB and ZF=
S. Some of the pools has compression enabled and you can save space with it=
. We have this FreeNAS connected to a hypervisor Xen and it works very well=
 and it's stable and sure. We have nine virtual servers some wirtualized an=
d other paravirtualized, and some Windows Server machine all about 2 years =
in production without any problem. My idea is connect this infrastructure w=
ith oVirt wo be able to have some resources for test VMs in that. Only want=
ed to share as another FreeNas success experience.=20

Juanjo.=20

On Fri, May 31, 2013 at 12:33 PM, < suporte@logicworks.pt > wrote:=20

<blockquote>

Thanks a lot Karli, you make my mind clear about deduplication, once again =
we cannot have the best of both worlds.=20

I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, running o=
n Linux, has no better performance but supports DRDB.=20

Jose=20

From: "Karli Sj=C3=B6berg" < Karli.Sjoberg@slu.se >=20
To: suporte@logicworks.pt=20
Cc: "Jiri Belka" < jbelka@redhat.com >, users@ovirt.org=20
Sent: Sexta-feira, 31 de Maio de 2013 10:45:41=20
Subject: Re: [Users] deduplication=20

fre 2013-05-31 klockan 09:50 +0100 skrev suporte@logicworks.pt :=20
<blockquote>
So, we can say that dedup has more disadvantages than advantages.=20

For a primary system; most definitely, yes.=20

But for a backup system, that has tons of RAM and SSD's for cache, and you =
have lots of virtual machines that are based off of the template, or are ve=
ry much the same, then you have a real use-case. I=C2=B4m active at the Fre=
eBSD forums where one person reports storing 150TB of data in only 30TB of =
physical disk. The best practice of scrubbing is once a week on "enterprise=
" systems, though he is only able to do it once a month, because that=C2=B4=
s how long it takes for a scrub to complete in that system. So you=C2=B4ve =
got to choose performance or savings, you can=C2=B4t have both.=20

<blockquote>

And what about dedup of Netapp?=20

</blockquote>

Much better implementation, in my opinion. You are able schedule dedup-runs=
 to go at night so your user=C2=B4s performance isn=C2=B4t impacted, and yo=
u get the savings. The question is if you value the savings enough to take =
on price-tag that is NetApp. Or just build your own FreeBSD/ZFS server with=
 compression enabled and buy in standard HDD's from anywhere... We did;)=20

/Karli=20

<blockquote>

Jose=20

</blockquote>

<blockquote>
From: "Karli Sj=C3=B6berg" < Karli.Sjoberg@slu.se >=20
To: suporte@logicworks.pt=20
Cc: "Jiri Belka" < jbelka@redhat.com >, users@ovirt.org=20
Sent: Quinta-feira, 30 de Maio de 2013 8:33:19=20
Subject: Re: [Users] deduplication=20

ons 2013-05-29 klockan 09:59 +0100 skrev suporte@logicworks.pt :=20

<blockquote>
Absolutely agree with you, planning is the best thing to do, but normally p=
eople want a plug'n'play system with all included, because there is not muc=
h time to think and planning, and there are many companies that know how to=
 take advantage of this people characteristics.=20
Any way, I think another solution for dedup is FreeNAS using ZFS.=20

</blockquote>

FreeNAS is just FreeBSD with a fancy web-ui ontop, so it=C2=B4s neither mor=
e or less of ZFS than you would have otherwise, And regarding dedup in ZFS;=
 Just don=C2=B4t, it=C2=B4s not worth it! It=C2=B4s said that it may increa=
se performance when you have a very suitable usecase, e.g. everything exact=
ly the same over and over. What=C2=B4s not said is that scrubbing and resil=
vering slows down to a snail (from hundreds of MB/s, or GB if your system i=
s large enough, down to less than 10), just from dedup. Also deleting snaps=
hots of datasets that have(or have had) dedup on can kill the entire system=
, and when I say kill, I mean really fubar. Been there, regretted that... N=
ow, compression on the other hand, you get basically for free and gives dec=
ent savings, I highly recommend that.=20

/Karli=20

<blockquote>

Jose=20

From: "Jiri Belka" < jbelka@redhat.com >=20
To: suporte@logicworks.pt=20
Cc: users@ovirt.org=20
Sent: Quarta-feira, 29 de Maio de 2013 7:33:10=20
Subject: Re: [Users] deduplication=20

On Tue, 28 May 2013 14:29:05 +0100 (WEST)=20
suporte@logicworks.pt wrote:=20
...
That's why I'm making this questions, to demystify some buzzwords around =
here.=20
But if you have a strong and good technology why not create buzzwords to =
get into as many people as possible? without trapped them.=20
Share a disk containing "static" data is a good idea, do you know from wh=
ere I can start?=20
...
<div>Thanks,</div><div><br></div><div>Juanjo.</div></div><div class=3D"gma=
il_extra"><br><br><div class=3D"gmail_quote">On Mon, Jun 3, 2013 at 6:12 PM=
,  <span dir=3D"ltr"><<a href=3D"mailto:suporte@logicworks.pt" target=3D=
"_blank">suporte@logicworks.pt</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; borde=
r-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div><div style=
=3D"font-size: 10pt; font-family: arial,helvetica,sans-serif;">Hi Juan,<br>=
<br>thanks for your info, I'll try to test FreeNAS with compression. Do you=
 use it with iSCSI or NFS?<br>
<br>Jose<br><br><hr><div style=3D"font-size: 12pt; font-style: normal; font=
-family: Helvetica,Arial,sans-serif; text-decoration: none; font-weight: no=
rmal;"><b>From: </b>"Juan Jose" <<a href=3D"mailto:jj197005@gmail.com" t=
arget=3D"_blank">jj197005@gmail.com</a>><br>
<b>To: </b><a href=3D"mailto:suporte@logicworks.pt" target=3D"_blank">supor=
te@logicworks.pt</a>, <a href=3D"mailto:users@ovirt.org" target=3D"_blank">=
users@ovirt.org</a><br><b>Sent: </b>Segunda-feira, 3 de Junho de 2013 13:37=
:21<br>
<b>Subject: </b>Re: [Users] deduplication<div><div class=3D"h5"><br><br><di=
v dir=3D"ltr"><br><div>Hello Jose,</div><div><br></div><div>We also have Fr=
eeNAS working in our infraestructure, with about 3 TB and ZFS. Some of the =
Everything depends on your needs, design planning. Maybe then sharing=20
disk would be better to share via NFS/iscsi. Of course if you have many=20
VMs each of them is different you will fail. But if you have mostly=20
homogeneous environment you can think about this approach. Sure you have=20
to have plan for upgrading "base" "static" shared OS data, you have to=20
have plan how to install additional software (different destination=20
than /usr or /usr/local)... If you already have your own build host=20
which builds for you OS packages and you have already your own plan for=20
deployment, you have done first steps. If you depend on upgrading each=20
machine separately from Internet, then first you should plan your=20
environment, configuration management etc.=20

Well, in many times people do not do any planning, they just think some=20
good technology would save their "poor" design.=20

j.=20

</blockquote>

=09--=20

Med V=C3=A4nliga H=C3=A4lsningar=20
---------------------------------------------------------------------------=
----=20
Karli Sj=C3=B6berg=20
Swedish University of Agricultural Sciences=20
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)=20
S-750 07 Uppsala, Sweden=20
Phone: +46-(0)18-67 15 66=20
karli.sjoberg@slu.se=20
</blockquote>

<blockquote>

</blockquote>

=09--=20

Med V=C3=A4nliga H=C3=A4lsningar=20
---------------------------------------------------------------------------=
----=20
Karli Sj=C3=B6berg=20
Swedish University of Agricultural Sciences=20
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)=20
S-750 07 Uppsala, Sweden=20
Phone: +46-(0)18-67 15 66=20
karli.sjoberg@slu.se=20

_______________________________________________=20
Users mailing list=20
Users@ovirt.org=20
http://lists.ovirt.org/mailman/listinfo/users=20

</blockquote>

</blockquote>

------=_Part_237_16733285.1370360396749
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><style type=3D'text/css'>p { margin: 0; }</style></head><body><=
div style=3D'font-family: arial,helvetica,sans-serif; font-size: 10pt; colo=
r: #000000'>Hi Juanjo,<br><br>Very useful, thanks a lot.You save a medium o=
f 60%. <br><br>Thanks <br><br><hr id=3D"zwchr"><div style=3D"color: rgb(0, =
0, 0); font-weight: normal; font-style: normal; text-decoration: none; font=
-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Juan J=
ose" <jj197005@gmail.com><br><b>To: </b>suporte@logicworks.pt, users@=
ovirt.org<br><b>Sent: </b>Ter=C3=A7a-feira, 4 de Junho de 2013 12:39:02<br>=
<b>Subject: </b>Re: [Users] deduplication<br><br><div dir=3D"ltr"><br><div>=
Hello Jose,</div><div><br></div><div>We are using only with iSCSI and as I =
have said it's working very well. We have three of this virtual servers tha=
t have the compression "lzjb" activated. The ratios are:</div>
<div><br></div><div>Little server with Dovecot mail server has 21 GB and it=
 has 8,6 GB real, save 60% space</div><div>Little server with many log file=
s has 11GB and it's using 2,2 GB real,  save 80% space</div><div>Littl=
e Windows Server OS for FlexLM and other software license control with 15 G=
B and it's using 9,2 GB real, save 40% space</div>
<div><br></div><div>I hope this data are useful to you.</div><div><br></div=
pools has compression enabled and you can save space with it. We have this =
FreeNAS connected to a hypervisor Xen and it works very well and it's stabl=
e and sure. We have nine virtual servers some wirtualized and other paravir=
tualized, and some Windows Server machine all about 2 years in production w=
ithout any problem. My idea is connect this infrastructure with oVirt wo be=
 able to have some resources for test VMs in that. Only wanted to share as =
another FreeNas success experience.</div>

<div><br></div><div>Juanjo.</div></div><div class=3D"gmail_extra"><br><br><=
div class=3D"gmail_quote">On Fri, May 31, 2013 at 12:33 PM,  <span dir=3D"l=
tr"><<a href=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@=
logicworks.pt</a>></span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0.8ex; borde=
r-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div><div style=
=3D"font-size: 10pt; font-family: arial,helvetica,sans-serif;">Thanks a lot=
 Karli, you make my mind clear about deduplication, once again we cannot ha=
ve the best of both worlds.<br>

I'll try FreeNAS despite my poor knowledge on FreeBSD. Openfiler, runni=
ng on Linux, has no better performance but supports DRDB. Jose <b=
r><hr><div style=3D"font-size: 12pt; font-style: normal; font-family: Helve=
tica,Arial,sans-serif; text-decoration: none; font-weight: normal;">

<div><b>From: </b>"Karli Sj=C3=B6berg" <<a href=3D"mailto:Karli.Sjoberg@=
slu.se" target=3D"_blank">Karli.Sjoberg@slu.se</a>><br><b>To: </b><a hre=
f=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt<=
/a><br>

<b>Cc: </b>"Jiri Belka" <<a href=3D"mailto:jbelka@redhat.com" target=3D"=
_blank">jbelka@redhat.com</a>>, <a href=3D"mailto:users@ovirt.org" targe=
t=3D"_blank">users@ovirt.org</a><br></div><b>Sent: </b>Sexta-feira, 31 de M=
aio de 2013 10:45:41<br>

<b>Subject: </b>Re: [Users] deduplication<div><div><br><br>

fre 2013-05-31 klockan 09:50 +0100 skrev <a href=3D"mailto:suporte@logicwor=
ks.pt" target=3D"_blank">suporte@logicworks.pt</a>:
<blockquote>So, we can say that dedup has more disa=
dvantages than advantages. 
</blockquote>
 
For a primary system; most definitely, yes. 
 
But for a backup system, that has tons of RAM and SSD's for cache, and you =
have lots of virtual machines that are based off of the template, or are ve=
ry much the same, then you have a real use-case. I=C2=B4m active at the Fre=
eBSD forums where one person reports
 storing 150TB of data in only 30TB of physical disk. The best practice of =
scrubbing is once a week on "enterprise" systems, though he is only able to=
 do it once a month, because that=C2=B4s how long it takes for a scrub to c=
omplete in that system. So you=C2=B4ve got
 to choose performance or savings, you can=C2=B4t have both. 
 
<blockquote> 
And what about dedup of Netapp? 
</blockquote>
 
Much better implementation, in my opinion. You are able schedule dedup-runs=
 to go at night so your user=C2=B4s performance isn=C2=B4t impacted, and yo=
u get the savings. The question is if you value the savings enough to take =
on price-tag that is NetApp. Or just build
 your own FreeBSD/ZFS server with compression enabled and buy in standard H=
DD's from anywhere... We did;) 
 
/Karli 
 
<blockquote> 
Jose 
 
<hr align=3D"center">
</blockquote>
<blockquote>From: "Karli Sj=C3=B6berg" <<a href=3D"mailto:Karli.Sjoberg@slu.se" targe=
t=3D"_blank">Karli.Sjoberg@slu.se</a>> 
To: <a href=
=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt</=
a> 
Cc: "Jiri Bel=
ka" <<a href=3D"mailto:jbelka@redhat.com" target=3D"_blank">jbelka@redha=
t.com</a>>, <a href=3D"mailto:users@ovirt.org" target=3D"_blank">users@o=
virt.org</a>

<b><font color=3D"#000000">Sent: </font></b><font color=3D"#000000">Quinta-=
feira, 30 de Maio de 2013 8:33:19</font><br>
<b><font color=3D"#000000">Subject: </font></b><font color=3D"#000000">Re: =
[Users] deduplication</font><br>
<br>
<font color=3D"#000000">ons 2013-05-29 klockan 09:59 +0100 skrev <a href=3D=
"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt</a>:
</font><br>
<blockquote><font color=3D"#000000">Absolutely agree with you, planning is =
the best thing to do, but normally people want a plug'n'play system with al=
l included, because there is not much time to think and planning, and there=
 are many companies that know how
 to take advantage of this people characteristics.</font><br>
<font color=3D"#000000">Any way, I think another solution for dedup is Free=
NAS using ZFS.</font><br>
</blockquote>
<br>
<font color=3D"#000000">FreeNAS is just FreeBSD with a fancy web-ui ontop, =
so it=C2=B4s neither more or less of ZFS than you would have otherwise, And=
 regarding dedup in ZFS; Just don=C2=B4t, it=C2=B4s not worth it! It=C2=B4s=
 said that it
</font><font color=3D"#000000"><b>may</b></font><font color=3D"#000000"> in=
crease performance when you have a very suitable usecase, e.g. everything
</font><font color=3D"#000000"><b>exactly</b></font><font color=3D"#000000"=
...
the same over and over. What=C2=B4s not said is that scrubbing and resilv=
ering slows down to a snail (from hundreds of MB/s, or GB if your system is=
 large enough, down to less than 10), just
 from dedup. Also deleting snapshots of datasets that have(or have had) ded=
up on can kill the entire system, and when I say kill, I mean really fubar.=
 Been there, regretted that... Now, compression on the other hand, you get =
basically for free and gives decent
 savings, I highly recommend that.</font><br>
<br>
<font color=3D"#000000">/Karli</font><br>
<br>
<blockquote><br>
<font color=3D"#000000">Jose</font><br>
<br>
<br>
<hr align=3D"center">
<br>
<b><font color=3D"#000000">From: </font></b><font color=3D"#000000">"Jiri B=
elka" <<a href=3D"mailto:jbelka@redhat.com" target=3D"_blank">jbelka@red=
hat.com</a>></font><br>
<b><font color=3D"#000000">To: </font></b><font color=3D"#000000"><a href=
=3D"mailto:suporte@logicworks.pt" target=3D"_blank">suporte@logicworks.pt</=
a></font><br>
<b><font color=3D"#000000">Cc: </font></b><font color=3D"#000000"><a href=
=3D"mailto:users@ovirt.org" target=3D"_blank">users@ovirt.org</a></font><br=
<b><font color=3D"#000000">Sent: </font></b><font color=3D"#000000">Quarta-=
feira, 29 de Maio de 2013 7:33:10</font><br>
<b><font color=3D"#000000">Subject: </font></b><font color=3D"#000000">Re: =
[Users] deduplication</font><br>
<br>
<font color=3D"#000000">On Tue, 28 May 2013 14:29:05 +0100 (WEST)</font><br=
...
<font color=3D"#000000"><a href=3D"mailto:suporte@logicworks.pt" target=3D"=
_blank">suporte@logicworks.pt</a> wrote:</font><br>
<br>
<font color=3D"#000000">> That's why I'm making this questions, to demys=
tify some buzzwords around here.</font><br>
<font color=3D"#000000">> But if you have a strong and good technology w=
hy not create buzzwords to get into as many people as possible? without tra=
pped them.</font><br>
<font color=3D"#000000">> Share a disk containing "static" data is a goo=
d idea, do you know from where I can start?</font><br>
<br>
<font color=3D"#000000">Everything depends on your needs, design planning. =
Maybe then sharing</font><br>
<font color=3D"#000000">disk would be better to share via NFS/iscsi. Of cou=
rse if you have many</font><br>
<font color=3D"#000000">VMs each of them is different you will fail. But if=
 you have mostly</font><br>
<font color=3D"#000000">homogeneous environment you can think about this ap=
proach. Sure you have</font><br>
<font color=3D"#000000">to have plan for upgrading "base" "static" shared O=
S data, you have to</font><br>
<font color=3D"#000000">have plan how to install additional software (diffe=
rent destination</font><br>
<font color=3D"#000000">than /usr or /usr/local)... If you already have you=
r own build host</font><br>
<font color=3D"#000000">which builds for you OS packages and you have alrea=
dy your own plan for</font><br>
<font color=3D"#000000">deployment, you have done first steps. If you depen=
d on upgrading each</font><br>
<font color=3D"#000000">machine separately from Internet, then first you sh=
ould plan your</font><br>
<font color=3D"#000000">environment, configuration management etc.</font><b=
r>
<br>
<font color=3D"#000000">Well, in many times people do not do any planning, =
they just think some</font><br>
<font color=3D"#000000">good technology would save their "poor" design.</fo=
nt><br>
<br>
<font color=3D"#000000">j.</font><br>
<br>
<br>
<br>
</blockquote>
<br>
<table cellpadding=3D"0" cellspacing=3D"0" width=3D"100%">
<tbody>
<tr>
<td>-- <br>
<br>
Med V=C3=A4nliga H=C3=A4lsningar<br>
---------------------------------------------------------------------------=
----<br>
Karli Sj=C3=B6berg<br>
Swedish University of Agricultural Sciences<br>
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)<br>
S-750 07 Uppsala, Sweden<br>
Phone:  +46-(0)18-67 15 66<br>
<a href=3D"mailto:karli.sjoberg@adm.slu.se" target=3D"_blank">karli.sjoberg=
@slu.se</a> </td>
</tr>
</tbody>
</table>
</blockquote>
<blockquote><br>
<br>
</blockquote>
<br>
<table cellpadding=3D"0" cellspacing=3D"0" width=3D"100%">
<tbody>
<tr>
<td>-- <br>
<br>
Med V=C3=A4nliga H=C3=A4lsningar<br>
---------------------------------------------------------------------------=
----<br>
Karli Sj=C3=B6berg<br>
Swedish University of Agricultural Sciences<br>
Box 7079 (Visiting Address Kron=C3=A5sv=C3=A4gen 8)<br>
S-750 07 Uppsala, Sweden<br>
Phone:  +46-(0)18-67 15 66<br>
<a href=3D"mailto:karli.sjoberg@adm.slu.se" target=3D"_blank">karli.sjoberg=
@slu.se</a> </td>
</tr>
</tbody>
</table>
</div></div></div><br></div></div><br>_____________________________________=
__________<br>
Users mailing list<br>
<a href=3D"mailto:Users@ovirt.org" target=3D"_blank">Users@ovirt.org</a><br=
...
<a href=3D"http://lists.ovirt.org/mailman/listinfo/users" target=3D"_blank"=
...
http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>
</div></div></div><br></div></div></blockquote></div><br></div>
</div><br></div></body></html>
------=_Part_237_16733285.1370360396749--

Re: [Users] deduplication

suporte＠logicworks.pt