6:24 a.m.
This is a multi-part message in MIME format.
------------15dc15fdd7045fa2768c750b77
Content-Type: text/plain; format=flowed; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On ovirt gluster uses sharding. So all large files are broken up in small=
=20
pieces on the gluster bricks.
/Johan
On August 8, 2017 12:19:39 Moacir Ferreira <moacirferreira(a)hotmail.com> w=
rote:
Thanks Johan, you brought "light" into my darkness! I went
looking for =
the=20
GlusterFS tiering how-to and it looks like quite simple to attach a
SSD=
as=20
hot tier. For those willing to read about it, go here:=20
http://blog.gluster.org/2016/03/automated-tiering-in-gluster/
Now, I still have a question: VMs are made of very large .qcow2 files. =
My=20
understanding is that files in Gluster are kept all together in a
singl=
e=20
brick. If so, I will not benefit from tiering as a single SSD will
not =
be=20
big enough to fit all my large VM .qcow2 files. This would not be
true =
if=20
Gluster can store "blocks" of data that compose a large
file spread on=20
several bricks. But if I am not wrong, this is one of key differences i=
n=20
between GlusterFS and Ceph. Can you comment?
Moacir
________________________________
From: Johan Bernhardsson <johan(a)kafit.se>
Sent: Tuesday, August 8, 2017 7:03 AM
To: Moacir Ferreira; Devin Acosta; users(a)ovirt.org
Subject: Re: [ovirt-users] Good practices
You attach the ssd as a hot tier with a gluster command. I don't think =
that=20
gdeploy or ovirt gui can do it.
The gluster docs and redhat docs explains tiering quite good.
/Johan
On August 8, 2017 07:06:42 Moacir Ferreira <moacirferreira(a)hotmail.com>=
wrote:
Hi Devin,
Please consider that for the OS I have a RAID 1. Now, lets say I use RA=
ID 5=20
to assemble a single disk on each server. In this case, the SSD will
no=
t=20
make any difference, right? I guess that to be possible to use it,
the =
SSD=20
should not be part of the RAID 5. In this case I could create a
logical=
=20
volume made of the RAIDed brick and then extend it using the SSD.
I.e.:=
=20
Using gdeploy:
[disktype]
jbod
....
[pv1]
action=3Dcreate
devices=3Dsdb, sdc
wipefs=3Dyes
ignore_vg_erros=3Dno
[vg1]
action=3Dcreate
vgname=3Dgluster_vg_jbod
pvname=3Dsdb
ignore_vg_erros=3Dno
[vg2]
action=3Dextend
vgname=3Dgluster_vg_jbod
pvname=3Dsdc
ignore_vg_erros=3Dno
But will Gluster be able to auto-detect and use this SSD brick for tier=
ing?=20
Do I have to do some other configurations? Also, as the VM files
(.qcow=
2)=20
are quite big will I benefit from tiering? This is wrong and my
approac=
h=20
should be other?
Thanks,
Moacir
________________________________
From: Devin Acosta <devin(a)pabstatencio.com>
Sent: Monday, August 7, 2017 7:46 AM
To: Moacir Ferreira; users(a)ovirt.org
Subject: Re: [ovirt-users] Good practices
Moacir,
I have recently installed multiple Red Hat Virtualization hosts for sev=
eral=20
different companies, and have dealt with the Red Hat Support Team in
de=
pth=20
about optimal configuration in regards to setting up GlusterFS
most=20
efficiently and I wanted to share with you what I learned.
In general Red Hat Virtualization team frowns upon using each DISK of t=
he=20
system as just a JBOD, sure there is some protection by having the
data=
=20
replicated, however, the recommendation is to use RAID 6 (preferred)
or=
=20
RAID-5, or at least RAID-1 at the very least.
Here is the direct quote from Red Hat when I asked about RAID and Brick=
s:
"A typical Gluster configuration would use RAID underneath the bricks. =
RAID=20
6 is most typical as it gives you 2 disk failure protection, but RAID
5=
=20
could be used too. Once you have the RAIDed bricks, you'd then
apply th=
e=20
desired replication on top of that. The most popular way of doing
this=20
would be distributed replicated with 2x replication. In general you'll =
get=20
better performance with larger bricks. 12 drives is often a sweet
spot.=
=20
Another option would be to create a separate tier using all
SSD=E2=80=99=
s.=E2=80=9D
In order to SSD tiering from my understanding you would need 1 x NVMe d=
rive=20
in each server, or 4 x SSD hot tier (it needs to be distributed,
replic=
ated=20
for the hot tier if not using NVME). So with you only having 1 SSD
driv=
e in=20
each server, I=E2=80=99d suggest maybe looking into the NVME option.
Since your using only 3-servers, what I=E2=80=99d probably suggest is t=
o do
(2=20
Replicas + Arbiter Node), this setup actually doesn=E2=80=99t require
t=
he 3rd=20
server to have big drives at all as it only stores meta-data about
the=20
files and not actually a full copy.
Please see the attached document that was given to me by Red Hat to get=
=20
more information on this. Hope this information helps you.
--
Devin Acosta, RHCA, RHVCA
Red Hat Certified Architect
On August 6, 2017 at 7:29:29 PM, Moacir Ferreira=20
(moacirferreira@hotmail.com<mailto:moacirferreira@hotmail.com>) wrote:
I am willing to assemble a oVirt "pod", made of 3 servers, each with 2 =
CPU=20
sockets of 12 cores, 256GB RAM, 7 HDD 10K, 1 SSD. The idea is to
use=20
GlusterFS to provide HA for the VMs. The 3 servers have a dual 40Gb NIC=
and=20
a dual 10Gb NIC. So my intention is to create a loop like a server
tria=
ngle=20
using the 40Gb NICs for virtualization files (VMs .qcow2) access and
to=
=20
move VMs around the pod (east /west traffic) while using the 10Gb=20
interfaces for giving services to the outside world (north/south traffi=
c).
This said, my first question is: How should I deploy GlusterFS in such=20
oVirt scenario? My questions are:
1 - Should I create 3 RAID (i.e.: RAID 5), one on each oVirt node, and =
then=20
create a GlusterFS using them?
2 - Instead, should I create a JBOD array made of all server's disks?
3 - What is the best Gluster configuration to provide for HA while not=20
consuming too much disk space?
4 - Does a oVirt hypervisor pod like I am planning to build, and the=20
virtualization environment, benefits from tiering when using a SSD disk=
?=20
And yes, will Gluster do it by default or I have to configure it to
do =
so?
At the bottom line, what is the good practice for using GlusterFS in sm=
all=20
pods for enterprises?
You opinion/feedback will be really appreciated!
Moacir
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users%40ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
------------15dc15fdd7045fa2768c750b77
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<style type=3D"text/css" style=3D"display:none;"><!-- P
{margin-top:0;margin-bottom:0;} --></style>
</head>
<body>
<div style=3D"color: black;">
<div style=3D"color: black;">
<p style=3D"margin: 0 0 1em 0; color: black;">On ovirt gluster uses
shard=
ing.
So all large files are broken up in small pieces on the gluster bricks.</=
p>
<p style=3D"margin: 0 0 1em 0; color: black;">/Johan</p>
</div>
<div style=3D"color: black;">
<p
style=3D"color: black; font-size: 10pt; font-family: Arial, sans-serif; m=
argin: 10pt 0;">On
August 8, 2017 12:19:39 Moacir Ferreira &lt;moacirferreira(a)hotmail.com&gt=
;
wrote:</p>
<blockquote type=3D"cite" class=3D"gmail_quote"
style=3D"margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-le=
ft: 0.75ex;">
<div id=3D"divtagdefaultwrapper"
style=3D"font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-=
serif;"
dir=3D"ltr">
<p>Thanks Johan, you brought "light" into my darkness! I went
looking for the GlusterFS tiering how-to and it looks like quite simple t=
o
attach a SSD as hot tier. For those willing to read about it, go here:
<a href=3D"http://blog.gluster.org/2016/03/automated-tiering-in-gluster...
class=3D"OWAAutoLink" id=3D"LPlnk939343"
previewremoved=3D"true">
http://blog.gluster.org/2016/03/automated-tiering-in-gluster/</a>&l...
<p><br>
</p>
<p>Now, I still have a question: VMs are made of very large .qcow2 files.
My understanding is that files in Gluster are kept all together in a sing=
le
brick. If so, I will not benefit from tiering as a single SSD will not be
big enough to fit all my large VM
.qcow2 files. This would not be true if Gluster can store
"blocks" of data that compose a large file spread on several
bricks. But if I am not wrong, this is one of key differences in between
GlusterFS and Ceph. Can you comment?</p>
<p><br>
</p>
<p>Moacir<br>
</p>
<br>
<br>
<div style=3D"color: rgb(49, 55, 57);">
<hr tabindex=3D"-1" style=3D"display:inline-block; width:98%">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font
style=3D"font-size:11pt"
face=3D"Calibri, sans-serif"
color=3D"#000000"><b>From:</b> Johan Bernhar=
dsson
&lt;johan(a)kafit.se&gt;<br>
<b>Sent:</b> Tuesday, August 8, 2017 7:03 AM<br>
<b>To:</b> Moacir Ferreira; Devin Acosta; users(a)ovirt.org<br>
<b>Subject:</b> Re: [ovirt-users] Good practices</font>
<div> </div>
</div>
<div>
<div style=3D"color:black">
<div style=3D"color:black">
<p style=3D"margin:0 0 1em 0; color:black">You attach the ssd as a hot
ti=
er
with a gluster command. I don't think that gdeploy or ovirt gui can do it=
.</p>
<p style=3D"margin:0 0 1em 0; color:black">The gluster docs and redhat
do=
cs
explains tiering quite good.</p>
<p style=3D"margin:0 0 1em 0; color:black">/Johan</p>
</div>
<div style=3D"color:black">
<p
style=3D"color:black; font-size:10pt; font-family:Arial,sans-serif; margi=
n:10pt 0">
On August 8, 2017 07:06:42 Moacir Ferreira
&lt;moacirferreira(a)hotmail.com&gt; wrote:</p>
<blockquote type=3D"cite" class=3D"gmail_quote"
style=3D"margin:0 0 0 0.75ex; border-left:1px solid #808080; padding-left=
:0.75ex">
<div id=3D"divtagdefaultwrapper" dir=3D"ltr"
style=3D"font-size:12pt; color:#000000; font-family:Calibri,Helvetica,san=
s-serif">
<p>Hi Devin,</p>
<p><br>
</p>
<p>Please consider that for the OS I have a RAID 1. Now, lets say I use
RAID 5 to assemble a single disk on each server. In this case, the SSD wi=
ll
not make any difference, right? I guess that to be possible to use it, th=
e
SSD should not be part of the RAID
5. In this case I could create a logical volume made of the RAIDed brick
and then extend it using the SSD. I.e.: Using gdeploy:</p>
<p><br>
</p>
<p>[disktype]</p>
<p>jbod</p>
<p>....</p>
<p>[pv1]</p>
<p>action=3Dcreate</p>
<p>devices=3Dsdb, sdc</p>
<p>wipefs=3Dyes</p>
<p></p>
<p>ignore_vg_erros=3Dno<br>
</p>
<p><br>
</p>
<p></p>
<p>[vg1]</p>
<p><b>action=3Dcreate</b></p>
<p>vgname=3Dgluster_vg_jbod</p>
<p>pvname=3Dsdb</p>
<p>ignore_vg_erros=3Dno<br>
</p>
<p><br>
</p>
<p>[vg2]</p>
<p><b>action=3Dextend</b></p>
<p>vgname=3Dgluster_vg_jbod</p>
<p>pvname=3Dsdc</p>
<p>ignore_vg_erros=3Dno<br>
</p>
<br>
<p>But will Gluster be able to auto-detect and use this SSD brick for
tiering? Do I have to do some other configurations? Also, as the VM files
(.qcow2) are quite big will I benefit from tiering? This is wrong and my
approach should be other?</p>
<p><br>
</p>
<p>Thanks,</p>
<p>Moacir<br>
</p>
<p><br>
</p>
<br>
<div style=3D"color:rgb(49,55,57)">
<hr tabindex=3D"-1" style=3D"display:inline-block; width:98%">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font
style=3D"font-size:11pt"
face=3D"Calibri, sans-serif"
color=3D"#000000"><b>From:</b> Devin Acosta
&lt;devin(a)pabstatencio.com&gt;<br>
<b>Sent:</b> Monday, August 7, 2017 7:46 AM<br>
<b>To:</b> Moacir Ferreira; users(a)ovirt.org<br>
<b>Subject:</b> Re: [ovirt-users] Good practices</font>
<div> </div>
</div>
<div>
<div id=3D"bloop_customfont" style=3D"color:rgb(0,0,0);
margin:0px"><font
face=3D"Input Mono"><br>
</font></div>
<div id=3D"bloop_customfont" style=3D"color:rgb(0,0,0);
margin:0px"><font
face=3D"Input Mono">Moacir,</font></div>
<div id=3D"bloop_customfont" style=3D"color:rgb(0,0,0);
margin:0px"><font
face=3D"Input Mono"><br>
</font></div>
<div id=3D"bloop_customfont" style=3D"color:rgb(0,0,0);
margin:0px"><font
face=3D"Input Mono">I have recently installed multiple Red Hat Virtualiza=
tion
hosts for several different companies, and have dealt with the Red Hat
Support Team in depth about optimal configuration
in regards to setting up GlusterFS most efficiently and I wanted to shar=
e
with you what I learned.</font></div>
<div id=3D"bloop_customfont" style=3D"color:rgb(0,0,0);
margin:0px"><font
face=3D"Input Mono"><br>
</font></div>
<div id=3D"bloop_customfont" style=3D"color:rgb(0,0,0);
margin:0px"><font
face=3D"Input Mono">In general Red Hat Virtualization team frowns upon us=
ing
each DISK of the system as just a JBOD, sure there is some protection by
having the data replicated, however, the
recommendation is to use RAID 6 (preferred) or RAID-5, or at least RAID-=
1
at the very least.</font></div>
<div id=3D"bloop_customfont" style=3D"color:rgb(0,0,0);
margin:0px"><font
face=3D"Input Mono"><br>
</font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mon=
o">Here
is the direct quote from Red Hat when I asked about RAID and
Bricks:</font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mon=
o"><i><br>
</i></font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mono"><i>"A typical Gluster configuration would
use RA=
ID
underneath the bricks. RAID 6 is most typical as it gives you 2 disk
failure protection, but RAID 5 could be used too. Once you have the
RAIDed bricks, you'd then apply the desired replication on top of that.
The most popular way of doing this would be distributed replicated with 2=
x
replication. In general you'll get better performance with larger
bricks. 12 drives is often a sweet spot. Another
option would be to create a separate tier using all
SSD=E2=80=99s.=E2=80=9D </i></font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><br>
</div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mon=
o"><i>In
order to SSD tiering from my understanding you would need 1 x NVMe
drive in each server, or 4 x SSD hot tier (it needs to be distributed,
replicated for the hot tier if not using NVME).
So with you only having 1 SSD drive in each server, I=E2=80=99d sug=
gest maybe
looking into the NVME option. </i></font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mon=
o"><i><br>
</i></font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mono"><i>Since your using only 3-servers, what
I=E2=80=99d =
probably
suggest is to do (2 Replicas + Arbiter Node), this setup actually
doesn=E2=80=99t require the 3rd server to have big drives at all as it on=
ly
stores meta-data about the files and not actually a full
copy. </i></font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mon=
o"><i><br>
</i></font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mono"><i>Please see the attached document that was given
to=
me
by Red Hat to get more information on this. Hope this information helps
you.</i></font></div>
<div id=3D"bloop_customfont" style=3D"margin:0px"><font
face=3D"Input Mon=
o"><i><br>
</i></font></div>
<br>
<div id=3D"bloop_sign_1502087376725469184"
class=3D"bloop_sign"><span
style=3D"font-family:'helvetica Neue',helvetica;
font-size:14px">--</span=
<br
style=3D"font-family:'helvetica
Neue',helvetica; font-size:14px">
<div class=3D"gmail_signature"
style=3D"font-family:'helvetica Neue',helvetica; font-size:14px">
<div dir=3D"ltr">
<div><br>
</div>
<div>Devin Acosta, RHCA, RHVCA</div>
<div>Red Hat Certified Architect</div>
<div></div>
</div>
</div>
</div>
<br>
<p class=3D"airmail_on">On August 6, 2017 at 7:29:29 PM, Moacir Ferreira
=
(<a
href=3D"mailto:moacirferreira@hotmail.com">moacirferreira@hotmail.com</a>=
)
wrote:</p>
<blockquote type=3D"cite" class=3D"clean_bq"><span>
<div dir=3D"ltr">
<div></div>
<div>
<div id=3D"divtagdefaultwrapper" dir=3D"ltr"
style=3D"font-size:12pt; color:#000000; font-family:Calibri,Helvetica,san=
s-serif">
<p><span>I am willing to assemble a oVirt "pod", made of 3
servers, each with 2 CPU sockets of 12 cores, 256GB RAM, 7 HDD 10K, 1 SSD=
.
The idea is to use GlusterFS to provide HA for the VMs. The 3 servers hav=
e
a dual 40Gb NIC and a dual 10Gb NIC. So my intention
is to create a loop like a server triangle using the 40Gb NICs for
virtualization files (VMs .qcow2) access and to move VMs around the pod
(east /west traffic) while using the 10Gb interfaces for giving services =
to
the outside world (north/south traffic).</span></p>
<p><br>
</p>
<p>This said, my first question is: How should I deploy GlusterFS in such
oVirt scenario? My questions are:</p>
<p><br>
</p>
<p>1 - Should I create 3 RAID (i.e.: RAID 5), one on each oVirt node, and
then create a GlusterFS using them?</p>
<p>2 - Instead, should I create a JBOD array made of all server's disks?<=
/p>
<p>3 - What is the best Gluster configuration to provide for HA while not
consuming too much disk space?<br>
</p>
<p>4 - Does a oVirt hypervisor pod like I am planning to build, and the
virtualization environment, benefits from tiering when using a SSD disk?
And yes, will Gluster do it by default or I have to configure it to do so=
?</p>
<p><br>
</p>
<p>At the bottom line, what is the good practice for using GlusterFS in
small pods for enterprises?<br>
</p>
<p><br>
</p>
<p>You opinion/feedback will be really appreciated!</p>
<p>Moacir<br>
</p>
</div>
_______________________________________________ <br>
Users mailing list <br>
<a href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a> <br>
<a
href=3D"http://lists.ovirt.org/mailman/listinfo/users">http:...
.org/mailman/listinfo/users</a>
<br>
</div>
</div>
</span></blockquote>
</div>
</div>
</div>
_______________________________________________<br>
Users mailing list<br>
<a class=3D"aqm-autolink aqm-autowrap"
href=3D"mailto:Users%40ovirt.org">Users@ovirt.org</a><br>
<a class=3D"aqm-autolink aqm-autowrap"
href=3D"http://lists.ovirt.org/mailman/listinfo/users">http:...
.org/mailman/listinfo/users</a><br>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>
------------15dc15fdd7045fa2768c750b77--