8:39 a.m.
Hi,
I am currently testing oVirt 3.4.3 + gluster 3.5.2 for use in my relatively
small home office environment on a single host. I have 2 Intel hosts with
SSD and magnetic disk and one AMD host with only magnetic disk. I have
been trying to figure out the best way to configure my environment given my
previous attempt with oVirt 3.3 encountered storage issues.
I will be hosting two types of VMs - VMs that can be tied to a particular
system (such as 3 node FreeIPA domain or some test VMs), and VMs which
could migrate between systems for improved uptime.
The processor issue seems straightforward. Have a single datacenter with
two clusters - one for the Intel systems and one for the AMD systems. Put
VMs which need to live migrate on the Intel cluster. If necessary VMs can
be manually switched between the Intel and AMD cluster with a downtime.
The Gluster side of the storage seems less clear. The bulk of the gluster
with oVirt issues I experienced and have seen on the list seem to be two
node setups with 2 bricks in the Gluster volume.
So here are my questions:
1) Should I avoid 2 brick Gluster volumes?
2) What is the risk in having the SSD volumes with only 2 bricks given that
there would be 3 gluster servers? How should I configure them?
3) Is there a way to use local storage for a host locked VM other than
creating a gluster volume with one brick?
4) Should I avoid using the hosted engine configuration? I do have an
external VMware ESXi system to host the engine for now but would like to
phase it out eventually.
5) If I do the hosted engine should I make the underlying gluster volume 3
brick replicated?
Thanks in advance for any help you can provide.
-David
5:13 p.m.
--Alternative_=_Boundary_=_1409264035
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
I'll try to answer some of these.<br>1) its not a serious problem persay th=
e issue is if one node goes down and you delete a file while the second nod=
e is down it will be restored when the second node comes back which may cau=
se orphaned files where as if you use 3 servers they will use quorum to fig=
ure out what needs to be restored or deleted. Further more your read and wr=
ite performance may suffer especially in comparison to having 1 replica of =
the file with stripping.<br><br>2) see answer 1 and just create the volume =
with 1 replica and only include the URI for bricks on two of the hosts when=
you create it.<br><br>3) I think so but have never tried it you just have =
to define it as a local storage domain.<br><br>4) well that's a
philosophic=
al question. You can theory have two hosted engines on separate VMs on two =
separate physical boxes but if for any reason they both go down you will "b=
e living in interesting times" (as in the Chinese curse)<br><br><span
style=
=3D"font-family:Prelude, Verdana, san-serif;">5) YES! And have more than on=
e.<br><br></span><span id=3D"signature"><div
style=3D"font-family: arial, s=
ans-serif; font-size: 12px;color: #999999;">-- Sent from my HP
Pre3</div><b=
r></span><span style=3D"color:navy; font-family:Prelude, Verdana,
san-serif=
; "><hr align=3D"left" style=3D"width:75%">On Aug 28,
2014 9:39 AM, David K=
ing &lt;david(a)rexden.us&gt; wrote: <br><br></span><div
dir=3D"ltr">Hi,<div>=
<br></div><div>I am currently testing oVirt 3.4.3 + gluster 3.5.2 for
use i=
n my relatively small home office environment on a single host. =C2=A0I hav=
e 2 =C2=A0Intel hosts with SSD and magnetic disk and one AMD host with only=
magnetic disk. =C2=A0I have been trying to figure out the best way to conf=
igure my environment given my previous attempt with oVirt 3.3 encountered s=
torage issues.</div>
<div><br></div><div>I will be hosting two types of VMs - VMs that
can be ti=
ed to a particular system (such as 3 node FreeIPA domain or some test VMs),=
and VMs which could migrate between systems for improved uptime.</div>
<div><br></div><div>The processor issue seems straightforward.
=C2=A0Have a=
single datacenter with two clusters - one for the Intel systems and one fo=
r the AMD systems. =C2=A0Put VMs which need to live migrate on the Intel cl=
uster. =C2=A0If necessary VMs can be manually switched between the Intel an=
d AMD cluster with a downtime.</div>
<div><br></div><div>The Gluster side of the storage seems less
clear. =C2=
=A0The bulk of the gluster with oVirt issues I experienced and have seen on=
the list seem to be two node setups with 2 bricks in the Gluster volume. =
=C2=A0</div>
<div><br></div><div>So here are my
questions:</div><div><br></div><div>1) S=
hould I avoid 2 brick Gluster volumes?
=C2=A0</div><div><br></div><div>2) W=
hat is the risk in having the SSD volumes with only 2 bricks given that the=
re would be 3 gluster servers? =C2=A0How should I configure them?</div>
<div><br></div><div>3) Is there a way to use local storage for a
host locke=
d VM other than creating a gluster volume with one brick? =C2=A0</div><div>=
<br></div><div>4) Should I avoid using the hosted engine configuration?
=C2=
=A0I do have an external VMware ESXi system to host the engine for now but =
would like to phase it out eventually.</div>
<div><br></div><div>5) If I do the hosted engine should I make the
underlyi=
ng gluster volume 3 brick
replicated?</div><div><br></div><div>Thanks in ad=
vance for any help you can provide.
=C2=A0</div><div><br></div><div>-David<=
/div>
</div>
--Alternative_=_Boundary_=_1409264035--
9:04 a.m.
Paul,
Thanks for the response.
You mention that the issue is orphaned files during updates when one node
is down. However I am less concerned about adding and removing files
because the file server will be predominately VM disks so the file
structure is fairly static. Those VM files will be quite active however -
will gluster be able to keep track of partial updates to a large file when
one out of two bricks are down?
Right now I am leaning towards using SSD for "host local" disk - single
brick gluster volumes intended for VMs which are node specific and then 3
way replicas for the higher availability zones which tend to be more read
oriented. I presume that read-only access only needs to get data from one
of the 3 replicas so that should be reasonably performant.
Thanks,
David
On Thu, Aug 28, 2014 at 6:13 PM, Paul Robert Marino <prmarino1(a)gmail.com>
wrote:
I'll try to answer some of these.
1) its not a serious problem persay the issue is if one node goes down and
you delete a file while the second node is down it will be restored when
the second node comes back which may cause orphaned files where as if you
use 3 servers they will use quorum to figure out what needs to be restored
or deleted. Further more your read and write performance may suffer
especially in comparison to having 1 replica of the file with stripping.
2) see answer 1 and just create the volume with 1 replica and only include
the URI for bricks on two of the hosts when you create it.
3) I think so but have never tried it you just have to define it as a
local storage domain.
4) well that's a philosophical question. You can theory have two hosted
engines on separate VMs on two separate physical boxes but if for any
reason they both go down you will "be living in interesting times" (as in
the Chinese curse)
5) YES! And have more than one.
-- Sent from my HP Pre3
------------------------------
On Aug 28, 2014 9:39 AM, David King <david(a)rexden.us> wrote:
Hi,
I am currently testing oVirt 3.4.3 + gluster 3.5.2 for use in my
relatively small home office environment on a single host. I have 2 Intel
hosts with SSD and magnetic disk and one AMD host with only magnetic disk.
I have been trying to figure out the best way to configure my environment
given my previous attempt with oVirt 3.3 encountered storage issues.
I will be hosting two types of VMs - VMs that can be tied to a particular
system (such as 3 node FreeIPA domain or some test VMs), and VMs which
could migrate between systems for improved uptime.
The processor issue seems straightforward. Have a single datacenter with
two clusters - one for the Intel systems and one for the AMD systems. Put
VMs which need to live migrate on the Intel cluster. If necessary VMs can
be manually switched between the Intel and AMD cluster with a downtime.
The Gluster side of the storage seems less clear. The bulk of the gluster
with oVirt issues I experienced and have seen on the list seem to be two
node setups with 2 bricks in the Gluster volume.
So here are my questions:
1) Should I avoid 2 brick Gluster volumes?
2) What is the risk in having the SSD volumes with only 2 bricks given
that there would be 3 gluster servers? How should I configure them?
3) Is there a way to use local storage for a host locked VM other than
creating a gluster volume with one brick?
4) Should I avoid using the hosted engine configuration? I do have an
external VMware ESXi system to host the engine for now but would like to
phase it out eventually.
5) If I do the hosted engine should I make the underlying gluster volume 3
brick replicated?
Thanks in advance for any help you can provide.
-David
11:25 a.m.
On 08/29/2014 07:34 PM, David King wrote:
Paul,
Thanks for the response.
You mention that the issue is orphaned files during updates when one
node is down. However I am less concerned about adding and removing
files because the file server will be predominately VM disks so the file
structure is fairly static. Those VM files will be quite active however
- will gluster be able to keep track of partial updates to a large file
when one out of two bricks are down?
Yes, gluster only updates regions of the file that need to be
synchronized during self-healing. More details on this synchronization
can be found in the self-healing section of afr's design document [1].
Right now I am leaning towards using SSD for "host local"
disk - single
brick gluster volumes intended for VMs which are node specific and then
3 way replicas for the higher availability zones which tend to be more
read oriented. I presume that read-only access only needs to get data
from one of the 3 replicas so that should be reasonably performant.
Yes, read operations are directed to only one of the replicas.
Regards,
Vijay
[1] https://github.com/gluster/glusterfs/blob/master/doc/features/afr-v1.md
2:01 p.m.
On Fri, Aug 29, 2014 at 12:25 PM, Vijay Bellur <vbellur(a)redhat.com> wrote:
On 08/29/2014 07:34 PM, David King wrote:
>
> Paul,
>
> Thanks for the response.
>
> You mention that the issue is orphaned files during updates when one
> node is down. However I am less concerned about adding and removing
> files because the file server will be predominately VM disks so the file
> structure is fairly static. Those VM files will be quite active however
> - will gluster be able to keep track of partial updates to a large file
> when one out of two bricks are down?
>
Yes, gluster only updates regions of the file that need to be synchronized
during self-healing. More details on this synchronization can be found in
the self-healing section of afr's design document [1].
> Right now I am leaning towards using SSD for "host local" disk - single
> brick gluster volumes intended for VMs which are node specific and then
I wouldn't use single brick gluster volumes for local disk you don't
need it and it will actually make it more complicated with no real
benefits.
> 3 way replicas for the higher availability zones which tend to be
more
> read oriented. I presume that read-only access only needs to get data
> from one of the 3 replicas so that should be reasonably performant.
Yes, read operations are directed to only one of the replicas.
Regards,
Vijay
[1] https://github.com/gluster/glusterfs/blob/master/doc/features/afr-v1.md
2:36 p.m.
Hi Paul,
I would prefer to do a direct mount for local disk. However I am not certain how to
configure a single system with both local storage and bluster replicated storage.
- The “Configure Local Storage” option for Hosts wants to make a datacenter and cluster
for the system. I presume that’s because oVirt wants to be able to mount the storage on
all hosts in a datacenter.
- Configuring a POSIX storage domain with local disk does not work as oVirt wants to mount
the disk on all systems in the datacenter.
I suppose my third option would be to put these systems as libvirt VMs and not manage them
with oVirt. This is fairly reasonable as I use Foreman for provisioning except that I
will need to figure out how to make them co-exist. Has anyone tried this?
Am I missing other options for local non-replicated disk?
Thanks,
David
--
David King
On August 29, 2014 at 3:01:49 PM, Paul Robert Marino (prmarino1(a)gmail.com) wrote:
On Fri, Aug 29, 2014 at 12:25 PM, Vijay Bellur <vbellur(a)redhat.com> wrote:
On 08/29/2014 07:34 PM, David King wrote:
>
> Paul,
>
> Thanks for the response.
>
> You mention that the issue is orphaned files during updates when one
> node is down. However I am less concerned about adding and removing
> files because the file server will be predominately VM disks so the file
> structure is fairly static. Those VM files will be quite active however
> - will gluster be able to keep track of partial updates to a large file
> when one out of two bricks are down?
>
Yes, gluster only updates regions of the file that need to be synchronized
during self-healing. More details on this synchronization can be found in
the self-healing section of afr's design document [1].
> Right now I am leaning towards using SSD for "host local" disk - single
> brick gluster volumes intended for VMs which are node specific and then
I wouldn't use single brick gluster volumes for local disk you don't
need it and it will actually make it more complicated with no real
benefits.
> 3 way replicas for the higher availability zones which tend to be
more
> read oriented. I presume that read-only access only needs to get data
> from one of the 3 replicas so that should be reasonably performant.
Yes, read operations are directed to only one of the replicas.
Regards,
Vijay
[1] https://github.com/gluster/glusterfs/blob/master/doc/features/afr-v1.md
3530
days inactive
3531
days old
5 comments
3 participants
participants (3)
-
David King -
Paul Robert Marino -
Vijay Bellur