3:33 a.m.
Hi all,
Not sure if this is the place to be asking this but I was wondering which hardware you all
are using and why in order for me to see what I would be needing.
I would like to set up a HA cluster consisting off 3 hosts to be able to run 30 vm’s.
The engine, I can run on an other server. The hosts can be fitted with the storage and
share the space through glusterfs. I would think I will be needing at least 3 nic’s but
would be able to install ovn. (Are 1gb nic’s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Thanks,
4:35 a.m.
This is a multi-part message in MIME format.
--------------D22D740E8090EF46CB19D891
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi,
HL ProLiant DL380, dual Xeon
120 GB RAID L1 for system
2 TB RAID L10 for VM disks
5 VMs, 3 Linux, 2 Windows
Total CPU load most of the time is=C2=A0 low, high level of activity rela=
ted
to disk.
Host engine under KVM appliance on SuSE, can be easily moved, backed up,
copied, experimented with, etc.
You'll have to use servers with more RAM and storage than main.
More then one NIC required if some of your VMs are on different subnets,
e.g. 1 in internal zone and 2nd on DMZ.
For your setup 10 GB NICs + L3 Switch for ovirtmgmt.
BTW, I would suggest to have several separate hardware RAIDs unless you
have SSD, otherwise limit of the disk system I/O will be a bottleneck.
Consider SSD L1 RAID for heavy-loaded databases.
*Please note many cheap SSDs do NOT work reliably with SAS controllers
even in SATA mode*.
For example, I supposed to use 2 x WD Green SSD configures as RAID L1
for OS.
It was possible to install system, yet under heavy load simulated with
iozone disk system freeze, rendering OS unbootable.
Same crash was experienced with 512GB KingFast SSD connected to
broadcom/AMCC SAS RAID Card.
On 03/24/2018 10:33 AM, Andy Michielsen wrote:
Hi all,
Not sure if this is the place to be asking this but I was wondering whi=
ch
hardware you all are using and why in order for me to see what I would=
be needing.
I would like to set up a HA cluster consisting off 3 hosts to be able t=
o run 30
vm=E2=80=99s.
The engine, I can run on an other server. The hosts can be fitted
with =
the storage and share the space through glusterfs. I would think I will b=
e needing at least 3 nic=E2=80=99s but would be able to install ovn. (Are=
1gb nic=E2=80=99s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Thanks,
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--------------D22D740E8090EF46CB19D891
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html;
charset=3Dutf=
-8">
</head>
<body bgcolor=3D"#FFFFFF" text=3D"#000000">
<div class=3D"moz-cite-prefix">Hi,<br>
<br>
HL ProLiant DL380, dual Xeon<br>
120 GB RAID L1 for system<br>
2 TB RAID L10 for VM disks<br>
5 VMs, 3 Linux, 2 Windows<br>
Total CPU load most of the time is=C2=A0 low, high level of activit=
y
related to disk.<br>
Host engine under KVM appliance on SuSE, can be easily moved,
backed up, copied, experimented with, etc.<br>
<br>
You'll have to use servers with more RAM and storage than main.<br>
More then one NIC required if some of your VMs are on different
subnets, e.g. 1 in internal zone and 2nd on DMZ.<br>
For your setup 10 GB NICs + L3 Switch for ovirtmgmt.<br>
<br>
BTW, I would suggest to have several separate hardware RAIDs
unless you have SSD, otherwise limit of the disk system I/O will
be a bottleneck. Consider SSD L1 RAID for heavy-loaded databases.<b=
r>
<br>
<font color=3D"#990000"><b>Please note many cheap SSDs do NOT
work
reliably with SAS controllers even in SATA
mode</b>.</font><br>
<br>
For example, I supposed to use 2 x WD Green SSD configures as RAID
L1 for OS. <br>
It was possible to install system, yet under heavy load simulated
with iozone disk system freeze, rendering OS unbootable.<br>
Same crash was experienced with 512GB KingFast SSD connected to
broadcom/AMCC SAS RAID Card.<br>
<br>
<br>
On 03/24/2018 10:33 AM, Andy Michielsen wrote:<br>
</div>
<blockquote type=3D"cite"
cite=3D"mid:815987B5-31DA-4316-809D-A03363A1E3C3@gmail.com">
<pre wrap=3D"">Hi all,
Not sure if this is the place to be asking this but I was wondering which=
hardware you all are using and why in order for me to see what I would b=
e needing.
I would like to set up a HA cluster consisting off 3 hosts to be able to =
run 30 vm=E2=80=99s.
The engine, I can run on an other server. The hosts can be fitted with th=
e storage and share the space through glusterfs. I would think I will be =
needing at least 3 nic=E2=80=99s but would be able to install ovn. (Are 1=
gb nic=E2=80=99s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Thanks,
_______________________________________________
Users mailing list
<a class=3D"moz-txt-link-abbreviated"
href=3D"mailto:Users@ovirt.org">Use=
rs(a)ovirt.org</a>
<a class=3D"moz-txt-link-freetext"
href=3D"http://lists.ovirt.org/mailman=
/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<p><br>
</p>
</body>
</html>
--------------D22D740E8090EF46CB19D891--
6:24 a.m.
--Apple-Mail-FF4BAD67-2D6B-4D37-B4DF-9283D336B1AF
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hello Andrei,
Thank you very much for sharing info on your hardware setup. Very informativ=
e.
At this moment I have my ovirt engine on our vmware environment which is fin=
e for good backup and restore.
I have 4 nodes running now all different in make and model with local storag=
e and it works but lacks performance a bit.
But I can get my hands on some old dell=E2=80=99s R415 with 96 Gb of ram and=
2 quadcores and 6 x 1 Gb nic=E2=80=99s. They all come with 2 x 146 Gb 15000=
rpm=E2=80=99s harddisks. This isn=E2=80=99t bad but I will add more RAM for=
starters. Also I would like to have some good redundant storage for this to=
o and the servers have limited space to add that.
Hopefully others will also share there setups and expirience like you did.
Kind regards.
=20
<meta http-equiv=3D"Content-Type" content=3D"text/html;
charset=3Dutf-8"=
=20
=20
<div class=3D"moz-cite-prefix">Hi,<br
<br HL ProLiant DL380, dual Xeon<br 120 GB RAID L1 for system<br 2 TB RAID L10 for VM disks<br 5 VMs, 3 Linux, 2 Windows<br Total CPU load most of the time is low, high
level of activity
related to disk.<br Host engine under KVM
appliance on SuSE, can be easily moved,
backed up, copied, experimented with, etc.<br
<br You'll have to use servers with more RAM and storage
than main.<br More then one NIC required if
some of your VMs are on different
subnets, e.g. 1 in internal zone and 2nd on DMZ.<br For
your setup 10 GB NICs + L3 Switch for ovirtmgmt.<br
<br BTW, I would suggest to have several separate hardware
RAIDs
unless you have SSD, otherwise limit of the disk system I/O will
be a bottleneck. Consider SSD L1 RAID for heavy-loaded databases.<br <br
<font color=3D"#990000"><b>Please note many cheap SSDs do NOT work
reliably with SAS controllers even in SATA
mode</b>.</font><br <br For example, I supposed to use 2 x WD Green SSD
configures as RAID
L1 for OS. <br It was possible to install
system, yet under heavy load simulated
with iozone disk system freeze, rendering OS unbootable.<br Same crash was experienced with 512GB KingFast SSD
connected to
broadcom/AMCC SAS RAID Card.<br
<br <br On
03/24/2018 10:33 AM, Andy Michielsen wrote:<br
</div <blockquote
type=3D"cite" cite=3D"mid:815987B5-31DA-4316-809D-A03363A1E3=
C3(a)gmail.com" <pre
wrap=3D"">Hi all,
Not sure if this is the place to be asking this but I was wondering which ha=
rdware you all are using and why in order for me to see what I would be need=
ing.
I would like to set up a HA cluster consisting off 3 hosts to be able to run=
30 vm=E2=80=99s.
The engine, I can run on an other server. The hosts can be fitted with the s=
torage and share the space through glusterfs. I would think I will be needin=
g at least 3 nic=E2=80=99s but would be able to install ovn. (Are 1gb nic=E2=
=80=99s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Thanks,
_______________________________________________
Users mailing list
<a class=3D"moz-txt-link-abbreviated"
href=3D"mailto:Users@ovirt.org">Users@=
ovirt.org</a <a
class=3D"moz-txt-link-freetext" href=3D"http://lists.ovirt.org/mailman/li=
stinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a </pre
</blockquote <p><br </p =20
</div></blockquote></body></html>=
--Apple-Mail-FF4BAD67-2D6B-4D37-B4DF-9283D336B1AF--
On 24 Mar 2018, at 10:35, Andrei Verovski
<andreil1(a)starlett.lv> wrote:
=20
Hi,
=20
HL ProLiant DL380, dual Xeon
120 GB RAID L1 for system
2 TB RAID L10 for VM disks
5 VMs, 3 Linux, 2 Windows
Total CPU load most of the time is low, high level of activity related to=
disk.
Host engine under KVM appliance on SuSE, can be easily moved, backed
up, c=
opied, experimented with, etc.
=20
You'll have to use servers with more RAM and storage than main.
More then one NIC required if some of your VMs are on different subnets, e=
.g. 1
in internal zone and 2nd on DMZ.
For your setup 10 GB NICs + L3 Switch for ovirtmgmt.
=20
BTW, I would suggest to have several separate hardware RAIDs unless you ha=
ve SSD,
otherwise limit of the disk system I/O will be a bottleneck. Conside=
r SSD L1 RAID for heavy-loaded databases.
=20
Please note many cheap SSDs do NOT work reliably with SAS controllers even=
in
SATA mode.
=20
For example, I supposed to use 2 x WD Green SSD configures as RAID L1 for O=
S.=20
It was possible to install system, yet under heavy load simulated
with ioz=
one disk system freeze, rendering OS unbootable.
Same crash was experienced with 512GB KingFast SSD connected to
broadcom/A=
MCC SAS RAID Card.
=20
=20
> On 03/24/2018 10:33 AM, Andy Michielsen wrote:
> Hi all,
>=20
> Not sure if this is the place to be asking this but I was wondering which=
hardware you all are using and why in order for me to see what I would be n=
eeding.
>=20
> I would like to set up a HA cluster consisting off 3 hosts to be able to r=
un
30 vm=E2=80=99s.
> The engine, I can run on an other server. The hosts can be fitted
with th=
e storage and share the space through glusterfs. I would think I will be
nee=
ding at least 3 nic=E2=80=99s but would be able to install ovn. (Are 1gb nic=
=E2=80=99s sufficient ?)
>=20
> Any input you guys would like to share would be greatly appriciated.
>=20
> Thanks,
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
=20
--Apple-Mail-FF4BAD67-2D6B-4D37-B4DF-9283D336B1AF
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><head><meta http-equiv=3D"content-type"
content=3D"text/html; charset=3D=
utf-8"></head><body
dir=3D"auto"><div></div><div>Hello
Andrei,</div><div><br=
</div><div>Thank you very much for sharing info on your
hardware setup. Ver=
y
informative.</div><div><br></div><div>At this moment I have
my ovirt engin=
e on our vmware environment which is fine for good backup and restore.</div>=
<div><br></div><div>I have 4 nodes running now all different in
make and mod=
el with local storage and it works but lacks performance a
bit.</div><div><b=
r></div><div>But I can get my hands on some old dell=E2=80=99s R415 with 96
G=
b of ram and 2 quadcores and 6 x 1 Gb nic=E2=80=99s. They all come with 2 x 1=
46 Gb 15000 rpm=E2=80=99s harddisks. This isn=E2=80=99t bad but I will add m=
ore RAM for starters. Also I would like to have some good redundant storage f=
or this too and the servers have limited space to add
that.</div><div><br></=
div><div>Hopefully others will also share there setups and expirience like y=
ou did.</div><div><br></div><div>Kind
regards.</div><div><br>On 24 Mar 2018,=
at 10:35, Andrei Verovski <<a
href=3D"mailto:andreil1@starlett.lv">andre=
il1(a)starlett.lv</a>&gt; wrote:<br><br></div><blockquote
type=3D"cite"><div
2:08 p.m.
I have 2 or 3 node clusters with following hardware (all with self-hosted
engine) :
2 node cluster:
RAM: 64 GB per host
CPU: 8 cores per host
Storage: 4x 1TB SAS in RAID10
NIC: 2x Gbit
VMs: 20
The above, although I would like to have had a third NIC for gluster
storage redundancy, it is running smoothly for quite some time and without
performance issues.
The VMs it is running are not high on IO (mostly small Linux servers).
3 node clusters:
RAM: 32 GB per host
CPU: 16 cores per host
Storage: 5x 600GB in RAID5 (not ideal but I had to gain some storage space
without purchasing extra disks)
NIC: 6x Gbit
VMs: less then 10 large Windows VMs (Windows 2016 server and Windows 10)
For your setup (30 VMs) I would rather go with RAID10 SAS disks and at
least a dual 10Gbit NIC dedicated to the gluster traffic only.
Alex
On Sat, Mar 24, 2018 at 1:24 PM, Andy Michielsen <andy.michielsen(a)gmail.com>
wrote:
Hello Andrei,
Thank you very much for sharing info on your hardware setup. Very
informative.
At this moment I have my ovirt engine on our vmware environment which is
fine for good backup and restore.
I have 4 nodes running now all different in make and model with local
storage and it works but lacks performance a bit.
But I can get my hands on some old dell’s R415 with 96 Gb of ram and 2
quadcores and 6 x 1 Gb nic’s. They all come with 2 x 146 Gb 15000 rpm’s
harddisks. This isn’t bad but I will add more RAM for starters. Also I
would like to have some good redundant storage for this too and the servers
have limited space to add that.
Hopefully others will also share there setups and expirience like you did.
Kind regards.
On 24 Mar 2018, at 10:35, Andrei Verovski <andreil1(a)starlett.lv> wrote:
Hi,
HL ProLiant DL380, dual Xeon
120 GB RAID L1 for system
2 TB RAID L10 for VM disks
5 VMs, 3 Linux, 2 Windows
Total CPU load most of the time is low, high level of activity related to
disk.
Host engine under KVM appliance on SuSE, can be easily moved, backed up,
copied, experimented with, etc.
You'll have to use servers with more RAM and storage than main.
More then one NIC required if some of your VMs are on different subnets,
e.g. 1 in internal zone and 2nd on DMZ.
For your setup 10 GB NICs + L3 Switch for ovirtmgmt.
BTW, I would suggest to have several separate hardware RAIDs unless you
have SSD, otherwise limit of the disk system I/O will be a bottleneck.
Consider SSD L1 RAID for heavy-loaded databases.
*Please note many cheap SSDs do NOT work reliably with SAS controllers
even in SATA mode*.
For example, I supposed to use 2 x WD Green SSD configures as RAID L1 for
OS.
It was possible to install system, yet under heavy load simulated with
iozone disk system freeze, rendering OS unbootable.
Same crash was experienced with 512GB KingFast SSD connected to
broadcom/AMCC SAS RAID Card.
On 03/24/2018 10:33 AM, Andy Michielsen wrote:
Hi all,
Not sure if this is the place to be asking this but I was wondering which hardware you
all are using and why in order for me to see what I would be needing.
I would like to set up a HA cluster consisting off 3 hosts to be able to run 30 vm’s.
The engine, I can run on an other server. The hosts can be fitted with the storage and
share the space through glusterfs. I would think I will be needing at least 3 nic’s but
would be able to install ovn. (Are 1gb nic’s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Thanks,
_______________________________________________
Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2:36 a.m.
--Apple-Mail-B8EBE151-6DD4-4679-A889-96283D26D5BF
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hello Alex,
Thanks for sharing. Much appriciated.
I believe my setup would need 96 Gb off RAM in each host, and would need abo=
ut at least 3 Tb of storage. Probably 4 Tb would be beter if I want to work w=
ith snapshots. (Will be running mostly windows 2016 servers or windows 10 de=
sktops with 6Gb off RAM and 100 Gb of disks)
I agree that a 10 Gb network for storage would be very beneficial.
Now If I can figure out how to set up a glusterfs on a 3 node cluster in oVi=
rt 4.2 just for the data storage. I =E2=80=98m golden to get started. :-)
Kind regards.
=20
=20
=20
=20
<div class=3D"m_2490472978721696341moz-cite-prefix">Hi,<br <br HL
ProLiant DL380, dual Xeon<br 120 GB RAID L1 for
system<br 2 TB RAID L10 for VM
disks<br 5 VMs, 3 Linux, 2
Windows<br Total CPU load most of the
time is low, high level of activity
related to disk.<br Host engine under KVM
appliance on SuSE, can be easily moved,
backed up, copied, experimented with, etc.<br
<br You'll have to use servers with more RAM and storage
than main.<br More then one NIC required if
some of your VMs are on different
subnets, e.g. 1 in internal zone and 2nd on DMZ.<br For
your setup 10 GB NICs + L3 Switch for ovirtmgmt.<br
<br BTW, I would suggest to have several separate hardware
RAIDs
unless you have SSD, otherwise limit of the disk system I/O will
be a bottleneck. Consider SSD L1 RAID for heavy-loaded databases.<br <br
<font color=3D"#990000"><b>Please note many cheap SSDs do NOT work
reliably with SAS controllers even in SATA
mode</b>.</font><br <br For example, I supposed to use 2 x WD Green SSD
configures as RAID
L1 for OS. <br It was possible to install
system, yet under heavy load simulated
with iozone disk system freeze, rendering OS unbootable.<br Same crash was experienced with 512GB KingFast SSD
connected to
broadcom/AMCC SAS RAID Card.<br
<br <br On
03/24/2018 10:33 AM, Andy Michielsen wrote:<br
</div <blockquote
type=3D"cite" <pre>Hi all,
Not sure if this is the place to be asking this but I was wondering which ha=
rdware you all are using and why in order for me to see what I would be need=
ing.
I would like to set up a HA cluster consisting off 3 hosts to be able to run=
30 vm=E2=80=99s.
The engine, I can run on an other server. The hosts can be fitted with the s=
torage and share the space through glusterfs. I would think I will be needin=
g at least 3 nic=E2=80=99s but would be able to install ovn. (Are 1gb nic=E2=
=80=99s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Thanks,
______________________________<wbr>_________________
Users mailing list
<a class=3D"m_2490472978721696341moz-txt-link-abbreviated"
href=3D"mailto:Us=
ers(a)ovirt.org" target=3D"_blank">Users(a)ovirt.org</a <a
class=3D"m_2490472978721696341moz-txt-link-freetext" href=3D"http://lists=
.ovirt.org/mailman/listinfo/users"
target=3D"_blank">http://lists.ovirt.org/=
<wbr>mailman/listinfo/users</a
</pre </blockquote <p><br
</p =20
</div></blockquote></div></div></div><br>______________________________<wbr>=
_________________<br Users mailing list<br <a
href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><br <a
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
rel=3D"noreferrer"=
target=3D"_blank">http://lists.ovirt.org/<wbr>mailman/...
<br></blockquote></div><br></div </div></blockquote></body></html>=
--Apple-Mail-B8EBE151-6DD4-4679-A889-96283D26D5BF--
On 24 Mar 2018, at 20:08, Alex K <rightkicktech(a)gmail.com>
wrote:
=20
I have 2 or 3 node clusters with following hardware (all with self-hosted e=
ngine)
:=20
=20
2 node cluster:=20
RAM: 64 GB per host
CPU: 8 cores per host
Storage: 4x 1TB SAS in RAID10
NIC: 2x Gbit
VMs: 20
=20
The above, although I would like to have had a third NIC for gluster stora=
ge
redundancy, it is running smoothly for quite some time and without perfor=
mance issues.=20
The VMs it is running are not high on IO (mostly small Linux
servers).=20
=20
3 node clusters:=20
RAM: 32 GB per host
CPU: 16 cores per host
Storage: 5x 600GB in RAID5 (not ideal but I had to gain some storage space=
without purchasing extra disks)
NIC: 6x Gbit
VMs: less then 10 large Windows VMs (Windows 2016 server and Windows 10)
=20
For your setup (30 VMs) I would rather go with RAID10 SAS disks and at lea=
st a
dual 10Gbit NIC dedicated to the gluster traffic only.=20
=20
Alex
=20
=20
> On Sat, Mar 24, 2018 at 1:24 PM, Andy Michielsen <andy.michielsen(a)gmail.c=
om> wrote:
> Hello Andrei,
>=20
> Thank you very much for sharing info on your hardware setup. Very informa=
tive.
>=20
> At this moment I have my ovirt engine on our vmware environment which is f=
ine
for good backup and restore.
>=20
> I have 4 nodes running now all different in make and model with local sto=
rage
and it works but lacks performance a bit.
>=20
> But I can get my hands on some old dell=E2=80=99s R415 with 96 Gb of ram a=
nd
2 quadcores and 6 x 1 Gb nic=E2=80=99s. They all come with 2 x 146 Gb 150=
00 rpm=E2=80=99s harddisks. This isn=E2=80=99t bad but I will add more RAM f=
or starters. Also I would like to have some good redundant storage for this t=
oo and the servers have limited space to add that.
>=20
> Hopefully others will also share there setups and expirience like you did=
.
>=20
> Kind regards.
>=20
>> On 24 Mar 2018, at 10:35, Andrei Verovski <andreil1(a)starlett.lv> wrote:
>>=20
>> Hi,
>>=20
>> HL ProLiant DL380, dual Xeon
>> 120 GB RAID L1 for system
>> 2 TB RAID L10 for VM disks
>> 5 VMs, 3 Linux, 2 Windows
>> Total CPU load most of the time is low, high level of activity related t=
o disk.
>> Host engine under KVM appliance on SuSE, can be easily moved,
backed up,=
copied, experimented with, etc.
>>=20
>> You'll have to use servers with more RAM and storage than main.
>> More then one NIC required if some of your VMs are on different subnets,=
e.g. 1 in internal zone and 2nd on DMZ.
>> For your setup 10 GB NICs + L3 Switch for ovirtmgmt.
>>=20
>> BTW, I would suggest to have several separate hardware RAIDs unless you h=
ave SSD, otherwise limit of the disk system I/O will be a bottleneck. Consid=
er SSD L1 RAID for heavy-loaded databases.
>>=20
>> Please note many cheap SSDs do NOT work reliably with SAS controllers ev=
en in SATA mode.
>>=20
>> For example, I supposed to use 2 x WD Green SSD configures as RAID L1 fo=
r
OS.=20
>> It was possible to install system, yet under heavy load
simulated with i=
ozone disk system freeze, rendering OS unbootable.
>> Same crash was experienced with 512GB KingFast SSD connected
to broadcom=
/AMCC SAS RAID Card.
>>=20
>>=20
>>> On 03/24/2018 10:33 AM, Andy Michielsen wrote:
>>> Hi all,
>>>=20
>>> Not sure if this is the place to be asking this but I was wondering whi=
ch hardware you all are using and why in order for me to see what I would be=
needing.
>>>=20
>>> I would like to set up a HA cluster consisting off 3 hosts to be able t=
o run 30 vm=E2=80=99s.
>>> The engine, I can run on an other server. The hosts can
be fitted with t=
he storage and share the space through glusterfs. I would think I
will be ne=
eding at least 3 nic=E2=80=99s but would be able to install ovn. (Are 1gb ni=
c=E2=80=99s sufficient ?)
>>>=20
>>> Any input you guys would like to share would be greatly appriciated.
>>>=20
>>> Thanks,
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>=20
>=20
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>=20
=20
--Apple-Mail-B8EBE151-6DD4-4679-A889-96283D26D5BF
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><head><meta http-equiv=3D"content-type"
content=3D"text/html; charset=3D=
utf-8"></head><body
dir=3D"auto"><div></div><div>Hello
Alex,</div><div><br><=
/div><div>Thanks for sharing. Much
appriciated.</div><div><br></div><div>I b=
elieve my setup would need 96 Gb off RAM in each host, and would need about a=
t least 3 Tb of storage. Probably 4 Tb would be beter if I want to work with=
snapshots. (Will be running mostly windows 2016 servers or windows 10 deskt=
ops with 6Gb off RAM and 100 Gb of
disks)</div><div><br></div><div>I agree t=
hat a 10 Gb network for storage would be very
beneficial.</div><div><br></di=
v><div>Now If I can figure out how to set up a glusterfs on a 3 node cluster=
in oVirt 4.2 just for the data storage. I =E2=80=98m golden to get started.=
:-)</div><div><br></div><div>Kind
regards.</div><div><br>On 24 Mar 2018, at=
20:08, Alex K <<a
href=3D"mailto:rightkicktech@gmail.com">rightkicktech@=
gmail.com</a>> wrote:<br><br></div><blockquote
type=3D"cite"><div><div di=
r=3D"ltr"><div><div>I have 2 or 3 node clusters with following
hardware (all=
with self-hosted engine) : <br><br></div>2 node cluster: <br>RAM:
64 GB per=
host<br></div><div>CPU: 8 cores per
host<br></div><div>Storage: 4x 1TB SAS i=
n RAID10<br></div><div>NIC: 2x Gbit<br></div><div>VMs:
20<br><br></div><div>=
The above, although I would like to have had a third NIC for gluster storage=
redundancy, it is running smoothly for quite some time and without performa=
nce issues. <br></div><div>The VMs it is running are not high on IO
(mostly s=
mall Linux servers). <br><br></div><div>3 node clusters:
<br></div><div>RAM:=
32 GB per host<br></div><div>CPU: 16 cores per
host<br></div><div>Storage: 5=
x 600GB in RAID5 (not ideal but I had to gain some storage space without pur=
chasing extra disks)<br></div><div>NIC: 6x
Gbit<br></div><div>VMs: less then=
10 large Windows VMs (Windows 2016 server and Windows
10)<br><br></div><div=
For your setup (30 VMs) I would rather go with RAID10 SAS disks and at
leas=
t a dual 10Gbit NIC dedicated to the gluster traffic only.
<br><br></div><di=
v>Alex<br></div><br></div><div
class=3D"gmail_extra"><br><div class=3D"gmail=
_quote">On Sat, Mar 24, 2018 at 1:24 PM, Andy Michielsen <span
dir=3D"ltr">&=
lt;<a href=3D"mailto:andy.michielsen@gmail.com"
target=3D"_blank">andy.michi=
elsen(a)gmail.com</a>&gt;</span> wrote:<br><blockquote
class=3D"gmail_quote" s=
tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex"><div d=
ir=3D"auto"><div></div><div>Hello
Andrei,</div><div><br></div><div>Thank you=
very much for sharing info on your hardware setup. Very informative.</div><=
div><br></div><div>At this moment I have my ovirt engine on our
vmware envir=
onment which is fine for good backup and
restore.</div><div><br></div><div>I=
have 4 nodes running now all different in make and model with local storage=
and it works but lacks performance a
bit.</div><div><br></div><div>But I ca=
n get my hands on some old dell=E2=80=99s R415 with 96 Gb of ram and 2 quadc=
ores and 6 x 1 Gb nic=E2=80=99s. They all come with 2 x 146 Gb 15000 rpm=E2=80=
=99s harddisks. This isn=E2=80=99t bad but I will add more RAM for starters.=
Also I would like to have some good redundant storage for this too and the s=
ervers have limited space to add
that.</div><div><br></div><div>Hopefully ot=
hers will also share there setups and expirience like you
did.</div><div><br=
</div><div>Kind regards.</div><div><div
class=3D"h5"><div><br>On 24 Mar 201=
8, at 10:35, Andrei
Verovski <<a href=3D"mailto:andreil1@starlett.lv" tar=
get=3D"_blank">andreil1(a)starlett.lv</a>&gt;
wrote:<br><br></div><blockquote t=
ype=3D"cite"><div
2:35 p.m.
Andy, Im using 2 node cluster:
-2 supermicro 6017 (2 Intel 2420(12C24T each node) 384Gb ram total, 10gbe.
all hosted engine via nfs
storage side: 2 SC836BE16-R1K28B(192gb arc cache) with raid 10 zfs+intel
slog serving iscsi at 10Gbe
80 VM's more or less.
regards,
2018-03-25 4:36 GMT-03:00 Andy Michielsen <andy.michielsen(a)gmail.com>:
Hello Alex,
Thanks for sharing. Much appriciated.
I believe my setup would need 96 Gb off RAM in each host, and would need
about at least 3 Tb of storage. Probably 4 Tb would be beter if I want to
work with snapshots. (Will be running mostly windows 2016 servers or
windows 10 desktops with 6Gb off RAM and 100 Gb of disks)
I agree that a 10 Gb network for storage would be very beneficial.
Now If I can figure out how to set up a glusterfs on a 3 node cluster in
oVirt 4.2 just for the data storage. I ‘m golden to get started. :-)
Kind regards.
On 24 Mar 2018, at 20:08, Alex K <rightkicktech(a)gmail.com> wrote:
I have 2 or 3 node clusters with following hardware (all with self-hosted
engine) :
2 node cluster:
RAM: 64 GB per host
CPU: 8 cores per host
Storage: 4x 1TB SAS in RAID10
NIC: 2x Gbit
VMs: 20
The above, although I would like to have had a third NIC for gluster
storage redundancy, it is running smoothly for quite some time and without
performance issues.
The VMs it is running are not high on IO (mostly small Linux servers).
3 node clusters:
RAM: 32 GB per host
CPU: 16 cores per host
Storage: 5x 600GB in RAID5 (not ideal but I had to gain some storage space
without purchasing extra disks)
NIC: 6x Gbit
VMs: less then 10 large Windows VMs (Windows 2016 server and Windows 10)
For your setup (30 VMs) I would rather go with RAID10 SAS disks and at
least a dual 10Gbit NIC dedicated to the gluster traffic only.
Alex
On Sat, Mar 24, 2018 at 1:24 PM, Andy Michielsen <
andy.michielsen(a)gmail.com> wrote:
> Hello Andrei,
>
> Thank you very much for sharing info on your hardware setup. Very
> informative.
>
> At this moment I have my ovirt engine on our vmware environment which is
> fine for good backup and restore.
>
> I have 4 nodes running now all different in make and model with local
> storage and it works but lacks performance a bit.
>
> But I can get my hands on some old dell’s R415 with 96 Gb of ram and 2
> quadcores and 6 x 1 Gb nic’s. They all come with 2 x 146 Gb 15000 rpm’s
> harddisks. This isn’t bad but I will add more RAM for starters. Also I
> would like to have some good redundant storage for this too and the servers
> have limited space to add that.
>
> Hopefully others will also share there setups and expirience like you did.
>
> Kind regards.
>
> On 24 Mar 2018, at 10:35, Andrei Verovski <andreil1(a)starlett.lv> wrote:
>
> Hi,
>
> HL ProLiant DL380, dual Xeon
> 120 GB RAID L1 for system
> 2 TB RAID L10 for VM disks
> 5 VMs, 3 Linux, 2 Windows
> Total CPU load most of the time is low, high level of activity related
> to disk.
> Host engine under KVM appliance on SuSE, can be easily moved, backed up,
> copied, experimented with, etc.
>
> You'll have to use servers with more RAM and storage than main.
> More then one NIC required if some of your VMs are on different subnets,
> e.g. 1 in internal zone and 2nd on DMZ.
> For your setup 10 GB NICs + L3 Switch for ovirtmgmt.
>
> BTW, I would suggest to have several separate hardware RAIDs unless you
> have SSD, otherwise limit of the disk system I/O will be a bottleneck.
> Consider SSD L1 RAID for heavy-loaded databases.
>
> *Please note many cheap SSDs do NOT work reliably with SAS controllers
> even in SATA mode*.
>
> For example, I supposed to use 2 x WD Green SSD configures as RAID L1 for
> OS.
> It was possible to install system, yet under heavy load simulated with
> iozone disk system freeze, rendering OS unbootable.
> Same crash was experienced with 512GB KingFast SSD connected to
> broadcom/AMCC SAS RAID Card.
>
>
> On 03/24/2018 10:33 AM, Andy Michielsen wrote:
>
> Hi all,
>
> Not sure if this is the place to be asking this but I was wondering which hardware
you all are using and why in order for me to see what I would be needing.
>
> I would like to set up a HA cluster consisting off 3 hosts to be able to run 30
vm’s.
> The engine, I can run on an other server. The hosts can be fitted with the storage
and share the space through glusterfs. I would think I will be needing at least 3 nic’s
but would be able to install ovn. (Are 1gb nic’s sufficient ?)
>
> Any input you guys would like to share would be greatly appriciated.
>
> Thanks,
> _______________________________________________
> Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2:42 a.m.
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Otw9dz9x11zAjj8g4jYNmTEKPlPgL4W9o
Content-Type: multipart/mixed; boundary="7NT1xWEOC6g4SNggAbskfGEZF9hpEIL4F";
protected-headers="v1"
From: Richard Neuboeck <hawk(a)tbi.univie.ac.at>
To: users(a)ovirt.org
Message-ID: <b71bed12-f9b6-bbda-64f2-e99b47160e30(a)tbi.univie.ac.at>
Subject: Re: [ovirt-users] Which hardware are you using for oVirt
References: <815987B5-31DA-4316-809D-A03363A1E3C3(a)gmail.com>
<cdca9ccd-8c15-1cdb-ead6-84567f226ea4(a)starlett.lv>
<1701ED12-9ED4-46B9-96E5-1FD3E32419DD(a)gmail.com>
<CABMULtKPvZRDRkByH3WyXbZQHuSZY4o7=hNwNqvRDsV=+6yjtg(a)mail.gmail.com>
<545F49C7-9AFE-4BE3-B03B-3FB1DEE1F09E(a)gmail.com>
In-Reply-To: <545F49C7-9AFE-4BE3-B03B-3FB1DEE1F09E(a)gmail.com>
--7NT1xWEOC6g4SNggAbskfGEZF9hpEIL4F
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
Hi Andy,
we have 3 hosts for virtualization. Each 40 Cores, 512GB RAM, RAID 1 for
the system, 4 bonded (onboard) 1Gbit NICs for client access (to the VMs)
and a 10GBit NIC for the storage network.
The storage is built of 3 hosts, 10Gbit NIC, RAID 6 (5TB HDDs and SSDs
for caching) and gluster in replica 3 mode.
Cheers
Richard
On 25.03.18 09:36, Andy Michielsen wrote:
Hello Alex,
=20
Thanks for sharing. Much appriciated.
=20
I believe my setup would need 96 Gb off RAM in each host, and would nee=
d
about at least 3 Tb of storage. Probably 4 Tb would be beter if I
want
to work with snapshots. (Will be running mostly windows 2016 servers or=
windows 10 desktops with 6Gb off RAM and 100 Gb of disks)
=20
I agree that a 10 Gb network for storage would be very beneficial.
=20
Now If I can figure out how to set up a glusterfs on a 3 node cluster i=
n
oVirt 4.2 just for the data storage. I =E2=80=98m golden to get
started=
=2E :-)
=20
Kind regards.
=20
On 24 Mar 2018, at 20:08, Alex K <rightkicktech(a)gmail.com
<mailto:rightkicktech@gmail.com>> wrote:
=20
> I have 2 or 3 node clusters with following hardware (all with
> self-hosted engine) :
>
> 2 node cluster:
> RAM: 64 GB per host
> CPU: 8 cores per host
> Storage: 4x 1TB SAS in RAID10
> NIC: 2x Gbit
> VMs: 20
>
> The above, although I would like to have had a third NIC for gluster
> storage redundancy, it is running smoothly for quite some time and
> without performance issues.
> The VMs it is running are not high on IO (mostly small Linux servers).=
>
> 3 node clusters:
> RAM: 32 GB per host
> CPU: 16 cores per host
> Storage: 5x 600GB in RAID5 (not ideal but I had to gain some storage
> space without purchasing extra disks)
> NIC: 6x Gbit
> VMs: less then 10 large Windows VMs (Windows 2016 server and Windows 1=
0)
>
> For your setup (30 VMs) I would rather go with RAID10 SAS disks and at=
> least a dual 10Gbit NIC dedicated to the gluster traffic only.
>
> Alex
>
>
> On Sat, Mar 24, 2018 at 1:24 PM, Andy Michielsen
> <andy.michielsen(a)gmail.com <mailto:andy.michielsen@gmail.com>> wrote:
>
> Hello Andrei,
>
> Thank you very much for sharing info on your hardware setup. Very
> informative.
>
> At this moment I have my ovirt engine on our vmware environment
> which is fine for good backup and restore.
>
> I have 4 nodes running now all different in make and model with
> local storage and it works but lacks performance a bit.
>
> But I can get my hands on some old dell=E2=80=99s R415 with 96 Gb =
of ram
> and 2 quadcores and 6 x 1 Gb nic=E2=80=99s. They all come
with 2 x=
146 Gb
> 15000 rpm=E2=80=99s harddisks. This isn=E2=80=99t bad but I
will a=
dd more RAM for
> starters. Also I would like to have some good redundant
storage
> for this too and the servers have limited space to add that.
>
> Hopefully others will also share there setups and expirience like
> you did.
>
> Kind regards.
>
> On 24 Mar 2018, at 10:35, Andrei Verovski <andreil1(a)starlett.lv
> <mailto:andreil1@starlett.lv>> wrote:
>
>> Hi,
>>
>> HL ProLiant DL380, dual Xeon
>> 120 GB RAID L1 for system
>> 2 TB RAID L10 for VM disks
>> 5 VMs, 3 Linux, 2 Windows
>> Total CPU load most of the time is=C2=A0 low, high level of activ=
ity
>> related to disk.
>> Host engine under KVM appliance on SuSE, can be easily moved,
>> backed up, copied, experimented with, etc.
>>
>> You'll have to use servers with more RAM and storage than main.
>> More then one NIC required if some of your VMs are on different
>> subnets, e.g. 1 in internal zone and 2nd on DMZ.
>> For your setup 10 GB NICs + L3 Switch for ovirtmgmt.
>>
>> BTW, I would suggest to have several separate hardware RAIDs
>> unless you have SSD, otherwise limit of the disk system I/O will
>> be a bottleneck. Consider SSD L1 RAID for heavy-loaded databases.=
>>
>> *Please note many cheap SSDs do NOT work reliably with SAS
>> controllers even in SATA mode*.
>>
>> For example, I supposed to use 2 x WD Green SSD configures as
>> RAID L1 for OS.
>> It was possible to install system, yet under heavy load simulated=
>> with iozone disk system freeze, rendering OS unbootable.
>> Same crash was experienced with 512GB KingFast SSD connected to
>> broadcom/AMCC SAS RAID Card.
>>
>>
>> On 03/24/2018 10:33 AM, Andy Michielsen wrote:
>>> Hi all,
>>>
>>> Not sure if this is the place to be asking this but I was wonder=
ing which hardware you all are using and why in order for me to see what =
I would be needing.
>>>
>>> I would like to set up a HA cluster consisting off 3 hosts to be=
able to run 30 vm=E2=80=99s.
>>> The engine, I can run on an other server. The hosts
can be fitte=
d with the storage and share the space through glusterfs. I would
think I=
will be needing at least 3 nic=E2=80=99s but would be able to install ov=
n. (Are 1gb nic=E2=80=99s sufficient ?)
>>>
>>> Any input you guys would like to share would be greatly appricia=
ted.
>>>
>>> Thanks,
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org <mailto:Users@ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users
>>> <http://lists.ovirt.org/mailman/listinfo/users>
>>
>>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org <mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>
>
>
=20
=20
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
=20
--7NT1xWEOC6g4SNggAbskfGEZF9hpEIL4F--
--Otw9dz9x11zAjj8g4jYNmTEKPlPgL4W9o
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iHQEARECADQWIQQH3Gy5jXqjdPfp3qKcaiGA6s70YQUCWrikgRYcaGF3a0B0Ymku
dW5pdmllLmFjLmF0AAoJEJxqIYDqzvRh7oUAoPhxFVG6Rq/9n12MqHFjD/VAIHPI
AKDJjxGOtzvHcCRN/H+h8UgEFBNmfQ==
=e9ez
-----END PGP SIGNATURE-----
--Otw9dz9x11zAjj8g4jYNmTEKPlPgL4W9o--
2:02 a.m.
On Sat, Mar 24, 2018 at 9:33 AM, Andy Michielsen
<andy.michielsen(a)gmail.com> wrote:
Hi all,
Not sure if this is the place to be asking this but I was wondering which hardware you
all are using and why in order for me to see what I would be needing.
I would like to set up a HA cluster consisting off 3 hosts to be able to run 30 vm’s.
The engine, I can run on an other server. The hosts can be fitted with the storage and
share the space through glusterfs. I would think I will be needing at least 3 nic’s but
would be able to install ovn. (Are 1gb nic’s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Hello Andy,
i'm not running hyperconverged setup, but just for reference i'll
describe my setup:
2 clusters (6+2) HPE BL460 G9 with 512 GB of ram each. On the cluster
composed by two nodes we're running self-hosted engine.
The storage backend is FC for both (EMC VNX8000 for the biggest one
and EMC VPLEX with VMAX as disk backend on the smallest one).
Luca
--
"E' assurdo impiegare gli uomini di intelligenza eccellente per fare
calcoli che potrebbero essere affidati a chiunque se si usassero delle
macchine"
Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)
"Internet è la più grande biblioteca del mondo.
Ma il problema è che i libri sono tutti sparsi sul pavimento"
John Allen Paulos, Matematico (1945-vivente)
Luca 'remix_tj' Lorenzetto, http://www.remixtj.net ,
<lorenzetto.luca(a)gmail.com>
2:19 a.m.
Hi Luca,
You have 2 node cluster with 512 GB each host to run engine only?
How many vms are u running at the compute nodes?
Alex
On Mon, Mar 26, 2018, 10:02 Luca 'remix_tj' Lorenzetto <
lorenzetto.luca(a)gmail.com> wrote:
On Sat, Mar 24, 2018 at 9:33 AM, Andy Michielsen
<andy.michielsen(a)gmail.com> wrote:
> Hi all,
>
> Not sure if this is the place to be asking this but I was wondering
which hardware you all are using and why in order for me to see what I
would be needing.
>
> I would like to set up a HA cluster consisting off 3 hosts to be able to
run 30 vm’s.
> The engine, I can run on an other server. The hosts can be fitted with
the storage and share the space through glusterfs. I would think I will be
needing at least 3 nic’s but would be able to install ovn. (Are 1gb nic’s
sufficient ?)
>
> Any input you guys would like to share would be greatly appriciated.
Hello Andy,
i'm not running hyperconverged setup, but just for reference i'll
describe my setup:
2 clusters (6+2) HPE BL460 G9 with 512 GB of ram each. On the cluster
composed by two nodes we're running self-hosted engine.
The storage backend is FC for both (EMC VNX8000 for the biggest one
and EMC VPLEX with VMAX as disk backend on the smallest one).
Luca
--
"E' assurdo impiegare gli uomini di intelligenza eccellente per fare
calcoli che potrebbero essere affidati a chiunque se si usassero delle
macchine"
Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)
"Internet è la più grande biblioteca del mondo.
Ma il problema è che i libri sono tutti sparsi sul pavimento"
John Allen Paulos, Matematico (1945-vivente)
Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , <
lorenzetto.luca(a)gmail.com>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2:27 a.m.
Sorry Andy,
forgot to write the density.
On the first cluster (6 nodes) we're running 225 VMs at the moment
On the second one (2 nodes) we're running 42 VMs + the engine.
This setup is running since last July.
Just for completness, we've just set up another environment with the
same sizing (2 sockets + 512gb ram) and distribution (6+2) but
different hardware (Lenovo x240 M5 IIRC) for hosting production VMs.
At the moment there are few VMs, but we're planning to migrate over
300 VMs. Same storage backend.
Luca
On Mon, Mar 26, 2018 at 9:19 AM, Alex K <rightkicktech(a)gmail.com> wrote:
Hi Luca,
You have 2 node cluster with 512 GB each host to run engine only?
How many vms are u running at the compute nodes?
Alex
On Mon, Mar 26, 2018, 10:02 Luca 'remix_tj' Lorenzetto
<lorenzetto.luca(a)gmail.com> wrote:
>
> On Sat, Mar 24, 2018 at 9:33 AM, Andy Michielsen
> <andy.michielsen(a)gmail.com> wrote:
> > Hi all,
> >
> > Not sure if this is the place to be asking this but I was wondering
> > which hardware you all are using and why in order for me to see what I would
> > be needing.
> >
> > I would like to set up a HA cluster consisting off 3 hosts to be able to
> > run 30 vm’s.
> > The engine, I can run on an other server. The hosts can be fitted with
> > the storage and share the space through glusterfs. I would think I will be
> > needing at least 3 nic’s but would be able to install ovn. (Are 1gb nic’s
> > sufficient ?)
> >
> > Any input you guys would like to share would be greatly appriciated.
>
> Hello Andy,
>
> i'm not running hyperconverged setup, but just for reference i'll
> describe my setup:
>
> 2 clusters (6+2) HPE BL460 G9 with 512 GB of ram each. On the cluster
> composed by two nodes we're running self-hosted engine.
>
> The storage backend is FC for both (EMC VNX8000 for the biggest one
> and EMC VPLEX with VMAX as disk backend on the smallest one).
>
> Luca
>
>
> --
> "E' assurdo impiegare gli uomini di intelligenza eccellente per fare
> calcoli che potrebbero essere affidati a chiunque se si usassero delle
> macchine"
> Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)
>
> "Internet è la più grande biblioteca del mondo.
> Ma il problema è che i libri sono tutti sparsi sul pavimento"
> John Allen Paulos, Matematico (1945-vivente)
>
> Luca 'remix_tj' Lorenzetto, http://www.remixtj.net ,
> <lorenzetto.luca(a)gmail.com>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
--
"E' assurdo impiegare gli uomini di intelligenza eccellente per fare
calcoli che potrebbero essere affidati a chiunque se si usassero delle
macchine"
Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)
"Internet è la più grande biblioteca del mondo.
Ma il problema è che i libri sono tutti sparsi sul pavimento"
John Allen Paulos, Matematico (1945-vivente)
Luca 'remix_tj' Lorenzetto, http://www.remixtj.net ,
<lorenzetto.luca(a)gmail.com>
11:03 a.m.
On 03/24/2018 03:33 AM, Andy Michielsen wrote:
Hi all,
Not sure if this is the place to be asking this but I was wondering which hardware you
all are using and why in order for me to see what I would be needing.
I would like to set up a HA cluster consisting off 3 hosts to be able to run 30 vm’s.
The engine, I can run on an other server. The hosts can be fitted with the storage and
share the space through glusterfs. I would think I will be needing at least 3 nic’s but
would be able to install ovn. (Are 1gb nic’s sufficient ?)
Just because you asked, but not because this is helpful to you....
But first, a comment on "3 hosts to be able to run 30 VMs". The SPM
node shouldn't run a lot of VMs. There are settings (the setting slips
my mind) on the engine to give it a "virtual set" of VMs in order to
keep VMs off of it.
With that said, CPU wise, it doesn't require a lot to run 30 VM's. The
costly thing is memory (in general). So while a cheap set of 3 machines
might handle the CPU requirements of 30 VM's, those cheap machines might
not be able to give you the memory you need (depends). You might be
fine. I mean, there are cheap desktop like machines that do 64G (and
sometimes more). Just something to keep in mind. Memory and storage
will be the most costly items. It's simple math. Linux hosts, of
course, don't necessarily need much memory (or storage). But Windows...
1Gbit NIC's are "ok", but again, depends on storage. Glusterfs is no
speed demon. But you might not need "fast" storage.
Lastly, your setup is just for "fun", right? Otherwise, read on.
Running oVirt 3.6 (this is a production setup)
ovirt engine (manager):
Dell PowerEdge 430, 32G
ovirt cluster nodes:
Dell m1000e 1.1 backplane Blade Enclosure
9 x M630 Blades (2xE5-2669v3, 384GB), 4 iSCSI paths, 4 bonded LAN, all
10GbE, CentOS 7.2
4 x MXL 10/40GbE (2x40Gbit LAN, 2x40Gbit iSCSI SAN to the S4810's)
120 VM's, CentOS 6, CentOS 7, Windows 10 Ent., Windows Server 2012
We've run on as few as 3 nodes.
Network, SAN and Storage (for ovirt Domains):
2 x S4810 (part is used for SAN, part for LAN)
Equallogic dual controller (note: passive/active) PS6610S (84 x 4TB 7.2K
SAS)
Equallogic dual controller (note: passive/active) PS6610X (84 x 1TB 10K SAS
ISO and Export Domains are handled by:
Dell PE R620, 32G, 2x10Gbit LAN, 2x10Gbit iSCSI to the SAN (above),
CentOS 7.4, NFS
What I like:
* Easy setup.
* Relatively good network and storage.
What I don't like:
* 2 "effective" networks, LAN and iSCSI. All networking uses the same
effective path. Would be nice to have more physical isolation for mgmt
vs motion vs VMs. QoS is provided in oVirt, but still, would be nice to
have the full pathways.
* Storage doesn't use active/active controllers, so controller failover
is VERY slow.
* We have a fast storage system, and somewhat slower storage system
(matter of IOPS), neither is SSD, so there isn't a huge difference. No
real redundancy or flexibility.
* vdsm can no longer respond fast enough for the amount of disks defined
(in the event of a new Storage Domain add). We have raised vdsTimeout,
but have not tested yet.
I inherited the "style" above. My recommendation of where to start for
a reasonable production instance, minimum (assumes the S4810's above,
not priced here):
1 x ovirt manager/engine, approx $1500
4 x Dell R620, 2xE5-2660, 768G, 6x10GbE (LAN, Storage, Motion), approx $42K
3 x Nexsan 18P 108TB, approx $96K
While significantly cheaper (by 6 figures), it provides active/active
controllers, storage reliability and flexibility and better network
pathways. Why 4 x nodes? Need at least N+1 for reliability. The extra
4th node is merely capacity. Why 3 x storage? Need at least N+1 for
reliability.
Obviously, you'll still want to back things up and test the ability to
restore components like the ovirt engine from scratch.
Btw, my recommended minimum above is regardless of hypervisor cluster
choice (could be VMware).
1:50 a.m.
Hello Chritopher,
Thank you very much for sharing.
It started out just for fun but now people at work are looking at me to have an
environment to do testing, simulate problems they have encountered, etc.
And more an more off them see the benifits off this. At work we are running vmware but
that was far to expencieve to use it for these test. But as I suspected that was in the
beginning and I knew I had to be able to expand so whenever an old server was
decommisioned from production I converted it to an node. I now have 4 in use and demands
keep growing.
So now I want to ask my boss to invest in new hardware as now people are asking me why I
do not have proper backups and even why the can not use the vm’s when I perform
administrative tasks or upgrades.
So that’s why I’m very inerested in what others are using.
Kind regards.
On 26 Mar 2018, at 18:03, Christopher Cox <ccox(a)endlessnow.com>
wrote:
> On 03/24/2018 03:33 AM, Andy Michielsen wrote:
> Hi all,
> Not sure if this is the place to be asking this but I was wondering which hardware
you all are using and why in order for me to see what I would be needing.
> I would like to set up a HA cluster consisting off 3 hosts to be able to run 30
vm’s.
> The engine, I can run on an other server. The hosts can be fitted with the storage
and share the space through glusterfs. I would think I will be needing at least 3 nic’s
but would be able to install ovn. (Are 1gb nic’s sufficient ?)
Just because you asked, but not because this is helpful to you....
But first, a comment on "3 hosts to be able to run 30 VMs". The SPM node
shouldn't run a lot of VMs. There are settings (the setting slips my mind) on the
engine to give it a "virtual set" of VMs in order to keep VMs off of it.
With that said, CPU wise, it doesn't require a lot to run 30 VM's. The costly
thing is memory (in general). So while a cheap set of 3 machines might handle the CPU
requirements of 30 VM's, those cheap machines might not be able to give you the memory
you need (depends). You might be fine. I mean, there are cheap desktop like machines
that do 64G (and sometimes more). Just something to keep in mind. Memory and storage
will be the most costly items. It's simple math. Linux hosts, of course, don't
necessarily need much memory (or storage). But Windows...
1Gbit NIC's are "ok", but again, depends on storage. Glusterfs is no speed
demon. But you might not need "fast" storage.
Lastly, your setup is just for "fun", right? Otherwise, read on.
Running oVirt 3.6 (this is a production setup)
ovirt engine (manager):
Dell PowerEdge 430, 32G
ovirt cluster nodes:
Dell m1000e 1.1 backplane Blade Enclosure
9 x M630 Blades (2xE5-2669v3, 384GB), 4 iSCSI paths, 4 bonded LAN, all 10GbE, CentOS 7.2
4 x MXL 10/40GbE (2x40Gbit LAN, 2x40Gbit iSCSI SAN to the S4810's)
120 VM's, CentOS 6, CentOS 7, Windows 10 Ent., Windows Server 2012
We've run on as few as 3 nodes.
Network, SAN and Storage (for ovirt Domains):
2 x S4810 (part is used for SAN, part for LAN)
Equallogic dual controller (note: passive/active) PS6610S (84 x 4TB 7.2K SAS)
Equallogic dual controller (note: passive/active) PS6610X (84 x 1TB 10K SAS
ISO and Export Domains are handled by:
Dell PE R620, 32G, 2x10Gbit LAN, 2x10Gbit iSCSI to the SAN (above), CentOS 7.4, NFS
What I like:
* Easy setup.
* Relatively good network and storage.
What I don't like:
* 2 "effective" networks, LAN and iSCSI. All networking uses the same
effective path. Would be nice to have more physical isolation for mgmt vs motion vs VMs.
QoS is provided in oVirt, but still, would be nice to have the full pathways.
* Storage doesn't use active/active controllers, so controller failover is VERY
slow.
* We have a fast storage system, and somewhat slower storage system (matter of IOPS),
neither is SSD, so there isn't a huge difference. No real redundancy or flexibility.
* vdsm can no longer respond fast enough for the amount of disks defined (in the event of
a new Storage Domain add). We have raised vdsTimeout, but have not tested yet.
I inherited the "style" above. My recommendation of where to start for a
reasonable production instance, minimum (assumes the S4810's above, not priced here):
1 x ovirt manager/engine, approx $1500
4 x Dell R620, 2xE5-2660, 768G, 6x10GbE (LAN, Storage, Motion), approx $42K
3 x Nexsan 18P 108TB, approx $96K
While significantly cheaper (by 6 figures), it provides active/active controllers,
storage reliability and flexibility and better network pathways. Why 4 x nodes? Need at
least N+1 for reliability. The extra 4th node is merely capacity. Why 3 x storage? Need
at least N+1 for reliability.
Obviously, you'll still want to back things up and test the ability to restore
components like the ovirt engine from scratch.
Btw, my recommended minimum above is regardless of hypervisor cluster choice (could be
VMware).
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
3:42 p.m.
On Mon, Mar 26, 2018, 7:04 PM Christopher Cox <ccox(a)endlessnow.com> wrote:
On 03/24/2018 03:33 AM, Andy Michielsen wrote:
> Hi all,
>
> Not sure if this is the place to be asking this but I was wondering
which hardware you all are using and why in order for me to see what I
would be needing.
>
> I would like to set up a HA cluster consisting off 3 hosts to be able to
run 30 vm’s.
> The engine, I can run on an other server. The hosts can be fitted with
the storage and share the space through glusterfs. I would think I will be
needing at least 3 nic’s but would be able to install ovn. (Are 1gb nic’s
sufficient ?)
Just because you asked, but not because this is helpful to you....
But first, a comment on "3 hosts to be able to run 30 VMs". The SPM
node shouldn't run a lot of VMs. There are settings (the setting slips
my mind) on the engine to give it a "virtual set" of VMs in order to
keep VMs off of it.
With that said, CPU wise, it doesn't require a lot to run 30 VM's. The
costly thing is memory (in general). So while a cheap set of 3 machines
might handle the CPU requirements of 30 VM's, those cheap machines might
not be able to give you the memory you need (depends). You might be
fine. I mean, there are cheap desktop like machines that do 64G (and
sometimes more). Just something to keep in mind. Memory and storage
will be the most costly items. It's simple math. Linux hosts, of
course, don't necessarily need much memory (or storage). But Windows...
1Gbit NIC's are "ok", but again, depends on storage. Glusterfs is no
speed demon. But you might not need "fast" storage.
Lastly, your setup is just for "fun", right? Otherwise, read on.
Running oVirt 3.6 (this is a production setup)
ovirt engine (manager):
Dell PowerEdge 430, 32G
ovirt cluster nodes:
Dell m1000e 1.1 backplane Blade Enclosure
9 x M630 Blades (2xE5-2669v3, 384GB), 4 iSCSI paths, 4 bonded LAN, all
10GbE, CentOS 7.2
4 x MXL 10/40GbE (2x40Gbit LAN, 2x40Gbit iSCSI SAN to the S4810's)
120 VM's, CentOS 6, CentOS 7, Windows 10 Ent., Windows Server 2012
We've run on as few as 3 nodes.
Network, SAN and Storage (for ovirt Domains):
2 x S4810 (part is used for SAN, part for LAN)
Equallogic dual controller (note: passive/active) PS6610S (84 x 4TB 7.2K
SAS)
Equallogic dual controller (note: passive/active) PS6610X (84 x 1TB 10K SAS
ISO and Export Domains are handled by:
Dell PE R620, 32G, 2x10Gbit LAN, 2x10Gbit iSCSI to the SAN (above),
CentOS 7.4, NFS
What I like:
* Easy setup.
* Relatively good network and storage.
What I don't like:
* 2 "effective" networks, LAN and iSCSI. All networking uses the same
effective path. Would be nice to have more physical isolation for mgmt
vs motion vs VMs. QoS is provided in oVirt, but still, would be nice to
have the full pathways.
* Storage doesn't use active/active controllers, so controller failover
is VERY slow.
* We have a fast storage system, and somewhat slower storage system
(matter of IOPS), neither is SSD, so there isn't a huge difference. No
real redundancy or flexibility.
* vdsm can no longer respond fast enough for the amount of disks defined
(in the event of a new Storage Domain add). We have raised vdsTimeout,
but have not tested yet.
We have substantially changed and improved VDSM for better scale since 3.6.
How many disks are defined, in how many storage domains and LUNs?
(also the OS itself has improved).
I inherited the "style" above. My recommendation of where
to start for
a reasonable production instance, minimum (assumes the S4810's above,
not priced here):
1 x ovirt manager/engine, approx $1500
What about high availability for the engine?
4 x Dell R620, 2xE5-2660, 768G, 6x10GbE (LAN, Storage, Motion), approx $42K
3 x Nexsan 18P 108TB, approx $96K
Alternatively, how many reasonable SSDs can you buy? Samsing 860 EVO, 4TB
costs in Amazon (US) $1300. You could buy tens (70+) of those and be left
with some change.
Can you instead use them in a fast storage setup?
https://www.backblaze.com/blog/open-source-data-storage-server/ for example
is interesting.
While significantly cheaper (by 6 figures), it provides
active/active
controllers, storage reliability and flexibility and better network
pathways. Why 4 x nodes? Need at least N+1 for reliability. The extra
4th node is merely capacity. Why 3 x storage? Need at least N+1 for
reliability.
Are they running in some cluster?
Obviously, you'll still want to back things up and test the
ability to
restore components like the ovirt engine from scratch.
+1.
Y.
Btw, my recommended minimum above is regardless of hypervisor
cluster
choice (could be VMware).
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2:20 p.m.
We run Dell Poweredge R720s and R730s with 32 GB of RAM and quad Xeon
processors. Storage is provided by Dell MD3800i and Promise arrays
using iSCSI. The network is all 10 gigabit interfaces using 802.3ad
bonds. We actually just upgraded from 1 gigabit NICs since there were
some performance issues with storage causing high IOwait on VMs. I'd
recommend avoiding 1 gigabit if you can.
On 3/24/18 4:33 AM, Andy Michielsen wrote:
Hi all,
Not sure if this is the place to be asking this but I was wondering which hardware you
all are using and why in order for me to see what I would be needing.
I would like to set up a HA cluster consisting off 3 hosts to be able to run 30 vm’s.
The engine, I can run on an other server. The hosts can be fitted with the storage and
share the space through glusterfs. I would think I will be needing at least 3 nic’s but
would be able to install ovn. (Are 1gb nic’s sufficient ?)
Any input you guys would like to share would be greatly appriciated.
Thanks,
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
3:10 p.m.
Lots of great hardware knowledge in this thread! I'm also making the move
to 10Gb.
I'm adding a 3rd host to my deployment and moving to Glusterfs on 3 nodes,
from my current NFS share on a separate storage server.
Each of the 3 nodes has dual E5-2640 V4s with128Gb ram. I have some
hardware choices I would love some advice about:
- Should I use Intel X520-DA2 or X710-DA2 nics for the storage network? No
significant price difference. The hosts are running oVirt Node 4.2. I
hope to use them in bridge mode so that I don't need a 10Gbe switch. I do
have a single 10Gbe port left on my router.
- the hosts have 12Gbps 520i SAS cards, should I spec 6 or 12 Gbps SSD
drives? Here there is a large price difference, also large difference
between Enterprise performance, Enterprise mainstream, and Enterprise
entry. I'm not sure how to estimate the value of those different options
in a Glusterfs deployment.
The workload is pretty I/O intensive with fairly small read/write
operations (under 128Kb) on windows VMs.
Any obvious weak links with this plan?
On Sat, Mar 31, 2018, 12:27 PM Michael Watters, <wattersm(a)watters.ws> wrote:
We run Dell Poweredge R720s and R730s with 32 GB of RAM and quad
Xeon
processors. Storage is provided by Dell MD3800i and Promise arrays
using iSCSI. The network is all 10 gigabit interfaces using 802.3ad
bonds. We actually just upgraded from 1 gigabit NICs since there were
some performance issues with storage causing high IOwait on VMs. I'd
recommend avoiding 1 gigabit if you can.
On 3/24/18 4:33 AM, Andy Michielsen wrote:
> Hi all,
>
> Not sure if this is the place to be asking this but I was wondering
which hardware you all are using and why in order for me to see what I
would be needing.
>
> I would like to set up a HA cluster consisting off 3 hosts to be able to
run 30 vm’s.
> The engine, I can run on an other server. The hosts can be fitted with
the storage and share the space through glusterfs. I would think I will be
needing at least 3 nic’s but would be able to install ovn. (Are 1gb nic’s
sufficient ?)
>
> Any input you guys would like to share would be greatly appriciated.
>
> Thanks,
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2576
days inactive
2583
days old
14 comments
10 participants
participants (10)
-
Alex K -
Andrei Verovski -
Andy Michielsen -
Christopher Cox -
Juan Pablo -
Luca 'remix_tj' Lorenzetto -
Michael Watters -
Richard Neuboeck -
Vincent Royer -
Yaniv Kaul