<div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr">On Mon, Mar 26, 2018, 7:04 PM Christopher Cox <<a href="mailto:ccox@endlessnow.com">ccox@endlessnow.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 03/24/2018 03:33 AM, Andy Michielsen wrote:<br>
> Hi all,<br>
><br>
> Not sure if this is the place to be asking this but I was wondering which hardware you all are using and why in order for me to see what I would be needing.<br>
><br>
> I would like to set up a HA cluster consisting off 3 hosts to be able to run 30 vm’s.<br>
> The engine, I can run on an other server. The hosts can be fitted with the storage and share the space through glusterfs. I would think I will be needing at least 3 nic’s but would be able to install ovn. (Are 1gb nic’s sufficient ?)<br>
<br>
Just because you asked, but not because this is helpful to you....<br>
<br>
But first, a comment on "3 hosts to be able to run 30 VMs". The SPM<br>
node shouldn't run a lot of VMs. There are settings (the setting slips<br>
my mind) on the engine to give it a "virtual set" of VMs in order to<br>
keep VMs off of it.<br>
<br>
With that said, CPU wise, it doesn't require a lot to run 30 VM's. The<br>
costly thing is memory (in general). So while a cheap set of 3 machines<br>
might handle the CPU requirements of 30 VM's, those cheap machines might<br>
not be able to give you the memory you need (depends). You might be<br>
fine. I mean, there are cheap desktop like machines that do 64G (and<br>
sometimes more). Just something to keep in mind. Memory and storage<br>
will be the most costly items. It's simple math. Linux hosts, of<br>
course, don't necessarily need much memory (or storage). But Windows...<br>
<br>
1Gbit NIC's are "ok", but again, depends on storage. Glusterfs is no<br>
speed demon. But you might not need "fast" storage.<br>
<br>
Lastly, your setup is just for "fun", right? Otherwise, read on.<br>
<br>
<br>
Running oVirt 3.6 (this is a production setup)<br>
<br>
ovirt engine (manager):<br>
Dell PowerEdge 430, 32G<br>
<br>
ovirt cluster nodes:<br>
Dell m1000e 1.1 backplane Blade Enclosure<br>
9 x M630 Blades (2xE5-2669v3, 384GB), 4 iSCSI paths, 4 bonded LAN, all<br>
10GbE, CentOS 7.2<br>
4 x MXL 10/40GbE (2x40Gbit LAN, 2x40Gbit iSCSI SAN to the S4810's)<br>
<br>
120 VM's, CentOS 6, CentOS 7, Windows 10 Ent., Windows Server 2012<br>
We've run on as few as 3 nodes.<br>
<br>
Network, SAN and Storage (for ovirt Domains):<br>
2 x S4810 (part is used for SAN, part for LAN)<br>
Equallogic dual controller (note: passive/active) PS6610S (84 x 4TB 7.2K<br>
SAS)<br>
Equallogic dual controller (note: passive/active) PS6610X (84 x 1TB 10K SAS<br>
<br>
ISO and Export Domains are handled by:<br>
Dell PE R620, 32G, 2x10Gbit LAN, 2x10Gbit iSCSI to the SAN (above),<br>
CentOS 7.4, NFS<br>
<br>
What I like:<br>
* Easy setup.<br>
* Relatively good network and storage.<br>
<br>
What I don't like:<br>
* 2 "effective" networks, LAN and iSCSI. All networking uses the same<br>
effective path. Would be nice to have more physical isolation for mgmt<br>
vs motion vs VMs. QoS is provided in oVirt, but still, would be nice to<br>
have the full pathways.<br>
* Storage doesn't use active/active controllers, so controller failover<br>
is VERY slow.<br>
* We have a fast storage system, and somewhat slower storage system<br>
(matter of IOPS), neither is SSD, so there isn't a huge difference. No<br>
real redundancy or flexibility.<br>
* vdsm can no longer respond fast enough for the amount of disks defined<br>
(in the event of a new Storage Domain add). We have raised vdsTimeout,<br>
but have not tested yet.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">We have substantially changed and improved VDSM for better scale since 3.6.</div><div dir="auto">How many disks are defined, in how many storage domains and LUNs? </div><div dir="auto">(also the OS itself has improved). </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I inherited the "style" above. My recommendation of where to start for<br>
a reasonable production instance, minimum (assumes the S4810's above,<br>
not priced here):<br>
<br>
1 x ovirt manager/engine, approx $1500<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">What about high availability for the engine? </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
4 x Dell R620, 2xE5-2660, 768G, 6x10GbE (LAN, Storage, Motion), approx $42K<br>
3 x Nexsan 18P 108TB, approx $96K<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">Alternatively, how many reasonable SSDs can you buy? Samsing 860 EVO, 4TB costs in Amazon (US) <span class="money">$1300</span>. You could buy tens (70+) of those and be left with some change. </div><div dir="auto">Can you instead use them in a fast storage setup? <a href="https://www.backblaze.com/blog/open-source-data-storage-server/">https://www.backblaze.com/blog/open-source-data-storage-server/</a> for example is interesting. </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
While significantly cheaper (by 6 figures), it provides active/active<br>
controllers, storage reliability and flexibility and better network<br>
pathways. Why 4 x nodes? Need at least N+1 for reliability. The extra<br>
4th node is merely capacity. Why 3 x storage? Need at least N+1 for<br>
reliability.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">Are they running in some cluster? </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Obviously, you'll still want to back things up and test the ability to<br>
restore components like the ovirt engine from scratch.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">+1.</div><div dir="auto">Y.</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Btw, my recommended minimum above is regardless of hypervisor cluster<br>
choice (could be VMware).<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank" rel="noreferrer">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</blockquote></div></div></div>