I have a problem in a productive system. We use three Servers which have a Raid 5 with four Harddrives SATA 8 TB 7200 rpms. The Systems are interconnected for Glusterfs with 1 GB/s. 

The Network Load between all three servers are between 5 and 8 % of the network. The Glusterfs has a separate VLAN which complete isolated from other networks. 

Find attached a small drawing. 

On that server we rum as HA and there are 8 VMs installed. Most of the VMs working well but if there is high load on the system some of the VM can't write the Data in a "good" speed on the disks. The Glusterfs is configured as Replica on the Raid5 disk which is as Jbob in Glusterfs defined. 

If i take a look on the iostats i can see that the Harddisk have round 50 open connections some of the glusterfs and a lot of the vms.  In side the VMs i can see long iowaiting times from 300 up to 600 ms. With is on case of a Posgresql database very long. 

Should we configure the Raid5 controller out that the controller only should 4 disks with 8 TB and Glusterfs should connect directly to the /dev/sd[b-e]? 

What do you think? Will SSD HDD will help on performance but the same config with Raid5 and Jbod could be the bigger problem. 

Thanks

Marcel