<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Hi,</p>
<p><br>
</p>
<p>Can you put some numbers ? what tests are you doing ?</p>
<p>Im running oVirt with Gluster without performance issues, but im running replica 2 all SSDs.</p>
<p>Gluster logs might help too.</p>
<p><br>
</p>
<div id="x_Signature"><br>
<div class="x_ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> users-bounces@ovirt.org <users-bounces@ovirt.org> on behalf of Chris Boot <bootc@bootc.net><br>
<b>Sent:</b> Monday, June 19, 2017 5:46:08 PM<br>
<b>To:</b> oVirt users<br>
<b>Subject:</b> [ovirt-users] Very poor GlusterFS performance</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi folks,<br>
<br>
I have 3x servers in a "hyper-converged" oVirt 4.1.2 + GlusterFS 3.10<br>
configuration. My VMs run off a replica 3 arbiter 1 volume comprised of<br>
6 bricks, which themselves live on two SSDs in each of the servers (one<br>
brick per SSD). The bricks are XFS on LVM thin volumes straight onto the<br>
SSDs. Connectivity is 10G Ethernet.<br>
<br>
Performance within the VMs is pretty terrible. I experience very low<br>
throughput and random IO is really bad: it feels like a latency issue.<br>
On my oVirt nodes the SSDs are not generally very busy. The 10G network<br>
seems to run without errors (iperf3 gives bandwidth measurements of >=<br>
9.20 Gbits/sec between the three servers).<br>
<br>
To put this into perspective: I was getting better behaviour from NFS4<br>
on a gigabit connection than I am with GlusterFS on 10G: that doesn't<br>
feel right at all.<br>
<br>
My volume configuration looks like this:<br>
<br>
Volume Name: vmssd<br>
Type: Distributed-Replicate<br>
Volume ID: d5a5ddd1-a140-4e0d-b514-701cfe464853<br>
Status: Started<br>
Snapshot Count: 0<br>
Number of Bricks: 2 x (2 + 1) = 6<br>
Transport-type: tcp<br>
Bricks:<br>
Brick1: ovirt3:/gluster/ssd0_vmssd/brick<br>
Brick2: ovirt1:/gluster/ssd0_vmssd/brick<br>
Brick3: ovirt2:/gluster/ssd0_vmssd/brick (arbiter)<br>
Brick4: ovirt3:/gluster/ssd1_vmssd/brick<br>
Brick5: ovirt1:/gluster/ssd1_vmssd/brick<br>
Brick6: ovirt2:/gluster/ssd1_vmssd/brick (arbiter)<br>
Options Reconfigured:<br>
nfs.disable: on<br>
transport.address-family: inet6<br>
performance.quick-read: off<br>
performance.read-ahead: off<br>
performance.io-cache: off<br>
performance.stat-prefetch: off<br>
performance.low-prio-threads: 32<br>
network.remote-dio: off<br>
cluster.eager-lock: enable<br>
cluster.quorum-type: auto<br>
cluster.server-quorum-type: server<br>
cluster.data-self-heal-algorithm: full<br>
cluster.locking-scheme: granular<br>
cluster.shd-max-threads: 8<br>
cluster.shd-wait-qlength: 10000<br>
features.shard: on<br>
user.cifs: off<br>
storage.owner-uid: 36<br>
storage.owner-gid: 36<br>
features.shard-block-size: 128MB<br>
performance.strict-o-direct: on<br>
network.ping-timeout: 30<br>
cluster.granular-entry-heal: enable<br>
<br>
I would really appreciate some guidance on this to try to improve things<br>
because at this rate I will need to reconsider using GlusterFS altogether.<br>
<br>
Cheers,<br>
Chris<br>
<br>
-- <br>
Chris Boot<br>
bootc@bootc.net<br>
_______________________________________________<br>
Users mailing list<br>
Users@ovirt.org<br>
<a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</div>
</span></font>
</body>
</html>