<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Mon, Nov 20, 2017 at 5:22 PM Jiří Sléžka <<a href="mailto:jiri.slezka@slu.cz">jiri.slezka@slu.cz</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
I am trying realize why is exporting of vm to export storage on<br>
glusterfs such slow.<br>
<br>
I am using oVirt and RHV, both instalations on version 4.1.7.<br>
<br>
Hosts have dedicated nics for rhevm network - 1gbps, data storage itself<br>
is on FC.<br>
<br>
GlusterFS cluster lives separate on 4 dedicated hosts. It has slow disks<br>
but I can achieve about 200-400mbit throughput in other applications (we<br>
are using it for "cold" data, backups mostly).<br>
<br>
I am using this glusterfs cluster as backend for export storage. When I<br>
am exporting vm I can see only about 60-80mbit throughput.<br>
<br>
What could be the bottleneck here?<br>
<br>
Could it be qemu-img utility?<br>
<br>
vdsm 97739 0.3 0.0 354212 29148 ? S<l 15:43 0:06<br>
/usr/bin/qemu-img convert -p -t none -T none -f raw<br>
/rhev/data-center/2ff6d0ee-a10b-473d-b77c-be9149945f5f/ff3cd56a-1005-4426-8137-8f422c0b47c1/images/ba42cbcc-c068-4df8-af3d-00f2077b1e27/c57acd5f-d6cf-48cc-ad0c-4a7d979c0c1e<br>
-O raw<br>
/rhev/data-center/mnt/glusterSD/10.20.30.41:_rhv__export/81094499-a392-4ea2-b081-7c6288fbb636/images/ba42cbcc-c068-4df8-af3d-00f2077b1e27/c57acd5f-d6cf-48cc-ad0c-4a7d979c0c1e<br>
<br>
Any idea how to make it work faster or what throughput should I expected?<br></blockquote><div><br></div><div>gluster storage operations are using fuse mount - so every write:</div><div>- travel to the kernel</div><div>- travel back to the gluster fuse helper process</div><div>- travel to all 3 replicas - replication is done on client side</div><div>- return to kernel when all writes succeeded</div><div>- return to caller</div><div><br></div><div>So gluster will never set any speed record.</div><div><br></div><div>Additionally, you are copying from raw lv on FC - qemu-img cannot do anything</div><div>smart and avoid copying unused clusters. Instead if copies gigabytes of zeros</div><div>from FC.</div><div><br></div><div>However 7.5-10 MiB/s sounds too slow.</div><div><br></div><div>I would try to test with dd - how much time it takes to copy</div><div>the same image from FC to your gluster storage?</div><div><br></div><div>dd if=/rhev/data-center/2ff6d0ee-a10b-473d-b77c-be9149945f5f/ff3cd56a-1005-4426-8137-8f422c0b47c1/images/ba42cbcc-c068-4df8-af3d-00f2077b1e27/c57acd5f-d6cf-48cc-ad0c-4a7d979c0c1e of=/rhev/data-center/mnt/glusterSD/10.20.30.41:_rhv__export/81094499-a392-4ea2-b081-7c6288fbb636/__test__ bs=8M oflag=direct status=progress</div><div><br></div><div>If dd can do this faster, please ask on qemu-discuss mailing list:</div><div><a href="https://lists.nongnu.org/mailman/listinfo/qemu-discuss">https://lists.nongnu.org/mailman/listinfo/qemu-discuss</a><br></div><div><br></div><div>If both give similar results, I think asking in gluster mailing list</div><div>about this can help. Maybe your gluster setup can be optimized.</div><div><br></div><div>Nir</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Cheers,<br>
<br>
Jiri<br>
<br>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</blockquote></div></div>