I have been fiddling with the drive settings in the windows guest and have been able to fix a lot of the issue.  I went into Device manager -> Disk Drives -> (right click) QEMU QEMU HARDDISK -> Policies and checked "Turn off Windows Write-Cache buffer flushing on the device"

I am not sure how safe this is to do in a virtual machine, but my 10Gb transfer speed is now much more stable.  Occasionally it dips to the low 100 MB/s or so, but generally stays at 200-300 MB/s.

On Thu, Nov 30, 2017 at 10:39 PM, Wesley Stewart <wstewart3@gmail.com> wrote:
I was curious if anyone else has seen this or had any suggestions.

I have recently began playing around with my two servers (Freenas Box and oVirt box) and their 10Gb Ethernet ports.

I can access the Freenas SMB share over the 10Gb port without issue and I have been playing around with the capabilities.  After finding out that my Linux Raid (MDADM) mirror is having horrible write performance, I decided to plug in an NVMe drive that I had lying around and check out its performance.

For my first test,I added the NVMe drive as a passthrough device to a Windows guest and was able to transfer to and from Freenas box without issue.  Speeds were typically ~350-400 MB/s but could drop down to 250 MB/s or so, and would top out around 525 MB/s, pretty slick!

For my second test, I decided to mount the NVMe drive on the CentOS ovirt host and make it a local datastore.  I migrated my Windows Guest to it, and decided to test and see what sort of transfer speeds I got and saw some weird results...

Writing TOO the NAS worked about the same.  Perhaps a little slower but at least had a steady 250-300 MB/s.

Writing to the Windows Guest had a very "Fast and then slow, fast and then slow" type of throughput.  I took a few screenshots:

(Writing TO the NAS was fairly consistent)

Inline image 1
https://i.imgur.com/jWNNvfp.png


(Writing TO the Windows Guest on NVMe storage)
Sometimes these hit the low 10-20 MB/s during the transfer.
Inline image 2
https://i.imgur.com/aizG6n0.png

Inline image 3
https://i.imgur.com/AjRpR0K.png