Terrible Disk Performance on Windows 10 VM
I recently installed a Windows 10 VM under oVirt 4.4.5.11-1.el8 Also installed the drivers using "virtio-win-1.9.16.iso" (Then re-installed them after updates just in case it helped) I found a similar complaint with VMWare (https://communities.vmware.com/t5/VMware-Workstation-Pro/vmware-workstation-...) So I looked into that and made a registry change for the AHCI controller, as well as setting the 'viodiskcache' to write-back (Seen in another thread on here) - those two changes seemed to help.. marginally. But not much at all. When I do just about anything, disk usage spikes to 100% and stays there for quite a while. Write speeds rarely break 100kb/sec. Not even sure what to look for next. My Linux VMs don't seem to have this issue and the host it's running on is barely working at all. CPU and memory stay close to unused. oVirt didn't show a lot, but in task manager in the Windows VM - you can see disk queue just pegged completely. I've given the VM 6GB of RAM, so that's not it. I even turned off paging in the Windows VM as well, to no avail. This is an example of disk usage, just opening 'Groove Music' for the first time. https://i.postimg.cc/FRLq28Mw/Disk-Activity.png Any ideas? :)
On Wed, Jul 28, 2021 at 2:51 PM <regloff@gmail.com> wrote:
I recently installed a Windows 10 VM under oVirt 4.4.5.11-1.el8
Also installed the drivers using "virtio-win-1.9.16.iso" (Then re-installed them after updates just in case it helped)
I found a similar complaint with VMWare ( https://communities.vmware.com/t5/VMware-Workstation-Pro/vmware-workstation-... )
So I looked into that and made a registry change for the AHCI controller, as well as setting the 'viodiskcache' to write-back (Seen in another thread on here) - those two changes seemed to help.. marginally. But not much at all.
When I do just about anything, disk usage spikes to 100% and stays there for quite a while. Write speeds rarely break 100kb/sec.
Not even sure what to look for next. My Linux VMs don't seem to have this issue and the host it's running on is barely working at all. CPU and memory stay close to unused. oVirt didn't show a lot, but in task manager in the Windows VM - you can see disk queue just pegged completely.
I've given the VM 6GB of RAM, so that's not it. I even turned off paging in the Windows VM as well, to no avail.
This is an example of disk usage, just opening 'Groove Music' for the first time.
https://i.postimg.cc/FRLq28Mw/Disk-Activity.png
Any ideas? :)
Hello, I've got a number of Windows 7/8/10 and 16 VMs running on a number of different oVirt clusters, ranging from single host setups (NFS localhost storage) up to multiple-host setups (Gluster storage) and I'm not seeing any performance issues. Can you share additional information concerning the host, storage and VM configuration? - Gilboa
This is a simple one desktop setup I use at home for being a nerd :) So it's a single host 'cluster' using local storage. Host Info: CentOS Linux 8 - 4.18.0-305.10.2.el8_4.x86_64 (I keep fairly well updated) Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz [Kaby Lake] {Skylake}, 14nm The disk it's on is: WDC WD40EZRZ-00G (5400 RPM platter disk) - it's not the fastest thing in the world, but it should be sufficient. VM info: Windows 10 Professional (Desktop not server) 6144 MB of RAM 2 Virtual CPUS - Some settings I ran across for 'Performance' mode and a couple I had seen on some similar issues (the similar issues were quite dated) Running in headless mode I/O Threads enabled = 1 Multi-Queues enabled Virt-IO-SCSI enabled Random Number generator enabled Added a custom property of 'viodiskcache' = writeback (Didn't seem to make any significant improvement) As I type this though - I was going to add this link as it's what I followed to install the storage driver during the Windows install and then in the OS after that: https://access.redhat.com/solutions/17463 I did notice something.. it says to create a new VM with the 'VirtIO disk interface' and I just noted my VM is setup as 'SATA'. Perhaps that is it. This is just my first attempt at running something other than a Linux Distro under oVirt. When I first installed the Windows guest, I didn't have the Virt-IO package downloaded initially. When Windows couldn't find a storage driver, I found this info out. I think I'll deploy a new Windows guest and try the 'VirtIO-SCSI' interface and see if my performance is any better. It's just a default install of Windows at this point, so that'll be easy. :) Will update this thread either way!
On Fri, Jul 30, 2021 at 4:17 PM <regloff@gmail.com> wrote:
I think I'll deploy a new Windows guest and try the 'VirtIO-SCSI' interface and see if my performance is any better. It's just a default install of Windows at this point, so that'll be easy. :)
I think it should work also this way: . hot add a disk to your Windows VM with size small as you want (eg 1Gb) and configure it as virtio-scsi. . in windows os when it asks for a driver, you provide what you need (the virtio-win iso) and let windows configure . eventually restart windows to verify all is ok . shutdown the vm and set the boot disk as virtio-scsi . power on the vm . windows should auto configure it and perhaps reboot and you are then done . remove the second disk Gianluca
Re-installation using the 'VirtIO-SCSI' interface along with the appropriate driver installs took care of the issue! It's still a little bit slow - but what I would expect for a 5400 RPM drive :) The performance had improved greatly! Thanks for the suggestions!
You should also know that preallocated disks are about 20-30% faster than thin disks in read/write speed (this is according to my measurements with caching turned off)
When I did the VM this time, I did pre-allocate all of the space. The first time, I was just curious if Windows would even run ok under oVirt and it does appear it will. Next.. I need to get a couple Solaris VMs actually working. I've seen some 'hacks' out there that others have reported success with. Just haven't had much time to mess with that also.
You found the issue! VirtIOSCSI can only do its magic, when it's actually used. And once the boot disks was running using AHCI emulation it's a little hard to make it "re-attach" to SCSI. I am pretty sure it could be done, like you could make Windows disks switch from IDE to SATA/AHCI with a bit of twiddling in the registry using recovery mode. I think I managed to feed the Windows installer a floppy image with the VirtIO drivers at one point, but it took me half a workday I think... That is one of the reasons I like importing my Windows VMs for oVirt from VirtualBox. And it doesn't seem to like the OVAs VMware produces.
On Fri, Jul 30, 2021 at 5:17 PM <regloff@gmail.com> wrote:
This is a simple one desktop setup I use at home for being a nerd :)
So it's a single host 'cluster' using local storage.
Sorry for the late reply. Define: local. NFS, Gluster or ISCSI? - Gilboa
Host Info: CentOS Linux 8 - 4.18.0-305.10.2.el8_4.x86_64 (I keep fairly well updated) Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz [Kaby Lake] {Skylake}, 14nm The disk it's on is: WDC WD40EZRZ-00G (5400 RPM platter disk) - it's not the fastest thing in the world, but it should be sufficient.
VM info: Windows 10 Professional (Desktop not server) 6144 MB of RAM 2 Virtual CPUS - Some settings I ran across for 'Performance' mode and a couple I had seen on some similar issues (the similar issues were quite dated) Running in headless mode I/O Threads enabled = 1 Multi-Queues enabled Virt-IO-SCSI enabled Random Number generator enabled Added a custom property of 'viodiskcache' = writeback (Didn't seem to make any significant improvement)
As I type this though - I was going to add this link as it's what I followed to install the storage driver during the Windows install and then in the OS after that:
https://access.redhat.com/solutions/17463
I did notice something.. it says to create a new VM with the 'VirtIO disk interface' and I just noted my VM is setup as 'SATA'.
Perhaps that is it. This is just my first attempt at running something other than a Linux Distro under oVirt. When I first installed the Windows guest, I didn't have the Virt-IO package downloaded initially. When Windows couldn't find a storage driver, I found this info out.
I think I'll deploy a new Windows guest and try the 'VirtIO-SCSI' interface and see if my performance is any better. It's just a default install of Windows at this point, so that'll be easy. :)
Will update this thread either way! _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4YC5E3MRPKJPFA...
I believe "local" in this context is using the local ovirt Host OS disk as VM storage ie "local storage". The disk info mentioned "WDC WD40EZRZ-00G" = a single 4TB disk, at 5400RPM. OP the seek time on that disk will be high. How many VMs are running off it? Are you able to try other storage? If you could run some monitoring on the host, I'd expect to see low throughput and high delay on that disk. Regards, Tony Pearce On Tue, 3 Aug 2021 at 16:54, Gilboa Davara <gilboad@gmail.com> wrote:
On Fri, Jul 30, 2021 at 5:17 PM <regloff@gmail.com> wrote:
This is a simple one desktop setup I use at home for being a nerd :)
So it's a single host 'cluster' using local storage.
Sorry for the late reply. Define: local. NFS, Gluster or ISCSI?
- Gilboa
Host Info: CentOS Linux 8 - 4.18.0-305.10.2.el8_4.x86_64 (I keep fairly well updated) Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz [Kaby Lake] {Skylake}, 14nm The disk it's on is: WDC WD40EZRZ-00G (5400 RPM platter disk) - it's not the fastest thing in the world, but it should be sufficient.
VM info: Windows 10 Professional (Desktop not server) 6144 MB of RAM 2 Virtual CPUS - Some settings I ran across for 'Performance' mode and a couple I had seen on some similar issues (the similar issues were quite dated) Running in headless mode I/O Threads enabled = 1 Multi-Queues enabled Virt-IO-SCSI enabled Random Number generator enabled Added a custom property of 'viodiskcache' = writeback (Didn't seem to make any significant improvement)
As I type this though - I was going to add this link as it's what I followed to install the storage driver during the Windows install and then in the OS after that:
https://access.redhat.com/solutions/17463
I did notice something.. it says to create a new VM with the 'VirtIO disk interface' and I just noted my VM is setup as 'SATA'.
Perhaps that is it. This is just my first attempt at running something other than a Linux Distro under oVirt. When I first installed the Windows guest, I didn't have the Virt-IO package downloaded initially. When Windows couldn't find a storage driver, I found this info out.
I think I'll deploy a new Windows guest and try the 'VirtIO-SCSI' interface and see if my performance is any better. It's just a default install of Windows at this point, so that'll be easy. :)
Will update this thread either way! _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4YC5E3MRPKJPFA...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CIZOXVW2N5ND4A...
Yes - local as in 5400 RPM SATA - standard desktop, slow storage.. :) It's still 'slow' being 5400 RPM SATA, but after setting the new VM to 'VirtIO-SCSI' and loading the driver, the performance is 'as expected'. I don't notice with with the Linux VMs because they don't do anything that requires a lot of disk I/O. Mostly Ansible/Python education and such. https://i.postimg.cc/28f764yb/Untitled.png I actually have some super fast Serial SCSI SSD drives I am going to use in the future. A storage vendor where I worked at ordered a bunch by mistake to upgrade our storage array and then left them sitting on-site for like 9 months. I contacted them to remind them we still had them in our data center and asked if they wanted to come and get them. I joked with our field engineer and told him if they didn't want them, I could find a use for them! He actually contacted his manager who gave us approval to just 'dispose' of them. So I thought why not recycle them? :) I'm in the process of moving soon for a new job. Once I get settled, I'm going to upgrade the storage I use for VMs. Either to those SSDs or maybe a small NAS device. Ideally.. a NAS device that can support Serial SCSI. I'll need to get a controller and a cable for them, but considering the performance... it should be well worth it. And no - I didn't get fired for swiping the drives! Too many years invested in IT for something that stupid and I'm just not that kind of person anyway. I took a position that's a bit more 'administrative' and less technical; but with better pay, so I want to keep my tech skills sharp, just because I enjoy it. This is just a 'home lab' - nothing that supports anything even remotely important. I'm so used to SSD now.. my desktop OS is on SSD, my CentOS machine is on SSD.. putting Windows on spinning platters is just painful anymore!
On Tue, Aug 3, 2021 at 3:18 PM <regloff@gmail.com> wrote:
Yes - local as in 5400 RPM SATA - standard desktop, slow storage.. :)
It's still 'slow' being 5400 RPM SATA, but after setting the new VM to 'VirtIO-SCSI' and loading the driver, the performance is 'as expected'. I don't notice with with the Linux VMs because they don't do anything that requires a lot of disk I/O. Mostly Ansible/Python education and such.
https://i.postimg.cc/28f764yb/Untitled.png
I actually have some super fast Serial SCSI SSD drives I am going to use in the future. A storage vendor where I worked at ordered a bunch by mistake to upgrade our storage array and then left them sitting on-site for like 9 months. I contacted them to remind them we still had them in our data center and asked if they wanted to come and get them. I joked with our field engineer and told him if they didn't want them, I could find a use for them! He actually contacted his manager who gave us approval to just 'dispose' of them. So I thought why not recycle them? :)
I'm in the process of moving soon for a new job. Once I get settled, I'm going to upgrade the storage I use for VMs. Either to those SSDs or maybe a small NAS device. Ideally.. a NAS device that can support Serial SCSI. I'll need to get a controller and a cable for them, but considering the performance... it should be well worth it. And no - I didn't get fired for swiping the drives! Too many years invested in IT for something that stupid and I'm just not that kind of person anyway. I took a position that's a bit more 'administrative' and less technical; but with better pay, so I want to keep my tech skills sharp, just because I enjoy it.
This is just a 'home lab' - nothing that supports anything even remotely important. I'm so used to SSD now.. my desktop OS is on SSD, my CentOS machine is on SSD.. putting Windows on spinning platters is just painful anymore!
While I do have big oVirt setups running on pure SSD storage, I must admit that Windows (and Linux VMs) are perfectly usable on HDD software RAIDs, *if* everything is configured correctly (and you have a lot of RAM). E.g. I'm typing this message on a Fedora VM running (w/ VFIO + nVidia GPU + USB passthrough) on a pretty beefy 8 y/o Xeon machine with 6 x 2TB MDRAID and ~10 other VMs (including Windows), and unless multiple VMs are trashing the disks, I get near bare-metal performance. (I even run 3D games on this VM...) - Gilboa
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/5EVF6FR7Z46A2P...
On Tue, Aug 3, 2021 at 12:24 PM Tony Pearce <tonyppe@gmail.com> wrote:
I believe "local" in this context is using the local ovirt Host OS disk as VM storage ie "local storage". The disk info mentioned "WDC WD40EZRZ-00G" = a single 4TB disk, at 5400RPM.
OP the seek time on that disk will be high. How many VMs are running off it?
Are you able to try other storage? If you could run some monitoring on the host, I'd expect to see low throughput and high delay on that disk.
Regards,
Tony Pearce
Stupid question: I must be missing something. AFAIR oVirt doesn't support local storage, one needs to choose localhost NFS, single-host-Gluster to localhost-iSCSI. Assuming I'm not mistaken, in my experience the type of storage domain used has a considerable impact on performance, especially latency. E.g. Running Bonnie++ on a Fedora Linux VM, single host GlusterFS has 2-3 times the latency of localhost NFS, and noticeably lower throughput. Hence my question. - Gilboa
On Tue, 3 Aug 2021 at 16:54, Gilboa Davara <gilboad@gmail.com> wrote:
On Fri, Jul 30, 2021 at 5:17 PM <regloff@gmail.com> wrote:
This is a simple one desktop setup I use at home for being a nerd :)
So it's a single host 'cluster' using local storage.
Sorry for the late reply. Define: local. NFS, Gluster or ISCSI?
- Gilboa
Host Info: CentOS Linux 8 - 4.18.0-305.10.2.el8_4.x86_64 (I keep fairly well updated) Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz [Kaby Lake] {Skylake}, 14nm The disk it's on is: WDC WD40EZRZ-00G (5400 RPM platter disk) - it's not the fastest thing in the world, but it should be sufficient.
VM info: Windows 10 Professional (Desktop not server) 6144 MB of RAM 2 Virtual CPUS - Some settings I ran across for 'Performance' mode and a couple I had seen on some similar issues (the similar issues were quite dated) Running in headless mode I/O Threads enabled = 1 Multi-Queues enabled Virt-IO-SCSI enabled Random Number generator enabled Added a custom property of 'viodiskcache' = writeback (Didn't seem to make any significant improvement)
As I type this though - I was going to add this link as it's what I followed to install the storage driver during the Windows install and then in the OS after that:
https://access.redhat.com/solutions/17463
I did notice something.. it says to create a new VM with the 'VirtIO disk interface' and I just noted my VM is setup as 'SATA'.
Perhaps that is it. This is just my first attempt at running something other than a Linux Distro under oVirt. When I first installed the Windows guest, I didn't have the Virt-IO package downloaded initially. When Windows couldn't find a storage driver, I found this info out.
I think I'll deploy a new Windows guest and try the 'VirtIO-SCSI' interface and see if my performance is any better. It's just a default install of Windows at this point, so that'll be easy. :)
Will update this thread either way! _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4YC5E3MRPKJPFA...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CIZOXVW2N5ND4A...
The storage domain lists the type as "Local on Host". It's been a while since I set it up, but I thought the engine deployment had an option for local storage. The VM's reside in a directory under /opt on the Physical Host. https://i.postimg.cc/z3mMNj3J/Capture.png I forget which version I actually deployed.. but it's at 4.4.5.11-1.el8 now. Prior to installing oVirt, I was using KVM, but where I worked at the time migrated from Oracle's old 'OVM' to 'OLVM' which is basically an older version of oVirt with an Oracle icon :) The Oracle version wouldn't allow for local storage in any form. So I was pleasantly surprised that the version of oVirt I installed at home did allow it. I already had a NFS share setup anticipating that I would need to use NFS for my home setup, but didn't actually need it. Thing is.. I'm just spoiled with SSD data transfer/access/write speeds now, lol. I do keep various data I don't access constantly on some spinning disks, like media, ISO files and such. But for OS booting speeds - completely accustomed to SSD! I'll be looking forward to using those SSDs I have. They have been sitting for almost a year now in a drawer.
participants (6)
-
Gianluca Cecchi -
Gilboa Davara -
Patrick Lomakin -
regloff＠gmail.com -
Thomas Hoberg -
Tony Pearce