VM resource allocation and IO Threads
Hello, can anyone give any pointer to deeper information about what in subject and the value for "Num Of IO Threads" configuration, best practices and to-be-expected improvements? I read also here: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/htm... but in some points it seems not so clear to me: eg: If a virtual machine has more than one disk, you can enable or change the number of IO threads to improve performance. but also Red Hat recommends using the default number of IO threads, which is 1. There is also a note about deactivation and activation of disks: does it mean that even if I poweroff the VM and change its config I have to make this step after? Anyone has run benchmarks? Does it make sense if my VM has 3 disks to configure 6 IO threads for example? Do IO threads map to SCSI controllers inside the guest or what? Thanks in advance, Gianluca
From: Gianluca Cecchi <gianluca.cecchi@gmail.com> Subject: [ovirt-users] VM resource allocation and IO Threads Date: October 27, 2017 at 9:26:59 AM CDT To: users =20 Hello, can anyone give any pointer to deeper information about what in = subject and the value for "Num Of IO Threads" configuration, best =
=20 I read also here: = https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/h= tml-single/virtual_machine_management_guide/#Editing_IO_Threads = <https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/=
=20 but in some points it seems not so clear to me: =20 eg: =20 If a virtual machine has more than one disk, you can enable or change =
--Apple-Mail=_D4C1048E-6102-423F-8DB0-C4D43E76E17F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Best explanation I=E2=80=99ve found is = https://wiki.mikejung.biz/KVM_/_Xen#virtio-blk_iothreads_.28x-data-plane.2= 9 If you google a bit, you=E2=80=99ll find some more under QEMU topics, = I saw some discussion of threads and queues in virtio-scsi, but that = seems to be a slightly different thing than this setting. In short, having at least 1 offers advantages for all your VM=E2=80=99s = disks, and if you want to be optimal (at the possible expense of extra = CPU for IO), one per drive attached. There is (currently) no benefit to = having more than 1 thread per drive. =46rom what I can tell, if you have = more drives than threads they share the threads evenly and are = statically assigned to a thread. Seems to be effective at QEMU start, so = you have to change it with the VM down or stop and start it again. I currently enable it on all VMs and assign 1 thread per drive on my = systems. practices and to-be-expected improvements? html-single/virtual_machine_management_guide/#Editing_IO_Threads> the number of IO threads to improve performance.
=20 but also =20 Red Hat recommends using the default number of IO threads, which is 1. =20 There is also a note about deactivation and activation of disks: does = it mean that even if I poweroff the VM and change its config I have to = make this step after? =20 Anyone has run benchmarks? Does it make sense if my VM has 3 disks to configure 6 IO threads for = example? Do IO threads map to SCSI controllers inside the guest or what? =20 Thanks in advance, Gianluca _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_D4C1048E-6102-423F-8DB0-C4D43E76E17F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">Best = explanation I=E2=80=99ve found is <a = href=3D"https://wiki.mikejung.biz/KVM_/_Xen#virtio-blk_iothreads_.28x-data= -plane.29" = class=3D"">https://wiki.mikejung.biz/KVM_/_Xen#virtio-blk_iothreads_.28x-d= ata-plane.29</a> If you google a bit, you=E2=80=99ll find some = more under QEMU topics, I saw some discussion of threads and queues in = virtio-scsi, but that seems to be a slightly different thing than this = setting.<div class=3D""><br class=3D""></div><div class=3D"">In short, = having at least 1 offers advantages for all your VM=E2=80=99s disks, and = if you want to be optimal (at the possible expense of extra CPU for IO), = one per drive attached. There is (currently) no benefit to having more = than 1 thread per drive. =46rom what I can tell, if you have more drives = than threads they share the threads evenly and are statically assigned = to a thread. Seems to be effective at QEMU start, so you have to change = it with the VM down or stop and start it again.</div><div class=3D""><br = class=3D""></div><div class=3D"">I currently enable it on all VMs and = assign 1 thread per drive on my systems.<br class=3D""><div><blockquote = type=3D"cite" class=3D""><hr style=3D"border:none;border-top:solid = #B5C4DF 1.0pt;padding:0 0 0 0;margin:10px 0 5px 0;" class=3D""><span = style=3D"margin: -1.3px 0.0px 0.0px 0.0px" id=3D"RwhHeaderAttributes" = class=3D""><font face=3D"Helvetica" size=3D"4" color=3D"#000000" = style=3D"font: 13.0px Helvetica; color: #000000" class=3D""><b = class=3D"">From:</b> Gianluca Cecchi <<a = href=3D"mailto:gianluca.cecchi@gmail.com" = class=3D"">gianluca.cecchi@gmail.com</a>></font></span><br class=3D""> <span style=3D"margin: -1.3px 0.0px 0.0px 0.0px" class=3D""><font = face=3D"Helvetica" size=3D"4" color=3D"#000000" style=3D"font: 13.0px = Helvetica; color: #000000" class=3D""><b class=3D"">Subject:</b> = [ovirt-users] VM resource allocation and IO Threads</font></span><br = class=3D""> <span style=3D"margin: -1.3px 0.0px 0.0px 0.0px" class=3D""><font = face=3D"Helvetica" size=3D"4" color=3D"#000000" style=3D"font: 13.0px = Helvetica; color: #000000" class=3D""><b class=3D"">Date:</b> October = 27, 2017 at 9:26:59 AM CDT</font></span><br class=3D""> <span style=3D"margin: -1.3px 0.0px 0.0px 0.0px" class=3D""><font = face=3D"Helvetica" size=3D"4" color=3D"#000000" style=3D"font: 13.0px = Helvetica; color: #000000" class=3D""><b class=3D"">To:</b> = users</font></span><br class=3D""> <br class=3D"Apple-interchange-newline"><div class=3D""><div dir=3D"ltr" = class=3D"">Hello,<div class=3D"">can anyone give any pointer to deeper = information about what in subject and the value for "Num Of IO Threads" = configuration, best practices and to-be-expected improvements?</div><div = class=3D""><br class=3D""></div><div class=3D"">I read also = here:</div><div class=3D""><a = href=3D"https://access.redhat.com/documentation/en-us/red_hat_virtualizati= on/4.1/html-single/virtual_machine_management_guide/#Editing_IO_Threads" = class=3D"">https://access.redhat.com/documentation/en-us/red_hat_virtualiz= ation/4.1/html-single/virtual_machine_management_guide/#Editing_IO_Threads= </a><br class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D"">but in some points it seems not so clear to me:</div><div = class=3D""><br class=3D""></div><div class=3D"">eg:</div><div = class=3D""><br class=3D""></div><div class=3D""><div class=3D"">If a = virtual machine has more than one disk, you can enable or change the = number of IO threads to improve performance.</div></div><div = class=3D""><br class=3D""></div><div class=3D"">but also</div><div = class=3D""><br class=3D""></div><div class=3D""><div class=3D"">Red Hat = recommends using the default number of IO threads, which is = 1.</div></div><div class=3D""><br class=3D""></div><div class=3D"">There = is also a note about deactivation and activation of disks: does it mean = that even if I poweroff the VM and change its config I have to make this = step after?</div><div class=3D""><br class=3D""></div><div = class=3D"">Anyone has run benchmarks?</div><div class=3D"">Does it make = sense if my VM has 3 disks to configure 6 IO threads for = example?</div><div class=3D"">Do IO threads map to SCSI controllers = inside the guest or what?</div><div class=3D""><br class=3D""></div><div = class=3D"">Thanks in advance,<br class=3D""></div><div = class=3D"">Gianluca</div></div> _______________________________________________<br class=3D"">Users = mailing list<br class=3D""><a href=3D"mailto:Users@ovirt.org" = class=3D"">Users@ovirt.org</a><br = class=3D"">http://lists.ovirt.org/mailman/listinfo/users<br = class=3D""></div></blockquote></div><br class=3D""></div></body></html>= --Apple-Mail=_D4C1048E-6102-423F-8DB0-C4D43E76E17F--
As stated previously, 1 IO thread is somewhat sane choice. Multiple IO threads make sense if you have multiple storage devices across different NUMA nodes. mpolednik On Mon, Oct 30, 2017 at 4:29 PM, Darrell Budic <budic@onholyground.com> wrote:
Best explanation I’ve found is https://wiki.mikejung.biz/KVM_/_Xen#virtio-blk_iothreads_.28x-data-plane.29 If you google a bit, you’ll find some more under QEMU topics, I saw some discussion of threads and queues in virtio-scsi, but that seems to be a slightly different thing than this setting.
In short, having at least 1 offers advantages for all your VM’s disks, and if you want to be optimal (at the possible expense of extra CPU for IO), one per drive attached. There is (currently) no benefit to having more than 1 thread per drive. From what I can tell, if you have more drives than threads they share the threads evenly and are statically assigned to a thread. Seems to be effective at QEMU start, so you have to change it with the VM down or stop and start it again.
I currently enable it on all VMs and assign 1 thread per drive on my systems.
________________________________ From: Gianluca Cecchi <gianluca.cecchi@gmail.com> Subject: [ovirt-users] VM resource allocation and IO Threads Date: October 27, 2017 at 9:26:59 AM CDT To: users
Hello, can anyone give any pointer to deeper information about what in subject and the value for "Num Of IO Threads" configuration, best practices and to-be-expected improvements?
I read also here: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/htm...
but in some points it seems not so clear to me:
eg:
If a virtual machine has more than one disk, you can enable or change the number of IO threads to improve performance.
but also
Red Hat recommends using the default number of IO threads, which is 1.
There is also a note about deactivation and activation of disks: does it mean that even if I poweroff the VM and change its config I have to make this step after?
Anyone has run benchmarks? Does it make sense if my VM has 3 disks to configure 6 IO threads for example? Do IO threads map to SCSI controllers inside the guest or what?
Thanks in advance, Gianluca _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Mon, Oct 30, 2017 at 6:16 PM, Martin Polednik <mpoledni@redhat.com> wrote:
As stated previously, 1 IO thread is somewhat sane choice. Multiple IO threads make sense if you have multiple storage devices across different NUMA nodes.
mpolednik
On Mon, Oct 30, 2017 at 4:29 PM, Darrell Budic <budic@onholyground.com> wrote:
Best explanation I’ve found is https://wiki.mikejung.biz/KVM_/_Xen#virtio-blk_iothreads_.28 x-data-plane.29
[snip]
I currently enable it on all VMs and assign 1 thread per drive on my systems.
Thanks for sharing your experience, Darrell. I'm testing some VMs with Oracle RDBMS where I configured 3 disks with virtio-scsi and now I have assigned 3 IO threads. Let's see the numbers. In the mean time I verified you have to deactivate/activate the disks in the guest, otherwise you do get 3 SCSI controllers, but all the 3 disks are bound to the first one (scsi0-0-0-0). Now instead on hypervisor I correctly get this with dumpxml: # virsh -r dumpxml my_guest | grep scsi <target dev='sda' bus='scsi'/> <alias name='scsi0-0-0-0'/> <target dev='sdb' bus='scsi'/> <alias name='scsi1-0-0-0'/> <target dev='sdc' bus='scsi'/> <alias name='scsi2-0-0-0'/> <controller type='scsi' index='0' model='virtio-scsi'> <alias name='scsi0'/> <controller type='scsi' index='1' model='virtio-scsi'> <alias name='scsi1'/> <controller type='scsi' index='2' model='virtio-scsi'> <alias name='scsi2'/> # Martin, I have not understood when you write about "if you have multiple storage devices across different NUMA nodes." Can you elaborate a bit or point to any documentation link? Thanks, Gianluca
participants (3)
-
Darrell Budic -
Gianluca Cecchi -
Martin Polednik