[RFC] VM thread pinning for High Performance

Hey list, As a part of High Performance VMs feature [1], the oVirt virt team is planning to introduce a fine grained CPU pinning control for various VM threads (vcpu, io, emulator). Current idea for the pinning is to allow per-host configuration of the number of CPUs per numa node to be reserved for io+emulator threads. If we assume that the reservation will be done by 'reserved_cpus' variable, let's use this as an example: reserved_cpus = [0, 1] Such configuration would imply that 2 cores, 0th and 1st in each numa node may be used for io+emulator thread. Additional logic is required to avoid wasting these cores if single machine spans >1 numa node, roughly if numa_node where vcpu0 contains reserved_cpus: pin io/emulator thread to reserved_cpus else: reserve reserved_cpus in numa_node with vcpu0 pin io/emulator thread to reserved_cpus Such logic won't waste extra vcpus for large VMs. Any comments on the idea are more than welcome! [1] https://trello.com/c/MHRDD8ZO/67-high-performance-vm-preset-in-vm-dialog

On Thu, Jun 22, 2017 at 1:16 PM, Martin Polednik <mpolednik@redhat.com> wrote:
Hey list,
As a part of High Performance VMs feature [1], the oVirt virt team is planning to introduce a fine grained CPU pinning control for various VM threads (vcpu, io, emulator).
Current idea for the pinning is to allow per-host configuration of the number of CPUs per numa node to be reserved for io+emulator threads.
If we assume that the reservation will be done by 'reserved_cpus' variable, let's use this as an example:
reserved_cpus = [0, 1]
Such configuration would imply that 2 cores, 0th and 1st in each numa node may be used for io+emulator thread. Additional logic is required to avoid wasting these cores if single machine spans >1 numa node, roughly
if numa_node where vcpu0 contains reserved_cpus: pin io/emulator thread to reserved_cpus else: reserve reserved_cpus in numa_node with vcpu0 pin io/emulator thread to reserved_cpus
Such logic won't waste extra vcpus for large VMs.
Any comments on the idea are more than welcome!
I like this idea a lot. It will much simplify the way how to configure it in engine. In engine we will be able to configure it as one global engine-config parameter (something like ReservedCpusForHPVms) which will be by default set to "0,1" and all the HP VMs will than be started with this param provided. This way we will not expose this low level configuration to the VM dialog but still will have the option to change it if you have some specific needs (but in absolutely most cases the default should be fine).
[1] https://trello.com/c/MHRDD8ZO/67-high-performance-vm-preset- in-vm-dialog _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (2)
-
Martin Polednik
-
Tomas Jelinek