1:57 p.m.
Hi all,
we plan to work on an improvement in VM definition for high performance workloads which do
not require desktop-class devices and generally favor highest possible performance in
expense of less flexibility.
We’re thinking of adding a new VM preset in addition to current Desktop and Server in New
VM dialog, which would automatically pre-select existing options in the right way, and
suggest/warn on suboptimal configuration
All the presets and warning can be changed and ignored. There are few things we already
identified as boosting performance and/or minimize the complexity of the VM, so we plan
the preset to:
- remove all graphical consoles and set the VM as headless, making it accessible by serial
console.
- disable all USB.
- disable soundcard.
- enable I/O Threads, just one for all disks by default.
- set host cpu passthrough (effectively disabling VM live migration), add I/O Thread
pinning in a similar way as the existing CPU pinning.
We plan the following checks and suggest to perform CPU pinning, host topology == guest
topology (number of cores per socket and threads per core should match), NUMA topology
host and guest match, check and suggest the I/O threads pinning.
A popup on a VM dialog save seems suitable.
currently identified task and status can be followed on trello card[1]
Please share your thoughts, questions, any kind of feedback…
Thanks,
michal
[1] https://trello.com/c/MHRDD8ZO
2:59 p.m.
On 24/05/17 12:57 +0200, Michal Skrivanek wrote:
Hi all,
we plan to work on an improvement in VM definition for high performance workloads which do
not require desktop-class devices and generally favor highest possible performance in
expense of less flexibility.
We’re thinking of adding a new VM preset in addition to current Desktop and Server in New
VM dialog, which would automatically pre-select existing options in the right way, and
suggest/warn on suboptimal configuration
All the presets and warning can be changed and ignored. There are few things we already
identified as boosting performance and/or minimize the complexity of the VM, so we plan
the preset to:
- remove all graphical consoles and set the VM as headless, making it accessible by serial
console.
- disable all USB.
- disable soundcard.
- enable I/O Threads, just one for all disks by default.
- set host cpu passthrough (effectively disabling VM live migration), add I/O Thread
pinning in a similar way as the existing CPU pinning.
We plan the following checks and suggest to perform CPU pinning, host topology == guest
topology (number of cores per socket and threads per core should match), NUMA topology
host and guest match, check and suggest the I/O threads pinning.
A popup on a VM dialog save seems suitable.
As for the checks, I'd prefer to see slightly more fine-grained
topology analysis. We don't need the guest topology to overlap with
hosts topology, but it needs to fit in. So the checks should be
(ordered by it's topology significance):
1) #guest_numa_nodes <= #host_numa_nodes
2) #guest_sockets_per_node <= #host_sockets_per_node
3) #guest_cpus_per_socket <= #host_cpus_per_socket (this check has to
account for cores X threads difference)
4) guest_ram_per_node <= host_ram_per_node
These four checks guarantee that each guest's numa fits onto host numa
node and that we're not requesting more nodes than we have at disposal.
Now if these don't pass, each check can have suggestion on how to
proceed:
1) lower the number of numa nodes but increase sockets/cores/memory
per node
2) increase number of numa nodes
3) increase number of sockets
4) increase number of numa nodes
Should these checks fail, the VM can't be started in high performance mode.
In short, we've relaxed the requirement that host topology == guest
topology to host topology >= guest topology.
If the checks pass, we should recommend pinning the numa nodes (and do
strict/prefered pinning rather than interleave), numa
memory and CPUs to make sure it will match the topology. Additionally,
if the VM uses host devices (incl. SR-IOV), the pinning should be as
close to the node where devices' MMIO resides as possible.
Additionally, we should suggest to leave some CPUs within numa node
and pin iothreads/emulator against these.
BTW what about virtio-scsi vs virtio-blk in this case? High
performance may be the case where virtio-blk is reasonable.
currently identified task and status can be followed on trello
card[1]
Please share your thoughts, questions, any kind of feedback…
Thanks,
michal
[1] https://trello.com/c/MHRDD8ZO
3:11 p.m.
On 05/24/2017 12:57 PM, Michal Skrivanek wrote:
Hi all,
we plan to work on an improvement in VM definition for high performance workloads which
do not require desktop-class devices and generally favor highest possible performance in
expense of less flexibility.
We’re thinking of adding a new VM preset in addition to current Desktop and Server in New
VM dialog, which would automatically pre-select existing options in the right way, and
suggest/warn on suboptimal configuration
All the presets and warning can be changed and ignored. There are few things we already
identified as boosting performance and/or minimize the complexity of the VM, so we plan
the preset to:
- remove all graphical consoles and set the VM as headless, making it accessible by
serial console.
- disable all USB.
- disable soundcard.
- enable I/O Threads, just one for all disks by default.
- set host cpu passthrough (effectively disabling VM live migration), add I/O Thread
pinning in a similar way as the existing CPU pinning.
We plan the following checks and suggest to perform CPU pinning, host topology == guest
topology (number of cores per socket and threads per core should match), NUMA topology
host and guest match, check and suggest the I/O threads pinning.
A popup on a VM dialog save seems suitable.
currently identified task and status can be followed on trello card[1]
Please share your thoughts, questions, any kind of feedback…
In order to maximize performance we may also want to limit the number of
other VMs (either regular or high performance) running on the same
host. This to minimize the interference and the resource stealing.
In the extreme case, just the selected high performance VM would be
allowed to run on one suitable host.
Bests,
--
Francesco Romani
Senior SW Eng., Virtualization R&D
Red Hat
IRC: fromani github: @fromanirh
3:20 p.m.
On 24 May 2017, at 14:11, Francesco Romani <fromani(a)redhat.com>
wrote:
On 05/24/2017 12:57 PM, Michal Skrivanek wrote:
> Hi all,
> we plan to work on an improvement in VM definition for high performance workloads
which do not require desktop-class devices and generally favor highest possible
performance in expense of less flexibility.
> We’re thinking of adding a new VM preset in addition to current Desktop and Server in
New VM dialog, which would automatically pre-select existing options in the right way, and
suggest/warn on suboptimal configuration
> All the presets and warning can be changed and ignored. There are few things we
already identified as boosting performance and/or minimize the complexity of the VM, so we
plan the preset to:
> - remove all graphical consoles and set the VM as headless, making it accessible by
serial console.
> - disable all USB.
> - disable soundcard.
> - enable I/O Threads, just one for all disks by default.
> - set host cpu passthrough (effectively disabling VM live migration), add I/O Thread
pinning in a similar way as the existing CPU pinning.
> We plan the following checks and suggest to perform CPU pinning, host topology ==
guest topology (number of cores per socket and threads per core should match), NUMA
topology host and guest match, check and suggest the I/O threads pinning.
> A popup on a VM dialog save seems suitable.
>
> currently identified task and status can be followed on trello card[1]
>
> Please share your thoughts, questions, any kind of feedback…
In order to maximize performance we may also want to limit the number of
other VMs (either regular or high performance) running on the same
host. This to minimize the interference and the resource stealing.
In the extreme case, just the selected high performance VM would be
allowed to run on one suitable host.
well, it certainly goes in that direction. E.g. for a real time workloads/VMs the host
needs to be configured in a way which makes it very cumbersome for other non-RT VMs - you
need to reserve resources, typically you do not want to do that at all as when you are
serious about performance or RT you really do not want any kind of overcommit.
Even for the most simple case of hugepages the pre-allocation of those pages seems to be
the only reliable way of getting them for sure (at least for 1GB pages), and such memory
is then unusable for non-hugepages VMs
Bests,
--
Francesco Romani
Senior SW Eng., Virtualization R&D
Red Hat
IRC: fromani github: @fromanirh
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
4:03 p.m.
In order to maximize performance we may also want to limit the number
of
other VMs (either regular or high performance) running on the same
host. This to minimize the interference and the resource stealing.
In the extreme case, just the selected high performance VM would be
allowed to run on one suitable host.
I would base this on cores. You can have two HPF VMs when each is
pinned to distinct set of CPUs.
Martin
On Wed, May 24, 2017 at 2:11 PM, Francesco Romani <fromani(a)redhat.com> wrote:
> On 05/24/2017 12:57 PM, Michal Skrivanek wrote:
>> Hi all,
>> we plan to work on an improvement in VM definition for high performance workloads
which do not require desktop-class devices and generally favor highest possible
performance in expense of less flexibility.
>> We’re thinking of adding a new VM preset in addition to current Desktop and
Server in New VM dialog, which would automatically pre-select existing options in the
right way, and suggest/warn on suboptimal configuration
>> All the presets and warning can be changed and ignored. There are few things we
already identified as boosting performance and/or minimize the complexity of the VM, so we
plan the preset to:
>> - remove all graphical consoles and set the VM as headless, making it accessible
by serial console.
>> - disable all USB.
>> - disable soundcard.
>> - enable I/O Threads, just one for all disks by default.
>> - set host cpu passthrough (effectively disabling VM live migration), add I/O
Thread pinning in a similar way as the existing CPU pinning.
>> We plan the following checks and suggest to perform CPU pinning, host topology ==
guest topology (number of cores per socket and threads per core should match), NUMA
topology host and guest match, check and suggest the I/O threads pinning.
>> A popup on a VM dialog save seems suitable.
>>
>> currently identified task and status can be followed on trello card[1]
>>
>> Please share your thoughts, questions, any kind of feedback…
>
In order to maximize performance we may also want to limit the number
of
other VMs (either regular or high performance) running on the same
host. This to minimize the interference and the resource stealing.
In the extreme case, just the selected high performance VM would be
allowed to run on one suitable host.
>
> Bests,
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R&D
> Red Hat
> IRC: fromani github: @fromanirh
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
4:06 p.m.
Or maybe on NUMA nodes.
On Wed, May 24, 2017 at 3:03 PM, Martin Sivak <msivak(a)redhat.com> wrote:
> In order to maximize performance we may also want to limit the
number of
> other VMs (either regular or high performance) running on the same
> host. This to minimize the interference and the resource stealing.
>
>
> In the extreme case, just the selected high performance VM would be
> allowed to run on one suitable host.
I would base this on cores. You can have two HPF VMs when each is
pinned to distinct set of CPUs.
Martin
On Wed, May 24, 2017 at 2:11 PM, Francesco Romani <fromani(a)redhat.com> wrote:
> On 05/24/2017 12:57 PM, Michal Skrivanek wrote:
>> Hi all,
>> we plan to work on an improvement in VM definition for high performance workloads
which do not require desktop-class devices and generally favor highest possible
performance in expense of less flexibility.
>> We’re thinking of adding a new VM preset in addition to current Desktop and
Server in New VM dialog, which would automatically pre-select existing options in the
right way, and suggest/warn on suboptimal configuration
>> All the presets and warning can be changed and ignored. There are few things we
already identified as boosting performance and/or minimize the complexity of the VM, so we
plan the preset to:
>> - remove all graphical consoles and set the VM as headless, making it accessible
by serial console.
>> - disable all USB.
>> - disable soundcard.
>> - enable I/O Threads, just one for all disks by default.
>> - set host cpu passthrough (effectively disabling VM live migration), add I/O
Thread pinning in a similar way as the existing CPU pinning.
>> We plan the following checks and suggest to perform CPU pinning, host topology ==
guest topology (number of cores per socket and threads per core should match), NUMA
topology host and guest match, check and suggest the I/O threads pinning.
>> A popup on a VM dialog save seems suitable.
>>
>> currently identified task and status can be followed on trello card[1]
>>
>> Please share your thoughts, questions, any kind of feedback…
>
> In order to maximize performance we may also want to limit the number of
> other VMs (either regular or high performance) running on the same
> host. This to minimize the interference and the resource stealing.
>
>
> In the extreme case, just the selected high performance VM would be
> allowed to run on one suitable host.
>
> Bests,
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R&D
> Red Hat
> IRC: fromani github: @fromanirh
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
2738
days inactive
2738
days old
5 comments
4 participants
participants (4)
-
Francesco Romani -
Martin Polednik -
Martin Sivak -
Michal Skrivanek