Subject: [oVirt 4.5.7] CCD/CCX and L3 cache topology awareness for AMD EPYC
Genoa guests
Hello oVirt community,
First, thank you to everyone who contributed to the oVirt 4.5.7 release. It
is great to see the project moving forward, and the addition of the AMD
EPYC Genoa CPU type in cluster level 4.8 is very welcome.
I am designing an oVirt-based virtualization platform on Dell PowerEdge
servers with AMD EPYC Genoa ( 9654 series) processors. On Genoa, each CCD
contains one CCX of up to 8 cores that share a 32 MB L3 cache, and a socket
can have up to 12 CCDs. For latency-sensitive workloads, keeping a VM's
vCPUs within one CCD, and making the guest aware of which vCPUs share an L3
cache, makes a measurable difference to performance.
I would like to understand what oVirt 4.5.7 supports out of the box in this
area:
1. CPU type scope: Does the new EPYC Genoa CPU type only expose the CPU
model and CPUID feature flags to the guest, or does it also influence the
virtual CPU topology (sockets/dies/cores/threads) and cache topology the
guest sees?
2. Die and cache topology in the domain XML: Does the engine/VDSM generate
a libvirt <topology> with the 'dies' attribute, or a <cache> element
(passthrough or emulated L3), so that the guest can see CCD/CCX boundaries?
If so, which VM types or settings enable it (for example, the High
Performance VM type or Pass-Through Host CPU)?
3. Guest view of L3 sharing: When vCPUs are pinned to the cores of a single
CCD, should the guest report correct L3 sharing (for example in 'lscpu -C'
or /sys/devices/system/cpu/cpu*/cache/index3/shared_cpu_list), or will it
typically present one L3 shared across all vCPUs?
4. Scheduling and pinning: Do the CPU pinning policies (Dedicated, Isolate
Threads, Resize and Pin NUMA) take CCD/CCX boundaries into account when
choosing host cores? Is there support for hosts configured with the BIOS
option 'L3 as NUMA' (LLC as NUMA node), where each CCX appears as its own
NUMA node?
5. Roadmap: If CCD/L3-aware topology is not available today, is it on the
roadmap for a future 4.5.x or later release? Is there an existing GitHub
issue or RFE I can follow or contribute details to? In the meantime, is a
VDSM hook that adds the dies/cache elements to the domain XML the
recommended workaround, or is there a better approach?
I am happy to test any builds or patches on Genoa hardware and share the
results (lscpu output, libvirt XML, and benchmark numbers) with the
community.
Thank you for your time and guidance.
Best regards,
Srini
