NUMA Pinning bug with Hugepages?
Hi, I seem to have found an issue when trying to setup a high performance VM utilising hugepages and NUMA pinning. The VM is configured for 32GB RAM and uses hugepages of size 1G. The host has two NUMA nodes each having 64GB RAM (for 128GB total system RAM). No other VMs are running or otherwise pinned to the host. The VM is pinned to node 0. And the required hugepages are allocated with echo 32 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages I then attempt to start the VM and get a pop-up error saying "cannot accommodate memory of VM's pinned virtual NUMA nodes within host's physical NUMA nodes". However, if I remove the hugepages from node 0 echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages Start the VM and then immediately re-create the hugepqages then everything works as expected. It seems to me that oVirt is considering the hugepages as allocated memory even if they are not in use. Is this correct? Thanks, Alan
Hi, I think you've hit similar issue as reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1812316 You're right, the hugepages are considered as allocated memory and it is a bug. I am working on a fix right now. Regards, Lucia On Wed, Apr 15, 2020 at 1:21 PM Alan G <alan+ovirt@griff.me.uk> wrote:
Hi,
I seem to have found an issue when trying to setup a high performance VM utilising hugepages and NUMA pinning.
The VM is configured for 32GB RAM and uses hugepages of size 1G.
The host has two NUMA nodes each having 64GB RAM (for 128GB total system RAM).
No other VMs are running or otherwise pinned to the host.
The VM is pinned to node 0. And the required hugepages are allocated with
echo 32 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
I then attempt to start the VM and get a pop-up error saying "cannot accommodate memory of VM's pinned virtual NUMA nodes within host's physical NUMA nodes".
However, if I remove the hugepages from node 0
echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
Start the VM and then immediately re-create the hugepqages then everything works as expected.
It seems to me that oVirt is considering the hugepages as allocated memory even if they are not in use.
Is this correct?
Thanks,
Alan
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/32QKZ2EQ5OKTMP...
Will this be fixed in 4.3 or only 4.4? Thanks, Alan ---- On Wed, 15 Apr 2020 12:37:38 +0100 Lucia Jelinkova <ljelinko@redhat.com> wrote ---- Hi, I think you've hit similar issue as reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1812316 You're right, the hugepages are considered as allocated memory and it is a bug. I am working on a fix right now. Regards, Lucia On Wed, Apr 15, 2020 at 1:21 PM Alan G <mailto:alan%2Bovirt@griff.me.uk> wrote: Hi, I seem to have found an issue when trying to setup a high performance VM utilising hugepages and NUMA pinning. The VM is configured for 32GB RAM and uses hugepages of size 1G. The host has two NUMA nodes each having 64GB RAM (for 128GB total system RAM). No other VMs are running or otherwise pinned to the host. The VM is pinned to node 0. And the required hugepages are allocated with echo 32 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages I then attempt to start the VM and get a pop-up error saying "cannot accommodate memory of VM's pinned virtual NUMA nodes within host's physical NUMA nodes". However, if I remove the hugepages from node 0 echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages Start the VM and then immediately re-create the hugepqages then everything works as expected. It seems to me that oVirt is considering the hugepages as allocated memory even if they are not in use. Is this correct? Thanks, Alan _______________________________________________ Users mailing list -- mailto:users@ovirt.org To unsubscribe send an email to mailto:users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/32QKZ2EQ5OKTMP...
Hi, I asked about a backport to 4.3, but this won't happen according to support. You should be aware that there are more issues with hugepages and 4.3: https://access.redhat.com/solutions/4904441 https://bugzilla.redhat.com/show_bug.cgi?id=1804037 https://bugzilla.redhat.com/show_bug.cgi?id=1804046 https://bugzilla.redhat.com/show_bug.cgi?id=1812316 https://bugzilla.redhat.com/show_bug.cgi?id=1806339 https://bugzilla.redhat.com/show_bug.cgi?id=1785507 Greetings Klaas On 15.04.20 13:42, Alan G wrote:
Will this be fixed in 4.3 or only 4.4?
Thanks,
Alan
---- On Wed, 15 Apr 2020 12:37:38 +0100 *Lucia Jelinkova <ljelinko@redhat.com>* wrote ----
Hi,
I think you've hit similar issue as reported here:
https://bugzilla.redhat.com/show_bug.cgi?id=1812316
You're right, the hugepages are considered as allocated memory and it is a bug. I am working on a fix right now.
Regards,
Lucia
On Wed, Apr 15, 2020 at 1:21 PM Alan G <alan+ovirt@griff.me.uk <mailto:alan%2Bovirt@griff.me.uk>> wrote:
Hi,
I seem to have found an issue when trying to setup a high performance VM utilising hugepages and NUMA pinning.
The VM is configured for 32GB RAM and uses hugepages of size 1G.
The host has two NUMA nodes each having 64GB RAM (for 128GB total system RAM).
No other VMs are running or otherwise pinned to the host.
The VM is pinned to node 0. And the required hugepages are allocated with
echo 32 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
I then attempt to start the VM and get a pop-up error saying "cannot accommodate memory of VM's pinned virtual NUMA nodes within host's physical NUMA nodes".
However, if I remove the hugepages from node 0
echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
Start the VM and then immediately re-create the hugepqages then everything works as expected.
It seems to me that oVirt is considering the hugepages as allocated memory even if they are not in use.
Is this correct?
Thanks,
Alan
_______________________________________________ Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> To unsubscribe send an email to users-leave@ovirt.org <mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/32QKZ2EQ5OKTMP...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PLIOIMOAOBVWJE...
participants (3)
-
Alan G -
klaasdemter＠gmail.com -
Lucia Jelinkova