[ovirt-users] Ovirt VM Performance abd CPU times

Thu Oct 30 09:46:31 UTC 2014

I haven't tested the performance differences with or without HT. We were
running pure kvm-libvirt hosts on these machines and we're migrating them
to oVirt and that's what triggered the problem with the Redhat 5 vms.

I'll probably give it a try and disable HT on the next hypervisor that
we'll be adding to the cluster next week and see if that solves the problem
or it just mitigates it.

X

On Thu, Oct 30, 2014 at 10:09 AM, Arman Khalatyan <arm2arm at gmail.com> wrote:

> On our clusters with pure computation we use always HT off, HT was slowing
> down scientific calculations.
> Just I'm interested if any benefit of enabling HT on the Ovirt host?
> At least on my system host-Centos6.5, guest-Centos7VM-LAMPstack with
> separate CentOS7-MariaDBVM did not show any performance boost if I switch
> on/off HT on 2650v2.
> a.
>
>
>
> On Wed, Oct 29, 2014 at 5:24 PM, Xavier Naveira <xnaveira at gmail.com>
> wrote:
>
>> On 10/29/2014 05:15 PM, Daniel Helgenberger wrote:
>>
>>>
>>>
>>> On 29.10.2014 16:44, Xavier Naveira wrote:
>>>
>>>> On 10/29/2014 04:29 PM, Xavier Naveira wrote:
>>>>
>>>>> On 10/29/2014 04:06 PM, Daniel Helgenberger wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 29.10.2014 15:57, Xavier Naveira wrote:
>>>>>>
>>>>>>> On 10/29/2014 03:07 PM, Xavier Naveira wrote:
>>>>>>>
>>>>>>>> On 10/29/2014 01:26 PM, Daniel Helgenberger wrote:
>>>>>>>>
>>>>>>>>> On 29.10.2014 11:48, Xavier Naveira wrote:
>>>>>>>>>
>>>>>>>>>> On 10/29/2014 11:47 AM, Xavier Naveira wrote:
>>>>>>>>>>
>>>>>>>>>>> On 10/29/2014 11:40 AM, Daniel Helgenberger wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 29.10.2014 10:21, Xavier Naveira wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> We are migrating our ifrastructure from kvm+libvirt
>>>>>>>>>>>>> hypervisors to
>>>>>>>>>>>>> ovirt.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Everything is working fine but we're noticing that all the
>>>>>>>>>>>>> qemu-kvm
>>>>>>>>>>>>> processes in the hypervisors take a lot of CPU.
>>>>>>>>>>>>>
>>>>>>>>>>>> Without further details of the workload this is hard tell. One
>>>>>>>>>>>> Reason I
>>>>>>>>>>>> can think of might be KSM [1]. Is it enabled on your cluster(s)?
>>>>>>>>>>>> What is
>>>>>>>>>>>> your mem over-commitment setting?
>>>>>>>>>>>>
>>>>>>>>>>>> Note, IIRC the KSM policy is currently hard coded; it will
>>>>>>>>>>>> start at
>>>>>>>>>>>> 80%
>>>>>>>>>>>> host mem usage.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] http://www.ovirt.org/Sla/host-mom-policy
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The typical example is an idle machine, running top from the
>>>>>>>>>>>>> machine
>>>>>>>>>>>>> itself it reports cpu use percentages below 10% and loads
>>>>>>>>>>>>> (with 2
>>>>>>>>>>>>> processors) of 0.0x. The process running that machine in the
>>>>>>>>>>>>> hypervisor
>>>>>>>>>>>>> rports cpu uses in the order of the 80-100%.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Should the values look like this? Why are the idle machines
>>>>>>>>>>>>> eating
>>>>>>>>>>>>> up so
>>>>>>>>>>>>> much CPU time?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>> Xavier
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>> Users at ovirt.org
>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>  Hi, thank you for the answer.
>>>>>>>>>>
>>>>>>>>>> I've been trying to work out some pattern and realized that the
>>>>>>>>>> VMs
>>>>>>>>>> using that much cpu all are Redhat 5.x, the Readhat 6.x doesn't
>>>>>>>>>> exhibit
>>>>>>>>>> this kind of high cpu use. (we run only redhat/centos 5.x/6.x on
>>>>>>>>>> the
>>>>>>>>>> cluster)
>>>>>>>>>>
>>>>>>>>> What OS are the hosts running? In case of EL6, make sure you have
>>>>>>>>> tuned-0.2.19-13.el6.noarch installed [1].
>>>>>>>>>
>>>>>>>>
>>>>>>>> That's exactly the version we've in the hypervisors.
>>>>>>>>
>>>>>>>>
>>>>>>>>> To further investigate please post Engine, VDSM, libvirt and kernel
>>>>>>>>> versions from the hosts.
>>>>>>>>>
>>>>>>>>
>>>>>>>> vdsm-xmlrpc-4.14.11.2-0.el6.noarch
>>>>>>>> vdsm-cli-4.14.11.2-0.el6.noarch
>>>>>>>> vdsm-python-4.14.11.2-0.el6.x86_64
>>>>>>>> vdsm-4.14.11.2-0.el6.x86_64
>>>>>>>> vdsm-python-zombiereaper-4.14.11.2-0.el6.noarch
>>>>>>>>
>>>>>>>> libvirt-client-0.10.2-29.el6_5.12.x86_64
>>>>>>>> libvirt-0.10.2-29.el6_5.12.x86_64
>>>>>>>> libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64
>>>>>>>> libvirt-python-0.10.2-29.el6_5.12.x86_64
>>>>>>>>
>>>>>>>> 2.6.32-431.23.3.el6.x86_64 #1 SMP Wed Jul 16 06:12:23 EDT 2014
>>>>>>>> x86_64
>>>>>>>> x86_64 x86_64 GNU/Linux
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> [1] https://access.redhat.com/solutions/358033
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'll take a look to the KSM config.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Xavier
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> Actually, this seems to be it. But I'm already at a newer kernel:
>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=705082
>>>>>>>
>>>>>> Well, I do not have such hardware so I never run into the issue. You
>>>>>> could disable HT as I suspect your physical cores are less then 64?
>>>>>>
>>>>>> Your workload might differ but my VMs usually do not benefit from
>>>>>> 'threaded' cores and I want HT disabled anyway. Also, you can check
>>>>>> cluster settings and disable 'count threads as cores' if enabled. But
>>>>>> I
>>>>>> think this might not make any difference.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>  Yeah, these are machines with 4 sockets, 6 core per socket and HT
>>>>> enabled, so total 48 "CPU".
>>>>>
>>>> Good to know; yet the largest host I have has 32 (2 sockets, 8 cores, HT
>>> enabled) CPUs and is not showing this issue (at least I just looked and
>>> everything seems fine).
>>>
>>>>
>>>>> So, are you implying that the problem is the number of "CPUs"? We were
>>>>> hoping to add some more hypervisors to the cluster next week that have
>>>>> even more cores...
>>>>>
>>>>> I can probably try to disable HT when we add the next hypervisor next
>>>>> week but it feels that it'd be just a workaround?
>>>>>
>>>> Maybe, but not a bad one as you should not have any disadvantages.
>>>
>>>>
>>>> I opened a bug at redhat just in
>>>> case:https://bugzilla.redhat.com/show_bug.cgi?id=1158547
>>>>
>>> I have to ask as I cannot see the BZ because I have no subscription any
>>> more. Against witch component did you open it?
>>>
>>>>
>>>>
>>>  I did as in the original bug: kernel, and then I took KVM as subsystem.
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20141030/a8da21ca/attachment-0001.html>