[ovirt-users] Ovirt VM Performance abd CPU times

[Xavier Naveira]

On 10/29/2014 05:15 PM, Daniel Helgenberger wrote:
>
>
> On 29.10.2014 16:44, Xavier Naveira wrote:
>> On 10/29/2014 04:29 PM, Xavier Naveira wrote:
>>> On 10/29/2014 04:06 PM, Daniel Helgenberger wrote:
>>>>
>>>>
>>>> On 29.10.2014 15:57, Xavier Naveira wrote:
>>>>> On 10/29/2014 03:07 PM, Xavier Naveira wrote:
>>>>>> On 10/29/2014 01:26 PM, Daniel Helgenberger wrote:
>>>>>>> On 29.10.2014 11:48, Xavier Naveira wrote:
>>>>>>>> On 10/29/2014 11:47 AM, Xavier Naveira wrote:
>>>>>>>>> On 10/29/2014 11:40 AM, Daniel Helgenberger wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 29.10.2014 10:21, Xavier Naveira wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> We are migrating our ifrastructure from kvm+libvirt hypervisors to
>>>>>>>>>>> ovirt.
>>>>>>>>>>>
>>>>>>>>>>> Everything is working fine but we're noticing that all the
>>>>>>>>>>> qemu-kvm
>>>>>>>>>>> processes in the hypervisors take a lot of CPU.
>>>>>>>>>> Without further details of the workload this is hard tell. One
>>>>>>>>>> Reason I
>>>>>>>>>> can think of might be KSM [1]. Is it enabled on your cluster(s)?
>>>>>>>>>> What is
>>>>>>>>>> your mem over-commitment setting?
>>>>>>>>>>
>>>>>>>>>> Note, IIRC the KSM policy is currently hard coded; it will start at
>>>>>>>>>> 80%
>>>>>>>>>> host mem usage.
>>>>>>>>>>
>>>>>>>>>> [1] http://www.ovirt.org/Sla/host-mom-policy
>>>>>>>>>>>
>>>>>>>>>>> The typical example is an idle machine, running top from the
>>>>>>>>>>> machine
>>>>>>>>>>> itself it reports cpu use percentages below 10% and loads (with 2
>>>>>>>>>>> processors) of 0.0x. The process running that machine in the
>>>>>>>>>>> hypervisor
>>>>>>>>>>> rports cpu uses in the order of the 80-100%.
>>>>>>>>>>>
>>>>>>>>>>> Should the values look like this? Why are the idle machines eating
>>>>>>>>>>> up so
>>>>>>>>>>> much CPU time?
>>>>>>>>>>>
>>>>>>>>>>> Thank you.
>>>>>>>>>>> Xavier
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Users mailing list
>>>>>>>>>>> Users at ovirt.org
>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> Hi, thank you for the answer.
>>>>>>>>
>>>>>>>> I've been trying to work out some pattern and realized that the VMs
>>>>>>>> using that much cpu all are Redhat 5.x, the Readhat 6.x doesn't
>>>>>>>> exhibit
>>>>>>>> this kind of high cpu use. (we run only redhat/centos 5.x/6.x on the
>>>>>>>> cluster)
>>>>>>> What OS are the hosts running? In case of EL6, make sure you have
>>>>>>> tuned-0.2.19-13.el6.noarch installed [1].
>>>>>>
>>>>>> That's exactly the version we've in the hypervisors.
>>>>>>
>>>>>>>
>>>>>>> To further investigate please post Engine, VDSM, libvirt and kernel
>>>>>>> versions from the hosts.
>>>>>>
>>>>>> vdsm-xmlrpc-4.14.11.2-0.el6.noarch
>>>>>> vdsm-cli-4.14.11.2-0.el6.noarch
>>>>>> vdsm-python-4.14.11.2-0.el6.x86_64
>>>>>> vdsm-4.14.11.2-0.el6.x86_64
>>>>>> vdsm-python-zombiereaper-4.14.11.2-0.el6.noarch
>>>>>>
>>>>>> libvirt-client-0.10.2-29.el6_5.12.x86_64
>>>>>> libvirt-0.10.2-29.el6_5.12.x86_64
>>>>>> libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64
>>>>>> libvirt-python-0.10.2-29.el6_5.12.x86_64
>>>>>>
>>>>>> 2.6.32-431.23.3.el6.x86_64 #1 SMP Wed Jul 16 06:12:23 EDT 2014 x86_64
>>>>>> x86_64 x86_64 GNU/Linux
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> [1] https://access.redhat.com/solutions/358033
>>>>>>>>
>>>>>>>> I'll take a look to the KSM config.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Xavier
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> Actually, this seems to be it. But I'm already at a newer kernel:
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=705082
>>>> Well, I do not have such hardware so I never run into the issue. You
>>>> could disable HT as I suspect your physical cores are less then 64?
>>>>
>>>> Your workload might differ but my VMs usually do not benefit from
>>>> 'threaded' cores and I want HT disabled anyway. Also, you can check
>>>> cluster settings and disable 'count threads as cores' if enabled. But I
>>>> think this might not make any difference.
>>>>>
>>>>
>>> Yeah, these are machines with 4 sockets, 6 core per socket and HT
>>> enabled, so total 48 "CPU".
> Good to know; yet the largest host I have has 32 (2 sockets, 8 cores, HT
> enabled) CPUs and is not showing this issue (at least I just looked and
> everything seems fine).
>>>
>>> So, are you implying that the problem is the number of "CPUs"? We were
>>> hoping to add some more hypervisors to the cluster next week that have
>>> even more cores...
>>>
>>> I can probably try to disable HT when we add the next hypervisor next
>>> week but it feels that it'd be just a workaround?
> Maybe, but not a bad one as you should not have any disadvantages.
>>
>> I opened a bug at redhat just in
>> case:https://bugzilla.redhat.com/show_bug.cgi?id=1158547
> I have to ask as I cannot see the BZ because I have no subscription any
> more. Against witch component did you open it?
>>
>
I did as in the original bug: kernel, and then I took KVM as subsystem.