Re: [ovirt-users] Ovirt VM Performance abd CPU times

On 29.10.2014 15:57, Xavier Naveira wrote: > On 10/29/2014 03:07 PM, Xavier Naveira wrote: >> On 10/29/2014 01:26 PM, Daniel Helgenberger wrote: >>> On 29.10.2014 11:48, Xavier Naveira wrote: >>>> On 10/29/2014 11:47 AM, Xavier Naveira wrote: >>>>> On 10/29/2014 11:40 AM, Daniel Helgenberger wrote: >>>>>> >>>>>> >>>>>> On 29.10.2014 10:21, Xavier Naveira wrote: >>>>>>> Hi, >>>>>>> >>>>>>> We are migrating our ifrastructure from kvm+libvirt hypervisors to >>>>>>> ovirt. >>>>>>> >>>>>>> Everything is working fine but we're noticing that all the qemu-kvm >>>>>>> processes in the hypervisors take a lot of CPU. >>>>>> Without further details of the workload this is hard tell. One >>>>>> Reason I >>>>>> can think of might be KSM [1]. Is it enabled on your cluster(s)? >>>>>> What is >>>>>> your mem over-commitment setting? >>>>>> >>>>>> Note, IIRC the KSM policy is currently hard coded; it will start at >>>>>> 80% >>>>>> host mem usage. >>>>>> >>>>>> [1] http://www.ovirt.org/Sla/host-mom-policy >>>>>>> >>>>>>> The typical example is an idle machine, running top from the machine >>>>>>> itself it reports cpu use percentages below 10% and loads (with 2 >>>>>>> processors) of 0.0x. The process running that machine in the >>>>>>> hypervisor >>>>>>> rports cpu uses in the order of the 80-100%. >>>>>>> >>>>>>> Should the values look like this? Why are the idle machines eating >>>>>>> up so >>>>>>> much CPU time? >>>>>>> >>>>>>> Thank you. >>>>>>> Xavier >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Users mailing list >>>>>>> Users(a)ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>>> >>>>>> >>>>> >>>> Hi, thank you for the answer. >>>> >>>> I've been trying to work out some pattern and realized that the VMs >>>> using that much cpu all are Redhat 5.x, the Readhat 6.x doesn't exhibit >>>> this kind of high cpu use. (we run only redhat/centos 5.x/6.x on the >>>> cluster) >>> What OS are the hosts running? In case of EL6, make sure you have >>> tuned-0.2.19-13.el6.noarch installed [1]. >> >> That's exactly the version we've in the hypervisors. >> >>> >>> To further investigate please post Engine, VDSM, libvirt and kernel >>> versions from the hosts. >> >> vdsm-xmlrpc-4.14.11.2-0.el6.noarch >> vdsm-cli-4.14.11.2-0.el6.noarch >> vdsm-python-4.14.11.2-0.el6.x86_64 >> vdsm-4.14.11.2-0.el6.x86_64 >> vdsm-python-zombiereaper-4.14.11.2-0.el6.noarch >> >> libvirt-client-0.10.2-29.el6_5.12.x86_64 >> libvirt-0.10.2-29.el6_5.12.x86_64 >> libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 >> libvirt-python-0.10.2-29.el6_5.12.x86_64 >> >> 2.6.32-431.23.3.el6.x86_64 #1 SMP Wed Jul 16 06:12:23 EDT 2014 x86_64 >> x86_64 x86_64 GNU/Linux >> >> >>> >>> [1] https://access.redhat.com/solutions/358033 >>>> >>>> I'll take a look to the KSM config. >>>> >>>> Cheers, >>>> >>>> Xavier >>>> >>> >> > > Actually, this seems to be it. But I'm already at a newer kernel: > https://bugzilla.redhat.com/show_bug.cgi?id=705082 Well, I do not have such hardware so I never run into the issue. You could disable HT as I suspect your physical cores are less then 64? Your workload might differ but my VMs usually do not benefit from 'threaded' cores and I want HT disabled anyway. Also, you can check cluster settings and disable 'count threads as cores' if enabled. But I think this might not make any difference. >