[Engine-devel] Updates in VdsUpdateRuntimeInfo

[Itamar Heim]

On 06/12/2013 04:03 PM, Liran Zelkha wrote:
> Hi all,
>
> Current status of VdsUpdateRuntimeInfo (please note - patch is not yet committed).
> Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB activity. Of that, about 25% of CPU time wasted on update behavior.
> After modification (moving updates to batch update) UpdateRuntimeInfo performance wasted roughly 40% of CPU time on DB activity. And only 12% of that was wasted on update behavior.

which changes are in place for above?
>
> My next task is to try and optimize the rest of overall database performance.
>
>
> ----- Original Message -----
> From: "Liran Zelkha" <liran.zelkha at gmail.com>
> To: "Barak Azulay" <bazulay at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Wednesday, June 12, 2013 12:35:03 PM
> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
>
> Hi guys,
>
> I'm working hard on this. I'll send summary mail and findings later today.
>
> On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:
>
>>
>>
>> ----- Original Message -----
>>> From: "Itamar Heim" <iheim at redhat.com>
>>> To: "Barak Azulay" <bazulay at redhat.com>
>>> Cc: "engine-devel" <engine-devel at ovirt.org>
>>> Sent: Wednesday, June 12, 2013 10:09:09 AM
>>> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
>>>
>>> On 06/12/2013 09:41 AM, Barak Azulay wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Itamar Heim" <iheim at redhat.com>
>>>>> To: "Liran Zelkha" <lzelkha at redhat.com>
>>>>> Cc: "engine-devel" <engine-devel at ovirt.org>
>>>>> Sent: Tuesday, June 11, 2013 4:00:21 PM
>>>>> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
>>>>>
>>>>> On 06/11/2013 03:26 PM, Liran Zelkha wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I'm checking performance for VdsUpdateRunTimeInfo.
>>>>>> Naturally, much of the performance surrounds database activity
>>>>>> (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
>>>>>>
>>>>>> Few questions:
>>>>>> 1. I have implemented batch updates for procedure
>>>>>> UpdateVmDeviceRuntimeInfo
>>>>>> for improved performance.
>>>>>> 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting
>>>>>> are
>>>>>> vm_id,vm_device_id,address and alias. Are those rapidly changing, or will
>>>>>> it be beneficial to implement caching on those updates (to ensure
>>>>>> not-required updates do not travel to the database).
>>>>>
>>>>> slowly changing, but how will you cover all flows changing these devices
>>>>> to invalidate the cache (iiuc, this table is modified by engine when
>>>>> adding devices to a VM as well?)
>>>>
>>>>
>>>> I don't think that in the device run time info we need to invalidate once
>>>> we add a device.
>>>> This is a specific case where we actually get the information from the VDSM
>>>> (addresses are received from libvirt)
>>>> The commands IIRC are first send to VDSM and than update the runtime info
>>>> only on changed info (we can also hash it),
>>>> It may put the placeholder in the DB first but it still relies on the data
>>>> received from VDSM.
>>>
>>> if this table is only updated from vdsm to save it, i agree.
>>> but isn't the engine also manipulating it?
>>> wasn't there a request to be able to maybe edit the addresses some day?
>>
>> Even if there was such a request, we still update when we receive the info from libvirt.
>>
>> Anyway there are obviously a few improvements to be done before caching.
>> - update only when info received from VDSM change (need to verify it is the practice here)
>> - do batch update for the new updated data received.
>> - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes
>> - caching ...
>>
>> Barak
>>
>>
>>
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>>
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>