Hi all, I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic) Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).

3. Any additional known performance problems you know of in regards to this class? Thanks... _______________________________________________ Engine-devel mailing list Engine-devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel

----- Original Message ----- > From: "Itamar Heim" <iheim(a)redhat.com&gt; > To: "Liran Zelkha" <lzelkha(a)redhat.com&gt; > Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; > Sent: Tuesday, June 11, 2013 4:00:21 PM > Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo > > On 06/11/2013 03:26 PM, Liran Zelkha wrote: >> Hi all, >> >> I'm checking performance for VdsUpdateRunTimeInfo. >> Naturally, much of the performance surrounds database activity >> (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic) >> >> Few questions: >> 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo >> for improved performance. >> 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are >> vm_id,vm_device_id,address and alias. Are those rapidly changing, or will >> it be beneficial to implement caching on those updates (to ensure >> not-required updates do not travel to the database). > > slowly changing, but how will you cover all flows changing these devices > to invalidate the cache (iiuc, this table is modified by engine when > adding devices to a VM as well?) I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.

----- Original Message ----- > From: "Itamar Heim" <iheim(a)redhat.com&gt; > To: "Barak Azulay" <bazulay(a)redhat.com&gt; > Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; > Sent: Wednesday, June 12, 2013 10:09:09 AM > Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo > > On 06/12/2013 09:41 AM, Barak Azulay wrote: >> >> >> ----- Original Message ----- >>> From: "Itamar Heim" <iheim(a)redhat.com&gt; >>> To: "Liran Zelkha" <lzelkha(a)redhat.com&gt; >>> Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; >>> Sent: Tuesday, June 11, 2013 4:00:21 PM >>> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo >>> >>> On 06/11/2013 03:26 PM, Liran Zelkha wrote: >>>> Hi all, >>>> >>>> I'm checking performance for VdsUpdateRunTimeInfo. >>>> Naturally, much of the performance surrounds database activity >>>> (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic) >>>> >>>> Few questions: >>>> 1. I have implemented batch updates for procedure >>>> UpdateVmDeviceRuntimeInfo >>>> for improved performance. >>>> 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting >>>> are >>>> vm_id,vm_device_id,address and alias. Are those rapidly changing, or will >>>> it be beneficial to implement caching on those updates (to ensure >>>> not-required updates do not travel to the database). >>> >>> slowly changing, but how will you cover all flows changing these devices >>> to invalidate the cache (iiuc, this table is modified by engine when >>> adding devices to a VM as well?) >> >> >> I don't think that in the device run time info we need to invalidate once >> we add a device. >> This is a specific case where we actually get the information from the VDSM >> (addresses are received from libvirt) >> The commands IIRC are first send to VDSM and than update the runtime info >> only on changed info (we can also hash it), >> It may put the placeholder in the DB first but it still relies on the data >> received from VDSM. > > if this table is only updated from vdsm to save it, i agree. > but isn't the engine also manipulating it? > wasn't there a request to be able to maybe edit the addresses some day? Even if there was such a request, we still update when we receive the info from libvirt. Anyway there are obviously a few improvements to be done before caching. - update only when info received from VDSM change (need to verify it is the practice here) - do batch update for the new updated data received. - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes - caching ... Barak > > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel > > > _______________________________________________ Engine-devel mailing list Engine-devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel

Hi all, Current status of VdsUpdateRuntimeInfo (please note - patch is not yet committed). Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB activity. Of that, about 25% of CPU time wasted on update behavior. After modification (moving updates to batch update) UpdateRuntimeInfo performance wasted roughly 40% of CPU time on DB activity. And only 12% of that was wasted on update behavior.

My next task is to try and optimize the rest of overall database performance. ----- Original Message ----- From: "Liran Zelkha" <liran.zelkha(a)gmail.com&gt; To: "Barak Azulay" <bazulay(a)redhat.com&gt; Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; Sent: Wednesday, June 12, 2013 12:35:03 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo Hi guys, I'm working hard on this. I'll send summary mail and findings later today. On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote: > > > ----- Original Message ----- >> From: "Itamar Heim" <iheim(a)redhat.com&gt; >> To: "Barak Azulay" <bazulay(a)redhat.com&gt; >> Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; >> Sent: Wednesday, June 12, 2013 10:09:09 AM >> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo >> >> On 06/12/2013 09:41 AM, Barak Azulay wrote: >>> >>> >>> ----- Original Message ----- >>>> From: "Itamar Heim" <iheim(a)redhat.com&gt; >>>> To: "Liran Zelkha" <lzelkha(a)redhat.com&gt; >>>> Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; >>>> Sent: Tuesday, June 11, 2013 4:00:21 PM >>>> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo >>>> >>>> On 06/11/2013 03:26 PM, Liran Zelkha wrote: >>>>> Hi all, >>>>> >>>>> I'm checking performance for VdsUpdateRunTimeInfo. >>>>> Naturally, much of the performance surrounds database activity >>>>> (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic) >>>>> >>>>> Few questions: >>>>> 1. I have implemented batch updates for procedure >>>>> UpdateVmDeviceRuntimeInfo >>>>> for improved performance. >>>>> 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting >>>>> are >>>>> vm_id,vm_device_id,address and alias. Are those rapidly changing, or will >>>>> it be beneficial to implement caching on those updates (to ensure >>>>> not-required updates do not travel to the database). >>>> >>>> slowly changing, but how will you cover all flows changing these devices >>>> to invalidate the cache (iiuc, this table is modified by engine when >>>> adding devices to a VM as well?) >>> >>> >>> I don't think that in the device run time info we need to invalidate once >>> we add a device. >>> This is a specific case where we actually get the information from the VDSM >>> (addresses are received from libvirt) >>> The commands IIRC are first send to VDSM and than update the runtime info >>> only on changed info (we can also hash it), >>> It may put the placeholder in the DB first but it still relies on the data >>> received from VDSM. >> >> if this table is only updated from vdsm to save it, i agree. >> but isn't the engine also manipulating it? >> wasn't there a request to be able to maybe edit the addresses some day? > > Even if there was such a request, we still update when we receive the info from libvirt. > > Anyway there are obviously a few improvements to be done before caching. > - update only when info received from VDSM change (need to verify it is the practice here) > - do batch update for the new updated data received. > - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes > - caching ... > > Barak > > > >> >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >> >> >> > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel _______________________________________________ Engine-devel mailing list Engine-devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel _______________________________________________ Engine-devel mailing list Engine-devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel

On 06/12/2013 04:03 PM, Liran Zelkha wrote: > Hi all, > > Current status of VdsUpdateRuntimeInfo (please note - patch is not yet committed). > Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB activity. Of that, about 25% of CPU time wasted on update behavior. > After modification (moving updates to batch update) UpdateRuntimeInfo performance wasted roughly 40% of CPU time on DB activity. And only 12% of that was wasted on update behavior. which changes are in place for above? > > My next task is to try and optimize the rest of overall database performance. > > > ----- Original Message ----- > From: "Liran Zelkha" <liran.zelkha(a)gmail.com&gt; > To: "Barak Azulay" <bazulay(a)redhat.com&gt; > Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; > Sent: Wednesday, June 12, 2013 12:35:03 PM > Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo > > Hi guys, > > I'm working hard on this. I'll send summary mail and findings later today. > > On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote: > >> >> >> ----- Original Message ----- >>> From: "Itamar Heim" <iheim(a)redhat.com&gt; >>> To: "Barak Azulay" <bazulay(a)redhat.com&gt; >>> Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; >>> Sent: Wednesday, June 12, 2013 10:09:09 AM >>> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo >>> >>> On 06/12/2013 09:41 AM, Barak Azulay wrote: >>>> >>>> >>>> ----- Original Message ----- >>>>> From: "Itamar Heim" <iheim(a)redhat.com&gt; >>>>> To: "Liran Zelkha" <lzelkha(a)redhat.com&gt; >>>>> Cc: "engine-devel" <engine-devel(a)ovirt.org&gt; >>>>> Sent: Tuesday, June 11, 2013 4:00:21 PM >>>>> Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo >>>>> >>>>> On 06/11/2013 03:26 PM, Liran Zelkha wrote: >>>>>> Hi all, >>>>>> >>>>>> I'm checking performance for VdsUpdateRunTimeInfo. >>>>>> Naturally, much of the performance surrounds database activity >>>>>> (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic) >>>>>> >>>>>> Few questions: >>>>>> 1. I have implemented batch updates for procedure >>>>>> UpdateVmDeviceRuntimeInfo >>>>>> for improved performance. >>>>>> 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting >>>>>> are >>>>>> vm_id,vm_device_id,address and alias. Are those rapidly changing, or will >>>>>> it be beneficial to implement caching on those updates (to ensure >>>>>> not-required updates do not travel to the database). >>>>> >>>>> slowly changing, but how will you cover all flows changing these devices >>>>> to invalidate the cache (iiuc, this table is modified by engine when >>>>> adding devices to a VM as well?) >>>> >>>> >>>> I don't think that in the device run time info we need to invalidate once >>>> we add a device. >>>> This is a specific case where we actually get the information from the VDSM >>>> (addresses are received from libvirt) >>>> The commands IIRC are first send to VDSM and than update the runtime info >>>> only on changed info (we can also hash it), >>>> It may put the placeholder in the DB first but it still relies on the data >>>> received from VDSM. >>> >>> if this table is only updated from vdsm to save it, i agree. >>> but isn't the engine also manipulating it? >>> wasn't there a request to be able to maybe edit the addresses some day? >> >> Even if there was such a request, we still update when we receive the info from libvirt. >> >> Anyway there are obviously a few improvements to be done before caching. >> - update only when info received from VDSM change (need to verify it is the practice here) >> - do batch update for the new updated data received. >> - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes >> - caching ... >> >> Barak >> >> >> >>> >>> _______________________________________________ >>> Engine-devel mailing list >>> Engine-devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>> >>> >>> >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel > > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel >