[Engine-devel] Updates in VdsUpdateRuntimeInfo
Hi all, I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic) Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database). 3. Any additional known performance problems you know of in regards to this class? Thanks...
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
3. Any additional known performance problems you know of in regards to this class?
Thanks... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
3. Any additional known performance problems you know of in regards to this class?
Thanks... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 06/12/2013 09:41 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
if this table is only updated from vdsm to save it, i agree. but isn't the engine also manipulating it? wasn't there a request to be able to maybe edit the addresses some day?
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 10:09:09 AM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/12/2013 09:41 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
if this table is only updated from vdsm to save it, i agree. but isn't the engine also manipulating it? wasn't there a request to be able to maybe edit the addresses some day?
Even if there was such a request, we still update when we receive the info from libvirt. Anyway there are obviously a few improvements to be done before caching. - update only when info received from VDSM change (need to verify it is the practice here) - do batch update for the new updated data received. - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes - caching ... Barak
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Hi guys, I'm working hard on this. I'll send summary mail and findings later today. On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 10:09:09 AM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/12/2013 09:41 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
if this table is only updated from vdsm to save it, i agree. but isn't the engine also manipulating it? wasn't there a request to be able to maybe edit the addresses some day?
Even if there was such a request, we still update when we receive the info from libvirt.
Anyway there are obviously a few improvements to be done before caching. - update only when info received from VDSM change (need to verify it is the practice here) - do batch update for the new updated data received. - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes - caching ...
Barak
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Hi all, Current status of VdsUpdateRuntimeInfo (please note - patch is not yet committed). Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB activity. Of that, about 25% of CPU time wasted on update behavior. After modification (moving updates to batch update) UpdateRuntimeInfo performance wasted roughly 40% of CPU time on DB activity. And only 12% of that was wasted on update behavior. My next task is to try and optimize the rest of overall database performance. ----- Original Message ----- From: "Liran Zelkha" <liran.zelkha@gmail.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 12:35:03 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo Hi guys, I'm working hard on this. I'll send summary mail and findings later today. On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 10:09:09 AM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/12/2013 09:41 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
if this table is only updated from vdsm to save it, i agree. but isn't the engine also manipulating it? wasn't there a request to be able to maybe edit the addresses some day?
Even if there was such a request, we still update when we receive the info from libvirt.
Anyway there are obviously a few improvements to be done before caching. - update only when info received from VDSM change (need to verify it is the practice here) - do batch update for the new updated data received. - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes - caching ...
Barak
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 06/12/2013 04:03 PM, Liran Zelkha wrote:
Hi all,
Current status of VdsUpdateRuntimeInfo (please note - patch is not yet committed). Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB activity. Of that, about 25% of CPU time wasted on update behavior. After modification (moving updates to batch update) UpdateRuntimeInfo performance wasted roughly 40% of CPU time on DB activity. And only 12% of that was wasted on update behavior.
which changes are in place for above?
My next task is to try and optimize the rest of overall database performance.
----- Original Message ----- From: "Liran Zelkha" <liran.zelkha@gmail.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 12:35:03 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
Hi guys,
I'm working hard on this. I'll send summary mail and findings later today.
On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 10:09:09 AM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/12/2013 09:41 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
if this table is only updated from vdsm to save it, i agree. but isn't the engine also manipulating it? wasn't there a request to be able to maybe edit the addresses some day?
Even if there was such a request, we still update when we receive the info from libvirt.
Anyway there are obviously a few improvements to be done before caching. - update only when info received from VDSM change (need to verify it is the practice here) - do batch update for the new updated data received. - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes - caching ...
Barak
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Itamar Heim" <iheim@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 10:45:25 AM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 10:09:09 AM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/12/2013 09:41 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote:
Hi all,
I'm checking performance for VdsUpdateRunTimeInfo. Naturally, much of the performance surrounds database activity (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
Few questions: 1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo for improved performance. 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are vm_id,vm_device_id,address and alias. Are those rapidly changing, or will it be beneficial to implement caching on those updates (to ensure not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
if this table is only updated from vdsm to save it, i agree. but isn't the engine also manipulating it? wasn't there a request to be able to maybe edit the addresses some day?
Even if there was such a request, we still update when we receive the info from libvirt.
Anyway there are obviously a few improvements to be done before caching. - update only when info received from VDSM change (need to verify it is the practice here)
This should be the behavior right now There is a hash column in vm_dynamic , this hash is passed by VDSM We are updating only VMs that change their hash value
- do batch update for the new updated data received. - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes - caching ...
Barak
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Batch updates for UpdateVmDeviceRuntimeInfo, UpdateVdsInterfaceStatistics and UpdateVdsDynamic. All improved execution performance by about 60-80%. On Jun 12, 2013, at 4:05 PM, Itamar Heim wrote:
On 06/12/2013 04:03 PM, Liran Zelkha wrote:
Hi all,
Current status of VdsUpdateRuntimeInfo (please note - patch is not yet committed). Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB activity. Of that, about 25% of CPU time wasted on update behavior. After modification (moving updates to batch update) UpdateRuntimeInfo performance wasted roughly 40% of CPU time on DB activity. And only 12% of that was wasted on update behavior.
which changes are in place for above?
My next task is to try and optimize the rest of overall database performance.
----- Original Message ----- From: "Liran Zelkha" <liran.zelkha@gmail.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 12:35:03 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
Hi guys,
I'm working hard on this. I'll send summary mail and findings later today.
On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Barak Azulay" <bazulay@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Wednesday, June 12, 2013 10:09:09 AM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/12/2013 09:41 AM, Barak Azulay wrote:
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Liran Zelkha" <lzelkha@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org> Sent: Tuesday, June 11, 2013 4:00:21 PM Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
On 06/11/2013 03:26 PM, Liran Zelkha wrote: > Hi all, > > I'm checking performance for VdsUpdateRunTimeInfo. > Naturally, much of the performance surrounds database activity > (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic) > > Few questions: > 1. I have implemented batch updates for procedure > UpdateVmDeviceRuntimeInfo > for improved performance. > 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting > are > vm_id,vm_device_id,address and alias. Are those rapidly changing, or will > it be beneficial to implement caching on those updates (to ensure > not-required updates do not travel to the database).
slowly changing, but how will you cover all flows changing these devices to invalidate the cache (iiuc, this table is modified by engine when adding devices to a VM as well?)
I don't think that in the device run time info we need to invalidate once we add a device. This is a specific case where we actually get the information from the VDSM (addresses are received from libvirt) The commands IIRC are first send to VDSM and than update the runtime info only on changed info (we can also hash it), It may put the placeholder in the DB first but it still relies on the data received from VDSM.
if this table is only updated from vdsm to save it, i agree. but isn't the engine also manipulating it? wasn't there a request to be able to maybe edit the addresses some day?
Even if there was such a request, we still update when we receive the info from libvirt.
Anyway there are obviously a few improvements to be done before caching. - update only when info received from VDSM change (need to verify it is the practice here) - do batch update for the new updated data received. - do the hashing (of vm device runtime info) on VDSM for the data , and update only when hash changes - caching ...
Barak
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
participants (5)
-
Barak Azulay -
Eli Mesika -
Itamar Heim -
Liran Zelkha -
Liran Zelkha