2:25 p.m.
This is a multi-part message in MIME format.
--------------020305070807050907000005
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Please find the prettier version on the wiki:
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
Proposal VDSM - Engine Data Statistics Retrieval
VDSM <=> Engine data retrieval optimization
Motivation:
Currently the RHEVM engine is polling the a lot of data from VDSM every
15 seconds. This should be optimized and the amount of data requested
should be more specific.
For each VM the data currently contains much more information than
actually needed which blows up the size of the XML content quite big. We
could optimize this by splitting the reply on the getVmStats based on
the request of the engine into sections. For this reason Omer Frenkel
and me have split up the data into parts based on their usage.
This data can and usually does change during the lifetime of the VM.
Rarely Changed:
This data is change not very frequent and it should be enough to update
this only once in a while. Most commonly this data changes after changes
made in the UI or after a migration of the VM to another Host.
*Status* = Running
*acpiEnable* = true
*vmType* = kvm
*guestName* = W864GUESTAGENTT
*displayType* = qxl
*guestOs* = Win 8
*kvmEnable* = true #/*this should be constant and never changed*/
*pauseCode* = NOERR
*monitorResponse* = 0
*session* = Locked # unused
*netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}]
*appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3',
'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64
3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3',
'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3',
'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
*pid* = 11314
*guestIPs* = 10.34.60.148 # duplicated info
*displayIp* = 0
*displayPort* = 5902
*displaySecurePort* = 5903
*username* = user@W864GUESTAGENTT
*clientIp* =
*lastLogin* = 1361976900.67
Often Changed:
This data is changed quite often however it is not necessary to update
this data every 15 seconds. As this is cumulative data and reflects the
current status, and it does not need to be snapshotted every 15 seconds
to retrieve statistics. The data can be retrieved in much more generous
time slices. (e.g. Every 5 minutes)
*network* = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0', 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate': '0.0',
'txErrors': '0', 'state': 'unknown', 'speed':
'100', 'name': 'vnet1'}}
*disksUsage* = [{'path': 'c:\\', 'total':
'64055406592', 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
*timeOffset* = 14422
*elapsedTime* = 68591
*hash* = 2335461227228498964
*statsAge* = 0.09 # unused
Often Changed but unused
This data does not seem to be used in the engine at all. It is *not*
even used in the data warehouse.
*memoryStats* = {'swap_out': '0', 'majflt': '0',
'mem_free': '1466884', 'swap_in': '0', 'pageflt':
'0', 'mem_total': '2096736', 'mem_unused':
'1466884'}
*balloonInfo* = {'balloon_max': 2097152, 'balloon_cur': 2097152}
*disks* = {'vda': {'readLatency': '0',
'apparentsize': '64424509440', 'writeLatency': '1754496',
'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize': '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency': '0', 'apparentsize': '0',
'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}}
Very frequent uppdates needed by webadmin portal:
This data is mostly needed for the webadmin portal and might be required
to be updated quite often. An exception here is the statsAge field,
which seems to be unused by the Engine. This data could be requested
every 15 seconds to keep things as they are now.
*cpuSys* = 2.32
*cpuUser* = 1.34
*memUsage* = 30
Proposed Solution for VDSM & Engine:
We will introduce new optional parameters to getVmStats, getAllVmStats
and list to allow a finer grained specification of data which should be
included.
*Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats only)
*Allowed values:*
* full (default to keep backwards compatibility)
* app-list (Just send the application list)
* rare (include everything from rarely changed to very frequent)
* often (include everything from often changed to very frequent)
* frequent (only send the very frequently changed items)
*Parameter:* *clientId*=*<string>* The client id is specified by the
client and should be unique however constantly used.
*Parameter:* *diff*=*<boolean>* In combination with the clientId VDSM
will send only differences to the previous request from the named
clientId. (if diff=true)
Additional Change:
Besides the introduction of the new parameters for list, getVmStats and
getAllVmStats it might make sense to include a hash for the appList into
the rarely changed section of the response which would allow to identify
changes and avoid having to sent the complete appList every so often and
only if the hash known to the client is outdated.
*Note:* The appList (Application List) reported by the guest agent could
be fully implemented on request only, as long as the guest agent
installed supports this. As there seems to be a request to have the
complete list of installed applications on all guests this data could be
quite extensive and a huge list. On the other hand this data is only
rarely visible and therefore it should not be requested all the time and
only on demand.
Improvement of the Guest Agent:
As part of the proposed solution it is necessary to improve the guest
agent as well. For the full application list there should be implemented
a caching system which will be fully reactive and should not poll the
application list for example all the time. The guest can create a
prepared data file containing all data in the JSON format (as used for
the communication with VDSM via VIO) and just have to read that file
from disk and directly sends it to VDSM. However it is quite possible
that this list is to big and it might have to be chunked into pieces.
(Multiple messages, which would have to be supported by VDSM then as
well) The solution for this is to make VDSM request this data and it
will retrieve the data necessary on request only.
--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
--------------020305070807050907000005
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Please find the prettier version on the wiki:
<a class="moz-txt-link-freetext"
href="http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Re...
<br>
<h1 id="firstHeading" class="firstHeading
page-header"><span
dir="auto">Proposal VDSM - Engine Data Statistics
Retrieval</span></h1>
<h2><span class="mw-headline"
id="VDSM_.3C.3D.3E_Engine_data_retrieval_optimization">VDSM
<=> Engine data retrieval optimization </span></h2>
<h3> <span class="mw-headline" id="Motivation:">
Motivation: </span></h3>
<p>Currently the RHEVM engine is polling the a lot of data from VDSM
every 15 seconds. This should be optimized and the amount of data
requested should be more specific.
</p>
<p>For each VM the data currently contains much more information
than actually needed which blows up the size of the XML content
quite big. We could optimize this by splitting the reply on the
getVmStats based on the request of the engine into sections.
For this reason Omer Frenkel and me have split up the data into
parts based on their usage.
</p>
<p>This data can and usually does change during the lifetime of the
VM.
</p>
<h4> <span class="mw-headline" id="Rarely_Changed:">
Rarely
Changed: </span></h4>
<p>This data is change not very frequent and it should be enough to
update this only once in a while.
Most commonly this data changes after changes made in the UI or
after a migration of the VM to another Host.
</p>
<pre> <b>Status</b> = Running
<b>acpiEnable</b> = true
<b>vmType</b> = kvm
<b>guestName</b> = W864GUESTAGENTT
<b>displayType</b> = qxl
<b>guestOs</b> = Win 8
<b>kvmEnable</b> = true # <i><b>this should be constant and
never changed</b></i>
<b>pauseCode</b> = NOERR
<b>monitorResponse</b> = 0
<b>session</b> = Locked # unused
<b>netIfaces</b> = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC', 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}]
<b>appsList</b> = ['RHEV-Tools 3.2.4', 'RHEV-Agent64
3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2',
'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64
3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB
3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
<b>pid</b> = 11314
<b>guestIPs</b> = 10.34.60.148 # duplicated info
<b>displayIp</b> = 0
<b>displayPort</b> = 5902
<b>displaySecurePort</b> = 5903
<b>username</b> = user@W864GUESTAGENTT
<b>clientIp</b> =
<b>lastLogin</b> = 1361976900.67
</pre>
<h4> <span class="mw-headline" id="Often_Changed:">
Often Changed:
</span></h4>
<p>This data is changed quite often however it is not necessary to
update this data every 15 seconds. As this is cumulative data and
reflects the current status, and it does not need to be
snapshotted every 15 seconds to retrieve statistics. The data can
be retrieved in much more generous time slices. (e.g. Every 5
minutes)
</p>
<pre> <b>network</b> = {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db', 'rxDropped': '0', 'txDropped':
'0', 'rxErrors': '0', 'txRate': '0.0',
'rxRate': '0.0', 'txErrors': '0', 'state':
'unknown', 'speed': '100', 'name': 'vnet1'}}
<b>disksUsage</b> = [{'path': 'c:\\', 'total':
'64055406592', 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
<b>timeOffset</b> = 14422
<b>elapsedTime</b> = 68591
<b>hash</b> = 2335461227228498964
<b>statsAge</b> = 0.09 # unused
</pre>
<h4> <span class="mw-headline"
id="Often_Changed_but_unused"> Often
Changed but unused </span></h4>
<p>This data does not seem to be used in the engine at all. It is
<b>not</b>
even used in the data warehouse.
</p>
<pre> <b>memoryStats</b> = {'swap_out': '0',
'majflt': '0', 'mem_free': '1466884', 'swap_in':
'0', 'pageflt': '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
<b>balloonInfo</b> = {'balloon_max': 2097152,
'balloon_cur': 2097152}
<b>disks</b> = {'vda': {'readLatency': '0',
'apparentsize': '64424509440', 'writeLatency': '1754496',
'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize': '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency': '0', 'apparentsize': '0',
'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}}
</pre>
<h4> <span class="mw-headline"
id="Very_frequent_uppdates_needed_by_webadmin_portal:"> Very
frequent uppdates needed by webadmin portal: </span></h4>
<p>This data is mostly needed for the webadmin portal and might be
required to be updated quite often. An exception here is the
statsAge field, which seems to be unused by the Engine.
This data could be requested every 15 seconds to keep things as
they are now.
</p>
<pre> <b>cpuSys</b> = 2.32
<b>cpuUser</b> = 1.34
<b>memUsage</b> = 30
</pre>
<h2> <span class="mw-headline"
id="Proposed_Solution_for_VDSM_.26_Engine:"> Proposed Solution
for VDSM & Engine: </span></h2>
<p>We will introduce new optional parameters to getVmStats,
getAllVmStats and list to allow a finer grained specification of
data which should be included.
</p>
<p><b>Parameter:</b>
<b>statsType</b>=<i><b><string></b></i>
(getVmStats, getAllVmStats only)
<b>Allowed values:</b>
</p>
<ul>
<li> full (default to keep backwards compatibility)
</li>
<li> app-list (Just send the application list)
</li>
<li> rare (include everything from rarely changed to very
frequent)
</li>
<li> often (include everything from often changed to very
frequent)
</li>
<li> frequent (only send the very frequently changed items)
</li>
</ul>
<p><br>
<b>Parameter:</b>
<b>clientId</b>=<b><string></b>
The client id is specified by the client and should be unique
however constantly used.
</p>
<p><b>Parameter:</b>
<b>diff</b>=<b><boolean></b>
In combination with the clientId VDSM will send only differences
to the previous request from the named clientId. (if diff=true)
</p>
<p><br>
</p>
<h3> <span class="mw-headline"
id="Additional_Change:"> Additional
Change: </span></h3>
<p>Besides the introduction of the new parameters for list,
getVmStats and getAllVmStats it might make sense to include a hash
for the appList into the rarely changed section of the response
which would allow to identify changes and avoid having to sent the
complete appList every so often and only if the hash known to the
client is outdated.
</p>
<p><b>Note:</b> The appList (Application List) reported by the
guest
agent could be fully implemented on request only, as long as the
guest agent installed supports this. As there seems to be a
request to have the complete list of installed applications on all
guests this data could be quite extensive and a huge list. On the
other hand this data is only rarely visible and therefore it
should not be requested all the time and only on demand.
</p>
<h3> <span class="mw-headline"
id="Improvement_of_the_Guest_Agent:">
Improvement of the Guest Agent: </span></h3>
<p>As part of the proposed solution it is necessary to improve the
guest agent as well. For the full application list there should be
implemented a caching system which will be fully reactive and
should not poll the application list for example all the time.
The guest can create a prepared data file containing all data in
the JSON format (as used for the communication with VDSM via VIO)
and just have to read that file from disk and directly sends it to
VDSM.
However it is quite possible that this list is to big and it might
have to be chunked into pieces. (Multiple messages, which would
have to be supported by VDSM then as well)
The solution for this is to make VDSM request this data and it
will retrieve the data necessary on request only.
</p>
<pre class="moz-signature" cols="72">--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com</pre>
</body>
</html>
--------------020305070807050907000005--
1:11 a.m.
On Thu, Mar 07, 2013 at 12:25:54PM +0100, Vinzenz Feenstra wrote:
Please find the prettier version on the wiki:
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
Proposal VDSM - Engine Data Statistics Retrieval
VDSM <=> Engine data retrieval optimization
Motivation:
Currently the RHEVM engine is polling the a lot of data from VDSM
every 15 seconds. This should be optimized and the amount of data
requested should be more specific.
It feels like a good idea, but do you have numbers? How much traffic
would be saved? Remember the added computation incurred on each host -
there's always a price to pay.
For each VM the data currently contains much more information than
actually needed which blows up the size of the XML content quite
big. We could optimize this by splitting the reply on the getVmStats
based on the request of the engine into sections. For this reason
Omer Frenkel and me have split up the data into parts based on their
usage.
This data can and usually does change during the lifetime of the VM.
Rarely Changed:
This data is change not very frequent and it should be enough to
update this only once in a while. Most commonly this data changes
after changes made in the UI or after a migration of the VM to
another Host.
*Status* = Running
Status does not change much, but when it does, it is important to report
that quickly.
*acpiEnable* = true
*vmType* = kvm
*guestName* = W864GUESTAGENTT
*displayType* = qxl
*guestOs* = Win 8
*kvmEnable* = true #/*this should be constant and never changed*/
*pauseCode* = NOERR
*monitorResponse* = 0
*session* = Locked # unused
*netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}]
*appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3',
'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64
3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3',
'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3',
'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
*pid* = 11314
*guestIPs* = 10.34.60.148 # duplicated info
*displayIp* = 0
*displayPort* = 5902
*displaySecurePort* = 5903
*username* = user@W864GUESTAGENTT
*clientIp* =
*lastLogin* = 1361976900.67
Often Changed:
This data is changed quite often however it is not necessary to
update this data every 15 seconds. As this is cumulative data and
reflects the current status, and it does not need to be snapshotted
every 15 seconds to retrieve statistics. The data can be retrieved
in much more generous time slices. (e.g. Every 5 minutes)
*network* = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0', 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate': '0.0',
'txErrors': '0', 'state': 'unknown', 'speed':
'100', 'name': 'vnet1'}}
*disksUsage* = [{'path': 'c:\\', 'total':
'64055406592', 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
*timeOffset* = 14422
*elapsedTime* = 68591
*hash* = 2335461227228498964
*statsAge* = 0.09 # unused
Often Changed but unused
This data does not seem to be used in the engine at all. It is *not*
even used in the data warehouse.
*memoryStats* = {'swap_out': '0', 'majflt': '0',
'mem_free': '1466884', 'swap_in': '0', 'pageflt':
'0', 'mem_total': '2096736', 'mem_unused':
'1466884'}
*balloonInfo* = {'balloon_max': 2097152, 'balloon_cur': 2097152}
*disks* = {'vda': {'readLatency': '0',
'apparentsize': '64424509440', 'writeLatency': '1754496',
'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize': '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency': '0', 'apparentsize': '0',
'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}}
I am pretty sure that {read,write,flush}Latency is collected and
reported by Engine. `git grep writeLatency` reinforces my vague memory.
Very frequent uppdates needed by webadmin portal:
This data is mostly needed for the webadmin portal and might be
required to be updated quite often. An exception here is the
statsAge field, which seems to be unused by the Engine. This data
could be requested every 15 seconds to keep things as they are now.
*cpuSys* = 2.32
*cpuUser* = 1.34
*memUsage* = 30
Proposed Solution for VDSM & Engine:
We will introduce new optional parameters to getVmStats,
getAllVmStats and list to allow a finer grained specification of
data which should be included.
*Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats
only) *Allowed values:*
* full (default to keep backwards compatibility)
* app-list (Just send the application list)
* rare (include everything from rarely changed to very frequent)
* often (include everything from often changed to very frequent)
* frequent (only send the very frequently changed items)
I think that a nice way to think of this, is that Engine ask for a set
of keys it is interested about. Asking for getVmStats(keys=[displayType,
netIfaces]) would return only the requrested values of the VM. "full",
"rare", "often" and "frequent" are simply pre-defined sets
of key names.
A side effect of this pov is that we can avoid the vague name
"statsType".
*Parameter:* *clientId*=*<string>* The client id is specified by the
client and should be unique however constantly used.
*Parameter:* *diff*=*<boolean>* In combination with the clientId
VDSM will send only differences to the previous request from the
named clientId. (if diff=true)
The semantics of "diff" is not completely defined: how about complex
structures like that of "network"? It is most likely to be reported
every time.
Since this requires a caching mechanism on vdsm side, Engine must expect
that the cache may be evicted in any moment, and that a full list is
received.
Additional Change:
Besides the introduction of the new parameters for list, getVmStats
and getAllVmStats it might make sense to include a hash for the
appList into the rarely changed section of the response which would
allow to identify changes and avoid having to sent the complete
appList every so often and only if the hash known to the client is
outdated.
*Note:* The appList (Application List) reported by the guest agent
could be fully implemented on request only, as long as the guest
agent installed supports this. As there seems to be a request to
have the complete list of installed applications on all guests this
data could be quite extensive and a huge list. On the other hand
this data is only rarely visible and therefore it should not be
requested all the time and only on demand.
Improvement of the Guest Agent:
As part of the proposed solution it is necessary to improve the
guest agent as well.
Improving the agent may be a good idea, but I do not see the necessity
in it. It's also important to improve the horrible multithreaded
vdsm/libvirt statistics acquisition, but just as unrelated to the core
of this feature.
For the full application list there should be
implemented a caching system which will be fully reactive and should
not poll the application list for example all the time. The guest
can create a prepared data file containing all data in the JSON
format (as used for the communication with VDSM via VIO) and just
have to read that file from disk and directly sends it to VDSM.
However it is quite possible that this list is to big and it might
have to be chunked into pieces. (Multiple messages, which would have
to be supported by VDSM then as well) The solution for this is to
make VDSM request this data and it will retrieve the data necessary
on request only.
5:30 a.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On 03/08/2013 06:11 AM, Dan Kenigsberg wrote:
On Thu, Mar 07, 2013 at 12:25:54PM +0100, Vinzenz Feenstra wrote:
> Please find the prettier version on the wiki:
> http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>
>
> Proposal VDSM - Engine Data Statistics Retrieval
>
>
> VDSM <=> Engine data retrieval optimization
>
>
> Motivation:
>
> Currently the RHEVM engine is polling the a lot of data from VDSM
> every 15 seconds. This should be optimized and the amount of data
> requested should be more specific.
It feels like a good idea, but do you have numbers? How much traffic
would be saved? Remember the added computation incurred on each host -
there's always a price to pay.
> For each VM the data currently contains much more information than
> actually needed which blows up the size of the XML content quite
> big. We could optimize this by splitting the reply on the getVmStats
> based on the request of the engine into sections. For this reason
> Omer Frenkel and me have split up the data into parts based on their
> usage.
>
> This data can and usually does change during the lifetime of the VM.
>
>
> Rarely Changed:
>
> This data is change not very frequent and it should be enough to
> update this only once in a while. Most commonly this data changes
> after changes made in the UI or after a migration of the VM to
> another Host.
>
> *Status* = Running
Status does not change much, but when it does, it is important to report
that quickly.
For this kind of data, it is suitable to use an event report, which
should be available in the jsonrpc API.
> *acpiEnable* = true
> *vmType* = kvm
> *guestName* = W864GUESTAGENTT
> *displayType* = qxl
> *guestOs* = Win 8
> *kvmEnable* = true #/*this should be constant and never changed*/
> *pauseCode* = NOERR
> *monitorResponse* = 0
> *session* = Locked # unused
> *netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}]
> *appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3',
'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64
3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3',
'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3',
'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
> *pid* = 11314
> *guestIPs* = 10.34.60.148 # duplicated info
> *displayIp* = 0
> *displayPort* = 5902
> *displaySecurePort* = 5903
> *username* = user@W864GUESTAGENTT
> *clientIp* =
> *lastLogin* = 1361976900.67
>
>
> Often Changed:
>
> This data is changed quite often however it is not necessary to
> update this data every 15 seconds. As this is cumulative data and
> reflects the current status, and it does not need to be snapshotted
> every 15 seconds to retrieve statistics. The data can be retrieved
> in much more generous time slices. (e.g. Every 5 minutes)
>
> *network* = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0', 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate': '0.0',
'txErrors': '0', 'state': 'unknown', 'speed':
'100', 'name': 'vnet1'}}
> *disksUsage* = [{'path': 'c:\\', 'total':
'64055406592', 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
> *timeOffset* = 14422
> *elapsedTime* = 68591
> *hash* = 2335461227228498964
> *statsAge* = 0.09 # unused
>
>
> Often Changed but unused
>
> This data does not seem to be used in the engine at all. It is *not*
> even used in the data warehouse.
>
> *memoryStats* = {'swap_out': '0', 'majflt': '0',
'mem_free': '1466884', 'swap_in': '0', 'pageflt':
'0', 'mem_total': '2096736', 'mem_unused':
'1466884'}
> *balloonInfo* = {'balloon_max': 2097152, 'balloon_cur':
2097152}
> *disks* = {'vda': {'readLatency': '0',
'apparentsize': '64424509440', 'writeLatency': '1754496',
'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize': '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency': '0', 'apparentsize': '0',
'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}}
I am pretty sure that {read,write,flush}Latency is collected and
reported by Engine. `git grep writeLatency` reinforces my vague memory.
>
> Very frequent uppdates needed by webadmin portal:
>
> This data is mostly needed for the webadmin portal and might be
> required to be updated quite often. An exception here is the
> statsAge field, which seems to be unused by the Engine. This data
> could be requested every 15 seconds to keep things as they are now.
>
> *cpuSys* = 2.32
> *cpuUser* = 1.34
> *memUsage* = 30
>
>
> Proposed Solution for VDSM & Engine:
>
> We will introduce new optional parameters to getVmStats,
> getAllVmStats and list to allow a finer grained specification of
> data which should be included.
>
> *Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats
> only) *Allowed values:*
>
> * full (default to keep backwards compatibility)
> * app-list (Just send the application list)
> * rare (include everything from rarely changed to very frequent)
> * often (include everything from often changed to very frequent)
> * frequent (only send the very frequently changed items)
I think that a nice way to think of this, is that Engine ask for a set
of keys it is interested about. Asking for getVmStats(keys=[displayType,
netIfaces]) would return only the requrested values of the VM.
+1. It could split
the information according to different functions,
not just change frequency.
"full",
"rare", "often" and "frequent" are simply pre-defined sets
of key names.
A side effect of this pov is that we can avoid the vague name
"statsType".
>
> *Parameter:* *clientId*=*<string>* The client id is specified by the
> client and should be unique however constantly used.
>
> *Parameter:* *diff*=*<boolean>* In combination with the clientId
> VDSM will send only differences to the previous request from the
> named clientId. (if diff=true)
The semantics of "diff" is not completely defined: how about complex
structures like that of "network"? It is most likely to be reported
every time.
Since this requires a caching mechanism on vdsm side, Engine must expect
that the cache may be evicted in any moment, and that a full list is
received.
Every data collector should be responsible to invalidate/update the
cache.
It could reduce the time to calculate the diff.
>
> Additional Change:
>
> Besides the introduction of the new parameters for list, getVmStats
> and getAllVmStats it might make sense to include a hash for the
> appList into the rarely changed section of the response which would
> allow to identify changes and avoid having to sent the complete
> appList every so often and only if the hash known to the client is
> outdated.
>
> *Note:* The appList (Application List) reported by the guest agent
> could be fully implemented on request only, as long as the guest
> agent installed supports this. As there seems to be a request to
> have the complete list of installed applications on all guests this
> data could be quite extensive and a huge list. On the other hand
> this data is only rarely visible and therefore it should not be
> requested all the time and only on demand.
>
>
> Improvement of the Guest Agent:
>
> As part of the proposed solution it is necessary to improve the
> guest agent as well.
Improving the agent may be a good idea, but I do not see the necessity
in it. It's also important to improve the horrible multithreaded
vdsm/libvirt statistics acquisition, but just as unrelated to the core
of this feature.
> For the full application list there should be
> implemented a caching system which will be fully reactive and should
> not poll the application list for example all the time. The guest
> can create a prepared data file containing all data in the JSON
> format (as used for the communication with VDSM via VIO) and just
> have to read that file from disk and directly sends it to VDSM.
> However it is quite possible that this list is to big and it might
> have to be chunked into pieces. (Multiple messages, which would have
> to be supported by VDSM then as well) The solution for this is to
> make VDSM request this data and it will retrieve the data necessary
> on request only.
_______________________________________________
vdsm-devel mailing list
vdsm-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
11:37 a.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On 03/08/2013 03:30 AM, Mark Wu wrote:
On 03/08/2013 06:11 AM, Dan Kenigsberg wrote:
> On Thu, Mar 07, 2013 at 12:25:54PM +0100, Vinzenz Feenstra wrote:
>> Please find the prettier version on the wiki:
>> http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>
>>
>> Proposal VDSM - Engine Data Statistics Retrieval
>>
>>
>> VDSM <=> Engine data retrieval optimization
>>
>>
>> Motivation:
>>
>> Currently the RHEVM engine is polling the a lot of data from VDSM
>> every 15 seconds. This should be optimized and the amount of data
>> requested should be more specific.
> It feels like a good idea, but do you have numbers? How much traffic
> would be saved? Remember the added computation incurred on each host -
> there's always a price to pay.
Well the data of a single really basic
simple VM has about 4 KiB data in
the output of vdsClient, the XMLRPC formatted body part has almost 16KiB.
The thing is that this data is queried every 15 seconds (previously 10)
with little value for having ALL data sent all the time, the engine is
not even using all of the data all the time.
This optimization must be seen on a bigger scale, if you have a
datacenter with let's say 1000 VMs then the data needed to be
transmitted and parsed by the engine every 15 seconds is about 16MiB.
This optimization wouldn't pay off that much in a 2 server 20 VM
datacenter however on a larger scale it has quite a big impact.
>
>> For each VM the data currently contains much more information than
>> actually needed which blows up the size of the XML content quite
>> big. We could optimize this by splitting the reply on the getVmStats
>> based on the request of the engine into sections. For this reason
>> Omer Frenkel and me have split up the data into parts based on their
>> usage.
>>
>> This data can and usually does change during the lifetime of the VM.
>>
>>
>> Rarely Changed:
>>
>> This data is change not very frequent and it should be enough to
>> update this only once in a while. Most commonly this data changes
>> after changes made in the UI or after a migration of the VM to
>> another Host.
>>
>> *Status* = Running
> Status does not change much, but when it does, it is important to report
> that quickly.
This is done by the list command which is executed every 2
seconds
(maybe 3?)
For this kind of data, it is suitable to use an event report, which
should be available in the jsonrpc API.
>
>> *acpiEnable* = true
>> *vmType* = kvm
>> *guestName* = W864GUESTAGENTT
>> *displayType* = qxl
>> *guestOs* = Win 8
>> *kvmEnable* = true #/*this should be constant and never changed*/
>> *pauseCode* = NOERR
>> *monitorResponse* = 0
>> *session* = Locked # unused
>> *netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC',
>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'],
>> 'hw': '00:1a:4a:22:3c:db'}]
>> *appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3',
>> 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2',
'RHEV-Network64
>> 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3',
>> 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB
3.2.3',
>> 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
>> *pid* = 11314
>> *guestIPs* = 10.34.60.148 # duplicated info
>> *displayIp* = 0
>> *displayPort* = 5902
>> *displaySecurePort* = 5903
>> *username* = user@W864GUESTAGENTT
>> *clientIp* =
>> *lastLogin* = 1361976900.67
>>
>>
>> Often Changed:
>>
>> This data is changed quite often however it is not necessary to
>> update this data every 15 seconds. As this is cumulative data and
>> reflects the current status, and it does not need to be snapshotted
>> every 15 seconds to retrieve statistics. The data can be retrieved
>> in much more generous time slices. (e.g. Every 5 minutes)
>>
>> *network* = {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db',
>> 'rxDropped': '0', 'txDropped': '0',
'rxErrors': '0', 'txRate':
>> '0.0', 'rxRate': '0.0', 'txErrors': '0',
'state': 'unknown',
>> 'speed': '100', 'name': 'vnet1'}}
>> *disksUsage* = [{'path': 'c:\\', 'total':
'64055406592', 'fs':
>> 'NTFS', 'used': '19223846912'}, {'path':
'd:\\', 'total':
>> '3490912256', 'fs': 'UDF', 'used':
'3490912256'}]
>> *timeOffset* = 14422
>> *elapsedTime* = 68591
>> *hash* = 2335461227228498964
>> *statsAge* = 0.09 # unused
>>
>>
>> Often Changed but unused
>>
>> This data does not seem to be used in the engine at all. It is *not*
>> even used in the data warehouse.
>>
>> *memoryStats* = {'swap_out': '0', 'majflt':
'0', 'mem_free':
>> '1466884', 'swap_in': '0', 'pageflt':
'0', 'mem_total': '2096736',
>> 'mem_unused': '1466884'}
>> *balloonInfo* = {'balloon_max': 2097152, 'balloon_cur':
2097152}
>> *disks* = {'vda': {'readLatency': '0',
'apparentsize':
>> '64424509440', 'writeLatency': '1754496',
'imageID':
>> '28abb923-7b89-4638-84f8-1700f0b76482', 'flushLatency':
'156549',
>> 'readRate': '0.00', 'truesize': '18855059456',
'writeRate':
>> '952.05'}, 'hdc': {'readLatency': '0',
'apparentsize': '0',
>> 'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00',
>> 'truesize': '0', 'writeRate': '0.00'}}
> I am pretty sure that {read,write,flush}Latency is collected and
> reported by Engine. `git grep writeLatency` reinforces my vague memory.
Ok,
well we did just a quite quick query about the usage and we searched
rather for the keys than for the individual entries. Good to know what
we need to be a bit more specific about the individual entries to
classify them more appropriate.
>>
>> Very frequent uppdates needed by webadmin portal:
>>
>> This data is mostly needed for the webadmin portal and might be
>> required to be updated quite often. An exception here is the
>> statsAge field, which seems to be unused by the Engine. This data
>> could be requested every 15 seconds to keep things as they are now.
>>
>> *cpuSys* = 2.32
>> *cpuUser* = 1.34
>> *memUsage* = 30
>>
>>
>> Proposed Solution for VDSM & Engine:
>>
>> We will introduce new optional parameters to getVmStats,
>> getAllVmStats and list to allow a finer grained specification of
>> data which should be included.
>>
>> *Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats
>> only) *Allowed values:*
>>
>> * full (default to keep backwards compatibility)
>> * app-list (Just send the application list)
>> * rare (include everything from rarely changed to very frequent)
>> * often (include everything from often changed to very frequent)
>> * frequent (only send the very frequently changed items)
> I think that a nice way to think of this, is that Engine ask for a set
> of keys it is interested about. Asking for getVmStats(keys=[displayType,
> netIfaces]) would return only the requrested values of the VM.
I was thinking
of that as well or a way to exclude things from the list.
+1. It could split the information according to different functions,
not just change frequency.
I would say to go for either or, both wouldn't make
much sense.
> "full",
> "rare", "often" and "frequent" are simply pre-defined
sets of key names.
>
> A side effect of this pov is that we can avoid the vague name
> "statsType".
>
>>
>> *Parameter:* *clientId*=*<string>* The client id is specified by the
>> client and should be unique however constantly used.
>>
>> *Parameter:* *diff*=*<boolean>* In combination with the clientId
>> VDSM will send only differences to the previous request from the
>> named clientId. (if diff=true)
> The semantics of "diff" is not completely defined: how about complex
> structures like that of "network"? It is most likely to be reported
> every time.
Well the idea was a per key evaluation, maybe in cases like network
and
disks per device/interface.
>
> Since this requires a caching mechanism on vdsm side, Engine must expect
> that the cache may be evicted in any moment, and that a full list is
> received.
Well the engine should always expect that.
Every data collector should be responsible to invalidate/update the
cache.
It could reduce the time to calculate the diff.
>>
>> Additional Change:
>>
>> Besides the introduction of the new parameters for list, getVmStats
>> and getAllVmStats it might make sense to include a hash for the
>> appList into the rarely changed section of the response which would
>> allow to identify changes and avoid having to sent the complete
>> appList every so often and only if the hash known to the client is
>> outdated.
>>
>> *Note:* The appList (Application List) reported by the guest agent
>> could be fully implemented on request only, as long as the guest
>> agent installed supports this. As there seems to be a request to
>> have the complete list of installed applications on all guests this
>> data could be quite extensive and a huge list. On the other hand
>> this data is only rarely visible and therefore it should not be
>> requested all the time and only on demand.
>>
>>
>> Improvement of the Guest Agent:
>>
>> As part of the proposed solution it is necessary to improve the
>> guest agent as well.
> Improving the agent may be a good idea, but I do not see the necessity
> in it.
The guest agent is doing 'expensive' queries (e.g.
"application_list")
way too often. And things like network interfaces, disk usage and
installed applications won't usually change every n minutes.
Those queries could be much more reactive then proactive.
> It's also important to improve the horrible multithreaded
> vdsm/libvirt statistics acquisition, but just as unrelated to the core
> of this feature.
>
>> For the full application list there should be
>> implemented a caching system which will be fully reactive and should
>> not poll the application list for example all the time. The guest
>> can create a prepared data file containing all data in the JSON
>> format (as used for the communication with VDSM via VIO) and just
>> have to read that file from disk and directly sends it to VDSM.
>> However it is quite possible that this list is to big and it might
>> have to be chunked into pieces. (Multiple messages, which would have
>> to be supported by VDSM then as well) The solution for this is to
>> make VDSM request this data and it will retrieve the data necessary
>> on request only.
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel(a)lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
12:47 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On Fri, Mar 08, 2013 at 09:37:03AM +0100, Vinzenz Feenstra wrote:
<snip>
>>>
>>>
>>> Improvement of the Guest Agent:
>>>
>>>As part of the proposed solution it is necessary to improve the
>>>guest agent as well.
>>Improving the agent may be a good idea, but I do not see the necessity
>>in it.
The guest agent is doing 'expensive' queries (e.g.
"application_list") way too often. And things like network
interfaces, disk usage and installed applications won't usually
change every n minutes.
Those queries could be much more reactive then proactive.
Of course, but as I said:
>>It's also important to improve the horrible
multithreaded
>>vdsm/libvirt statistics acquisition, but just as unrelated to the core
>>of this feature.
I think it is misleading to include this in the discussion about
vdsm/Engine interface change.
2 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On 03/08/2013 12:11 AM, Dan Kenigsberg wrote:
> This data is change not very frequent and it should be enough to
> >update this only once in a while. Most commonly this data changes
> >after changes made in the UI or after a migration of the VM to
> >another Host.
> >
> > *Status* = Running
Status does not change much, but when it does, it is important to report
that quickly.
actually, status and one other field are polled every 3 seconds iirc
regardless of getAllVmStats (list table or something like that)
5:21 a.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
This is a multi-part message in MIME format.
--------------050703010307000605090101
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
On 03/07/2013 07:25 PM, Vinzenz Feenstra wrote:
Please find the prettier version on the wiki:
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
Proposal VDSM - Engine Data Statistics Retrieval
VDSM <=> Engine data retrieval optimization
Motivation:
Currently the RHEVM engine is polling the a lot of data from VDSM
every 15 seconds. This should be optimized and the amount of data
requested should be more specific.
If the data size really matters, we could also consider to pack the
information into binary. I am not sure if it's suitable in the
transmission of XMLRPC.
For each VM the data currently contains much more information than
actually needed which blows up the size of the XML content quite big.
We could optimize this by splitting the reply on the getVmStats based
on the request of the engine into sections. For this reason Omer
Frenkel and me have split up the data into parts based on their usage.
This data can and usually does change during the lifetime of the VM.
Rarely Changed:
This data is change not very frequent and it should be enough to
update this only once in a while. Most commonly this data changes
after changes made in the UI or after a migration of the VM to another
Host.
*Status* = Running
*acpiEnable* = true
*vmType* = kvm
*guestName* = W864GUESTAGENTT
*displayType* = qxl
*guestOs* = Win 8
*kvmEnable* = true #/*this should be constant and never changed*/
Then it
should be removed from vm stats. In my opinion, any information
belongs to vm's static configuration, it shouldn't be included in vm
stats. For the fields above, except 'Status', engine can get the
information without querying the vdsm host. It could not be changed by
vdsm itself, right?
*pauseCode* = NOERR
*monitorResponse* = 0
*session* = Locked # unused
*netIfaces* = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}]
*appsList* = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3',
'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64
3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64 3.2.3',
'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB 3.2.3',
'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
*pid* = 11314
*guestIPs* = 10.34.60.148 # duplicated info
*displayIp* = 0
*displayPort* = 5902
*displaySecurePort* = 5903
*username* = user@W864GUESTAGENTT
*clientIp* =
*lastLogin* = 1361976900.67
Often Changed:
This data is changed quite often however it is not necessary to update
this data every 15 seconds. As this is cumulative data and reflects
the current status, and it does not need to be snapshotted every 15
seconds to retrieve statistics. The data can be retrieved in much more
generous time slices. (e.g. Every 5 minutes)
*network* = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0', 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate': '0.0',
'txErrors': '0', 'state': 'unknown', 'speed':
'100', 'name': 'vnet1'}}
macAddr and name don't change
either.
*disksUsage* = [{'path': 'c:\\',
'total': '64055406592', 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
*timeOffset* = 14422
*elapsedTime* = 68591
*hash* = 2335461227228498964
*statsAge* = 0.09 # unused
Often Changed but unused
This data does not seem to be used in the engine at all. It is *not*
even used in the data warehouse.
*memoryStats* = {'swap_out': '0', 'majflt': '0',
'mem_free': '1466884', 'swap_in': '0', 'pageflt':
'0', 'mem_total': '2096736', 'mem_unused':
'1466884'}
*balloonInfo* = {'balloon_max': 2097152, 'balloon_cur': 2097152}
It's used by mom to adjust memory overcommitment dynamically.
*disks* = {'vda': {'readLatency': '0',
'apparentsize': '64424509440', 'writeLatency': '1754496',
'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize': '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency': '0', 'apparentsize': '0',
'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}}
Very frequent uppdates needed by webadmin portal:
This data is mostly needed for the webadmin portal and might be
required to be updated quite often. An exception here is the statsAge
field, which seems to be unused by the Engine. This data could be
requested every 15 seconds to keep things as they are now.
*cpuSys* = 2.32
*cpuUser* = 1.34
*memUsage* = 30
Proposed Solution for VDSM & Engine:
We will introduce new optional parameters to getVmStats, getAllVmStats
and list to allow a finer grained specification of data which should
be included.
*Parameter:* *statsType*=/*<string>*/ (getVmStats, getAllVmStats only)
*Allowed values:*
* full (default to keep backwards compatibility)
* app-list (Just send the application list)
* rare (include everything from rarely changed to very frequent)
* often (include everything from often changed to very frequent)
* frequent (only send the very frequently changed items)
*Parameter:* *clientId*=*<string>* The client id is specified by the
client and should be unique however constantly used.
*Parameter:* *diff*=*<boolean>* In combination with the clientId VDSM
will send only differences to the previous request from the named
clientId. (if diff=true)
Additional Change:
Besides the introduction of the new parameters for list, getVmStats
and getAllVmStats it might make sense to include a hash for the
appList into the rarely changed section of the response which would
allow to identify changes and avoid having to sent the complete
appList every so often and only if the hash known to the client is
outdated.
*Note:* The appList (Application List) reported by the guest agent
could be fully implemented on request only, as long as the guest agent
installed supports this. As there seems to be a request to have the
complete list of installed applications on all guests this data could
be quite extensive and a huge list. On the other hand this data is
only rarely visible and therefore it should not be requested all the
time and only on demand.
Improvement of the Guest Agent:
As part of the proposed solution it is necessary to improve the guest
agent as well. For the full application list there should be
implemented a caching system which will be fully reactive and should
not poll the application list for example all the time. The guest can
create a prepared data file containing all data in the JSON format (as
used for the communication with VDSM via VIO) and just have to read
that file from disk and directly sends it to VDSM. However it is quite
possible that this list is to big and it might have to be chunked into
pieces. (Multiple messages, which would have to be supported by VDSM
then as well) The solution for this is to make VDSM request this data
and it will retrieve the data necessary on request only.
--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
_______________________________________________
vdsm-devel mailing list
vdsm-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
--------------050703010307000605090101
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 03/07/2013 07:25 PM, Vinzenz
Feenstra wrote:<br>
</div>
<blockquote cite="mid:51387942.606@redhat.com" type="cite">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
Please find the prettier version on the wiki: <a
moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Re...
<br>
<h1 id="firstHeading" class="firstHeading
page-header"><span
dir="auto">Proposal VDSM - Engine Data Statistics
Retrieval</span></h1>
<h2><span class="mw-headline"
id="VDSM_.3C.3D.3E_Engine_data_retrieval_optimization">VDSM
<=> Engine data retrieval optimization </span></h2>
<h3> <span class="mw-headline" id="Motivation:">
Motivation: </span></h3>
<p>Currently the RHEVM engine is polling the a lot of data from
VDSM every 15 seconds. This should be optimized and the amount
of data requested should be more specific. </p>
</blockquote>
If the data size really matters, we could also consider to pack the
information into binary. I am not sure if it's suitable in the
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
transmission of XMLRPC.<br>
<blockquote cite="mid:51387942.606@redhat.com" type="cite">
<p>For each VM the data currently contains much more information
than actually needed which blows up the size of the XML content
quite big. We could optimize this by splitting the reply on the
getVmStats based on the request of the engine into sections. For
this reason Omer Frenkel and me have split up the data into
parts based on their usage. </p>
<p>This data can and usually does change during the lifetime of
the VM. </p>
<h4> <span class="mw-headline" id="Rarely_Changed:">
Rarely
Changed: </span></h4>
<p>This data is change not very frequent and it should be enough
to update this only once in a while. Most commonly this data
changes after changes made in the UI or after a migration of the
VM to another Host. </p>
<pre> <b>Status</b> = Running
<b>acpiEnable</b> = true
<b>vmType</b> = kvm
<b>guestName</b> = W864GUESTAGENTT
<b>displayType</b> = qxl
<b>guestOs</b> = Win 8
<b>kvmEnable</b> = true # <i><b>this should be constant and
never changed</b></i></pre>
</blockquote>
Then it should be removed from vm stats. In my opinion, any
information belongs to vm's static configuration, it shouldn't be
included in vm stats. For the fields above, except 'Status', engine
can get the information without querying the vdsm host. It could not
be changed by vdsm itself, right?<br>
<blockquote cite="mid:51387942.606@redhat.com" type="cite">
<pre>
<b>pauseCode</b> = NOERR
<b>monitorResponse</b> = 0
<b>session</b> = Locked # unused
<b>netIfaces</b> = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC', 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'], 'hw': '00:1a:4a:22:3c:db'}]
<b>appsList</b> = ['RHEV-Tools 3.2.4', 'RHEV-Agent64
3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64 3.2.2',
'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64
3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB
3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
<b>pid</b> = 11314
<b>guestIPs</b> = 10.34.60.148 # duplicated info
<b>displayIp</b> = 0
<b>displayPort</b> = 5902
<b>displaySecurePort</b> = 5903
<b>username</b> = user@W864GUESTAGENTT
<b>clientIp</b> =
<b>lastLogin</b> = 1361976900.67
</pre>
<h4> <span class="mw-headline" id="Often_Changed:">
Often
Changed: </span></h4>
<p>This data is changed quite often however it is not necessary to
update this data every 15 seconds. As this is cumulative data
and reflects the current status, and it does not need to be
snapshotted every 15 seconds to retrieve statistics. The data
can be retrieved in much more generous time slices. (e.g. Every
5 minutes) </p>
<pre> <b>network</b> = {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db', 'rxDropped': '0', 'txDropped':
'0', 'rxErrors': '0', 'txRate': '0.0',
'rxRate': '0.0', 'txErrors': '0', 'state':
'unknown', 'speed': '100', 'name':
'vnet1'}}</pre>
</blockquote>
macAddr and name don't change either.<br>
<blockquote cite="mid:51387942.606@redhat.com" type="cite">
<pre>
<b>disksUsage</b> = [{'path': 'c:\\', 'total':
'64055406592', 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
<b>timeOffset</b> = 14422
<b>elapsedTime</b> = 68591
<b>hash</b> = 2335461227228498964
<b>statsAge</b> = 0.09 # unused
</pre>
<h4> <span class="mw-headline"
id="Often_Changed_but_unused">
Often Changed but unused </span></h4>
<p>This data does not seem to be used in the engine at all. It is
<b>not</b> even used in the data warehouse. </p>
<pre> <b>memoryStats</b> = {'swap_out': '0',
'majflt': '0', 'mem_free': '1466884', 'swap_in':
'0', 'pageflt': '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
<b>balloonInfo</b> = {'balloon_max': 2097152,
'balloon_cur': 2097152}
</pre>
</blockquote>
It's used by mom to adjust memory overcommitment dynamically.<br>
<blockquote cite="mid:51387942.606@redhat.com" type="cite">
<pre>
<b>disks</b> = {'vda': {'readLatency': '0',
'apparentsize': '64424509440', 'writeLatency': '1754496',
'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize': '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency': '0', 'apparentsize': '0',
'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}}
</pre>
<h4> <span class="mw-headline"
id="Very_frequent_uppdates_needed_by_webadmin_portal:"> Very
frequent uppdates needed by webadmin portal: </span></h4>
<p>This data is mostly needed for the webadmin portal and might be
required to be updated quite often. An exception here is the
statsAge field, which seems to be unused by the Engine. This
data could be requested every 15 seconds to keep things as they
are now. </p>
<pre> <b>cpuSys</b> = 2.32
<b>cpuUser</b> = 1.34
<b>memUsage</b> = 30
</pre>
<h2> <span class="mw-headline"
id="Proposed_Solution_for_VDSM_.26_Engine:"> Proposed Solution
for VDSM & Engine: </span></h2>
<p>We will introduce new optional parameters to getVmStats,
getAllVmStats and list to allow a finer grained specification of
data which should be included. </p>
<p><b>Parameter:</b>
<b>statsType</b>=<i><b><string></b></i>
(getVmStats, getAllVmStats only) <b>Allowed values:</b> </p>
<ul>
<li> full (default to keep backwards compatibility) </li>
<li> app-list (Just send the application list) </li>
<li> rare (include everything from rarely changed to very
frequent) </li>
<li> often (include everything from often changed to very
frequent) </li>
<li> frequent (only send the very frequently changed items) </li>
</ul>
<p><br>
<b>Parameter:</b>
<b>clientId</b>=<b><string></b> The
client id is specified by the client and should be unique
however constantly used. </p>
<p><b>Parameter:</b>
<b>diff</b>=<b><boolean></b> In
combination with the clientId VDSM will send only differences to
the previous request from the named clientId. (if diff=true) </p>
<p><br>
</p>
<h3> <span class="mw-headline"
id="Additional_Change:">
Additional Change: </span></h3>
<p>Besides the introduction of the new parameters for list,
getVmStats and getAllVmStats it might make sense to include a
hash for the appList into the rarely changed section of the
response which would allow to identify changes and avoid having
to sent the complete appList every so often and only if the hash
known to the client is outdated. </p>
<p><b>Note:</b> The appList (Application List) reported by the
guest agent could be fully implemented on request only, as long
as the guest agent installed supports this. As there seems to be
a request to have the complete list of installed applications on
all guests this data could be quite extensive and a huge list.
On the other hand this data is only rarely visible and therefore
it should not be requested all the time and only on demand. </p>
<h3> <span class="mw-headline"
id="Improvement_of_the_Guest_Agent:"> Improvement of the Guest
Agent: </span></h3>
<p>As part of the proposed solution it is necessary to improve the
guest agent as well. For the full application list there should
be implemented a caching system which will be fully reactive and
should not poll the application list for example all the time.
The guest can create a prepared data file containing all data in
the JSON format (as used for the communication with VDSM via
VIO) and just have to read that file from disk and directly
sends it to VDSM. However it is quite possible that this list is
to big and it might have to be chunked into pieces. (Multiple
messages, which would have to be supported by VDSM then as well)
The solution for this is to make VDSM request this data and it
will retrieve the data necessary on request only. </p>
<pre class="moz-signature" cols="72">--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
vdsm-devel mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:vdsm-devel@lists.fedorahosted.org">vdsm-devel@lists.fedorahosted.org</a>
<a class="moz-txt-link-freetext"
href="https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel&quo...
</pre>
</blockquote>
<br>
</body>
</html>
--------------050703010307000605090101--
12:53 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On Fri, Mar 08, 2013 at 10:21:27AM +0800, Mark Wu wrote:
On 03/07/2013 07:25 PM, Vinzenz Feenstra wrote:
>Please find the prettier version on the wiki:
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>
>
> Proposal VDSM - Engine Data Statistics Retrieval
>
>
> VDSM <=> Engine data retrieval optimization
>
>
> Motivation:
>
>Currently the RHEVM engine is polling the a lot of data from VDSM
>every 15 seconds. This should be optimized and the amount of data
>requested should be more specific.
>
If the data size really matters, we could also consider to pack the
information into binary. I am not sure if it's suitable in the
transmission of XMLRPC.
I do not think we should embed binary in XMLRPC. I'd consider
compressing the data at the transport layer - but that would be a
completely deferent feature.
>
>For each VM the data currently contains much more information than
>actually needed which blows up the size of the XML content quite
>big. We could optimize this by splitting the reply on the
>getVmStats based on the request of the engine into sections. For
>this reason Omer Frenkel and me have split up the data into parts
>based on their usage.
>
>This data can and usually does change during the lifetime of the VM.
>
>
> Rarely Changed:
>
>This data is change not very frequent and it should be enough to
>update this only once in a while. Most commonly this data changes
>after changes made in the UI or after a migration of the VM to
>another Host.
>
> *Status* = Running
> *acpiEnable* = true
> *vmType* = kvm
> *guestName* = W864GUESTAGENTT
> *displayType* = qxl
> *guestOs* = Win 8
> *kvmEnable* = true #/*this should be constant and never changed*/
Then it should be removed from vm stats. In my opinion, any
information belongs to vm's static configuration, it shouldn't be
included in vm stats. For the fields above, except 'Status', engine
can get the information without querying the vdsm host. It could not
be changed by vdsm itself, right?
actually, guestName and guestOs may change - for example by installing
Linux on that Windows guest.
12:15 a.m.
New subject: [Engine-devel] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
I am completely against this.
It make the return value differ according to input which
is a big no no when talking about type safe APIs.
The only reason we have this problem is because there is this
thing against making multiple calls.
Just split it up.
getVmRuntimeStats() - transient things like mem and cpu%
getVmInformation() - (semi)static things like disk\networking layout etc.
Each updated at different intervals.
----- Original Message -----
From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
Sent: Thursday, March 7, 2013 6:25:54 AM
Subject: [Engine-devel] Proposal VDSM <=> Engine Data Statistics
Retrieval Optimization
Please find the prettier version on the wiki:
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
Proposal VDSM - Engine Data Statistics Retrieval
VDSM <=> Engine data retrieval optimization
Motivation:
Currently the RHEVM engine is polling the a lot of data from VDSM
every 15 seconds. This should be optimized and the amount of data
requested should be more specific.
For each VM the data currently contains much more information than
actually needed which blows up the size of the XML content quite
big. We could optimize this by splitting the reply on the getVmStats
based on the request of the engine into sections. For this reason
Omer Frenkel and me have split up the data into parts based on their
usage.
This data can and usually does change during the lifetime of the VM.
Rarely Changed:
This data is change not very frequent and it should be enough to
update this only once in a while. Most commonly this data changes
after changes made in the UI or after a migration of the VM to
another Host. Status = Running acpiEnable = true vmType = kvm
guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
kvmEnable = true # this should be constant and never changed
pauseCode = NOERR monitorResponse = 0 session = Locked # unused
netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'],
'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools 3.2.4',
'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64
3.2.2',
'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64
3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB
3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] pid = 11314
guestIPs = 10.34.60.148 # duplicated info displayIp = 0 displayPort
= 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
clientIp = lastLogin = 1361976900.67 Often Changed:
This data is changed quite often however it is not necessary to
update this data every 15 seconds. As this is cumulative data and
reflects the current status, and it does not need to be snapshotted
every 15 seconds to retrieve statistics. The data can be retrieved
in much more generous time slices. (e.g. Every 5 minutes) network =
{'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped':
'0',
'txDropped': '0', 'rxErrors': '0', 'txRate':
'0.0', 'rxRate': '0.0',
'txErrors': '0', 'state': 'unknown', 'speed':
'100', 'name':
'vnet1'}} disksUsage = [{'path': 'c:\\', 'total':
'64055406592',
'fs': 'NTFS', 'used': '19223846912'}, {'path':
'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
timeOffset = 14422
elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09 #
unused Often Changed but unused
This data does not seem to be used in the engine at all. It is not
even used in the data warehouse. memoryStats = {'swap_out': '0',
'majflt': '0', 'mem_free': '1466884', 'swap_in':
'0', 'pageflt':
'0', 'mem_total': '2096736', 'mem_unused':
'1466884'} balloonInfo =
{'balloon_max': 2097152, 'balloon_cur': 2097152} disks = {'vda':
{'readLatency': '0', 'apparentsize': '64424509440',
'writeLatency':
'1754496', 'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize':
'18855059456', 'writeRate': '952.05'}, 'hdc':
{'readLatency': '0',
'apparentsize': '0', 'writeLatency': '0',
'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}} Very
frequent uppdates needed by webadmin portal:
This data is mostly needed for the webadmin portal and might be
required to be updated quite often. An exception here is the
statsAge field, which seems to be unused by the Engine. This data
could be requested every 15 seconds to keep things as they are now.
cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution for
VDSM & Engine:
We will introduce new optional parameters to getVmStats,
getAllVmStats and list to allow a finer grained specification of
data which should be included.
Parameter: statsType = <string> (getVmStats, getAllVmStats only)
Allowed values:
* full (default to keep backwards compatibility)
* app-list (Just send the application list)
* rare (include everything from rarely changed to very frequent)
* often (include everything from often changed to very frequent)
* frequent (only send the very frequently changed items)
Parameter: clientId = <string> The client id is specified by the
client and should be unique however constantly used.
Parameter: diff = <boolean> In combination with the clientId VDSM
will send only differences to the previous request from the named
clientId. (if diff=true)
Additional Change:
Besides the introduction of the new parameters for list, getVmStats
and getAllVmStats it might make sense to include a hash for the
appList into the rarely changed section of the response which would
allow to identify changes and avoid having to sent the complete
appList every so often and only if the hash known to the client is
outdated.
Note: The appList (Application List) reported by the guest agent
could be fully implemented on request only, as long as the guest
agent installed supports this. As there seems to be a request to
have the complete list of installed applications on all guests this
data could be quite extensive and a huge list. On the other hand
this data is only rarely visible and therefore it should not be
requested all the time and only on demand. Improvement of the Guest
Agent:
As part of the proposed solution it is necessary to improve the guest
agent as well. For the full application list there should be
implemented a caching system which will be fully reactive and should
not poll the application list for example all the time. The guest
can create a prepared data file containing all data in the JSON
format (as used for the communication with VDSM via VIO) and just
have to read that file from disk and directly sends it to VDSM.
However it is quite possible that this list is to big and it might
have to be chunked into pieces. (Multiple messages, which would have
to be supported by VDSM then as well) The solution for this is to
make VDSM request this data and it will retrieve the data necessary
on request only. --
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community
collaboration.
See how it works at redhat.com
_______________________________________________
Engine-devel mailing list
Engine-devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
12:39 a.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
I am completely against this.
It make the return value differ according to input which
is a big no no when talking about type safe APIs.
The only reason we have this problem is because there is this
thing against making multiple calls.
Just split it up.
getVmRuntimeStats() - transient things like mem and cpu%
getVmInformation() - (semi)static things like disk\networking layout
etc.
Each updated at different intervals.
+1 on splitting the data up into 2 separate API calls.
You could potentially add a checksum (md5, or any other way) of the "static"
data to getVmRuntimeStats and not bother even with polling the VmInformation if this
hasn't changed. Then you could poll as often as you'd like the stats and
immediately see if you also need to retrieve VmInfo or not (you rarely would).
----- Original Message -----
> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
> Sent: Thursday, March 7, 2013 6:25:54 AM
> Subject: [Engine-devel] Proposal VDSM <=> Engine Data Statistics
> Retrieval Optimization
>
>
> Please find the prettier version on the wiki:
> http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>
> Proposal VDSM - Engine Data Statistics Retrieval
> VDSM <=> Engine data retrieval optimization
> Motivation:
>
>
> Currently the RHEVM engine is polling the a lot of data from VDSM
> every 15 seconds. This should be optimized and the amount of data
> requested should be more specific.
>
> For each VM the data currently contains much more information than
> actually needed which blows up the size of the XML content quite
> big. We could optimize this by splitting the reply on the
> getVmStats
> based on the request of the engine into sections. For this reason
> Omer Frenkel and me have split up the data into parts based on
> their
> usage.
>
> This data can and usually does change during the lifetime of the
> VM.
> Rarely Changed:
>
>
> This data is change not very frequent and it should be enough to
> update this only once in a while. Most commonly this data changes
> after changes made in the UI or after a migration of the VM to
> another Host. Status = Running acpiEnable = true vmType = kvm
> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
> kvmEnable = true # this should be constant and never changed
> pauseCode = NOERR monitorResponse = 0 session = Locked # unused
> netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
> 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'],
> 'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools 3.2.4',
> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64
> 3.2.2',
> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64
> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2',
'RHEV-USB
> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] pid =
11314
> guestIPs = 10.34.60.148 # duplicated info displayIp = 0 displayPort
> = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
> clientIp = lastLogin = 1361976900.67 Often Changed:
>
>
> This data is changed quite often however it is not necessary to
> update this data every 15 seconds. As this is cumulative data and
> reflects the current status, and it does not need to be snapshotted
> every 15 seconds to retrieve statistics. The data can be retrieved
> in much more generous time slices. (e.g. Every 5 minutes) network =
> {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0',
> 'txDropped': '0', 'rxErrors': '0', 'txRate':
'0.0', 'rxRate':
> '0.0',
> 'txErrors': '0', 'state': 'unknown',
'speed': '100', 'name':
> 'vnet1'}} disksUsage = [{'path': 'c:\\', 'total':
'64055406592',
> 'fs': 'NTFS', 'used': '19223846912'},
{'path': 'd:\\', 'total':
> '3490912256', 'fs': 'UDF', 'used':
'3490912256'}] timeOffset =
> 14422
> elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09 #
> unused Often Changed but unused
>
>
> This data does not seem to be used in the engine at all. It is not
> even used in the data warehouse. memoryStats = {'swap_out': '0',
> 'majflt': '0', 'mem_free': '1466884',
'swap_in': '0', 'pageflt':
> '0', 'mem_total': '2096736', 'mem_unused':
'1466884'} balloonInfo =
> {'balloon_max': 2097152, 'balloon_cur': 2097152} disks =
{'vda':
> {'readLatency': '0', 'apparentsize': '64424509440',
'writeLatency':
> '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
> 'flushLatency': '156549', 'readRate': '0.00',
'truesize':
> '18855059456', 'writeRate': '952.05'}, 'hdc':
{'readLatency': '0',
> 'apparentsize': '0', 'writeLatency': '0',
'flushLatency': '0',
> 'readRate': '0.00', 'truesize': '0',
'writeRate': '0.00'}} Very
> frequent uppdates needed by webadmin portal:
>
>
> This data is mostly needed for the webadmin portal and might be
> required to be updated quite often. An exception here is the
> statsAge field, which seems to be unused by the Engine. This data
> could be requested every 15 seconds to keep things as they are now.
> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution for
> VDSM & Engine:
>
>
> We will introduce new optional parameters to getVmStats,
> getAllVmStats and list to allow a finer grained specification of
> data which should be included.
>
> Parameter: statsType = <string> (getVmStats, getAllVmStats only)
> Allowed values:
>
> * full (default to keep backwards compatibility)
> * app-list (Just send the application list)
> * rare (include everything from rarely changed to very
> frequent)
> * often (include everything from often changed to very
> frequent)
> * frequent (only send the very frequently changed items)
>
>
>
> Parameter: clientId = <string> The client id is specified by the
> client and should be unique however constantly used.
>
> Parameter: diff = <boolean> In combination with the clientId VDSM
> will send only differences to the previous request from the named
> clientId. (if diff=true)
>
>
> Additional Change:
>
>
> Besides the introduction of the new parameters for list, getVmStats
> and getAllVmStats it might make sense to include a hash for the
> appList into the rarely changed section of the response which would
> allow to identify changes and avoid having to sent the complete
> appList every so often and only if the hash known to the client is
> outdated.
>
> Note: The appList (Application List) reported by the guest agent
> could be fully implemented on request only, as long as the guest
> agent installed supports this. As there seems to be a request to
> have the complete list of installed applications on all guests this
> data could be quite extensive and a huge list. On the other hand
> this data is only rarely visible and therefore it should not be
> requested all the time and only on demand. Improvement of the Guest
> Agent:
>
>
> As part of the proposed solution it is necessary to improve the
> guest
> agent as well. For the full application list there should be
> implemented a caching system which will be fully reactive and
> should
> not poll the application list for example all the time. The guest
> can create a prepared data file containing all data in the JSON
> format (as used for the communication with VDSM via VIO) and just
> have to read that file from disk and directly sends it to VDSM.
> However it is quite possible that this list is to big and it might
> have to be chunked into pieces. (Multiple messages, which would
> have
> to be supported by VDSM then as well) The solution for this is to
> make VDSM request this data and it will retrieve the data necessary
> on request only. --
> Regards,
>
> Vinzenz Feenstra | Senior Software Engineer
> RedHat Engineering Virtualization R & D
> Phone: +420 532 294 625
> IRC: vfeenstr or evilissimo
>
> Better technology. Faster innovation. Powered by community
> collaboration.
> See how it works at redhat.com
> _______________________________________________
> Engine-devel mailing list
> Engine-devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
_______________________________________________
vdsm-devel mailing list
vdsm-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
12:50 a.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
From: "Ayal Baron" <abaron(a)redhat.com>
To: "Saggi Mizrahi" <smizrahi(a)redhat.com>
Cc: engine-devel(a)ovirt.org, vdsm-devel(a)lists.fedorahosted.org, "Vinzenz
Feenstra" <vfeenstr(a)redhat.com>
Sent: Wednesday, March 13, 2013 5:39:24 PM
Subject: Re: [vdsm] [Engine-devel] Proposal VDSM <=> Engine Data Statistics
Retrieval Optimization
----- Original Message -----
> I am completely against this.
> It make the return value differ according to input which
> is a big no no when talking about type safe APIs.
>
> The only reason we have this problem is because there is this
> thing against making multiple calls.
>
> Just split it up.
> getVmRuntimeStats() - transient things like mem and cpu%
> getVmInformation() - (semi)static things like disk\networking
> layout
> etc.
> Each updated at different intervals.
+1 on splitting the data up into 2 separate API calls.
You could potentially add a checksum (md5, or any other way) of the
"static" data to getVmRuntimeStats and not bother even with polling
the VmInformation if this hasn't changed. Then you could poll as
often as you'd like the stats and immediately see if you also need
to retrieve VmInfo or not (you rarely would).
+1 To Ayal's suggestion
except that instead of the engine hashing the data VDSM sends the
key which is opaque to the engine.
This can be a local timestap or a generation number.
But, we might want to consider that when we add events polling
becomes (much) less frequent so maybe it'll be an overkill.
>
> ----- Original Message -----
> > From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
> > To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
> > Sent: Thursday, March 7, 2013 6:25:54 AM
> > Subject: [Engine-devel] Proposal VDSM <=> Engine Data Statistics
> > Retrieval Optimization
> >
> >
> > Please find the prettier version on the wiki:
> > http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
> >
> > Proposal VDSM - Engine Data Statistics Retrieval
> > VDSM <=> Engine data retrieval optimization
> > Motivation:
> >
> >
> > Currently the RHEVM engine is polling the a lot of data from VDSM
> > every 15 seconds. This should be optimized and the amount of data
> > requested should be more specific.
> >
> > For each VM the data currently contains much more information
> > than
> > actually needed which blows up the size of the XML content quite
> > big. We could optimize this by splitting the reply on the
> > getVmStats
> > based on the request of the engine into sections. For this reason
> > Omer Frenkel and me have split up the data into parts based on
> > their
> > usage.
> >
> > This data can and usually does change during the lifetime of the
> > VM.
> > Rarely Changed:
> >
> >
> > This data is change not very frequent and it should be enough to
> > update this only once in a while. Most commonly this data changes
> > after changes made in the UI or after a migration of the VM to
> > another Host. Status = Running acpiEnable = true vmType = kvm
> > guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
> > kvmEnable = true # this should be constant and never changed
> > pauseCode = NOERR monitorResponse = 0 session = Locked # unused
> > netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
> > 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
> > ['10.34.60.148'],
> > 'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools
3.2.4',
> > 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
> > 3.2.2',
> > 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
> > 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2',
'RHEV-USB
> > 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] pid =
11314
> > guestIPs = 10.34.60.148 # duplicated info displayIp = 0
> > displayPort
> > = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
> > clientIp = lastLogin = 1361976900.67 Often Changed:
> >
> >
> > This data is changed quite often however it is not necessary to
> > update this data every 15 seconds. As this is cumulative data and
> > reflects the current status, and it does not need to be
> > snapshotted
> > every 15 seconds to retrieve statistics. The data can be
> > retrieved
> > in much more generous time slices. (e.g. Every 5 minutes) network
> > =
> > {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0',
> > 'txDropped': '0', 'rxErrors': '0',
'txRate': '0.0', 'rxRate':
> > '0.0',
> > 'txErrors': '0', 'state': 'unknown',
'speed': '100', 'name':
> > 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total': '64055406592',
> > 'fs': 'NTFS', 'used': '19223846912'},
{'path': 'd:\\', 'total':
> > '3490912256', 'fs': 'UDF', 'used':
'3490912256'}] timeOffset =
> > 14422
> > elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09 #
> > unused Often Changed but unused
> >
> >
> > This data does not seem to be used in the engine at all. It is
> > not
> > even used in the data warehouse. memoryStats = {'swap_out':
'0',
> > 'majflt': '0', 'mem_free': '1466884',
'swap_in': '0', 'pageflt':
> > '0', 'mem_total': '2096736', 'mem_unused':
'1466884'} balloonInfo
> > =
> > {'balloon_max': 2097152, 'balloon_cur': 2097152} disks =
{'vda':
> > {'readLatency': '0', 'apparentsize':
'64424509440',
> > 'writeLatency':
> > '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
> > 'flushLatency': '156549', 'readRate': '0.00',
'truesize':
> > '18855059456', 'writeRate': '952.05'}, 'hdc':
{'readLatency':
> > '0',
> > 'apparentsize': '0', 'writeLatency': '0',
'flushLatency': '0',
> > 'readRate': '0.00', 'truesize': '0',
'writeRate': '0.00'}} Very
> > frequent uppdates needed by webadmin portal:
> >
> >
> > This data is mostly needed for the webadmin portal and might be
> > required to be updated quite often. An exception here is the
> > statsAge field, which seems to be unused by the Engine. This data
> > could be requested every 15 seconds to keep things as they are
> > now.
> > cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution for
> > VDSM & Engine:
> >
> >
> > We will introduce new optional parameters to getVmStats,
> > getAllVmStats and list to allow a finer grained specification of
> > data which should be included.
> >
> > Parameter: statsType = <string> (getVmStats, getAllVmStats only)
> > Allowed values:
> >
> > * full (default to keep backwards compatibility)
> > * app-list (Just send the application list)
> > * rare (include everything from rarely changed to very
> > frequent)
> > * often (include everything from often changed to very
> > frequent)
> > * frequent (only send the very frequently changed items)
> >
> >
> >
> > Parameter: clientId = <string> The client id is specified by the
> > client and should be unique however constantly used.
> >
> > Parameter: diff = <boolean> In combination with the clientId VDSM
> > will send only differences to the previous request from the named
> > clientId. (if diff=true)
> >
> >
> > Additional Change:
> >
> >
> > Besides the introduction of the new parameters for list,
> > getVmStats
> > and getAllVmStats it might make sense to include a hash for the
> > appList into the rarely changed section of the response which
> > would
> > allow to identify changes and avoid having to sent the complete
> > appList every so often and only if the hash known to the client
> > is
> > outdated.
> >
> > Note: The appList (Application List) reported by the guest agent
> > could be fully implemented on request only, as long as the guest
> > agent installed supports this. As there seems to be a request to
> > have the complete list of installed applications on all guests
> > this
> > data could be quite extensive and a huge list. On the other hand
> > this data is only rarely visible and therefore it should not be
> > requested all the time and only on demand. Improvement of the
> > Guest
> > Agent:
> >
> >
> > As part of the proposed solution it is necessary to improve the
> > guest
> > agent as well. For the full application list there should be
> > implemented a caching system which will be fully reactive and
> > should
> > not poll the application list for example all the time. The guest
> > can create a prepared data file containing all data in the JSON
> > format (as used for the communication with VDSM via VIO) and just
> > have to read that file from disk and directly sends it to VDSM.
> > However it is quite possible that this list is to big and it
> > might
> > have to be chunked into pieces. (Multiple messages, which would
> > have
> > to be supported by VDSM then as well) The solution for this is to
> > make VDSM request this data and it will retrieve the data
> > necessary
> > on request only. --
> > Regards,
> >
> > Vinzenz Feenstra | Senior Software Engineer
> > RedHat Engineering Virtualization R & D
> > Phone: +420 532 294 625
> > IRC: vfeenstr or evilissimo
> >
> > Better technology. Faster innovation. Powered by community
> > collaboration.
> > See how it works at redhat.com
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> >
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel(a)lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>
12:55 a.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
----- Original Message -----
> From: "Ayal Baron" <abaron(a)redhat.com>
> To: "Saggi Mizrahi" <smizrahi(a)redhat.com>
> Cc: engine-devel(a)ovirt.org, vdsm-devel(a)lists.fedorahosted.org,
> "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
> Sent: Wednesday, March 13, 2013 5:39:24 PM
> Subject: Re: [vdsm] [Engine-devel] Proposal VDSM <=> Engine Data
> Statistics Retrieval Optimization
>
>
>
> ----- Original Message -----
> > I am completely against this.
> > It make the return value differ according to input which
> > is a big no no when talking about type safe APIs.
> >
> > The only reason we have this problem is because there is this
> > thing against making multiple calls.
> >
> > Just split it up.
> > getVmRuntimeStats() - transient things like mem and cpu%
> > getVmInformation() - (semi)static things like disk\networking
> > layout
> > etc.
> > Each updated at different intervals.
>
> +1 on splitting the data up into 2 separate API calls.
> You could potentially add a checksum (md5, or any other way) of the
> "static" data to getVmRuntimeStats and not bother even with polling
> the VmInformation if this hasn't changed. Then you could poll as
> often as you'd like the stats and immediately see if you also need
> to retrieve VmInfo or not (you rarely would).
+1 To Ayal's suggestion
except that instead of the engine hashing the data VDSM sends the
key which is opaque to the engine.
This can be a local timestap or a generation number.
Of course vdsm does the hash, otherwise you'd need to pass all the data to engine
which would beat the purpose.
But, we might want to consider that when we add events polling
becomes (much) less frequent so maybe it'll be an overkill.
You'd still need to compare versions of the data in vdsm and send only if it changed.
If you don't persist what was received last then potentially you could have a monday
morning effect where upon on system startup you'd be sending everything. So I still
think you'd want to have the hash.
>
> >
> > ----- Original Message -----
> > > From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
> > > To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
> > > Sent: Thursday, March 7, 2013 6:25:54 AM
> > > Subject: [Engine-devel] Proposal VDSM <=> Engine Data
> > > Statistics
> > > Retrieval Optimization
> > >
> > >
> > > Please find the prettier version on the wiki:
> > > http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
> > >
> > > Proposal VDSM - Engine Data Statistics Retrieval
> > > VDSM <=> Engine data retrieval optimization
> > > Motivation:
> > >
> > >
> > > Currently the RHEVM engine is polling the a lot of data from
> > > VDSM
> > > every 15 seconds. This should be optimized and the amount of
> > > data
> > > requested should be more specific.
> > >
> > > For each VM the data currently contains much more information
> > > than
> > > actually needed which blows up the size of the XML content
> > > quite
> > > big. We could optimize this by splitting the reply on the
> > > getVmStats
> > > based on the request of the engine into sections. For this
> > > reason
> > > Omer Frenkel and me have split up the data into parts based on
> > > their
> > > usage.
> > >
> > > This data can and usually does change during the lifetime of
> > > the
> > > VM.
> > > Rarely Changed:
> > >
> > >
> > > This data is change not very frequent and it should be enough
> > > to
> > > update this only once in a while. Most commonly this data
> > > changes
> > > after changes made in the UI or after a migration of the VM to
> > > another Host. Status = Running acpiEnable = true vmType = kvm
> > > guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
> > > kvmEnable = true # this should be constant and never changed
> > > pauseCode = NOERR monitorResponse = 0 session = Locked # unused
> > > netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC',
> > > 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
> > > ['10.34.60.148'],
> > > 'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools
3.2.4',
> > > 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
> > > 3.2.2',
> > > 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
> > > 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2',
'RHEV-USB
> > > 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
pid =
> > > 11314
> > > guestIPs = 10.34.60.148 # duplicated info displayIp = 0
> > > displayPort
> > > = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
> > > clientIp = lastLogin = 1361976900.67 Often Changed:
> > >
> > >
> > > This data is changed quite often however it is not necessary to
> > > update this data every 15 seconds. As this is cumulative data
> > > and
> > > reflects the current status, and it does not need to be
> > > snapshotted
> > > every 15 seconds to retrieve statistics. The data can be
> > > retrieved
> > > in much more generous time slices. (e.g. Every 5 minutes)
> > > network
> > > =
> > > {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0',
> > > 'txDropped': '0', 'rxErrors': '0',
'txRate': '0.0', 'rxRate':
> > > '0.0',
> > > 'txErrors': '0', 'state': 'unknown',
'speed': '100', 'name':
> > > 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total':
> > > '64055406592',
> > > 'fs': 'NTFS', 'used': '19223846912'},
{'path': 'd:\\', 'total':
> > > '3490912256', 'fs': 'UDF', 'used':
'3490912256'}] timeOffset =
> > > 14422
> > > elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
> > > #
> > > unused Often Changed but unused
> > >
> > >
> > > This data does not seem to be used in the engine at all. It is
> > > not
> > > even used in the data warehouse. memoryStats = {'swap_out':
> > > '0',
> > > 'majflt': '0', 'mem_free': '1466884',
'swap_in': '0',
> > > 'pageflt':
> > > '0', 'mem_total': '2096736', 'mem_unused':
'1466884'}
> > > balloonInfo
> > > =
> > > {'balloon_max': 2097152, 'balloon_cur': 2097152} disks =
> > > {'vda':
> > > {'readLatency': '0', 'apparentsize':
'64424509440',
> > > 'writeLatency':
> > > '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
> > > 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
> > > '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
> > > '0',
> > > 'apparentsize': '0', 'writeLatency': '0',
'flushLatency': '0',
> > > 'readRate': '0.00', 'truesize': '0',
'writeRate': '0.00'}} Very
> > > frequent uppdates needed by webadmin portal:
> > >
> > >
> > > This data is mostly needed for the webadmin portal and might be
> > > required to be updated quite often. An exception here is the
> > > statsAge field, which seems to be unused by the Engine. This
> > > data
> > > could be requested every 15 seconds to keep things as they are
> > > now.
> > > cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
> > > for
> > > VDSM & Engine:
> > >
> > >
> > > We will introduce new optional parameters to getVmStats,
> > > getAllVmStats and list to allow a finer grained specification
> > > of
> > > data which should be included.
> > >
> > > Parameter: statsType = <string> (getVmStats, getAllVmStats
> > > only)
> > > Allowed values:
> > >
> > > * full (default to keep backwards compatibility)
> > > * app-list (Just send the application list)
> > > * rare (include everything from rarely changed to very
> > > frequent)
> > > * often (include everything from often changed to very
> > > frequent)
> > > * frequent (only send the very frequently changed items)
> > >
> > >
> > >
> > > Parameter: clientId = <string> The client id is specified by
> > > the
> > > client and should be unique however constantly used.
> > >
> > > Parameter: diff = <boolean> In combination with the clientId
> > > VDSM
> > > will send only differences to the previous request from the
> > > named
> > > clientId. (if diff=true)
> > >
> > >
> > > Additional Change:
> > >
> > >
> > > Besides the introduction of the new parameters for list,
> > > getVmStats
> > > and getAllVmStats it might make sense to include a hash for the
> > > appList into the rarely changed section of the response which
> > > would
> > > allow to identify changes and avoid having to sent the complete
> > > appList every so often and only if the hash known to the client
> > > is
> > > outdated.
> > >
> > > Note: The appList (Application List) reported by the guest
> > > agent
> > > could be fully implemented on request only, as long as the
> > > guest
> > > agent installed supports this. As there seems to be a request
> > > to
> > > have the complete list of installed applications on all guests
> > > this
> > > data could be quite extensive and a huge list. On the other
> > > hand
> > > this data is only rarely visible and therefore it should not be
> > > requested all the time and only on demand. Improvement of the
> > > Guest
> > > Agent:
> > >
> > >
> > > As part of the proposed solution it is necessary to improve the
> > > guest
> > > agent as well. For the full application list there should be
> > > implemented a caching system which will be fully reactive and
> > > should
> > > not poll the application list for example all the time. The
> > > guest
> > > can create a prepared data file containing all data in the JSON
> > > format (as used for the communication with VDSM via VIO) and
> > > just
> > > have to read that file from disk and directly sends it to VDSM.
> > > However it is quite possible that this list is to big and it
> > > might
> > > have to be chunked into pieces. (Multiple messages, which would
> > > have
> > > to be supported by VDSM then as well) The solution for this is
> > > to
> > > make VDSM request this data and it will retrieve the data
> > > necessary
> > > on request only. --
> > > Regards,
> > >
> > > Vinzenz Feenstra | Senior Software Engineer
> > > RedHat Engineering Virtualization R & D
> > > Phone: +420 532 294 625
> > > IRC: vfeenstr or evilissimo
> > >
> > > Better technology. Faster innovation. Powered by community
> > > collaboration.
> > > See how it works at redhat.com
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel(a)ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel(a)lists.fedorahosted.org
> > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> >
>
2:04 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On 03/13/2013 11:55 PM, Ayal Baron wrote:
...
>>> The only reason we have this problem is because there is
this
>>> thing against making multiple calls.
>>>
>>> Just split it up.
>>> getVmRuntimeStats() - transient things like mem and cpu%
>>> getVmInformation() - (semi)static things like disk\networking
>>> layout
>>> etc.
>>> Each updated at different intervals.
>>
>> +1 on splitting the data up into 2 separate API calls.
>> You could potentially add a checksum (md5, or any other way) of the
>> "static" data to getVmRuntimeStats and not bother even with polling
>> the VmInformation if this hasn't changed. Then you could poll as
>> often as you'd like the stats and immediately see if you also need
>> to retrieve VmInfo or not (you rarely would).
> +1 To Ayal's suggestion
> except that instead of the engine hashing the data VDSM sends the
> key which is opaque to the engine.
> This can be a local timestap or a generation number.
Of course vdsm does the hash, otherwise you'd need to pass all the data to engine
which would beat the purpose.
I thought you meant engine will be sending the hash of previous requests
per VM to vdsm, then vdsm will reply back with vm's removed, vm's added,
and the details for vm's that changed (i.e., engine would be doing
something like if-modified-since-checksum per vm).
benefit is reducing a round trip.
but first would need to split to calls of stats (always changing) and
slowly/never changing data.
>
> But, we might want to consider that when we add events polling
> becomes (much) less frequent so maybe it'll be an overkill.
You'd still need to compare versions of the data in vdsm and send only if it changed.
If you don't persist what was received last then potentially you could have a monday
morning effect where upon on system startup you'd be sending everything. So I still
think you'd want to have the hash.
>
>>
>>>
>>> ----- Original Message -----
>>>> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>>> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
>>>> Statistics
>>>> Retrieval Optimization
>>>>
>>>>
>>>> Please find the prettier version on the wiki:
>>>> http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>>>
>>>> Proposal VDSM - Engine Data Statistics Retrieval
>>>> VDSM <=> Engine data retrieval optimization
>>>> Motivation:
>>>>
>>>>
>>>> Currently the RHEVM engine is polling the a lot of data from
>>>> VDSM
>>>> every 15 seconds. This should be optimized and the amount of
>>>> data
>>>> requested should be more specific.
>>>>
>>>> For each VM the data currently contains much more information
>>>> than
>>>> actually needed which blows up the size of the XML content
>>>> quite
>>>> big. We could optimize this by splitting the reply on the
>>>> getVmStats
>>>> based on the request of the engine into sections. For this
>>>> reason
>>>> Omer Frenkel and me have split up the data into parts based on
>>>> their
>>>> usage.
>>>>
>>>> This data can and usually does change during the lifetime of
>>>> the
>>>> VM.
>>>> Rarely Changed:
>>>>
>>>>
>>>> This data is change not very frequent and it should be enough
>>>> to
>>>> update this only once in a while. Most commonly this data
>>>> changes
>>>> after changes made in the UI or after a migration of the VM to
>>>> another Host. Status = Running acpiEnable = true vmType = kvm
>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
>>>> kvmEnable = true # this should be constant and never changed
>>>> pauseCode = NOERR monitorResponse = 0 session = Locked # unused
>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC',
>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
>>>> ['10.34.60.148'],
>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools
3.2.4',
>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
>>>> 3.2.2',
>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
>>>> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2',
'RHEV-USB
>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
pid =
>>>> 11314
>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
>>>> displayPort
>>>> = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
>>>>
>>>>
>>>> This data is changed quite often however it is not necessary to
>>>> update this data every 15 seconds. As this is cumulative data
>>>> and
>>>> reflects the current status, and it does not need to be
>>>> snapshotted
>>>> every 15 seconds to retrieve statistics. The data can be
>>>> retrieved
>>>> in much more generous time slices. (e.g. Every 5 minutes)
>>>> network
>>>> =
>>>> {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0',
>>>> 'txDropped': '0', 'rxErrors': '0',
'txRate': '0.0', 'rxRate':
>>>> '0.0',
>>>> 'txErrors': '0', 'state': 'unknown',
'speed': '100', 'name':
>>>> 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total':
>>>> '64055406592',
>>>> 'fs': 'NTFS', 'used': '19223846912'},
{'path': 'd:\\', 'total':
>>>> '3490912256', 'fs': 'UDF', 'used':
'3490912256'}] timeOffset =
>>>> 14422
>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
>>>> #
>>>> unused Often Changed but unused
>>>>
>>>>
>>>> This data does not seem to be used in the engine at all. It is
>>>> not
>>>> even used in the data warehouse. memoryStats = {'swap_out':
>>>> '0',
>>>> 'majflt': '0', 'mem_free': '1466884',
'swap_in': '0',
>>>> 'pageflt':
>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
>>>> balloonInfo
>>>> =
>>>> {'balloon_max': 2097152, 'balloon_cur': 2097152} disks =
>>>> {'vda':
>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
>>>> 'writeLatency':
>>>> '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
>>>> 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
>>>> '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
>>>> '0',
>>>> 'apparentsize': '0', 'writeLatency': '0',
'flushLatency': '0',
>>>> 'readRate': '0.00', 'truesize': '0',
'writeRate': '0.00'}} Very
>>>> frequent uppdates needed by webadmin portal:
>>>>
>>>>
>>>> This data is mostly needed for the webadmin portal and might be
>>>> required to be updated quite often. An exception here is the
>>>> statsAge field, which seems to be unused by the Engine. This
>>>> data
>>>> could be requested every 15 seconds to keep things as they are
>>>> now.
>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
>>>> for
>>>> VDSM & Engine:
>>>>
>>>>
>>>> We will introduce new optional parameters to getVmStats,
>>>> getAllVmStats and list to allow a finer grained specification
>>>> of
>>>> data which should be included.
>>>>
>>>> Parameter: statsType = <string> (getVmStats, getAllVmStats
>>>> only)
>>>> Allowed values:
>>>>
>>>> * full (default to keep backwards compatibility)
>>>> * app-list (Just send the application list)
>>>> * rare (include everything from rarely changed to very
>>>> frequent)
>>>> * often (include everything from often changed to very
>>>> frequent)
>>>> * frequent (only send the very frequently changed items)
>>>>
>>>>
>>>>
>>>> Parameter: clientId = <string> The client id is specified by
>>>> the
>>>> client and should be unique however constantly used.
>>>>
>>>> Parameter: diff = <boolean> In combination with the clientId
>>>> VDSM
>>>> will send only differences to the previous request from the
>>>> named
>>>> clientId. (if diff=true)
>>>>
>>>>
>>>> Additional Change:
>>>>
>>>>
>>>> Besides the introduction of the new parameters for list,
>>>> getVmStats
>>>> and getAllVmStats it might make sense to include a hash for the
>>>> appList into the rarely changed section of the response which
>>>> would
>>>> allow to identify changes and avoid having to sent the complete
>>>> appList every so often and only if the hash known to the client
>>>> is
>>>> outdated.
>>>>
>>>> Note: The appList (Application List) reported by the guest
>>>> agent
>>>> could be fully implemented on request only, as long as the
>>>> guest
>>>> agent installed supports this. As there seems to be a request
>>>> to
>>>> have the complete list of installed applications on all guests
>>>> this
>>>> data could be quite extensive and a huge list. On the other
>>>> hand
>>>> this data is only rarely visible and therefore it should not be
>>>> requested all the time and only on demand. Improvement of the
>>>> Guest
>>>> Agent:
>>>>
>>>>
>>>> As part of the proposed solution it is necessary to improve the
>>>> guest
>>>> agent as well. For the full application list there should be
>>>> implemented a caching system which will be fully reactive and
>>>> should
>>>> not poll the application list for example all the time. The
>>>> guest
>>>> can create a prepared data file containing all data in the JSON
>>>> format (as used for the communication with VDSM via VIO) and
>>>> just
>>>> have to read that file from disk and directly sends it to VDSM.
>>>> However it is quite possible that this list is to big and it
>>>> might
>>>> have to be chunked into pieces. (Multiple messages, which would
>>>> have
>>>> to be supported by VDSM then as well) The solution for this is
>>>> to
>>>> make VDSM request this data and it will retrieve the data
>>>> necessary
>>>> on request only. --
>>>> Regards,
>>>>
>>>> Vinzenz Feenstra | Senior Software Engineer
>>>> RedHat Engineering Virtualization R & D
>>>> Phone: +420 532 294 625
>>>> IRC: vfeenstr or evilissimo
>>>>
>>>> Better technology. Faster innovation. Powered by community
>>>> collaboration.
>>>> See how it works at redhat.com
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>> _______________________________________________
>>> vdsm-devel mailing list
>>> vdsm-devel(a)lists.fedorahosted.org
>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>
>>
>
_______________________________________________
vdsm-devel mailing list
vdsm-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
4:13 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
On 03/13/2013 11:55 PM, Ayal Baron wrote:
...
>>>> The only reason we have this problem is because there is this
>>>> thing against making multiple calls.
>>>>
>>>> Just split it up.
>>>> getVmRuntimeStats() - transient things like mem and cpu%
>>>> getVmInformation() - (semi)static things like disk\networking
>>>> layout
>>>> etc.
>>>> Each updated at different intervals.
>>>
>>> +1 on splitting the data up into 2 separate API calls.
>>> You could potentially add a checksum (md5, or any other way) of
>>> the
>>> "static" data to getVmRuntimeStats and not bother even with
>>> polling
>>> the VmInformation if this hasn't changed. Then you could poll as
>>> often as you'd like the stats and immediately see if you also
>>> need
>>> to retrieve VmInfo or not (you rarely would).
>> +1 To Ayal's suggestion
>> except that instead of the engine hashing the data VDSM sends the
>> key which is opaque to the engine.
>> This can be a local timestap or a generation number.
>
> Of course vdsm does the hash, otherwise you'd need to pass all the
> data to engine which would beat the purpose.
I thought you meant engine will be sending the hash of previous
requests
per VM to vdsm, then vdsm will reply back with vm's removed, vm's
added,
and the details for vm's that changed (i.e., engine would be doing
something like if-modified-since-checksum per vm).
benefit is reducing a round trip.
but first would need to split to calls of stats (always changing) and
slowly/never changing data.
If vdms accepts the hash then in your method engine would have to periodically call
getVmInfo(hash).
What I was suggesting is that getVmStats would return vmInfo hash so that we could avoid
calling getVmInfo altogether.
The stats *always* change so there is no need for checking if that info has changed.
What we could do is avoid the split into 2 verbs by calling getVmStats(hash) and then have
getVmStats return everything if the hash has changed or only the stats if it hasn't.
This would be the least number of roundtrips and avoid the split. If you don't pass a
hash it would return everything so this way it's also fully backward compatible.
>
>>
>> But, we might want to consider that when we add events polling
>> becomes (much) less frequent so maybe it'll be an overkill.
>
> You'd still need to compare versions of the data in vdsm and send
> only if it changed. If you don't persist what was received last
> then potentially you could have a monday morning effect where upon
> on system startup you'd be sending everything. So I still think
> you'd want to have the hash.
>
>
>>
>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>>>> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
>>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
>>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
>>>>> Statistics
>>>>> Retrieval Optimization
>>>>>
>>>>>
>>>>> Please find the prettier version on the wiki:
>>>>>
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>>>>
>>>>> Proposal VDSM - Engine Data Statistics Retrieval
>>>>> VDSM <=> Engine data retrieval optimization
>>>>> Motivation:
>>>>>
>>>>>
>>>>> Currently the RHEVM engine is polling the a lot of data from
>>>>> VDSM
>>>>> every 15 seconds. This should be optimized and the amount of
>>>>> data
>>>>> requested should be more specific.
>>>>>
>>>>> For each VM the data currently contains much more information
>>>>> than
>>>>> actually needed which blows up the size of the XML content
>>>>> quite
>>>>> big. We could optimize this by splitting the reply on the
>>>>> getVmStats
>>>>> based on the request of the engine into sections. For this
>>>>> reason
>>>>> Omer Frenkel and me have split up the data into parts based on
>>>>> their
>>>>> usage.
>>>>>
>>>>> This data can and usually does change during the lifetime of
>>>>> the
>>>>> VM.
>>>>> Rarely Changed:
>>>>>
>>>>>
>>>>> This data is change not very frequent and it should be enough
>>>>> to
>>>>> update this only once in a while. Most commonly this data
>>>>> changes
>>>>> after changes made in the UI or after a migration of the VM to
>>>>> another Host. Status = Running acpiEnable = true vmType = kvm
>>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
>>>>> kvmEnable = true # this should be constant and never changed
>>>>> pauseCode = NOERR monitorResponse = 0 session = Locked # unused
>>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC',
>>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'],
'inet':
>>>>> ['10.34.60.148'],
>>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList =
['RHEV-Tools 3.2.4',
>>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
>>>>> 3.2.2',
>>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
>>>>> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64
3.2.2', 'RHEV-USB
>>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64
3.2.2'] pid =
>>>>> 11314
>>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
>>>>> displayPort
>>>>> = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
>>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
>>>>>
>>>>>
>>>>> This data is changed quite often however it is not necessary to
>>>>> update this data every 15 seconds. As this is cumulative data
>>>>> and
>>>>> reflects the current status, and it does not need to be
>>>>> snapshotted
>>>>> every 15 seconds to retrieve statistics. The data can be
>>>>> retrieved
>>>>> in much more generous time slices. (e.g. Every 5 minutes)
>>>>> network
>>>>> =
>>>>> {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0',
>>>>> 'txDropped': '0', 'rxErrors': '0',
'txRate': '0.0', 'rxRate':
>>>>> '0.0',
>>>>> 'txErrors': '0', 'state': 'unknown',
'speed': '100', 'name':
>>>>> 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total':
>>>>> '64055406592',
>>>>> 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
>>>>> '3490912256', 'fs': 'UDF', 'used':
'3490912256'}] timeOffset =
>>>>> 14422
>>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
>>>>> #
>>>>> unused Often Changed but unused
>>>>>
>>>>>
>>>>> This data does not seem to be used in the engine at all. It is
>>>>> not
>>>>> even used in the data warehouse. memoryStats = {'swap_out':
>>>>> '0',
>>>>> 'majflt': '0', 'mem_free':
'1466884', 'swap_in': '0',
>>>>> 'pageflt':
>>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
>>>>> balloonInfo
>>>>> =
>>>>> {'balloon_max': 2097152, 'balloon_cur': 2097152}
disks =
>>>>> {'vda':
>>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
>>>>> 'writeLatency':
>>>>> '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
>>>>> 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
>>>>> '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
>>>>> '0',
>>>>> 'apparentsize': '0', 'writeLatency':
'0', 'flushLatency': '0',
>>>>> 'readRate': '0.00', 'truesize': '0',
'writeRate': '0.00'}} Very
>>>>> frequent uppdates needed by webadmin portal:
>>>>>
>>>>>
>>>>> This data is mostly needed for the webadmin portal and might be
>>>>> required to be updated quite often. An exception here is the
>>>>> statsAge field, which seems to be unused by the Engine. This
>>>>> data
>>>>> could be requested every 15 seconds to keep things as they are
>>>>> now.
>>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
>>>>> for
>>>>> VDSM & Engine:
>>>>>
>>>>>
>>>>> We will introduce new optional parameters to getVmStats,
>>>>> getAllVmStats and list to allow a finer grained specification
>>>>> of
>>>>> data which should be included.
>>>>>
>>>>> Parameter: statsType = <string> (getVmStats, getAllVmStats
>>>>> only)
>>>>> Allowed values:
>>>>>
>>>>> * full (default to keep backwards compatibility)
>>>>> * app-list (Just send the application list)
>>>>> * rare (include everything from rarely changed to very
>>>>> frequent)
>>>>> * often (include everything from often changed to very
>>>>> frequent)
>>>>> * frequent (only send the very frequently changed items)
>>>>>
>>>>>
>>>>>
>>>>> Parameter: clientId = <string> The client id is specified by
>>>>> the
>>>>> client and should be unique however constantly used.
>>>>>
>>>>> Parameter: diff = <boolean> In combination with the clientId
>>>>> VDSM
>>>>> will send only differences to the previous request from the
>>>>> named
>>>>> clientId. (if diff=true)
>>>>>
>>>>>
>>>>> Additional Change:
>>>>>
>>>>>
>>>>> Besides the introduction of the new parameters for list,
>>>>> getVmStats
>>>>> and getAllVmStats it might make sense to include a hash for the
>>>>> appList into the rarely changed section of the response which
>>>>> would
>>>>> allow to identify changes and avoid having to sent the complete
>>>>> appList every so often and only if the hash known to the client
>>>>> is
>>>>> outdated.
>>>>>
>>>>> Note: The appList (Application List) reported by the guest
>>>>> agent
>>>>> could be fully implemented on request only, as long as the
>>>>> guest
>>>>> agent installed supports this. As there seems to be a request
>>>>> to
>>>>> have the complete list of installed applications on all guests
>>>>> this
>>>>> data could be quite extensive and a huge list. On the other
>>>>> hand
>>>>> this data is only rarely visible and therefore it should not be
>>>>> requested all the time and only on demand. Improvement of the
>>>>> Guest
>>>>> Agent:
>>>>>
>>>>>
>>>>> As part of the proposed solution it is necessary to improve the
>>>>> guest
>>>>> agent as well. For the full application list there should be
>>>>> implemented a caching system which will be fully reactive and
>>>>> should
>>>>> not poll the application list for example all the time. The
>>>>> guest
>>>>> can create a prepared data file containing all data in the JSON
>>>>> format (as used for the communication with VDSM via VIO) and
>>>>> just
>>>>> have to read that file from disk and directly sends it to VDSM.
>>>>> However it is quite possible that this list is to big and it
>>>>> might
>>>>> have to be chunked into pieces. (Multiple messages, which would
>>>>> have
>>>>> to be supported by VDSM then as well) The solution for this is
>>>>> to
>>>>> make VDSM request this data and it will retrieve the data
>>>>> necessary
>>>>> on request only. --
>>>>> Regards,
>>>>>
>>>>> Vinzenz Feenstra | Senior Software Engineer
>>>>> RedHat Engineering Virtualization R & D
>>>>> Phone: +420 532 294 625
>>>>> IRC: vfeenstr or evilissimo
>>>>>
>>>>> Better technology. Faster innovation. Powered by community
>>>>> collaboration.
>>>>> See how it works at redhat.com
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>>> _______________________________________________
>>>> vdsm-devel mailing list
>>>> vdsm-devel(a)lists.fedorahosted.org
>>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>
>>>
>>
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel(a)lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>
4:38 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On 17/03/13 15:13, Ayal Baron wrote:
----- Original Message -----
> On 03/13/2013 11:55 PM, Ayal Baron wrote:
> ...
>>>>> The only reason we have this problem is because there is this
>>>>> thing against making multiple calls.
>>>>>
>>>>> Just split it up.
>>>>> getVmRuntimeStats() - transient things like mem and cpu%
>>>>> getVmInformation() - (semi)static things like disk\networking
>>>>> layout
>>>>> etc.
>>>>> Each updated at different intervals.
>>>> +1 on splitting the data up into 2 separate API calls.
>>>> You could potentially add a checksum (md5, or any other way) of
>>>> the
>>>> "static" data to getVmRuntimeStats and not bother even with
>>>> polling
>>>> the VmInformation if this hasn't changed. Then you could poll as
>>>> often as you'd like the stats and immediately see if you also
>>>> need
>>>> to retrieve VmInfo or not (you rarely would).
>>> +1 To Ayal's suggestion
>>> except that instead of the engine hashing the data VDSM sends the
>>> key which is opaque to the engine.
>>> This can be a local timestap or a generation number.
>> Of course vdsm does the hash, otherwise you'd need to pass all the
>> data to engine which would beat the purpose.
> I thought you meant engine will be sending the hash of previous
> requests
> per VM to vdsm, then vdsm will reply back with vm's removed, vm's
> added,
> and the details for vm's that changed (i.e., engine would be doing
> something like if-modified-since-checksum per vm).
> benefit is reducing a round trip.
> but first would need to split to calls of stats (always changing) and
> slowly/never changing data.
If vdms accepts the hash then in your method engine would have to periodically call
getVmInfo(hash).
What I was suggesting is that getVmStats would return vmInfo hash so that we could avoid
calling getVmInfo altogether.
The stats *always* change so there is no need for checking if that info has changed.
What we could do is avoid the split into 2 verbs by calling getVmStats(hash) and then
have getVmStats return everything if the hash has changed or only the stats if it
hasn't. This would be the least number of roundtrips and avoid the split. If you
don't pass a hash it would return everything so this way it's also fully backward
compatible.
For the 'static' data, why is there a need for a hash?
If VDSM sends in each update a timestamp, can't RHEVM just use
if-modified-since with the last timestamp it got from VDSM?
Is it cheaper for VDSM to calculate the hash, than update the timestamp
per change in any of the fields? It doesn't really need to update the
timestamp per change, only for the first change since last update sent
actually (so 'dirty' flag in a way, to signify data that RHEVM hasn't
seen yet).
Y.
>>> But, we might want to consider that when we add events polling
>>> becomes (much) less frequent so maybe it'll be an overkill.
>> You'd still need to compare versions of the data in vdsm and send
>> only if it changed. If you don't persist what was received last
>> then potentially you could have a monday morning effect where upon
>> on system startup you'd be sending everything. So I still think
>> you'd want to have the hash.
>>
>>
>>>>> ----- Original Message -----
>>>>>> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>>>>> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
>>>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
>>>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
>>>>>> Statistics
>>>>>> Retrieval Optimization
>>>>>>
>>>>>>
>>>>>> Please find the prettier version on the wiki:
>>>>>>
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>>>>>
>>>>>> Proposal VDSM - Engine Data Statistics Retrieval
>>>>>> VDSM <=> Engine data retrieval optimization
>>>>>> Motivation:
>>>>>>
>>>>>>
>>>>>> Currently the RHEVM engine is polling the a lot of data from
>>>>>> VDSM
>>>>>> every 15 seconds. This should be optimized and the amount of
>>>>>> data
>>>>>> requested should be more specific.
>>>>>>
>>>>>> For each VM the data currently contains much more information
>>>>>> than
>>>>>> actually needed which blows up the size of the XML content
>>>>>> quite
>>>>>> big. We could optimize this by splitting the reply on the
>>>>>> getVmStats
>>>>>> based on the request of the engine into sections. For this
>>>>>> reason
>>>>>> Omer Frenkel and me have split up the data into parts based on
>>>>>> their
>>>>>> usage.
>>>>>>
>>>>>> This data can and usually does change during the lifetime of
>>>>>> the
>>>>>> VM.
>>>>>> Rarely Changed:
>>>>>>
>>>>>>
>>>>>> This data is change not very frequent and it should be enough
>>>>>> to
>>>>>> update this only once in a while. Most commonly this data
>>>>>> changes
>>>>>> after changes made in the UI or after a migration of the VM to
>>>>>> another Host. Status = Running acpiEnable = true vmType = kvm
>>>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
>>>>>> kvmEnable = true # this should be constant and never changed
>>>>>> pauseCode = NOERR monitorResponse = 0 session = Locked # unused
>>>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast
Ethernet NIC',
>>>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'],
'inet':
>>>>>> ['10.34.60.148'],
>>>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList =
['RHEV-Tools 3.2.4',
>>>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
>>>>>> 3.2.2',
>>>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
>>>>>> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64
3.2.2', 'RHEV-USB
>>>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64
3.2.2'] pid =
>>>>>> 11314
>>>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
>>>>>> displayPort
>>>>>> = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
>>>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
>>>>>>
>>>>>>
>>>>>> This data is changed quite often however it is not necessary to
>>>>>> update this data every 15 seconds. As this is cumulative data
>>>>>> and
>>>>>> reflects the current status, and it does not need to be
>>>>>> snapshotted
>>>>>> every 15 seconds to retrieve statistics. The data can be
>>>>>> retrieved
>>>>>> in much more generous time slices. (e.g. Every 5 minutes)
>>>>>> network
>>>>>> =
>>>>>> {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db', 'rxDropped': '0',
>>>>>> 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate':
>>>>>> '0.0',
>>>>>> 'txErrors': '0', 'state':
'unknown', 'speed': '100', 'name':
>>>>>> 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total':
>>>>>> '64055406592',
>>>>>> 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
>>>>>> '3490912256', 'fs': 'UDF',
'used': '3490912256'}] timeOffset =
>>>>>> 14422
>>>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
>>>>>> #
>>>>>> unused Often Changed but unused
>>>>>>
>>>>>>
>>>>>> This data does not seem to be used in the engine at all. It is
>>>>>> not
>>>>>> even used in the data warehouse. memoryStats =
{'swap_out':
>>>>>> '0',
>>>>>> 'majflt': '0', 'mem_free':
'1466884', 'swap_in': '0',
>>>>>> 'pageflt':
>>>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
>>>>>> balloonInfo
>>>>>> =
>>>>>> {'balloon_max': 2097152, 'balloon_cur': 2097152}
disks =
>>>>>> {'vda':
>>>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
>>>>>> 'writeLatency':
>>>>>> '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
>>>>>> 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
>>>>>> '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
>>>>>> '0',
>>>>>> 'apparentsize': '0', 'writeLatency':
'0', 'flushLatency': '0',
>>>>>> 'readRate': '0.00', 'truesize':
'0', 'writeRate': '0.00'}} Very
>>>>>> frequent uppdates needed by webadmin portal:
>>>>>>
>>>>>>
>>>>>> This data is mostly needed for the webadmin portal and might be
>>>>>> required to be updated quite often. An exception here is the
>>>>>> statsAge field, which seems to be unused by the Engine. This
>>>>>> data
>>>>>> could be requested every 15 seconds to keep things as they are
>>>>>> now.
>>>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
>>>>>> for
>>>>>> VDSM & Engine:
>>>>>>
>>>>>>
>>>>>> We will introduce new optional parameters to getVmStats,
>>>>>> getAllVmStats and list to allow a finer grained specification
>>>>>> of
>>>>>> data which should be included.
>>>>>>
>>>>>> Parameter: statsType = <string> (getVmStats, getAllVmStats
>>>>>> only)
>>>>>> Allowed values:
>>>>>>
>>>>>> * full (default to keep backwards compatibility)
>>>>>> * app-list (Just send the application list)
>>>>>> * rare (include everything from rarely changed to very
>>>>>> frequent)
>>>>>> * often (include everything from often changed to very
>>>>>> frequent)
>>>>>> * frequent (only send the very frequently changed items)
>>>>>>
>>>>>>
>>>>>>
>>>>>> Parameter: clientId = <string> The client id is specified
by
>>>>>> the
>>>>>> client and should be unique however constantly used.
>>>>>>
>>>>>> Parameter: diff = <boolean> In combination with the
clientId
>>>>>> VDSM
>>>>>> will send only differences to the previous request from the
>>>>>> named
>>>>>> clientId. (if diff=true)
>>>>>>
>>>>>>
>>>>>> Additional Change:
>>>>>>
>>>>>>
>>>>>> Besides the introduction of the new parameters for list,
>>>>>> getVmStats
>>>>>> and getAllVmStats it might make sense to include a hash for the
>>>>>> appList into the rarely changed section of the response which
>>>>>> would
>>>>>> allow to identify changes and avoid having to sent the complete
>>>>>> appList every so often and only if the hash known to the client
>>>>>> is
>>>>>> outdated.
>>>>>>
>>>>>> Note: The appList (Application List) reported by the guest
>>>>>> agent
>>>>>> could be fully implemented on request only, as long as the
>>>>>> guest
>>>>>> agent installed supports this. As there seems to be a request
>>>>>> to
>>>>>> have the complete list of installed applications on all guests
>>>>>> this
>>>>>> data could be quite extensive and a huge list. On the other
>>>>>> hand
>>>>>> this data is only rarely visible and therefore it should not be
>>>>>> requested all the time and only on demand. Improvement of the
>>>>>> Guest
>>>>>> Agent:
>>>>>>
>>>>>>
>>>>>> As part of the proposed solution it is necessary to improve the
>>>>>> guest
>>>>>> agent as well. For the full application list there should be
>>>>>> implemented a caching system which will be fully reactive and
>>>>>> should
>>>>>> not poll the application list for example all the time. The
>>>>>> guest
>>>>>> can create a prepared data file containing all data in the JSON
>>>>>> format (as used for the communication with VDSM via VIO) and
>>>>>> just
>>>>>> have to read that file from disk and directly sends it to VDSM.
>>>>>> However it is quite possible that this list is to big and it
>>>>>> might
>>>>>> have to be chunked into pieces. (Multiple messages, which would
>>>>>> have
>>>>>> to be supported by VDSM then as well) The solution for this is
>>>>>> to
>>>>>> make VDSM request this data and it will retrieve the data
>>>>>> necessary
>>>>>> on request only. --
>>>>>> Regards,
>>>>>>
>>>>>> Vinzenz Feenstra | Senior Software Engineer
>>>>>> RedHat Engineering Virtualization R & D
>>>>>> Phone: +420 532 294 625
>>>>>> IRC: vfeenstr or evilissimo
>>>>>>
>>>>>> Better technology. Faster innovation. Powered by community
>>>>>> collaboration.
>>>>>> See how it works at redhat.com
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>>> _______________________________________________
>>>>> vdsm-devel mailing list
>>>>> vdsm-devel(a)lists.fedorahosted.org
>>>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>>
>> _______________________________________________
>> vdsm-devel mailing list
>> vdsm-devel(a)lists.fedorahosted.org
>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>
>
_______________________________________________
Engine-devel mailing list
Engine-devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
5:28 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
On 17/03/13 15:13, Ayal Baron wrote:
>
> ----- Original Message -----
>> On 03/13/2013 11:55 PM, Ayal Baron wrote:
>> ...
>>>>>> The only reason we have this problem is because there is this
>>>>>> thing against making multiple calls.
>>>>>>
>>>>>> Just split it up.
>>>>>> getVmRuntimeStats() - transient things like mem and cpu%
>>>>>> getVmInformation() - (semi)static things like disk\networking
>>>>>> layout
>>>>>> etc.
>>>>>> Each updated at different intervals.
>>>>> +1 on splitting the data up into 2 separate API calls.
>>>>> You could potentially add a checksum (md5, or any other way) of
>>>>> the
>>>>> "static" data to getVmRuntimeStats and not bother even
with
>>>>> polling
>>>>> the VmInformation if this hasn't changed. Then you could poll
>>>>> as
>>>>> often as you'd like the stats and immediately see if you also
>>>>> need
>>>>> to retrieve VmInfo or not (you rarely would).
>>>> +1 To Ayal's suggestion
>>>> except that instead of the engine hashing the data VDSM sends
>>>> the
>>>> key which is opaque to the engine.
>>>> This can be a local timestap or a generation number.
>>> Of course vdsm does the hash, otherwise you'd need to pass all
>>> the
>>> data to engine which would beat the purpose.
>> I thought you meant engine will be sending the hash of previous
>> requests
>> per VM to vdsm, then vdsm will reply back with vm's removed, vm's
>> added,
>> and the details for vm's that changed (i.e., engine would be doing
>> something like if-modified-since-checksum per vm).
>> benefit is reducing a round trip.
>> but first would need to split to calls of stats (always changing)
>> and
>> slowly/never changing data.
> If vdms accepts the hash then in your method engine would have to
> periodically call getVmInfo(hash).
> What I was suggesting is that getVmStats would return vmInfo hash
> so that we could avoid calling getVmInfo altogether.
> The stats *always* change so there is no need for checking if that
> info has changed.
> What we could do is avoid the split into 2 verbs by calling
> getVmStats(hash) and then have getVmStats return everything if the
> hash has changed or only the stats if it hasn't. This would be
> the least number of roundtrips and avoid the split. If you don't
> pass a hash it would return everything so this way it's also fully
> backward compatible.
For the 'static' data, why is there a need for a hash?
If VDSM sends in each update a timestamp, can't RHEVM just use
if-modified-since with the last timestamp it got from VDSM?
Is it cheaper for VDSM to calculate the hash, than update the
timestamp
per change in any of the fields? It doesn't really need to update the
timestamp per change, only for the first change since last update
sent
actually (so 'dirty' flag in a way, to signify data that RHEVM hasn't
seen yet).
Y.
As Saggi mentioned: "VDSM sends the key which is opaque to the engine. This can be a
local timestap or a generation number."
The content doesn't matter, what matters is that it has changed.
timestamp assumes that vdsm will track changes and send only delta. Although possible
this would be an overkill (for every value in the dict you'd have to hold a timestamp
of last change and send only those which have changed since the timestamp which was passed
by the user).
Either way, I don't care what the 'hash' is, the point was that there is a
simple way to keep a single API call, keep BC and toggle returning all data or just
statistics (data that changes frequently) since last time user checked while minimizing
API calls.
>
>>>> But, we might want to consider that when we add events polling
>>>> becomes (much) less frequent so maybe it'll be an overkill.
>>> You'd still need to compare versions of the data in vdsm and send
>>> only if it changed. If you don't persist what was received last
>>> then potentially you could have a monday morning effect where
>>> upon
>>> on system startup you'd be sending everything. So I still think
>>> you'd want to have the hash.
>>>
>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Vinzenz Feenstra"
<vfeenstr(a)redhat.com>
>>>>>>> To: vdsm-devel(a)lists.fedorahosted.org,
engine-devel(a)ovirt.org
>>>>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
>>>>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
>>>>>>> Statistics
>>>>>>> Retrieval Optimization
>>>>>>>
>>>>>>>
>>>>>>> Please find the prettier version on the wiki:
>>>>>>>
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>>>>>>
>>>>>>> Proposal VDSM - Engine Data Statistics Retrieval
>>>>>>> VDSM <=> Engine data retrieval optimization
>>>>>>> Motivation:
>>>>>>>
>>>>>>>
>>>>>>> Currently the RHEVM engine is polling the a lot of data
from
>>>>>>> VDSM
>>>>>>> every 15 seconds. This should be optimized and the amount
of
>>>>>>> data
>>>>>>> requested should be more specific.
>>>>>>>
>>>>>>> For each VM the data currently contains much more
information
>>>>>>> than
>>>>>>> actually needed which blows up the size of the XML content
>>>>>>> quite
>>>>>>> big. We could optimize this by splitting the reply on the
>>>>>>> getVmStats
>>>>>>> based on the request of the engine into sections. For this
>>>>>>> reason
>>>>>>> Omer Frenkel and me have split up the data into parts based
>>>>>>> on
>>>>>>> their
>>>>>>> usage.
>>>>>>>
>>>>>>> This data can and usually does change during the lifetime
of
>>>>>>> the
>>>>>>> VM.
>>>>>>> Rarely Changed:
>>>>>>>
>>>>>>>
>>>>>>> This data is change not very frequent and it should be
enough
>>>>>>> to
>>>>>>> update this only once in a while. Most commonly this data
>>>>>>> changes
>>>>>>> after changes made in the UI or after a migration of the VM
>>>>>>> to
>>>>>>> another Host. Status = Running acpiEnable = true vmType =
kvm
>>>>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win
8
>>>>>>> kvmEnable = true # this should be constant and never
changed
>>>>>>> pauseCode = NOERR monitorResponse = 0 session = Locked #
>>>>>>> unused
>>>>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast
Ethernet NIC',
>>>>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'],
'inet':
>>>>>>> ['10.34.60.148'],
>>>>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList =
['RHEV-Tools 3.2.4',
>>>>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
>>>>>>> 3.2.2',
>>>>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64
3.2.3', 'RHEV-Balloon64
>>>>>>> 3.2.3', 'RHEV-Balloon64 3.2.2',
'RHEV-Agent64 3.2.2',
>>>>>>> 'RHEV-USB
>>>>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64
3.2.2'] pid =
>>>>>>> 11314
>>>>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
>>>>>>> displayPort
>>>>>>> = 5902 displaySecurePort = 5903 username =
>>>>>>> user@W864GUESTAGENTT
>>>>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
>>>>>>>
>>>>>>>
>>>>>>> This data is changed quite often however it is not
necessary
>>>>>>> to
>>>>>>> update this data every 15 seconds. As this is cumulative
data
>>>>>>> and
>>>>>>> reflects the current status, and it does not need to be
>>>>>>> snapshotted
>>>>>>> every 15 seconds to retrieve statistics. The data can be
>>>>>>> retrieved
>>>>>>> in much more generous time slices. (e.g. Every 5 minutes)
>>>>>>> network
>>>>>>> =
>>>>>>> {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db', 'rxDropped': '0',
>>>>>>> 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate':
>>>>>>> '0.0',
>>>>>>> 'txErrors': '0', 'state':
'unknown', 'speed': '100', 'name':
>>>>>>> 'vnet1'}} disksUsage = [{'path':
'c:\\', 'total':
>>>>>>> '64055406592',
>>>>>>> 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\',
>>>>>>> 'total':
>>>>>>> '3490912256', 'fs': 'UDF',
'used': '3490912256'}] timeOffset
>>>>>>> =
>>>>>>> 14422
>>>>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge =
>>>>>>> 0.09
>>>>>>> #
>>>>>>> unused Often Changed but unused
>>>>>>>
>>>>>>>
>>>>>>> This data does not seem to be used in the engine at all. It
>>>>>>> is
>>>>>>> not
>>>>>>> even used in the data warehouse. memoryStats =
{'swap_out':
>>>>>>> '0',
>>>>>>> 'majflt': '0', 'mem_free':
'1466884', 'swap_in': '0',
>>>>>>> 'pageflt':
>>>>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
>>>>>>> balloonInfo
>>>>>>> =
>>>>>>> {'balloon_max': 2097152, 'balloon_cur':
2097152} disks =
>>>>>>> {'vda':
>>>>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
>>>>>>> 'writeLatency':
>>>>>>> '1754496', 'imageID':
>>>>>>> '28abb923-7b89-4638-84f8-1700f0b76482',
>>>>>>> 'flushLatency': '156549',
'readRate': '0.00', 'truesize':
>>>>>>> '18855059456', 'writeRate':
'952.05'}, 'hdc': {'readLatency':
>>>>>>> '0',
>>>>>>> 'apparentsize': '0', 'writeLatency':
'0', 'flushLatency':
>>>>>>> '0',
>>>>>>> 'readRate': '0.00', 'truesize':
'0', 'writeRate': '0.00'}}
>>>>>>> Very
>>>>>>> frequent uppdates needed by webadmin portal:
>>>>>>>
>>>>>>>
>>>>>>> This data is mostly needed for the webadmin portal and
might
>>>>>>> be
>>>>>>> required to be updated quite often. An exception here is
the
>>>>>>> statsAge field, which seems to be unused by the Engine.
This
>>>>>>> data
>>>>>>> could be requested every 15 seconds to keep things as they
>>>>>>> are
>>>>>>> now.
>>>>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed
Solution
>>>>>>> for
>>>>>>> VDSM & Engine:
>>>>>>>
>>>>>>>
>>>>>>> We will introduce new optional parameters to getVmStats,
>>>>>>> getAllVmStats and list to allow a finer grained
specification
>>>>>>> of
>>>>>>> data which should be included.
>>>>>>>
>>>>>>> Parameter: statsType = <string> (getVmStats,
getAllVmStats
>>>>>>> only)
>>>>>>> Allowed values:
>>>>>>>
>>>>>>> * full (default to keep backwards compatibility)
>>>>>>> * app-list (Just send the application list)
>>>>>>> * rare (include everything from rarely changed to
very
>>>>>>> frequent)
>>>>>>> * often (include everything from often changed to
very
>>>>>>> frequent)
>>>>>>> * frequent (only send the very frequently changed
>>>>>>> items)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Parameter: clientId = <string> The client id is
specified by
>>>>>>> the
>>>>>>> client and should be unique however constantly used.
>>>>>>>
>>>>>>> Parameter: diff = <boolean> In combination with the
clientId
>>>>>>> VDSM
>>>>>>> will send only differences to the previous request from the
>>>>>>> named
>>>>>>> clientId. (if diff=true)
>>>>>>>
>>>>>>>
>>>>>>> Additional Change:
>>>>>>>
>>>>>>>
>>>>>>> Besides the introduction of the new parameters for list,
>>>>>>> getVmStats
>>>>>>> and getAllVmStats it might make sense to include a hash for
>>>>>>> the
>>>>>>> appList into the rarely changed section of the response
which
>>>>>>> would
>>>>>>> allow to identify changes and avoid having to sent the
>>>>>>> complete
>>>>>>> appList every so often and only if the hash known to the
>>>>>>> client
>>>>>>> is
>>>>>>> outdated.
>>>>>>>
>>>>>>> Note: The appList (Application List) reported by the guest
>>>>>>> agent
>>>>>>> could be fully implemented on request only, as long as the
>>>>>>> guest
>>>>>>> agent installed supports this. As there seems to be a
request
>>>>>>> to
>>>>>>> have the complete list of installed applications on all
>>>>>>> guests
>>>>>>> this
>>>>>>> data could be quite extensive and a huge list. On the other
>>>>>>> hand
>>>>>>> this data is only rarely visible and therefore it should
not
>>>>>>> be
>>>>>>> requested all the time and only on demand. Improvement of
the
>>>>>>> Guest
>>>>>>> Agent:
>>>>>>>
>>>>>>>
>>>>>>> As part of the proposed solution it is necessary to improve
>>>>>>> the
>>>>>>> guest
>>>>>>> agent as well. For the full application list there should
be
>>>>>>> implemented a caching system which will be fully reactive
and
>>>>>>> should
>>>>>>> not poll the application list for example all the time. The
>>>>>>> guest
>>>>>>> can create a prepared data file containing all data in the
>>>>>>> JSON
>>>>>>> format (as used for the communication with VDSM via VIO)
and
>>>>>>> just
>>>>>>> have to read that file from disk and directly sends it to
>>>>>>> VDSM.
>>>>>>> However it is quite possible that this list is to big and
it
>>>>>>> might
>>>>>>> have to be chunked into pieces. (Multiple messages, which
>>>>>>> would
>>>>>>> have
>>>>>>> to be supported by VDSM then as well) The solution for this
>>>>>>> is
>>>>>>> to
>>>>>>> make VDSM request this data and it will retrieve the data
>>>>>>> necessary
>>>>>>> on request only. --
>>>>>>> Regards,
>>>>>>>
>>>>>>> Vinzenz Feenstra | Senior Software Engineer
>>>>>>> RedHat Engineering Virtualization R & D
>>>>>>> Phone: +420 532 294 625
>>>>>>> IRC: vfeenstr or evilissimo
>>>>>>>
>>>>>>> Better technology. Faster innovation. Powered by community
>>>>>>> collaboration.
>>>>>>> See how it works at redhat.com
>>>>>>> _______________________________________________
>>>>>>> Engine-devel mailing list
>>>>>>> Engine-devel(a)ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>
>>>>>> _______________________________________________
>>>>>> vdsm-devel mailing list
>>>>>> vdsm-devel(a)lists.fedorahosted.org
>>>>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>>>
>>> _______________________________________________
>>> vdsm-devel mailing list
>>> vdsm-devel(a)lists.fedorahosted.org
>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
6:30 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On Sun, Mar 17, 2013 at 10:28:15AM -0400, Ayal Baron wrote:
----- Original Message -----
> On 17/03/13 15:13, Ayal Baron wrote:
> >
> > ----- Original Message -----
> >> On 03/13/2013 11:55 PM, Ayal Baron wrote:
> >> ...
> >>>>>> The only reason we have this problem is because there is
this
> >>>>>> thing against making multiple calls.
> >>>>>>
> >>>>>> Just split it up.
> >>>>>> getVmRuntimeStats() - transient things like mem and cpu%
> >>>>>> getVmInformation() - (semi)static things like
disk\networking
> >>>>>> layout
> >>>>>> etc.
> >>>>>> Each updated at different intervals.
> >>>>> +1 on splitting the data up into 2 separate API calls.
> >>>>> You could potentially add a checksum (md5, or any other way)
of
> >>>>> the
> >>>>> "static" data to getVmRuntimeStats and not bother
even with
> >>>>> polling
> >>>>> the VmInformation if this hasn't changed. Then you could
poll
> >>>>> as
> >>>>> often as you'd like the stats and immediately see if you
also
> >>>>> need
> >>>>> to retrieve VmInfo or not (you rarely would).
> >>>> +1 To Ayal's suggestion
> >>>> except that instead of the engine hashing the data VDSM sends
> >>>> the
> >>>> key which is opaque to the engine.
> >>>> This can be a local timestap or a generation number.
> >>> Of course vdsm does the hash, otherwise you'd need to pass all
> >>> the
> >>> data to engine which would beat the purpose.
> >> I thought you meant engine will be sending the hash of previous
> >> requests
> >> per VM to vdsm, then vdsm will reply back with vm's removed, vm's
> >> added,
> >> and the details for vm's that changed (i.e., engine would be doing
> >> something like if-modified-since-checksum per vm).
> >> benefit is reducing a round trip.
> >> but first would need to split to calls of stats (always changing)
> >> and
> >> slowly/never changing data.
> > If vdms accepts the hash then in your method engine would have to
> > periodically call getVmInfo(hash).
> > What I was suggesting is that getVmStats would return vmInfo hash
> > so that we could avoid calling getVmInfo altogether.
> > The stats *always* change so there is no need for checking if that
> > info has changed.
> > What we could do is avoid the split into 2 verbs by calling
> > getVmStats(hash) and then have getVmStats return everything if the
> > hash has changed or only the stats if it hasn't. This would be
> > the least number of roundtrips and avoid the split. If you don't
> > pass a hash it would return everything so this way it's also fully
> > backward compatible.
>
> For the 'static' data, why is there a need for a hash?
> If VDSM sends in each update a timestamp, can't RHEVM just use
> if-modified-since with the last timestamp it got from VDSM?
> Is it cheaper for VDSM to calculate the hash, than update the
> timestamp
> per change in any of the fields? It doesn't really need to update the
> timestamp per change, only for the first change since last update
> sent
> actually (so 'dirty' flag in a way, to signify data that RHEVM hasn't
> seen yet).
> Y.
As Saggi mentioned: "VDSM sends the key which is opaque to the engine. This can be a
local timestap or a generation number."
The content doesn't matter, what matters is that it has changed.
timestamp assumes that vdsm will track changes and send only delta.
Although possible this would be an overkill (for every value in the
dict you'd have to hold a timestamp of last change and send only those
which have changed since the timestamp which was passed by the user).
If we're in the spirit of quoting Saggi, this suggestion is not
compatible with "...mak[ing] the return value differ according to input
... is a big no no when talking about type safe APIs.".
Dan.
3 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
This is a multi-part message in MIME format.
--------------090700020909000108020704
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Hi,
I finally have an update for the proposal ready, after some thinking and
throwing some ideas around with colleagues we came to the conclusion
that the best approach might be as followed (without having dynamic
return values)
I have set up some test engine with 'FakeVDSM' and I am tracking the
data sent. The system has 250 hosts and about 10000 VMs running. I have
captured 1.3 GiB TCP data within about 20 minutes only from the
communication between the engine and the fake VDSM host. In a frame of
23 minutes I have captured 13.560 getAllVMStats and getAllVdsStats
calls, and 67827 calls to list.
getAllVmStats replies have produced in that time about 510MiB of data
transmitted to the engine. This proposal below would reduce the overall
amount of data significantly and we're preparing to work on a prototype
for the engine backend as well so we can backup this statement with hard
numbers.
Here's the proposal.
VDSM <=> Engine data retrieval optimization
Motivation:
Currently the oVirt Engine is polling a lot of data from VDSM every 15
seconds. This should be optimized and the amount of data requested
should be more specific.
For each VM the data currently contains much more information than
actually needed which blows up the size of the XML content quite big. We
could optimize this by splitting the reply on the getVmStats based on
the needs of the engine into different requests. For this reason Omer
Frenkel and me have split up the data into parts based on their usage.
Changes
New Verbs
getAllRuntimeStats
Get runtime information of all VMs
Returns for each VM a map with UUID and a value of:
* *@cpuSys* Ratio of CPU time spent by qemu on other than guest time
* *@cpuUser* Ratio of CPU time spent by the guest VM
* *@memUsage* The percent of memory in use by the guest
* *@elapsedTime* The number of seconds that the VM has been running
* *@status* The current status of the given VM
* *@statsAge* The age of these statistics in seconds
* *@hashes* Hashes of several statistics and information around VMs
Hashes consists of:
* *@info* Hash for VmConfInfo data
* *@config* Hash of the VM configuration XML
* *@status* Hash of the VmStatusInfo data
* *@guestDetails* Hash of the VmGuestDetails data
getStatus
Get status information about a list of VMs
Parameters:
* *@vmIDs* a list of UUIDs for VMs to query
Returns for each VM in vmIDs a map with UUID and a value of:
* *timeOffset* The time difference from host to the VM in seconds
* *monitorResponse* Indicates if the qemu monitor is responsive
* *clientIp* The IP address of the client connected to the display
* *username* the username associated with the current session
* *session* The current state of user interaction with the VM
* *guestIPs* A space separated string of assigned IPv4 addresses
* *pauseCode* Indicates the reason a VM has been paused
getConfInfo
Get configuration information about a list of VMs
Parameters:
* *@vmIDs* a list of UUIDs for VMs to query
Returns for each VM in vmIDs a map with UUID and a value of:
* *acpiEnable* Indicates if ACPI is enabled inside the VM
* *displayPort* The port in use for unencrypted display data
* *displaySecurePort* The port in use for encrypted display data
* *displayType* The type of display in use
* *displayIp* The IP address to use for accessing the VM display
* *pid* The process ID of the underlying qemu process
* *vmType* The type of VM
* *kvmEnable* Indicates if KVM hardware acceleration is enabled
* *cdrom* /*optional*/ The path to an ISO image used in the VM's
CD-ROM device
* *boot* /*optional*/ An alias for the type of device used to boot the VM
getAllDeviceStats
VM device statistics containing information for getting statistics and
SLA information
Returns for each VM a map with UUID and a value of:
* *memoryStats* Memory statistics as reported by the guest agent
* *balloonInfo* Guest memory balloon information
* *disksUsage* Info about mounted filesystems as reported by the agent
* *network* Network bandwidth/utilization statistics
* *disks* Disk bandwidth/utilization statistics
getGuestDetails
Get details from the guest OS from a list of VMs
Parameters:
* *@vmIDs* a list of UUIDs for VMs to query
Returns for each VM in vmIDs a map with UUID and a value of:
* *appsList* A list of installed applications with their versions
* *netIfaces* Network device address info as reported by the agent
Usage
Currently the engine is requesting currently every 3 seconds the vm list
from each vdsm host and every 15 seconds all the data mentioned above
for all VMs.
The change would be as follows:
The engine requests every 3 seconds getAllRuntimeStats from vdsm which
will give the engine the most used data. If the engine has a mismatch of
the hashes returned by getAllRuntimeStats it should request the data
changed.
if hashes.info changed => request getConfInfo with all vmIDs on that
host where the hash changed if hashes.status changed => request
getStatus with all vmIDs on that host where the hash changed if
hashes.guestDetails changed => request getGuestDetails with all vmIDs on
that host where the hash changed
Request getAllDeviceStats periodically e.g. every 5 minutes, which
should be sufficient for the DWH, in case it is not it could be even
configurable.
On 03/17/2013 04:30 PM, Dan Kenigsberg wrote:
On Sun, Mar 17, 2013 at 10:28:15AM -0400, Ayal Baron wrote:
>
> ----- Original Message -----
>> On 17/03/13 15:13, Ayal Baron wrote:
>>> ----- Original Message -----
>>>> On 03/13/2013 11:55 PM, Ayal Baron wrote:
>>>> ...
>>>>>>>> The only reason we have this problem is because there is
this
>>>>>>>> thing against making multiple calls.
>>>>>>>>
>>>>>>>> Just split it up.
>>>>>>>> getVmRuntimeStats() - transient things like mem and cpu%
>>>>>>>> getVmInformation() - (semi)static things like
disk\networking
>>>>>>>> layout
>>>>>>>> etc.
>>>>>>>> Each updated at different intervals.
>>>>>>> +1 on splitting the data up into 2 separate API calls.
>>>>>>> You could potentially add a checksum (md5, or any other way)
of
>>>>>>> the
>>>>>>> "static" data to getVmRuntimeStats and not bother
even with
>>>>>>> polling
>>>>>>> the VmInformation if this hasn't changed. Then you could
poll
>>>>>>> as
>>>>>>> often as you'd like the stats and immediately see if you
also
>>>>>>> need
>>>>>>> to retrieve VmInfo or not (you rarely would).
>>>>>> +1 To Ayal's suggestion
>>>>>> except that instead of the engine hashing the data VDSM sends
>>>>>> the
>>>>>> key which is opaque to the engine.
>>>>>> This can be a local timestap or a generation number.
>>>>> Of course vdsm does the hash, otherwise you'd need to pass all
>>>>> the
>>>>> data to engine which would beat the purpose.
>>>> I thought you meant engine will be sending the hash of previous
>>>> requests
>>>> per VM to vdsm, then vdsm will reply back with vm's removed,
vm's
>>>> added,
>>>> and the details for vm's that changed (i.e., engine would be doing
>>>> something like if-modified-since-checksum per vm).
>>>> benefit is reducing a round trip.
>>>> but first would need to split to calls of stats (always changing)
>>>> and
>>>> slowly/never changing data.
>>> If vdms accepts the hash then in your method engine would have to
>>> periodically call getVmInfo(hash).
>>> What I was suggesting is that getVmStats would return vmInfo hash
>>> so that we could avoid calling getVmInfo altogether.
>>> The stats *always* change so there is no need for checking if that
>>> info has changed.
>>> What we could do is avoid the split into 2 verbs by calling
>>> getVmStats(hash) and then have getVmStats return everything if the
>>> hash has changed or only the stats if it hasn't. This would be
>>> the least number of roundtrips and avoid the split. If you don't
>>> pass a hash it would return everything so this way it's also fully
>>> backward compatible.
>> For the 'static' data, why is there a need for a hash?
>> If VDSM sends in each update a timestamp, can't RHEVM just use
>> if-modified-since with the last timestamp it got from VDSM?
>> Is it cheaper for VDSM to calculate the hash, than update the
>> timestamp
>> per change in any of the fields? It doesn't really need to update the
>> timestamp per change, only for the first change since last update
>> sent
>> actually (so 'dirty' flag in a way, to signify data that RHEVM
hasn't
>> seen yet).
>> Y.
> As Saggi mentioned: "VDSM sends the key which is opaque to the engine. This can
be a local timestap or a generation number."
>
> The content doesn't matter, what matters is that it has changed.
> timestamp assumes that vdsm will track changes and send only delta.
> Although possible this would be an overkill (for every value in the
> dict you'd have to hold a timestamp of last change and send only those
> which have changed since the timestamp which was passed by the user).
If we're in the spirit of quoting Saggi, this suggestion is not
compatible with "...mak[ing] the return value differ according to input
... is a big no no when talking about type safe APIs.".
Dan.
_______________________________________________
vdsm-devel mailing list
vdsm-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
--------------090700020909000108020704
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi,<br>
<br>
I finally have an update for the proposal ready, after some
thinking and throwing some ideas around with colleagues we came to
the conclusion that the best approach might be as followed
(without having dynamic return values)<br>
<br>
I have set up some test engine with 'FakeVDSM' and I am tracking
the data sent. The system has 250 hosts and about 10000 VMs
running. I have captured 1.3 GiB TCP data within about 20 minutes
only from the communication between the engine and the fake VDSM
host. In a frame of 23 minutes I have captured
<meta http-equiv="CONTENT-TYPE" content="text/html;
charset=UTF-8">
<title></title>
<meta name="GENERATOR" content="LibreOffice 3.6 (Linux)">
<style>
<!--
BODY,DIV,TABLE,THEAD,TBODY,TFOOT,TR,TH,TD,P { font-family:"Liberation Sans";
font-size:x-small }
-->
</style>13.560 getAllVMStats and getAllVdsStats calls, and 67827 calls
to list.<br>
<br>
getAllVmStats replies have produced in that time about 510MiB of
data transmitted to the engine. This proposal below would reduce
the overall amount of data significantly and we're preparing to
work on a prototype for the engine backend as well so we can
backup this statement with hard numbers.<br>
<br>
<br>
Here's the proposal.<br>
<br>
<h2><span class="mw-headline"
id="VDSM_.3C.3D.3E_Engine_data_retrieval_optimization">VDSM
<=> Engine data retrieval optimization </span></h2>
<h3> <span class="mw-headline" id="Motivation:">
Motivation: </span></h3>
<p>Currently the oVirt Engine is polling a lot of data from VDSM
every 15 seconds. This should be optimized and the amount of
data requested should be more specific.
</p>
<p>For each VM the data currently contains much more information
than actually needed which blows up the size of the XML content
quite big. We could optimize this by splitting the reply on the
getVmStats based on the needs of the engine into different
requests.
For this reason Omer Frenkel and me have split up the data into
parts based on their usage.
</p>
<h3> <span class="mw-headline" id="Changes"> Changes
</span></h3>
<h4> <span class="mw-headline" id="New_Verbs"> New
Verbs </span></h4>
<h5> <span class="mw-headline"
id="getAllRuntimeStats">
getAllRuntimeStats </span></h5>
<p>Get runtime information of all VMs<br>
Returns for each VM a map with UUID and a value of:
</p>
<ul>
<li> <b>@cpuSys</b> Ratio of CPU time spent by qemu on other
than guest time
</li>
<li> <b>@cpuUser</b> Ratio of CPU time spent by the guest VM
</li>
<li> <b>@memUsage</b> The percent of memory in use by the guest
</li>
<li> <b>@elapsedTime</b> The number of seconds that the VM has
been running
</li>
<li> <b>@status</b> The current status of the given VM
</li>
<li> <b>@statsAge</b> The age of these statistics in seconds
</li>
<li> <b>@hashes</b> Hashes of several statistics and
information around VMs
</li>
</ul>
<p>Hashes consists of:
</p>
<ul>
<li> <b>@info</b> Hash for VmConfInfo data
</li>
<li> <b>@config</b> Hash of the VM configuration XML
</li>
<li> <b>@status</b> Hash of the VmStatusInfo data
</li>
<li> <b>@guestDetails</b> Hash of the VmGuestDetails data
</li>
</ul>
<h5> <span class="mw-headline" id="getStatus">
getStatus </span></h5>
<p>Get status information about a list of VMs<br>
Parameters: </p>
<ul>
<li> <b>@vmIDs</b> a list of UUIDs for VMs to query
</li>
</ul>
<p>Returns for each VM in vmIDs a map with UUID and a value of:
</p>
<ul>
<li> <b>timeOffset</b> The time difference from host to the VM
in seconds
</li>
<li> <b>monitorResponse</b> Indicates if the qemu monitor is
responsive
</li>
<li> <b>clientIp</b> The IP address of the client connected to
the display
</li>
<li> <b>username</b> the username associated with the current
session
</li>
<li> <b>session</b> The current state of user interaction with
the VM
</li>
<li> <b>guestIPs</b> A space separated string of assigned IPv4
addresses
</li>
<li> <b>pauseCode</b> Indicates the reason a VM has been paused
</li>
</ul>
<h5> <span class="mw-headline" id="getConfInfo">
getConfInfo </span></h5>
<p>Get configuration information about a list of VMs<br>
Parameters: </p>
<ul>
<li> <b>@vmIDs</b> a list of UUIDs for VMs to query
</li>
</ul>
<p>Returns for each VM in vmIDs a map with UUID and a value of:
</p>
<ul>
<li> <b>acpiEnable</b> Indicates if ACPI is enabled inside the
VM
</li>
<li> <b>displayPort</b> The port in use for unencrypted display
data
</li>
<li> <b>displaySecurePort</b> The port in use for encrypted
display data
</li>
<li> <b>displayType</b> The type of display in use
</li>
<li> <b>displayIp</b> The IP address to use for accessing the
VM display
</li>
<li> <b>pid</b> The process ID of the underlying qemu process
</li>
<li> <b>vmType</b> The type of VM
</li>
<li> <b>kvmEnable</b> Indicates if KVM hardware acceleration is
enabled
</li>
<li> <b>cdrom</b> <i><b>optional</b></i>
The path to an ISO
image used in the VM's CD-ROM device
</li>
<li> <b>boot</b> <i><b>optional</b></i>
An alias for the type
of device used to boot the VM
</li>
</ul>
<h5> <span class="mw-headline"
id="getAllDeviceStats">
getAllDeviceStats </span></h5>
<p>VM device statistics containing information for getting
statistics and SLA information<br>
Returns for each VM a map with UUID and a value of:
</p>
<ul>
<li> <b>memoryStats</b> Memory statistics as reported by the
guest agent
</li>
<li> <b>balloonInfo</b> Guest memory balloon information
</li>
<li> <b>disksUsage</b> Info about mounted filesystems as
reported by the agent
</li>
<li> <b>network</b> Network bandwidth/utilization statistics
</li>
<li> <b>disks</b> Disk bandwidth/utilization statistics
</li>
</ul>
<h5> <span class="mw-headline"
id="getGuestDetails">
getGuestDetails </span></h5>
<p>Get details from the guest OS from a list of VMs<br>
Parameters: </p>
<ul>
<li> <b>@vmIDs</b> a list of UUIDs for VMs to query
</li>
</ul>
<p>Returns for each VM in vmIDs a map with UUID and a value of:
</p>
<ul>
<li> <b>appsList</b> A list of installed applications with
their versions
</li>
<li> <b>netIfaces</b> Network device address info as reported
by the agent
</li>
</ul>
<h4> <span class="mw-headline" id="Usage"> Usage
</span></h4>
<p>Currently the engine is requesting currently every 3 seconds
the vm list from each vdsm host and every 15 seconds all the
data mentioned above for all VMs.
</p>
<p>The change would be as follows:
</p>
<p>The engine requests every 3 seconds getAllRuntimeStats from
vdsm which will give the engine the most used data.
If the engine has a mismatch of the hashes returned by
getAllRuntimeStats it should request the data changed.
</p>
<p>if hashes.info changed => request getConfInfo with all vmIDs
on that host where the hash changed
if hashes.status changed => request getStatus with all vmIDs
on that host where the hash changed
if hashes.guestDetails changed => request getGuestDetails
with all vmIDs on that host where the hash changed
</p>
<p>Request getAllDeviceStats periodically e.g. every 5 minutes,
which should be sufficient for the DWH, in case it is not it
could be even configurable.
</p>
<br>
<br>
On 03/17/2013 04:30 PM, Dan Kenigsberg wrote:<br>
</div>
<blockquote cite="mid:20130317153038.GB4891@redhat.com"
type="cite">
<pre wrap="">On Sun, Mar 17, 2013 at 10:28:15AM -0400, Ayal Baron
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
----- Original Message -----
</pre>
<blockquote type="cite">
<pre wrap="">On 17/03/13 15:13, Ayal Baron wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
----- Original Message -----
</pre>
<blockquote type="cite">
<pre wrap="">On 03/13/2013 11:55 PM, Ayal Baron wrote:
...
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">The only reason we have this problem is
because there is this
thing against making multiple calls.
Just split it up.
getVmRuntimeStats() - transient things like mem and cpu%
getVmInformation() - (semi)static things like disk\networking
layout
etc.
Each updated at different intervals.
</pre>
</blockquote>
<pre wrap="">+1 on splitting the data up into 2
separate API calls.
You could potentially add a checksum (md5, or any other way) of
the
"static" data to getVmRuntimeStats and not bother even with
polling
the VmInformation if this hasn't changed. Then you could poll
as
often as you'd like the stats and immediately see if you also
need
to retrieve VmInfo or not (you rarely would).
</pre>
</blockquote>
<pre wrap="">+1 To Ayal's suggestion
except that instead of the engine hashing the data VDSM sends
the
key which is opaque to the engine.
This can be a local timestap or a generation number.
</pre>
</blockquote>
<pre wrap="">Of course vdsm does the hash, otherwise
you'd need to pass all
the
data to engine which would beat the purpose.
</pre>
</blockquote>
<pre wrap="">I thought you meant engine will be sending the
hash of previous
requests
per VM to vdsm, then vdsm will reply back with vm's removed, vm's
added,
and the details for vm's that changed (i.e., engine would be doing
something like if-modified-since-checksum per vm).
benefit is reducing a round trip.
but first would need to split to calls of stats (always changing)
and
slowly/never changing data.
</pre>
</blockquote>
<pre wrap="">If vdms accepts the hash then in your method
engine would have to
periodically call getVmInfo(hash).
What I was suggesting is that getVmStats would return vmInfo hash
so that we could avoid calling getVmInfo altogether.
The stats *always* change so there is no need for checking if that
info has changed.
What we could do is avoid the split into 2 verbs by calling
getVmStats(hash) and then have getVmStats return everything if the
hash has changed or only the stats if it hasn't. This would be
the least number of roundtrips and avoid the split. If you don't
pass a hash it would return everything so this way it's also fully
backward compatible.
</pre>
</blockquote>
<pre wrap="">
For the 'static' data, why is there a need for a hash?
If VDSM sends in each update a timestamp, can't RHEVM just use
if-modified-since with the last timestamp it got from VDSM?
Is it cheaper for VDSM to calculate the hash, than update the
timestamp
per change in any of the fields? It doesn't really need to update the
timestamp per change, only for the first change since last update
sent
actually (so 'dirty' flag in a way, to signify data that RHEVM hasn't
seen yet).
Y.
</pre>
</blockquote>
<pre wrap="">
As Saggi mentioned: "VDSM sends the key which is opaque to the engine. This can be a
local timestap or a generation number."
The content doesn't matter, what matters is that it has changed.
timestamp assumes that vdsm will track changes and send only delta.
Although possible this would be an overkill (for every value in the
dict you'd have to hold a timestamp of last change and send only those
which have changed since the timestamp which was passed by the user).
</pre>
</blockquote>
<pre wrap="">
If we're in the spirit of quoting Saggi, this suggestion is not
compatible with "...mak[ing] the return value differ according to input
... is a big no no when talking about type safe APIs.".
Dan.
_______________________________________________
vdsm-devel mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:vdsm-devel@lists.fedorahosted.org">vdsm-devel@lists.fedorahosted.org</a>
<a class="moz-txt-link-freetext"
href="https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel&quo...
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com</pre>
</body>
</html>
--------------090700020909000108020704--
5:11 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
From: "Ayal Baron" <abaron(a)redhat.com>
To: "Itamar Heim" <iheim(a)redhat.com>
Cc: engine-devel(a)ovirt.org, vdsm-devel(a)lists.fedorahosted.org
Sent: Sunday, March 17, 2013 3:13:09 PM
Subject: Re: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics
Retrieval Optimization
----- Original Message -----
> On 03/13/2013 11:55 PM, Ayal Baron wrote:
> ...
> >>>> The only reason we have this problem is because there is this
> >>>> thing against making multiple calls.
> >>>>
> >>>> Just split it up.
> >>>> getVmRuntimeStats() - transient things like mem and cpu%
> >>>> getVmInformation() - (semi)static things like disk\networking
> >>>> layout
> >>>> etc.
> >>>> Each updated at different intervals.
> >>>
> >>> +1 on splitting the data up into 2 separate API calls.
> >>> You could potentially add a checksum (md5, or any other way) of
> >>> the
> >>> "static" data to getVmRuntimeStats and not bother even with
> >>> polling
> >>> the VmInformation if this hasn't changed. Then you could poll
> >>> as
> >>> often as you'd like the stats and immediately see if you also
> >>> need
> >>> to retrieve VmInfo or not (you rarely would).
> >> +1 To Ayal's suggestion
> >> except that instead of the engine hashing the data VDSM sends
> >> the
> >> key which is opaque to the engine.
> >> This can be a local timestap or a generation number.
> >
> > Of course vdsm does the hash, otherwise you'd need to pass all
> > the
> > data to engine which would beat the purpose.
>
> I thought you meant engine will be sending the hash of previous
> requests
> per VM to vdsm, then vdsm will reply back with vm's removed, vm's
> added,
> and the details for vm's that changed (i.e., engine would be doing
> something like if-modified-since-checksum per vm).
> benefit is reducing a round trip.
> but first would need to split to calls of stats (always changing)
> and
> slowly/never changing data.
If vdms accepts the hash then in your method engine would have to
periodically call getVmInfo(hash).
What I was suggesting is that getVmStats would return vmInfo hash so
that we could avoid calling getVmInfo altogether.
The stats *always* change so there is no need for checking if that
info has changed.
What we could do is avoid the split into 2 verbs by calling
getVmStats(hash) and then have getVmStats return everything if the
hash has changed or only the stats if it hasn't. This would be the
least number of roundtrips and avoid the split. If you don't pass a
hash it would return everything so this way it's also fully backward
compatible.
Actually, I assume we can pass hash 0 (to have vdsm return "everything"). I
assume that the chances for md5 on "real data" (i.e -
real data that is known to engine) to be 0 are very slim.
>
> >
> >>
> >> But, we might want to consider that when we add events polling
> >> becomes (much) less frequent so maybe it'll be an overkill.
> >
> > You'd still need to compare versions of the data in vdsm and send
> > only if it changed. If you don't persist what was received last
> > then potentially you could have a monday morning effect where
> > upon
> > on system startup you'd be sending everything. So I still think
> > you'd want to have the hash.
> >
> >
> >>
> >>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
> >>>>> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
> >>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
> >>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
> >>>>> Statistics
> >>>>> Retrieval Optimization
> >>>>>
> >>>>>
> >>>>> Please find the prettier version on the wiki:
> >>>>>
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
> >>>>>
> >>>>> Proposal VDSM - Engine Data Statistics Retrieval
> >>>>> VDSM <=> Engine data retrieval optimization
> >>>>> Motivation:
> >>>>>
> >>>>>
> >>>>> Currently the RHEVM engine is polling the a lot of data from
> >>>>> VDSM
> >>>>> every 15 seconds. This should be optimized and the amount of
> >>>>> data
> >>>>> requested should be more specific.
> >>>>>
> >>>>> For each VM the data currently contains much more information
> >>>>> than
> >>>>> actually needed which blows up the size of the XML content
> >>>>> quite
> >>>>> big. We could optimize this by splitting the reply on the
> >>>>> getVmStats
> >>>>> based on the request of the engine into sections. For this
> >>>>> reason
> >>>>> Omer Frenkel and me have split up the data into parts based
> >>>>> on
> >>>>> their
> >>>>> usage.
> >>>>>
> >>>>> This data can and usually does change during the lifetime of
> >>>>> the
> >>>>> VM.
> >>>>> Rarely Changed:
> >>>>>
> >>>>>
> >>>>> This data is change not very frequent and it should be enough
> >>>>> to
> >>>>> update this only once in a while. Most commonly this data
> >>>>> changes
> >>>>> after changes made in the UI or after a migration of the VM
> >>>>> to
> >>>>> another Host. Status = Running acpiEnable = true vmType = kvm
> >>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
> >>>>> kvmEnable = true # this should be constant and never changed
> >>>>> pauseCode = NOERR monitorResponse = 0 session = Locked #
> >>>>> unused
> >>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast
Ethernet NIC',
> >>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'],
'inet':
> >>>>> ['10.34.60.148'],
> >>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList =
['RHEV-Tools 3.2.4',
> >>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
> >>>>> 3.2.2',
> >>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
> >>>>> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64
3.2.2',
> >>>>> 'RHEV-USB
> >>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64
3.2.2'] pid =
> >>>>> 11314
> >>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
> >>>>> displayPort
> >>>>> = 5902 displaySecurePort = 5903 username =
> >>>>> user@W864GUESTAGENTT
> >>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
> >>>>>
> >>>>>
> >>>>> This data is changed quite often however it is not necessary
> >>>>> to
> >>>>> update this data every 15 seconds. As this is cumulative data
> >>>>> and
> >>>>> reflects the current status, and it does not need to be
> >>>>> snapshotted
> >>>>> every 15 seconds to retrieve statistics. The data can be
> >>>>> retrieved
> >>>>> in much more generous time slices. (e.g. Every 5 minutes)
> >>>>> network
> >>>>> =
> >>>>> {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db', 'rxDropped': '0',
> >>>>> 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate':
> >>>>> '0.0',
> >>>>> 'txErrors': '0', 'state':
'unknown', 'speed': '100', 'name':
> >>>>> 'vnet1'}} disksUsage = [{'path':
'c:\\', 'total':
> >>>>> '64055406592',
> >>>>> 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\',
> >>>>> 'total':
> >>>>> '3490912256', 'fs': 'UDF',
'used': '3490912256'}] timeOffset
> >>>>> =
> >>>>> 14422
> >>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge =
> >>>>> 0.09
> >>>>> #
> >>>>> unused Often Changed but unused
> >>>>>
> >>>>>
> >>>>> This data does not seem to be used in the engine at all. It
> >>>>> is
> >>>>> not
> >>>>> even used in the data warehouse. memoryStats =
{'swap_out':
> >>>>> '0',
> >>>>> 'majflt': '0', 'mem_free':
'1466884', 'swap_in': '0',
> >>>>> 'pageflt':
> >>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
> >>>>> balloonInfo
> >>>>> =
> >>>>> {'balloon_max': 2097152, 'balloon_cur':
2097152} disks =
> >>>>> {'vda':
> >>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
> >>>>> 'writeLatency':
> >>>>> '1754496', 'imageID':
> >>>>> '28abb923-7b89-4638-84f8-1700f0b76482',
> >>>>> 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
> >>>>> '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
> >>>>> '0',
> >>>>> 'apparentsize': '0', 'writeLatency':
'0', 'flushLatency':
> >>>>> '0',
> >>>>> 'readRate': '0.00', 'truesize':
'0', 'writeRate': '0.00'}}
> >>>>> Very
> >>>>> frequent uppdates needed by webadmin portal:
> >>>>>
> >>>>>
> >>>>> This data is mostly needed for the webadmin portal and might
> >>>>> be
> >>>>> required to be updated quite often. An exception here is the
> >>>>> statsAge field, which seems to be unused by the Engine. This
> >>>>> data
> >>>>> could be requested every 15 seconds to keep things as they
> >>>>> are
> >>>>> now.
> >>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
> >>>>> for
> >>>>> VDSM & Engine:
> >>>>>
> >>>>>
> >>>>> We will introduce new optional parameters to getVmStats,
> >>>>> getAllVmStats and list to allow a finer grained specification
> >>>>> of
> >>>>> data which should be included.
> >>>>>
> >>>>> Parameter: statsType = <string> (getVmStats,
getAllVmStats
> >>>>> only)
> >>>>> Allowed values:
> >>>>>
> >>>>> * full (default to keep backwards compatibility)
> >>>>> * app-list (Just send the application list)
> >>>>> * rare (include everything from rarely changed to very
> >>>>> frequent)
> >>>>> * often (include everything from often changed to very
> >>>>> frequent)
> >>>>> * frequent (only send the very frequently changed items)
> >>>>>
> >>>>>
> >>>>>
> >>>>> Parameter: clientId = <string> The client id is specified
by
> >>>>> the
> >>>>> client and should be unique however constantly used.
> >>>>>
> >>>>> Parameter: diff = <boolean> In combination with the
clientId
> >>>>> VDSM
> >>>>> will send only differences to the previous request from the
> >>>>> named
> >>>>> clientId. (if diff=true)
> >>>>>
> >>>>>
> >>>>> Additional Change:
> >>>>>
> >>>>>
> >>>>> Besides the introduction of the new parameters for list,
> >>>>> getVmStats
> >>>>> and getAllVmStats it might make sense to include a hash for
> >>>>> the
> >>>>> appList into the rarely changed section of the response which
> >>>>> would
> >>>>> allow to identify changes and avoid having to sent the
> >>>>> complete
> >>>>> appList every so often and only if the hash known to the
> >>>>> client
> >>>>> is
> >>>>> outdated.
> >>>>>
> >>>>> Note: The appList (Application List) reported by the guest
> >>>>> agent
> >>>>> could be fully implemented on request only, as long as the
> >>>>> guest
> >>>>> agent installed supports this. As there seems to be a request
> >>>>> to
> >>>>> have the complete list of installed applications on all
> >>>>> guests
> >>>>> this
> >>>>> data could be quite extensive and a huge list. On the other
> >>>>> hand
> >>>>> this data is only rarely visible and therefore it should not
> >>>>> be
> >>>>> requested all the time and only on demand. Improvement of the
> >>>>> Guest
> >>>>> Agent:
> >>>>>
> >>>>>
> >>>>> As part of the proposed solution it is necessary to improve
> >>>>> the
> >>>>> guest
> >>>>> agent as well. For the full application list there should be
> >>>>> implemented a caching system which will be fully reactive and
> >>>>> should
> >>>>> not poll the application list for example all the time. The
> >>>>> guest
> >>>>> can create a prepared data file containing all data in the
> >>>>> JSON
> >>>>> format (as used for the communication with VDSM via VIO) and
> >>>>> just
> >>>>> have to read that file from disk and directly sends it to
> >>>>> VDSM.
> >>>>> However it is quite possible that this list is to big and it
> >>>>> might
> >>>>> have to be chunked into pieces. (Multiple messages, which
> >>>>> would
> >>>>> have
> >>>>> to be supported by VDSM then as well) The solution for this
> >>>>> is
> >>>>> to
> >>>>> make VDSM request this data and it will retrieve the data
> >>>>> necessary
> >>>>> on request only. --
> >>>>> Regards,
> >>>>>
> >>>>> Vinzenz Feenstra | Senior Software Engineer
> >>>>> RedHat Engineering Virtualization R & D
> >>>>> Phone: +420 532 294 625
> >>>>> IRC: vfeenstr or evilissimo
> >>>>>
> >>>>> Better technology. Faster innovation. Powered by community
> >>>>> collaboration.
> >>>>> See how it works at redhat.com
> >>>>> _______________________________________________
> >>>>> Engine-devel mailing list
> >>>>> Engine-devel(a)ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>>
> >>>> _______________________________________________
> >>>> vdsm-devel mailing list
> >>>> vdsm-devel(a)lists.fedorahosted.org
> >>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>>
> >>>
> >>
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel(a)lists.fedorahosted.org
> > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> >
>
>
_______________________________________________
Engine-devel mailing list
Engine-devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
5:29 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
----- Original Message -----
> From: "Ayal Baron" <abaron(a)redhat.com>
> To: "Itamar Heim" <iheim(a)redhat.com>
> Cc: engine-devel(a)ovirt.org, vdsm-devel(a)lists.fedorahosted.org
> Sent: Sunday, March 17, 2013 3:13:09 PM
> Subject: Re: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data
> Statistics Retrieval Optimization
>
>
>
> ----- Original Message -----
> > On 03/13/2013 11:55 PM, Ayal Baron wrote:
> > ...
> > >>>> The only reason we have this problem is because there is
> > >>>> this
> > >>>> thing against making multiple calls.
> > >>>>
> > >>>> Just split it up.
> > >>>> getVmRuntimeStats() - transient things like mem and cpu%
> > >>>> getVmInformation() - (semi)static things like
> > >>>> disk\networking
> > >>>> layout
> > >>>> etc.
> > >>>> Each updated at different intervals.
> > >>>
> > >>> +1 on splitting the data up into 2 separate API calls.
> > >>> You could potentially add a checksum (md5, or any other way)
> > >>> of
> > >>> the
> > >>> "static" data to getVmRuntimeStats and not bother even
with
> > >>> polling
> > >>> the VmInformation if this hasn't changed. Then you could
> > >>> poll
> > >>> as
> > >>> often as you'd like the stats and immediately see if you also
> > >>> need
> > >>> to retrieve VmInfo or not (you rarely would).
> > >> +1 To Ayal's suggestion
> > >> except that instead of the engine hashing the data VDSM sends
> > >> the
> > >> key which is opaque to the engine.
> > >> This can be a local timestap or a generation number.
> > >
> > > Of course vdsm does the hash, otherwise you'd need to pass all
> > > the
> > > data to engine which would beat the purpose.
> >
> > I thought you meant engine will be sending the hash of previous
> > requests
> > per VM to vdsm, then vdsm will reply back with vm's removed, vm's
> > added,
> > and the details for vm's that changed (i.e., engine would be
> > doing
> > something like if-modified-since-checksum per vm).
> > benefit is reducing a round trip.
> > but first would need to split to calls of stats (always changing)
> > and
> > slowly/never changing data.
>
> If vdms accepts the hash then in your method engine would have to
> periodically call getVmInfo(hash).
> What I was suggesting is that getVmStats would return vmInfo hash
> so
> that we could avoid calling getVmInfo altogether.
> The stats *always* change so there is no need for checking if that
> info has changed.
> What we could do is avoid the split into 2 verbs by calling
> getVmStats(hash) and then have getVmStats return everything if the
> hash has changed or only the stats if it hasn't. This would be the
> least number of roundtrips and avoid the split. If you don't pass
> a
> hash it would return everything so this way it's also fully
> backward
> compatible.
Actually, I assume we can pass hash 0 (to have vdsm return
"everything"). I assume that the chances for md5 on "real data" (i.e
-
real data that is known to engine) to be 0 are very slim.
We'd need to support hash=None to keep backward compatibility, plus there are no
assumptions this way on hash algorithm so why bother with hash=0?
>
> >
> > >
> > >>
> > >> But, we might want to consider that when we add events polling
> > >> becomes (much) less frequent so maybe it'll be an overkill.
> > >
> > > You'd still need to compare versions of the data in vdsm and
> > > send
> > > only if it changed. If you don't persist what was received
> > > last
> > > then potentially you could have a monday morning effect where
> > > upon
> > > on system startup you'd be sending everything. So I still
> > > think
> > > you'd want to have the hash.
> > >
> > >
> > >>
> > >>>
> > >>>>
> > >>>> ----- Original Message -----
> > >>>>> From: "Vinzenz Feenstra"
<vfeenstr(a)redhat.com>
> > >>>>> To: vdsm-devel(a)lists.fedorahosted.org,
> > >>>>> engine-devel(a)ovirt.org
> > >>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
> > >>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine
Data
> > >>>>> Statistics
> > >>>>> Retrieval Optimization
> > >>>>>
> > >>>>>
> > >>>>> Please find the prettier version on the wiki:
> > >>>>>
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
> > >>>>>
> > >>>>> Proposal VDSM - Engine Data Statistics Retrieval
> > >>>>> VDSM <=> Engine data retrieval optimization
> > >>>>> Motivation:
> > >>>>>
> > >>>>>
> > >>>>> Currently the RHEVM engine is polling the a lot of data
> > >>>>> from
> > >>>>> VDSM
> > >>>>> every 15 seconds. This should be optimized and the amount
> > >>>>> of
> > >>>>> data
> > >>>>> requested should be more specific.
> > >>>>>
> > >>>>> For each VM the data currently contains much more
> > >>>>> information
> > >>>>> than
> > >>>>> actually needed which blows up the size of the XML
content
> > >>>>> quite
> > >>>>> big. We could optimize this by splitting the reply on the
> > >>>>> getVmStats
> > >>>>> based on the request of the engine into sections. For
this
> > >>>>> reason
> > >>>>> Omer Frenkel and me have split up the data into parts
based
> > >>>>> on
> > >>>>> their
> > >>>>> usage.
> > >>>>>
> > >>>>> This data can and usually does change during the lifetime
> > >>>>> of
> > >>>>> the
> > >>>>> VM.
> > >>>>> Rarely Changed:
> > >>>>>
> > >>>>>
> > >>>>> This data is change not very frequent and it should be
> > >>>>> enough
> > >>>>> to
> > >>>>> update this only once in a while. Most commonly this data
> > >>>>> changes
> > >>>>> after changes made in the UI or after a migration of the
VM
> > >>>>> to
> > >>>>> another Host. Status = Running acpiEnable = true vmType =
> > >>>>> kvm
> > >>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs =
Win
> > >>>>> 8
> > >>>>> kvmEnable = true # this should be constant and never
> > >>>>> changed
> > >>>>> pauseCode = NOERR monitorResponse = 0 session = Locked #
> > >>>>> unused
> > >>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast
Ethernet
> > >>>>> NIC',
> > >>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'],
'inet':
> > >>>>> ['10.34.60.148'],
> > >>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList =
['RHEV-Tools 3.2.4',
> > >>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64
3.2.3',
> > >>>>> 'RHEV-Network64
> > >>>>> 3.2.2',
> > >>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64
3.2.3',
> > >>>>> 'RHEV-Balloon64
> > >>>>> 3.2.3', 'RHEV-Balloon64 3.2.2',
'RHEV-Agent64 3.2.2',
> > >>>>> 'RHEV-USB
> > >>>>> 3.2.3', 'RHEV-Block64 3.2.2',
'RHEV-Serial64 3.2.2'] pid =
> > >>>>> 11314
> > >>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
> > >>>>> displayPort
> > >>>>> = 5902 displaySecurePort = 5903 username =
> > >>>>> user@W864GUESTAGENTT
> > >>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
> > >>>>>
> > >>>>>
> > >>>>> This data is changed quite often however it is not
> > >>>>> necessary
> > >>>>> to
> > >>>>> update this data every 15 seconds. As this is cumulative
> > >>>>> data
> > >>>>> and
> > >>>>> reflects the current status, and it does not need to be
> > >>>>> snapshotted
> > >>>>> every 15 seconds to retrieve statistics. The data can be
> > >>>>> retrieved
> > >>>>> in much more generous time slices. (e.g. Every 5 minutes)
> > >>>>> network
> > >>>>> =
> > >>>>> {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db', 'rxDropped':
> > >>>>> '0',
> > >>>>> 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0',
> > >>>>> 'rxRate':
> > >>>>> '0.0',
> > >>>>> 'txErrors': '0', 'state':
'unknown', 'speed': '100',
> > >>>>> 'name':
> > >>>>> 'vnet1'}} disksUsage = [{'path':
'c:\\', 'total':
> > >>>>> '64055406592',
> > >>>>> 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\',
> > >>>>> 'total':
> > >>>>> '3490912256', 'fs': 'UDF',
'used': '3490912256'}]
> > >>>>> timeOffset
> > >>>>> =
> > >>>>> 14422
> > >>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge =
> > >>>>> 0.09
> > >>>>> #
> > >>>>> unused Often Changed but unused
> > >>>>>
> > >>>>>
> > >>>>> This data does not seem to be used in the engine at all.
It
> > >>>>> is
> > >>>>> not
> > >>>>> even used in the data warehouse. memoryStats =
{'swap_out':
> > >>>>> '0',
> > >>>>> 'majflt': '0', 'mem_free':
'1466884', 'swap_in': '0',
> > >>>>> 'pageflt':
> > >>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
> > >>>>> balloonInfo
> > >>>>> =
> > >>>>> {'balloon_max': 2097152, 'balloon_cur':
2097152} disks =
> > >>>>> {'vda':
> > >>>>> {'readLatency': '0',
'apparentsize': '64424509440',
> > >>>>> 'writeLatency':
> > >>>>> '1754496', 'imageID':
> > >>>>> '28abb923-7b89-4638-84f8-1700f0b76482',
> > >>>>> 'flushLatency': '156549',
'readRate': '0.00', 'truesize':
> > >>>>> '18855059456', 'writeRate':
'952.05'}, 'hdc':
> > >>>>> {'readLatency':
> > >>>>> '0',
> > >>>>> 'apparentsize': '0',
'writeLatency': '0', 'flushLatency':
> > >>>>> '0',
> > >>>>> 'readRate': '0.00', 'truesize':
'0', 'writeRate': '0.00'}}
> > >>>>> Very
> > >>>>> frequent uppdates needed by webadmin portal:
> > >>>>>
> > >>>>>
> > >>>>> This data is mostly needed for the webadmin portal and
> > >>>>> might
> > >>>>> be
> > >>>>> required to be updated quite often. An exception here is
> > >>>>> the
> > >>>>> statsAge field, which seems to be unused by the Engine.
> > >>>>> This
> > >>>>> data
> > >>>>> could be requested every 15 seconds to keep things as
they
> > >>>>> are
> > >>>>> now.
> > >>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed
> > >>>>> Solution
> > >>>>> for
> > >>>>> VDSM & Engine:
> > >>>>>
> > >>>>>
> > >>>>> We will introduce new optional parameters to getVmStats,
> > >>>>> getAllVmStats and list to allow a finer grained
> > >>>>> specification
> > >>>>> of
> > >>>>> data which should be included.
> > >>>>>
> > >>>>> Parameter: statsType = <string> (getVmStats,
getAllVmStats
> > >>>>> only)
> > >>>>> Allowed values:
> > >>>>>
> > >>>>> * full (default to keep backwards compatibility)
> > >>>>> * app-list (Just send the application list)
> > >>>>> * rare (include everything from rarely changed to
very
> > >>>>> frequent)
> > >>>>> * often (include everything from often changed to
very
> > >>>>> frequent)
> > >>>>> * frequent (only send the very frequently changed
> > >>>>> items)
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> Parameter: clientId = <string> The client id is
specified
> > >>>>> by
> > >>>>> the
> > >>>>> client and should be unique however constantly used.
> > >>>>>
> > >>>>> Parameter: diff = <boolean> In combination with the
> > >>>>> clientId
> > >>>>> VDSM
> > >>>>> will send only differences to the previous request from
the
> > >>>>> named
> > >>>>> clientId. (if diff=true)
> > >>>>>
> > >>>>>
> > >>>>> Additional Change:
> > >>>>>
> > >>>>>
> > >>>>> Besides the introduction of the new parameters for list,
> > >>>>> getVmStats
> > >>>>> and getAllVmStats it might make sense to include a hash
for
> > >>>>> the
> > >>>>> appList into the rarely changed section of the response
> > >>>>> which
> > >>>>> would
> > >>>>> allow to identify changes and avoid having to sent the
> > >>>>> complete
> > >>>>> appList every so often and only if the hash known to the
> > >>>>> client
> > >>>>> is
> > >>>>> outdated.
> > >>>>>
> > >>>>> Note: The appList (Application List) reported by the
guest
> > >>>>> agent
> > >>>>> could be fully implemented on request only, as long as
the
> > >>>>> guest
> > >>>>> agent installed supports this. As there seems to be a
> > >>>>> request
> > >>>>> to
> > >>>>> have the complete list of installed applications on all
> > >>>>> guests
> > >>>>> this
> > >>>>> data could be quite extensive and a huge list. On the
other
> > >>>>> hand
> > >>>>> this data is only rarely visible and therefore it should
> > >>>>> not
> > >>>>> be
> > >>>>> requested all the time and only on demand. Improvement of
> > >>>>> the
> > >>>>> Guest
> > >>>>> Agent:
> > >>>>>
> > >>>>>
> > >>>>> As part of the proposed solution it is necessary to
improve
> > >>>>> the
> > >>>>> guest
> > >>>>> agent as well. For the full application list there should
> > >>>>> be
> > >>>>> implemented a caching system which will be fully reactive
> > >>>>> and
> > >>>>> should
> > >>>>> not poll the application list for example all the time.
The
> > >>>>> guest
> > >>>>> can create a prepared data file containing all data in
the
> > >>>>> JSON
> > >>>>> format (as used for the communication with VDSM via VIO)
> > >>>>> and
> > >>>>> just
> > >>>>> have to read that file from disk and directly sends it to
> > >>>>> VDSM.
> > >>>>> However it is quite possible that this list is to big and
> > >>>>> it
> > >>>>> might
> > >>>>> have to be chunked into pieces. (Multiple messages, which
> > >>>>> would
> > >>>>> have
> > >>>>> to be supported by VDSM then as well) The solution for
this
> > >>>>> is
> > >>>>> to
> > >>>>> make VDSM request this data and it will retrieve the data
> > >>>>> necessary
> > >>>>> on request only. --
> > >>>>> Regards,
> > >>>>>
> > >>>>> Vinzenz Feenstra | Senior Software Engineer
> > >>>>> RedHat Engineering Virtualization R & D
> > >>>>> Phone: +420 532 294 625
> > >>>>> IRC: vfeenstr or evilissimo
> > >>>>>
> > >>>>> Better technology. Faster innovation. Powered by
community
> > >>>>> collaboration.
> > >>>>> See how it works at redhat.com
> > >>>>> _______________________________________________
> > >>>>> Engine-devel mailing list
> > >>>>> Engine-devel(a)ovirt.org
> > >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>>>>
> > >>>> _______________________________________________
> > >>>> vdsm-devel mailing list
> > >>>> vdsm-devel(a)lists.fedorahosted.org
> > >>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> > >>>>
> > >>>
> > >>
> > > _______________________________________________
> > > vdsm-devel mailing list
> > > vdsm-devel(a)lists.fedorahosted.org
> > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> > >
> >
> >
> _______________________________________________
> Engine-devel mailing list
> Engine-devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
6:25 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
This is a multi-part message in MIME format.
--------------070408040808030806010106
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
I have left this for a while without continuing because I had to focus
on other things. However this is still in progress :-)
On 03/13/2013 10:55 PM, Ayal Baron wrote:
----- Original Message -----
>
> ----- Original Message -----
>> From: "Ayal Baron" <abaron(a)redhat.com>
>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com>
>> Cc: engine-devel(a)ovirt.org, vdsm-devel(a)lists.fedorahosted.org,
>> "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>> Sent: Wednesday, March 13, 2013 5:39:24 PM
>> Subject: Re: [vdsm] [Engine-devel] Proposal VDSM <=> Engine Data
>> Statistics Retrieval Optimization
>>
>>
>>
>> ----- Original Message -----
>>> I am completely against this.
>>> It make the return value differ according to input which
>>> is a big no no when talking about type safe APIs.
>>>
>>> The only reason we have this problem is because there is this
>>> thing against making multiple calls.
Which is totally contra productive
because multiple calls, if properly
split up, will actually lead to less data sent for frequent needed data
calls. And the others shall be triggered when necessary.
>>>
>>> Just split it up.
>>> getVmRuntimeStats() - transient things like mem and cpu%
>>> getVmInformation() - (semi)static things like disk\networking
>>> layout
>>> etc.
>>> Each updated at different intervals.
>> +1 on splitting the data up into 2 separate API calls.
>> You could potentially add a checksum (md5, or any other way) of the
>> "static" data to getVmRuntimeStats and not bother even with polling
>> the VmInformation if this hasn't changed. Then you could poll as
>> often as you'd like the stats and immediately see if you also need
>> to retrieve VmInfo or not (you rarely would).
> +1 To Ayal's suggestion
> except that instead of the engine hashing the data VDSM sends the
> key which is opaque to the engine.
> This can be a local timestap or a generation number.
Of course vdsm does the hash, otherwise you'd need to pass all the data to engine
which would beat the purpose.
We need the hash if we can't have dynamic content.
Generation numbers
aren't really helpful as every call aggregates the statistics data
newly, at the moment at least.
> But, we might want to consider that when we add events polling
> becomes (much) less frequent so maybe it'll be an overkill.
You'd still need to compare versions of the data in vdsm and send only if it changed.
If you don't persist what was received last then potentially you could have a monday
morning effect where upon on system startup you'd be sending everything. So I still
think you'd want to have the hash.
We do a hash already on the XML and include
it in the getStats response.
Hashes should show enough difference.
Now to the non-dynamic responses and 'type-safe' API: If we would go for
non dynamic responses we would need for sure 5 new API calls to achieve
some gain on the amount of data sent.
*getAllVmRuntimeStats() "returns a map of vmId/data pairs for all vms"*
# All the time changing data which is needed by the oVirt Engine, or so
often changing that it does not make sense
# to place it anywhere else
{
VmId: {
cpuSys --> Could be potentially summarized
cpuUser -/
memUsage
elapsedTime,
status
statsAge
hashes = {
conf, # Hased information of the XML
(This one is called "hash" in getStats())
info, # Hashed information of semi
static items
statusHash: # Hashed information of items with are
likely to change however not that often
guestDetails: # Hashed value of the guest details
(applicationList, network information)
}
}
**getVmStatuses([vmId1, vmId2, ...])*****"Returns a vmId/data pair for
each vm requested"**
*# This data does not change that often and can be retrieved on demand
once the hash changes
return {
vmId: {
timeOffset,
monitorResponse
clientIp,
lastLogin,
username,
session,
guestIPs,
}
}
*getAllVmDeviceStatistics():**"Returns a vmId/data pair for all vms"*
# This data has to be requested all the time however in lower
intervals (e.g. every 5 minutes)
# And is usually needed for all the VMs anyway
return {
vmId: {
network,
disksUsage, # Might be improved by summarizing?
disks,
balloonInfo,
memoryStats
}
}
*getVmInfo([vmId1, vmId2, ...]) "Returns a vmId/data pair for each vm
requested"
* # Basically this should be almost constant, except if there have
been changes like migrations, pausing, errors etc
return {
vmId: {
acpiEnable,
vmType,
guestName,
guestOS,
kvmEnable,
pauseCode,
displayIp,
displayPort,
displaySecurePort,
pid,
}
}
*getVmGuestDetails*([vmId1, vmId2, ...])
# Data which changes seldom and these changes can be reflected in
the hash when this needs to be requested
# This data is really only necessary when it really has been
changed or needs to be refreshed for whatever reason.
return {
vmId: {
appsList,
netIfaces,
}
}
>>> ----- Original Message -----
>>>> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>>> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
>>>> Statistics
>>>> Retrieval Optimization
>>>>
>>>>
>>>> Please find the prettier version on the wiki:
>>>> http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>>>
>>>> Proposal VDSM - Engine Data Statistics Retrieval
>>>> VDSM <=> Engine data retrieval optimization
>>>> Motivation:
>>>>
>>>>
>>>> Currently the RHEVM engine is polling the a lot of data from
>>>> VDSM
>>>> every 15 seconds. This should be optimized and the amount of
>>>> data
>>>> requested should be more specific.
>>>>
>>>> For each VM the data currently contains much more information
>>>> than
>>>> actually needed which blows up the size of the XML content
>>>> quite
>>>> big. We could optimize this by splitting the reply on the
>>>> getVmStats
>>>> based on the request of the engine into sections. For this
>>>> reason
>>>> Omer Frenkel and me have split up the data into parts based on
>>>> their
>>>> usage.
>>>>
>>>> This data can and usually does change during the lifetime of
>>>> the
>>>> VM.
>>>> Rarely Changed:
>>>>
>>>>
>>>> This data is change not very frequent and it should be enough
>>>> to
>>>> update this only once in a while. Most commonly this data
>>>> changes
>>>> after changes made in the UI or after a migration of the VM to
>>>> another Host. Status = Running acpiEnable = true vmType = kvm
>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
>>>> kvmEnable = true # this should be constant and never changed
>>>> pauseCode = NOERR monitorResponse = 0 session = Locked # unused
>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC',
>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
>>>> ['10.34.60.148'],
>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools
3.2.4',
>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
>>>> 3.2.2',
>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
>>>> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2',
'RHEV-USB
>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
pid =
>>>> 11314
>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
>>>> displayPort
>>>> = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
>>>>
>>>>
>>>> This data is changed quite often however it is not necessary to
>>>> update this data every 15 seconds. As this is cumulative data
>>>> and
>>>> reflects the current status, and it does not need to be
>>>> snapshotted
>>>> every 15 seconds to retrieve statistics. The data can be
>>>> retrieved
>>>> in much more generous time slices. (e.g. Every 5 minutes)
>>>> network
>>>> =
>>>> {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0',
>>>> 'txDropped': '0', 'rxErrors': '0',
'txRate': '0.0', 'rxRate':
>>>> '0.0',
>>>> 'txErrors': '0', 'state': 'unknown',
'speed': '100', 'name':
>>>> 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total':
>>>> '64055406592',
>>>> 'fs': 'NTFS', 'used': '19223846912'},
{'path': 'd:\\', 'total':
>>>> '3490912256', 'fs': 'UDF', 'used':
'3490912256'}] timeOffset =
>>>> 14422
>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
>>>> #
>>>> unused Often Changed but unused
>>>>
>>>>
>>>> This data does not seem to be used in the engine at all. It is
>>>> not
>>>> even used in the data warehouse. memoryStats = {'swap_out':
>>>> '0',
>>>> 'majflt': '0', 'mem_free': '1466884',
'swap_in': '0',
>>>> 'pageflt':
>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
>>>> balloonInfo
>>>> =
>>>> {'balloon_max': 2097152, 'balloon_cur': 2097152} disks =
>>>> {'vda':
>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
>>>> 'writeLatency':
>>>> '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
>>>> 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
>>>> '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
>>>> '0',
>>>> 'apparentsize': '0', 'writeLatency': '0',
'flushLatency': '0',
>>>> 'readRate': '0.00', 'truesize': '0',
'writeRate': '0.00'}} Very
>>>> frequent uppdates needed by webadmin portal:
>>>>
>>>>
>>>> This data is mostly needed for the webadmin portal and might be
>>>> required to be updated quite often. An exception here is the
>>>> statsAge field, which seems to be unused by the Engine. This
>>>> data
>>>> could be requested every 15 seconds to keep things as they are
>>>> now.
>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
>>>> for
>>>> VDSM & Engine:
>>>>
>>>>
>>>> We will introduce new optional parameters to getVmStats,
>>>> getAllVmStats and list to allow a finer grained specification
>>>> of
>>>> data which should be included.
>>>>
>>>> Parameter: statsType = <string> (getVmStats, getAllVmStats
>>>> only)
>>>> Allowed values:
>>>>
>>>> * full (default to keep backwards compatibility)
>>>> * app-list (Just send the application list)
>>>> * rare (include everything from rarely changed to very
>>>> frequent)
>>>> * often (include everything from often changed to very
>>>> frequent)
>>>> * frequent (only send the very frequently changed items)
>>>>
>>>>
>>>>
>>>> Parameter: clientId = <string> The client id is specified by
>>>> the
>>>> client and should be unique however constantly used.
>>>>
>>>> Parameter: diff = <boolean> In combination with the clientId
>>>> VDSM
>>>> will send only differences to the previous request from the
>>>> named
>>>> clientId. (if diff=true)
>>>>
>>>>
>>>> Additional Change:
>>>>
>>>>
>>>> Besides the introduction of the new parameters for list,
>>>> getVmStats
>>>> and getAllVmStats it might make sense to include a hash for the
>>>> appList into the rarely changed section of the response which
>>>> would
>>>> allow to identify changes and avoid having to sent the complete
>>>> appList every so often and only if the hash known to the client
>>>> is
>>>> outdated.
>>>>
>>>> Note: The appList (Application List) reported by the guest
>>>> agent
>>>> could be fully implemented on request only, as long as the
>>>> guest
>>>> agent installed supports this. As there seems to be a request
>>>> to
>>>> have the complete list of installed applications on all guests
>>>> this
>>>> data could be quite extensive and a huge list. On the other
>>>> hand
>>>> this data is only rarely visible and therefore it should not be
>>>> requested all the time and only on demand. Improvement of the
>>>> Guest
>>>> Agent:
>>>>
>>>>
>>>> As part of the proposed solution it is necessary to improve the
>>>> guest
>>>> agent as well. For the full application list there should be
>>>> implemented a caching system which will be fully reactive and
>>>> should
>>>> not poll the application list for example all the time. The
>>>> guest
>>>> can create a prepared data file containing all data in the JSON
>>>> format (as used for the communication with VDSM via VIO) and
>>>> just
>>>> have to read that file from disk and directly sends it to VDSM.
>>>> However it is quite possible that this list is to big and it
>>>> might
>>>> have to be chunked into pieces. (Multiple messages, which would
>>>> have
>>>> to be supported by VDSM then as well) The solution for this is
>>>> to
>>>> make VDSM request this data and it will retrieve the data
>>>> necessary
>>>> on request only. --
>>>> Regards,
>>>>
>>>> Vinzenz Feenstra | Senior Software Engineer
>>>> RedHat Engineering Virtualization R & D
>>>> Phone: +420 532 294 625
>>>> IRC: vfeenstr or evilissimo
>>>>
>>>> Better technology. Faster innovation. Powered by community
>>>> collaboration.
>>>> See how it works at redhat.com
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>> _______________________________________________
>>> vdsm-devel mailing list
>>> vdsm-devel(a)lists.fedorahosted.org
>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>
--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
--------------070408040808030806010106
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">I have left this for a while without
continuing because I had to focus on other things. However this is
still in progress :-)<br>
On 03/13/2013 10:55 PM, Ayal Baron wrote:<br>
</div>
<blockquote
cite="mid:25893083.7640290.1363211722007.JavaMail.root@redhat.com"
type="cite">
<pre wrap="">
----- Original Message -----
</pre>
<blockquote type="cite">
<pre wrap="">
----- Original Message -----
</pre>
<blockquote type="cite">
<pre wrap="">From: "Ayal Baron" <a
class="moz-txt-link-rfc2396E"
href="mailto:abaron@redhat.com"><abaron@redhat.com></a>
To: "Saggi Mizrahi" <a class="moz-txt-link-rfc2396E"
href="mailto:smizrahi@redhat.com"><smizrahi@redhat.com></a>
Cc: <a class="moz-txt-link-abbreviated"
href="mailto:engine-devel@ovirt.org">engine-devel@ovirt.org</a>, <a
class="moz-txt-link-abbreviated"
href="mailto:vdsm-devel@lists.fedorahosted.org">vdsm-devel@lists.fedorahosted.org</a>,
"Vinzenz Feenstra" <a class="moz-txt-link-rfc2396E"
href="mailto:vfeenstr@redhat.com"><vfeenstr@redhat.com></a>
Sent: Wednesday, March 13, 2013 5:39:24 PM
Subject: Re: [vdsm] [Engine-devel] Proposal VDSM <=> Engine Data
Statistics Retrieval Optimization
----- Original Message -----
</pre>
<blockquote type="cite">
<pre wrap="">I am completely against this.
It make the return value differ according to input which
is a big no no when talking about type safe APIs.
The only reason we have this problem is because there is this
thing against making multiple calls.</pre>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
Which is totally contra productive because multiple calls, if
properly split up, will actually lead to less data sent for frequent
needed data calls. And the others shall be triggered when necessary.<br>
<blockquote
cite="mid:25893083.7640290.1363211722007.JavaMail.root@redhat.com"
type="cite">
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">
Just split it up.
getVmRuntimeStats() - transient things like mem and cpu%
getVmInformation() - (semi)static things like disk\networking
layout
etc.
Each updated at different intervals.
</pre>
</blockquote>
<pre wrap="">
+1 on splitting the data up into 2 separate API calls.
You could potentially add a checksum (md5, or any other way) of the
"static" data to getVmRuntimeStats and not bother even with polling
the VmInformation if this hasn't changed. Then you could poll as
often as you'd like the stats and immediately see if you also need
to retrieve VmInfo or not (you rarely would).
</pre>
</blockquote>
<pre wrap="">+1 To Ayal's suggestion
except that instead of the engine hashing the data VDSM sends the
key which is opaque to the engine.
This can be a local timestap or a generation number.
</pre>
</blockquote>
<pre wrap="">
Of course vdsm does the hash, otherwise you'd need to pass all the data to engine
which would beat the purpose.</pre>
</blockquote>
We need the hash if we can't have dynamic content. Generation
numbers aren't really helpful as every call aggregates the
statistics data newly, at the moment at least.
<blockquote
cite="mid:25893083.7640290.1363211722007.JavaMail.root@redhat.com"
type="cite">
<blockquote type="cite">
<pre wrap="">
But, we might want to consider that when we add events polling
becomes (much) less frequent so maybe it'll be an overkill.
</pre>
</blockquote>
<pre wrap="">
You'd still need to compare versions of the data in vdsm and send only if it changed.
If you don't persist what was received last then potentially you could have a monday
morning effect where upon on system startup you'd be sending everything. So I still
think you'd want to have the hash.</pre>
</blockquote>
We do a hash already on the XML and include it in the getStats
response.<br>
Hashes should show enough difference.<br>
<br>
Now to the non-dynamic responses and 'type-safe' API: If we would go
for non dynamic responses we would need for sure 5 new API calls to
achieve some gain on the amount of data sent.<br>
<br>
<b>getAllVmRuntimeStats() "returns a map of vmId/data pairs for all
vms"</b><br>
# All the time changing data which is needed by the oVirt Engine, or
so often changing that it does not make sense<br>
# to place it anywhere else<br>
{<br>
VmId: {<br>
cpuSys --> Could be potentially summarized<br>
cpuUser -/<br>
memUsage<br>
elapsedTime,<br>
status<br>
statsAge<br>
<br>
hashes = {<br>
conf, # Hased information of the XML
(This one is called "hash" in getStats())<br>
info, # Hashed information of semi
static items<br>
statusHash: # Hashed information of items with
are likely to change however not that often<br>
guestDetails: # Hashed value of the guest details
(applicationList, network information)<br>
}<br>
}<br>
<br>
<b><b>getVmStatuses([vmId1, vmId2,
...])</b></b><b><b><b> "Returns a
vmId/data pair for each vm requested"</b></b><br>
</b># This data does not change that often and can be
retrieved on demand once the hash changes<br>
return {<br>
vmId: {<br>
timeOffset,<br>
monitorResponse<br>
clientIp,<br>
lastLogin,<br>
username,<br>
session,<br>
guestIPs,<br>
}<br>
}<br>
<br>
<b>getAllVmDeviceStatistics():</b><b> "Returns a vmId/data pair
for
all vms"</b><br>
# This data has to be requested all the time however in lower
intervals (e.g. every 5 minutes)<br>
# And is usually needed for all the VMs anyway<br>
return {<br>
vmId: {<br>
network,<br>
disksUsage, # Might be improved by summarizing?<br>
disks,<br>
balloonInfo,<br>
memoryStats<br>
}<br>
}<br>
<br>
<b>getVmInfo([vmId1, vmId2, ...]) "Returns a vmId/data pair for each
vm requested"<br>
</b> # Basically this should be almost constant, except if there
have been changes like migrations, pausing, errors etc<br>
return {<br>
vmId: {<br>
acpiEnable,<br>
vmType,<br>
guestName,<br>
guestOS,<br>
kvmEnable,<br>
pauseCode,<br>
displayIp,<br>
displayPort,<br>
displaySecurePort,<br>
pid,<br>
}<br>
}<br>
<br>
<b>getVmGuestDetails</b>([vmId1, vmId2, ...])<br>
# Data which changes seldom and these changes can be reflected
in the hash when this needs to be requested<br>
# This data is really only necessary when it really has been
changed or needs to be refreshed for whatever reason. <br>
return {<br>
vmId: {<br>
appsList,<br>
netIfaces,<br>
}<br>
}<br>
<br>
<blockquote
cite="mid:25893083.7640290.1363211722007.JavaMail.root@redhat.com"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">
----- Original Message -----
</pre>
<blockquote type="cite">
<pre wrap="">From: "Vinzenz Feenstra" <a
class="moz-txt-link-rfc2396E"
href="mailto:vfeenstr@redhat.com"><vfeenstr@redhat.com></a>
To: <a class="moz-txt-link-abbreviated"
href="mailto:vdsm-devel@lists.fedorahosted.org">vdsm-devel@lists.fedorahosted.org</a>,
<a class="moz-txt-link-abbreviated"
href="mailto:engine-devel@ovirt.org">engine-devel@ovirt.org</a>
Sent: Thursday, March 7, 2013 6:25:54 AM
Subject: [Engine-devel] Proposal VDSM <=> Engine Data
Statistics
Retrieval Optimization
Please find the prettier version on the wiki:
<a class="moz-txt-link-freetext"
href="http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Re...
Proposal VDSM - Engine Data Statistics Retrieval
VDSM <=> Engine data retrieval optimization
Motivation:
Currently the RHEVM engine is polling the a lot of data from
VDSM
every 15 seconds. This should be optimized and the amount of
data
requested should be more specific.
For each VM the data currently contains much more information
than
actually needed which blows up the size of the XML content
quite
big. We could optimize this by splitting the reply on the
getVmStats
based on the request of the engine into sections. For this
reason
Omer Frenkel and me have split up the data into parts based on
their
usage.
This data can and usually does change during the lifetime of
the
VM.
Rarely Changed:
This data is change not very frequent and it should be enough
to
update this only once in a while. Most commonly this data
changes
after changes made in the UI or after a migration of the VM to
another Host. Status = Running acpiEnable = true vmType = kvm
guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
kvmEnable = true # this should be constant and never changed
pauseCode = NOERR monitorResponse = 0 session = Locked # unused
netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC',
'inet6': ['fe80::490c:92bb:bbcc:9f87'], 'inet':
['10.34.60.148'],
'hw': '00:1a:4a:22:3c:db'}] appsList = ['RHEV-Tools 3.2.4',
'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3', 'RHEV-Network64
3.2.2',
'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 'RHEV-Balloon64
3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 'RHEV-USB
3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2'] pid =
11314
guestIPs = 10.34.60.148 # duplicated info displayIp = 0
displayPort
= 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
clientIp = lastLogin = 1361976900.67 Often Changed:
This data is changed quite often however it is not necessary to
update this data every 15 seconds. As this is cumulative data
and
reflects the current status, and it does not need to be
snapshotted
every 15 seconds to retrieve statistics. The data can be
retrieved
in much more generous time slices. (e.g. Every 5 minutes)
network
=
{'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped':
'0',
'txDropped': '0', 'rxErrors': '0', 'txRate':
'0.0', 'rxRate':
'0.0',
'txErrors': '0', 'state': 'unknown', 'speed':
'100', 'name':
'vnet1'}} disksUsage = [{'path': 'c:\\', 'total':
'64055406592',
'fs': 'NTFS', 'used': '19223846912'}, {'path':
'd:\\', 'total':
'3490912256', 'fs': 'UDF', 'used': '3490912256'}]
timeOffset =
14422
elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
#
unused Often Changed but unused
This data does not seem to be used in the engine at all. It is
not
even used in the data warehouse. memoryStats = {'swap_out':
'0',
'majflt': '0', 'mem_free': '1466884', 'swap_in':
'0',
'pageflt':
'0', 'mem_total': '2096736', 'mem_unused':
'1466884'}
balloonInfo
=
{'balloon_max': 2097152, 'balloon_cur': 2097152} disks =
{'vda':
{'readLatency': '0', 'apparentsize': '64424509440',
'writeLatency':
'1754496', 'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize':
'18855059456', 'writeRate': '952.05'}, 'hdc':
{'readLatency':
'0',
'apparentsize': '0', 'writeLatency': '0',
'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}} Very
frequent uppdates needed by webadmin portal:
This data is mostly needed for the webadmin portal and might be
required to be updated quite often. An exception here is the
statsAge field, which seems to be unused by the Engine. This
data
could be requested every 15 seconds to keep things as they are
now.
cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
for
VDSM & Engine:
We will introduce new optional parameters to getVmStats,
getAllVmStats and list to allow a finer grained specification
of
data which should be included.
Parameter: statsType = <string> (getVmStats, getAllVmStats
only)
Allowed values:
* full (default to keep backwards compatibility)
* app-list (Just send the application list)
* rare (include everything from rarely changed to very
frequent)
* often (include everything from often changed to very
frequent)
* frequent (only send the very frequently changed items)
Parameter: clientId = <string> The client id is specified by
the
client and should be unique however constantly used.
Parameter: diff = <boolean> In combination with the clientId
VDSM
will send only differences to the previous request from the
named
clientId. (if diff=true)
Additional Change:
Besides the introduction of the new parameters for list,
getVmStats
and getAllVmStats it might make sense to include a hash for the
appList into the rarely changed section of the response which
would
allow to identify changes and avoid having to sent the complete
appList every so often and only if the hash known to the client
is
outdated.
Note: The appList (Application List) reported by the guest
agent
could be fully implemented on request only, as long as the
guest
agent installed supports this. As there seems to be a request
to
have the complete list of installed applications on all guests
this
data could be quite extensive and a huge list. On the other
hand
this data is only rarely visible and therefore it should not be
requested all the time and only on demand. Improvement of the
Guest
Agent:
As part of the proposed solution it is necessary to improve the
guest
agent as well. For the full application list there should be
implemented a caching system which will be fully reactive and
should
not poll the application list for example all the time. The
guest
can create a prepared data file containing all data in the JSON
format (as used for the communication with VDSM via VIO) and
just
have to read that file from disk and directly sends it to VDSM.
However it is quite possible that this list is to big and it
might
have to be chunked into pieces. (Multiple messages, which would
have
to be supported by VDSM then as well) The solution for this is
to
make VDSM request this data and it will retrieve the data
necessary
on request only. --
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community
collaboration.
See how it works at redhat.com
_______________________________________________
Engine-devel mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Engine-devel@ovirt.org">Engine-devel@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/engine-devel">...
</pre>
</blockquote>
<pre wrap="">_______________________________________________
vdsm-devel mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:vdsm-devel@lists.fedorahosted.org">vdsm-devel@lists.fedorahosted.org</a>
<a class="moz-txt-link-freetext"
href="https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel&quo...
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com</pre>
</body>
</html>
--------------070408040808030806010106--
11:39 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
----- Original Message -----
I have left this for a while without continuing because I had to
focus
on other things. However this is still in progress :-)
Are you writing patches? (if so, what solution are you pursuing)
On 03/13/2013 10:55 PM, Ayal Baron wrote:
>
> ----- Original Message -----
>>
>> ----- Original Message -----
>>> From: "Ayal Baron" <abaron(a)redhat.com>
>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com>
>>> Cc: engine-devel(a)ovirt.org, vdsm-devel(a)lists.fedorahosted.org,
>>> "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>> Sent: Wednesday, March 13, 2013 5:39:24 PM
>>> Subject: Re: [vdsm] [Engine-devel] Proposal VDSM <=> Engine Data
>>> Statistics Retrieval Optimization
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> I am completely against this.
>>>> It make the return value differ according to input which
>>>> is a big no no when talking about type safe APIs.
>>>>
>>>> The only reason we have this problem is because there is this
>>>> thing against making multiple calls.
Which is totally contra productive because multiple calls, if properly
split up, will actually lead to less data sent for frequent needed data
calls. And the others shall be triggered when necessary.
>>>>
>>>> Just split it up.
>>>> getVmRuntimeStats() - transient things like mem and cpu%
>>>> getVmInformation() - (semi)static things like disk\networking
>>>> layout
>>>> etc.
>>>> Each updated at different intervals.
>>> +1 on splitting the data up into 2 separate API calls.
>>> You could potentially add a checksum (md5, or any other way) of the
>>> "static" data to getVmRuntimeStats and not bother even with
polling
>>> the VmInformation if this hasn't changed. Then you could poll as
>>> often as you'd like the stats and immediately see if you also need
>>> to retrieve VmInfo or not (you rarely would).
>> +1 To Ayal's suggestion
>> except that instead of the engine hashing the data VDSM sends the
>> key which is opaque to the engine.
>> This can be a local timestap or a generation number.
> Of course vdsm does the hash, otherwise you'd need to pass all the data to
> engine which would beat the purpose.
We need the hash if we can't have dynamic content. Generation numbers
aren't really helpful as every call aggregates the statistics data
newly, at the moment at least.
>> But, we might want to consider that when we add events polling
>> becomes (much) less frequent so maybe it'll be an overkill.
> You'd still need to compare versions of the data in vdsm and send only if
> it changed. If you don't persist what was received last then potentially
> you could have a monday morning effect where upon on system startup you'd
> be sending everything. So I still think you'd want to have the hash.
We do a hash already on the XML and include it in the getStats response.
Hashes should show enough difference.
Now to the non-dynamic responses and 'type-safe' API: If we would go for
non dynamic responses we would need for sure 5 new API calls to achieve
some gain on the amount of data sent.
*getAllVmRuntimeStats() "returns a map of vmId/data pairs for all vms"*
# All the time changing data which is needed by the oVirt Engine, or so
often changing that it does not make sense
# to place it anywhere else
{
VmId: {
cpuSys --> Could be potentially summarized
cpuUser -/
memUsage
elapsedTime,
status
statsAge
hashes = {
conf, # Hased information of the XML
(This one is called "hash" in getStats())
info, # Hashed information of semi
static items
statusHash: # Hashed information of items with are
likely to change however not that often
guestDetails: # Hashed value of the guest details
(applicationList, network information)
}
}
**getVmStatuses([vmId1, vmId2, ...])*****"Returns a vmId/data pair for
each vm requested"**
*# This data does not change that often and can be retrieved on demand
once the hash changes
return {
vmId: {
timeOffset,
monitorResponse
clientIp,
lastLogin,
username,
session,
guestIPs,
}
}
*getAllVmDeviceStatistics():**"Returns a vmId/data pair for all vms"*
# This data has to be requested all the time however in lower
intervals (e.g. every 5 minutes)
# And is usually needed for all the VMs anyway
return {
vmId: {
network,
disksUsage, # Might be improved by summarizing?
disks,
balloonInfo,
memoryStats
}
}
*getVmInfo([vmId1, vmId2, ...]) "Returns a vmId/data pair for each vm
requested"
* # Basically this should be almost constant, except if there have
been changes like migrations, pausing, errors etc
return {
vmId: {
acpiEnable,
vmType,
guestName,
guestOS,
kvmEnable,
pauseCode,
displayIp,
displayPort,
displaySecurePort,
pid,
}
}
*getVmGuestDetails*([vmId1, vmId2, ...])
# Data which changes seldom and these changes can be reflected in
the hash when this needs to be requested
# This data is really only necessary when it really has been
changed or needs to be refreshed for whatever reason.
return {
vmId: {
appsList,
netIfaces,
}
}
>
>
>>>> ----- Original Message -----
>>>>> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>>>> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
>>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
>>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
>>>>> Statistics
>>>>> Retrieval Optimization
>>>>>
>>>>>
>>>>> Please find the prettier version on the wiki:
>>>>>
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>>>>
>>>>> Proposal VDSM - Engine Data Statistics Retrieval
>>>>> VDSM <=> Engine data retrieval optimization
>>>>> Motivation:
>>>>>
>>>>>
>>>>> Currently the RHEVM engine is polling the a lot of data from
>>>>> VDSM
>>>>> every 15 seconds. This should be optimized and the amount of
>>>>> data
>>>>> requested should be more specific.
>>>>>
>>>>> For each VM the data currently contains much more information
>>>>> than
>>>>> actually needed which blows up the size of the XML content
>>>>> quite
>>>>> big. We could optimize this by splitting the reply on the
>>>>> getVmStats
>>>>> based on the request of the engine into sections. For this
>>>>> reason
>>>>> Omer Frenkel and me have split up the data into parts based on
>>>>> their
>>>>> usage.
>>>>>
>>>>> This data can and usually does change during the lifetime of
>>>>> the
>>>>> VM.
>>>>> Rarely Changed:
>>>>>
>>>>>
>>>>> This data is change not very frequent and it should be enough
>>>>> to
>>>>> update this only once in a while. Most commonly this data
>>>>> changes
>>>>> after changes made in the UI or after a migration of the VM to
>>>>> another Host. Status = Running acpiEnable = true vmType = kvm
>>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
>>>>> kvmEnable = true # this should be constant and never changed
>>>>> pauseCode = NOERR monitorResponse = 0 session = Locked # unused
>>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast Ethernet
NIC',
>>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'],
'inet':
>>>>> ['10.34.60.148'],
>>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList =
['RHEV-Tools 3.2.4',
>>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
>>>>> 3.2.2',
>>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
>>>>> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64
3.2.2', 'RHEV-USB
>>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64
3.2.2'] pid =
>>>>> 11314
>>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
>>>>> displayPort
>>>>> = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
>>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
>>>>>
>>>>>
>>>>> This data is changed quite often however it is not necessary to
>>>>> update this data every 15 seconds. As this is cumulative data
>>>>> and
>>>>> reflects the current status, and it does not need to be
>>>>> snapshotted
>>>>> every 15 seconds to retrieve statistics. The data can be
>>>>> retrieved
>>>>> in much more generous time slices. (e.g. Every 5 minutes)
>>>>> network
>>>>> =
>>>>> {'vnet1': {'macAddr': '00:1a:4a:22:3c:db',
'rxDropped': '0',
>>>>> 'txDropped': '0', 'rxErrors': '0',
'txRate': '0.0', 'rxRate':
>>>>> '0.0',
>>>>> 'txErrors': '0', 'state': 'unknown',
'speed': '100', 'name':
>>>>> 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total':
>>>>> '64055406592',
>>>>> 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
>>>>> '3490912256', 'fs': 'UDF', 'used':
'3490912256'}] timeOffset =
>>>>> 14422
>>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
>>>>> #
>>>>> unused Often Changed but unused
>>>>>
>>>>>
>>>>> This data does not seem to be used in the engine at all. It is
>>>>> not
>>>>> even used in the data warehouse. memoryStats = {'swap_out':
>>>>> '0',
>>>>> 'majflt': '0', 'mem_free':
'1466884', 'swap_in': '0',
>>>>> 'pageflt':
>>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
>>>>> balloonInfo
>>>>> =
>>>>> {'balloon_max': 2097152, 'balloon_cur': 2097152}
disks =
>>>>> {'vda':
>>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
>>>>> 'writeLatency':
>>>>> '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
>>>>> 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
>>>>> '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
>>>>> '0',
>>>>> 'apparentsize': '0', 'writeLatency':
'0', 'flushLatency': '0',
>>>>> 'readRate': '0.00', 'truesize': '0',
'writeRate': '0.00'}} Very
>>>>> frequent uppdates needed by webadmin portal:
>>>>>
>>>>>
>>>>> This data is mostly needed for the webadmin portal and might be
>>>>> required to be updated quite often. An exception here is the
>>>>> statsAge field, which seems to be unused by the Engine. This
>>>>> data
>>>>> could be requested every 15 seconds to keep things as they are
>>>>> now.
>>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
>>>>> for
>>>>> VDSM & Engine:
>>>>>
>>>>>
>>>>> We will introduce new optional parameters to getVmStats,
>>>>> getAllVmStats and list to allow a finer grained specification
>>>>> of
>>>>> data which should be included.
>>>>>
>>>>> Parameter: statsType = <string> (getVmStats, getAllVmStats
>>>>> only)
>>>>> Allowed values:
>>>>>
>>>>> * full (default to keep backwards compatibility)
>>>>> * app-list (Just send the application list)
>>>>> * rare (include everything from rarely changed to very
>>>>> frequent)
>>>>> * often (include everything from often changed to very
>>>>> frequent)
>>>>> * frequent (only send the very frequently changed items)
>>>>>
>>>>>
>>>>>
>>>>> Parameter: clientId = <string> The client id is specified by
>>>>> the
>>>>> client and should be unique however constantly used.
>>>>>
>>>>> Parameter: diff = <boolean> In combination with the clientId
>>>>> VDSM
>>>>> will send only differences to the previous request from the
>>>>> named
>>>>> clientId. (if diff=true)
>>>>>
>>>>>
>>>>> Additional Change:
>>>>>
>>>>>
>>>>> Besides the introduction of the new parameters for list,
>>>>> getVmStats
>>>>> and getAllVmStats it might make sense to include a hash for the
>>>>> appList into the rarely changed section of the response which
>>>>> would
>>>>> allow to identify changes and avoid having to sent the complete
>>>>> appList every so often and only if the hash known to the client
>>>>> is
>>>>> outdated.
>>>>>
>>>>> Note: The appList (Application List) reported by the guest
>>>>> agent
>>>>> could be fully implemented on request only, as long as the
>>>>> guest
>>>>> agent installed supports this. As there seems to be a request
>>>>> to
>>>>> have the complete list of installed applications on all guests
>>>>> this
>>>>> data could be quite extensive and a huge list. On the other
>>>>> hand
>>>>> this data is only rarely visible and therefore it should not be
>>>>> requested all the time and only on demand. Improvement of the
>>>>> Guest
>>>>> Agent:
>>>>>
>>>>>
>>>>> As part of the proposed solution it is necessary to improve the
>>>>> guest
>>>>> agent as well. For the full application list there should be
>>>>> implemented a caching system which will be fully reactive and
>>>>> should
>>>>> not poll the application list for example all the time. The
>>>>> guest
>>>>> can create a prepared data file containing all data in the JSON
>>>>> format (as used for the communication with VDSM via VIO) and
>>>>> just
>>>>> have to read that file from disk and directly sends it to VDSM.
>>>>> However it is quite possible that this list is to big and it
>>>>> might
>>>>> have to be chunked into pieces. (Multiple messages, which would
>>>>> have
>>>>> to be supported by VDSM then as well) The solution for this is
>>>>> to
>>>>> make VDSM request this data and it will retrieve the data
>>>>> necessary
>>>>> on request only. --
>>>>> Regards,
>>>>>
>>>>> Vinzenz Feenstra | Senior Software Engineer
>>>>> RedHat Engineering Virtualization R & D
>>>>> Phone: +420 532 294 625
>>>>> IRC: vfeenstr or evilissimo
>>>>>
>>>>> Better technology. Faster innovation. Powered by community
>>>>> collaboration.
>>>>> See how it works at redhat.com
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>>> _______________________________________________
>>>> vdsm-devel mailing list
>>>> vdsm-devel(a)lists.fedorahosted.org
>>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>
--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
8:16 a.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On 04/22/2013 10:39 PM, Ayal Baron wrote:
----- Original Message -----
> I have left this for a while without continuing because I had to focus
> on other things. However this is still in progress :-)
Are you writing patches? (if so, what solution are you pursuing)
I haven't
started writing patches for this particular thing yet, however
I want to, but before I want to know that the approach will be
acceptable and so I won't have to throw away work because it's not
accepted by the project that way.
My ultimate goal is to reduce the overall traffic and the amount of data
the engine has to parse on a per Host basis.
This can be achieved without having push notifications working already.
Imagine that oVirt engine currently has to parse for 1000 VMs running,
~16MiB of XML data and that every 15 seconds. And that's just the result
from getAllVmStats.
Additionally it's polling every 2-3 seconds 'list' to retrieve the VmIds
and statuses.
Now I have split the values which the engine would return in getVmStats
into groups based on how likely the are to change AND their size.
getAllVmRuntimeStats() for example, returns the hashes to the different
parts where it makes sense, however it won't give you the devices for
example. Because the engine still has to poll that every 5 minutes or so
to retrieve stats for the the data warehouse.
I am personally for having push notifications properly implemented and
working however the base of this idea is reducing parser and validation
work and will help the engine backend to scale better. Especially it
should theoretically reduce the load on the database as well, at least
theoretically (I have no numbers for this one ;-))
> On 03/13/2013 10:55 PM, Ayal Baron wrote:
>> ----- Original Message -----
>>> ----- Original Message -----
>>>> From: "Ayal Baron" <abaron(a)redhat.com>
>>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com>
>>>> Cc: engine-devel(a)ovirt.org, vdsm-devel(a)lists.fedorahosted.org,
>>>> "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>>> Sent: Wednesday, March 13, 2013 5:39:24 PM
>>>> Subject: Re: [vdsm] [Engine-devel] Proposal VDSM <=> Engine Data
>>>> Statistics Retrieval Optimization
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> I am completely against this.
>>>>> It make the return value differ according to input which
>>>>> is a big no no when talking about type safe APIs.
>>>>>
>>>>> The only reason we have this problem is because there is this
>>>>> thing against making multiple calls.
> Which is totally contra productive because multiple calls, if properly
> split up, will actually lead to less data sent for frequent needed data
> calls. And the others shall be triggered when necessary.
>>>>> Just split it up.
>>>>> getVmRuntimeStats() - transient things like mem and cpu%
>>>>> getVmInformation() - (semi)static things like disk\networking
>>>>> layout
>>>>> etc.
>>>>> Each updated at different intervals.
>>>> +1 on splitting the data up into 2 separate API calls.
>>>> You could potentially add a checksum (md5, or any other way) of the
>>>> "static" data to getVmRuntimeStats and not bother even with
polling
>>>> the VmInformation if this hasn't changed. Then you could poll as
>>>> often as you'd like the stats and immediately see if you also need
>>>> to retrieve VmInfo or not (you rarely would).
>>> +1 To Ayal's suggestion
>>> except that instead of the engine hashing the data VDSM sends the
>>> key which is opaque to the engine.
>>> This can be a local timestap or a generation number.
>> Of course vdsm does the hash, otherwise you'd need to pass all the data to
>> engine which would beat the purpose.
> We need the hash if we can't have dynamic content. Generation numbers
> aren't really helpful as every call aggregates the statistics data
> newly, at the moment at least.
>>> But, we might want to consider that when we add events polling
>>> becomes (much) less frequent so maybe it'll be an overkill.
>> You'd still need to compare versions of the data in vdsm and send only if
>> it changed. If you don't persist what was received last then potentially
>> you could have a monday morning effect where upon on system startup you'd
>> be sending everything. So I still think you'd want to have the hash.
> We do a hash already on the XML and include it in the getStats response.
> Hashes should show enough difference.
>
> Now to the non-dynamic responses and 'type-safe' API: If we would go for
> non dynamic responses we would need for sure 5 new API calls to achieve
> some gain on the amount of data sent.
>
> *getAllVmRuntimeStats() "returns a map of vmId/data pairs for all vms"*
> # All the time changing data which is needed by the oVirt Engine, or so
> often changing that it does not make sense
> # to place it anywhere else
> {
> VmId: {
> cpuSys --> Could be potentially summarized
> cpuUser -/
> memUsage
> elapsedTime,
> status
> statsAge
>
> hashes = {
> conf, # Hased information of the XML
> (This one is called "hash" in getStats())
> info, # Hashed information of semi
> static items
> statusHash: # Hashed information of items with are
> likely to change however not that often
> guestDetails: # Hashed value of the guest details
> (applicationList, network information)
> }
> }
>
> **getVmStatuses([vmId1, vmId2, ...])*****"Returns a vmId/data pair for
> each vm requested"**
> *# This data does not change that often and can be retrieved on demand
> once the hash changes
> return {
> vmId: {
> timeOffset,
> monitorResponse
> clientIp,
> lastLogin,
> username,
> session,
> guestIPs,
> }
> }
>
> *getAllVmDeviceStatistics():**"Returns a vmId/data pair for all vms"*
> # This data has to be requested all the time however in lower
> intervals (e.g. every 5 minutes)
> # And is usually needed for all the VMs anyway
> return {
> vmId: {
> network,
> disksUsage, # Might be improved by summarizing?
> disks,
> balloonInfo,
> memoryStats
> }
> }
>
> *getVmInfo([vmId1, vmId2, ...]) "Returns a vmId/data pair for each vm
> requested"
> * # Basically this should be almost constant, except if there have
> been changes like migrations, pausing, errors etc
> return {
> vmId: {
> acpiEnable,
> vmType,
> guestName,
> guestOS,
> kvmEnable,
> pauseCode,
> displayIp,
> displayPort,
> displaySecurePort,
> pid,
> }
> }
>
> *getVmGuestDetails*([vmId1, vmId2, ...])
> # Data which changes seldom and these changes can be reflected in
> the hash when this needs to be requested
> # This data is really only necessary when it really has been
> changed or needs to be refreshed for whatever reason.
> return {
> vmId: {
> appsList,
> netIfaces,
> }
> }
>
>>
>>>>> ----- Original Message -----
>>>>>> From: "Vinzenz Feenstra" <vfeenstr(a)redhat.com>
>>>>>> To: vdsm-devel(a)lists.fedorahosted.org, engine-devel(a)ovirt.org
>>>>>> Sent: Thursday, March 7, 2013 6:25:54 AM
>>>>>> Subject: [Engine-devel] Proposal VDSM <=> Engine Data
>>>>>> Statistics
>>>>>> Retrieval Optimization
>>>>>>
>>>>>>
>>>>>> Please find the prettier version on the wiki:
>>>>>>
http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
>>>>>>
>>>>>> Proposal VDSM - Engine Data Statistics Retrieval
>>>>>> VDSM <=> Engine data retrieval optimization
>>>>>> Motivation:
>>>>>>
>>>>>>
>>>>>> Currently the RHEVM engine is polling the a lot of data from
>>>>>> VDSM
>>>>>> every 15 seconds. This should be optimized and the amount of
>>>>>> data
>>>>>> requested should be more specific.
>>>>>>
>>>>>> For each VM the data currently contains much more information
>>>>>> than
>>>>>> actually needed which blows up the size of the XML content
>>>>>> quite
>>>>>> big. We could optimize this by splitting the reply on the
>>>>>> getVmStats
>>>>>> based on the request of the engine into sections. For this
>>>>>> reason
>>>>>> Omer Frenkel and me have split up the data into parts based on
>>>>>> their
>>>>>> usage.
>>>>>>
>>>>>> This data can and usually does change during the lifetime of
>>>>>> the
>>>>>> VM.
>>>>>> Rarely Changed:
>>>>>>
>>>>>>
>>>>>> This data is change not very frequent and it should be enough
>>>>>> to
>>>>>> update this only once in a while. Most commonly this data
>>>>>> changes
>>>>>> after changes made in the UI or after a migration of the VM to
>>>>>> another Host. Status = Running acpiEnable = true vmType = kvm
>>>>>> guestName = W864GUESTAGENTT displayType = qxl guestOs = Win 8
>>>>>> kvmEnable = true # this should be constant and never changed
>>>>>> pauseCode = NOERR monitorResponse = 0 session = Locked # unused
>>>>>> netIfaces = [{'name': 'Realtek RTL8139C+ Fast
Ethernet NIC',
>>>>>> 'inet6': ['fe80::490c:92bb:bbcc:9f87'],
'inet':
>>>>>> ['10.34.60.148'],
>>>>>> 'hw': '00:1a:4a:22:3c:db'}] appsList =
['RHEV-Tools 3.2.4',
>>>>>> 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 3.2.3',
'RHEV-Network64
>>>>>> 3.2.2',
>>>>>> 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3',
'RHEV-Balloon64
>>>>>> 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64
3.2.2', 'RHEV-USB
>>>>>> 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64
3.2.2'] pid =
>>>>>> 11314
>>>>>> guestIPs = 10.34.60.148 # duplicated info displayIp = 0
>>>>>> displayPort
>>>>>> = 5902 displaySecurePort = 5903 username = user@W864GUESTAGENTT
>>>>>> clientIp = lastLogin = 1361976900.67 Often Changed:
>>>>>>
>>>>>>
>>>>>> This data is changed quite often however it is not necessary to
>>>>>> update this data every 15 seconds. As this is cumulative data
>>>>>> and
>>>>>> reflects the current status, and it does not need to be
>>>>>> snapshotted
>>>>>> every 15 seconds to retrieve statistics. The data can be
>>>>>> retrieved
>>>>>> in much more generous time slices. (e.g. Every 5 minutes)
>>>>>> network
>>>>>> =
>>>>>> {'vnet1': {'macAddr':
'00:1a:4a:22:3c:db', 'rxDropped': '0',
>>>>>> 'txDropped': '0', 'rxErrors':
'0', 'txRate': '0.0', 'rxRate':
>>>>>> '0.0',
>>>>>> 'txErrors': '0', 'state':
'unknown', 'speed': '100', 'name':
>>>>>> 'vnet1'}} disksUsage = [{'path': 'c:\\',
'total':
>>>>>> '64055406592',
>>>>>> 'fs': 'NTFS', 'used':
'19223846912'}, {'path': 'd:\\', 'total':
>>>>>> '3490912256', 'fs': 'UDF',
'used': '3490912256'}] timeOffset =
>>>>>> 14422
>>>>>> elapsedTime = 68591 hash = 2335461227228498964 statsAge = 0.09
>>>>>> #
>>>>>> unused Often Changed but unused
>>>>>>
>>>>>>
>>>>>> This data does not seem to be used in the engine at all. It is
>>>>>> not
>>>>>> even used in the data warehouse. memoryStats =
{'swap_out':
>>>>>> '0',
>>>>>> 'majflt': '0', 'mem_free':
'1466884', 'swap_in': '0',
>>>>>> 'pageflt':
>>>>>> '0', 'mem_total': '2096736',
'mem_unused': '1466884'}
>>>>>> balloonInfo
>>>>>> =
>>>>>> {'balloon_max': 2097152, 'balloon_cur': 2097152}
disks =
>>>>>> {'vda':
>>>>>> {'readLatency': '0', 'apparentsize':
'64424509440',
>>>>>> 'writeLatency':
>>>>>> '1754496', 'imageID':
'28abb923-7b89-4638-84f8-1700f0b76482',
>>>>>> 'flushLatency': '156549', 'readRate':
'0.00', 'truesize':
>>>>>> '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency':
>>>>>> '0',
>>>>>> 'apparentsize': '0', 'writeLatency':
'0', 'flushLatency': '0',
>>>>>> 'readRate': '0.00', 'truesize':
'0', 'writeRate': '0.00'}} Very
>>>>>> frequent uppdates needed by webadmin portal:
>>>>>>
>>>>>>
>>>>>> This data is mostly needed for the webadmin portal and might be
>>>>>> required to be updated quite often. An exception here is the
>>>>>> statsAge field, which seems to be unused by the Engine. This
>>>>>> data
>>>>>> could be requested every 15 seconds to keep things as they are
>>>>>> now.
>>>>>> cpuSys = 2.32 cpuUser = 1.34 memUsage = 30 Proposed Solution
>>>>>> for
>>>>>> VDSM & Engine:
>>>>>>
>>>>>>
>>>>>> We will introduce new optional parameters to getVmStats,
>>>>>> getAllVmStats and list to allow a finer grained specification
>>>>>> of
>>>>>> data which should be included.
>>>>>>
>>>>>> Parameter: statsType = <string> (getVmStats, getAllVmStats
>>>>>> only)
>>>>>> Allowed values:
>>>>>>
>>>>>> * full (default to keep backwards compatibility)
>>>>>> * app-list (Just send the application list)
>>>>>> * rare (include everything from rarely changed to very
>>>>>> frequent)
>>>>>> * often (include everything from often changed to very
>>>>>> frequent)
>>>>>> * frequent (only send the very frequently changed items)
>>>>>>
>>>>>>
>>>>>>
>>>>>> Parameter: clientId = <string> The client id is specified
by
>>>>>> the
>>>>>> client and should be unique however constantly used.
>>>>>>
>>>>>> Parameter: diff = <boolean> In combination with the
clientId
>>>>>> VDSM
>>>>>> will send only differences to the previous request from the
>>>>>> named
>>>>>> clientId. (if diff=true)
>>>>>>
>>>>>>
>>>>>> Additional Change:
>>>>>>
>>>>>>
>>>>>> Besides the introduction of the new parameters for list,
>>>>>> getVmStats
>>>>>> and getAllVmStats it might make sense to include a hash for the
>>>>>> appList into the rarely changed section of the response which
>>>>>> would
>>>>>> allow to identify changes and avoid having to sent the complete
>>>>>> appList every so often and only if the hash known to the client
>>>>>> is
>>>>>> outdated.
>>>>>>
>>>>>> Note: The appList (Application List) reported by the guest
>>>>>> agent
>>>>>> could be fully implemented on request only, as long as the
>>>>>> guest
>>>>>> agent installed supports this. As there seems to be a request
>>>>>> to
>>>>>> have the complete list of installed applications on all guests
>>>>>> this
>>>>>> data could be quite extensive and a huge list. On the other
>>>>>> hand
>>>>>> this data is only rarely visible and therefore it should not be
>>>>>> requested all the time and only on demand. Improvement of the
>>>>>> Guest
>>>>>> Agent:
>>>>>>
>>>>>>
>>>>>> As part of the proposed solution it is necessary to improve the
>>>>>> guest
>>>>>> agent as well. For the full application list there should be
>>>>>> implemented a caching system which will be fully reactive and
>>>>>> should
>>>>>> not poll the application list for example all the time. The
>>>>>> guest
>>>>>> can create a prepared data file containing all data in the JSON
>>>>>> format (as used for the communication with VDSM via VIO) and
>>>>>> just
>>>>>> have to read that file from disk and directly sends it to VDSM.
>>>>>> However it is quite possible that this list is to big and it
>>>>>> might
>>>>>> have to be chunked into pieces. (Multiple messages, which would
>>>>>> have
>>>>>> to be supported by VDSM then as well) The solution for this is
>>>>>> to
>>>>>> make VDSM request this data and it will retrieve the data
>>>>>> necessary
>>>>>> on request only. --
>>>>>> Regards,
>>>>>>
>>>>>> Vinzenz Feenstra | Senior Software Engineer
>>>>>> RedHat Engineering Virtualization R & D
>>>>>> Phone: +420 532 294 625
>>>>>> IRC: vfeenstr or evilissimo
>>>>>>
>>>>>> Better technology. Faster innovation. Powered by community
>>>>>> collaboration.
>>>>>> See how it works at redhat.com
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>>> _______________________________________________
>>>>> vdsm-devel mailing list
>>>>> vdsm-devel(a)lists.fedorahosted.org
>>>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>>
>
> --
> Regards,
>
> Vinzenz Feenstra | Senior Software Engineer
> RedHat Engineering Virtualization R & D
> Phone: +420 532 294 625
> IRC: vfeenstr or evilissimo
>
> Better technology. Faster innovation. Powered by community collaboration.
> See how it works at redhat.com
>
>
--
Regards,
Vinzenz Feenstra | Senior Software Engineer
RedHat Engineering Virtualization R & D
Phone: +420 532 294 625
IRC: vfeenstr or evilissimo
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
2:06 p.m.
New subject: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization
On 03/07/2013 01:25 PM, Vinzenz Feenstra wrote:
Often Changed but unused
This data does not seem to be used in the engine at all. It is *not*
even used in the data warehouse.
*memoryStats* = {'swap_out': '0', 'majflt': '0',
'mem_free': '1466884', 'swap_in': '0', 'pageflt':
'0', 'mem_total': '2096736', 'mem_unused':
'1466884'}
*balloonInfo* = {'balloon_max': 2097152, 'balloon_cur': 2097152}
these were added for better balloon control iirc. so api needs them for
mom probbaly. not sure if engine needs them or not.
*disks* = {'vda': {'readLatency': '0',
'apparentsize': '64424509440', 'writeLatency': '1754496',
'imageID': '28abb923-7b89-4638-84f8-1700f0b76482',
'flushLatency': '156549', 'readRate': '0.00',
'truesize': '18855059456', 'writeRate': '952.05'},
'hdc': {'readLatency': '0', 'apparentsize': '0',
'writeLatency': '0', 'flushLatency': '0',
'readRate': '0.00', 'truesize': '0', 'writeRate':
'0.00'}}
these i remember we added per request to the system - are you sure they
are not in the dwh?
4193
days inactive
4277
days old
23 comments
8 participants
participants (8)
-
Ayal Baron -
Dan Kenigsberg -
Itamar Heim -
Mark Wu -
Saggi Mizrahi -
Vinzenz Feenstra -
Yair Zaslavsky -
Yaniv Kaul