<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 11/20/2014 09:20 AM, Amador Pahim
wrote:<br>
</div>
<blockquote cite="mid:546DDC77.3090803@redhat.com" type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<div class="moz-cite-prefix">Hi Lior,<br>
<br>
Thank you for this. Indeed I have seen multiple requests for
this. I also have a bugzilla for it: <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="https://bugzilla.redhat.com/show_bug.cgi?id=1108144">https://bugzilla.redhat.com/show_bug.cgi?id=1108144</a>.
Some comments bellow.<br>
<br>
On 11/11/2014 07:07 AM, Lior Vernia wrote:<br>
</div>
<blockquote cite="mid:5461DFFE.9060006@redhat.com" type="cite">
<pre wrap="">Hello,
The need to monitor cumulative VM network usage has come up several
times in the past; while this should be handled as part of
(<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://bugzilla.redhat.com/show_bug.cgi?id=1063343">https://bugzilla.redhat.com/show_bug.cgi?id=1063343</a>), in the mean time
I've written a small Python script that monitors those statistics,
attached here.
The script polls the engine via RESTful API periodically and dumps the
up-to-date total usage into a file. The output is a multi-level
map/dictionary in JSON format, where:
* The top level keys are VM names.
* Under each VM, the next level keys are vNIC names.
* Under each vNIC, there are keys for total 'rx' (received) and 'tx'
(transmitted), where the values are in Bytes.
The script is built to run forever. It may be stopped at any time, but
while it's not running VM network usage data will "be lost". When it's
re-run, it'll go back to accumulating data on top of its previous data.</pre>
</blockquote>
<br>
This could be mitigated if along with rx and tx data, vdsm was
reporting a timestamp reflecting the time when data was collected.
So, even with gaps, we should be able to calculate the cumulative
information.<br>
</blockquote>
<br>
Actually vdsm is not reporting rx/tx bytes. They are "tx/rx rate".
So, we're only able to see the average consumption for the time
between pooling periods.<br>
<br>
<blockquote cite="mid:546DDC77.3090803@redhat.com" type="cite"> <br>
<blockquote cite="mid:5461DFFE.9060006@redhat.com" type="cite">
<pre wrap="">
A few disclaimers:
* I haven't tested this with any edge cases (engine service dies, etc.).
* Tested this with tens of VMs, not sure it'll work fine with hundreds.
* The PERIOD_TIME (polling interval) should be set so that it matches
both the engine's and vdsm's polling interval (see comments inside the
script), otherwise data will be either lost or counted multiple times.
>From 3.4 onwards, default configuration should be fine with 15 seconds.</pre>
</blockquote>
<br>
Here we have another issue. In 3.4, 15 seconds is fine... backend
and vdsm are in line with 15 seconds. But up to 3.3, vdsm is
pooling the data every 5 seconds and backend is collecting data
every 15 seconds. So 2 in 3 vdsm poolings are droped. Since you're
handling total bytes, this might not be a big issue.<br>
</blockquote>
<br>
Forget the last sentence. It's a big issue since the data is not
cumulative, but the average of the period between vdsm checks.
bz#1066570 is the solution for precise calculations here.<br>
<br>
<blockquote cite="mid:546DDC77.3090803@redhat.com" type="cite"> <br>
<blockquote cite="mid:5461DFFE.9060006@redhat.com" type="cite">
<pre wrap="">* The precision of traffic measurement on a NIC is 0.1% of the
interface's speed over each PERIOD_TIME interval. For example, on a
1Gbps vNIC, when PERIOD_TIME = 15s, data will only be measured in 15Mb
(~2MB) quanta. Specifically what this means is, that in this example,
any traffic smaller than 2MB over a 15-second period would be negligible
and wouldn't be recorded.</pre>
</blockquote>
<br>
Looking to the code, if "overhead" is bigger than "PERIOD_TIME",
cumulative data for a given period will never be accurate. Anyway
the script will fall in exception when that is the case (negative
value for time.sleep()). The mentioned timestamp reported by vdsm
could drop the need for the "overhead" calculation.<br>
<br>
<blockquote cite="mid:5461DFFE.9060006@redhat.com" type="cite">
<pre wrap="">
Knock yourselves out :)
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</body>
</html>