On 10/27/2015 05:59 PM, Paulo Ricardo Paz Vital wrote:
On Sun, 2014-12-07 at 23:16 -0200, Daniel Henrique Barboza wrote:
I was looking at the following item in the backlog:

Guests	Stats: Display memory utilization (use virt-df or virt
-top or ...)
If I understood it right, the idea here is to show the -inner- memory
allocation of the guest. If you have a VM with 4Gb of RAM running an
Ubuntu, we want to know how much memory the Ubuntu OS and its
processes are using. 

I've done an investigation and I haven't found any tool to accomplish
this. "virt-top", "virsh dommemstat" and the libvirt API retrieves
the information of the memory usage of the guest relative to the
host. In the example mentioned before, supposing that the host has
64Gb of RAM, all these tools would show that the VM is using 12% of
the host RAM. They do not dive in the VM and shows the actual mem
usage of the Ubuntu and its processes running there.

Haven't found anything useful in other MLs and forums. The common
answer is 'run top in a terminal inside the VM', which of course does
not suit us. My question is: any thoughts about how we can implement
this feature? Because I am starting to think that, in the end, this
kind of info is strict to the guest OS and can't be polled from the

Looking again how to solve this, I could see that virt-manager and
virsh have interfaces to provide the guest's memory usage. For example,
using virsh you can get this kind of information:

$ sudo virsh dommemstat fedora22-mig
actual 2097152
swap_in 0
swap_out 0
major_fault 1150
minor_fault 356159
unused 1689844
available 2049328
rss 496976

This output says that the VM has 2097152 kB allocated and at the moment
when command was executed, it was using 496976 Kb (the rss value).

So, I looked for how those information is provided, and the Libvirt API
provides all necessary information by the https://libvirt.org/html/libv
irt-libvirt-domain.html#virDomainMemoryStats method.

So, I'm going to work in this tasks and submit patch to provide the
guest's memory usage in back-end. For the moment, I'm planning to
return to following information:

### Resource: Virtual Machine

**URI:** /plugins/kimchi/vms/*:name*


* **GET**: Retrieve the full description of a Virtual Machine
    * name: The name of the VM.  Used to identify the VM in this API
    * state: Indicates the current state in the VM lifecycle
        * running: The VM is powered on
        * paused: The VMs virtual CPUs are paused
        * shutoff: The VM is powered off
    * stats: Virtual machine statistics:
        * cpu_utilization: A number between 0 and 100 which indicates
          the percentage of CPU utilization.
Since we are planning to provide virtual machine statistics, is it good idea to include cpu steal time of guest as part of this api? I see many of the cloud providers discussing on steal information and how that affects guest performance.

By the definition - Steal time is the amount of CPU time desired by a guest virtual machine that is not provided by the host. Steal time occurs when the host allocates these resources elsewhere: for example, to another guest.

If steal time exceeds certain %, it would need to revisit cloud set up, changing how cpu is getting shared or increase cpu for particular guest.

This feature will benefit customers using our product, to monitor and manage performance of guests . :-)

Here are some of the links which talks about steal time:
* https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/sect-KVM_guest_timing_management-Steal_time_accounting.html

* http://blog.scoutapp.com/articles/2013/07/25/understanding-cpu-steal-time-when-should-you-be-worried

* http://www.stackdriver.com/understanding-cpu-steal-experiment/

* https://www.datadoghq.com/blog/understanding-aws-stolen-cpu-and-how-it-affects-your-apps/
+        * mem_utilization: A number between 0 and 100 which indicates
+          the percentage of memory utilization.
        * net_throughput: Expresses total network throughput for reads 
          and writes across all virtual interfaces (kb/s).
        * net_throughput_peak: The highest recent value of
        * io_throughput: Expresses the total IO throughput for reads
          and writes across all virtual disks (kb/s).
        * io_throughput_peak: The highest recent value of 
    * uuid: UUID of the VM.

Feedbacks are welcome.
Best regards,
Paulo Ricardo Paz Vital
IBM Linux Technology Center

Kimchi-devel mailing list