March 2015 - Kimchi-devel - oVirt List Archives

RFC - Guest memory usage
by Daniel Henrique Barboza 27 Oct '15

27 Oct '15

I was looking at the following item in the backlog: Guests Stats: Display memory utilization (use virt-df or virt-top or ...) If I understood it right, the idea here is to show the -inner- memory allocation of the guest. If you have a VM with 4Gb of RAM running an Ubuntu, we want to know how much memory the Ubuntu OS and its processes are using. I've done an investigation and I haven't found any tool to accomplish this. "virt-top", "virsh dommemstat" and the libvirt API retrieves the information of the memory usage of the guest relative to the host. In the example mentioned before, supposing that the host has 64Gb of RAM, all these tools would show that the VM is using 12% of the host RAM. They do not dive in the VM and shows the actual mem usage of the Ubuntu and its processes running there. Haven't found anything useful in other MLs and forums. The common answer is 'run top in a terminal inside the VM', which of course does not suit us. My question is: any thoughts about how we can implement this feature? Because I am starting to think that, in the end, this kind of info is strict to the guest OS and can't be polled from the outside. Thanks!

5 5

[RFC] Split Kimchi into Base Framework and Virtualization layers
by Lucio Correia 05 May '15

05 May '15

Hi, Gustavo (on cc) and me have been analyzing how to split Kimchi Virtualization layer form its Base web framework. We would like to leverage all infrastructure kimchi base framework provides (URI routing, error handling, i18n, authentication, API validation, plugin tabs loading) in other projects, and contribute this back to kimchi-ginger project. The general idea for the web framework is that it provides basic UI with login and tabs menu, and each plugin installed extends the menu with some tabs. We started by making kimchi a plugin, like ginger is. We are way advanced on that and already have most of kimchi's UI running as a plugin over an almost fully running base web framework. Ginger needs minimal changes, mainly on import statements. We can provide changes needed back to ginger too. Currently, we are splitting kimchi specific stuff like this way: (after renaming the dir src/kimchi to src/backend *) * we are using the name "backend" just for disambiguation from virtualization layer (plugins/kimchi). We will need to define a better name for that basic framework. One suggestion is to call it "wok", the base where kimchi is cooked. {src/backend => plugins/kimchi}/API.json | 0 {src/backend/control/vm => plugins/kimchi/control}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/control}/__init__.py | 2 +- {src/backend => plugins/kimchi}/control/config.py | 0 {src/backend => plugins/kimchi}/control/cpuinfo.py | 0 {src/backend => plugins/kimchi}/control/debugreports.py | 0 {src/backend => plugins/kimchi}/control/groups.py | 0 {src/backend => plugins/kimchi}/control/host.py | 0 {src/backend => plugins/kimchi}/control/interfaces.py | 0 {src/backend => plugins/kimchi}/control/networks.py | 0 {src/backend => plugins/kimchi}/control/peers.py | 0 {src/backend => plugins/kimchi}/control/storagepools.py | 0 {src/backend => plugins/kimchi}/control/storageservers.py | 0 {src/backend => plugins/kimchi}/control/storagevolumes.py | 0 {src/backend => plugins/kimchi}/control/templates.py | 0 {src/backend => plugins/kimchi}/control/users.py | 0 {src/backend => plugins/kimchi}/control/vm/Makefile.am | 0 {src/backend => plugins/kimchi}/control/vm/__init__.py | 0 {src/backend => plugins/kimchi}/control/vm/hostdevs.py | 0 {src/backend => plugins/kimchi}/control/vm/ifaces.py | 0 {src/backend => plugins/kimchi}/control/vm/snapshots.py | 0 {src/backend => plugins/kimchi}/control/vm/storages.py | 0 {src/backend => plugins/kimchi}/control/vms.py | 0 {src/backend => plugins/kimchi}/disks.py | 0 {src/backend => plugins/kimchi}/imageinfo.py | 0 {src/backend => plugins/kimchi}/iscsi.py | 0 {src/backend => plugins/kimchi}/isoinfo.py | 0 {src/backend => plugins/kimchi}/kvmusertests.py | 0 {src/backend => plugins/kimchi}/mockmodel.py | 0 {src/backend/control/vm => plugins/kimchi/model}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/model}/__init__.py | 8 --- {src/backend => plugins/kimchi}/model/config.py | 0 {src/backend => plugins/kimchi}/model/cpuinfo.py | 0 {src/backend => plugins/kimchi}/model/debugreports.py | 0 {src/backend => plugins/kimchi}/model/diskutils.py | 0 {src/backend => plugins/kimchi}/model/featuretests.py | 0 {src/backend => plugins/kimchi}/model/groups.py | 0 {src/backend => plugins/kimchi}/model/host.py | 0 {src/backend => plugins/kimchi}/model/hostdev.py | 0 {src/backend => plugins/kimchi}/model/interfaces.py | 0 {src/backend => plugins/kimchi}/model/libvirtconnection.py | 0 {src/backend => plugins/kimchi}/model/libvirtstoragepool.py | 0 plugins/kimchi/model/model.py | 93 ++++++++++++++++++++++++++++ {src/backend => plugins/kimchi}/model/networks.py | 0 {src/backend => plugins/kimchi}/model/peers.py | 0 {src/backend => plugins/kimchi}/model/storagepools.py | 0 {src/backend => plugins/kimchi}/model/storageservers.py | 0 {src/backend => plugins/kimchi}/model/storagetargets.py | 0 {src/backend => plugins/kimchi}/model/storagevolumes.py | 0 {src/backend => plugins/kimchi}/model/templates.py | 0 {src/backend => plugins/kimchi}/model/users.py | 0 {src/backend => plugins/kimchi}/model/utils.py | 0 {src/backend => plugins/kimchi}/model/vmhostdevs.py | 0 {src/backend => plugins/kimchi}/model/vmifaces.py | 0 {src/backend => plugins/kimchi}/model/vms.py | 0 {src/backend => plugins/kimchi}/model/vmsnapshots.py | 0 {src/backend => plugins/kimchi}/model/vmstorages.py | 0 {src/backend => plugins/kimchi}/osinfo.py | 0 {src/backend => plugins/kimchi}/scan.py | 0 {config => plugins/kimchi/ui}/Makefile.am | 0 {config/ui => plugins/kimchi/ui/config}/Makefile.am | 0 config/ui/tabs.xml => plugins/kimchi/ui/config/tab-ext.xml | 0 {src/backend => plugins/kimchi}/vmtemplate.py | 0 The remaining generic stuff on the "backend" side, usable or already in use by other plugins, would be: backend |___model | |___model.py | |___plugins.py | |___tasks.py |___control | |___base.py | |___plugins.py | |___tasks.py | |___utils.py |___xmlutils | |___network.py | |___disk.py | |___utils.py | |___graphics.py | |___qemucmdline.py | |___interface.py |___asynctask.py |___auth.py |___basemodel.py |___cachebust.py |___distroloader.py |___exception.py |___i18n.py |___netinfo.py |___network.py |___objectstore.py |___proxy.py |___repositories.py |___rollbackcontext.py |___root.py |___screenshot.py |___server.py |___sslcert.py |___swupdate.py |___template.py |___utils.py |___vnc.py We don't want to make this a too long e-mail and lose the focus on the idea. We have further technical details to discuss in a follow up. Please let us know your thoughts on that, any directions and requirements you have on this, and how could we proceed with that plan. -- Lucio Correia Software Engineer IBM LTC Brazil

2 3

[RFC] Do not block operations due to ref_cnt
by Crístian Viana 03 Apr '15

03 Apr '15

Hi, I'd like to propose a change to how Kimchi uses the resource field "ref_cnt". Currently, "ref_cnt" - which stands for "reference count" - is one of the fields returned when looking up a storage volume. Its purpose is to indicate how many times that resource is being used at the moment. For example, if the resource /storagepools/pool/storagevolumes/vol has ref_cnt=1, it means that the disk is attached to 1 VM right now and thus it cannot be attached to another VM. I believe the original idea of this feature is to prevent the same resource from being attached more than once at the same time. However, IMO, that might not always be the desired behavior and there's no way to enforce it completely as those resources can be used outside of Kimchi, where "ref_cnt" doesn't exist. For example, if I have one VM which uses the disk "vol", I'm not able to attach it to another VM via Kimchi; but if I use another libvirt-based VM manager (e.g. virsh), I am able to attach that disk to a different VM. This becomes even trickier when we consider other operations, like snapshots, which can attach/detach disks while they're being reverted to. Also, suppose I might want to inspect one VM's disk from another VM, and then I'd need to attach one disk twice; Kimchi wouldn't allow that by stating that the disk is already in use. I propose Kimchi should stop using "ref_cnt" as a blocking method. The field may still exist for information/warning messages (e.g. "are you sure you want to attach this disk? it's already being used by another VM.") but no operation should be blocked because of it, as it is the case now. As inconsistencies with that value may happen and we have no way to make sure it will always work, we shouldn't annoy the user by stopping them from doing something that may be perfectly valid. Any feedback will be very welcome. Best regards, Crístian.

3 3

[kimchi-devel][PATCHv3 0/6] Upload storage volume
by lvroyce＠linux.vnet.ibm.com 03 Apr '15

03 Apr '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> known issue: When using api to send requests without setting up session, cherrypy server will report error after hundred of calls, because race condition below: https://bitbucket.org/cherrypy/cherrypy/pull-request/50/fix-race-condition-… But it works well with session setup. v3>v1, Use libvirt upload api instead of r/w file Use lock to synchronize upload to prevent Royce Lv (6): Update docs and json schema of storage volume upload Update controller to make update accept formdata params Add lock facility for storage volume upload Update model for storage volume update Fix incomplete record when uploading update test case for storage volume upload docs/API.md | 7 ++-- src/kimchi/API.json | 19 +++++++++++ src/kimchi/control/base.py | 6 ++-- src/kimchi/i18n.py | 4 +++ src/kimchi/isoinfo.py | 5 ++- src/kimchi/mockmodel.py | 8 ++--- src/kimchi/model/storagevolumes.py | 61 +++++++++++++++------------------- src/kimchi/model/utils.py | 46 ++++++++++++++++++++++++++ tests/test_model.py | 67 ++++++++++++++++++++++++++++++++++++++ 9 files changed, 177 insertions(+), 46 deletions(-) -- 2.1.0

3 20

[PATCH] RFC patch to make nginx proxy optional
by Julien Goodwin 03 Apr '15

03 Apr '15

Sending this as an RFC patch, I expect people will want changes from this. Implements Issue #570. Julien Goodwin (1): Initial prototype, make nginx proxy optional. docs/Makefile.am | 1 + docs/apache.conf.ex | 35 +++++++++++++++++++++++++++++++++++ src/kimchi.conf.in | 3 +++ src/kimchi/config.py.in | 1 + src/kimchi/proxy.py | 6 ++++++ 5 files changed, 46 insertions(+) create mode 100644 docs/apache.conf.ex -- 2.1.4

4 18

[RFC] New API to create a random guest console password
by Aline Manera 02 Apr '15

02 Apr '15

Today user may set/change the guest console password and also its expiration time through Kimchi API. When passing an empty password, a random password is automatically generated. curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"passwd": ""}}' That way is difficult to handle when user wants to reset the guest password. We have a similar issue when we automatically change the passwdValidTo when it is expired - increasing it in 30 seconds. My proposal is simple: only change "passwd" and "passwdValidTo" when user wants to do it. curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"passwd": "123456", "passwdValidTo": "<some datetime format>"}}' curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"passwdValidTo": "<some datetime format>"}}' And make sure the passwdValidTo is only acceptable when there is a passwd set. And to reset those values, we only need to send an empty string: curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"password": "", "passwdValidTo": ""}}' And create a new API: POST /vms/blah/ticket to automatically generate a random password valid only for 30 seconds. curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah/ticket -X POST -d'{}' What do you think about it? Regards, Aline Manera

2 3

[RFC] Hotplug a new VM disk
by Crístian Viana 01 Apr '15

01 Apr '15

Hi, I'd like to propose a new feature to hotplug a new disk to a VM. This feature will be implemented only on the UI side. The backend already has the methods needed to make it work. Suppose a user wants to add a new disk to a running VM. Currently, they have to create an empty disk using other tools outside of Kimchi and put it into an existing storage pool. This might be challenging for some users, as it requires the use of other tools, possibly involving the command line (e.g. qemu-img). With this new feature, the user will only select the storage pool and the size of the new disk they want to add; Kimchi will create the disk in the appropriate pool and attach it to the VM. In order to perform this feature, the UI will have to be updated to allow the following steps: 1) Select an existing VM, "Actions" > "Edit". 2) Switch to the tab "Storage". 3) Click on the button "+". 4) On "Device type", select "disk" . 5) On "Storage pool", select any storage pool. *6) On "Storage volume", select "New...".** **7) Enter the desired disk size.* 8) Click on the button "Attach". Steps 6 and 7 are the ones related to this feature and they need to be implemented. The other steps already exist in the current UI. When the user performs step 8, the UI should send the following requests to the server: POST /storagepools/<pool>/storagevolumes {'name': <vol>, 'capacity': <size>} POST /vms/<vm>/storages {'name': <vol>} Those requests already work, they just have to be executed in that order in order to perform this new feature. Any feedback will be very welcome. Best regards, Crístian.

2 1

[RFC] Hotplug a VM network interface
by Crístian Viana 01 Apr '15

01 Apr '15

Hi, I'd like to propose a new feature to hotplug a network interface to a running VM. A similar feature is already available for stopped VMs by selecting a VM and clicking on "Actions" > "Edit", tab "Interface"; the user is able to add or remove a network interface in that dialog. However, if the VM is running, that tab disables the buttons to add and remove network interfaces. This feature will allow those two operations to be performed also on running VMs. The API for adding and removing network interfaces to a VM already exist: POST /vms/<vm>/ifaces {<iface data>} DELETE /vms/<vm>/ifaces/<iface> However, they currently raise an exception when the VM is running. This feature will use those same commands in order to hotplug a network interface, keeping the existing REST interface. Any feedback will be very welcome. Best regards, Crístian.

2 1

[RFC] Add suport for VM suspend/resume
by Crístian Viana 01 Apr '15

01 Apr '15

Hi, I'd like to propose two new operations on virtual machines: *suspend* and *resume*: Suspend ---------- From libvirt.org: /Suspend an active domain, the process is frozen without further access to CPU resources and I/O but the memory used by the domain at the hypervisor level will stay allocated./ REST syntax: POST /vms/<vm-name>/suspend Return status: 200: OK 4xx: error Return data: None Resume ---------- From libvirt.org: /Resume a suspended domain, the process is restarted from the state where it was frozen by calling "suspend"./ REST syntax: POST /vms/<vm-name>/resume Return status: 200: OK 4xx: error Return data: None -------------------- This feature has been partially implemented in previous patches ("[PATCH 0/3] VM suspend/resume", from Jan 21st), and its progress should be resumed soon. The UI will have to be updated accordingly by adding buttons to suspend and resume a VM, somewhat like the buttons which start and stop a VM. Any feedback will be very welcome. Best regards, Crístian.

2 1

[PATCH v2 0/3] Fix issue #461 (backend)
by Crístian Viana 01 Apr '15

01 Apr '15

This is the changelog between this and the previous patchset: - Don't allow mirrorlist and metalink to be specified at the same time; according to the YUM documentation, that situation is invalid. Crístian Viana (3): Handle empty variables when updating YUM repository Use more generic message in repo mirror list error issue #461: Add 'metalink' support for YUM repositories src/kimchi/API.json | 5 +++++ src/kimchi/i18n.py | 6 ++++-- src/kimchi/repositories.py | 31 +++++++++++++++++++++++++------ 3 files changed, 34 insertions(+), 8 deletions(-) -- 2.1.0

2 4