February 2014 - Kimchi-devel - oVirt List Archives

[RFC] Host's repositories support
by Paulo Ricardo Paz Vital 05 Feb '14

05 Feb '14

Hello guys. This is the RFC for the Host's repositories support (task "Register YUM, apt, zypper repos" of 1.2 sprint 3). The support will be provided by adding in the control/host.py a new collection (Repositories), responsible to manage all repositories of the system, and a new resource (Repository), responsible to operate with a specific repository. All this management/operations will be transparent to host's package management system, being provided by one of the new back end classes. The back end support provides 4 new classes: 1) Repositories (object): Class to represent and operate with repositories information in Kimchi's perspective. It's transparent to host's package management system, and can execute all operations necessary: add repository, get all repositories list, get information about one repository, update a repository, enable and disable a repository and remove a repository. This class will load in run time the necessary classes to work with the host's package management: YumRepo for YUM systems based, AptRepo for APT systems based and ZypperRepo for Zypper systems based; 2) YumRepo (object): Class to represent and operate with YUM repositories. Loaded only on those systems that supports YUM (e.g. Fedora and RHEL), it's responsible to connect, collect and provide information of YUM repositories in the system. Also it's responsible to create/delete the files in disk to maintain the repositories in system after disconnection. 3) AptRepo (object): Class to represent and operate with APT repositories. Loaded only on those systems that supports APT (e.g. Debian and Ubuntu), it's responsible to connect, collect and provide information of APT repositories in the system. Also it's responsible to create/delete the files in disk to maintain the repositories in system after disconnection. 4) ZypperRepo (object): Class to represent and operate with Zypper repositories. Loaded only on those systems that supports Zypper (e.g. OpenSuse), it's responsible to connect, collect and provide information of Zypper repositories in the system. Also it's responsible to create/delete the files in disk to maintain the repositories in system after disconnection. Bellow I present the REST API for the repositories support. ### Collection: Host Repositories **URI:** /host/repositories **Methods:** * **GET**: Retrieve a summarized list of all repositories available * **POST**: Add a new repository * repo_id : Unique repository name for each repository, one word. * repo_name: Human-readable string describing the repository. * baseurl: URL to the repodata directory when "is_mirror" is false. Otherwise, it can be URL to the mirror system for YUM. Can be an http://, ftp:// or file:// URL. * is_mirror *(optional)*: Set the given URI of baseurl as a mirror list, instead of use baseurl in repository configuration. * url_args *(optional)*: Arguments to be passed to baseurl, like the list of APT repositories provided by the same baseurl. * enabled *(optional)*: Indicates if repository should be included as a package source: * false: Do not include the repository. * true: Include the repository. * gpgcheck *(optional)*: Indicates if a GPG signature check on the packages gotten from repository should be performed: * false: Do not check GPG signature * true: Check GPG signature * gpgkey *(optional)*: URL pointing to the ASCII-armored GPG key file for the repository. This option is used if yum needs a public key to verify a package and the required key hasn't been imported into the RPM database. ### Resource: Repository **URI:** /host/repositories/*:repo-id* **Methods:** * **GET**: Retrieve the full description of a Repository * repo_id : Unique repository name for each repository, one word. * repo_type: Indicates which type of repository is: * yum: Indicates it is a YUM repository. * deb: Indicates it is a APT repository. * zyp: Indicates it is a Zypper repository. * repo_name: Human-readable string describing the repository. * baseurl: URL to the repodata directory when "is_mirror" is false. Otherwise, it can be URL to the mirror system for YUM. Can be an http://, ftp:// or file:// URL. * url_args *(optional)*: Arguments to be passed to baseurl, like the list of APT repositories provided by the same baseurl. * enabled *(optional)*: Indicates if repository should be included as a package source: * false: Do not include the repository. * true: Include the repository. * gpgcheck *(optional)*: Indicates if a GPG signature check on the packages gotten from repository should be performed: * false: Do not check GPG signature * true: Check GPG signature * gpgkey *(optional)*: URL pointing to the ASCII-armored GPG key file for the repository. This option is used if yum needs a public key to verify a package and the required key hasn't been imported into the RPM database. * **DELETE**: Remove the Repository * **POST**: *See Repository Actions* * **PUT**: update the parameters of existing Repository * repo_id *(otional)*: Unique repository name for each repository, one word. * repo_name *(otional)*: Human-readable string describing the repository. * baseurl *(optional)*: URL to the repodata directory when "is_mirror" is false. Otherwise, it can be URL to the mirror system for YUM. Can be an http://, ftp:// or file:// URL. * is_mirror *(optional)*: Set the given URI of baseurl as a mirror list, instead of use baseurl in repository configuration. * url_args *(optional)*: Arguments to be passed to baseurl, like the list of APT repositories provided by the same baseurl. * enabled *(optional)*: Indicates if repository should be included as a package source: * false: Do not include the repository. * true: Include the repository. * gpgcheck *(optional)*: Indicates if a GPG signature check on the packages gotten from repository should be performed: * false: Do not check GPG signature * true: Check GPG signature * gpgkey *(optional)*: URL pointing to the ASCII-armored GPG key file for the repository. This option is used if yum needs a public key to verify a package and the required key hasn't been imported into the RPM database. **Actions (POST):** * enable: Enable the Repository as package source * disable: Disable the Repository as package source

3 4

[PATCH] bug fix: Set full path to guest page file in guest tab
by Aline Manera 05 Feb '14

05 Feb '14

From: Aline Manera <alinefm(a)br.ibm.com> Commit 6bcbb3f0 changed how the guest tab is loaded to be more robust, but it was using relative path to guest page which causes errors when Kimchi is running outside its top directory. So use full path to avoid those kind of problems. Signed-off-by: Aline Manera <alinefm(a)br.ibm.com> --- src/kimchi/root.py | 8 +++++++- ui/pages/tabs/guests.html.tmpl | 2 +- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/kimchi/root.py b/src/kimchi/root.py index c3d471e..ce4a49c 100644 --- a/src/kimchi/root.py +++ b/src/kimchi/root.py @@ -75,8 +75,14 @@ class Root(Resource): @cherrypy.expose def tabs(self, page, **kwargs): + # In order to load the Guests tab, we also use Cheetah in the tab + # template to save the delay of the extra get to the guest page + # For that, the tab template needs to know the correct path to ui files + data = {} + data['ui_dir'] = paths.ui_dir + if page.endswith('.html'): - return template.render('tabs/' + page, None) + return template.render('tabs/' + page, data) raise cherrypy.HTTPError(404) diff --git a/ui/pages/tabs/guests.html.tmpl b/ui/pages/tabs/guests.html.tmpl index e2ab173..8b530c7 100644 --- a/ui/pages/tabs/guests.html.tmpl +++ b/ui/pages/tabs/guests.html.tmpl @@ -52,7 +52,7 @@ $_("No guests found.") </div> <script id="guest-tmpl" type="kimchi/template"> - $ht(file="ui/pages/guest.html.tmpl",searchList=[self,{'lang':$lang}]) + $ht(file=$data.ui_dir + "/pages/guest.html.tmpl", searchList=[self, {'lang':$lang}]) </script> <script type="text/javascript"> kimchi.guest_main(); -- 1.7.10.4

3 3

[PATCH 0/3 RFC] Refactor exception
by Aline Manera 04 Feb '14

04 Feb '14

From: Aline Manera <alinefm(a)br.ibm.com> Hi all, This is the RFC for the refactor exception task. Sheldon has already done something related to that some time ago. He also created a wiki for that: - https://github.com/kimchi-project/kimchi/wiki/refactor-exception The only change I made in him approach is translating the message in backend and send it to UI That way we don't need to duplicate the messages on backend and UI (as it is explained on wiki) The steps to get it done: 1) Change cherrypy error handler to send the exception data to UI 2) Create a common Exception class (KimchiError) to translate the error message and proper set the parameters 3) Update UI to show the message received from backend to the user I also would like to create a file on backend: __messages__.py which will contain all messages used on backend. For example: # __messages__.py VM_START_FAILED = "<code>: Failed to start virtual machine %(name)s" The contants names should be: <resource>_<action>_<error> The <code> need to be a unique value between all messages. It is needed in order to we identify the error independent of the message language. That way a user running Kimchi in German can easily post his error on mail list or github, and we can identify where the problem is by the message code. I also will add this code in the UI messages. For the code, we can build it like: <resource><BACK|UI><type-of-error><ident> And on backend: VMBACKINFO01 VMBACKERRO01 VMBACKWARN01 And on UI: VMUIINFO01 VMUIERRO01 VMUIWARN01 So we will also need: 4) Create a __messages__ file with all messages used on backend 5) Update build process to add the messages in this file to .po files 6) Update UI messages to add a code for them I made a small patch set with my proposal. This patch set does not include the items 4,5 and 6 In fact, I used an existent message for test (as you can notice on patches) And *just for example* it will *always* raise an exception while starting a VM. Any comments/suggestions are welcome. =) Aline Manera (2): refactor exception: Create a common Exception to translate error messages refator exception: Example of use ShaoHe Feng (1): refactor exception: Add a kimchi special error handler src/kimchi/control/base.py | 5 ++++- src/kimchi/exception.py | 29 ++++++++++++++++++++++------- src/kimchi/model/vms.py | 4 +++- src/kimchi/template.py | 14 ++++++++++++++ ui/js/src/kimchi.guest_main.js | 4 ++-- 5 files changed, 45 insertions(+), 11 deletions(-) -- 1.7.10.4

3 7

[PATCH V3] Avoid useless libvirt error log produced by featuretest
by Aline Manera 04 Feb '14

04 Feb '14

From: Aline Manera <alinefm(a)br.ibm.com> I am sending the next version for the apporc patch as he may be out for the Chinese holidays. v2 -> v3: 1. Any error on feature tests will be log into kimchi error log file For that, silent cherrypy screen log to avoid displyaing the message on screen. v1 -> -v2: 1. Just hide the portocol type error, leave other error messages as it is. (Thanks Cristian) 2. Unregister the error handler when necessary. (Thanks Cristian) 3. Keep back kimchi log "*** Running feature tests ***", etc. (Thanks Cristian) 4. Move libvirt error handler function inside of Featuretest. (Thanks Aline) v1: Avoid useless libvirt error log produced by featuretest apporc (1): Avoid useless libvirt error log produced by featuretests src/kimchi/featuretests.py | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) -- 1.7.10.4

4 5

[RFC] Manage guest CDROM devices
by Rodrigo Trujillo 04 Feb '14

04 Feb '14

Reference: https://github.com/kimchi-project/kimchi/wiki/Todo-1.2 https://github.com/kimchi-project/kimchi/wiki/customize-VM Royce has created the structure to add/list/lookup cdrom devices in the guests. There is a patch in the list (still under revision) with the code: " Kimchi-devel] [PATCH 0/5] cdrom: support add and query cdrom " This RFC aims to continue the second part of this work and modify the API to allow user to change (UPDATE/POST) attached cd ISOs and delete (DELETE) cdrom devices. If necessary I can continue the Royces work. Aline has suggested to use the API as "/vms/<name>/storages" : """ . . . Maybe we can use /vms/<name>/storages That way we can use the same API to add/remove different kind of storages to the vm: cdrom, scsi disk, ide disk, etc ... """ This is the first decision here (1) Use api as follow ? (I propose to use API params to differentiate the storages, like '?type=cdrom') **URI:** /vms/*:name*/storages * **GET**: Retrieve all storages attached to the VM * **POST**: Attatch new storage to specified virtual machine. * name : The name of the storage in the VM. * type: The type of the storage: cdrom, disk * path *(optional)*: Path of ISO or DISK. ### Sub-resource: storage **URI:** /vms/*:name*/storage/*:name* * **GET**: Retrieve the storage information * path: Path of the storage * type: Type of the storage * **PUT**: Update storage information * name *(optional)*: The name of the storage in the VM. * path *(optional)*: Path of ISO or DISK. * **DELETE**: Remove the storage Examples: UPDATE {'path': ''} /vms/vm-1/storage/cdrom-1--> to eject the iso UPDATE {'path': '/root/ubuntu.iso'} /vms/vm-1/storage/cdrom-1-->to change or insert a media POST {'path':'abcd', 'bus': 'ide', 'name': 'cdrom-2'} /vms/vm-1/storage --> to attach a new cdrom DELETE {} /vms/vm-1/cdroms/cdrom-1-->to detach a cdrom (2) Should be possible to add and remove storages with the guest running ? (Like hot plug) Does libvirt provide this ? (3) The new attached isos should be shown in the guests like changing a CD ? (4) The specification mention tuning of the devices. Should tuning be always done in the background, or allow users to specify tuning parameters ? Regards, Rodrigo Trujillo

3 3

[PATCH] bug fix: Fix shallow scan broken after model refactoring
by Aline Manera 04 Feb '14

04 Feb '14

From: Aline Manera <alinefm(a)br.ibm.com> The shallow scan is always return an empty array as it is trying to access the pool name as a libvirt pool object. Fix it to restore shallow scan functionality. Signed-off-by: Aline Manera <alinefm(a)br.ibm.com> --- src/kimchi/model/storagevolumes.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/kimchi/model/storagevolumes.py b/src/kimchi/model/storagevolumes.py index 7ff570b..8440a76 100644 --- a/src/kimchi/model/storagevolumes.py +++ b/src/kimchi/model/storagevolumes.py @@ -160,8 +160,9 @@ class IsoVolumesModel(object): pools = conn.listStoragePools() pools += conn.listDefinedStoragePools() - for pool in pools: + for pool_name in pools: try: + pool = StoragePoolModel.get_storagepool(pool_name, self.conn) pool.refresh(0) volumes = pool.listVolumes() except InvalidOperation: @@ -169,7 +170,7 @@ class IsoVolumesModel(object): continue for volume in volumes: - res = self.storagevolume.lookup(pool, volume) + res = self.storagevolume.lookup(pool_name, volume) if res['format'] == 'iso': res['name'] = '%s' % volume iso_volumes.append(res) -- 1.7.10.4

2 2

RFC: "Add disk to existing logical storage pool"
by Daniel H Barboza 04 Feb '14

04 Feb '14

Hello! I am almost delivering a first version of this feature but now I am having doubts in how it might work or not. My implementation would use vgextend and add a disk to an existing LVM pool. The question is that, in this process, all existing data in the added partition will be deleted. Is this ok? We can warn the user about it, of course. But I am not sure if this is the intended design. I've spoken with Aline a while ago and she agreed that the implementation would function similar to what we have today when creating a new LVM pool. We simply "do not care" about the potential data loss when adding the disk to an existing pool. Any thoughts? thanks!

3 5

[PATCH 0/3] Fix permission of cdrom files
by Aline Manera 04 Feb '14

04 Feb '14

From: Aline Manera <alinefm(a)br.ibm.com> It is a continuation of Sheldon's work. What I added: - Run setfacl to add read permission to user who own the file - Fix permission of cdrom file before starting the virtual machine Aline Manera (2): Remove kvmusertests.py Make sure ISO files have read permission while starting VM ShaoHe Feng (1): Add a method to fix read permission on ISO files src/kimchi/kvmusertests.py | 66 -------------------------------------------- src/kimchi/model/vms.py | 10 ++++++- src/kimchi/utils.py | 9 ++++-- 3 files changed, 16 insertions(+), 69 deletions(-) delete mode 100644 src/kimchi/kvmusertests.py -- 1.7.10.4

2 7

[PATCH] Fix get vms list function name
by Rodrigo Trujillo 04 Feb '14

04 Feb '14

The host shutdown function was using a function that does not exist in the class. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/model/host.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index d068614..816e2e8 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -66,7 +66,7 @@ class HostModel(object): def shutdown(self, args=None): # Check for running vms before shutdown - running_vms = self.vms_get_list_by_state('running') + running_vms = self._get_vms_list_by_state('running') if len(running_vms) > 0: raise OperationFailed("Shutdown not allowed: VMs are running!") kimchi_log.info('Host is going to shutdown.') -- 1.8.5.3

3 2

[PATCH] Packaging: add the basemodel.py to src/kimchi/Makefile.am
by Paulo Vital 04 Feb '14

04 Feb '14

src/kimchi/basemodel.py is a common model builder and is imported by several models from src/kimchi/model, and it is not in src/kimchi/Makefile.am, so make install, make dist and make rpm do not install or package this file. Signed-off-by: Paulo Vital <pvital(a)linux.vnet.ibm.com> --- src/kimchi/Makefile.am | 1 + 1 file changed, 1 insertion(+) diff --git a/src/kimchi/Makefile.am b/src/kimchi/Makefile.am index ffcdcad..855eb9e 100644 --- a/src/kimchi/Makefile.am +++ b/src/kimchi/Makefile.am @@ -26,6 +26,7 @@ kimchi_PYTHON = \ __init__.py \ asynctask.py \ auth.py \ + basemodel.py \ cachebust.py \ disks.py \ distroloader.py \ -- 1.8.3.1

2 1

[ RFC V2 ] Manage guest CDROM devices
by Rodrigo Trujillo 04 Feb '14

04 Feb '14

Reference: https://github.com/kimchi-project/kimchi/wiki/Todo-1.2 https://github.com/kimchi-project/kimchi/wiki/customize-VM According to discussion in other thread: _API__definition_ **URI:** /vms/*:name*/storages * **GET**: Retrieve all storages attached to the VM * **POST**: Attatch new storage to specified virtual machine * dev : The guest device name of the storage * type: The type of the storage: cdrom, disk * path: Path of ISO or DISK. ### Sub-resource: storage **URI:** /vms/*:name*/storages/*:dev* * **GET**: Retrieve the storage information * dev: The guest device name of the storage * path: Path of the storage * type: Type of the storage * **PUT**: Update storage information * name: The name of the storage in the VM * path: Path of ISO or DISK * **DELETE**: Remove the storage _User __possibilities_ * List cdroms (GET - collection) * Add new cdrom (will be used to attach cd) (POST - collection) * Lookup information of a cdrom (GET - resource) * Update parameters of a cdrom (PUT - resource) * Delete a cdrom (will be used to detach) (DELETE - resource) - Will not be possible to detach and attach new cd on the same device. This operation will be done removing and adding new storage - Eject cd operation will not be provided _Perfomance/Tunning_ * Still to be defined

1 0

[PATCH V3 0/4] (WIP) Storagepool SCSI/FC
by Rodrigo Trujillo 03 Feb '14

03 Feb '14

Still working in test cases and mockmodel V3: - Changed API to only receive the scsi host name when creating new pool - Changed API to require LUN when creating new VM on SCSI pool - Created feature test and removed libvirt test function - Rebased with new model structure - Added error function handlers to UIs - Fixed LUN selection window V2: - Implements Fibre Channel devices discover in the host - Allow vms_create receive a volume to create the disk (if pool is SCSI) - Create basic UI to select SCSI Host when creating SCSI FC pool - Draft of UI to select LUN to create new VM when template has a SCSI pool configured. (Need help of UI guys here!) Rodrigo Trujillo (4): Storagepool SCSI/FC: Implement node devices API backend Storagepool SCSI/FC: Backend implementation Storagepool SCSI/FC: Implement UI for FC scsi_host pool Storagepool SCSI/FC: Modifies UI flow to select a LUN to new VM docs/API.md | 5 ++- src/kimchi/API.json | 14 ++++++- src/kimchi/control/host.py | 16 +++++++ src/kimchi/featuretests.py | 27 ++++++++++++ src/kimchi/model/config.py | 5 ++- src/kimchi/model/host.py | 54 ++++++++++++++++++++++++ src/kimchi/model/libvirtstoragepool.py | 45 ++++++++++++++++++++ src/kimchi/model/storagepools.py | 22 ++++++++-- src/kimchi/model/templates.py | 5 +++ src/kimchi/model/vms.py | 25 ++++++++++- src/kimchi/vmtemplate.py | 31 +++++++++++++- ui/js/src/kimchi.api.js | 24 +++++++++++ ui/js/src/kimchi.guest_add_main.js | 72 ++++++++++++++++++++++++++++++-- ui/js/src/kimchi.storagepool_add_main.js | 46 +++++++++++++++++++- ui/pages/i18n.html.tmpl | 4 ++ ui/pages/storagepool-add.html.tmpl | 12 ++++++ 16 files changed, 392 insertions(+), 15 deletions(-) -- 1.8.5.3

1 4

[PATCH V2 0/7] (WIP) Storagepool SCSI/FC
by Rodrigo Trujillo 03 Feb '14

03 Feb '14

There is still some work to be done in this functionality. V2: - Implements Fibre Channel devices discover in the host - Allow vms_create receive a volume to create the disk (if pool is SCSI) - Create basic UI to select SCSI Host when creating SCSI FC pool - Draft of UI to select LUN to create new VM when template has a SCSI pool configured. (Need help of UI guys here!) Rodrigo Trujillo (7): Storagepool SCSI/FC: SCSI/Fibre Channel backend implementation Storagepool SCSI/FC: Function to check libvirt version Storagepool SCSI/FC: Assign SCSI fibre channel LUN as disk to a new guest Storagepool SCSI/FC: Implement node devices API backend Storagepool SCSI/FC: Implement UI when a FC scsi_host will be the pool Storagepool SCSI/FC: Create new VM with given SCSI FC LUN (backend) Storagepool SCSI/FC: Draft UI to allow user to select a LUN to new VM docs/API.md | 7 ++ src/kimchi/API.json | 19 ++++- src/kimchi/control/devices.py | 40 +++++++++++ src/kimchi/control/host.py | 3 + src/kimchi/model.py | 117 +++++++++++++++++++++++++++++-- src/kimchi/utils.py | 18 ++++- src/kimchi/vmtemplate.py | 48 ++++++++++++- ui/js/src/kimchi.api.js | 24 +++++++ ui/js/src/kimchi.guest_add_main.js | 32 +++++++++ ui/js/src/kimchi.storagepool_add_main.js | 40 ++++++++++- ui/pages/i18n.html.tmpl | 1 + ui/pages/storagepool-add.html.tmpl | 12 ++++ 12 files changed, 350 insertions(+), 11 deletions(-) create mode 100644 src/kimchi/control/devices.py -- 1.8.4.2

4 34