February 2014 - Kimchi-devel - oVirt List Archives

[PATCH 0/5 V2] Host's repositories management support
by Paulo Vital 13 Feb '14

13 Feb '14

V1 -> V2: * Removed the enable/disable from docs/API.md and src/kimchi/API.json * fixed wrong URI on test_rest.py * PEP8 compatibility * Updated repositories.py to make consistent with docs/API.md * Raised correct exceptions * Changed model/host.py and mockmodel.py to generate repo_id if not provided * Changed mockmodel.py to declare MockRepositories class V1: This patch set provides support to host's repositories management operations. At this point, an agnostic class is providing support to backend and REST API operations. In addition, YUM (for RHEL, Fedora, SLES and OpenSuse) and APT (for Debian and Ubuntu) specific classes are provided to support the operation os each software update system. To test the backend execute the following commands: $ cd tests $ sudo ./run_tests.sh test_model.ModelTests.test_repository_create $ sudo ./run_tests.sh test_model.ModelTests.test_repository_update $ sudo ./run_tests.sh test_model.ModelTests.test_repository_disable_enable To test the REST API, execute the following commands (all them are agnostic of the host's distro): 1) Get list of all repositories enabled in the host: $ curl -u <USER> -H 'Content-type: application/json' -H 'Accept: application/json' http://localhost:8000/host/repositories/ -X GET 2) Create a new repository: $ curl -u <USER> -H 'Content-type: application/json' -H 'Accept: application/json' http://localhost:8000/host/repositories/ -X POST -d' { "repo_id": "fedora-fake", "baseurl": "http://www.fedora.org", "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-fake-19" } ' 3) Get information from a specific repository: $ curl -u <USER> -H 'Content-type: application/json' -H 'Accept: application/json' http://localhost:8000/host/repositories/fedora-fake 4) Update a specific repository: $ curl -u <USER> -H 'Content-type: application/json' -H 'Accept: application/json' http://localhost:8000/host/repositories/fedora-fake -X PUT -d' { "repo_id":"fedora-fake", "repo_name":"Fedora 19 FAKEs", "baseurl": "http://www.fedora.org/downloads" } ' 5) Disable a specific repository: $ curl -u <USER> -H 'Content-type: application/json' -H 'Accept: application/json' http://localhost:8000/host/repositories/fedora-fake/disable -X POST -d '' 6) Enable a specific repository: $ curl -u <USER> -H 'Content-type: application/json' -H 'Accept: application/json' http://localhost:8000/host/repositories/fedora-fake/enable -X POST -d '' 7) Delete a specific repository: $ curl -u <USER> -H 'Content-type: application/json' -H 'Accept: application/json' http://localhost:8000/host/repositories/fedora-fake -X DELETE Paulo Vital (5): Host's repositories management: Update API.md Host's repositories management: Update REST API Host's repositories management: Update backend. Host's repositories management: Update Makefile Host's repositories management: Update test-cases. Makefile.am | 1 + docs/API.md | 77 ++++++ src/kimchi/API.json | 59 +++++ src/kimchi/Makefile.am | 1 + src/kimchi/control/host.py | 21 ++ src/kimchi/mockmodel.py | 140 +++++++++++ src/kimchi/model/host.py | 56 ++++- src/kimchi/repositories.py | 590 +++++++++++++++++++++++++++++++++++++++++++++ tests/test_model.py | 100 ++++++++ tests/test_rest.py | 33 +++ 10 files changed, 1077 insertions(+), 1 deletion(-) create mode 100644 src/kimchi/repositories.py -- 1.8.3.1

2 8

[PATCH 0/5] Initial authorization support
by Leonardo Garcia 13 Feb '14

13 Feb '14

From: Leonardo Garcia <lagarcia(a)br.ibm.com> Current, in Kimchi, no real authorization support is implemented. We do have authentication support, and, apart from that, no other kind of control is provided in order to authorize or not a Kimchi user to access its features. IOW, today, a user can access everything or nothing Kimchi provides. This patch series tries to implement an initial support for user authorization in Kimchi back-end. Some has already been discussed in the community about this feature [1, 2, 3]. The RFC proposed in [2] and the WIP sent in [3] seems to be diverging from the simple proposal first put in [1] and sustained in replies to [2]. So, the purpose of this patch series is to try to be, as much as possible, compliant to [1] and keep things as simple as possible. In summary, that means we will identify users as having sudo rights or not. This information will be passed to the UI by the /login REST API during logging in. With this information the UI will be able to decide which components (tabs, buttons, etc.) it will show to the user. Additionally, an infrastructure was also built in order to identify a REST API as one that needs sudo rights or not to be accessed. So, if the UI, for some reason, tries to access a REST API in a session whose user does not have sudo rights, the REST API call will return HTTP error 401. [1] https://github.com/kimchi-project/kimchi/wiki/authorization [2] http://lists.ovirt.org/pipermail/kimchi-devel/2014-January/001218.html [3] http://lists.ovirt.org/pipermail/kimchi-devel/2014-January/001898.html Leonardo Garcia (5): Improve parse_cmd_output to split lines based on a given separator. Code cleanup. Find out user groups and sudo status during login. Enhance UrlSubNode decorator and kimchiauth tool to check for sudo rights Limit REST API /host to user with sudo rights. src/kimchi/auth.py | 79 +++++++++++++++++++++++++++++++++++++-------- src/kimchi/control/host.py | 2 +- src/kimchi/control/utils.py | 4 ++- src/kimchi/root.py | 4 +-- src/kimchi/server.py | 2 ++ src/kimchi/utils.py | 4 +-- 6 files changed, 76 insertions(+), 19 deletions(-) -- 1.8.5.3

7 22

Error in kimchi model test
by Rodrigo Trujillo 13 Feb '14

13 Feb '14

I have seem this error in my machine: - Fedora 20 - x86_64 - libvirt-1.1.3.3-2.fc20.x86_64 Can anyone try this test please, and let me know if you see the error ? *************************************************************************** [rotru@dhcp-9-18-235-188 tests]$ sudo ./run_tests.sh test_model.ModelTests.test_delete_running_vm libvirt: DBus Utils error : Invalid argument E ====================================================================== ERROR: test_delete_running_vm (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 630, in test_delete_running_vm inst.vm_start(u'kīмkhī-∨м') File "/home/rotru/Projetos/Kimchi/kimchi/src/kimchi/model/vms.py", line 371, in start dom.create() File "/home/rotru/Projetos/Kimchi/kimchi/src/kimchi/model/libvirtconnection.py", line 65, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 708, in create if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self) libvirtError: Invalid argument ---------------------------------------------------------------------- Ran 1 test in 0.369s FAILED (errors=1) *************************************************************************** I could get rid of this error changing the name of the vm created. The current name uses weird characters and unicode, but this in intentional to check unicode in libvirt. You can see the code in commit '5f04ac0750aacc97befa0088ae69fd036236349a' Thanks, Rodrigo Trujillo

3 4

[PATCH V3] Fix host debug report for Fedora 20
by Rodrigo Trujillo 13 Feb '14

13 Feb '14

The tool sosreport does not save the report file in /tmp in F20, this causes a error in Kimchi, which has /tmp hardcoded. This fixes this problem searching report files in /var/tmp too. Notice that is possible to pass --tmp-dir as the directory to save the report file, but this option is crashing in Fedora 20, so the code is not using it. This patch also improves the logging. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/model/debugreports.py | 57 ++++++++++++++++++++++++---------------- 1 file changed, 35 insertions(+), 22 deletions(-) diff --git a/src/kimchi/model/debugreports.py b/src/kimchi/model/debugreports.py index a1cb19c..e48fe1a 100644 --- a/src/kimchi/model/debugreports.py +++ b/src/kimchi/model/debugreports.py @@ -32,6 +32,7 @@ from kimchi import config from kimchi.exception import NotFoundError, OperationFailed from kimchi.model.tasks import TaskModel from kimchi.utils import add_task, kimchi_log +from kimchi.utils import run_command class DebugReportsModel(object): @@ -63,36 +64,48 @@ class DebugReportsModel(object): @staticmethod def sosreport_generate(cb, name): - command = 'sosreport --batch --name "%s"' % name try: - retcode = subprocess.call(command, shell=True, - stdout=subprocess.PIPE) + command = ['sosreport', '--batch', '--name=%s' % name] + output, error, retcode = run_command(command) + if retcode < 0: raise OperationFailed('Command terminated with signal') elif retcode > 0: raise OperationFailed('Command failed: rc = %i' % retcode) - pattern = '/tmp/sosreport-%s-*' % name - for reportFile in glob.glob(pattern): - if not fnmatch.fnmatch(reportFile, '*.md5'): - output = reportFile + + # SOSREPORT might create file in /tmp or /var/tmp + # FIXME: The right way should be passing the tar.xz file directory + # though the parameter '--tmp-dir', but it is failing in Fedora 20 + patterns = ['/tmp/sosreport-%s-*', '/var/tmp/sosreport-%s-*'] + reports = [] + reportFile = None + for p in patterns: + reports = reports + [f for f in glob.glob(p % name)] + for f in reports: + if not fnmatch.fnmatch(f, '*.md5'): + reportFile = f break - else: - # sosreport tends to change the name mangling rule and - # compression file format between different releases. - # It's possible to fail to match a report file even sosreport - # runs successfully. In future we might have a general name - # mangling function in kimchi to format the name before passing - # it to sosreport. Then we can delete this exception. - raise OperationFailed('Can not find generated debug report ' - 'named by %s' % pattern) - ext = output.split('.', 1)[1] + # Some error in sosreport happened + if reportFile is None: + kimchi_log.error('Debug report file not found. See sosreport ' + 'output for detail:\n%s', output) + raise OperationFailed('Debug report file not found.') + + md5_report_file = reportFile + '.md5' + report_file_extension = '.' + reportFile.split('.', 1)[1] path = config.get_debugreports_path() - target = os.path.join(path, name) - target_file = '%s.%s' % (target, ext) - shutil.move(output, target_file) - os.remove('%s.md5' % output) + target = os.path.join(path, name + report_file_extension) + # Moving report + msg = 'Moving debug report file "%s" to "%s"' % (reportFile, target) + kimchi_log.info(msg) + shutil.move(reportFile, target) + # Deleting md5 + msg = 'Deleting report md5 file: "%s"' % (md5_report_file) + kimchi_log.info(msg) + md5 = open(md5_report_file).read().strip() + kimchi_log.info('Md5 file content: "%s"', md5) + os.remove(md5_report_file) cb('OK', True) - return except OSError: -- 1.8.5.3

2 1

[PATCH v2 0/3] Documentation improvements
by Leonardo Garcia 13 Feb '14

13 Feb '14

From: Leonardo Garcia <lagarcia(a)br.ibm.com> Leonardo Garcia (3): Add hypervisor information on README.md. Limit README.md to 80 columns. Add mailing list information. CONTRIBUTE.md | 6 +++--- docs/README.md | 31 +++++++++++++++++++++++-------- 2 files changed, 26 insertions(+), 11 deletions(-) -- 1.8.5.3

4 6

[PATCH] : Dir Storagepool path: Git hub issue:316
by Pradeep K Surisetty 13 Feb '14

13 Feb '14

Defining a New DIR based Storage Pool fails, if user appends "/" to path. For ex: If user use "/home/user/vms/" instead of "/home/user/vms", it fails with " Not a valid linux path" Issue: https://github.com/kimchi-project/kimchi/issues/316 Signed-off-by: Chandan Kumar<psuriset(a)linux.vnet.ibm.com> Reviewed-by: Pradeep K Surisetty <psuriset(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.storagepool_add_main.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/js/src/kimchi.storagepool_add_main.js b/ui/js/src/kimchi.storagepool_add_main.js index e5922b3..a5a12f7 100644 --- a/ui/js/src/kimchi.storagepool_add_main.js +++ b/ui/js/src/kimchi.storagepool_add_main.js @@ -167,7 +167,7 @@ kimchi.validateDirForm = function () { kimchi.message.error(i18n['msg.pool.edit.path.blank']); return false; } - if (!/((\/([0-9a-zA-Z-_\.]+)))$/.test(path)) { + if (!/((\/([0-9a-zA-Z-_\.]+)))(\/|[0-9a-zA-Z-_\.])$/.test(path)) { kimchi.message.error(i18n['msg.validate.pool.edit.path']); return false; } -- 1.8.3.1

3 3

[PATCH v3 0/4] CDROM Management
by Daniel Barboza 13 Feb '14

13 Feb '14

From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com> This patch set implements host storage devices management. It implements full CDROM device add, remove and update functionality. It implements basic Disk functionalities. To test this contribution: - get info of all storages (cdrom and disks) of a VM curl -u <user> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8000/vms/<vm_name>/storages -X GET - add a new cdrom to a VM curl -u <user> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8000/vms/<vm_name>/storages -X POST -d'{"type": "cdrom", "path": "<path_to_iso>"}' - get specific info from cdrom device curl -u <user> -H "Content-Type: application/json" -H "Accet: application/json" http://localhost:8000/vms/<vm_name>/storages/<dev_name> -X GET - delete a cdrom device from a VM curl -u <user> -H "Content-Type: application/json" -H "Accet: application/json" http://localhost:8000/vms/<vm_name>/storages/<dev_name> -X DELETE - update a cdrom device from a VM curl -u <user> -H "Content-Type: application/json" -H "Accet: application/json" http://localhost:8000/vms/<vm_name>/storages/<dev_name> -X PUT -d '{"path":<path_to_iso>}' Changelog: V3: - Using lxml to parse the XML information in model/vms.py - Addressed comments/suggestions from the ML in other files V2: - Add devices to mockmodel and add test cases to rest API - Assign name automatically to new devices, if not passed by user - Fix minor errors Rodrigo Trujillo (4): CDROM Management: Add storage sub-collection to sub-resource to guest resource CDROM Management: Update controller and API.json for guest storages CDROM Management: Devices management model implementation CDROM Management: Guest vm storage devices mockmodel and rest api test cases docs/API.md | 21 +++++ src/kimchi/API.json | 33 +++++++ src/kimchi/control/vm/storages.py | 49 ++++++++++ src/kimchi/mockmodel.py | 66 ++++++++++++++ src/kimchi/model/vms.py | 184 +++++++++++++++++++++++++++++++++++++- src/kimchi/xmlutils.py | 5 ++ tests/test_rest.py | 74 +++++++++++++++ 7 files changed, 428 insertions(+), 4 deletions(-) create mode 100644 src/kimchi/control/vm/storages.py -- 1.8.3.1

5 17

[PATCH 0/5] template cloning
by shaohef＠linux.vnet.ibm.com 12 Feb '14

12 Feb '14

From: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> The user may clone a template from an existing template with different name. Later he can customize some parts of the template to save the effort to create a full new template. For example, he can update the network of the template cloned to have a new different template. ShaoHe Feng (5): clone template: update API.md API.json: move templates_create close to template_update clone template: update controller clone template: update model and mockmodel clone template: update test case docs/API.md | 27 +++++++++- src/kimchi/API.json | 112 ++++++++++++++++++++++++++++++++++------ src/kimchi/control/templates.py | 1 + src/kimchi/mockmodel.py | 19 +++++++ src/kimchi/model/templates.py | 20 +++++++ tests/test_rest.py | 34 ++++++++++++ 6 files changed, 196 insertions(+), 17 deletions(-) -- 1.8.4.2

4 13

[PATCH V5 0/6] Storagepool SCSI/FC
by Rodrigo Trujillo 12 Feb '14

12 Feb '14

V5: - Add entries to API.md related to host pci devices API - Fix issues with wwpn and wwnn fields in lookup returns - Fix issues in mock model - Fix UI minor issues - Add UI strings to i18n - Refactored code accordig to code review suggestions V4: - Implements mockmodel and tests - Fix UI - Fix other minor issues V3: - Changed API to only receive the scsi host name when creating new pool - Changed API to require LUN when creating new VM on SCSI pool - Created feature test and removed libvirt test function - Rebased with new model structure - Added error function handlers to UIs - Fixed LUN selection window V2: - Implements Fibre Channel devices discover in the host - Allow vms_create receive a volume to create the disk (if pool is SCSI) - Create basic UI to select SCSI Host when creating SCSI FC pool - Draft of UI to select LUN to create new VM when template has a SCSI pool configured. (Need help of UI guys here!) Rodrigo Trujillo (5): Storagepool SCSI/FC: Implement node devices API backend Storagepool SCSI/FC: Backend implementation Storagepool SCSI/FC: Implement UI for FC scsi_host pool Storagepool SCSI/FC: Modifies UI flow to select a LUN to new VM Storagepool SCSI/FC: Modifies mockmodel and implements tests for FC pool Rodrigo Trujillo (6): Storagepool SCSI/FC: Implement node devices API backend Storagepool SCSI/FC: API.md entries related to host pci devices Storagepool SCSI/FC: Backend implementation Storagepool SCSI/FC: Implement UI for FC scsi_host pool Storagepool SCSI/FC: Modifies UI flow to select a LUN to new VM Storagepool SCSI/FC: Modifies mockmodel and implements tests for FC pool docs/API.md | 34 ++++++++++++++- src/kimchi/API.json | 14 +++++- src/kimchi/control/host.py | 16 +++++++ src/kimchi/featuretests.py | 29 +++++++++++++ src/kimchi/mockmodel.py | 55 +++++++++++++++++++++--- src/kimchi/model/config.py | 2 + src/kimchi/model/host.py | 55 ++++++++++++++++++++++++ src/kimchi/model/libvirtstoragepool.py | 48 ++++++++++++++++++++- src/kimchi/model/storagepools.py | 30 +++++++++++-- src/kimchi/model/templates.py | 5 +++ src/kimchi/model/vms.py | 27 ++++++++++-- src/kimchi/vmtemplate.py | 31 +++++++++++++- tests/test_rest.py | 62 +++++++++++++++++++++++++++ tests/test_storagepool.py | 21 +++++++++ ui/js/src/kimchi.api.js | 24 +++++++++++ ui/js/src/kimchi.guest_add_main.js | 73 ++++++++++++++++++++++++++++++-- ui/js/src/kimchi.storagepool_add_main.js | 46 +++++++++++++++++++- ui/pages/i18n.html.tmpl | 8 ++++ ui/pages/storagepool-add.html.tmpl | 12 ++++++ 19 files changed, 569 insertions(+), 23 deletions(-) -- 1.8.5.3

2 12

[PATCH 0/8 V2] Refactor exception
by Aline Manera 12 Feb '14

12 Feb '14

From: Aline Manera <alinefm(a)br.ibm.com> V1 -> V2: - Add missing messages to i18n.py - Fix inconsistency with error message KCHTMPL6001M - Replace "Storage Pool" to "storage pool" - Replace "Storage Volume" to "storage volume" - Replace "VM" to "virtual machine" - And other minor changes suggested by Leonardo This patch set includes: - Create a common Exception class (KimchiError) to translate the error message and proper set the parameters - Create a i18n.py file with all messages used on backend - Update build process to add the messages in this file to .po files - Update UI to show the message received from backend to the user - Update UI messages to add a code for them - Display error message like: <code>: <msg> Aline Manera (8): refactor exception: Create a common Exception to translate error messages refactor exception: Add messages to be translated on backend refactor exception: Set error messages for jsonschema validation refator exception: Update control to raise the exception message refactor exception: Update all exceptions refactor exception: Update gen-pot script to get messages from i18n.py refactor exception: Update UI references refactor exception: Update all po files plugins/sample/API.json | 23 +- plugins/sample/Makefile.am | 1 + plugins/sample/__init__.py | 7 +- plugins/sample/i18n.py | 44 ++ plugins/sample/model.py | 12 +- po/POTFILES.in | 1 + po/en_US.po | 739 ++++++++++++++++++++++++------ po/gen-pot.in | 8 +- po/kimchi.pot | 664 ++++++++++++++++++++++----- po/pt_BR.po | 738 +++++++++++++++++++++++------ po/zh_CN.po | 726 +++++++++++++++++++++++------ src/kimchi/API.json | 144 ++++-- src/kimchi/Makefile.am | 1 + src/kimchi/asynctask.py | 4 +- src/kimchi/auth.py | 8 +- src/kimchi/control/base.py | 116 ++--- src/kimchi/control/utils.py | 14 +- src/kimchi/disks.py | 5 +- src/kimchi/distroloader.py | 5 +- src/kimchi/exception.py | 42 +- src/kimchi/i18n.py | 181 ++++++++ src/kimchi/iscsi.py | 6 +- src/kimchi/isoinfo.py | 17 +- src/kimchi/mockmodel.py | 113 +++-- src/kimchi/model/config.py | 2 +- src/kimchi/model/debugreports.py | 18 +- src/kimchi/model/host.py | 10 +- src/kimchi/model/interfaces.py | 4 +- src/kimchi/model/libvirtstoragepool.py | 14 +- src/kimchi/model/networks.py | 37 +- src/kimchi/model/storagepools.py | 39 +- src/kimchi/model/storageservers.py | 2 +- src/kimchi/model/storagevolumes.py | 37 +- src/kimchi/model/templates.py | 44 +- src/kimchi/model/utils.py | 2 +- src/kimchi/model/vmifaces.py | 15 +- src/kimchi/model/vms.py | 28 +- src/kimchi/objectstore.py | 4 +- src/kimchi/root.py | 10 +- src/kimchi/template.py | 2 +- src/kimchi/utils.py | 12 +- src/kimchi/vmtemplate.py | 6 +- tests/test_exception.py | 12 +- tests/test_rest.py | 4 +- tests/utils.py | 5 +- ui/js/src/kimchi.api.js | 4 +- ui/js/src/kimchi.guest_add_main.js | 7 +- ui/js/src/kimchi.guest_main.js | 32 +- ui/js/src/kimchi.host.js | 46 +- ui/js/src/kimchi.line-chart.js | 2 +- ui/js/src/kimchi.login_window.js | 8 +- ui/js/src/kimchi.main.js | 6 +- ui/js/src/kimchi.message.js | 4 + ui/js/src/kimchi.network.js | 22 +- ui/js/src/kimchi.report_add_main.js | 8 +- ui/js/src/kimchi.storage_main.js | 18 +- ui/js/src/kimchi.storagepool_add_main.js | 28 +- ui/js/src/kimchi.template_add_main.js | 18 +- ui/js/src/kimchi.template_main.js | 16 +- ui/js/widgets/filter-select.js | 4 +- ui/js/widgets/select-menu.js | 4 +- ui/pages/guest-add.html.tmpl | 2 +- ui/pages/i18n.html.tmpl | 172 ++++--- ui/pages/storagepool-add.html.tmpl | 4 +- ui/pages/tabs/network.html.tmpl | 2 +- 65 files changed, 3243 insertions(+), 1090 deletions(-) create mode 100644 plugins/sample/i18n.py create mode 100644 src/kimchi/i18n.py -- 1.7.10.4

2 17