December 2015 - Kimchi-devel - oVirt List Archives

[PATCH V2] Create new storage volume when attaching disk to a guest.
by pvital＠linux.vnet.ibm.com 29 Jan '16

29 Jan '16

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> Changes in V2: * Updated test cases * Modified dictionary return of parameters V1: Add back-end support to create new storage volume (new virtual disk) when attaching disk to a guest created before. There are three essential parameters to create the new volume: * vol: Storage volume name of disk image, that should be 'new_vol'. * capacity: The total space which can be used to store new volumes. The unit is bytes. * format: The format of the defined Storage Volume. Only used when creating a storage volume with 'capacity'. To test this patch: $ curl -k -u test -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/kimchi/vms/kimchi-vm-new/storages' -X POST -d '{ "vol": "new_vol", "type": "disk", "pool": "default", "format": "qcow2", "capacity": 1024 }' Enter host password for user 'test': { "bus":"virtio", "path":"/var/lib/libvirt/images/2477bfd8-a9e2-4887-a683-e89015b3ba11 -1.img", "type":"disk", "dev":"vdb", "format":"qcow2" } $ curl -k -u test -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/kimchi/vms/kimchi-vm-new/storages' -X GET Enter host password for user 'test': [ { "bus":"virtio", "path":"/var/lib/libvirt/images/2477bfd8-a9e2-4887-a683 -e89015b3ba11-0.img", "type":"disk", "dev":"vda", "format":"qcow2" }, { "bus":"virtio", "path":"/var/lib/libvirt/images/2477bfd8-a9e2-4887-a683 -e89015b3ba11-1.img", "type":"disk", "dev":"vdb", "format":"qcow2" }, { "bus":"ide", "path":"/var/lib/kimchi/tests/ubuntu14.04.iso", "type":"cdrom", "dev":"hdc", "format":"raw" } ] Paulo Vital (1): Create new storage volume when attaching disk to a guest. src/wok/plugins/kimchi/API.json | 12 ++++++++++++ src/wok/plugins/kimchi/docs/API.md | 6 +++++- src/wok/plugins/kimchi/i18n.py | 1 + src/wok/plugins/kimchi/model/vmstorages.py | 28 +++++++++++++++++++++++++++- src/wok/plugins/kimchi/tests/test_model.py | 25 +++++++++++++++++++++++++ 5 files changed, 70 insertions(+), 2 deletions(-) -- 2.4.3

5 7

[RFC] Usage of Bootstrap Switch (http://www.bootstrap-switch.org/) for plugins ginger new ui developments
by Atreye Mukhopadhyay 22 Jan '16

22 Jan '16

Hi All, I would like propose the use of Bootstrap Switch (https://github.com/nostalgiaz/bootstrap-switch) for showing checkbox/radio element in form of switch button .This toggle switch can be used for showing status of certain component in form of switch and trigger action based on the current state. Official Github location of Bootstrap Switch implementation is @ https://github.com/nostalgiaz/bootstrap-switch. Bootstrap Switch code is of license type MIT so I guess it can be used safely. Please feel free to comment on this topic and have your opinion. Thanks & Regards, Atreyee

6 7

Re: [Kimchi-devel] Guest disk: Allocate storage volume when attaching disk to vm
by Aline Manera 12 Jan '16

12 Jan '16

Adding Samuel and Andre to this discussion and also forward to Kimchi ML. On 16/11/2015 14:21, Aline Manera wrote: > > Hi Socorro, > > On 12/11/2015 21:20, Socorro Stoppler wrote: >> Hi Aline, >> >> I chatted w/Paulo about the UI portion of this. He stated that the >> user has the option to select to add a new volume (in which the the >> UI must execute two requests: one to create a volume, and other to >> attach the new volume to the guest being edited). So in this screen >> (I know it's the old UI but this shows better than current new UI :)): >> >> >> >> What do you think of 3. being radio buttons? - > > I think 2 should be that radio buttons. > > So: > > 1. Device type > <disk> > > 2. ( ) Create a new disk image > > ( ) Select existing disk image > > 3. Storage pool > <pools> > > 4. Capacity or Volume (depends on selection on 2.) > > Below is how virt-manage does it. > > > >> <New Volume> - >> Volume: Storage volume name of 'new_vol' --- we prob do not >> need to show this in the UI, right? >> Capacity: in bytes >> Format: qcow2 -- is this the only option we want to provide? >> <Create Button> >> >> <Existing Volume> - which consists of current combobox above >> >> Also, if user selects to do the new volume, then created it, but >> decided that they didn't want to attach it, is that ok? Not sure of >> the different scenarios we have >> or if I'm totally missing the intent here. >> >> Let me know if you had envisioned something else for this UI. >> >> Thanks >> -Socorro >> >> >

3 4

[PATCH v2][Wok] Change plugins get_list to return only loaded plugins
by Rodrigo Trujillo 07 Jan '16

07 Jan '16

When using multiple plugins, it is possible that some plugin has a problem and is not loaded (like when there is a missing module). In this cases, Wok is not going to load the plugin in cherrypy configuration, but plugins API return the missing plugin name. When this occours, UI becomes broken and it is not possible to use any other plugin, because Wok UI is blank. This patch fix this problem, returning only plugins names that were loaded by Wok. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/wok/model/plugins.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/wok/model/plugins.py b/src/wok/model/plugins.py index 3a5a89e..85d60b6 100644 --- a/src/wok/model/plugins.py +++ b/src/wok/model/plugins.py @@ -19,6 +19,8 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +import cherrypy + from wok.utils import get_enabled_plugins @@ -27,4 +29,7 @@ class PluginsModel(object): pass def get_list(self): - return [plugin for (plugin, config) in get_enabled_plugins()] + # Will only return plugins that were loaded correctly by WOK and are + # properly configured in cherrypy + return [plugin for (plugin, config) in get_enabled_plugins() + if config.get('wok').get('uri') in cherrypy.tree.apps.keys()] -- 2.1.0

3 3

[PATCH V5] [Wok] Make run_command log its output into a file.
by pvital＠linux.vnet.ibm.com 07 Jan '16

07 Jan '16

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> V4 -> V5: - Added try/exception to control file IO. V3 -> V4: - PEP8 fixes. V2 -> V3: - Changed the place of the log_file creation control. V1 -> V2: - Removed fixed log file, changing parameter of run_command() method to accept the path of the log file instead of a boolean. V1: Adds the ability of run_command to log the output of a console command into the file /tmp/wok_tee_log_file, like the "tee" command does. Using this patch, a command that takes too many time to execute can be monitored by reading the log file while other tasks are executed. Paulo Vital (1): Make run_command log its output into a file. src/wok/utils.py | 50 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 48 insertions(+), 2 deletions(-) -- 2.5.0

4 5

[PATCH] [Wok] Remove cython requirment from configure.ac
by Lucio Correia 07 Jan '16

07 Jan '16

Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> --- configure.ac | 1 - 1 file changed, 1 deletion(-) diff --git a/configure.ac b/configure.ac index 692f4ac..d753b67 100644 --- a/configure.ac +++ b/configure.ac @@ -38,7 +38,6 @@ AM_INIT_AUTOMAKE([-Wno-portability]) AM_PATH_PYTHON([2.6]) AC_PATH_PROG([PEP8], [pep8], [/usr/bin/pep8]) AC_PYTHON_MODULE([unittest]) -AC_PYTHON_MODULE([cython], [fatal]) AC_SUBST([HAVE_PYMOD_UNITTEST]) AC_SUBST([PYTHON_VERSION]) AM_GNU_GETTEXT([external]) -- 1.9.1

3 3

[Kimchi v2] Identify opensuse-LEAP-42.1 ISO
by Ramon Medeiros 07 Jan '16

07 Jan '16

Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- Changes: v2: fix pep8 issues isoinfo.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/isoinfo.py b/isoinfo.py index 731c45a..c232fa8 100644 --- a/isoinfo.py +++ b/isoinfo.py @@ -111,6 +111,8 @@ iso_dir = [ 'openSUSE-DVD-i586-Build0039|openSUSE-DVD-x86_640039'), ('opensuse', '12.2', 'openSUSE-DVD-i586-Build0167|openSUSE-DVD-x86_640167'), + ('opensuse', '42.1', + 'openSUSE-Leap-42.1-DVD-x86_64026|openSUSE-Leap-42.1-NET-x86_64026'), ('rhel', '4.8', 'RHEL/4-U8'), ('rhel', lambda m: m.group(2), 'RHEL(-LE)?[_/-](\d+\.\d+)'), ('debian', lambda m: m.group(1), 'Debian (\d+\.\d+)'), -- 2.1.0

3 2

Aline is out on vacations, Daniel is taking temp maintainership ***use the temp repo*** !
by Daniel Henrique Barboza 07 Jan '16

07 Jan '16

Hello everyone, Aline is out on well deserved vacations. I'll be her replacement while she's out. As we did earlier this year, I'll resume her duties in the community reviewing and applying patches, but I'll do it so in my fork @ github: WoK temp repo: https://github.com/danielhb/wok.git Kimchi temp repo: https://github.com/danielhb/kimchi.git So, in these 2-3 weeks she's out, use these temp repos to get the latest changes in Wok and Kimchi. Aline will merge this fork with the official WoK/Kimchi repo when she comes back. All coding standards and best practices she enforces in WoK/Kimchi contributions will be enforced by me as well. So no need to worry, but no reason to get lazy either! So, long story short, use those 2 temp repos and keep doing the same thing as always. Happy coding! Daniel

2 1

Re: [Kimchi-devel] [PATCH] Fixed the comments for debugreports in report tool and removed the check for platform s390x and introduced wok_log errors in log_errors.
by Megha Smriti 29 Dec '15

29 Dec '15

Hi Aline, Please find my comments below. Regards, Megha > Subject: Re: [Kimchi-devel] [PATCH] Fixed the comments for > debugreports in report tool and removed the check for platform s390x > and introduced wok_log errors in log_errors. > Date: Mon, 30 Nov 2015 12:03:46 -0200 > From: Aline Manera <alinefm(a)linux.vnet.ibm.com> > To: mesmriti(a)linux.vnet.ibm.com, Kimchi Devel <kimchi-devel(a)ovirt.org> > > > > Almost there! =) > > Some comments below > > On 19/11/2015 02:56,mesmriti@linux.vnet.ibm.com wrote: > > From: root<root(a)localhost.localdomain> > > > > Signed-off-by:mesmriti@linux.vnet.ibm.com > > --- > > src/wok/plugins/gingerbase/i18n.py | 4 +- > > src/wok/plugins/gingerbase/model/debugreports.py | 181 ++++++++++++++++++----- > > 2 files changed, 143 insertions(+), 42 deletions(-) > > > > diff --git a/src/wok/plugins/gingerbase/i18n.py b/src/wok/plugins/gingerbase/i18n.py > > index af75c70..6ff783b 100644 > > --- a/src/wok/plugins/gingerbase/i18n.py > > +++ b/src/wok/plugins/gingerbase/i18n.py > > @@ -36,7 +36,9 @@ messages = { > > "GGBDR0007E": _("Debug report name must be a string. Only letters, digits, underscore ('_') and " > > "hyphen ('-') are allowed."), > > "GGBDR0008E": _("The debug report with specified name \"%(name)s\" already exists. Please use another one."), > > - > > + "GGBDR0009E": _("Unable to create dbginfo report with %(retcode)s. Details: %(err)s"), > > + "GGBDR0010E": _("Unable to compress the final debug report tar file with %(retcode)s. Details: %(error)s"), > > + "GGBDR0011E": _("Unable to generate final debug report %(name)s. Details: %(err)s"), > > "GGBHOST0001E": _("Unable to shutdown host machine as there are running virtual machines"), > > "GGBHOST0002E": _("Unable to reboot host machine as there are running virtual machines"), > > "GGBHOST0003E": _("There may be virtual machines running on the host"), > > diff --git a/src/wok/plugins/gingerbase/model/debugreports.py b/src/wok/plugins/gingerbase/model/debugreports.py > > index 94ab7fe..c6b60e8 100644 > > --- a/src/wok/plugins/gingerbase/model/debugreports.py > > +++ b/src/wok/plugins/gingerbase/model/debugreports.py > > @@ -19,13 +19,13 @@ > > # License along with this library; if not, write to the Free Software > > # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > > > > -import fnmatch > > import glob > > import logging > > import os > > import shutil > > import subprocess > > import time > > +import re > > > > from wok.exception import InvalidParameter, NotFoundError, OperationFailed > > from wok.exception import WokException > > @@ -71,54 +71,107 @@ class DebugReportsModel(object): > > raise OperationFailed("GGBDR0002E") > > > > @staticmethod > > - def sosreport_generate(cb, name): > > + def debugreport_generate(cb, name): > > def log_error(e): > > - log = logging.getLogger('Model') > > - log.warning('Exception in generating debug file: %s', e) > > + wok_log = logging.getLogger('Model') > > + wok_log.warning('Exception in generating debug file: %s', e) > > > > try: > > - command = ['sosreport', '--batch', '--name=%s' % name] > > + # Sosreport generation > > + sosreport_file = sosreport_collection(name) > > + md5_report_file = sosreport_file + '.md5' > > + report_file_extension = '.' + sosreport_file.split('.', 1)[1] > > + # If the platform is a system Z machine. > > + path_debugreport = '/var/tmp/' > > + dbgreport_regex = '(\S+\s+)(' + path_debugreport + \ > > + 'DBGINFO-[\d+]{4}-[\d+]{2}' \ > > + '-[\d+]{2}-[\d+]{2}-[\d+]{2}' \ > > + '-[\d+]{2}-\w+-\d+\S+)(\s+\S+)' > > + command = ['/usr/sbin/dbginfo.sh', '-d', path_debugreport] > > output, error, retcode = run_command(command) > > - > > if retcode != 0: > > - raise OperationFailed("GGBDR0003E", {'name': name, > > - 'err': retcode}) > > - > > - # SOSREPORT might create file in /tmp or /var/tmp > > - # FIXME: The right way should be passing the tar.xz file directory > > - # though the parameter '--tmp-dir', but it is failing in Fedora 20 > > - patterns = ['/tmp/sosreport-%s-*', '/var/tmp/sosreport-%s-*'] > > - reports = [] > > - reportFile = None > > - for p in patterns: > > - reports = reports + [f for f in glob.glob(p % name)] > > - for f in reports: > > - if not fnmatch.fnmatch(f, '*.md5'): > > - reportFile = f > > + raise OperationFailed("GGBDR0009E", > > + {'retcode': retcode, 'err': error}) > > > + output = output.splitlines() > > + dbginfo_report = None > > + for line in output: > > + line = line.strip() > > + n = re.match(dbgreport_regex, line) > > + if n: > > + dbginfo_report = n.groups()[1] > > break > > As you know where dbg report will be created, you can change the > dbgreport_regex to be a string pattern and use glob. > > import glob > > file = glob.glob(dgbreport_pattern) > if len(file) == 0: > # error [Megha:] Will incorporate this in my next patch. > I included I more detailed example on how use glob below. > > > - # Some error in sosreport happened > > - if reportFile is None: > > - wok_log.error('Debug report file not found. See sosreport ' > > - 'output for detail:\n%s', output) > > - fname = (patterns[0] % name).split('/')[-1] > > - raise OperationFailed('GGBDR0004E', {'name': fname}) > > - > > - md5_report_file = reportFile + '.md5' > > - report_file_extension = '.' + reportFile.split('.', 1)[1] > > + final_tar_report_name = name + report_file_extension > > + sosreport_tar = sosreport_file.split('/', 3)[3] > > + dbginfo_tar = dbginfo_report.split('/', 3)[3] > > + msg = 'Compressing the sosreport and debug info files into ' \ > > + 'final report file' > > + wok_log.info(msg) > > + # Compressing the sosreport and dbginfo reports into one > > + # tar file > > + command = ['tar', '-cvzf', '%s' % final_tar_report_name, > > + '-C', path_debugreport, dbginfo_tar, > > + sosreport_tar] > > + output, error, retcode = run_command(command) > > + if retcode != 0: > > + raise OperationFailed("GGBDR0010E", > > + {'retcode': retcode, > > + 'error': error}) > > path = config.get_debugreports_path() > > - target = os.path.join(path, name + report_file_extension) > > - # Moving report > > - msg = 'Moving debug report file "%s" to "%s"' % (reportFile, > > - target) > > + dbg_target = os.path.join(path, > > + name + report_file_extension) > > + # Moving final tar file to debugreports path > > + msg = 'Moving final debug report file "%s" to "%s"' % \ > > + (final_tar_report_name, dbg_target) > > + wok_log.info(msg) > > + shutil.move(final_tar_report_name, dbg_target) > > + # Deleting the sosreport md5 file > > + delete_the_sosreport_md5_file(md5_report_file) > > + # Deleting the dbingo report file > > + msg = 'Deleting the dbginfo file "%s" ' \ > > + % dbginfo_report > > wok_log.info(msg) > > - shutil.move(reportFile, target) > > - # Deleting md5 > > - msg = 'Deleting report md5 file: "%s"' % (md5_report_file) > > + os.remove(dbginfo_report) > > + # Deleting the sosreport file > > + msg = 'Deleting the sosreport file "%s" ' % sosreport_file > > wok_log.info(msg) > > - with open(md5_report_file) as f: > > - md5 = f.read().strip() > > - wok_log.info('Md5 file content: "%s"', md5) > > - os.remove(md5_report_file) > > + os.remove(sosreport_file) > > + wok_log.info('The debug report file has been moved') > > + cb('OK', True) > > + return > > + > > > + except WokException as e: > > + log_error(e) > > + raise > > + > > + except OSError as e: > > + log_error(e) > > + raise > > + > > Please, raise an appropriated exception. Otherwise, the user will not be > able to understand what went wrong. > [Megha]: As far as exceptions are concerned These are the exception I think can be thrown back from the try block and it is also similar to what is done in sosreport. Please correct me if I any suggestions. > > + except Exception, e: > > + # No need to call cb to update the task status here. > > + # The task object will catch the exception raised here > > + # and update the task status there > > + log_error(e) > > + raise OperationFailed("GGBDR0011E", {'name': name, 'err': e}) > > + > > + @staticmethod > > + def sosreport_generate(cb, name): > > + def log_error(e): > > + wok_log = logging.getLogger('Model') > > + wok_log.warning('Exception in generating debug file: %s', e) > > + try: > > + # Sosreport collection > > + sosreport_file = sosreport_collection(name) > > + md5_report_file = sosreport_file + '.md5' > > + report_file_extension = '.' + sosreport_file.split('.', 1)[1] > > + path = config.get_debugreports_path() > > + sosreport_target = os.path.join(path, > > + name + report_file_extension) > > + msg = 'Moving debug report file "%s" to "%s"' \ > > + % (sosreport_file, sosreport_target) > > + wok_log.info(msg) > > + shutil.move(sosreport_file, sosreport_target) > > + delete_the_sosreport_md5_file(md5_report_file) > > cb('OK', True) > > return > > > > @@ -142,7 +195,9 @@ class DebugReportsModel(object): > > # Please add new possible debug report command here > > # and implement the report generating function > > # based on the new report command > > - report_tools = ({'cmd': 'sosreport --help', > > + report_tools = ({'cmd': '/usr/sbin/dbginfo.sh --help', > > + 'fn': DebugReportsModel.debugreport_generate}, > > + {'cmd': 'sosreport --help', > > 'fn': DebugReportsModel.sosreport_generate},) > > > > # check if the command can be found by shell one by one > > @@ -213,3 +268,47 @@ class DebugReportContentModel(object): > > > > def lookup(self, name): > > return self._debugreport.lookup(name) > > + > > + > > +def delete_the_sosreport_md5_file(md5_file): > > + """ > > + Deleting md5 file and displaying the contents of the same. > > + """ > > + msg = 'Deleting report md5 file: "%s"' % md5_file > > + wok_log.info(msg) > > + with open(md5_file) as f: > > + md5 = f.read().strip() > > + wok_log.info('Md5 file content: "%s"', md5) > > + os.remove(md5_file) > > + > > + > > +def sosreport_collection(name): > > + """ > > + Code for the collection of sosreport n the path > > + /var/tmp as specified in the command. > > + """ > > + path_sosreport = '/var/tmp/' > > + command = ['sosreport', '--batch', '--name=%s' % name, > > + '--tmp-dir=%s' % path_sosreport] > > + output, error, retcode = run_command(command) > > + if retcode != 0: > > + raise OperationFailed("GGBDR0003E", {'name': name, > > + 'err': retcode}) > > + > > > + sosreport_name_regex = '(\s+)(' + path_sosreport + \ > > + 'sosreport-' +\ > > + name + '-\d+.tar.xz)' > > + sosreport_file = None > > + output = output.splitlines() > > + for line in output: > > + if line: > > + matched_name = re.match(sosreport_name_regex, line) > > + if matched_name: > > + sosreport_file = matched_name.groups()[1] > > + break > > + # Some error in sosreport happened > > + if sosreport_file is None: > > + wok_log.error('Debug report file not found. See sosreport ' > > + 'output for detail:\n%s', output) > > + raise OperationFailed('GGBDR0004E', {'name': name}) > > + return sosreport_file > > You know the sosreport file was created under /var/tmp so you don't need > to query the command output to know the file was created or not. > > If you want to confirm the file was created you can use 'glob' > > import glob > > sosreport_pattern = path_sosreport + 'sosreport-' + name + '.tar.xz' > sosreport_file = glob.glob(sosreport_pattern) > > if len(sosreport_file) == 0: > raise OperationFailed() > > return sosreport_file[0] > > [Megha]: Will incorporate in my next patch > _______________________________________________ > Kimchi-devel mailing list > Kimchi-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/kimchi-devel > > >

2 1

[Kimchi] Remove pyflakes checking on .git
by Ramon Medeiros 29 Dec '15

29 Dec '15

.git isn't a directory on submodles, it's handler for a directory in Wok. This will avoid this kind of error: ./.git:1:7: invalid syntax gitdir: ../../../../.git/modules/src/wok/plugins/<plugin> ^ Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile.am b/Makefile.am index 1c5af68..bb3311b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -59,7 +59,7 @@ I18N_FILES = ./i18n.py \ check-local: contrib/check_i18n.py $(I18N_FILES) - find . -path './.git' -prune -type f -o \ + find . -path -prune -type f -o \ -name '*.py' -o -name '*.py.in' | xargs $(PYFLAKES) | \ while read LINE; do echo "$$LINE"; false; done -- 2.1.0

2 1