May 2016 - Kimchi-devel - oVirt List Archives

[PATCH V2][Wok 06/12] FVT: Install all the dependencies from requirements.txt and runs FVT testcases
by archus＠linux.vnet.ibm.com 30 May '16

30 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Script to run FVT test cases which also take care of installing all dependencies from requirements.txt. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/run_tests.sh.in | 92 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100755 tests/fvt/run_tests.sh.in diff --git a/tests/fvt/run_tests.sh.in b/tests/fvt/run_tests.sh.in new file mode 100755 index 0000000..ee31b95 --- /dev/null +++ b/tests/fvt/run_tests.sh.in @@ -0,0 +1,92 @@ +#!/bin/bash +# +# Project Wok +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +mkdir -p venv + +HAVE_UNITTEST=@HAVE_PYMOD_UNITTEST@ +PYTHON_VER=@PYTHON_VERSION@ + +# Verify if the required commands exists on the system +command -v virtualenv >/dev/null 2>&1 || { echo >&2 "virtualenv must be installed for your distribution. Aborting."; exit 1; } +command -v pip >/dev/null 2>&1 || { echo >&2 "pip must be installed for your distribution. Aborting."; exit 1; } + +# Get absolute path of this script +pushd `dirname $0` > /dev/null +SCRIPTPATH=`pwd -P` +reqfile=$SCRIPTPATH'/requirements.txt' + +# Start the virtual environment +virtualenv venv --no-site-packages + + +# Actiate the virtual environment +source venv/bin/activate + +while read line; do + + case "$line" in + \#*) + continue ;; # skip comments + "") + continue ;; # skip empty lines + *) + venv/bin/python$PYTHON_VER -c "import $line" > /dev/null 2>&1 + status=$? + if [ $status -ne 0 ]; then + pip install -r $reqfile # Install the required modules to run tests + break + fi + esac +done < $reqfile + +# Execute the test suite +#python registered_tests.py +#nosetests --with-html --html-file=test_report.html registered_tests.py + +if [ "$1" = "-v" ]; then + OPTS="-v" + shift +else + OPTS="" +fi + +if [ $# -ne 0 ]; then + ARGS="$@" +else + ARGS=`find -name "fvt_*.py" | xargs -I @ basename @ .py` +fi + +if [ "$HAVE_UNITTEST" != "yes" -o "$PYTHON_VER" == "2.6" ]; then + CMD="unit2" +else + CMD="python -m unittest" +fi + +LIST=($ARGS) +FVT_LIST=() +for ((i=0;i<${#LIST[@]};i++)); do + FVT_LIST+=(${LIST[$i]}) +done +PYTHONPATH=../ $CMD $OPTS ${FVT_LIST[@]} + +# Deativate the virtual environment +deactivate + +rm -fr venv -- 2.5.0

1 0

[PATCH V2][Wok 05/12] FVT: Base test class, takes care common actions required for any FVT test cases.
by archus＠linux.vnet.ibm.com 30 May '16

30 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Common action methods like creating/destoring session, authorization using wok level fvt config file, creating JSON validator and logging. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/fvt_base.py | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 tests/fvt/fvt_base.py diff --git a/tests/fvt/fvt_base.py b/tests/fvt/fvt_base.py new file mode 100644 index 0000000..8d152a5 --- /dev/null +++ b/tests/fvt/fvt_base.py @@ -0,0 +1,92 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# +# Project Wok +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +import unittest +import os +from restapilib import APISession, APIError, APIRequestError +from restapilib import Validator + +DEFAULT_CONF = os.path.dirname(os.path.abspath(__file__)) + 'config' + + +class TestBase(unittest.TestCase): + """Represents an API session setup with the host Web Services API""" + + session = None + + def __init__(self, method='runTest'): + """init with default method as runTest""" + super(TestBase, self).__init__(method) + + @classmethod + def setUpClass(cls): + """ + Hook method for setting up class fixture + before running tests in the class. + Create session and set auth to the session + """ + print '--> TestBase.setUpClass(): Create session ' + cls.session = APISession() + # Log on to the API. An exception will be raised if this fails. + cls.logging = cls.session.logging + cls.logging.debug('--> TestBase.setUpClass()') + cls.validator = Validator(cls.logging) + cls.logging.debug('TestBase.setUpClass(): Setting auth to session') + try: + cls.session.auth() + cls.logging.debug('TestBase.setUpClass(): Auth details set to ' + 'session and base URI created as %s' + % cls.session._base_uri) + except APIError, err: + print 'ERROR %s' % err + print err.__str__() + if cls.session is not None: + cls.logging.error('TestBase.setUpClass(): Ending session' + ' as API error happened') + cls.session.end_session() + finally: + cls.logging.debug('<-- TestBase.setUpClass()') + + def setUp(self): + """Hook method for setting up the test fixture before exercising it.""" + pass + + def tearDown(self): + """Hook method for deconstructing the test fixture after testing it.""" + pass + + @classmethod + def tearDownClass(cls): + """ + Hook method for deconstructing the class + fixture after running all tests in the class. + """ + if cls.session is not None: + print 'TestBase.tearDownClass(): Ending session' + cls.session.end_session() + + def get(self): + try: + resp_net = self.session.request_get_json('/') + print resp_net + except APIRequestError as err: + print 'ERROR %s' % err + print err.__str__() -- 2.5.0

1 0

[PATCH V2][Wok 04/12] FVT: Common classes/methods for API calls as per config file configuration.
by archus＠linux.vnet.ibm.com 30 May '16

30 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Wrapper classes/methods for making API calls by reading the config file configuration. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/restapilib.py | 738 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 738 insertions(+) create mode 100644 tests/fvt/restapilib.py diff --git a/tests/fvt/restapilib.py b/tests/fvt/restapilib.py new file mode 100644 index 0000000..19634f1 --- /dev/null +++ b/tests/fvt/restapilib.py @@ -0,0 +1,738 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# +# Project Wok +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +import ConfigParser +import jsonschema +import json +import requests +import logging +import os + +requests.packages.urllib3.disable_warnings() + +__all__ = ['APIError', 'APIRequestError', 'APISession', 'SessionParameters'] + +DEFAULT_CONF = os.path.dirname(os.path.abspath(__file__)) + os.sep + 'config' + +_HTTP_SUCCESS_STATUS_RANGE = range(200, 299) # All of the 2XX status codes + + +class APIError(Exception): + """Base class for exceptions raised by this module.""" + + pass + + +class APISession(object): + """Represents an API session with the host Web Services API""" + + def __init__(self, conffile=DEFAULT_CONF): + """ Creates an APISession object for use with the specified Host. + """ + self._session = None + self._base_uri = None + self.username = None + self.passwd = None + self.host = None + self.port = None + # Holds the session from requests if self._session is None. + self.session() + self.sessionparams = SessionParameters(conffile=conffile) + self.logging = self.sessionparams.getLogger() + # base URI of connected machine + self._base_uri = self.create_base_uri() + + def session(self): + """ + Create the session object for API Request if no existing session. + """ + if self._session is not None: + raise APIError('Already have a session') + + self._session = requests.Session() + return self._session + + def auth(self): + """ + Set authentication to API session. + """ + + if self._session is None: + self.session() + + self.username, self.passwd = self.sessionparams.getCredentials() + + if self.username is None: + raise APIError('Username is None') + if self.passwd is None: + raise APIError('password is None') + # Set auth at session level + self._session.auth = (self.username, self.passwd) + + # set password expiration currently it default value + # self._session.expires = '1432091741' + # For now keeping it False to disable SSL verification + self._session.verify = False + return + + def create_base_uri(self): + """ + Create the base URI using host and port. + if no host is provided _base_uri is None. + if port is None then _base_uri is just host + if both is provided then append port to host. + """ + + if self._base_uri is not None: + return self._base_uri + + self.host, self.port = self.sessionparams.getUrlParams() + if self.host is None: + return None + if self.port is None: + return self.host + + self._base_uri = 'https://' + self.host + ':' + self.port + return self._base_uri + + def end_session(self): + """Ends an API session""" + + if self._session is None: + return None # Silently ignore + # The session to be terminated by setting None in session object we + # have. + self._session = None + return + + def request( + self, + method, + uri, + body=None, + expected_status_values=None, + headers=None, ): + """ + Issue an WS API request and return the response body, if any. + + \param method + The HTTP method to issue (eg. GET, PUT, POST, DELETE). + + \param uri + The URI path and query parameter string for the request. + + \param body + The request body that is input to the request, as a Unicode + or String, or None if there is no input body. + + \param expected_status_values + The HTTP status code that is expected on completion of the request. + If this parameter is specified, the function raises an exception + if the HTTP status from the request was not exactly as specified. + Otherwise, it raises an error if the HTTP status is not 2XX. + + \param headers + Request headers for this request, in the form of a Python Dict. + Optional. This function automatically augments these headers + with the headers needed to specify the API session. Note that if + input headers are provided, the supplied Dict object will be + modified by this function. + """ + + # Start with the headers supplied by caller, if any + + if headers is not None: + hdrs = headers # Use caller's dict, not a copy + else: + hdrs = dict() + + # If no session then throw error. + + if self._session is None: + raise APIError('You have no session') + + resp = self._session.request(method, uri, data=body, headers=hdrs) + + # If an expected status was specified, check that the status exactly + # matches wbat was expected, otherwise check that the status is one of + # the success Statuses. + + if expected_status_values is not None: + raise_exc = resp.status_code not in expected_status_values + else: + raise_exc = resp.status_code not in _HTTP_SUCCESS_STATUS_RANGE + + # Raise an exceptoin if the result is not as intended. + + if raise_exc: + + # If the request fails in some way WEb Services API response + # will usually include a standard error response body + # in JSON format that includes a more detailed reason + # code (and message) for the failure. It provides this data + # in JSON format even if the request would return some other + # format if the request had been successful. + # So if the request has failed, grab that additional info + # for use in raising exceptions below. + + failure_reason = 0 + failure_code = None + # if response.status_code not in _HTTP_SUCCESS_STATUS_RANGE: + + # The API provides the JSON error response in all usual error + # cases. But for certain less common errors this does not occur + # because the error is caught higher in the processing stack. + # So try to interpret the response as a JSON response body, but + # just silently ignore problems if we can't do this. + + try: + error_resp = json.loads(resp.text) + failure_reason = error_resp['reason'] + failure_code = error_resp['code'] + except (ValueError, KeyError): + pass + + raise APIRequestError(resp.status_code, failure_reason, + failure_code) + + # Return the response + return resp + + def request_octet( + self, + method, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + + # Start with the request headers the caller supplied if any, + # otherwise start with an empty set of headers. + + if headers is not None: + hdrs = headers # Using caller's dict, not a copy + else: + hdrs = dict() + + # If a request body was specified, convert it from its assumed Dict + # or List form into JSON using the Python JSON library. Also, add in + # the required Content-Type HTTP message header to let the API know + # that we are supplying input in JSON form. (A Content-Length header + # is also needed to specify the body length, but for not not set.) + + body_json = body + if body is not None: + body_json = json.dumps(body) + + # Supply an HTTP Accepts header indicating we accept/expect that any + # response body be JSON. The Accepts header is optional, and the API + # will defautl to supplying JSON for any operation defined in API V1.1 + # to return JSON (even if in the future support for other formats would + # be added). But its a good idea to specify this header as a safeguard + # in case this function were called for an URI that is not capable of + # returning JSON, as this function as currently coded would choke on + # non-JSON responses. + + hdrs['Accept'] = 'application/json' + + # Now issue the request, and retrieve the response object and the + # response body if provided. + + resp = self.request( + method, + uri, + body=body_json, + expected_status_values=expected_status_values, + headers=hdrs, + ) + + return resp + + def request_json( + self, + method, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """ + Issue an WS API request that is defined to take JSON input and + produce JSON output. (Nearly all WS API requests are like this.) + + Input and output bodies are specified as Python Dict or List objects, + which are converted to/from JSON by this function. + + \param method + The HTTP method to issue (eg. GET, PUT, POST, DELETE). + + \param uri + The URI path and query parameter string for the request. + + \param body + The request body that is input to the request, in the form of a + Python Dict or List object. This object is automatically converted + to corresponding JSON by this function. Optional. + + \param expected_status_values + The HTTP status code that is expected on completion of the request. + See corresponding parameter on request() method for more details. + + \param headers + Request headers for this request, in the form of a Python Dict. + Optional. This function automatically augments these headers + with the headers needed to specify the JSON content type for input + and output, and to specify the API session. Note that if input + headers are provided, the supplied Dict object will be modified + by this function. + + """ + + # Start with the request headers the caller supplied if any, + # otherwise start with an empty set of headers. + + if headers is not None: + hdrs = headers # Using caller's dict, not a copy + else: + hdrs = dict() + + # If a request body was specified, convert it from its assumed Dict + # or List form into JSON using the Python JSON library. Also, add in + # the required Content-Type HTTP message header to let the API know + # that we are supplying input in JSON form. (A Content-Length header + # is also needed to specify the body length, but for now have not set.) + + body_json = None + if body is not None: + body_json = json.dumps(body) + hdrs['Content-Type'] = 'application/json' + + # Supply an HTTP Accepts header indicating we accept/expect that any + # response body be JSON. The Accepts header is optional, and the API + # will default to supplying JSON for any operation defined in API V1.1 + # to return JSON (even if in the future support for other formats would + # be added). But its a good idea to specify this header as a safeguard + # in case this function were called for an URI that is not capable of + # returning JSON, as this function as currently coded would choke on + # non-JSON responses. + + hdrs['Accept'] = 'application/json' + + # Now issue the request, and retrieve the response object and the + # response body if provided. + + response_body = self.request( + method, + uri, + body=body_json, + expected_status_values=expected_status_values, + headers=hdrs, + ) + + # If a response body was returned, we presume it is JSON and + # convert it into a Python dict we will return. + + resp_json = None + if response_body is not None: + try: + resp_json = response_body.json() + except ValueError, err: + print 'response_body %s' % response_body + print 'response_body headers %s' % response_body.headers + raise APIError('Response body expected to be JSON but is' + + 'not valid %s', err) + return resp_json + + def request_get_json( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a GET request that returns JSON.""" + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json('GET', uri, + expected_status_values=expected_status_values, + headers=headers) + + def request_put_json( + self, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a GET request that returns JSON.""" + + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json( + 'PUT', + uri, + body=body, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_post_json( + self, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a POST request that + provides/returns JSON.""" + + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json( + 'POST', + uri, + body=body, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_delete_json( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a DELETE request that returns JSON.""" + + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json( + 'DELETE', + uri, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_get( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a GET request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + if headers is None: + headers = dict() + headers['Accept'] = 'application/json' + + return self.request('GET', uri, + expected_status_values=expected_status_values, + headers=headers) + + def request_put( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a PUT request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + if headers is None: + headers = dict() + headers['Accept'] = 'application/json' + + return self.request( + 'PUT', + uri, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_post( + self, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a POST request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + # If a request body was specified, convert it from its assumed Dict + # or List form into JSON using the Python JSON library. Also, add in + # the required Content-Type HTTP message header to let the API know + # that we are supplying input in JSON form. (A Content-Length header + # is also needed to specify the body length, but for now have not set.) + + body_json = None + if body is not None: + body_json = json.dumps(body) + + if headers is None: + headers = dict() + headers['Content-Type'] = 'application/json' + headers['Accept'] = 'application/json' + + return self.request( + 'POST', + uri, + body=body_json, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_delete( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a DELETE request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + if headers is None: + headers = dict() + headers['Accept'] = 'application/json' + + return self.request( + 'DELETE', + uri, + expected_status_values=expected_status_values, + headers=headers, + ) + + +class APIRequestError(APIError): + """ + Raised when an API request ends in error or not as expected. + + Attributes: + \param status + HTTP status code from the request + + \param reason + API reason code from the request + + \param message + API diagnostic message from the request + + \param stack + Internal diagnostic info for selected Status 500 errors + """ + + def __init__( + self, + status, + reason=None, + code=None, + ): + self.status = status + self.reason = reason + self.code = code + + def __str__(self): + if self.code is not None: + s = 'Request ended with status %s-%s (%s)' \ + % (self.status, self.reason, self.code) + else: + s = 'Request ended with status %s-%s' % (self.status, + self.reason) + return s + + +class BadJSONResponse(APIError): + """exceptions raised for Bad Json Response.""" + + def __init__( + self, + error, + ): + self.error = error + + def __str__(self): + s = 'Bad response: %s' % self.error + return s + + +class InvalidInput(APIError): + """exceptions raised for Invalid Input.""" + + def __init__( + self, + error, + ): + self.error = error + + def __str__(self): + s = 'Bad response: %s' % self.error + return s + + +class SessionParameters(object): + """ + Represents functionalities that could help in getting the + session information. + """ + + def __init__( + self, + conffile=DEFAULT_CONF): + """ + Attributes: + \param conffile + config file which contains all configuration + information with sections + """ + self.conffile = conffile + self.username = None + self.passwd = None + self.host = '127.0.0.1' + self.port = '8001' + self.params = 'config' + self.logfile = 'wok-api-test-suite.log' + self.session_section = 'Session' + self.logging = logging + self.loglevel = 'ERROR' + self.LEVELS = {'INFO': self.logging.INFO, + 'DEBUG': self.logging.DEBUG, + 'WARNING': self.logging.WARNING, + 'ERROR': self.logging.ERROR, + 'CRITICAL': self.logging.CRITICAL} + if self.conffile is None: + print 'Configuration file required %s' \ + % self.conffile + else: + print 'Reading configuration file %s' % self.conffile + self.params = ConfigParser.ConfigParser() + print self.params.read(self.conffile) + if self.params.has_section(self.session_section): + if self.params.has_option(self.session_section, 'logfile'): + self.logfile = self.params.get( + self.session_section, 'logfile') + + if self.params.has_option(self.session_section, 'loglevel'): + self.loglevel = self.params.get( + self.session_section, 'loglevel') + else: + print "Section %s is not available in the config file " \ + % self.session_section + + self.logging.basicConfig(format='%(asctime)s %(levelname)s: \ + %(message)s', + filename=self.logfile, + level=self.LEVELS[self.loglevel]) + + def getCredentials(self): + """ + Returns user and password details for a session obtained from config + file + """ + if self.params.has_section(self.session_section): + if self.params.has_option(self.session_section, 'user'): + self.username = self.params.get(self.session_section, 'user') + + if self.params.has_option(self.session_section, 'passwd'): + self.passwd = self.params.get(self.session_section, 'passwd') + else: + print "Section %s is not available in the config file " \ + % self.session_section + raise APIError('Session section not present in config') + + return self.username, self.passwd + + def getUrlParams(self): + """ + Returns host and port details for a session obtained from config file + """ + if self.params.has_section(self.session_section): + if self.params.has_option(self.session_section, 'host'): + self.host = self.params.get(self.session_section, 'host') + + if self.params.has_option(self.session_section, 'port'): + self.port = self.params.get(self.session_section, 'port') + else: + print "Section %s is not available in the config file " \ + % self.session_section + + return self.host, self.port + + def getLogger(self): + return self.logging + + +class Validator(object): + """ + validator class having different validate methods + """ + + def __init__(self, logger=None): + """ + :param logger: + :return: + """ + if logger is not None: + self.logging = logger + + def validate_json(self, jsn, schma): + """ + validates json against schema using jsonschema.validate library + + :param jsn: JSON to validate + :param schma: Schema against which JSON should get validated + :return: + """ + if self.logging is not None: + self.logging.info('--> validate_json()') + self.logging.debug( + 'validate_json(): jsn:%s' % jsn) + self.logging.debug( + 'validate_json(): schma:%s' % schma) + jsonschema.validate(jsn, schma) + if self.logging is not None: + self.logging.debug( + 'validate_json(): jsn is valid') + self.logging.info('<-- validate_json()') -- 2.5.0

1 0

[PATCH V2][Wok 03/12] FVT: Lists all dependecies for fvt testcases.
by archus＠linux.vnet.ibm.com 30 May '16

30 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> All the dependecies to be installed which is required by fvt testcases should be added here. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/requirements.txt | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 tests/fvt/requirements.txt diff --git a/tests/fvt/requirements.txt b/tests/fvt/requirements.txt new file mode 100644 index 0000000..9941793 --- /dev/null +++ b/tests/fvt/requirements.txt @@ -0,0 +1,23 @@ +# This requirements file lists all dependecies for fvt testcases. +# Run 'pip install -r requirements.txt' to install these dependencies +# +# Project Wok +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +jsonschema +requests -- 2.5.0

1 0

[PATCH V2][Wok 02/12] FVT: Wok level config file to have 'sectionsi required for fvt common across plugins.
by archus＠linux.vnet.ibm.com 30 May '16

30 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Wok level config file to have 'sections' required for functional verification test common across plugins. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/config | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 tests/fvt/config diff --git a/tests/fvt/config b/tests/fvt/config new file mode 100644 index 0000000..13acc0b --- /dev/null +++ b/tests/fvt/config @@ -0,0 +1,7 @@ +[Session] +user : +passwd : +host : localhost +port : 8001 +logfile : wok-api-test-suite.log +loglevel : DEBUG -- 2.5.0

1 0

[PATCH V2][Wok 01/12] FVT: Package for functional verification testcases.
by archus＠linux.vnet.ibm.com 30 May '16

30 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> init python file for fvt package. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/__init__.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 tests/fvt/__init__.py diff --git a/tests/fvt/__init__.py b/tests/fvt/__init__.py new file mode 100644 index 0000000..9316f95 --- /dev/null +++ b/tests/fvt/__init__.py @@ -0,0 +1,18 @@ +# +# Project Wok +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -- 2.5.0

1 0

[PATCH][Wok 04/11] FVT: Common classes/methods for API calls as per config file configuration.
by archus＠linux.vnet.ibm.com 30 May '16

30 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Wrapper classes/methods for making API calls by reading the config file configuration. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/restapilib.py | 760 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 760 insertions(+) create mode 100644 tests/fvt/restapilib.py diff --git a/tests/fvt/restapilib.py b/tests/fvt/restapilib.py new file mode 100644 index 0000000..dec7c43 --- /dev/null +++ b/tests/fvt/restapilib.py @@ -0,0 +1,760 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# +# Project Wok +# +# Copyright IBM, Corp. 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +import ConfigParser +import jsonschema +import json +import requests +import logging +import os + +requests.packages.urllib3.disable_warnings() + +__all__ = ['APIError', 'APIRequestError', 'APISession', 'SessionParameters'] + +# DEFAULT_CONNECTION_TIMEOUT = 5 # Default connection timeout, in secs +# DEFAULT_REQUEST_TIMEOUT = 300 # Default request timeout, in secs +DEFAULT_CONF = os.path.dirname(os.path.abspath(__file__)) + os.sep + 'config' + +_HTTP_SUCCESS_STATUS_RANGE = range(200, 299) # All of the 2XX status codes + + +class APIError(Exception): + """Base class for exceptions raised by this module.""" + + pass + + +class APISession(object): + """Represents an API session with the host Web Services API""" + + def __init__(self, conffile=DEFAULT_CONF): + """ Creates an APISession object for use with the specified Host. + """ + self._session = None + self._base_uri = None + self.username = None + self.passwd = None + self.host = None + self.port = None + self._session = self.session() # Holds the session from requests + self.sessionparams = SessionParameters(conffile=conffile) + self.logging = self.sessionparams.getlogger() + # base URI of connected machine + self._base_uri = self.create_base_uri() + + def session(self): + """ + Create the session object for API Request if no existing session. + """ + if self._session is not None: + raise APIError('Already have a session') + + self._session = requests.Session() + return self._session + + def auth(self): + """ + Set authentication to API session. + + \param username + The username name to use for this session. + + \param password + The password associated with username. + + """ + + if self._session is None: + self.session() + + self.username, self.passwd = self.sessionparams.getcredentials() + + if self.username is None: + raise APIError('Username is None') + if self.passwd is None: + raise APIError('password is None') + # Set auth at session level + self._session.auth = (self.username, self.passwd) + + # set password expiration currently it default value + # self._session.expires = '1432091741' + # For now keeping it False to disable SSL verification + self._session.verify = False + return + + def create_base_uri(self): + """ + Create the base URI using host and port. + if no host is provided _base_uri is None. + if port is None then _base_uri is just host + if both is provided then append port to host. + """ + + if self._base_uri is not None: + return self._base_uri + + self.host, self.port = self.sessionparams.geturlparams() + if self.host is None: + return None + if self.port is None: + return self.host + + self._base_uri = 'https://' + self.host + ':' + self.port + return self._base_uri + + def end_session(self): + """Ends an API session""" + + if self._session is None: + return None # Silently ignore + # The session to be terminated by setting None in session object we + # have. + self._session = None + return + + def request( + self, + method, + uri, + body=None, + expected_status_values=None, + headers=None, ): + """ + Issue an WS API request and return the response body, if any. + + \param method + The HTTP method to issue (eg. GET, PUT, POST, DELETE). + + \param uri + The URI path and query parameter string for the request. + + \param body + The request body that is input to the request, as a Unicode + or String, or None if there is no input body. + + \param expected_status_values + The HTTP status code that is expected on completion of the request. + If this parameter is specified, the function raises an exception + if the HTTP status from the request was not exactly as specified. + Otherwise, it raises an error if the HTTP status is not 2XX. + + \param expected_reason + The API reason code tha tis expected on completion of the request. + If this parameter is specified in addition to the + expected_status_values patameter, this function raises an exception + if either the status or reason is not as specified. + Otherwise, the reason code from the request is not considered. + + \param headers + Request headers for this request, in the form of a Python Dict. + Optional. This function automatically augments these headers + with the headers needed to specify the API session. Note that if + input headers are provided, the supplied Dict object will be + modified by this function. + """ + + # Start with the headers supplied by caller, if any + + if headers is not None: + hdrs = headers # Use caller's dict, not a copy + else: + hdrs = dict() + + # If no session then throw error. + + if self._session is None: + raise APIError('You have no session') + + resp = self._session.request(method, uri, data=body, headers=hdrs) + + # If an expected status was specified, check that the status exactly + # matches wbat was expected, otherwise check that the status is one of + # the success Statuses. + + if expected_status_values is not None: + raise_exc = resp.status_code not in expected_status_values + else: + raise_exc = resp.status_code not in _HTTP_SUCCESS_STATUS_RANGE + + # Raise an exceptoin if the result is not as intended. + + if raise_exc: + + # If the request fails in some way WEb Services API response + # will usually include a standard error response body + # in JSON format that includes a more detailed reason + # code (and message) for the failure. It provides this data + # in JSON format even if the request would return some other + # format if the request had been successful. + # So if the request has failed, grab that additional info + # for use in raising exceptions below. + + failure_reason = 0 + failure_code = None + # if response.status_code not in _HTTP_SUCCESS_STATUS_RANGE: + + # The API provides the JSON error response in all usual error + # cases. But for certain less common errors this does not occur + # because the error is caught higher in the processing stack. + # So try to interpret the response as a JSON response body, but + # just silently ignore problems if we can't do this. + + try: + error_resp = json.loads(resp.text) + failure_reason = error_resp['reason'] + failure_code = error_resp['code'] + except (ValueError, KeyError): + pass + + raise APIRequestError(resp.status_code, failure_reason, + failure_code) + + # Return the response + return resp + + def request_octet( + self, + method, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + + # Start with the request headers the caller supplied if any, + # otherwise start with an empty set of headers. + + if headers is not None: + hdrs = headers # Using caller's dict, not a copy + else: + hdrs = dict() + + # If a request body was specified, convert it from its assumed Dict + # or List form into JSON using the Python JSON library. Also, add in + # the required Content-Type HTTP message header to let the API know + # that we are supplying input in JSON form. (A Content-Length header + # is also needed to specify the body length, but for not not set.) + + body_json = body + if body is not None: + body_json = json.dumps(body) + + # Supply an HTTP Accepts header indicating we accept/expect that any + # response body be JSON. The Accepts header is optional, and the API + # will defautl to supplying JSON for any operation defined in API V1.1 + # to return JSON (even if in the future support for other formats would + # be added). But its a good idea to specify this header as a safeguard + # in case this function were called for an URI that is not capable of + # returning JSON, as this function as currently coded would choke on + # non-JSON responses. + + hdrs['Accept'] = 'application/json' + + # Now issue the request, and retrieve the response object and the + # response body if provided. + + resp = self.request( + method, + uri, + body=body_json, + expected_status_values=expected_status_values, + headers=hdrs, + ) + + return resp + + def request_json( + self, + method, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """ + Issue an WS API request that is defined to take JSON input and + produce JSON output. (Nearly all WS API requests are like this.) + + Input and output bodies are specified as Python Dict or List objects, + which are converted to/from JSON by this function. + + \param method + The HTTP method to issue (eg. GET, PUT, POST, DELETE). + + \param uri + The URI path and query parameter string for the request. + + \param body + The request body that is input to the request, in the form of a + Python Dict or List object. This object is automatically converted + to corresponding JSON by this function. Optional. + + \param expected_status_values + The HTTP status code that is expected on completion of the request. + See corresponding parameter on request() method for more details. + + \param expected_reason + The API reason code that is expected on completion of the request. + See corresponding parameter on request() method for more details. + + \param headers + Request headers for this request, in the form of a Python Dict. + Optional. This function automatically augments these headers + with the headers needed to specify the JSON content type for input + and output, and to specify the API session. Note that if input + headers are provided, the supplied Dict object will be modified + by this function. + + """ + + # Start with the request headers the caller supplied if any, + # otherwise start with an empty set of headers. + + if headers is not None: + hdrs = headers # Using caller's dict, not a copy + else: + hdrs = dict() + + # If a request body was specified, convert it from its assumed Dict + # or List form into JSON using the Python JSON library. Also, add in + # the required Content-Type HTTP message header to let the API know + # that we are supplying input in JSON form. (A Content-Length header + # is also needed to specify the body length, but for now have not set.) + + body_json = None + if body is not None: + body_json = json.dumps(body) + hdrs['Content-Type'] = 'application/json' + + # Supply an HTTP Accepts header indicating we accept/expect that any + # response body be JSON. The Accepts header is optional, and the API + # will default to supplying JSON for any operation defined in API V1.1 + # to return JSON (even if in the future support for other formats would + # be added). But its a good idea to specify this header as a safeguard + # in case this function were called for an URI that is not capable of + # returning JSON, as this function as currently coded would choke on + # non-JSON responses. + + hdrs['Accept'] = 'application/json' + + # Now issue the request, and retrieve the response object and the + # response body if provided. + + response_body = self.request( + method, + uri, + body=body_json, + expected_status_values=expected_status_values, + headers=hdrs, + ) + + # If a response body was returned, we presume it is JSON and + # convert it into a Python dict we will return. + + resp_json = None + if response_body is not None: + try: + resp_json = response_body.json() + except ValueError, err: + print 'response_body %s' % response_body + print 'response_body headers %s' % response_body.headers + raise APIError('Response body expected to be JSON but is' + + 'not valid %s', err) + return resp_json + + def request_get_json( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a GET request that returns JSON.""" + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json('GET', uri, + expected_status_values=expected_status_values, + headers=headers) + + def request_put_json( + self, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a GET request that returns JSON.""" + + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json( + 'PUT', + uri, + body=body, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_post_json( + self, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a POST request that + provides/returns JSON.""" + + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json( + 'POST', + uri, + body=body, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_delete_json( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a DELETE request that returns JSON.""" + + if self._base_uri is not None: + uri = self._base_uri + uri + + return self.request_json( + 'DELETE', + uri, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_get( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a GET request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + if headers is None: + headers = dict() + headers['Accept'] = 'application/json' + + return self.request('GET', uri, + expected_status_values=expected_status_values, + headers=headers) + + def request_put( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a PUT request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + if headers is None: + headers = dict() + headers['Accept'] = 'application/json' + + return self.request( + 'PUT', + uri, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_post( + self, + uri, + body=None, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a POST request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + # If a request body was specified, convert it from its assumed Dict + # or List form into JSON using the Python JSON library. Also, add in + # the required Content-Type HTTP message header to let the API know + # that we are supplying input in JSON form. (A Content-Length header + # is also needed to specify the body length, but for now have not set.) + + body_json = None + if body is not None: + body_json = json.dumps(body) + + if headers is None: + headers = dict() + headers['Content-Type'] = 'application/json' + headers['Accept'] = 'application/json' + + return self.request( + 'POST', + uri, + body=body_json, + expected_status_values=expected_status_values, + headers=headers, + ) + + def request_delete( + self, + uri, + expected_status_values=None, + headers=None, + ): + """Convenience function to do a DELETE request with header['Accept'] + as application/json that returns the response. + Does not convert the response to JSON + """ + + if self._base_uri is not None: + uri = self._base_uri + uri + + if headers is None: + headers = dict() + headers['Accept'] = 'application/json' + + return self.request( + 'DELETE', + uri, + expected_status_values=expected_status_values, + headers=headers, + ) + + +class APIRequestError(APIError): + """ + Raised when an API request ends in error or not as expected. + + Attributes: + \param status + HTTP status code from the request + + \param reason + API reason code from the request + + \param message + API diagnostic message from the request + + \param stack + Internal diagnostic info for selected Status 500 errors + """ + + def __init__( + self, + status, + reason=None, + code=None, + ): + self.status = status + self.reason = reason + self.code = code + + def __str__(self): + if self.code is not None: + s = 'Request ended with status %s-%s (%s)' \ + % (self.status, self.reason, self.code) + else: + s = 'Request ended with status %s-%s' % (self.status, + self.reason) + return s + + +class BadJSONResponse(APIError): + """exceptions raised for Bad Json Response.""" + + def __init__( + self, + error, + ): + self.error = error + + def __str__(self): + s = 'Bad response: %s' % self.error + return s + + +class InvalidInput(APIError): + """exceptions raised for Invalid Input.""" + + def __init__( + self, + error, + ): + self.error = error + + def __str__(self): + s = 'Bad response: %s' % self.error + return s + + +class SessionParameters(object): + """ + Represents functionalities that could help in getting the + session information. + """ + + def __init__( + self, + conffile=DEFAULT_CONF): + """ + Attributes: + \param conffile + config file which contains all configuration + information with sections + """ + self.conffile = conffile + self.username = None + self.passwd = None + self.host = '127.0.0.1' + self.port = '8001' + self.params = 'config' + self.logfile = 'kimchi-api-test-suite.log' + self.session_section = 'Session' + self.logging = logging + self.loglevel = 'ERROR' + self.LEVELS = {'INFO': self.logging.INFO, + 'DEBUG': self.logging.DEBUG, + 'WARNING': self.logging.WARNING, + 'ERROR': self.logging.ERROR, + 'CRITICAL': self.logging.CRITICAL} + if self.conffile is None: + print 'Configuration file required %s' \ + % self.conffile + else: + print 'Reading configuration file %s' % self.conffile + self.params = ConfigParser.ConfigParser() + print self.params.read(self.conffile) + if self.params.has_section(self.session_section): + if self.params.has_option(self.session_section, 'logfile'): + self.logfile = self.params.get( + self.session_section, 'logfile') + + if self.params.has_option(self.session_section, 'loglevel'): + self.loglevel = self.params.get( + self.session_section, 'loglevel') + else: + print "Section %s is not available in the config file " \ + % self.session_section + + self.logging.basicConfig(format='%(asctime)s %(levelname)s: \ + %(message)s', + datefmt='%m-%d-%Y %I:%M:%S %p', + filename=self.logfile, + level=self.LEVELS[self.loglevel]) + + def getcredentials(self): + """ + Returns user and password details for a session obtained from config + file + """ + if self.params.has_section(self.session_section): + if self.params.has_option(self.session_section, 'user'): + self.username = self.params.get(self.session_section, 'user') + + if self.params.has_option(self.session_section, 'passwd'): + self.passwd = self.params.get(self.session_section, 'passwd') + else: + print "Section %s is not available in the config file " \ + % self.session_section + raise APIError('Session section not present in config') + + return self.username, self.passwd + + def geturlparams(self): + """ + Returns host and port details for a session obtained from config file + """ + if self.params.has_section(self.session_section): + if self.params.has_option(self.session_section, 'host'): + self.host = self.params.get(self.session_section, 'host') + + if self.params.has_option(self.session_section, 'port'): + self.port = self.params.get(self.session_section, 'port') + else: + print "Section %s is not available in the config file " \ + % self.session_section + + return self.host, self.port + + def getlogger(self): + return self.logging + + +class Validator(object): + """ + validator class having different validate methods + """ + + def __init__(self, logger=None): + """ + :param logger: + :return: + """ + if logger is not None: + self.logging = logger + + def validate_json(self, jsn, schma): + """ + validates json against schema using jsonschema.validate library + + :param jsn: JSON to validate + :param schma: Schema against which JSON should get validated + :return: + """ + if logging is not None: + self.logging.info('--> validate_json()') + if logging is not None: + self.logging.debug( + 'validate_json(): jsn:%s' % jsn) + self.logging.debug( + 'validate_json(): schma:%s' % schma) + jsonschema.validate(jsn, schma) + if logging is not None: + self.logging.debug( + 'validate_json(): jsn is valid') + if logging is not None: + self.logging.info('<-- validate_json()') -- 2.5.0

3 2

[PATCH][Wok] Expose delete notification function
by Rodrigo Trujillo 28 May '16

28 May '16

This patch exposes the delete notifications operation, then plugins can use the function directly instead of import and instanciate the model to make this operation. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/wok/model/notifications.py | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/wok/model/notifications.py b/src/wok/model/notifications.py index 79bed03..55bd178 100644 --- a/src/wok/model/notifications.py +++ b/src/wok/model/notifications.py @@ -44,6 +44,15 @@ def add_notification(code, args={}, plugin_name=None): notificationsStore[code] = args +def del_notification(code): + global notificationsStore + + try: + del notificationsStore[str(code)] + except Exception as e: + raise OperationFailed("WOKNOT0002E", {'id': str(code), 'msg': e.msg()}) + + class NotificationsModel(object): def __init__(self, **kargs): pass @@ -71,10 +80,4 @@ class NotificationModel(object): raise NotFoundError("WOKNOT0001E", {'id': str(id)}) def delete(self, id): - global notificationsStore - - try: - del notificationsStore[str(id)] - except Exception as e: - raise OperationFailed("WOKNOT0002E", {'id': str(id), - 'msg': e.msg()}) + del_notification(id) -- 2.1.0

1 0

Re: [Kimchi-devel] [PATCH][Kimchi] Improve Kimchi feature tests output
by Aline Manera 27 May '16

27 May '16

Applied. Thanks. Regards, Aline Manera

1 0

[PATCH] [Kimchi] Fixed Clone Guest modal window
by sguimaraes943＠gmail.com 27 May '16

27 May '16

From: Samuel Guimarães <sguimaraes943(a)gmail.com> Fixed a minor issue with a <div> tag not closed in Clone Guest modal window. Samuel Guimarães (1): Fixed Clone Guest modal window ui/pages/guest-clone.html.tmpl | 1 + 1 file changed, 1 insertion(+) -- 1.9.3

3 3

Re: [Kimchi-devel] [PATCH V4] [Kimchi] Handle Libvirt host ENOSPC event
by Aline Manera 27 May '16

27 May '16

Applied. Thanks. Regards, Aline Manera

1 0

Re: [Kimchi-devel] [PATCH v2] [Kimchi] Decrease the sleep time for libvirt event timout
by Aline Manera 27 May '16

27 May '16

Applied. Thanks. Regards, Aline Manera

1 0

[PATCH v2] [Kimchi 2/2] Implement multi-function pci hotplug support
by Jose Ricardo Ziviani 27 May '16

27 May '16

Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- model/vmhostdevs.py | 100 +++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 91 insertions(+), 9 deletions(-) diff --git a/model/vmhostdevs.py b/model/vmhostdevs.py index 7051324..2130ac4 100644 --- a/model/vmhostdevs.py +++ b/model/vmhostdevs.py @@ -97,6 +97,22 @@ class VMHostDevsModel(object): return self.task.lookup(taskid) + def _get_pci_devices_xml(self, pci_infos, slot, driver): + hostdevs = '' + # all devices included in the xml will be sorted in reverse (the + # function 0 will be the last one) and will include the guest + # address details + for dev_info in sorted(pci_infos, + key=itemgetter('function'), + reverse=True): + + dev_info['detach_driver'] = driver + hostdevs += self._get_pci_device_xml(dev_info, + slot, + True) + + return '<devices>%s</devices>' % hostdevs + def _get_pci_device_xml(self, dev_info, slot, is_multifunction): if 'detach_driver' not in dev_info: dev_info['detach_driver'] = 'kvm' @@ -108,12 +124,6 @@ class VMHostDevsModel(object): driver = E.driver(name=dev_info['detach_driver']) if is_multifunction: - multi = E.address(type='pci', - domain='0', - bus='0', - slot=str(slot), - function=str(dev_info['function'])) - if dev_info['function'] == 0: multi = E.address(type='pci', domain='0', @@ -122,6 +132,13 @@ class VMHostDevsModel(object): function=str(dev_info['function']), multifunction='on') + else: + multi = E.address(type='pci', + domain='0', + bus='0', + slot=str(slot), + function=str(dev_info['function'])) + host_dev = E.hostdev(source, driver, multi, mode='subsystem', type='pci', managed='yes') @@ -168,7 +185,7 @@ class VMHostDevsModel(object): if free < slot: return free - return free+1 + return free + 1 def _attach_pci_device(self, cb, params): cb('Attaching PCI device') @@ -198,8 +215,7 @@ class VMHostDevsModel(object): pci_infos = [dev_model.lookup(dev_name) for dev_name in group_names] pci_infos.append(dev_info) - is_multifunction = len(pci_infos) > 1 and \ - DOM_STATE_MAP[dom.info()[0]] == "shutoff" + is_multifunction = len(pci_infos) > 1 pci_infos = sorted(pci_infos, key=itemgetter('name')) # does not allow hot-plug of 3D graphic cards @@ -234,8 +250,32 @@ class VMHostDevsModel(object): slot = 0 if is_multifunction: + # search for the first available slot in guest xml slot = self._available_slot(dom) + with RollbackContext() as rollback: + # multifunction hotplug is a special case where all functions + # must be attached together within one xml file, the same does + # not happen to multifunction coldplug - where each function is + # attached individually + if DOM_STATE_MAP[dom.info()[0]] != 'shutoff' and is_multifunction: + xmlstr = self._get_pci_devices_xml(pci_infos, slot, driver) + + try: + dom.attachDeviceFlags(xmlstr, device_flags) + + except libvirt.libvirtError: + wok_log.error( + 'Failed to attach mutifunction device VM %s: \n%s', + vmid, xmlstr) + raise + + rollback.prependDefer(dom.detachDeviceFlags, xmlstr, + device_flags) + rollback.commitAll() + cb('OK', True) + return + for pci_info in pci_infos: pci_info['detach_driver'] = driver cb('Reading source device XML') @@ -486,6 +526,14 @@ class VMHostDevModel(object): raise InvalidOperation('KCHVMHDEV0006E', {'name': dev_info['name']}) + if self._hotunplug_multifunction_pci(dom, hostdev, dev_name): + if is_3D_device: + cb('Updating MMIO from VM...') + devsmodel = VMHostDevsModel(conn=self.conn) + devsmodel.update_mmio_guest(vmid, False) + cb('OK', True) + return + for e in hostdev: if DeviceModel.deduce_dev_name(e, self.conn) == dev_name: xmlstr = etree.tostring(e) @@ -506,6 +554,40 @@ class VMHostDevModel(object): cb('OK', True) + def _get_devices_same_addr(self, hostdev, domain, bus, slot): + devices = [] + for device in hostdev: + if device.attrib['type'] != 'pci': + continue + + address = device.source.address + if int(address.attrib['domain'], 16) != domain or \ + int(address.attrib['bus'], 16) != bus or \ + int(address.attrib['slot'], 16) != slot: + continue + + devices.append(etree.tostring(device)) + + return devices + + def _hotunplug_multifunction_pci(self, dom, hostdev, dev_name): + domain, bus, slot, _ = dev_name.split('_')[1:] + # get all devices attached to the guest in the same domain+bus+slot + # that the one we are going to detach because they must be detached + # together + devices = self._get_devices_same_addr(hostdev, + int(domain, 16), + int(bus, 16), + int(slot, 16)) + if len(devices) <= 1: + return False + + devices_xml = '<devices>%s</devices>' % ''.join(devices) + dom.detachDeviceFlags(devices_xml, + get_vm_config_flag(dom, mode='all')) + + return True + def _delete_affected_pci_devices(self, dom, dev_name, pci_devs): dev_model = DeviceModel(conn=self.conn) try: -- 2.7.4

1 0

[PATCH v2] [Kimchi 1/2] Enable hot-plug multi-function pci on front-end
by Jose Ricardo Ziviani 27 May '16

27 May '16

Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.guest_edit_main.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/js/src/kimchi.guest_edit_main.js b/ui/js/src/kimchi.guest_edit_main.js index 66edbcb..4cee8fe 100644 --- a/ui/js/src/kimchi.guest_edit_main.js +++ b/ui/js/src/kimchi.guest_edit_main.js @@ -486,7 +486,7 @@ kimchi.guest_edit_main = function() { $('.body', '#form-guest-edit-pci').append(itemNode); pciEnabled || $('button', itemNode).remove(); $('button i', itemNode).addClass(iconClass); - if (kimchi.thisVMState === "running" && (arrPCIDevices[i].multifunction || arrPCIDevices[i].vga3d)) { + if (kimchi.thisVMState === "running" && arrPCIDevices[i].vga3d) { $('button', itemNode).prop("disabled", true); } $('button', itemNode).on('click', function(event) { -- 2.7.4

1 0

[PATCH] [Kimchi 0/2] Implement multi-function pci hotplug
by Jose Ricardo Ziviani 27 May '16

27 May '16

This patchset implements multi-function pci hotplug for Kimchi. The special point in it is the need to attach all functions at once (one xml containing all devices) sorted (in reverse, the function 0 is the last one). This logic also applies to hot unplug. How to test the backend: - hotplug curl -u username -H "Content-Type: application/json" -H "Accept: application/json" https://host:8001/plugins/kimchi/vms/0fed23/hostdevs -X POST -d '{"name": "pci_0001_05_00_1"}' - hotunplug curl -u username -H "Content-Type: application/json" -H "Accept: application/json" https://host:8001/plugins/kimchi/vms/0fed23/hostdevs/pci_0001_05_00_0 -X DELETE Note: it doesn't matter the function is chosen (pci_0001_05_00_0 or pci_0001_05_00_1 in this case), both functions will be attached/detached accordingly. Note2: attach/detach are async methods but UI is not reflecting it, it's necessary to close/reopen the Edit window to see the PCI list updated. Jose Ricardo Ziviani (2): Enable hot-plug multi-function pci on front-end Implement multi-function pci hotplug support model/vmhostdevs.py | 96 +++++++++++++++++++++++++++++++++++-- ui/js/src/kimchi.guest_edit_main.js | 2 +- 2 files changed, 94 insertions(+), 4 deletions(-) -- 2.7.4

3 2

[PATCH V4] [Kimchi] Handle Libvirt host ENOSPC event
by Lucio Correia 27 May '16

27 May '16

When storage pool is out of space and guest requires more, guest is paused and nothing is told to the user. This patch uses Libvirt Event handling and Asynchronous Notifications mechanism to tell web users about what happened. Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> --- i18n.py | 1 + model/libvirtevents.py | 28 ++++++++++++++++++++++++++++ model/model.py | 3 +++ 3 files changed, 32 insertions(+) V4: try/except added for libvirt event callback registration diff --git a/i18n.py b/i18n.py index 3a5a9d1..2d8390f 100644 --- a/i18n.py +++ b/i18n.py @@ -335,6 +335,7 @@ messages = { "KCHEVENT0001E": _("Failed to register the default event implementation."), "KCHEVENT0002E": _("Failed to register timeout event."), "KCHEVENT0003E": _("Failed to Run the default event implementation."), + "KCHEVENT0004W": _("I/O error on guest '%(vm)s': storage pool out of space for %(devAlias)s (%(srcPath)s)."), # These messages (ending with L) are for user log purposes "KCHNET0001L": _("Created %(connection)s virtual network '%(name)s'"), diff --git a/model/libvirtevents.py b/model/libvirtevents.py index ab5c17d..6425f9b 100644 --- a/model/libvirtevents.py +++ b/model/libvirtevents.py @@ -22,6 +22,9 @@ import libvirt import time from wok.exception import OperationFailed +from wok.message import WokMessage +from wok.model.notifications import add_notification +from wok.utils import wok_log class LibvirtEvents(object): @@ -57,3 +60,28 @@ class LibvirtEvents(object): # Event loop handler used to limit length of waiting for any other event. def _kimchi_EventTimeout(self, timer, opaque): time.sleep(1) + + def event_enospc_cb(self, conn, dom, path, dev, action, reason, args): + if reason == "enospc": + info = { + "vm": dom.name(), + "srcPath": path, + "devAlias": dev, + } + add_notification("KCHEVENT0004W", info, '/plugins/kimchi') + msg = WokMessage("KCHEVENT0004W", info, '/plugins/kimchi') + wok_log.warning(msg.get_text()) + + def handleEnospc(self, conn): + """ + Register Libvirt IO_ERROR_REASON event to handle host ENOSPC + """ + try: + conn.get().domainEventRegisterAny( + None, + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON, + self.event_enospc_cb, + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON + ) + except libvirt.libvirtError as e: + wok_log.error("Register of ENOSPC event failed: %s" % e.message) diff --git a/model/model.py b/model/model.py index ed474d2..327d5a7 100644 --- a/model/model.py +++ b/model/model.py @@ -50,6 +50,9 @@ class Model(BaseModel): 'eventsloop': self.events} models = [] + # Register for Libvirt's host ENOSPC event and notify UI if it happens + self.events.handleEnospc(self.conn) + # Import task model from Wok instances = get_instances('wok.model.tasks') for instance in instances: -- 1.9.1

2 2

[PATCH] [Kimchi 2/2] Implement multi-function pci hotplug support
by Jose Ricardo Ziviani 27 May '16

27 May '16

Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- model/vmhostdevs.py | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 93 insertions(+), 3 deletions(-) diff --git a/model/vmhostdevs.py b/model/vmhostdevs.py index 7051324..f643b6e 100644 --- a/model/vmhostdevs.py +++ b/model/vmhostdevs.py @@ -97,6 +97,24 @@ class VMHostDevsModel(object): return self.task.lookup(taskid) + def _get_pci_devices_xml(self, pci_infos, slot, driver): + hostdevs = '' + # all devices included in the xml will be sorted in reverse (the + # function 0 will be the last one) and will include the guest + # address details + for dev_info in sorted(pci_infos, + key=itemgetter('function'), + reverse=True): + + has_multif_flag = True if dev_info['function'] == 0 else False + + dev_info['detach_driver'] = driver + hostdevs += self._get_pci_device_xml(dev_info, + slot, + has_multif_flag) + + return '<devices>%s</devices>' % hostdevs + def _get_pci_device_xml(self, dev_info, slot, is_multifunction): if 'detach_driver' not in dev_info: dev_info['detach_driver'] = 'kvm' @@ -122,6 +140,13 @@ class VMHostDevsModel(object): function=str(dev_info['function']), multifunction='on') + else: + multi = E.address(type='pci', + domain='0', + bus='0', + slot=str(slot), + function=str(dev_info['function'])) + host_dev = E.hostdev(source, driver, multi, mode='subsystem', type='pci', managed='yes') @@ -168,7 +193,7 @@ class VMHostDevsModel(object): if free < slot: return free - return free+1 + return free + 1 def _attach_pci_device(self, cb, params): cb('Attaching PCI device') @@ -198,8 +223,7 @@ class VMHostDevsModel(object): pci_infos = [dev_model.lookup(dev_name) for dev_name in group_names] pci_infos.append(dev_info) - is_multifunction = len(pci_infos) > 1 and \ - DOM_STATE_MAP[dom.info()[0]] == "shutoff" + is_multifunction = len(pci_infos) > 1 pci_infos = sorted(pci_infos, key=itemgetter('name')) # does not allow hot-plug of 3D graphic cards @@ -234,8 +258,32 @@ class VMHostDevsModel(object): slot = 0 if is_multifunction: + # search for the first available slot in guest xml slot = self._available_slot(dom) + with RollbackContext() as rollback: + # multifunction hotplug is a special case where all functions + # must be attached together within one xml file, the same does + # not happen to multifunction coldplug - where each function is + # attached individually + if DOM_STATE_MAP[dom.info()[0]] != 'shutoff' and is_multifunction: + xmlstr = self._get_pci_devices_xml(pci_infos, slot, driver) + + try: + dom.attachDeviceFlags(xmlstr, device_flags) + + except libvirt.libvirtError: + wok_log.error( + 'Failed to attach mutifunction device VM %s: \n%s', + vmid, xmlstr) + raise + + rollback.prependDefer(dom.detachDeviceFlags, xmlstr, + device_flags) + rollback.commitAll() + cb('OK', True) + return + for pci_info in pci_infos: pci_info['detach_driver'] = driver cb('Reading source device XML') @@ -486,6 +534,14 @@ class VMHostDevModel(object): raise InvalidOperation('KCHVMHDEV0006E', {'name': dev_info['name']}) + if self._hotunplug_multifunction_pci(dom, hostdev, dev_name): + if is_3D_device: + cb('Updating MMIO from VM...') + devsmodel = VMHostDevsModel(conn=self.conn) + devsmodel.update_mmio_guest(vmid, False) + cb('OK', True) + return + for e in hostdev: if DeviceModel.deduce_dev_name(e, self.conn) == dev_name: xmlstr = etree.tostring(e) @@ -506,6 +562,40 @@ class VMHostDevModel(object): cb('OK', True) + def _get_devices_same_addr(self, hostdev, domain, bus, slot): + devices = [] + for device in hostdev: + if device.attrib['type'] != 'pci': + continue + + address = device.source.address + if int(address.attrib['domain'], 16) != domain or \ + int(address.attrib['bus'], 16) != bus or \ + int(address.attrib['slot'], 16) != slot: + continue + + devices.append(etree.tostring(device)) + + return devices + + def _hotunplug_multifunction_pci(self, dom, hostdev, dev_name): + domain, bus, slot, _ = dev_name.split('_')[1:] + # get all devices attached to the guest in the same domain+bus+slot + # that the one we are going to detach because they must be detached + # together + devices = self._get_devices_same_addr(hostdev, + int(domain, 16), + int(bus, 16), + int(slot, 16)) + if len(devices) <= 1: + return False + + devices_xml = '<devices>%s</devices>' % ''.join(devices) + dom.detachDeviceFlags(devices_xml, + get_vm_config_flag(dom, mode='all')) + + return True + def _delete_affected_pci_devices(self, dom, dev_name, pci_devs): dev_model = DeviceModel(conn=self.conn) try: -- 2.7.4

1 0

[PATCH] [Kimchi 1/2] Enable hot-plug multi-function pci on front-end
by Jose Ricardo Ziviani 27 May '16

27 May '16

Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.guest_edit_main.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/js/src/kimchi.guest_edit_main.js b/ui/js/src/kimchi.guest_edit_main.js index 66edbcb..4cee8fe 100644 --- a/ui/js/src/kimchi.guest_edit_main.js +++ b/ui/js/src/kimchi.guest_edit_main.js @@ -486,7 +486,7 @@ kimchi.guest_edit_main = function() { $('.body', '#form-guest-edit-pci').append(itemNode); pciEnabled || $('button', itemNode).remove(); $('button i', itemNode).addClass(iconClass); - if (kimchi.thisVMState === "running" && (arrPCIDevices[i].multifunction || arrPCIDevices[i].vga3d)) { + if (kimchi.thisVMState === "running" && arrPCIDevices[i].vga3d) { $('button', itemNode).prop("disabled", true); } $('button', itemNode).on('click', function(event) { -- 2.7.4

1 0

[PATCH] [Wok] Added extended locale list arranged it in alphabetic order.
by archus＠linux.vnet.ibm.com 26 May '16

26 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- ui/pages/login.html.tmpl | 194 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 172 insertions(+), 22 deletions(-) diff --git a/ui/pages/login.html.tmpl b/ui/pages/login.html.tmpl index 52b81fd..81b4e7a 100644 --- a/ui/pages/login.html.tmpl +++ b/ui/pages/login.html.tmpl @@ -138,28 +138,178 @@ <div class="form-group"> <label for="userLocale">$_("Locale")</label> <select id="userLocale" class="selectpicker col-md-12 col-lg-12"> - <option value="af-NA">Afrikaans (Namibia)</option> - <option value="ar-DZ">Arabic (Algeria)</option> - <option value="az-AZ">Azerbaijani (Azerbaijan)</option> - <option value="en-US">English (US)</option> - <option value="hy-AM">Armenian (Armenia)</option> - <option value="hi-IN">Hindi (India)</option> - <option value="nl-BE">Dutch (Belgium)</option> - <option value="ms-MY">Malay (Malaysia)</option> - <option value="no-NO">Norwegian (Norway)</option> - <option value="sq-AL">Albanian (Albania)</option> - <option value="sv-SE">Swedish (Sweden)</option> - <option value="uz-UZ">Uzbek (Uzbekistan)</option> - <option value="zh-CN">中文（简体）</option> - <option value="pt-BR">Português (Brasil)</option> - <option value="de-DE">Deutsch (Deutschland)</option> - <option value="es-ES">Español (España)</option> - <option value="fr-FR">Français (France)</option> - <option value="it-IT">Italiano (Italia)</option> - <option value="ja-JP">日本語 (日本)</option> - <option value="ko-KR">한국어 (대한민국)</option> - <option value="ru-RU">Русский (Россия)</option> - <option value="zh-TW">中文（繁體）</option> + <option value="af-ZA">Afrikaans (South Africa)</option> + <option value="sq-AL">Albanian (Albania)</option> + <option value="am-ET">Amharic (Ethiopia)</option> + <option value="ar-DZ">Arabic (Algeria)</option> + <option value="ar-BH">Arabic (Bahrain)</option> + <option value="ar-EG">Arabic (Egypt)</option> + <option value="ar-IQ">Arabic (Iraq)</option> + <option value="ar-JO">Arabic (Jordan)</option> + <option value="ar-KW">Arabic (Kuwait)</option> + <option value="ar-LB">Arabic (Lebanon)</option> + <option value="ar-LY">Arabic (Libya)</option> + <option value="ar-MR">Arabic (Mauritania)</option> + <option value="ar-MA">Arabic (Morocco)</option> + <option value="ar-OM">Arabic (Oman)</option> + <option value="ar-QA">Arabic (Qatar)</option> + <option value="ar-SA">Arabic (Saudi Arabia)</option> + <option value="ar-SY">Arabic (Syria)</option> + <option value="ar-TN">Arabic (Tunisia)</option> + <option value="ar-AE">Arabic (United Arab Emirates)</option> + <option value="ar-YE">Arabic (Yemen)</option> + <option value="hy-AM">Armenian (Armenia)</option> + <option value="as-IN">Assamese (India)</option> + <option value="az-AZ">Azerbaijani (Azerbaijan)</option> + <option value="eu-ES">Basque (Spain)</option> + <option value="be-BY">Belarusian (Belarus)</option> + <option value="bn-BD">Bengali (Bangladesh)</option> + <option value="bn-IN">Bengali (India)</option> + <option value="bs-BA">Bosnian (Bosnia)</option> + <option value="bg-BG">Bulgarian (Bulgaria)</option> + <option value="my-MM">Burmese (Myanmar)</option> + <option value="ca-ES">Catalan (Spain)</option> + <option value="zh-CN">Chinese-simplified (China)</option> + <option value="zh-SG">Chinese-simplified (Singapore)</option> + <option value="zh-HK">Chinese-traditional (Hong Kong S.A.R. of China)</option> + <option value="zh-MO">Chinese-traditional (Macao)</option> + <option value="zh-TW">Chinese-traditional (Taiwan)</option> + <option value="hr-HR">Croatian (Croatia)</option> + <option value="cs-CZ">Czech (Czech Republic)</option> + <option value="da-DK">Danish (Denmark)</option> + <option value="nl-BE">Dutch (Belgium)</option> + <option value="nl-NL">Dutch (The Netherlands)</option> + <option value="en-AU">English (Australia)</option> + <option value="en-BE">English (Belgium)</option> + <option value="en-CM">English (Cameroon)</option> + <option value="en-CA">English (Canada)</option> + <option value="en-GH">English (Ghana)</option> + <option value="en-HK">English (Hong Kong S.A.R. of China)</option> + <option value="en-IN">English (India)</option> + <option value="en-IE">English (Ireland)</option> + <option value="en-KE">English (Kenya)</option> + <option value="en-MU">English (Mauritius)</option> + <option value="en-NZ">English (New Zealand)</option> + <option value="en-NG">English (Nigeria)</option> + <option value="en-PH">English (Philippines)</option> + <option value="en-SG">English (Singapore)</option> + <option value="en-ZA">English (South Africa)</option> + <option value="en-TZ">English (Tanzania)</option> + <option value="en-GB">English (United Kingdom)</option> + <option value="en-US">English (United States)</option> + <option value="en-ZM">English (Zambia)</option> + <option value="et-EE">Estonian (Estonia)</option> + <option value="tl-PH">Filipino (Philippines)</option> + <option value="fi-FI">Finnish (Finland)</option> + <option value="fr-DZ">French (Algeria)</option> + <option value="fr-CM">French (Cameroon)</option> + <option value="fr-CD">French (Democratic Republic of the Congo)</option> + <option value="fr-BE">French (Belgium)</option> + <option value="fr-CA">French (Canada)</option> + <option value="fr-FR">French (France)</option> + <option value="fr-CI">French (Ivory Coast [Côte d’Ivoire])</option> + <option value="fr-LU">French (Luxembourg)</option> + <option value="fr-MR">French (Mauritania)</option> + <option value="fr-MU">French (Mauritius)</option> + <option value="fr-MA">French (Morocco)</option> + <option value="fr-SN">French (Senegal)</option> + <option value="fr-CH">French (Switzerland)</option> + <option value="fr-TN">French (Tunisia)</option> + <option value="gl-ES">Galician (Spain)</option> + <option value="lg-UG">Ganda (Uganda)</option> + <option value="ka-GE">Georgian (Georgia)</option> + <option value="de-AT">German (Austria)</option> + <option value="de-DE">German (Germany)</option> + <option value="de-LU">German (Luxembourg)</option> + <option value="de-CH">German (Switzerland)</option> + <option value="el-GR">Greek (Greece)</option> + <option value="gu-IN">Gujarati (India)</option> + <option value="ha-NG">Hausa (Nigeria)</option> + <option value="he-IL">Hebrew (Israel)</option> + <option value="hi-IN">Hindi (India)</option> + <option value="hu-HU">Hungarian (Hungary)</option> + <option value="is-IS">Icelandic (Iceland)</option> + <option value="ig-NG">Igbo (Nigeria)</option> + <option value="id-ID">Indonesian (Indonesia)</option> + <option value="it-IT">Italian (Italy)</option> + <option value="it-CH">Italian (Switzerland)</option> + <option value="ja-JP">Japanese (Japan)</option> + <option value="kn-IN">Kannada (India)</option> + <option value="kk-KZ">Kazakh (Kazakhstan)</option> + <option value="km-KH">Khmer (Cambodia)</option> + <option value="rw-RW">Kinyarwanda (Rwanda)</option> + <option value="kok-IN">Konkani (India)</option> + <option value="ko-KR">Korean (India)</option> + <option value="lo-LA">Lao (Korea, South)</option> + <option value="lv-LV">Latvian (Latvia)</option> + <option value="lt-LT">Lithuanian (Lithuania)</option> + <option value="mk-MK">Macedonian (Macedonia)</option> + <option value="ms-MY">Malay-Latin (Malaysia)</option> + <option value="ml-IN">Malayalam (India)</option> + <option value="mt-MT">Maltese (Malta)</option> + <option value="mr-IN">Marathi (India)</option> + <option value="mn-Cyrl-MN">Mongolian-Cyrillic (Mongolia)</option> + <option value="ne-IN">Nepali (India)</option> + <option value="ne-NP">Nepali (Nepal)</option> + <option value="nb-NO">Norwegian Bokmål (Norway)</option> + <option value="nn-NO">Norwegian Nynorsk (Norway)</option> + <option value="or-IN">Oriya [aka, Odia] (India)</option> + <option value="om-ET">Oromo (Ethiopia)</option> + <option value="pl-PL">Polish (Poland)</option> + <option value="pt-AO">Portuguese Bokmål (Angola)</option> + <option value="pt-BR">Portuguese Nynorsk (Brazil)</option> + <option value="pt-MO">Portuguese (Macao)</option> + <option value="pt-MZ">Portuguese (Mozambique)</option> + <option value="pt-PT">Portuguese (Portugal)</option> + <option value="pa-IN">Punjabi (India)</option> + <option value="ro-RO">Romanian (Romania)</option> + <option value="ru-RU">Russian (Russia)</option> + <option value="sr-RS">Serbian-Cyrillic (Serbia)</option> + <option value="sr-ME">Serbian-Latin (Montenegro)</option> + <option value="sr-Latn-RS">Serbian-Latin (Serbia)</option> + <option value="si-LK">Sinhala (Sri Lanka)</option> + <option value="sk-SK">Slovak (Slovakia)</option> + <option value="sl-SI">Slovenian (Slovenia)</option> + <option value="es-AR">Spanish (Argentina)</option> + <option value="es-BO">Spanish (Bolivia)</option> + <option value="es-CL">Spanish (Chile)</option> + <option value="es-CO">Spanish (Colombia)</option> + <option value="en-CR">Spanish (Costa Rica)</option> + <option value="es-DO">Spanish (Dominican Republic)</option> + <option value="es-EC">Spanish (Ecuador)</option> + <option value="es-SV">Spanish (El Salvador)</option> + <option value="es-GT">Spanish (Guatemala)</option> + <option value="en-HN">Spanish (Honduras)</option> + <option value="es-MX">Spanish (Mexico)</option> + <option value="es-NI">Spanish (Nicaragua)</option> + <option value="es-PA">Spanish (Panama)</option> + <option value="es-PY">Spanish (Paraguay)</option> + <option value="en-PE">Spanish (Peru)</option> + <option value="es-PR">Spanish (Puerto Rico)</option> + <option value="es-ES">Spanish (Spain)</option> + <option value="es-US">Spanish (United States)</option> + <option value="es-UY">Spanish (Uruguay)</option> + <option value="es-VE">Spanish (Venezuela)</option> + <option value="sw-KE">Swahili (Kenya)</option> + <option value="sw-TZ">Swahili (Tanzania)</option> + <option value="sv-SE">Swedish (Sweden)</option> + <option value="ta-IN">Tamil (India)</option> + <option value="te-IN">Telugu (Venezuela)</option> + <option value="th-TH">Thai (Thailand)</option> + <option value="tr-TR">Turkish (Turkey)</option> + <option value="uk-UA">Ukrainian (Ukraine)</option> + <option value="ur-IN">Urdu (India)</option> + <option value="ur-PK">Urdu (Pakistan)</option> + <option value="uz-Cyrl-UZ">Uzbek-Cyrillic (Uzbekistan)</option> + <option value="uz-Latn-UZ">Uzbek-Latin (Uzbekistan)</option> + <option value="vi-VN">Vietnamese (Vietnam)</option> + <option value="cy-GB">Welsh (United Kingdom)</option> + <option value="yo-NG">Yoruba (Nigeria)</option> + <option value="ZU-ZA">Zulu (South Africa)</option> + <option value="az-AZ">Azerbaijani-Latin (Azerbaijan)</option> + <option value="sr-CS">Serbian-Cyrillic ()</option> + <option value="sr-Cyrl-CS">Serbian-Cyrillic (Serbia and Montenegro)</option> + <option value="sr-Latn-CS">Serbian-Latin (Serbia and Montenegro)</option> </select> </div> </form> -- 1.8.3.1

2 1

[PATCH v2] [Kimchi] Decrease the sleep time for libvirt event timout
by Jose Ricardo Ziviani 25 May '16

25 May '16

- The current 1 second of sleeping time for libvirt event timeout causes impact in the webserial console because the event is blocked here for 1sec before arriving there. This commit decreases this value to a smaller arbitrary value. Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- model/libvirtevents.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/libvirtevents.py b/model/libvirtevents.py index ab5c17d..813d727 100644 --- a/model/libvirtevents.py +++ b/model/libvirtevents.py @@ -56,4 +56,4 @@ class LibvirtEvents(object): # Event loop handler used to limit length of waiting for any other event. def _kimchi_EventTimeout(self, timer, opaque): - time.sleep(1) + time.sleep(0.01) -- 2.7.4

3 2

Re: [Kimchi-devel] [PATCH V4] [Kimchi] Handle Libvirt host ENOSPC event
by Lucio Correia 25 May '16

25 May '16

On 24-05-2016 17:39, joserz(a)linux.vnet.ibm.com wrote: > Hello Lucio, > > Can I ask you to test your patch using my patch "[Kimchi] Decrease the > sleep time for libvirt event timout" as base to see if you > get any side effect? > > Thank you That patch broke this fix. The fix works without that patch, but doesn't work with it. The enospc event simply is not caught. -- Lucio Correia Software Engineer IBM LTC Brazil

1 0

[PATCH][Wok 11/11] FVT: Added Readme instruction for running fvt testcases.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Added Readme instruction for running fvt testcases. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- docs/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/README.md b/docs/README.md index 8e14672..56e8144 100644 --- a/docs/README.md +++ b/docs/README.md @@ -74,6 +74,14 @@ Build and Install # Optional if running from the source tree $ sudo make install +FVT Testing +----------------- + + $ sudo pip install virtualenv + $ make check-fvt + +After all fvt tests are executed, a summary will be displayed containing any errors/failures which might have occurred. + Starting up Wok --------------- -- 2.5.0

1 0

[PATCH][Wok 10/11] FVT: Added check-fvt to run FVT testcases using make and venv dir to be cleaned.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Added check-fvt to run FVT testcases using make and venv dir to be cleaned. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- Makefile.am | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/Makefile.am b/Makefile.am index 325d0c9..9827e66 100644 --- a/Makefile.am +++ b/Makefile.am @@ -52,6 +52,7 @@ check-local: find . -path './.git' -prune -o \ -name '*.py' -o -name '*.py.in' | \ xargs $(PYFLAKES) | \ + grep -w -v "venv" | \ while read LINE; do echo "$$LINE"; false; done \ else \ find . -name '*.py' -o -name '*.py.in' | \ @@ -70,6 +71,10 @@ check-local: /bin/bash check-IBM-license-header.sh ; \ fi +check-fvt: + tests/fvt/run_tests.sh + rm -fr venv + # Link built mo files in the source tree to enable use of translations from # within the source tree all-local: @@ -185,5 +190,6 @@ VERSION: clean-local: rm -rf mo rpm + rm -rf mo rpm tests/fvt/venv CLEANFILES = wok.spec `find "$(top_srcdir)" -type f -name "*.pyc" -print` -- 2.5.0

1 0

[PATCH][Wok 09/11] FVT: Added FVT Makefile path in AC_CONFIG_FILES list.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Added fvt's Makefile path in AC_CONFIG_FILES list. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- configure.ac | 1 + 1 file changed, 1 insertion(+) diff --git a/configure.ac b/configure.ac index 002774d..1c64367 100644 --- a/configure.ac +++ b/configure.ac @@ -153,6 +153,7 @@ AC_CONFIG_FILES([ contrib/wok.spec.fedora contrib/wok.spec.suse tests/Makefile + tests/fvt/Makefile ],[ chmod +x po/gen-pot ]) -- 2.5.0

1 0

[PATCH][Wok 08/11] FVT: Added fvt as subdirs and check-fvt to run fvt testcases using make.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Added fvt as subdirs and check-fvt to run fvt testcases using make. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/Makefile.am | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/Makefile.am b/tests/Makefile.am index 9ead666..656cfae 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -17,6 +17,8 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +SUBDIRS = fvt + EXTRA_DIST = \ Makefile.am \ run_tests.sh.in \ @@ -46,5 +48,10 @@ check-local: $(MKDIR_P) $(top_srcdir)/data/screenshots ./run_tests.sh +check-fvt: + fvt/run_tests.sh + rm -fr venv + + BUILT_SOURCES = test_config.py CLEANFILES = run_tests.sh test_config.py -- 2.5.0

1 0

[PATCH][Wok 07/11] FVT: Makefile needed for build and run fvt.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Makefile for building and running fvt. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/Makefile.am | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 tests/fvt/Makefile.am diff --git a/tests/fvt/Makefile.am b/tests/fvt/Makefile.am new file mode 100644 index 0000000..c1d3934 --- /dev/null +++ b/tests/fvt/Makefile.am @@ -0,0 +1,43 @@ +# +# Project Wok +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +EXTRA_DIST = \ + Makefile.am \ + run_tests.sh.in \ + $(wildcard *.py) \ + $(NULL) + +noinst_SCRIPTS = run_tests.sh + +do_substitution = \ + sed -e 's,[@]HAVE_PYMOD_UNITTEST[@],$(HAVE_PYMOD_UNITTEST),g' \ + -e 's,[@]PYTHON_VERSION[@],$(PYTHON_VERSION),g' + +run_tests.sh: run_tests.sh.in Makefile + $(do_substitution) < $(srcdir)/run_tests.sh.in > run_tests.sh + chmod +x run_tests.sh + +CLEANFILES = run_tests.sh + +check-fvt: + ./run_tests.sh + rm -fr venv + +clean-local: + rm -fr venv -- 2.5.0

1 0

[PATCH][Wok 06/11] FVT: Install all the dependencies from requirements.txt and runs FVT testcases
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Script to run FVT test cases which also take care of installing all dependencies from requirements.txt. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/run_tests.sh.in | 92 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100755 tests/fvt/run_tests.sh.in diff --git a/tests/fvt/run_tests.sh.in b/tests/fvt/run_tests.sh.in new file mode 100755 index 0000000..7f8046d --- /dev/null +++ b/tests/fvt/run_tests.sh.in @@ -0,0 +1,92 @@ +#!/bin/bash +# +# Project Wok +# +# Copyright IBM, Corp. 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +mkdir -p venv + +HAVE_UNITTEST=@HAVE_PYMOD_UNITTEST@ +PYTHON_VER=@PYTHON_VERSION@ + +# Verify if the required commands exists on the system +command -v virtualenv >/dev/null 2>&1 || { echo >&2 "virtualenv must be installed for your distribution. Aborting."; exit 1; } +command -v pip >/dev/null 2>&1 || { echo >&2 "pip must be installed for your distribution. Aborting."; exit 1; } + +# Get absolute path of this script +pushd `dirname $0` > /dev/null +SCRIPTPATH=`pwd -P` +reqfile=$SCRIPTPATH'/requirements.txt' + +# Start the virtual environment +virtualenv venv --no-site-packages + + +# Actiate the virtual environment +source venv/bin/activate + +while read line; do + + case "$line" in + \#*) + continue ;; # skip comments + "") + continue ;; # skip empty lines + *) + venv/bin/python$PYTHON_VER -c "import $line" > /dev/null 2>&1 + status=$? + if [ $status -ne 0 ]; then + pip install -r $reqfile # Install the required modules to run tests + break + fi + esac +done < $reqfile + +# Execute the test suite +#python registered_tests.py +#nosetests --with-html --html-file=test_report.html registered_tests.py + +if [ "$1" = "-v" ]; then + OPTS="-v" + shift +else + OPTS="" +fi + +if [ $# -ne 0 ]; then + ARGS="$@" +else + ARGS=`find -name "fvt_*.py" | xargs -I @ basename @ .py` +fi + +if [ "$HAVE_UNITTEST" != "yes" -o "$PYTHON_VER" == "2.6" ]; then + CMD="unit2" +else + CMD="python -m unittest" +fi + +LIST=($ARGS) +FVT_LIST=() +for ((i=0;i<${#LIST[@]};i++)); do + FVT_LIST+=(${LIST[$i]}) +done +PYTHONPATH=../ $CMD $OPTS ${FVT_LIST[@]} + +# Deativate the virtual environment +deactivate + +rm -fr venv -- 2.5.0

1 0

[PATCH][Wok 05/11] FVT: Base test class, takes care common actions required for any FVT test cases.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Common action methods like creating/destoring session, authorization using wok level fvt config file, creating JSON validator and logging. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/fvt_base.py | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 tests/fvt/fvt_base.py diff --git a/tests/fvt/fvt_base.py b/tests/fvt/fvt_base.py new file mode 100644 index 0000000..87df4fd --- /dev/null +++ b/tests/fvt/fvt_base.py @@ -0,0 +1,92 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# +# Project Wok +# +# Copyright IBM, Corp. 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +import unittest +import os +from restapilib import APISession, APIError, APIRequestError +from restapilib import Validator + +DEFAULT_CONF = os.path.dirname(os.path.abspath(__file__)) + 'config' + + +class TestBase(unittest.TestCase): + """Represents an API session setup with the host Web Services API""" + + session = None + + def __init__(self, method='runTest'): + """init with default method as runTest""" + super(TestBase, self).__init__(method) + + @classmethod + def setUpClass(cls): + """ + Hook method for setting up class fixture + before running tests in the class. + Create session and set auth to the session + """ + print '--> TestBase.setUpClass(): Create session ' + cls.session = APISession() + # Log on to the API. An exception will be raised if this fails. + cls.logging = cls.session.logging + cls.logging.debug('--> TestBase.setUpClass()') + cls.validator = Validator(cls.logging) + cls.logging.debug('TestBase.setUpClass(): Setting auth to session') + try: + cls.session.auth() + cls.logging.debug('TestBase.setUpClass(): Auth details set to ' + 'session and base URI created as %s' + % cls.session._base_uri) + except APIError, err: + print 'ERROR %s' % err + print err.__str__() + if cls.session is not None: + cls.logging.error('TestBase.setUpClass(): Ending session' + ' as API error happened') + cls.session.end_session() + finally: + cls.logging.debug('<-- TestBase.setUpClass()') + + def setUp(self): + """Hook method for setting up the test fixture before exercising it.""" + pass + + def tearDown(self): + """Hook method for deconstructing the test fixture after testing it.""" + pass + + @classmethod + def tearDownClass(cls): + """ + Hook method for deconstructing the class + fixture after running all tests in the class. + """ + if cls.session is not None: + print 'TestBase.tearDownClass(): Ending session' + cls.session.end_session() + + def get(self): + try: + resp_net = self.session.request_get_json('/') + print resp_net + except APIRequestError as err: + print 'ERROR %s' % err + print err.__str__() -- 2.5.0

1 0

[PATCH][Wok 03/11] FVT: Lists all dependecies for fvt testcases.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> All the dependecies to be installed which is required by fvt testcases should be added here. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/requirements.txt | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 tests/fvt/requirements.txt diff --git a/tests/fvt/requirements.txt b/tests/fvt/requirements.txt new file mode 100644 index 0000000..aa20241 --- /dev/null +++ b/tests/fvt/requirements.txt @@ -0,0 +1,23 @@ +# This requirements file lists all dependecies for fvt testcases. +# Run 'pip install -r requirements.txt' to install these dependencies +# +# Project Wok +# +# Copyright IBM, Corp. 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA + +jsonschema +requests -- 2.5.0

1 0

[PATCH][Wok 02/11] FVT: Wok level config file to have 'sectionsi required for fvt common across plugins.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Wok level config file to have 'sections' required for functional verification test common across plugins. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/config | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 tests/fvt/config diff --git a/tests/fvt/config b/tests/fvt/config new file mode 100644 index 0000000..7b20c01 --- /dev/null +++ b/tests/fvt/config @@ -0,0 +1,7 @@ +[Session] +user : +passwd : +host : localhost +port : 8001 +logfile : wok-api-test-suite.log +loglevel : DEBUG -- 2.5.0

1 0

[PATCH][Wok 01/11] FVT: Package for functional verification testcases.
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> init python file for fvt package. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- tests/fvt/__init__.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 tests/fvt/__init__.py diff --git a/tests/fvt/__init__.py b/tests/fvt/__init__.py new file mode 100644 index 0000000..4f44650 --- /dev/null +++ b/tests/fvt/__init__.py @@ -0,0 +1,18 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -- 2.5.0

1 0

[PATCH][Wok 00/11] FVT testcases wok framework
by archus＠linux.vnet.ibm.com 25 May '16

25 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> Patch for adding FVT testcases wok framework which can be used across plugins. The changes are: 1) Added fvt package inside tests directory. 2) Added config file inside fvt package to have session details. 3) restapilib.py to have common classes/methods for REST API calls. 4) fvt_base.py, a base test class to take care of doing common setup and treadown required for any fvt test cases like creating/destroying session using config file and restapilib.py. 5) run_test.sh.in script to install all the dependencies and to run all FVT. 6) make file changes to have 'make check-fvt' for running all the FVT. 7) Instruction added in REDME.md for make check-fvt. Archana Singh (11): FVT: Package for functional verification testcases. FVT: Wok level config file to have 'sectionsi required for fvt common across plugins. FVT: Lists all dependecies for fvt testcases. FVT: Common classes/methods for API calls as per config file configuration. FVT: Base test class, takes care common actions required for any FVT test cases. FVT: Install all the dependencies from requirements.txt and runs FVT testcases FVT: Makefile needed for build and run fvt. FVT: Added fvt as subdirs and check-fvt to run fvt testcases using make. FVT: Added FVT Makefile path in AC_CONFIG_FILES list. FVT: Added check-fvt to run FVT testcases using make and venv dir to be cleaned. FVT: Added Readme instruction for running fvt testcases. Makefile.am | 6 + configure.ac | 1 + docs/README.md | 8 + tests/Makefile.am | 7 + tests/fvt/Makefile.am | 43 +++ tests/fvt/__init__.py | 18 ++ tests/fvt/config | 7 + tests/fvt/fvt_base.py | 92 ++++++ tests/fvt/requirements.txt | 23 ++ tests/fvt/restapilib.py | 760 +++++++++++++++++++++++++++++++++++++++++++++ tests/fvt/run_tests.sh.in | 92 ++++++ 11 files changed, 1057 insertions(+) create mode 100644 tests/fvt/Makefile.am create mode 100644 tests/fvt/__init__.py create mode 100644 tests/fvt/config create mode 100644 tests/fvt/fvt_base.py create mode 100644 tests/fvt/requirements.txt create mode 100644 tests/fvt/restapilib.py create mode 100755 tests/fvt/run_tests.sh.in -- 2.5.0

1 0

[RFC] [Kimchi] How to handle more than one interface when listing networks
by Socorro Stoppler 24 May '16

24 May '16

Hi All, When working w/Edit Virtual Network, Lucio pointed out that when more than one interface is chosen (in the case of VEPA and Passthrough), only one interface is being shown in the list of Networks either after creation or editing of the network. User should be able to see all interfaces for that network. Here are the options of handling the interfaces: 1. Show only 1 interface on the list and when hovering over interfaces, show a tooltip that lists all interfaces for that network regardless of how many are chosen for that network (e.g. even for ones that can only have one interface, show tooltip anyways for consistency) 2. Same as #1 except to show 2 interfaces as there's real estate for that 3. Only show tooltip if there's more than 1 or 2 (whichever we decide) interface(s) Any other ideas? Thanks -Socorro

3 3

[PATCH] [Kimchi] Decrease the sleep time for libvirt event timout
by Jose Ricardo Ziviani 24 May '16

24 May '16

- The current 1 second of sleeping time for libvirt event timeout causes impact in the webserial console because the event is blocked here for 1sec before arriving there. This commit decreases this value to a smaller arbitrary value. Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- model/libvirtevents.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/model/libvirtevents.py b/model/libvirtevents.py index ab5c17d..0876d3a 100644 --- a/model/libvirtevents.py +++ b/model/libvirtevents.py @@ -56,4 +56,4 @@ class LibvirtEvents(object): # Event loop handler used to limit length of waiting for any other event. def _kimchi_EventTimeout(self, timer, opaque): - time.sleep(1) + time.sleep(0.0001) -- 2.7.4

1 0

[PATCH][Kimchi] Improve Kimchi feature tests output
by Rodrigo Trujillo 24 May '16

24 May '16

This patch adds more debug information about feature tests results when Kimchi loads. Ex.: - NFS Target Probe support ...: True - Fibre Channel Host support .: True - Kernel VFIO support ........: True - Network Manager running ....: True - Memory Hotplug support .....: True - Service Libvirtd running ...: True - QEMU stream support .......: True - Libvirt Stream Protocols ..: ['http', 'https', 'ftp', 'ftps', 'tftp'] Then developers/admins will have more information when debugging issues. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- model/config.py | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/model/config.py b/model/config.py index 7889896..36fce74 100644 --- a/model/config.py +++ b/model/config.py @@ -87,28 +87,43 @@ class CapabilitiesModel(object): FeatureTests.enable_libvirt_error_logging() def _set_depend_capabilities(self): - wok_log.info("*** Running dependable feature tests ***") + wok_log.info("\n*** Kimchi: Running dependable feature tests ***") conn = self.conn.get() self.qemu_stream = FeatureTests.qemu_supports_iso_stream() + msg = "QEMU stream support .......: %s" + wok_log.info(msg % str(self.qemu_stream)) self.libvirt_stream_protocols = [] for p in ['http', 'https', 'ftp', 'ftps', 'tftp']: if FeatureTests.libvirt_supports_iso_stream(conn, p): self.libvirt_stream_protocols.append(p) - - wok_log.info("*** Dependable feature tests completed ***") + msg = "Libvirt Stream Protocols ..: %s" + wok_log.info(msg % str(self.libvirt_stream_protocols)) + wok_log.info("*** Kimchi: Dependable feature tests completed ***\n") _set_depend_capabilities.priority = 90 def _set_capabilities(self): - wok_log.info("*** Running feature tests ***") + wok_log.info("\n*** Kimchi: Running feature tests ***") conn = self.conn.get() self.nfs_target_probe = FeatureTests.libvirt_support_nfs_probe(conn) + msg = "NFS Target Probe support ...: %s" + wok_log.info(msg % str(self.nfs_target_probe)) self.fc_host_support = FeatureTests.libvirt_support_fc_host(conn) + msg = "Fibre Channel Host support .: %s" + wok_log.info(msg % str(self.fc_host_support)) self.kernel_vfio = FeatureTests.kernel_support_vfio() + msg = "Kernel VFIO support ........: %s" + wok_log.info(msg % str(self.kernel_vfio)) self.nm_running = FeatureTests.is_nm_running() + msg = "Network Manager running ....: %s" + wok_log.info(msg % str(self.nm_running)) self.mem_hotplug_support = FeatureTests.has_mem_hotplug_support(conn) + msg = "Memory Hotplug support .....: %s" + wok_log.info(msg % str(self.mem_hotplug_support)) self.libvirtd_running = is_libvirtd_up() - wok_log.info("*** Feature tests completed ***") + msg = "Service Libvirtd running ...: %s" + wok_log.info(msg % str(self.libvirtd_running)) + wok_log.info("*** Kimchi: Feature tests completed ***\n") _set_capabilities.priority = 90 def _qemu_support_spice(self): -- 2.1.0

1 0

Re: [Kimchi-devel] [PATCH] [Wok] Fix advanced log search by time
by Aline Manera 24 May '16

24 May '16

Applied. Thanks. Regards, Aline Manera

1 0

Re: [Kimchi-devel] [PATCH v4] [Kimchi] Storage Volume management
by Samuel Henrique De Oliveira Guimaraes 24 May '16

24 May '16

-----Original Message----- From: Aline Manera [mailto:alinefm@linux.vnet.ibm.com] Sent: terça-feira, 24 de maio de 2016 11:40 To: Samuel Henrique De Oliveira Guimaraes <samuel.guimaraes(a)eldorado.org.br>; sguimaraes943(a)gmail.com; Kimchi Devel <kimchi-devel(a)ovirt.org> Subject: Re: [Kimchi-devel] [PATCH v4] [Kimchi] Storage Volume management On 05/24/2016 10:27 AM, Samuel Henrique De Oliveira Guimaraes wrote: > > -----Original Message----- > From: kimchi-devel-bounces(a)ovirt.org > [mailto:kimchi-devel-bounces@ovirt.org] On Behalf Of Aline Manera > Sent: segunda-feira, 23 de maio de 2016 17:41 > To: sguimaraes943(a)gmail.com; Kimchi Devel <kimchi-devel(a)ovirt.org> > Subject: Re: [Kimchi-devel] [PATCH v4] [Kimchi] Storage Volume > management > > Hi Samuel, > > Some comments: > > 1. When I switch to the "Gallery view" I can see all volumes selected > and after one second, they are deselected > > Hi. I tried to reproduce this here but it is working fine. Tested with Firefox and Chrome and watched for DOM changes and after a minute the checkboxes were still selected. Even If I run an asynchronous task in another Storage Pool it won't change the volumes in another Storage Pool. I am saying, when you are in the storage volumes list view without any storage volume selected and switch to the gallery view, all the storage volumes appear selected for a second and then the page is refreshed to deselect all them. Do you mean that black border that appears for a brief second? This is not happening on Chrome but Firefox and IE indeed have this problem. I suspect it is because of the border-color transition to "transparent", as Chrome supports RGBA while Firefox and IE only supports RGB for transitions. > 2. When you select multiple volumes to perform an action, there is no feedback to user that an operation will be done on those selected volumes. > I can select others, deselect the ones I have selected before and so. We should block the volume selected, add a loading icon, for example, and a mark check when the operation is completed. > > I think my machine is running these tasks too fast so that's why I didn't foresaw this as a requirement. I'll disable the checkboxes and add the spinner icon. > > 3. Usually we don't use success messages on Wok and its plugins. Any reason to add them now? > > I've added some to System Services and OVS Bridges there was no feedback to the user pointing a change in the tables or that an action was being performed but with multiple selection sometimes it may render multiple messages in the area so I'll remove them. > > 4. The message are duplicated. I can see 2 messages when trying to delete multiple items "Volume selected were deleted" + "Volume X was successfully deleted". > > See above. > > 5. The same messages for delete operation are shown when I wipe volumes. > > See above. > > 6. The wipe operation will change the allocation value and so it must be updated in the volume box. > > You mean the Storage Pool line in the parent table? Once the wipe process is completed the line is updated with 0.0B in the allocated column, however the parent %Used and Allocated columns are not refreshed. Are these two the only fields that should be updated? Should it be done for Clone and Delete as well? I was talking about the storage volume settings. When I do wipe multiple volumes, only one of them get the settings updated. I've selected five different .qcow files and wiped them. According to chrome the list refreshed 6 times, the last time when the selectedVolumes array is empty. For large files displaying in MB when I wipe them, they immediately turn the "allocated" column to 0.0B but If I choose to wipe a file in KB, the allocation and capacity has the same value: Before: { "allocation":131072, "isvalid":false, "capacity":127314, "name":"0003-WOK-Deleting-unecessary-files-conflicting-with-new-u.patch", "format":"raw", "has_permission":true, "path":"/root/Downloads/0003-WOK-Deleting-unecessary-files-conflicting-with-new-u.patch", "used_by":[], "type":"file" }, After: { "allocation":131072, "isvalid":true, "capacity":131072, "name":"0003-WOK-Deleting-unecessary-files-conflicting-with-new-u.patch", "format":"raw", "has_permission":true, "path":"/root/Downloads/0003-WOK-Deleting-unecessary-files-conflicting-with-new-u.patch", "used_by":[], "type":"file" } They keep the same value for allocation and capacity. With a large qcow file: Before: { "allocation":1014439936, "isvalid":true, "capacity":8589934592, "name":"2c2f4ef0-7ce0-4aad-b6c8-b891d21fbf40-0.qcow2", "format":"qcow2", "has_permission":true, "path":"/var/lib/libvirt/images/2c2f4ef0-7ce0-4aad-b6c8-b891d21fbf40-0.qcow2", "used_by":[ "vm1" ], "type":"file" } After: { "allocation":0, "isvalid":true, "capacity":1014497280, "name":"2c2f4ef0-7ce0-4aad-b6c8-b891d21fbf40-0.qcow2", "format":"raw", "has_permission":true, "path":"/var/lib/libvirt/images/2c2f4ef0-7ce0-4aad-b6c8-b891d21fbf40-0.qcow2", "used_by":[], "type":"file" } Note that the wiped volume capacity has a little bit more of its previous allocation. Is it correct? About what you said, I agree we should update the storage pool settings as new volumes or deleting volumes may change the storage pool allocation. I think only %used and allocated must be updated. Ok, will create a new function to replace the old values. > 7. The scroll bar should only scroll the volumes. It is hard to get back all time to have the Actions menu available. > > I'm ok with this for the list view appending a scrollbar on the "table" element but with the Gallery view I think it looks odd. See attached screenshots. IMO It is OK for me on both views (list and gallery). Do you have any other idea to have the same behavior on gallery view which does not look odd as you said? I was trying to attach the scrollbar on the same position as it is today but this doesn't work well with just CSS, I would have to write some lines in JS to keep this parallax effect. I'll keep the same as the screenshots but with different max-height values for list view and gallery view and the scrollbar attached to the tables. > > I think that is all from my side. > > Regards, > Aline Manera > > On 05/23/2016 12:46 PM, sguimaraes943(a)gmail.com wrote: >> From: Samuel Guimarães <sguimaraes943(a)gmail.com> >> >> This patch adds Storage Volume management functions Wipe, Clone, Resize and Delete with multiple selection. It also includes a filter input for each Storage Pool and Gallery View for Storage Volumes. >> >> Changes from [RFC] version: >> >> v1: >> - HTML and CSS >> >> v2: >> - Delete and Wipe with multi-selection >> - Confirm messages with list of selected volumes when wiping or deleting volumes (requires SCSS/CSS patch sent to Wok) >> - Filter working >> - Removed "Add Volume" link from Storage Pool action button >> - Added "Add Volume" to Volume box action button >> >> v3: >> - Clone function working with multiple selection >> - Progress bar working for clone and create volume >> - Temporary volume added to the volumes when cloning >> - Seamless refresh on the volumes once each task is finished >> - Fixed issue when list wouldn't refresh when all volumes are removed from the storage pool. >> >> v4: >> - Prevent scroll when Drop-down in volumes list is clicked >> - Dropdown is not clipped from volumes list when there's only one or two items on the list >> - Added Media Queries for small screen resolutions >> >> Samuel Guimarães (1): >> Storage Volume management >> >> model/storagevolumes.py | 2 +- >> ui/css/kimchi.css | 369 ++++++++++++++-- >> ui/css/src/modules/_storage.scss | 347 ++++++++++++--- >> ui/js/src/kimchi.api.js | 50 +++ >> ui/js/src/kimchi.storage_main.js | 465 +++++++++++++++++---- >> ui/js/src/kimchi.storagepool_add_volume_main.js | 2 +- >> ui/js/src/kimchi.storagepool_resize_volume_main.js | 59 +++ >> ui/pages/i18n.json.tmpl | 6 + >> ui/pages/storagepool-resize-volume.html.tmpl | 51 +++ >> ui/pages/tabs/storage.html.tmpl | 155 +++---- >> 10 files changed, 1249 insertions(+), 257 deletions(-) >> create mode 100644 ui/js/src/kimchi.storagepool_resize_volume_main.js >> create mode 100644 ui/pages/storagepool-resize-volume.html.tmpl >> > _______________________________________________ > Kimchi-devel mailing list > Kimchi-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/kimchi-devel

2 1

[PATCH v4] [Kimchi] Storage Volume management
by sguimaraes943＠gmail.com 24 May '16

24 May '16

From: Samuel Guimarães <sguimaraes943(a)gmail.com> This patch adds Storage Volume management functions Wipe, Clone, Resize and Delete with multiple selection. It also includes a filter input for each Storage Pool and Gallery View for Storage Volumes. Changes from [RFC] version: v1: - HTML and CSS v2: - Delete and Wipe with multi-selection - Confirm messages with list of selected volumes when wiping or deleting volumes (requires SCSS/CSS patch sent to Wok) - Filter working - Removed "Add Volume" link from Storage Pool action button - Added "Add Volume" to Volume box action button v3: - Clone function working with multiple selection - Progress bar working for clone and create volume - Temporary volume added to the volumes when cloning - Seamless refresh on the volumes once each task is finished - Fixed issue when list wouldn't refresh when all volumes are removed from the storage pool. v4: - Prevent scroll when Drop-down in volumes list is clicked - Dropdown is not clipped from volumes list when there's only one or two items on the list - Added Media Queries for small screen resolutions Samuel Guimarães (1): Storage Volume management model/storagevolumes.py | 2 +- ui/css/kimchi.css | 369 ++++++++++++++-- ui/css/src/modules/_storage.scss | 347 ++++++++++++--- ui/js/src/kimchi.api.js | 50 +++ ui/js/src/kimchi.storage_main.js | 465 +++++++++++++++++---- ui/js/src/kimchi.storagepool_add_volume_main.js | 2 +- ui/js/src/kimchi.storagepool_resize_volume_main.js | 59 +++ ui/pages/i18n.json.tmpl | 6 + ui/pages/storagepool-resize-volume.html.tmpl | 51 +++ ui/pages/tabs/storage.html.tmpl | 155 +++---- 10 files changed, 1249 insertions(+), 257 deletions(-) create mode 100644 ui/js/src/kimchi.storagepool_resize_volume_main.js create mode 100644 ui/pages/storagepool-resize-volume.html.tmpl -- 1.9.3

3 4

Re: [Kimchi-devel] [PATCH V3] [Kimchi] Handle Libvirt host ENOSPC event
by Aline Manera 23 May '16

23 May '16

Hi Lucio, Some test cases are failing with this patch. Could you fix it and resend? Thanks, Aline Manera --------------------- [alinefm@alinefm-TP440 kimchi]$ sudo cat /tmp/tmp.QLQ6K1z9cG *** Running feature tests *** *** Feature tests completed *** KCHVM0056E: KCHVM0056E libvirt: error : invalid argument: event callback already tracked Traceback (most recent call last): File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/lib64/python2.7/unittest/__main__.py", line 12, in <module> main(module=None) File "/usr/lib64/python2.7/unittest/main.py", line 94, in __init__ self.parseArgs(argv) File "/usr/lib64/python2.7/unittest/main.py", line 149, in parseArgs self.createTests() File "/usr/lib64/python2.7/unittest/main.py", line 158, in createTests self.module) File "/usr/lib64/python2.7/unittest/loader.py", line 130, in loadTestsFromNames suites = [self.loadTestsFromName(name, module) for name in names] File "/usr/lib64/python2.7/unittest/loader.py", line 91, in loadTestsFromName module = __import__('.'.join(parts_copy)) File "test_livemigration.py", line 91, in <module> class LiveMigrationTests(unittest.TestCase): File "test_livemigration.py", line 273, in LiveMigrationTests @unittest.skipUnless(check_if_vm_migration_test_possible(), File "test_livemigration.py", line 76, in check_if_vm_migration_test_possible inst = model.Model(objstore_loc='/tmp/kimchi-store-test') File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirt.libvirtError: invalid argument: event callback already tracked .*** Running feature tests *** libvirt: Domain Config error : this function is not supported by the connection driver: virDomainAttachDeviceFlags *** Feature tests completed *** cpu_info topology not supported. libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices ..libvirt: error : invalid argument: event callback already tracked Ecpu_info topology not supported. libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices .cpu_info topology not supported. libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices .libvirt: error : invalid argument: event callback already tracked Eslibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Ecpu_info topology not supported. libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices cpu_info topology not supported. cpu_info topology not supported. WOKOBJST0001E: WOKOBJST0001E .libvirt: error : invalid argument: event callback already tracked Ecpu_info topology not supported. libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices WOKOBJST0001E: WOKOBJST0001E libvirt: Domain Config error : this function is not supported by the connection driver: virDomainMemoryStats WOKOBJST0001E: WOKOBJST0001E libvirt: Domain Config error : this function is not supported by the connection driver: virDomainMemoryStats KCHVM0033E: KCHVM0033E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E WOKOBJST0001E: WOKOBJST0001E .libvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Ecpu_info topology not supported. libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices libvirt: Node Device Driver error : this function is not supported by the connection driver: virConnectListAllNodeDevices WOKOBJST0001E: WOKOBJST0001E libvirt: Domain Config error : this function is not supported by the connection driver: virDomainMemoryStats libvirt: Test Driver error : Domain not found KCHVM0002E: KCHVM0002E .libvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked Elibvirt: error : invalid argument: event callback already tracked E ====================================================================== ERROR: test_delete_running_vm (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 1198, in test_delete_running_vm inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_image_based_template (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 266, in test_image_based_template inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_template_creates_user_defined_vol_format_instead_default (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 753, in test_template_creates_user_defined_vol_format_instead_default inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_template_get_default_vol_format_from_conf (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 728, in test_template_get_default_vol_format_from_conf inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_template_uses_qcow2_format_if_no_user_or_default_defined (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 782, in test_template_uses_qcow2_format_if_no_user_or_default_defined inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_cdrom (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 581, in test_vm_cdrom inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_disk (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 485, in test_vm_disk inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_edit (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 955, in test_vm_edit objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_graphics (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 313, in test_vm_graphics inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_ifaces (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 365, in test_vm_ifaces inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_lifecycle (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 142, in test_vm_lifecycle inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_list_sorted (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 1225, in test_vm_list_sorted inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_memory_hotplug (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 807, in test_vm_memory_hotplug inst = model.Model(None, objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_netboot (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 436, in test_vm_netboot inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_serial (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 344, in test_vm_serial inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ====================================================================== ERROR: test_vm_storage_provisioning (test_model.ModelTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_model.py", line 671, in test_vm_storage_provisioning inst = model.Model(objstore_loc=self.tmp_store) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/model.py", line 54, in __init__ self.events.handleEnospc(self.conn) File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtevents.py", line 83, in handleEnospc libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON File "/home/alinefm/wok/src/wok/plugins/kimchi/model/libvirtconnection.py", line 67, in wrapper ret = f(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4919, in domainEventRegisterAny raise libvirtError ('virConnectDomainEventRegisterAny() failed', conn=self) libvirtError: invalid argument: event callback already tracked ---------------------------------------------------------------------- Ran 25 tests in 15.769s FAILED (errors=16, skipped=1) On 05/23/2016 05:46 PM, Lucio Correia wrote: > When storage pool is out of space and guest requires more, > guest is paused and nothing is told to the user. This patch > uses Libvirt Event handling and Asynchronous Notifications > mechanism to tell web users about what happened. > > Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> > --- > i18n.py | 1 + > model/libvirtevents.py | 25 +++++++++++++++++++++++++ > model/model.py | 3 +++ > 3 files changed, 29 insertions(+) > > Changes in v3: rebased with master > > diff --git a/i18n.py b/i18n.py > index 3a5a9d1..2d8390f 100644 > --- a/i18n.py > +++ b/i18n.py > @@ -335,6 +335,7 @@ messages = { > "KCHEVENT0001E": _("Failed to register the default event implementation."), > "KCHEVENT0002E": _("Failed to register timeout event."), > "KCHEVENT0003E": _("Failed to Run the default event implementation."), > + "KCHEVENT0004W": _("I/O error on guest '%(vm)s': storage pool out of space for %(devAlias)s (%(srcPath)s)."), > > # These messages (ending with L) are for user log purposes > "KCHNET0001L": _("Created %(connection)s virtual network '%(name)s'"), > diff --git a/model/libvirtevents.py b/model/libvirtevents.py > index ab5c17d..00e9433 100644 > --- a/model/libvirtevents.py > +++ b/model/libvirtevents.py > @@ -22,6 +22,9 @@ import libvirt > import time > > from wok.exception import OperationFailed > +from wok.message import WokMessage > +from wok.model.notifications import add_notification > +from wok.utils import wok_log > > > class LibvirtEvents(object): > @@ -57,3 +60,25 @@ class LibvirtEvents(object): > # Event loop handler used to limit length of waiting for any other event. > def _kimchi_EventTimeout(self, timer, opaque): > time.sleep(1) > + > + def event_enospc_cb(self, conn, dom, path, dev, action, reason, args): > + if reason == "enospc": > + info = { > + "vm": dom.name(), > + "srcPath": path, > + "devAlias": dev, > + } > + add_notification("KCHEVENT0004W", info, '/plugins/kimchi') > + msg = WokMessage("KCHEVENT0004W", info, '/plugins/kimchi') > + wok_log.warning(msg.get_text()) > + > + def handleEnospc(self, conn): > + """ > + Register Libvirt IO_ERROR_REASON event to handle host ENOSPC > + """ > + conn.get().domainEventRegisterAny( > + None, > + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON, > + self.event_enospc_cb, > + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON > + ) > diff --git a/model/model.py b/model/model.py > index ed474d2..327d5a7 100644 > --- a/model/model.py > +++ b/model/model.py > @@ -50,6 +50,9 @@ class Model(BaseModel): > 'eventsloop': self.events} > models = [] > > + # Register for Libvirt's host ENOSPC event and notify UI if it happens > + self.events.handleEnospc(self.conn) > + > # Import task model from Wok > instances = get_instances('wok.model.tasks') > for instance in instances:

1 0

Wok Notifications using Notifications API
by Samuel Henrique De Oliveira Guimaraes 23 May '16

23 May '16

Hi team, I was wondering if we could update wok.message.notification to support HTML5 Notification API: https://developer.mozilla.org/en-US/docs/Web/API/notification For now the support is restricted to Chrome and Firefox and Edge 14 (latest Windows 10 update. I think this feature could be very valuable to Kimchi / Ginger users. Regards, Samuel

2 1

[PATCH V3] [Kimchi] Handle Libvirt host ENOSPC event
by Lucio Correia 23 May '16

23 May '16

When storage pool is out of space and guest requires more, guest is paused and nothing is told to the user. This patch uses Libvirt Event handling and Asynchronous Notifications mechanism to tell web users about what happened. Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> --- i18n.py | 1 + model/libvirtevents.py | 25 +++++++++++++++++++++++++ model/model.py | 3 +++ 3 files changed, 29 insertions(+) Changes in v3: rebased with master diff --git a/i18n.py b/i18n.py index 3a5a9d1..2d8390f 100644 --- a/i18n.py +++ b/i18n.py @@ -335,6 +335,7 @@ messages = { "KCHEVENT0001E": _("Failed to register the default event implementation."), "KCHEVENT0002E": _("Failed to register timeout event."), "KCHEVENT0003E": _("Failed to Run the default event implementation."), + "KCHEVENT0004W": _("I/O error on guest '%(vm)s': storage pool out of space for %(devAlias)s (%(srcPath)s)."), # These messages (ending with L) are for user log purposes "KCHNET0001L": _("Created %(connection)s virtual network '%(name)s'"), diff --git a/model/libvirtevents.py b/model/libvirtevents.py index ab5c17d..00e9433 100644 --- a/model/libvirtevents.py +++ b/model/libvirtevents.py @@ -22,6 +22,9 @@ import libvirt import time from wok.exception import OperationFailed +from wok.message import WokMessage +from wok.model.notifications import add_notification +from wok.utils import wok_log class LibvirtEvents(object): @@ -57,3 +60,25 @@ class LibvirtEvents(object): # Event loop handler used to limit length of waiting for any other event. def _kimchi_EventTimeout(self, timer, opaque): time.sleep(1) + + def event_enospc_cb(self, conn, dom, path, dev, action, reason, args): + if reason == "enospc": + info = { + "vm": dom.name(), + "srcPath": path, + "devAlias": dev, + } + add_notification("KCHEVENT0004W", info, '/plugins/kimchi') + msg = WokMessage("KCHEVENT0004W", info, '/plugins/kimchi') + wok_log.warning(msg.get_text()) + + def handleEnospc(self, conn): + """ + Register Libvirt IO_ERROR_REASON event to handle host ENOSPC + """ + conn.get().domainEventRegisterAny( + None, + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON, + self.event_enospc_cb, + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON + ) diff --git a/model/model.py b/model/model.py index ed474d2..327d5a7 100644 --- a/model/model.py +++ b/model/model.py @@ -50,6 +50,9 @@ class Model(BaseModel): 'eventsloop': self.events} models = [] + # Register for Libvirt's host ENOSPC event and notify UI if it happens + self.events.handleEnospc(self.conn) + # Import task model from Wok instances = get_instances('wok.model.tasks') for instance in instances: -- 1.9.1

1 0

[PATCH V2] [Kimchi] Add support to Libvirt Events.
by pvital＠linux.vnet.ibm.com 23 May '16

23 May '16

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> This patch adds support to handle in Kimchi any Libvirt Event, just by adding a callback to process the event and register the event with the callback. None event is being registered by this patch and they need be added by developer by demand in the code. To see how register an event, the unit test test_libvirtevents.py register two events and then use them in the code. Signed-off-by: Paulo Vital <pvital(a)linux.vnet.ibm.com> --- i18n.py | 4 + model/libvirtevents.py | 59 +++++++++++ model/model.py | 5 +- tests/test_model_libvirtevents.py | 212 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 279 insertions(+), 1 deletion(-) create mode 100644 model/libvirtevents.py create mode 100644 tests/test_model_libvirtevents.py diff --git a/i18n.py b/i18n.py index db245c0..9d7a597 100644 --- a/i18n.py +++ b/i18n.py @@ -331,4 +331,8 @@ messages = { "KCHLVMS0001E": _("Invalid volume group name parameter: %(name)s."), + "KCHEVENT0001E": _("Failed to register the default event implementation."), + "KCHEVENT0002E": _("Failed to register timeout event."), + "KCHEVENT0003E": _("Failed to Run the default event implementation."), + } diff --git a/model/libvirtevents.py b/model/libvirtevents.py new file mode 100644 index 0000000..ab5c17d --- /dev/null +++ b/model/libvirtevents.py @@ -0,0 +1,59 @@ +# +# Project Kimchi +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import cherrypy +import libvirt +import time + +from wok.exception import OperationFailed + + +class LibvirtEvents(object): + def __init__(self): + # Register default implementation of event handlers + if libvirt.virEventRegisterDefaultImpl() < 0: + raise OperationFailed('KCHEVENT0001E') + + # Run a background thread with the event loop. Using cherrypy + # BackgroundTask class due to issues when using threading module with + # cherrypy. + self.event_loop_thread = cherrypy.process.plugins.BackgroundTask( + 2, + self._event_loop_run + ) + self.event_loop_thread.setName('KimchiLibvirtEventLoop') + self.event_loop_thread.setDaemon(True) + self.event_loop_thread.start() + + # Set an event timeout to control the self._event_loop_run + if libvirt.virEventAddTimeout(0, self._kimchi_EventTimeout, None) < 0: + raise OperationFailed('KCHEVENT0002E') + + # Event loop method to be executed in background as thread + def _event_loop_run(self): + while True: + if libvirt.virEventRunDefaultImpl() < 0: + raise OperationFailed('KCHEVENT0003E') + + def is_event_loop_alive(self): + return self.event_loop_thread.isAlive() + + # Event loop handler used to limit length of waiting for any other event. + def _kimchi_EventTimeout(self, timer, opaque): + time.sleep(1) diff --git a/model/model.py b/model/model.py index e44f804..ed474d2 100644 --- a/model/model.py +++ b/model/model.py @@ -26,6 +26,7 @@ from wok.plugins.kimchi import config from wok.utils import import_module, listPathModules from wok.plugins.kimchi.model.libvirtconnection import LibvirtConnection +from wok.plugins.kimchi.model.libvirtevents import LibvirtEvents class Model(BaseModel): @@ -43,8 +44,10 @@ class Model(BaseModel): return instances self.objstore = ObjectStore(objstore_loc or config.get_object_store()) + self.events = LibvirtEvents() self.conn = LibvirtConnection(libvirt_uri) - kargs = {'objstore': self.objstore, 'conn': self.conn} + kargs = {'objstore': self.objstore, 'conn': self.conn, + 'eventsloop': self.events} models = [] # Import task model from Wok diff --git a/tests/test_model_libvirtevents.py b/tests/test_model_libvirtevents.py new file mode 100644 index 0000000..420e63a --- /dev/null +++ b/tests/test_model_libvirtevents.py @@ -0,0 +1,212 @@ +# -*- coding: utf-8 -*- +# +# Project Kimchi +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import json +import libvirt +import os +import shutil +import tempfile +import time +import unittest + +import tests.utils as utils + +from wok.basemodel import Singleton +from wok.rollbackcontext import RollbackContext + +from wok.plugins.kimchi.model import model + +import iso_gen + + +TMP_DIR = '/var/lib/kimchi/tests/' +UBUNTU_ISO = TMP_DIR + 'ubuntu14.04.iso' +TMP_EVENT = None +EVENT_ID = 0 + + +def setUpModule(): + global TMP_DIR, TMP_EVENT + + if not os.path.exists(TMP_DIR): + os.makedirs(TMP_DIR) + + TMP_EVENT = temp.mktemp() + + iso_gen.construct_fake_iso(UBUNTU_ISO, True, '14.04', 'ubuntu') + + # Some FeatureTests functions depend on server to validate their result. + # As CapabilitiesModel is a Singleton class it will get the first result + # from FeatureTests which may be wrong when using the Model instance + # directly - the case of this test_model.py + # So clean Singleton instances to make sure to get the right result when + # running the following tests. + Singleton._instances = {} + + +def tearDownModule(): + global TMP_DIR, TMP_EVENT + + os.unlink(TMP_EVENT) + shutil.rmtree(TMP_DIR) + + +def _get_next_event_id(): + global EVENT_ID + EVENT_ID += 1 + return EVENT_ID + + +def _get_event_id(): + global EVENT_ID + return EVENT_ID + + +def _store_event(data): + global TMP_EVENT + with open(TMP_EVENT, "a") as file: + file.write("%s\n" % data) + + +def _get_event(id): + global TMP_EVENT + with open(TMP_EVENT, "r") as file: + for event in [line.rstrip('\n') for line in file.readlines()]: + fields = event.split('|') + if fields[0] == id: + return fields[1] + + +class LibvirtEventsTests(unittest.TestCase): + def setUp(self): + self.tmp_store = tempfile.mktemp() + + def tearDown(self): + os.unlink(self.tmp_store) + + def domain_event_lifecycle_cb(self, conn, dom, event, detail, *args): + """ + Callback to handle Domain (VMs) events - VM Livecycle. + """ + evStrings = ("Defined", "Undefined", "Started", "Suspended", "Resumed", + "Stopped", "Shutdown", "PMSuspended", "Crashed") + evDetails = (("Added", "Updated"), + ("Removed", ), + ("Booted", "Migrated", "Restored", "Snapshot", "Wakeup"), + ("Paused", "Migrated", "IOError", "Watchdog", "Restored", + "Snapshot", "API error"), + ("Unpaused", "Migrated", "Snapshot"), + ("Shutdown", "Destroyed", "Crashed", "Migrated", "Saved", + "Failed", "Snapshot"), + ("Finished", ), + ("Memory", "Disk"), + ("Panicked")) + + data = {'domain': dom.name(), 'event': evStrings[event], + 'event_detail': evDetails[event][detail]} + _store_event('%s|%s' % (_get_next_event_id(), json.dumps(data))) + + def domain_event_reboot_cb(self, conn, dom, *args): + """ + Callback to handle Domain (VMs) events - VM Reboot. + """ + data = {'domain': dom.name(), 'event': 'Rebooted'} + _store_event('%s|%s' % (_get_next_event_id(), json.dumps(data))) + + @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') + def test_events_vm_lifecycle(self): + inst = model.Model(objstore_loc=self.tmp_store) + self.objstore = inst.objstore + + # Register the most common Libvirt domain events to be handled. + conn = inst.conn.get() + event_map = [(libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, + self.domain_event_lifecycle_cb), + (libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, + self.domain_event_reboot_cb)] + + for event, event_cb in event_map: + try: + conn.domainEventRegisterAny(None, event, event_cb, None) + except libvirt.libvirtError as e: + # It's not a big deal if an event could not be registered. + print "Could not register event %s. Details: %s" % (event, e) + + # Create a template and VM to test, and start lifecycle tests + with RollbackContext() as rollback: + template_params = {'name': 'ttest', + 'source_media': {'type': 'disk', + 'path': UBUNTU_ISO}} + + inst.templates_create(template_params) + rollback.prependDefer(inst.template_delete, 'ttest') + + vm_params = {'name': 'kimchi-vm1', + 'template': '/plugins/kimchi/templates/ttest'} + task = inst.vms_create(vm_params) + inst.task_wait(task['id'], 10) + task = inst.task_lookup(task['id']) + self.assertEquals('finished', task['status']) + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Defined', res['event']) + self.assertEquals('Added', res['event_detail']) + + inst.vm_start('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Started', res['event']) + self.assertEquals('Booted', res['event_detail']) + + inst.vm_suspend('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Suspended', res['event']) + self.assertEquals('Paused', res['event_detail']) + + inst.vm_resume('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Resumed', res['event']) + self.assertEquals('Unpaused', res['event_detail']) + + inst.vm_reset('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Rebooted', res['event']) + + inst.vm_poweroff('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Stopped', res['event']) + self.assertEquals('Destroyed', res['event_detail']) + + inst.vm_delete('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Undefined', res['event']) + self.assertEquals('Removed', res['event_detail']) -- 2.5.5

2 3

Re: [Kimchi-devel] [PATCH V2] [Kimchi] Isolate unit tests execution.
by Aline Manera 23 May '16

23 May '16

Applied. Thanks. Regards, Aline Manera

1 0

Re: [Kimchi-devel] [PATCH V4] [Kimchi] Add support to Libvirt Events.
by Aline Manera 23 May '16

23 May '16

Applied. Thanks. Regards, Aline Manera

1 0

Re: [Kimchi-devel] [PATCH V2] [Kimchi] Handle Libvirt host ENOSPC event
by Aline Manera 23 May '16

23 May '16

Hi Lucio, I was not able to apply this patch. Could you rebase and resend? Thanks, Aline Manera On 05/17/2016 11:39 AM, Lucio Correia wrote: > When storage pool is out of space and guest requires more, > guest is paused and nothing is told to the user. This patch > uses Libvirt Event handling and Asynchronous Notifications > mechanism to tell web users about what happened. > > Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> > --- > i18n.py | 1 + > model/libvirtevents.py | 25 +++++++++++++++++++++++++ > model/model.py | 3 +++ > 3 files changed, 29 insertions(+) > > Change in V2: > - Rebase with latest master > > diff --git a/i18n.py b/i18n.py > index 49b7f1c..c697d52 100644 > --- a/i18n.py > +++ b/i18n.py > @@ -337,4 +337,5 @@ messages = { > "KCHEVENT0001E": _("Failed to register the default event implementation."), > "KCHEVENT0002E": _("Failed to register timeout event."), > "KCHEVENT0003E": _("Failed to Run the default event implementation."), > + "KCHEVENT0004W": _("I/O error on guest '%(vm)s': storage pool out of space for %(devAlias)s (%(srcPath)s)."), > } > diff --git a/model/libvirtevents.py b/model/libvirtevents.py > index ab5c17d..00e9433 100644 > --- a/model/libvirtevents.py > +++ b/model/libvirtevents.py > @@ -22,6 +22,9 @@ import libvirt > import time > > from wok.exception import OperationFailed > +from wok.message import WokMessage > +from wok.model.notifications import add_notification > +from wok.utils import wok_log > > > class LibvirtEvents(object): > @@ -57,3 +60,25 @@ class LibvirtEvents(object): > # Event loop handler used to limit length of waiting for any other event. > def _kimchi_EventTimeout(self, timer, opaque): > time.sleep(1) > + > + def event_enospc_cb(self, conn, dom, path, dev, action, reason, args): > + if reason == "enospc": > + info = { > + "vm": dom.name(), > + "srcPath": path, > + "devAlias": dev, > + } > + add_notification("KCHEVENT0004W", info, '/plugins/kimchi') > + msg = WokMessage("KCHEVENT0004W", info, '/plugins/kimchi') > + wok_log.warning(msg.get_text()) > + > + def handleEnospc(self, conn): > + """ > + Register Libvirt IO_ERROR_REASON event to handle host ENOSPC > + """ > + conn.get().domainEventRegisterAny( > + None, > + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON, > + self.event_enospc_cb, > + libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON > + ) > diff --git a/model/model.py b/model/model.py > index ed474d2..327d5a7 100644 > --- a/model/model.py > +++ b/model/model.py > @@ -50,6 +50,9 @@ class Model(BaseModel): > 'eventsloop': self.events} > models = [] > > + # Register for Libvirt's host ENOSPC event and notify UI if it happens > + self.events.handleEnospc(self.conn) > + > # Import task model from Wok > instances = get_instances('wok.model.tasks') > for instance in instances:

1 0

Re: [Kimchi-devel] [PATCH V2] [Kimchi] Add support to Libvirt Events.
by Paulo Ricardo Paz Vital 23 May '16

23 May '16

On May 23 03:46PM, Aline Manera wrote: > > Hi Paulo, > > Should I apply this patch with "[PATCH V2] [Kimchi] Isolate unit tests > execution."? > Yes > If so, could you rebase and resend? I was not able to apply it on top of > master branch. > Rebased and V4 patch submitted. V4 also needs "[PATCH V2] [Kimchi] Isolate unit tests execution." to be applied before. > > Thanks, > > Aline Manera > > > On 05/12/2016 04:07 PM, pvital(a)linux.vnet.ibm.com wrote: > > From: Paulo Vital <pvital(a)linux.vnet.ibm.com> > > > > This patch adds support to handle in Kimchi any Libvirt Event, just by adding > > a callback to process the event and register the event with the callback. > > > > None event is being registered by this patch and they need be added by developer > > by demand in the code. To see how register an event, the unit test > > test_libvirtevents.py register two events and then use them in the code. > > > > Signed-off-by: Paulo Vital <pvital(a)linux.vnet.ibm.com> > > --- > > i18n.py | 4 + > > model/libvirtevents.py | 59 +++++++++++ > > model/model.py | 5 +- > > tests/test_model_libvirtevents.py | 212 ++++++++++++++++++++++++++++++++++++++ > > 4 files changed, 279 insertions(+), 1 deletion(-) > > create mode 100644 model/libvirtevents.py > > create mode 100644 tests/test_model_libvirtevents.py > > > > diff --git a/i18n.py b/i18n.py > > index db245c0..9d7a597 100644 > > --- a/i18n.py > > +++ b/i18n.py > > @@ -331,4 +331,8 @@ messages = { > > > > "KCHLVMS0001E": _("Invalid volume group name parameter: %(name)s."), > > > > + "KCHEVENT0001E": _("Failed to register the default event implementation."), > > + "KCHEVENT0002E": _("Failed to register timeout event."), > > + "KCHEVENT0003E": _("Failed to Run the default event implementation."), > > + > > } > > diff --git a/model/libvirtevents.py b/model/libvirtevents.py > > new file mode 100644 > > index 0000000..ab5c17d > > --- /dev/null > > +++ b/model/libvirtevents.py > > @@ -0,0 +1,59 @@ > > +# > > +# Project Kimchi > > +# > > +# Copyright IBM Corp, 2016 > > +# > > +# This library is free software; you can redistribute it and/or > > +# modify it under the terms of the GNU Lesser General Public > > +# License as published by the Free Software Foundation; either > > +# version 2.1 of the License, or (at your option) any later version. > > +# > > +# This library is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > +# Lesser General Public License for more details. > > +# > > +# You should have received a copy of the GNU Lesser General Public > > +# License along with this library; if not, write to the Free Software > > +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > > + > > +import cherrypy > > +import libvirt > > +import time > > + > > +from wok.exception import OperationFailed > > + > > + > > +class LibvirtEvents(object): > > + def __init__(self): > > + # Register default implementation of event handlers > > + if libvirt.virEventRegisterDefaultImpl() < 0: > > + raise OperationFailed('KCHEVENT0001E') > > + > > + # Run a background thread with the event loop. Using cherrypy > > + # BackgroundTask class due to issues when using threading module with > > + # cherrypy. > > + self.event_loop_thread = cherrypy.process.plugins.BackgroundTask( > > + 2, > > + self._event_loop_run > > + ) > > + self.event_loop_thread.setName('KimchiLibvirtEventLoop') > > + self.event_loop_thread.setDaemon(True) > > + self.event_loop_thread.start() > > + > > + # Set an event timeout to control the self._event_loop_run > > + if libvirt.virEventAddTimeout(0, self._kimchi_EventTimeout, None) < 0: > > + raise OperationFailed('KCHEVENT0002E') > > + > > + # Event loop method to be executed in background as thread > > + def _event_loop_run(self): > > + while True: > > + if libvirt.virEventRunDefaultImpl() < 0: > > + raise OperationFailed('KCHEVENT0003E') > > + > > + def is_event_loop_alive(self): > > + return self.event_loop_thread.isAlive() > > + > > + # Event loop handler used to limit length of waiting for any other event. > > + def _kimchi_EventTimeout(self, timer, opaque): > > + time.sleep(1) > > diff --git a/model/model.py b/model/model.py > > index e44f804..ed474d2 100644 > > --- a/model/model.py > > +++ b/model/model.py > > @@ -26,6 +26,7 @@ from wok.plugins.kimchi import config > > from wok.utils import import_module, listPathModules > > > > from wok.plugins.kimchi.model.libvirtconnection import LibvirtConnection > > +from wok.plugins.kimchi.model.libvirtevents import LibvirtEvents > > > > > > class Model(BaseModel): > > @@ -43,8 +44,10 @@ class Model(BaseModel): > > return instances > > > > self.objstore = ObjectStore(objstore_loc or config.get_object_store()) > > + self.events = LibvirtEvents() > > self.conn = LibvirtConnection(libvirt_uri) > > - kargs = {'objstore': self.objstore, 'conn': self.conn} > > + kargs = {'objstore': self.objstore, 'conn': self.conn, > > + 'eventsloop': self.events} > > models = [] > > > > # Import task model from Wok > > diff --git a/tests/test_model_libvirtevents.py b/tests/test_model_libvirtevents.py > > new file mode 100644 > > index 0000000..420e63a > > --- /dev/null > > +++ b/tests/test_model_libvirtevents.py > > @@ -0,0 +1,212 @@ > > +# -*- coding: utf-8 -*- > > +# > > +# Project Kimchi > > +# > > +# Copyright IBM Corp, 2016 > > +# > > +# This library is free software; you can redistribute it and/or > > +# modify it under the terms of the GNU Lesser General Public > > +# License as published by the Free Software Foundation; either > > +# version 2.1 of the License, or (at your option) any later version. > > +# > > +# This library is distributed in the hope that it will be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > +# Lesser General Public License for more details. > > +# > > +# You should have received a copy of the GNU Lesser General Public > > +# License along with this library; if not, write to the Free Software > > +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > > + > > +import json > > +import libvirt > > +import os > > +import shutil > > +import tempfile > > +import time > > +import unittest > > + > > +import tests.utils as utils > > + > > +from wok.basemodel import Singleton > > +from wok.rollbackcontext import RollbackContext > > + > > +from wok.plugins.kimchi.model import model > > + > > +import iso_gen > > + > > + > > +TMP_DIR = '/var/lib/kimchi/tests/' > > +UBUNTU_ISO = TMP_DIR + 'ubuntu14.04.iso' > > +TMP_EVENT = None > > +EVENT_ID = 0 > > + > > + > > +def setUpModule(): > > + global TMP_DIR, TMP_EVENT > > + > > + if not os.path.exists(TMP_DIR): > > + os.makedirs(TMP_DIR) > > + > > + TMP_EVENT = temp.mktemp() > > + > > + iso_gen.construct_fake_iso(UBUNTU_ISO, True, '14.04', 'ubuntu') > > + > > + # Some FeatureTests functions depend on server to validate their result. > > + # As CapabilitiesModel is a Singleton class it will get the first result > > + # from FeatureTests which may be wrong when using the Model instance > > + # directly - the case of this test_model.py > > + # So clean Singleton instances to make sure to get the right result when > > + # running the following tests. > > + Singleton._instances = {} > > + > > + > > +def tearDownModule(): > > + global TMP_DIR, TMP_EVENT > > + > > + os.unlink(TMP_EVENT) > > + shutil.rmtree(TMP_DIR) > > + > > + > > +def _get_next_event_id(): > > + global EVENT_ID > > + EVENT_ID += 1 > > + return EVENT_ID > > + > > + > > +def _get_event_id(): > > + global EVENT_ID > > + return EVENT_ID > > + > > + > > +def _store_event(data): > > + global TMP_EVENT > > + with open(TMP_EVENT, "a") as file: > > + file.write("%s\n" % data) > > + > > + > > +def _get_event(id): > > + global TMP_EVENT > > + with open(TMP_EVENT, "r") as file: > > + for event in [line.rstrip('\n') for line in file.readlines()]: > > + fields = event.split('|') > > + if fields[0] == id: > > + return fields[1] > > + > > + > > +class LibvirtEventsTests(unittest.TestCase): > > + def setUp(self): > > + self.tmp_store = tempfile.mktemp() > > + > > + def tearDown(self): > > + os.unlink(self.tmp_store) > > + > > + def domain_event_lifecycle_cb(self, conn, dom, event, detail, *args): > > + """ > > + Callback to handle Domain (VMs) events - VM Livecycle. > > + """ > > + evStrings = ("Defined", "Undefined", "Started", "Suspended", "Resumed", > > + "Stopped", "Shutdown", "PMSuspended", "Crashed") > > + evDetails = (("Added", "Updated"), > > + ("Removed", ), > > + ("Booted", "Migrated", "Restored", "Snapshot", "Wakeup"), > > + ("Paused", "Migrated", "IOError", "Watchdog", "Restored", > > + "Snapshot", "API error"), > > + ("Unpaused", "Migrated", "Snapshot"), > > + ("Shutdown", "Destroyed", "Crashed", "Migrated", "Saved", > > + "Failed", "Snapshot"), > > + ("Finished", ), > > + ("Memory", "Disk"), > > + ("Panicked")) > > + > > + data = {'domain': dom.name(), 'event': evStrings[event], > > + 'event_detail': evDetails[event][detail]} > > + _store_event('%s|%s' % (_get_next_event_id(), json.dumps(data))) > > + > > + def domain_event_reboot_cb(self, conn, dom, *args): > > + """ > > + Callback to handle Domain (VMs) events - VM Reboot. > > + """ > > + data = {'domain': dom.name(), 'event': 'Rebooted'} > > + _store_event('%s|%s' % (_get_next_event_id(), json.dumps(data))) > > + > > + @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') > > + def test_events_vm_lifecycle(self): > > + inst = model.Model(objstore_loc=self.tmp_store) > > + self.objstore = inst.objstore > > + > > + # Register the most common Libvirt domain events to be handled. > > + conn = inst.conn.get() > > + event_map = [(libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, > > + self.domain_event_lifecycle_cb), > > + (libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, > > + self.domain_event_reboot_cb)] > > + > > + for event, event_cb in event_map: > > + try: > > + conn.domainEventRegisterAny(None, event, event_cb, None) > > + except libvirt.libvirtError as e: > > + # It's not a big deal if an event could not be registered. > > + print "Could not register event %s. Details: %s" % (event, e) > > + > > + # Create a template and VM to test, and start lifecycle tests > > + with RollbackContext() as rollback: > > + template_params = {'name': 'ttest', > > + 'source_media': {'type': 'disk', > > + 'path': UBUNTU_ISO}} > > + > > + inst.templates_create(template_params) > > + rollback.prependDefer(inst.template_delete, 'ttest') > > + > > + vm_params = {'name': 'kimchi-vm1', > > + 'template': '/plugins/kimchi/templates/ttest'} > > + task = inst.vms_create(vm_params) > > + inst.task_wait(task['id'], 10) > > + task = inst.task_lookup(task['id']) > > + self.assertEquals('finished', task['status']) > > + time.sleep(5) > > + res = json.loads(_get_event(str(_get_event_id()))) > > + self.assertEquals('kimchi-vm1', res['domain']) > > + self.assertEquals('Defined', res['event']) > > + self.assertEquals('Added', res['event_detail']) > > + > > + inst.vm_start('kimchi-vm1') > > + time.sleep(5) > > + res = json.loads(_get_event(str(_get_event_id()))) > > + self.assertEquals('kimchi-vm1', res['domain']) > > + self.assertEquals('Started', res['event']) > > + self.assertEquals('Booted', res['event_detail']) > > + > > + inst.vm_suspend('kimchi-vm1') > > + time.sleep(5) > > + res = json.loads(_get_event(str(_get_event_id()))) > > + self.assertEquals('kimchi-vm1', res['domain']) > > + self.assertEquals('Suspended', res['event']) > > + self.assertEquals('Paused', res['event_detail']) > > + > > + inst.vm_resume('kimchi-vm1') > > + time.sleep(5) > > + res = json.loads(_get_event(str(_get_event_id()))) > > + self.assertEquals('kimchi-vm1', res['domain']) > > + self.assertEquals('Resumed', res['event']) > > + self.assertEquals('Unpaused', res['event_detail']) > > + > > + inst.vm_reset('kimchi-vm1') > > + time.sleep(5) > > + res = json.loads(_get_event(str(_get_event_id()))) > > + self.assertEquals('kimchi-vm1', res['domain']) > > + self.assertEquals('Rebooted', res['event']) > > + > > + inst.vm_poweroff('kimchi-vm1') > > + time.sleep(5) > > + res = json.loads(_get_event(str(_get_event_id()))) > > + self.assertEquals('kimchi-vm1', res['domain']) > > + self.assertEquals('Stopped', res['event']) > > + self.assertEquals('Destroyed', res['event_detail']) > > + > > + inst.vm_delete('kimchi-vm1') > > + time.sleep(5) > > + res = json.loads(_get_event(str(_get_event_id()))) > > + self.assertEquals('kimchi-vm1', res['domain']) > > + self.assertEquals('Undefined', res['event']) > > + self.assertEquals('Removed', res['event_detail']) > > -- > > 2.5.5 > -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/

1 0

[PATCH V4] [Kimchi] Add support to Libvirt Events.
by pvital＠linux.vnet.ibm.com 23 May '16

23 May '16

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> This patch adds support to handle in Kimchi any Libvirt Event, just by adding a callback to process the event and register the event with the callback. None event is being registered by this patch and they need be added by developer by demand in the code. To see how register an event, the unit test test_libvirtevents.py register two events and then use them in the code. Signed-off-by: Paulo Vital <pvital(a)linux.vnet.ibm.com> --- i18n.py | 4 + model/libvirtevents.py | 59 ++++++++++ model/model.py | 5 +- tests/test_model_libvirtevents.py | 219 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 286 insertions(+), 1 deletion(-) create mode 100644 model/libvirtevents.py create mode 100644 tests/test_model_libvirtevents.py diff --git a/i18n.py b/i18n.py index c789b7f..3a5a9d1 100644 --- a/i18n.py +++ b/i18n.py @@ -332,6 +332,10 @@ messages = { "KCHCONN0001E": _("Unable to establish connection with libvirt. Please check your libvirt URI which is often defined in /etc/libvirt/libvirt.conf"), "KCHCONN0002E": _("Libvirt service is not active. Please start the libvirt service in your host system."), + "KCHEVENT0001E": _("Failed to register the default event implementation."), + "KCHEVENT0002E": _("Failed to register timeout event."), + "KCHEVENT0003E": _("Failed to Run the default event implementation."), + # These messages (ending with L) are for user log purposes "KCHNET0001L": _("Created %(connection)s virtual network '%(name)s'"), "KCHNET0002L": _("Removed virtual network '%(ident)s'"), diff --git a/model/libvirtevents.py b/model/libvirtevents.py new file mode 100644 index 0000000..ab5c17d --- /dev/null +++ b/model/libvirtevents.py @@ -0,0 +1,59 @@ +# +# Project Kimchi +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import cherrypy +import libvirt +import time + +from wok.exception import OperationFailed + + +class LibvirtEvents(object): + def __init__(self): + # Register default implementation of event handlers + if libvirt.virEventRegisterDefaultImpl() < 0: + raise OperationFailed('KCHEVENT0001E') + + # Run a background thread with the event loop. Using cherrypy + # BackgroundTask class due to issues when using threading module with + # cherrypy. + self.event_loop_thread = cherrypy.process.plugins.BackgroundTask( + 2, + self._event_loop_run + ) + self.event_loop_thread.setName('KimchiLibvirtEventLoop') + self.event_loop_thread.setDaemon(True) + self.event_loop_thread.start() + + # Set an event timeout to control the self._event_loop_run + if libvirt.virEventAddTimeout(0, self._kimchi_EventTimeout, None) < 0: + raise OperationFailed('KCHEVENT0002E') + + # Event loop method to be executed in background as thread + def _event_loop_run(self): + while True: + if libvirt.virEventRunDefaultImpl() < 0: + raise OperationFailed('KCHEVENT0003E') + + def is_event_loop_alive(self): + return self.event_loop_thread.isAlive() + + # Event loop handler used to limit length of waiting for any other event. + def _kimchi_EventTimeout(self, timer, opaque): + time.sleep(1) diff --git a/model/model.py b/model/model.py index e44f804..ed474d2 100644 --- a/model/model.py +++ b/model/model.py @@ -26,6 +26,7 @@ from wok.plugins.kimchi import config from wok.utils import import_module, listPathModules from wok.plugins.kimchi.model.libvirtconnection import LibvirtConnection +from wok.plugins.kimchi.model.libvirtevents import LibvirtEvents class Model(BaseModel): @@ -43,8 +44,10 @@ class Model(BaseModel): return instances self.objstore = ObjectStore(objstore_loc or config.get_object_store()) + self.events = LibvirtEvents() self.conn = LibvirtConnection(libvirt_uri) - kargs = {'objstore': self.objstore, 'conn': self.conn} + kargs = {'objstore': self.objstore, 'conn': self.conn, + 'eventsloop': self.events} models = [] # Import task model from Wok diff --git a/tests/test_model_libvirtevents.py b/tests/test_model_libvirtevents.py new file mode 100644 index 0000000..d7dc675 --- /dev/null +++ b/tests/test_model_libvirtevents.py @@ -0,0 +1,219 @@ +# -*- coding: utf-8 -*- +# +# Project Kimchi +# +# Copyright IBM Corp, 2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import json +import libvirt +import os +import shutil +import tempfile +import time +import unittest + +import tests.utils as utils + +from wok.basemodel import Singleton +from wok.rollbackcontext import RollbackContext + +from wok.plugins.kimchi.model import model + +import iso_gen + + +TMP_DIR = '/var/lib/kimchi/tests/' +UBUNTU_ISO = TMP_DIR + 'ubuntu14.04.iso' +TMP_EVENT = None +EVENT_ID = 0 + + +def setUpModule(): + global TMP_DIR, TMP_EVENT + + if not os.path.exists(TMP_DIR): + os.makedirs(TMP_DIR) + + TMP_EVENT = tempfile.mktemp() + + iso_gen.construct_fake_iso(UBUNTU_ISO, True, '14.04', 'ubuntu') + + # Some FeatureTests functions depend on server to validate their result. + # As CapabilitiesModel is a Singleton class it will get the first result + # from FeatureTests which may be wrong when using the Model instance + # directly - the case of this test_model.py + # So clean Singleton instances to make sure to get the right result when + # running the following tests. + Singleton._instances = {} + + +def tearDownModule(): + global TMP_DIR, TMP_EVENT + + os.unlink(TMP_EVENT) + shutil.rmtree(TMP_DIR) + + +def _get_next_event_id(): + global EVENT_ID + EVENT_ID += 1 + return EVENT_ID + + +def _get_event_id(): + global EVENT_ID + return EVENT_ID + + +def _store_event(data): + global TMP_EVENT + with open(TMP_EVENT, "a") as file: + file.write("%s\n" % data) + + +def _get_event(id): + global TMP_EVENT + with open(TMP_EVENT, "r") as file: + for event in [line.rstrip('\n') for line in file.readlines()]: + fields = event.split('|') + if fields[0] == id: + return fields[1] + + +class LibvirtEventsTests(unittest.TestCase): + def setUp(self): + self.tmp_store = tempfile.mktemp() + + def tearDown(self): + os.unlink(self.tmp_store) + + def domain_event_lifecycle_cb(self, conn, dom, event, detail, *args): + """ + Callback to handle Domain (VMs) events - VM Livecycle. + """ + evStrings = ("Defined", "Undefined", "Started", "Suspended", "Resumed", + "Stopped", "Shutdown", "PMSuspended", "Crashed") + evDetails = (("Added", "Updated"), + ("Removed", ), + ("Booted", "Migrated", "Restored", "Snapshot", "Wakeup"), + ("Paused", "Migrated", "IOError", "Watchdog", "Restored", + "Snapshot", "API error"), + ("Unpaused", "Migrated", "Snapshot"), + ("Shutdown", "Destroyed", "Crashed", "Migrated", "Saved", + "Failed", "Snapshot"), + ("Finished", ), + ("Memory", "Disk"), + ("Panicked")) + + data = {'domain': dom.name(), 'event': evStrings[event], + 'event_detail': evDetails[event][detail]} + _store_event('%s|%s' % (_get_next_event_id(), json.dumps(data))) + + def domain_event_reboot_cb(self, conn, dom, *args): + """ + Callback to handle Domain (VMs) events - VM Reboot. + """ + data = {'domain': dom.name(), 'event': 'Rebooted'} + _store_event('%s|%s' % (_get_next_event_id(), json.dumps(data))) + + @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') + def test_events_vm_lifecycle(self): + inst = model.Model(objstore_loc=self.tmp_store) + self.objstore = inst.objstore + conn = inst.conn.get() + + # Create a template and VM to test, and start lifecycle tests + with RollbackContext() as rollback: + # Register the most common Libvirt domain events to be handled. + event_map = [(libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, + self.domain_event_lifecycle_cb), + (libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, + self.domain_event_reboot_cb)] + + for event, event_cb in event_map: + ev_id = conn.domainEventRegisterAny(None, event, event_cb, + None) + rollback.prependDefer(conn.domainEventDeregisterAny, ev_id) + + # Create a template + template_params = {'name': 'ttest', + 'source_media': {'type': 'disk', + 'path': UBUNTU_ISO}} + + inst.templates_create(template_params) + rollback.prependDefer(inst.template_delete, 'ttest') + + # Create a VM (guest) + vm_params = {'name': 'kimchi-vm1', + 'template': '/plugins/kimchi/templates/ttest'} + task = inst.vms_create(vm_params) + inst.task_wait(task['id'], 10) + task = inst.task_lookup(task['id']) + self.assertEquals('finished', task['status']) + time.sleep(5) + # Check event of domain definition (addition) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Defined', res['event']) + self.assertEquals('Added', res['event_detail']) + + # Start the VM and check the event + inst.vm_start('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Started', res['event']) + self.assertEquals('Booted', res['event_detail']) + + # Suspend the VM and check the event + inst.vm_suspend('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Suspended', res['event']) + self.assertEquals('Paused', res['event_detail']) + + # Resume the VM and check the event + inst.vm_resume('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Resumed', res['event']) + self.assertEquals('Unpaused', res['event_detail']) + + # Reboot the VM and check the event + inst.vm_reset('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Rebooted', res['event']) + + # PowerOff (hard stop) the VM and check the event + inst.vm_poweroff('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Stopped', res['event']) + self.assertEquals('Destroyed', res['event_detail']) + + # Delete the VM and check the event + inst.vm_delete('kimchi-vm1') + time.sleep(5) + res = json.loads(_get_event(str(_get_event_id()))) + self.assertEquals('kimchi-vm1', res['domain']) + self.assertEquals('Undefined', res['event']) + self.assertEquals('Removed', res['event_detail']) -- 2.5.5

1 0

[PATCH V2] [Kimchi] Isolate unit tests execution.
by pvital＠linux.vnet.ibm.com 23 May '16

23 May '16

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> This patch isolates each unit test execution reducing the occurrences of tests using data from previous unit tests and making results not reliable. In addition, it also simplifies the output of tests execution, printing the tags PASSED and FAILED and the error/failed banners only when necessary. Signed-off-by: Paulo Vital <pvital(a)linux.vnet.ibm.com> --- tests/run_tests.sh.in | 52 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 40 insertions(+), 12 deletions(-) diff --git a/tests/run_tests.sh.in b/tests/run_tests.sh.in index 68dfa2e..a43a90a 100644 --- a/tests/run_tests.sh.in +++ b/tests/run_tests.sh.in @@ -21,6 +21,10 @@ HAVE_UNITTEST=@HAVE_PYMOD_UNITTEST@ PYTHON_VER=@PYTHON_VERSION@ +RED='\033[0;31m' +GREEN='\033[0;32m' +NC='\033[0m' # No Color + if [ "$1" = "-v" ]; then OPTS="-v" shift @@ -40,19 +44,43 @@ else CMD="python -m unittest" fi -LIST=($ARGS) -MODEL_LIST=() -MOCK_LIST=() -for ((i=0;i<${#LIST[@]};i++)); do - - if [[ ${LIST[$i]} == test_model* ]]; then - MODEL_LIST+=(${LIST[$i]}) +NUM_TESTS=0 +TIME=0 +declare -A FAILED_UT # dict to store failed unit tests and its OUTPUT_FILE +OUTPUT_FILE=$(mktemp) +for UT in $ARGS; do + echo -n "***** Running unit test: $UT... " + # ../../../../../ refers to the project root + # ../../../../ refers to wok directory + # ../../../ refers to plugins directory + OUTPUT=$(PYTHONPATH=../../../../../:../../../../:../../../ \ + PYTHONWARNINGS="ignore:Unverified HTTPS request" \ + $CMD $OPTS $UT 2>&1) + RES=$? + if [ $RES -ne 0 ]; then + # unit test has failed, so keep the OUTPUT_FILE and print the results + echo -e "\t ${RED}FAILED${NC}" + ERROR_LOG_BEGIN=$(echo "$OUTPUT" | grep -n "^\==*" | head -n1 | cut -d":" -f1) + ERROR_LOG_END=$(echo "$OUTPUT" | wc -l) + echo "$OUTPUT" | tail -n $((${ERROR_LOG_END}-${ERROR_LOG_BEGIN}+1)) + STATS=$(echo "$OUTPUT" | grep "Ran") + FAILED_UT+=([$UT]=$(echo "$OUTPUT" | grep FAILED | cut -d" " -f2-4)) + echo "$OUTPUT" >> $OUTPUT_FILE else - MOCK_LIST+=(${LIST[$i]}) + # unit test has passed, so print the results and delete the OUTPUT_FILE + STATS=$(echo "$OUTPUT" | grep "Ran") + echo -e "\t ${GREEN}PASSED${NC} - $STATS" fi + TES=$(echo $STATS | cut -d" " -f2) + NUM_TESTS=$(echo "$NUM_TESTS + $TES" | bc) + TES=$(echo $STATS | cut -d" " -f5) + TIME=$(echo "$TIME + ${TES:0:-1}" | bc) done -# ../../../../../ refers to the project root -# ../../../../ refers to wok directory -# ../../../ refers to plugins directory -PYTHONPATH=../../../../../:../../../../:../../../ PYTHONWARNINGS="ignore:Unverified HTTPS request" $CMD $OPTS ${MODEL_LIST[@]} ${MOCK_LIST[@]} +# Print summary results +echo -e "======================================================================" +echo -e "===================== Kimchi Unit Tests Summary ======================" +echo -e "Ran $NUM_TESTS tests in $TIME seconds." +for i in "${!FAILED_UT[@]}"; do + echo -e "$i FAILED: ${FAILED_UT[$i]} - full log available at $OUTPUT_FILE" +done -- 2.5.5

1 0

[PATCH] [Kimchi] Storage Volume management
by sguimaraes943＠gmail.com 23 May '16

23 May '16

From: Samuel Guimarães <sguimaraes943(a)gmail.com> This patch adds Storage Volume management functions Wipe, Clone, Resize and Delete with multiple selection. It also includes a filter input for each Storage Pool and Gallery View for Storage Volumes. Changes from [RFC] version: v1: - HTML and CSS v2: - Delete and Wipe with multi-selection - Confirm messages with list of selected volumes when wiping or deleting volumes (requires SCSS/CSS patch sent to Wok) - Filter working - Removed "Add Volume" link from Storage Pool action button - Added "Add Volume" to Volume box action button v3 (this patch): - Clone function working with multiple selection - Progress bar working for clone and create volume - Temporary volume added to the volumes when cloning - Seamless refresh on the volumes once each task is finished - Fixed issue when list wouldn't refresh when all volumes are removed from the storage pool. Samuel Guimarães (1): Storage Volume management model/storagevolumes.py | 2 +- ui/css/kimchi.css | 287 ++++++++++--- ui/css/src/modules/_storage.scss | 309 +++++++++++--- ui/js/src/kimchi.api.js | 50 +++ ui/js/src/kimchi.storage_main.js | 443 ++++++++++++++++----- ui/js/src/kimchi.storagepool_add_volume_main.js | 2 +- ui/js/src/kimchi.storagepool_resize_volume_main.js | 59 +++ ui/pages/i18n.json.tmpl | 6 + ui/pages/storagepool-resize-volume.html.tmpl | 51 +++ ui/pages/tabs/storage.html.tmpl | 155 +++---- 10 files changed, 1107 insertions(+), 257 deletions(-) create mode 100644 ui/js/src/kimchi.storagepool_resize_volume_main.js create mode 100644 ui/pages/storagepool-resize-volume.html.tmpl -- 1.9.3

1 1

[PATCH] [Wok 2/2] Issue #115: _get_resources wok_log.error result into unicode error
by archus＠linux.vnet.ibm.com 23 May '16

23 May '16

From: Archana Singh <archus(a)linux.vnet.ibm.com> As ident is in encoded value but e.message is in unicode value, combination of which result into unicode value having encoded ident value. So when wok_log.error try to encode(encoded value) it result into error. This patch fix the unicode error by encoding e.message. Signed-off-by: Archana Singh <archus(a)linux.vnet.ibm.com> --- src/wok/control/base.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/wok/control/base.py b/src/wok/control/base.py index be5f618..d4fd4e4 100644 --- a/src/wok/control/base.py +++ b/src/wok/control/base.py @@ -373,9 +373,13 @@ class Collection(object): res.lookup() except Exception as e: # In case of errors when fetching a resource info, pass and - # log the error, so, other resources are returned + # log the error, so, other resources are returned. + # Encoding error message as ident is also encoded value. + # This has to be done to avoid unicode error, + # as combination of encoded and unicode value results into + # unicode error. wok_log.error("Problem in lookup of resource '%s'. " - "Detail: %s" % (ident, e.message)) + "Detail: %s" % (ident, encode_value(e.message))) continue res_list.append(res) return res_list -- 1.8.3.1

1 0