Just a minor comment below On 02/10/2014 10:50 AM, Paulo Vital wrote:

Define get and POST action update API for software update resource.

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- docs/API.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)

diff --git a/docs/API.md b/docs/API.md index 580728c..d45298e 100644 --- a/docs/API.md +++ b/docs/API.md @@ -741,3 +741,24 @@ Contains the host sample data. * size: The total size of the partition, in bytes * mountpoint: If the partition is mounted, represents the mountpoint. Otherwise blank. + +### Resource: Host Package Update + +**URI:** /host/packageupdate + +Contains the information and action of package/software update in the host. + +**Methods:** + +* **GET**: Retrieves a list of all packages to be updated in the host. + * packages to be updated in the format: IMO, the indent is not right? Not sure the format of the above sentence is what you want.

I covert it to html as follow: *Methods:* *GET*: Retrieves a list of all packages to be updated in the host. * packages to be updated in the format: * package: The name of the package to be updated * arch: The architecture of the package * version: The new version of the package * repository: The repository name from where package will be downloaded

+ * package: The name of the package to be updated + * arch: The architecture of the package + * version: The new version of the package + * repository: The repository name from where package will be downloaded + +* **POST**: *See Software Update Actions* + +**Actions (POST):** + +* update: Start the update of packages in background

-- Thanks and best regards! Sheldon Feng(???)<shaohef@linux.vnet.ibm.com> IBM Linux Technology Center

On 02/10/2014 12:50 AM, Paulo Vital wrote:

Define get and POST action update API for software update resource.

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- docs/API.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)

diff --git a/docs/API.md b/docs/API.md index 580728c..d45298e 100644 --- a/docs/API.md +++ b/docs/API.md @@ -741,3 +741,24 @@ Contains the host sample data. * size: The total size of the partition, in bytes * mountpoint: If the partition is mounted, represents the mountpoint. Otherwise blank. + +### Resource: Host Package Update + +**URI:** /host/packageupdate +

You will have the Resource and Collection /host/packageupdate = Resource /host/packagesupdates = Collection The Collection return a list of Resources And the Resource returns the data ### Collection: Host packages to be updated **URI:** /host/packagesupdate **Methods:** * **GET**: Retrieve a summarized list of all defined Virtual Machines ### Resource: Host package set to be update **URI:** /host/packageupdate/*:name* **Methods:** * **GET**: Retrieve the full description of a package * name: ... * arch: ... * version: ... * repository: ...

+Contains the information and action of package/software update in the host. + +**Methods:** + +* **GET**: Retrieves a list of all packages to be updated in the host. + * packages to be updated in the format: + * package: The name of the package to be updated + * arch: The architecture of the package + * version: The new version of the package + * repository: The repository name from where package will be downloaded + +* **POST**: *See Software Update Actions* + +**Actions (POST):** + +* update: Start the update of packages in background

On 02/10/2014 12:50 AM, Paulo Vital wrote:

Define PackageUpdate resource features according to API.md Activate auth support to new resource

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- src/kimchi/control/host.py | 12 ++++++++++++ 1 file changed, 12 insertions(+)

diff --git a/src/kimchi/control/host.py b/src/kimchi/control/host.py index 053c822..f382eb3 100644 --- a/src/kimchi/control/host.py +++ b/src/kimchi/control/host.py @@ -36,6 +36,8 @@ class Host(Resource): self.shutdown = self.generate_action_handler('shutdown') self.stats = HostStats(self.model) self.partitions = Partitions(self.model) + self.packageupdate = PackageUpdate(self.model)

+ self.packageupdate.expose = True

There is no need to expose it here.

@property def data(self): @@ -61,3 +63,13 @@ class Partition(Resource): @property def data(self): return self.info + + +class PackageUpdate(Resource): + def __init__(self, model, id=None): + super(PackageUpdate, self).__init__(model, id)

+ self.update = self.generate_action_handler('update')

As we will update all the packages at once the update action should be in the PackagesUpdate(Collection)

+ + @property + def data(self): + return self.info

Applying: Host's software update: Update backend. /kimchi/.git/rebase-apply/patch:50: trailing whitespace. 'package_name': 'sysconfig-network'}, warning: 1 line adds whitespace errors. On 02/10/2014 10:50 AM, Paulo Vital wrote:

Update model and mockmodel to support backend opertions. Add new file implementing backend operations, with four new classes:

1) SoftwareUpdate (object): Class to represent and operate with OS software update system in Kimchi's perspective. It's agnostic to host's package management system, and can execute all operations necessary: get all packages to update, get information about one package and execute the update. This class will load in runtime the necessary classes to work with the host's package management: YumUpdate for YUM systems based, AptUpdate for APT systems based and ZypperUpdate for Zypper systems based.

2) YumUpdate (object): Class to represent and operate with YUM. Loaded only on those systems that supports YUM, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

3) AptUpdate (object): Class to represent and operate with APT. Loaded only on those systems that supports APT, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

4) ZypperUpdate (object): Class to represent and operate with Zypper. Loaded only on those systems that supports Zypper, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- src/kimchi/mockmodel.py | 46 +++++++++ src/kimchi/model/host.py | 20 +++- src/kimchi/swupdate.py | 264 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 329 insertions(+), 1 deletion(-) create mode 100644 src/kimchi/swupdate.py

diff --git a/src/kimchi/mockmodel.py b/src/kimchi/mockmodel.py index 4e276eb..235c72f 100644 --- a/src/kimchi/mockmodel.py +++ b/src/kimchi/mockmodel.py @@ -75,6 +75,7 @@ class MockModel(object): self._mock_storagepools = {'default': MockStoragePool('default')} self._mock_networks = {'default': MockNetwork('default')} self._mock_interfaces = self.dummy_interfaces() + self._mock_swupdate = MockPackageUpdate() self.next_taskid = 1 self.storagepool_activate('default')

@@ -665,6 +666,16 @@ class MockModel(object): 'display_proxy_port': kconfig.get('display', 'display_proxy_port')}

+ def packageupdate_get_list(self): + return self._mock_swupdate.getUpdates() + + def packageupdate_lookup(self, pkg_name): + return self._mock_swupdate.getUpdate(pkg_name) + + def packageupdate_update(self, args=None): + kimchi_log.info('Host is going to be updated.') + self._mock_swupdate.doUpdate() +

class MockVMTemplate(VMTemplate): def __init__(self, args, mockmodel_inst=None): @@ -825,6 +836,41 @@ class MockVMScreenshot(VMScreenshot): image.save(thumbnail)

+class MockPackageUpdate(object): + def __init__(self): + self._packages = {'udevmountd': {'repo': 'openSUSE-13.1-Update', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'udevmountd'}, + 'sysconfig-network': {'repo': 'openSUSE-13.1-Extras', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'sysconfig-network'}, + 'libzypp': {'repo': 'openSUSE-13.1-Update', + 'version': '13.9.0-10.1', + 'arch': 'noarch', + 'package_name': 'libzypp'}} + self._num2update = 3 + + def getUpdates(self): + return self._packages + + def getUpdate(self, name): + if not name in self._packages.keys(): + raise NotFoundError("Package %s is not marked to be updated." % + name) + return self._packages[name] + + def getNumOfUpdates(self): + return self._num2update + + def doUpdate(self): + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + sleep(60) + + def get_mock_environment(): model = MockModel() for i in xrange(5): diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index a3d9e38..1bccd53 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -30,10 +30,12 @@ from cherrypy.process.plugins import BackgroundTask

from kimchi import disks from kimchi import netinfo +from kimchi import swupdate from kimchi.basemodel import Singleton from kimchi.exception import NotFoundError, OperationFailed +from kimchi.model.tasks import TaskModel from kimchi.model.vms import DOM_STATE_MAP -from kimchi.utils import kimchi_log +from kimchi.utils import add_task, kimchi_log

HOST_STATS_INTERVAL = 1 @@ -199,3 +201,19 @@ class PartitionModel(object): raise NotFoundError("Partition %s not found in the host" % name) return disks.get_partition_details(name) + + +class PackageUpdateModel(object): + def __init__(self, **kargs): + self.host_swupdate = swupdate.SoftwareUpdate() + self.objstore = kargs['objstore'] + self.task = TaskModel(**kargs) + + def lookup(self, *name): + return self.host_swupdate.getUpdates() + + def update(self, **kargs): + kimchi_log.info('Host is going to be updated.') + gen_cmd = self.host_swupdate.doUpdate() + taskid = add_task('', gen_cmd, self.objstore, 'packageupdate') + return self.task.lookup(taskid) diff --git a/src/kimchi/swupdate.py b/src/kimchi/swupdate.py new file mode 100644 index 0000000..5cda9c1 --- /dev/null +++ b/src/kimchi/swupdate.py @@ -0,0 +1,264 @@ +# +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# Authors: +# Paulo Vital <pvital@linux.vnet.ibm.com> +# Ramon Medeiros <ramonn@linux.vnet.ibm.com> +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import platform +import subprocess + +from kimchi.exception import NotFoundError, OperationFailed +from kimchi.utils import kimchi_log, run_command + +YUM_DISTROS = ['fedora', 'red hat enterprise linux', + 'red hat enterprise linux server'] +APT_DISTROS = ['debian', 'ubuntu'] +ZYPPER_DISTROS = ['opensuse ', 'suse linux enterprise server '] + + +class SoftwareUpdate(object): + """ + Class to represent and operate with OS software update. + """ + def __init__(self): + # This stores all packages to be updated for Kimchi perspective. It's a + # dictionary of dictionaries, in the format {'package_name': package}, + # where: + # package = {'package_name': <string>, 'version': <string>, + # 'arch': <string>, 'repo': <string> + # } + self._packages = {} + + # This stores the number of packages to update + self._num2update = 0 + + # Get the distro of host machine and creates an object related to + # correct package management system + self._distro = platform.linux_distribution()[0].lower() + if (self._distro in YUM_DISTROS): + kimchi_log.info("Loading YumUpdate features.") + self._pkg_mnger = YumUpdate() + elif (self._distro in APT_DISTROS): + kimchi_log.info("Loading AptUpdate features.") + self._pkg_mnger = AptUpdate() + elif (self._distro in ZYPPER_DISTROS): + kimchi_log.info("Loading ZypperUpdate features.") + self._pkg_mnger = ZypperUpdate() + else: + self._pkg_mnger = None + + if not self._pkg_mnger: + kimchi_log.info("There is no compatible package manager for \ + this system.") + + def _scanUpdates(self): + """ + Update self._packages with packages to be updated. + """ + self._packages = {} + self._num2update = 0 + + # Call system pkg_mnger to get the packages as list of dictionaries. + for pkg in self._pkg_mnger.getPackagesList(): + + # Check if already exist a package in self._packages + pkg_id = pkg.get('package_name') + if pkg_id in self._packages.keys(): + # package already listed to update. do nothing + continue + + # Update the self._packages and self._num2update + self._packages[pkg_id] = pkg + self._num2update = self._num2update + 1 + print self._num2update + print self._packages + + def getUpdates(self): + """ + Return the self._packages. + """ + self._scanUpdates() + return self._packages + + def getNumOfUpdates(self): + """ + Return the number of packages to be updated. + """ + self._scanUpdates() + return self._num2update + + def doUpdate(self): + """ + Execute the update + """ + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + return self._pkg_mnger.update() + + +class YumUpdate(object): + """ + Class to represent and operate with YUM software update system. + It's loaded only on those systems listed at YUM_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._yb = getattr(__import__('yum'), 'YumBase')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = self._yb.doPackageLists('updates') + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.name, + 'version': "%s-%s" % (pkg.version, pkg.release), + 'arch': pkg.arch, 'repo': pkg.ui_from_repo} + pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + # FIXME: Due to incompatabilities between cherrypy and yum/sqlite3 + # threading, we need execute the YUM command line to execute the update + cmd = ["yum", "-y", "-d", "0", "-e", "0", "update"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + +class AptUpdate(object): + """ + Class to represent and operate with APT software update system. + It's loaded only on those systems listed at APT_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._apt_cache = getattr(__import__('apt'), 'Cache')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._apt_cache.update() + self._apt_cache.upgrade() + self._pkgs = self._apt_cache.get_changes() + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.shortname, + 'version': pkg.candidate.version, + 'arch': pkg.architecture(), + 'repo': pkg.candidate.origins[0].label} + pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + try: + self._apt_cache.update() + self._apt_cache.open(None) + self._apt_cache.upgrade() + self._apt_cache.commit() + except Exception, e: + raise OperationFailed("ERROR when executing command: %s" % e) + + +class ZypperUpdate(object): + """ + Class to represent and operate with Zypper software update system. + It's loaded only on those systems listed at ZYPPER_DISTROS and loads + necessary modules in runtime. + """ + def __init__(self): + self._pkgs = {} + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = {} + cmd = ["zypper", "list-updates"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + for line in stdout.split('\n'): + if line.find('v |') >= 0: + info = line.split(' | ') + package = {'package_name': info[2], 'version': info[4], + 'arch': info[5], 'repo': info[1]} + self._pkgs[info[2]] = package + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + pkg_list.append(pkg) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + cmd = ["zypper", "--non-interactive", "update", + "--auto-agree-with-licenses"] + proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + stdout, stderr = proc.communicate() + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr)

Hello Sheldon, My answers about your comments inline: On Mon, 2014-02-10 at 16:57 +0800, Sheldon wrote:

Just a minor comment below

On 02/10/2014 10:50 AM, Paulo Vital wrote:

...

Update model and mockmodel to support backend opertions. Add new file implementing backend operations, with four new classes:

1) SoftwareUpdate (object): Class to represent and operate with OS software update system in Kimchi's perspective. It's agnostic to host's package management system, and can execute all operations necessary: get all packages to update, get information about one package and execute the update. This class will load in runtime the necessary classes to work with the host's package management: YumUpdate for YUM systems based, AptUpdate for APT systems based and ZypperUpdate for Zypper systems based.

2) YumUpdate (object): Class to represent and operate with YUM. Loaded only on those systems that supports YUM, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

3) AptUpdate (object): Class to represent and operate with APT. Loaded only on those systems that supports APT, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

4) ZypperUpdate (object): Class to represent and operate with Zypper. Loaded only on those systems that supports Zypper, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- src/kimchi/mockmodel.py | 46 +++++++++ src/kimchi/model/host.py | 20 +++- src/kimchi/swupdate.py | 264 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 329 insertions(+), 1 deletion(-) create mode 100644 src/kimchi/swupdate.py

diff --git a/src/kimchi/mockmodel.py b/src/kimchi/mockmodel.py index 4e276eb..235c72f 100644 --- a/src/kimchi/mockmodel.py +++ b/src/kimchi/mockmodel.py @@ -75,6 +75,7 @@ class MockModel(object): self._mock_storagepools = {'default': MockStoragePool('default')} self._mock_networks = {'default': MockNetwork('default')} self._mock_interfaces = self.dummy_interfaces() + self._mock_swupdate = MockPackageUpdate() self.next_taskid = 1 self.storagepool_activate('default')

@@ -665,6 +666,16 @@ class MockModel(object): 'display_proxy_port': kconfig.get('display', 'display_proxy_port')}

+ def packageupdate_get_list(self): + return self._mock_swupdate.getUpdates() + + def packageupdate_lookup(self, pkg_name): + return self._mock_swupdate.getUpdate(pkg_name) + + def packageupdate_update(self, args=None): + kimchi_log.info('Host is going to be updated.') + self._mock_swupdate.doUpdate() +

class MockVMTemplate(VMTemplate): def __init__(self, args, mockmodel_inst=None): @@ -825,6 +836,41 @@ class MockVMScreenshot(VMScreenshot): image.save(thumbnail)

+class MockPackageUpdate(object): + def __init__(self): + self._packages = {'udevmountd': {'repo': 'openSUSE-13.1-Update', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'udevmountd'}, + 'sysconfig-network': {'repo': 'openSUSE-13.1-Extras', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'sysconfig-network'}, + 'libzypp': {'repo': 'openSUSE-13.1-Update', + 'version': '13.9.0-10.1', + 'arch': 'noarch', + 'package_name': 'libzypp'}} + self._num2update = 3 + + def getUpdates(self): + return self._packages + + def getUpdate(self, name): + if not name in self._packages.keys(): + raise NotFoundError("Package %s is not marked to be updated." % + name) + return self._packages[name] + + def getNumOfUpdates(self): + return self._num2update + + def doUpdate(self): + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + sleep(60) + + def get_mock_environment(): model = MockModel() for i in xrange(5): diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index a3d9e38..1bccd53 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -30,10 +30,12 @@ from cherrypy.process.plugins import BackgroundTask

from kimchi import disks from kimchi import netinfo +from kimchi import swupdate from kimchi.basemodel import Singleton from kimchi.exception import NotFoundError, OperationFailed +from kimchi.model.tasks import TaskModel from kimchi.model.vms import DOM_STATE_MAP -from kimchi.utils import kimchi_log +from kimchi.utils import add_task, kimchi_log

HOST_STATS_INTERVAL = 1 @@ -199,3 +201,19 @@ class PartitionModel(object): raise NotFoundError("Partition %s not found in the host" % name) return disks.get_partition_details(name) + + +class PackageUpdateModel(object): + def __init__(self, **kargs): + self.host_swupdate = swupdate.SoftwareUpdate() + self.objstore = kargs['objstore'] + self.task = TaskModel(**kargs) + + def lookup(self, *name): + return self.host_swupdate.getUpdates() + + def update(self, **kargs): + kimchi_log.info('Host is going to be updated.') + gen_cmd = self.host_swupdate.doUpdate() + taskid = add_task('', gen_cmd, self.objstore, 'packageupdate') + return self.task.lookup(taskid) diff --git a/src/kimchi/swupdate.py b/src/kimchi/swupdate.py new file mode 100644 index 0000000..5cda9c1 --- /dev/null +++ b/src/kimchi/swupdate.py @@ -0,0 +1,264 @@ +# +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# Authors: +# Paulo Vital <pvital@linux.vnet.ibm.com> +# Ramon Medeiros <ramonn@linux.vnet.ibm.com> +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import platform +import subprocess + +from kimchi.exception import NotFoundError, OperationFailed +from kimchi.utils import kimchi_log, run_command + +YUM_DISTROS = ['fedora', 'red hat enterprise linux', + 'red hat enterprise linux server'] +APT_DISTROS = ['debian', 'ubuntu'] +ZYPPER_DISTROS = ['opensuse ', 'suse linux enterprise server '] + + +class SoftwareUpdate(object): + """ + Class to represent and operate with OS software update. + """ + def __init__(self): + # This stores all packages to be updated for Kimchi perspective. It's a + # dictionary of dictionaries, in the format {'package_name': package}, + # where: + # package = {'package_name': <string>, 'version': <string>, + # 'arch': <string>, 'repo': <string> + # } + self._packages = {} + + # This stores the number of packages to update + self._num2update = 0 + + # Get the distro of host machine and creates an object related to + # correct package management system + self._distro = platform.linux_distribution()[0].lower() + if (self._distro in YUM_DISTROS): + kimchi_log.info("Loading YumUpdate features.") + self._pkg_mnger = YumUpdate() + elif (self._distro in APT_DISTROS): + kimchi_log.info("Loading AptUpdate features.") + self._pkg_mnger = AptUpdate() + elif (self._distro in ZYPPER_DISTROS): + kimchi_log.info("Loading ZypperUpdate features.") + self._pkg_mnger = ZypperUpdate() + else: + self._pkg_mnger = None + + if not self._pkg_mnger: + kimchi_log.info("There is no compatible package manager for \ + this system.") + + def _scanUpdates(self): + """ + Update self._packages with packages to be updated. + """ + self._packages = {} + self._num2update = 0 + + # Call system pkg_mnger to get the packages as list of dictionaries. + for pkg in self._pkg_mnger.getPackagesList(): + + # Check if already exist a package in self._packages + pkg_id = pkg.get('package_name') + if pkg_id in self._packages.keys(): + # package already listed to update. do nothing + continue + + # Update the self._packages and self._num2update + self._packages[pkg_id] = pkg + self._num2update = self._num2update + 1 + print self._num2update + print self._packages log?

LOL! Actually it was a little debug print :-D Fix on V3.

...

+ + def getUpdates(self): + """ + Return the self._packages. + """ + self._scanUpdates() + return self._packages + + def getNumOfUpdates(self): + """ + Return the number of packages to be updated. + """ + self._scanUpdates() + return self._num2update + + def doUpdate(self): + """ + Execute the update + """ + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + return self._pkg_mnger.update() + + +class YumUpdate(object): + """ + Class to represent and operate with YUM software update system. + It's loaded only on those systems listed at YUM_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._yb = getattr(__import__('yum'), 'YumBase')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = self._yb.doPackageLists('updates') + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } indent?

Thanks! Fix on V3.

...

+ """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.name, + 'version': "%s-%s" % (pkg.version, pkg.release), + 'arch': pkg.arch, 'repo': pkg.ui_from_repo} + pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + # FIXME: Due to incompatabilities between cherrypy and yum/sqlite3 + # threading, we need execute the YUM command line to execute the update what incompatabilities?

The builtin yum module executes some sqlite queries to search RPM's database in a different thread, and at the moment to execute one of builtin yum module methods to execute the update, it breaks complaining about the thread which is calling this method is not the correct. Looks like the class responsible to execute this queries prevents that a different thread than thread running/importing the YumBase class execute the sqlite queries. Doing a basic debug, I could check that the CherryPy thread is the main thread responsible to execute this queries - instead of the thread running kimchi.server module. So, once the logic to update the packages using the builtin yum methods is the same of shell command "yum -y -d 0 -e 0 update", I decided to use this command until I debug better the problem and find an "elegant" solution.

...

+ cmd = ["yum", "-y", "-d", "0", "-e", "0", "update"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + +class AptUpdate(object): + """ + Class to represent and operate with APT software update system. + It's loaded only on those systems listed at APT_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._apt_cache = getattr(__import__('apt'), 'Cache')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._apt_cache.update() + self._apt_cache.upgrade() + self._pkgs = self._apt_cache.get_changes() + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } indent?

Thanks! Fix on V3.

...

+ """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.shortname, + 'version': pkg.candidate.version, + 'arch': pkg.architecture(), + 'repo': pkg.candidate.origins[0].label} + pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + try: + self._apt_cache.update() + self._apt_cache.open(None) + self._apt_cache.upgrade() + self._apt_cache.commit() + except Exception, e: + raise OperationFailed("ERROR when executing command: %s" % e) + + +class ZypperUpdate(object): + """ + Class to represent and operate with Zypper software update system. + It's loaded only on those systems listed at ZYPPER_DISTROS and loads + necessary modules in runtime. + """ + def __init__(self): + self._pkgs = {} + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = {} + cmd = ["zypper", "list-updates"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + for line in stdout.split('\n'): + if line.find('v |') >= 0: what does "v" means ?

It's related to the first column of the output. I understood that is about the status of the package in the system. All packages with "v" has update candidates to be installed. The output of this command, only prints packages with update candidates.

...

+ info = line.split(' | ') + package = {'package_name': info[2], 'version': info[4], + 'arch': info[5], 'repo': info[1]} + self._pkgs[info[2]] = package + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + pkg_list.append(pkg) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + cmd = ["zypper", "--non-interactive", "update", + "--auto-agree-with-licenses"] + proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + stdout, stderr = proc.communicate() why not? stdout, stderr, returncode = run_command(cmd) if len(stderr) > 0:

I explained this in feedback from Aline's comments in the V1 patch set.

...

+ + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr)

On 02/10/2014 12:50 AM, Paulo Vital wrote:

Update model and mockmodel to support backend opertions. Add new file implementing backend operations, with four new classes:

1) SoftwareUpdate (object): Class to represent and operate with OS software update system in Kimchi's perspective. It's agnostic to host's package management system, and can execute all operations necessary: get all packages to update, get information about one package and execute the update. This class will load in runtime the necessary classes to work with the host's package management: YumUpdate for YUM systems based, AptUpdate for APT systems based and ZypperUpdate for Zypper systems based.

2) YumUpdate (object): Class to represent and operate with YUM. Loaded only on those systems that supports YUM, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

3) AptUpdate (object): Class to represent and operate with APT. Loaded only on those systems that supports APT, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

4) ZypperUpdate (object): Class to represent and operate with Zypper. Loaded only on those systems that supports Zypper, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- src/kimchi/mockmodel.py | 46 +++++++++ src/kimchi/model/host.py | 20 +++- src/kimchi/swupdate.py | 264 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 329 insertions(+), 1 deletion(-) create mode 100644 src/kimchi/swupdate.py

diff --git a/src/kimchi/mockmodel.py b/src/kimchi/mockmodel.py index 4e276eb..235c72f 100644 --- a/src/kimchi/mockmodel.py +++ b/src/kimchi/mockmodel.py @@ -75,6 +75,7 @@ class MockModel(object): self._mock_storagepools = {'default': MockStoragePool('default')} self._mock_networks = {'default': MockNetwork('default')} self._mock_interfaces = self.dummy_interfaces() + self._mock_swupdate = MockPackageUpdate() self.next_taskid = 1 self.storagepool_activate('default')

@@ -665,6 +666,16 @@ class MockModel(object): 'display_proxy_port': kconfig.get('display', 'display_proxy_port')}

+ def packageupdate_get_list(self): + return self._mock_swupdate.getUpdates() +

It should be packagesupdate_get_list() after you declaring the PackagesUpdate(Collection)

+ def packageupdate_lookup(self, pkg_name): + return self._mock_swupdate.getUpdate(pkg_name) + + def packageupdate_update(self, args=None): + kimchi_log.info('Host is going to be updated.') + self._mock_swupdate.doUpdate() +

MockModel and Model need to have the same behavior So here you also should return a Task resource taskid = add_task('', self._mock_swupdate.doUpdate, objstore, None) return task_lookup(task_id)

class MockVMTemplate(VMTemplate): def __init__(self, args, mockmodel_inst=None): @@ -825,6 +836,41 @@ class MockVMScreenshot(VMScreenshot): image.save(thumbnail)

+class MockPackageUpdate(object): + def __init__(self): + self._packages = {'udevmountd': {'repo': 'openSUSE-13.1-Update', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'udevmountd'},

From API.md we have: package:... version:... arch:... repository:... The backend and API.md should be consistent. Fix it

+ 'sysconfig-network': {'repo': 'openSUSE-13.1-Extras', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'sysconfig-network'}, + 'libzypp': {'repo': 'openSUSE-13.1-Update', + 'version': '13.9.0-10.1', + 'arch': 'noarch', + 'package_name': 'libzypp'}} + self._num2update = 3 + + def getUpdates(self): + return self._packages + + def getUpdate(self, name): + if not name in self._packages.keys(): + raise NotFoundError("Package %s is not marked to be updated." % + name) + return self._packages[name] + + def getNumOfUpdates(self): + return self._num2update + + def doUpdate(self): + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + sleep(60) + + def get_mock_environment(): model = MockModel() for i in xrange(5): diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index a3d9e38..1bccd53 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -30,10 +30,12 @@ from cherrypy.process.plugins import BackgroundTask

from kimchi import disks from kimchi import netinfo +from kimchi import swupdate from kimchi.basemodel import Singleton from kimchi.exception import NotFoundError, OperationFailed +from kimchi.model.tasks import TaskModel from kimchi.model.vms import DOM_STATE_MAP -from kimchi.utils import kimchi_log +from kimchi.utils import add_task, kimchi_log

HOST_STATS_INTERVAL = 1 @@ -199,3 +201,19 @@ class PartitionModel(object): raise NotFoundError("Partition %s not found in the host" % name) return disks.get_partition_details(name) + + +class PackageUpdateModel(object): + def __init__(self, **kargs): + self.host_swupdate = swupdate.SoftwareUpdate() + self.objstore = kargs['objstore'] + self.task = TaskModel(**kargs) + + def lookup(self, *name): + return self.host_swupdate.getUpdates() +

+ def update(self, **kargs): + kimchi_log.info('Host is going to be updated.') + gen_cmd = self.host_swupdate.doUpdate() + taskid = add_task('', gen_cmd, self.objstore, 'packageupdate') + return self.task.lookup(taskid)

As I said before the update() action should be in PackagesUpdate(Collection) as we update all packages at once.

diff --git a/src/kimchi/swupdate.py b/src/kimchi/swupdate.py new file mode 100644 index 0000000..5cda9c1 --- /dev/null +++ b/src/kimchi/swupdate.py @@ -0,0 +1,264 @@ +# +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# Authors: +# Paulo Vital <pvital@linux.vnet.ibm.com> +# Ramon Medeiros <ramonn@linux.vnet.ibm.com> +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import platform +import subprocess + +from kimchi.exception import NotFoundError, OperationFailed +from kimchi.utils import kimchi_log, run_command + +YUM_DISTROS = ['fedora', 'red hat enterprise linux', + 'red hat enterprise linux server'] +APT_DISTROS = ['debian', 'ubuntu'] +ZYPPER_DISTROS = ['opensuse ', 'suse linux enterprise server '] + + +class SoftwareUpdate(object): + """ + Class to represent and operate with OS software update. + """ + def __init__(self): + # This stores all packages to be updated for Kimchi perspective. It's a + # dictionary of dictionaries, in the format {'package_name': package}, + # where: + # package = {'package_name': <string>, 'version': <string>, + # 'arch': <string>, 'repo': <string> + # } + self._packages = {} + + # This stores the number of packages to update + self._num2update = 0 + + # Get the distro of host machine and creates an object related to + # correct package management system + self._distro = platform.linux_distribution()[0].lower() + if (self._distro in YUM_DISTROS): + kimchi_log.info("Loading YumUpdate features.") + self._pkg_mnger = YumUpdate() + elif (self._distro in APT_DISTROS): + kimchi_log.info("Loading AptUpdate features.") + self._pkg_mnger = AptUpdate() + elif (self._distro in ZYPPER_DISTROS): + kimchi_log.info("Loading ZypperUpdate features.") + self._pkg_mnger = ZypperUpdate() + else: + self._pkg_mnger = None + + if not self._pkg_mnger: + kimchi_log.info("There is no compatible package manager for \ + this system.") + + def _scanUpdates(self): + """ + Update self._packages with packages to be updated. + """ + self._packages = {} + self._num2update = 0 + + # Call system pkg_mnger to get the packages as list of dictionaries. + for pkg in self._pkg_mnger.getPackagesList(): + + # Check if already exist a package in self._packages + pkg_id = pkg.get('package_name') + if pkg_id in self._packages.keys(): + # package already listed to update. do nothing + continue + + # Update the self._packages and self._num2update + self._packages[pkg_id] = pkg + self._num2update = self._num2update + 1 + print self._num2update + print self._packages + + def getUpdates(self): + """ + Return the self._packages. + """ + self._scanUpdates() + return self._packages + + def getNumOfUpdates(self): + """ + Return the number of packages to be updated. + """ + self._scanUpdates() + return self._num2update + + def doUpdate(self): + """ + Execute the update + """ + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + return self._pkg_mnger.update() + + +class YumUpdate(object): + """ + Class to represent and operate with YUM software update system. + It's loaded only on those systems listed at YUM_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._yb = getattr(__import__('yum'), 'YumBase')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = self._yb.doPackageLists('updates') + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.name, + 'version': "%s-%s" % (pkg.version, pkg.release), + 'arch': pkg.arch, 'repo': pkg.ui_from_repo}

In API.md you used: package:... version:... arch:... repository:... Make it consistent.

+ pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + # FIXME: Due to incompatabilities between cherrypy and yum/sqlite3 + # threading, we need execute the YUM command line to execute the update + cmd = ["yum", "-y", "-d", "0", "-e", "0", "update"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + +class AptUpdate(object): + """ + Class to represent and operate with APT software update system. + It's loaded only on those systems listed at APT_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._apt_cache = getattr(__import__('apt'), 'Cache')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._apt_cache.update() + self._apt_cache.upgrade() + self._pkgs = self._apt_cache.get_changes() + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.shortname, + 'version': pkg.candidate.version, + 'arch': pkg.architecture(), + 'repo': pkg.candidate.origins[0].label} + pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + try: + self._apt_cache.update() + self._apt_cache.open(None) + self._apt_cache.upgrade() + self._apt_cache.commit() + except Exception, e: + raise OperationFailed("ERROR when executing command: %s" % e) + + +class ZypperUpdate(object): + """ + Class to represent and operate with Zypper software update system. + It's loaded only on those systems listed at ZYPPER_DISTROS and loads + necessary modules in runtime. + """ + def __init__(self): + self._pkgs = {} + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = {} + cmd = ["zypper", "list-updates"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + for line in stdout.split('\n'): + if line.find('v |') >= 0: + info = line.split(' | ') + package = {'package_name': info[2], 'version': info[4], + 'arch': info[5], 'repo': info[1]} + self._pkgs[info[2]] = package + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + pkg_list.append(pkg) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + cmd = ["zypper", "--non-interactive", "update", + "--auto-agree-with-licenses"] + proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + stdout, stderr = proc.communicate() + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr)

Some small grammatical suggestions below. On Mon, 2014-02-10 at 00:50 -0200, Paulo Vital wrote:

Update model and mockmodel to support backend opertions. Add new file implementing backend operations, with four new classes:

1) SoftwareUpdate (object): Class to represent and operate with OS software update system in Kimchi's perspective. It's agnostic to host's package management system, and can execute all operations necessary: get all packages to update, get information about one package and execute the update. This class will load in runtime the necessary classes to work with the host's package management: YumUpdate for YUM systems based, AptUpdate for APT systems based and ZypperUpdate for Zypper systems based. Change those to: "[tool]-based systems" (e.g. APT-based systems).

2) YumUpdate (object): Class to represent and operate with YUM. Loaded only on those systems that supports YUM, it's responsible to connect and collect

The little blurbs in 1-4 can all be updated similarly: Class to represent and operate with [tool]. Loaded only if the host supports [tool], it is responsible for querying and collecting information on the packages to be updated. It also executes the system update.

information of the packages to be updated. Also it's responsible to execute the update of the system.

3) AptUpdate (object): Class to represent and operate with APT. Loaded only on those systems that supports APT, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

4) ZypperUpdate (object): Class to represent and operate with Zypper. Loaded only on those systems that supports Zypper, it's responsible to connect and collect information of the packages to be updated. Also it's responsible to execute the update of the system.

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- src/kimchi/mockmodel.py | 46 +++++++++ src/kimchi/model/host.py | 20 +++- src/kimchi/swupdate.py | 264 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 329 insertions(+), 1 deletion(-) create mode 100644 src/kimchi/swupdate.py

diff --git a/src/kimchi/mockmodel.py b/src/kimchi/mockmodel.py index 4e276eb..235c72f 100644 --- a/src/kimchi/mockmodel.py +++ b/src/kimchi/mockmodel.py @@ -75,6 +75,7 @@ class MockModel(object): self._mock_storagepools = {'default': MockStoragePool('default')} self._mock_networks = {'default': MockNetwork('default')} self._mock_interfaces = self.dummy_interfaces() + self._mock_swupdate = MockPackageUpdate() self.next_taskid = 1 self.storagepool_activate('default')

@@ -665,6 +666,16 @@ class MockModel(object): 'display_proxy_port': kconfig.get('display', 'display_proxy_port')}

+ def packageupdate_get_list(self): + return self._mock_swupdate.getUpdates() + + def packageupdate_lookup(self, pkg_name): + return self._mock_swupdate.getUpdate(pkg_name) + + def packageupdate_update(self, args=None): + kimchi_log.info('Host is going to be updated.') + self._mock_swupdate.doUpdate() +

class MockVMTemplate(VMTemplate): def __init__(self, args, mockmodel_inst=None): @@ -825,6 +836,41 @@ class MockVMScreenshot(VMScreenshot): image.save(thumbnail)

+class MockPackageUpdate(object): + def __init__(self): + self._packages = {'udevmountd': {'repo': 'openSUSE-13.1-Update', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'udevmountd'}, + 'sysconfig-network': {'repo': 'openSUSE-13.1-Extras', + 'version': '0.81.5-14.1', + 'arch': 'x86_64', + 'package_name': 'sysconfig-network'}, + 'libzypp': {'repo': 'openSUSE-13.1-Update', + 'version': '13.9.0-10.1', + 'arch': 'noarch', + 'package_name': 'libzypp'}} + self._num2update = 3 + + def getUpdates(self): + return self._packages + + def getUpdate(self, name): + if not name in self._packages.keys(): + raise NotFoundError("Package %s is not marked to be updated." % + name) + return self._packages[name] + + def getNumOfUpdates(self): + return self._num2update + + def doUpdate(self): + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + sleep(60) + + def get_mock_environment(): model = MockModel() for i in xrange(5): diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index a3d9e38..1bccd53 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -30,10 +30,12 @@ from cherrypy.process.plugins import BackgroundTask

from kimchi import disks from kimchi import netinfo +from kimchi import swupdate from kimchi.basemodel import Singleton from kimchi.exception import NotFoundError, OperationFailed +from kimchi.model.tasks import TaskModel from kimchi.model.vms import DOM_STATE_MAP -from kimchi.utils import kimchi_log +from kimchi.utils import add_task, kimchi_log

HOST_STATS_INTERVAL = 1 @@ -199,3 +201,19 @@ class PartitionModel(object): raise NotFoundError("Partition %s not found in the host" % name) return disks.get_partition_details(name) + + +class PackageUpdateModel(object): + def __init__(self, **kargs): + self.host_swupdate = swupdate.SoftwareUpdate() + self.objstore = kargs['objstore'] + self.task = TaskModel(**kargs) + + def lookup(self, *name): + return self.host_swupdate.getUpdates() + + def update(self, **kargs): + kimchi_log.info('Host is going to be updated.') + gen_cmd = self.host_swupdate.doUpdate() + taskid = add_task('', gen_cmd, self.objstore, 'packageupdate') + return self.task.lookup(taskid) diff --git a/src/kimchi/swupdate.py b/src/kimchi/swupdate.py new file mode 100644 index 0000000..5cda9c1 --- /dev/null +++ b/src/kimchi/swupdate.py @@ -0,0 +1,264 @@ +# +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# Authors: +# Paulo Vital <pvital@linux.vnet.ibm.com> +# Ramon Medeiros <ramonn@linux.vnet.ibm.com> +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import platform +import subprocess + +from kimchi.exception import NotFoundError, OperationFailed +from kimchi.utils import kimchi_log, run_command + +YUM_DISTROS = ['fedora', 'red hat enterprise linux', + 'red hat enterprise linux server'] +APT_DISTROS = ['debian', 'ubuntu'] +ZYPPER_DISTROS = ['opensuse ', 'suse linux enterprise server '] + + +class SoftwareUpdate(object): + """ + Class to represent and operate with OS software update. + """ + def __init__(self): + # This stores all packages to be updated for Kimchi perspective. It's a + # dictionary of dictionaries, in the format {'package_name': package}, + # where: + # package = {'package_name': <string>, 'version': <string>, + # 'arch': <string>, 'repo': <string> + # } + self._packages = {} + + # This stores the number of packages to update + self._num2update = 0 + + # Get the distro of host machine and creates an object related to + # correct package management system + self._distro = platform.linux_distribution()[0].lower() + if (self._distro in YUM_DISTROS): + kimchi_log.info("Loading YumUpdate features.") + self._pkg_mnger = YumUpdate() + elif (self._distro in APT_DISTROS): + kimchi_log.info("Loading AptUpdate features.") + self._pkg_mnger = AptUpdate() + elif (self._distro in ZYPPER_DISTROS): + kimchi_log.info("Loading ZypperUpdate features.") + self._pkg_mnger = ZypperUpdate() + else: + self._pkg_mnger = None + + if not self._pkg_mnger: + kimchi_log.info("There is no compatible package manager for \ + this system.") + + def _scanUpdates(self): + """ + Update self._packages with packages to be updated. + """ + self._packages = {} + self._num2update = 0 + + # Call system pkg_mnger to get the packages as list of dictionaries. + for pkg in self._pkg_mnger.getPackagesList(): What if somehow there's no pkg manager? Should this check first? + + # Check if already exist a package in self._packages + pkg_id = pkg.get('package_name') + if pkg_id in self._packages.keys(): + # package already listed to update. do nothing + continue + + # Update the self._packages and self._num2update + self._packages[pkg_id] = pkg + self._num2update = self._num2update + 1 + print self._num2update + print self._packages + + def getUpdates(self): + """ + Return the self._packages. + """ + self._scanUpdates() + return self._packages + + def getNumOfUpdates(self): + """ + Return the number of packages to be updated. + """ + self._scanUpdates() + return self._num2update + + def doUpdate(self): + """ + Execute the update + """ + if self._num2update == 0: + kimchi_log.info("No packages marked for update") + raise OperationFailed("No packages marked for update") + return self._pkg_mnger.update() + +class YumUpdate(object): + """ + Class to represent and operate with YUM software update system. + It's loaded only on those systems listed at YUM_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._yb = getattr(__import__('yum'), 'YumBase')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = self._yb.doPackageLists('updates') + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.name, + 'version': "%s-%s" % (pkg.version, pkg.release), + 'arch': pkg.arch, 'repo': pkg.ui_from_repo} + pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + # FIXME: Due to incompatabilities between cherrypy and yum/sqlite3 + # threading, we need execute the YUM command line to execute the update + cmd = ["yum", "-y", "-d", "0", "-e", "0", "update"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + +class AptUpdate(object): + """ + Class to represent and operate with APT software update system. + It's loaded only on those systems listed at APT_DISTROS and loads necessary + modules in runtime. + """ + def __init__(self): + self._pkgs = {} + self._apt_cache = getattr(__import__('apt'), 'Cache')() + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._apt_cache.update() + self._apt_cache.upgrade() + self._pkgs = self._apt_cache.get_changes() + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + package = {'package_name': pkg.shortname, + 'version': pkg.candidate.version, + 'arch': pkg.architecture(), + 'repo': pkg.candidate.origins[0].label} + pkg_list.append(package) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + try: + self._apt_cache.update() + self._apt_cache.open(None) + self._apt_cache.upgrade() + self._apt_cache.commit() + except Exception, e: + raise OperationFailed("ERROR when executing command: %s" % e) + + +class ZypperUpdate(object): + """ + Class to represent and operate with Zypper software update system. + It's loaded only on those systems listed at ZYPPER_DISTROS and loads + necessary modules in runtime. + """ + def __init__(self): + self._pkgs = {} + + def _refreshUpdateList(self): + """ + Update the list of packages to be updated in the system. + """ + self._pkgs = {} + cmd = ["zypper", "list-updates"] + (stdout, stderr, returncode) = run_command(cmd) + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr) + + for line in stdout.split('\n'): + if line.find('v |') >= 0: + info = line.split(' | ') + package = {'package_name': info[2], 'version': info[4], + 'arch': info[5], 'repo': info[1]} + self._pkgs[info[2]] = package + + def getPackagesList(self): + """ + Return a list of package's dictionaries. Each dictionary contains the + information about a package, in the format + package = {'package_name': <string>, 'version': <string>, + 'arch': <string>, 'repo': <string> + } + """ + self._refreshUpdateList() + pkg_list = [] + for pkg in self._pkgs: + pkg_list.append(pkg) + return pkg_list + + def update(self): + """ + Execute the update of all packages marked to be update. + """ + cmd = ["zypper", "--non-interactive", "update", + "--auto-agree-with-licenses"] + proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + stdout, stderr = proc.communicate() + + if len(stderr) > 0: + raise OperationFailed("ERROR when executing command: %s" % stderr)

Reviewed-by: ShaoHe Feng <shaohef@linux.vnet.ibm.com> On 02/10/2014 10:50 AM, Paulo Vital wrote:

Update Makefile to provide software update support

Signed-off-by: Paulo Vital <pvital@linux.vnet.ibm.com> Signed-off-by: Ramon Medeiros <ramonn@linux.vnet.ibm.com> --- Makefile.am | 1 + src/kimchi/Makefile.am | 1 + 2 files changed, 2 insertions(+)

diff --git a/Makefile.am b/Makefile.am index 266f78f..f89dcf2 100644 --- a/Makefile.am +++ b/Makefile.am @@ -57,6 +57,7 @@ PEP8_WHITELIST = \ src/kimchi/rollbackcontext.py \ src/kimchi/root.py \ src/kimchi/server.py \ + src/kimchi/swupdate.py \ src/kimchi/utils.py \ tests/test_config.py.in \ tests/test_mockmodel.py \ diff --git a/src/kimchi/Makefile.am b/src/kimchi/Makefile.am index 1653c0c..1c65bea 100644 --- a/src/kimchi/Makefile.am +++ b/src/kimchi/Makefile.am @@ -46,6 +46,7 @@ kimchi_PYTHON = \ screenshot.py \ server.py \ sslcert.py \ + swupdate.py \ template.py \ utils.py \ vmtemplate.py \

-- Thanks and best regards! Sheldon Feng(冯少合)<shaohef@linux.vnet.ibm.com> IBM Linux Technology Center