Re: [Kimchi-devel] [ginger-dev-list] [PATCH V3] [Gingerbase 0/4] Issue #9 - Update selected packages.
On Fri, Mar 11, 2016 at 06:55:02PM -0300, Aline Manera wrote:
On 03/11/2016 04:48 PM, Aline Manera wrote:
On 03/10/2016 05:51 PM, Paulo Ricardo Paz Vital wrote:
On 03/10/2016 04:23 PM, Daniel Henrique Barboza wrote:
I've tested the patch in a real environment (my dev PC) and noticed the following behavior:
- pick a single package to be updated. In this case I've chosen the 'libv4l' package:
[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l' -X GET Enter host password for user 'root': { "depends":[ "libv4l" ], "version":"1.10.0-2.fc23", "arch":"i686", "repository":"updates", "package_name":"libv4l" }
[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l/upgrade'
-X POST -d'{}' Enter host password for user 'root': { "status":"running", "message":"", "id":"1", "target_uri":"/plugins/gingerbase/host/packagesupdate/libv4l/upgrade" }[danielhb@arthas gingerbase]$
- after the update process, the lookup was still returning package info, as if it was still available for updating:
[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l' -X GET Enter host password for user 'root': { "depends":[ "libv4l" ], "version":"1.10.0-2.fc23", "arch":"i686", "repository":"updates", "package_name":"libv4l" }
- after that I've listed all the packages to be updated, wondering if there was some sort of refresh problem after the single update:
}[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: aplication/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/' -X GET (... long output)
- and finally tried the lookup of the libv4l package again:
][danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: aplication/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l' -X GET Enter host password for user 'root': { "reason":"GGBPKGUPD0002E: Package libv4l is not marked to be updated.", "code":"404 Not Found", "call_stack":"Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py\", line 670, in respond\n response.body = self.handler()\n File \"/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py\", line 217, in __call__\n self.body = self.oldhandler(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py\", line 61, in __call__\n return self.callable(*self.args, **self.kwargs)\n File \"/home/danielhb/kimchi/wok_ginger/src/wok/control/base.py\", line 210, in index\n raise cherrypy.HTTPError(404, e.message)\nHTTPError: (404, u'GGBPKGUPD0002E: Package libv4l is not marked to be updated.')\n" }[
My conclusion is that, after the single package update, we need to do a list of the all the packages again to update/refresh some internal model info.
Yes, this happens because the lookup of one single package returns the dictionary from the internal list, after the first overall lookup.
In my opinion the package list needs to be updated after every single update. Perhaps there is a way of simply removing the updated package(s) from this list instead of refreshing the whole list again, operation that is now even slower than before due to all the additional info being fetched.
Ok, I'll add something to remove from the internal list of packages the package and its dependencies in the V4
My suggestion is to save the timestamp when a request to get_list() is made. And on lookup() verifies the timestamp does not exceed X seconds and if so, grab the information from yum again for the given resource.
Instead of doing that, it would be better to do not cache the information on get_list(). get_list() should return only the list of packages names and lookup() grabs the detailed information for each package.
As you are calling a new command on lookup() for a given package with this patch (to get the list of dependencies), I believe this command you are using also returns the information cached on get_list() - repository, version, etc.
The way PackagesUpdate works is different from what you described here. PackagesUpdate.get_list() returns the same return of SoftwareUpdate.getUpdates() that is, basically, the cached dictionary (called SoftwareUpdate._packages) that contains all packages eligible to update and it's information (package_name, version, arch, repository and, now, list of dependencies). All these information are collected by the private method SoftwareUpdate._scanUpdates() that is executed at first time, when the first PackagesUpdate.get_list() is requested by user. PackageUpdate.lookup() returns the information of one package by calling SoftwareUpdate.getUpdate(<package_name>), that is nothing more that the information of that specific package cached in SoftwareUpdate._packages.
Said that, I'd say to you change get_list() to only returns the packages names. And improve your command output on lookup() to grab the information before cached on get_list().
Ok, interesting idea, but doesn't this change how UI and user see the information - I mean, how API should works? If following your idea, when user access the Updates tab on UI, he/she will see only the list of packages. Today, it's provided the information of the bew version number of that package, the arch and repository that provides the package.
If you fail to parse the command output on lookup() that means the package is not marked to be upgraded so raise an error.
Something like below on lookup():
out = run_command(<command to get the dependencies>) name = get_name_from_out() if name is None: # package is not marked to be upgraded raise NotFound()
deps = get_deps_from_out() ...
return info
Thanks,
Daniel
On 03/09/2016 04:01 PM, pvital@linux.vnet.ibm.com wrote:
From: Paulo Vital <pvital@linux.vnet.ibm.com>
V2 - V3: * Modified _get_dnf_info_requires() of DnfUpdate class to prevent the addition of the package in the dependencies list, and how to process the name of the dependencies to be added.
V2:
This patch-set is the V2 of the feature to add backend support to select one or more packages to be updated by host package manager.
All modifications for V2 were based on the feedback provided by the mail thread "[RFC] Gingerbase Issue#9: Update selected packages."
To test, use:
# List all packages selected by host package manager, as eligible to be updated $ curl -k -u test -H "Content-Type: application/json" -H \ "Accept: application/json" \ 'https://localhost:8001/plugins/gingerbase/host/packagesupdate' -X GET Enter host password for user 'test': [ { "depends":[ "libudisks2", "udisks2" ], "version":"2.1.7-1.fc23", "arch":"x86_64", "repository":"updates", "package_name":"libudisks2" }, { "depends":[ "glusterfs-client-xlators", "glusterfs", "glusterfs-fuse", "glusterfs-libs", "glusterfs-api" ], "version":"3.7.8-2.fc23", "arch":"x86_64", "repository":"updates", "package_name":"glusterfs-client-xlators" }, { "depends":[ "parted" ], "version":"3.2-16.fc23", "arch":"x86_64", "repository":"updates", "package_name":"parted" },
< result cut, due to a large number of packages >
{ "depends":[ "openssl", "openssl-libs" ], "version":"1:1.0.2g-2.fc23", "arch":"x86_64", "repository":"updates", "package_name":"openssl" }, { "depends":[ "gnome-shell" ], "version":"3.18.4-1.fc23", "arch":"x86_64", "repository":"updates", "package_name":"gnome-shell" }, { "depends":[ "openssl", "openssl-libs" ], "version":"1:1.0.2g-2.fc23", "arch":"i686", "repository":"updates", "package_name":"openssl-libs" },
< result cut, due to a large number of packages >
]
# Print information about one selected package $ curl -k -u test -H "Content-Type: application/json" -H \ "Accept: application/json" \ 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/openssl' -X GET Enter host password for user 'test': { "depends":[ "openssl", "openssl-libs" ], "version":"1:1.0.2g-2.fc23", "arch":"x86_64", "repository":"updates", "package_name":"openssl" }
# Select only one package to be updated. All packages that were listed as # eligible to be update and are dependencies of this package will also be # updated automatically. $ curl -k -u test -H "Content-Type: application/json" -H \ "Accept: application/json" \ 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/openssl/upgrad...'
\ -X POST -d '{}' Enter host password for user 'test': { "status":"running", "message":"", "id":"1", "target_uri":"/plugins/gingerbase/host/packagesupdate/openssl/upgrade" }
Paulo Vital (4): Issue #9 - Reorganize Package Update classes. Issue #9 - List dependencies of packages to update Issue #9 - Add support to update selected packages Issue #9 - Test cases to update selected packages
control/host.py | 36 ++-------- control/packagesupdate.py | 53 ++++++++++++++ docs/API.md | 18 +++-- mockmodel.py | 39 ++++++++--- model/host.py | 43 ------------ model/packagesupdate.py | 116 +++++++++++++++++++++++++++++++ swupdate.py | 172 +++++++++++++++++++++++++++++++++++++++++----- tests/test_host.py | 37 ++++++++-- yumparser.py | 16 ++--- 9 files changed, 404 insertions(+), 126 deletions(-) create mode 100644 control/packagesupdate.py create mode 100644 model/packagesupdate.py
-- 2.5.0
-- You received this message because you are subscribed to the Google Groups "Ginger Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an email to ginger-dev-list+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
-- Paulo Ricardo Paz Vital IBM Linux Technology Center http://www.ibm.com
OOOpps! Sent to wrong ML :-P On Mon, Mar 14, 2016 at 10:20:00AM -0300, Paulo Ricardo Paz Vital wrote:
On Fri, Mar 11, 2016 at 06:55:02PM -0300, Aline Manera wrote:
On 03/11/2016 04:48 PM, Aline Manera wrote:
On 03/10/2016 05:51 PM, Paulo Ricardo Paz Vital wrote:
On 03/10/2016 04:23 PM, Daniel Henrique Barboza wrote:
I've tested the patch in a real environment (my dev PC) and noticed the following behavior:
- pick a single package to be updated. In this case I've chosen the 'libv4l' package:
[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l' -X GET Enter host password for user 'root': { "depends":[ "libv4l" ], "version":"1.10.0-2.fc23", "arch":"i686", "repository":"updates", "package_name":"libv4l" }
[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l/upgrade'
-X POST -d'{}' Enter host password for user 'root': { "status":"running", "message":"", "id":"1", "target_uri":"/plugins/gingerbase/host/packagesupdate/libv4l/upgrade" }[danielhb@arthas gingerbase]$
- after the update process, the lookup was still returning package info, as if it was still available for updating:
[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l' -X GET Enter host password for user 'root': { "depends":[ "libv4l" ], "version":"1.10.0-2.fc23", "arch":"i686", "repository":"updates", "package_name":"libv4l" }
- after that I've listed all the packages to be updated, wondering if there was some sort of refresh problem after the single update:
}[danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: aplication/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/' -X GET (... long output)
- and finally tried the lookup of the libv4l package again:
][danielhb@arthas gingerbase]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: aplication/json" 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/libv4l' -X GET Enter host password for user 'root': { "reason":"GGBPKGUPD0002E: Package libv4l is not marked to be updated.", "code":"404 Not Found", "call_stack":"Traceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py\", line 670, in respond\n response.body = self.handler()\n File \"/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py\", line 217, in __call__\n self.body = self.oldhandler(*args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py\", line 61, in __call__\n return self.callable(*self.args, **self.kwargs)\n File \"/home/danielhb/kimchi/wok_ginger/src/wok/control/base.py\", line 210, in index\n raise cherrypy.HTTPError(404, e.message)\nHTTPError: (404, u'GGBPKGUPD0002E: Package libv4l is not marked to be updated.')\n" }[
My conclusion is that, after the single package update, we need to do a list of the all the packages again to update/refresh some internal model info.
Yes, this happens because the lookup of one single package returns the dictionary from the internal list, after the first overall lookup.
In my opinion the package list needs to be updated after every single update. Perhaps there is a way of simply removing the updated package(s) from this list instead of refreshing the whole list again, operation that is now even slower than before due to all the additional info being fetched.
Ok, I'll add something to remove from the internal list of packages the package and its dependencies in the V4
My suggestion is to save the timestamp when a request to get_list() is made. And on lookup() verifies the timestamp does not exceed X seconds and if so, grab the information from yum again for the given resource.
Instead of doing that, it would be better to do not cache the information on get_list(). get_list() should return only the list of packages names and lookup() grabs the detailed information for each package.
As you are calling a new command on lookup() for a given package with this patch (to get the list of dependencies), I believe this command you are using also returns the information cached on get_list() - repository, version, etc.
The way PackagesUpdate works is different from what you described here.
PackagesUpdate.get_list() returns the same return of SoftwareUpdate.getUpdates() that is, basically, the cached dictionary (called SoftwareUpdate._packages) that contains all packages eligible to update and it's information (package_name, version, arch, repository and, now, list of dependencies). All these information are collected by the private method SoftwareUpdate._scanUpdates() that is executed at first time, when the first PackagesUpdate.get_list() is requested by user.
PackageUpdate.lookup() returns the information of one package by calling SoftwareUpdate.getUpdate(<package_name>), that is nothing more that the information of that specific package cached in SoftwareUpdate._packages.
Said that, I'd say to you change get_list() to only returns the packages names. And improve your command output on lookup() to grab the information before cached on get_list().
Ok, interesting idea, but doesn't this change how UI and user see the information - I mean, how API should works?
If following your idea, when user access the Updates tab on UI, he/she will see only the list of packages. Today, it's provided the information of the bew version number of that package, the arch and repository that provides the package.
If you fail to parse the command output on lookup() that means the package is not marked to be upgraded so raise an error.
Something like below on lookup():
out = run_command(<command to get the dependencies>) name = get_name_from_out() if name is None: # package is not marked to be upgraded raise NotFound()
deps = get_deps_from_out() ...
return info
Thanks,
Daniel
On 03/09/2016 04:01 PM, pvital@linux.vnet.ibm.com wrote:
From: Paulo Vital <pvital@linux.vnet.ibm.com>
V2 - V3: * Modified _get_dnf_info_requires() of DnfUpdate class to prevent the addition of the package in the dependencies list, and how to process the name of the dependencies to be added.
V2:
This patch-set is the V2 of the feature to add backend support to select one or more packages to be updated by host package manager.
All modifications for V2 were based on the feedback provided by the mail thread "[RFC] Gingerbase Issue#9: Update selected packages."
To test, use:
# List all packages selected by host package manager, as eligible to be updated $ curl -k -u test -H "Content-Type: application/json" -H \ "Accept: application/json" \ 'https://localhost:8001/plugins/gingerbase/host/packagesupdate' -X GET Enter host password for user 'test': [ { "depends":[ "libudisks2", "udisks2" ], "version":"2.1.7-1.fc23", "arch":"x86_64", "repository":"updates", "package_name":"libudisks2" }, { "depends":[ "glusterfs-client-xlators", "glusterfs", "glusterfs-fuse", "glusterfs-libs", "glusterfs-api" ], "version":"3.7.8-2.fc23", "arch":"x86_64", "repository":"updates", "package_name":"glusterfs-client-xlators" }, { "depends":[ "parted" ], "version":"3.2-16.fc23", "arch":"x86_64", "repository":"updates", "package_name":"parted" },
< result cut, due to a large number of packages >
{ "depends":[ "openssl", "openssl-libs" ], "version":"1:1.0.2g-2.fc23", "arch":"x86_64", "repository":"updates", "package_name":"openssl" }, { "depends":[ "gnome-shell" ], "version":"3.18.4-1.fc23", "arch":"x86_64", "repository":"updates", "package_name":"gnome-shell" }, { "depends":[ "openssl", "openssl-libs" ], "version":"1:1.0.2g-2.fc23", "arch":"i686", "repository":"updates", "package_name":"openssl-libs" },
< result cut, due to a large number of packages >
]
# Print information about one selected package $ curl -k -u test -H "Content-Type: application/json" -H \ "Accept: application/json" \ 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/openssl' -X GET Enter host password for user 'test': { "depends":[ "openssl", "openssl-libs" ], "version":"1:1.0.2g-2.fc23", "arch":"x86_64", "repository":"updates", "package_name":"openssl" }
# Select only one package to be updated. All packages that were listed as # eligible to be update and are dependencies of this package will also be # updated automatically. $ curl -k -u test -H "Content-Type: application/json" -H \ "Accept: application/json" \ 'https://localhost:8001/plugins/gingerbase/host/packagesupdate/openssl/upgrad...'
\ -X POST -d '{}' Enter host password for user 'test': { "status":"running", "message":"", "id":"1", "target_uri":"/plugins/gingerbase/host/packagesupdate/openssl/upgrade" }
Paulo Vital (4): Issue #9 - Reorganize Package Update classes. Issue #9 - List dependencies of packages to update Issue #9 - Add support to update selected packages Issue #9 - Test cases to update selected packages
control/host.py | 36 ++-------- control/packagesupdate.py | 53 ++++++++++++++ docs/API.md | 18 +++-- mockmodel.py | 39 ++++++++--- model/host.py | 43 ------------ model/packagesupdate.py | 116 +++++++++++++++++++++++++++++++ swupdate.py | 172 +++++++++++++++++++++++++++++++++++++++++----- tests/test_host.py | 37 ++++++++-- yumparser.py | 16 ++--- 9 files changed, 404 insertions(+), 126 deletions(-) create mode 100644 control/packagesupdate.py create mode 100644 model/packagesupdate.py
-- 2.5.0
-- You received this message because you are subscribed to the Google Groups "Ginger Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an email to ginger-dev-list+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
-- Paulo Ricardo Paz Vital IBM Linux Technology Center http://www.ibm.com
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
-- Paulo Ricardo Paz Vital IBM Linux Technology Center http://www.ibm.com
participants (1)
-
Paulo Ricardo Paz Vital