September 2015 - Kimchi-devel - oVirt List Archives

RFC - Guest memory usage
by Daniel Henrique Barboza 27 Oct '15

27 Oct '15

I was looking at the following item in the backlog: Guests Stats: Display memory utilization (use virt-df or virt-top or ...) If I understood it right, the idea here is to show the -inner- memory allocation of the guest. If you have a VM with 4Gb of RAM running an Ubuntu, we want to know how much memory the Ubuntu OS and its processes are using. I've done an investigation and I haven't found any tool to accomplish this. "virt-top", "virsh dommemstat" and the libvirt API retrieves the information of the memory usage of the guest relative to the host. In the example mentioned before, supposing that the host has 64Gb of RAM, all these tools would show that the VM is using 12% of the host RAM. They do not dive in the VM and shows the actual mem usage of the Ubuntu and its processes running there. Haven't found anything useful in other MLs and forums. The common answer is 'run top in a terminal inside the VM', which of course does not suit us. My question is: any thoughts about how we can implement this feature? Because I am starting to think that, in the end, this kind of info is strict to the guest OS and can't be polled from the outside. Thanks!

5 5

[WOK RFC] Update objectstore information while migrating to new structure.
by Paulo Ricardo Paz Vital 14 Oct '15

14 Oct '15

Hello guys. Doing some tests in the new Wok and Kimchi (as plugin) code I discovered something that can cause some issues for us. Let'me show a possible user case to explain better: The user uses some version of the current Kimchi (previous of 1.5.1 and called for now as, Kimchi_old) totally functional and operating. The user has a few templates created, which are stored in the the objectstore function of Kimchi, and for each template has the storagepool URI allocated for that template. The storagepool information has the following properties in Kimchi-old: "storagepool": { "description": "Location of the storage pool", "type": "string", "pattern": "^/storagepools/[^/]+/?$", "error": "KCHTMPL0015E" } However, the new Wok+Kimchi_plugin structure changed the URI for all virtualization features and the pattern now is: "^/plugins/kimchi/storagepools/[^/]+/?$" There's an issue when getting the information of all Kimchi-old templates from the objectstore, the URI information continues with the older pattern and some checks in the code will fail. My suggestion is to create an info-update process when the user upgrade from Kimchi-old to Wok+Kimchi_plugin, modifying the storagepool information of all templates that not follow the new pattern in the database already created. I'd like to know what do you guys think about and ask for more suggestions also. Best regards, -- Paulo Ricardo Paz Vital <pvital(a)linux.vnet.ibm.com> IBM Linux Technology Center

3 5

[PATCH] Avoid show user/password in url browser
by Rodrigo Trujillo 05 Oct '15

05 Oct '15

There is a remote, but real, possibility that kimchi.min.js breaks and is not loaded for some reason in Kimchi login page. If this happen, the form submmit action is not going to be binded to a javascript function that calls a AJAX POST request. Then the browser is going to submmit the form in the default way: using a GET request. GET requests add form data in the URL, so user will be able to see the user and password in the URL field and in the log: "GET /login.html?username=321&password=234 HTTP/1.0" 200 2936 "https://localhost:8001/login.html" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:39.0) Gecko/20100101 Firefox/39.0" This patch fixes this problem adding 'method="post"' in the login html form. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- ui/pages/login.html.tmpl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/pages/login.html.tmpl b/ui/pages/login.html.tmpl index e2f6855..55848b4 100644 --- a/ui/pages/login.html.tmpl +++ b/ui/pages/login.html.tmpl @@ -79,7 +79,7 @@ <div id="messUserPass" class="err-mess" style="display: none;">$_("The username or password you entered is incorrect. Please try again.")</div> <div id="messSession" class="err-mess" style="display: none;">$_("Session timeout, please re-login.")</div> </div> - <form id="form-login" class="login-panel"> + <form id="form-login" class="login-panel" method="post"> <div class="row"> <input type="text" id="username" name="username" required="required" placeholder="$_("User Name")" autofocus/> <div id="username-msg" class="msg-required"></div> -- 2.1.0

6 7

[PATCH 01/15] V2 Ginger Base : Taking off the host tab functionality from kimchi
by chandra＠linux.vnet.ibm.com 05 Oct '15

05 Oct '15

From: Chandra Shekhar Reddy Potula <chandra(a)linux.vnet.ibm.com> --- plugins/kimchi/API.json | 134 ------------ plugins/kimchi/Makefile.am | 3 +- plugins/kimchi/config.py.in | 4 - plugins/kimchi/control/host.py | 96 --------- plugins/kimchi/docs/README.md | 6 +- plugins/kimchi/i18n.py | 47 +---- plugins/kimchi/kimchi.conf | 7 - plugins/kimchi/mockmodel.py | 18 -- plugins/kimchi/model/config.py | 21 -- plugins/kimchi/model/host.py | 322 +---------------------------- plugins/kimchi/root.py | 1 - plugins/kimchi/tests/test_authorization.py | 6 - plugins/kimchi/tests/test_config.py.in | 7 - plugins/kimchi/tests/test_host.py | 96 +-------- plugins/kimchi/tests/test_model.py | 237 --------------------- plugins/kimchi/tests/test_rest.py | 54 ----- plugins/kimchi/ui/config/tab-ext.xml | 7 - 17 files changed, 7 insertions(+), 1059 deletions(-) diff --git a/plugins/kimchi/API.json b/plugins/kimchi/API.json index f1f58ff..e75b35f 100644 --- a/plugins/kimchi/API.json +++ b/plugins/kimchi/API.json @@ -59,30 +59,6 @@ } }, "properties": { - "debugreports_create": { - "type": "object", - "error": "KCHDR0006E", - "properties": { - "name": { - "description": "The name for the debug report file.", - "type": "string", - "pattern": "^[_A-Za-z0-9-]*$", - "error": "KCHDR0007E" - } - } - }, - "debugreport_update": { - "type": "object", - "properties": { - "name": { - "description": "New name of debug report", - "type": "string", - "pattern": "^[_A-Za-z0-9-]*$", - "error": "KCHDR0007E" - } - }, - "additionalProperties": false - }, "storagepools_create": { "type": "object", "error": "KCHPOOL0026E", @@ -684,116 +660,6 @@ "additionalProperties": false, "error": "KCHAPI0001E" }, - "repositories_create": { - "type": "object", - "properties": { - "repo_id": { - "description": "Repository ID used for YUM repository.", - "type": "string", - "error": "KCHREPOS0001E" - }, - "baseurl": { - "description": "URL to the directory where the repodata directory of a repository is located. Can be an http://, ftp:// or file:// URL.", - "type": "string", - "error": "KCHREPOS0002E" - }, - "config": { - "description": "Dictionary containing repository configuration", - "type": "object", - "error": "KCHREPOS0003E", - "properties": { - "dist": { - "description": "Distribution to DEB repository", - "type": "string", - "error": "KCHREPOS0004E" - }, - "comps": { - "description": "List of components to DEB repository", - "type": "array", - "error": "KCHREPOS0005E", - "uniqueItems": true, - "items": { - "description": "Component name", - "type": "string", - "error": "KCHREPOS0006E" - } - }, - "repo_name": { - "description": "YUM repository name", - "type": "string", - "error": "KCHREPOS0023E" - }, - "mirrorlist": { - "description": "URL to a file containing a list of baseurls", - "type": "string", - "error": "KCHREPOS0007E" - }, - "metalink": { - "description": "URL to a metalink file for the repomd.xml", - "type": "string", - "error": "KCHREPOS0029E" - } - } - } - }, - "additionalProperties": false, - "error": "KCHAPI0001E" - }, - "repository_update": { - "type": "object", - "properties": { - "baseurl": { - "description": "URL to the directory where the repodata directory of a repository is located. Can be an http://, ftp:// or file:// URL.", - "type": "string", - "error": "KCHREPOS0002E" - }, - "config": { - "description": "Dictionary containing repository configuration", - "type": "object", - "error": "KCHREPOS0003E", - "properties": { - "dist": { - "description": "Distribution to DEB repository", - "type": "string", - "error": "KCHREPOS0004E" - }, - "comps": { - "description": "List of components to DEB repository", - "type": "array", - "error": "KCHREPOS0005E", - "uniqueItems": true, - "items": { - "description": "Component name", - "type": "string", - "error": "KCHREPOS0006E" - } - }, - "repo_name": { - "description": "Human-readable string describing the YUM repository.", - "type": "string", - "error": "KCHREPOS0008E" - }, - "mirrorlist": { - "description": "URL to a file containing a list of baseurls for YUM repository", - "type": "string", - "error": "KCHREPOS0007E" - }, - "gpgcheck": { - "description": "Indicates if a GPG signature check on the packages gotten from repository should be performed.", - "type": "boolean", - "error": "KCHREPOS0009E" - }, - "gpgkey": { - "description": "URL pointing to the ASCII-armored GPG key file for the repository.", - "type": "string", - "error": "KCHREPOS0010E" - } - } - } - }, - "additionalProperties": false, - "error": "KCHAPI0001E" - }, "devices_get_list": { "type": "object", "properties": { diff --git a/plugins/kimchi/Makefile.am b/plugins/kimchi/Makefile.am index 49c835e..c30e7b9 100644 --- a/plugins/kimchi/Makefile.am +++ b/plugins/kimchi/Makefile.am @@ -99,7 +99,6 @@ config.py: config.py.in Makefile install-deb: install cp -R $(top_srcdir)/contrib/DEBIAN $(DESTDIR)/ mkdir -p $(DESTDIR)/var/lib/kimchi/vnc-tokens - mkdir -p $(DESTDIR)/var/lib/kimchi/debugreports mkdir -p $(DESTDIR)/var/lib/kimchi/screenshots mkdir -p $(DESTDIR)/var/lib/kimchi/isos @@ -140,7 +139,7 @@ install-data-local: $(MKDIR_P) $(DESTDIR)$(kimchidir) $(INSTALL_DATA) API.json $(DESTDIR)$(kimchidir)/API.json mkdir -p $(DESTDIR)/var/lib/kimchi/vnc-tokens - mkdir -p $(DESTDIR)/var/lib/kimchi/{debugreports,isos,screenshots} + mkdir -p $(DESTDIR)/var/lib/kimchi/{isos,screenshots} uninstall-local: $(RM) $(DESTDIR)$(kimchidir)/API.json diff --git a/plugins/kimchi/config.py.in b/plugins/kimchi/config.py.in index 6ae0ccd..43e7811 100644 --- a/plugins/kimchi/config.py.in +++ b/plugins/kimchi/config.py.in @@ -38,10 +38,6 @@ def get_distros_store(): return os.path.join(PluginPaths('kimchi').conf_dir, 'distros.d') -def get_debugreports_path(): - return os.path.join(PluginPaths('kimchi').state_dir, 'debugreports') - - def get_screenshot_path(): return os.path.join(PluginPaths('kimchi').state_dir, 'screenshots') diff --git a/plugins/kimchi/control/host.py b/plugins/kimchi/control/host.py index 0a40f1b..c71c47b 100644 --- a/plugins/kimchi/control/host.py +++ b/plugins/kimchi/control/host.py @@ -19,7 +19,6 @@ from wok.control.base import Collection, Resource, SimpleCollection from wok.control.utils import UrlSubNode -from wok.exception import NotFoundError from cpuinfo import CPUInfo @@ -31,14 +30,7 @@ class Host(Resource): self.role_key = 'host' self.admin_methods = ['GET', 'POST'] self.uri_fmt = '/host/%s' - self.reboot = self.generate_action_handler('reboot') - self.shutdown = self.generate_action_handler('shutdown') - self.stats = HostStats(self.model) - self.partitions = Partitions(self.model) self.devices = Devices(self.model) - self.packagesupdate = PackagesUpdate(self.model) - self.repositories = Repositories(self.model) - self.swupdate = self.generate_action_handler_task('swupdate') self.cpuinfo = CPUInfo(self.model) @property @@ -46,54 +38,6 @@ class Host(Resource): return self.info -class HostStats(Resource): - def __init__(self, model, id=None): - super(HostStats, self).__init__(model, id) - self.role_key = 'host' - self.admin_methods = ['GET'] - self.history = HostStatsHistory(self.model) - - @property - def data(self): - return self.info - - -class HostStatsHistory(Resource): - @property - def data(self): - return self.info - - -class Partitions(Collection): - def __init__(self, model): - super(Partitions, self).__init__(model) - self.role_key = 'storage' - self.admin_methods = ['GET'] - self.resource = Partition - - # Defining get_resources in order to return list of partitions in UI - # sorted by their path - def _get_resources(self, flag_filter): - res_list = super(Partitions, self)._get_resources(flag_filter) - res_list = filter(lambda x: x.info['available'], res_list) - res_list.sort(key=lambda x: x.info['path']) - return res_list - - -class Partition(Resource): - def __init__(self, model, id): - self.role_key = 'storage' - self.admin_methods = ['GET'] - super(Partition, self).__init__(model, id) - - @property - def data(self): - if not self.info['available']: - raise NotFoundError("KCHPART0001E", {'name': self.info['name']}) - - return self.info - - class Devices(Collection): def __init__(self, model): super(Devices, self).__init__(model) @@ -115,43 +59,3 @@ class Device(Resource): def data(self): return self.info - -class PackagesUpdate(Collection): - def __init__(self, model): - super(PackagesUpdate, self).__init__(model) - self.role_key = 'host' - self.admin_methods = ['GET'] - self.resource = PackageUpdate - - -class PackageUpdate(Resource): - def __init__(self, model, id=None): - super(PackageUpdate, self).__init__(model, id) - self.role_key = 'host' - self.admin_methods = ['GET'] - - @property - def data(self): - return self.info - - -class Repositories(Collection): - def __init__(self, model): - super(Repositories, self).__init__(model) - self.role_key = 'host' - self.admin_methods = ['GET', 'POST'] - self.resource = Repository - - -class Repository(Resource): - def __init__(self, model, id): - super(Repository, self).__init__(model, id) - self.role_key = 'host' - self.admin_methods = ['GET', 'PUT', 'POST', 'DELETE'] - self.uri_fmt = "/host/repositories/%s" - self.enable = self.generate_action_handler('enable') - self.disable = self.generate_action_handler('disable') - - @property - def data(self): - return self.info diff --git a/plugins/kimchi/docs/README.md b/plugins/kimchi/docs/README.md index f400333..ee6bfed 100644 --- a/plugins/kimchi/docs/README.md +++ b/plugins/kimchi/docs/README.md @@ -196,13 +196,13 @@ Usage Connect your browser to https://localhost:8001. You should see a screen like: -![Wok Login Screen](/docs/kimchi-login.png) +![Wok Login Screen](docs/kimchi-login.png) Wok uses PAM to authenticate users so you can log in with the same username and password that you would use to log in to the machine itself. Once logged in you will see a screen like: -![Kimchi Guest View](/docs/kimchi-guest.png) +![Kimchi Guest View](docs/kimchi-guest.png) This shows you the list of running guests including a live screenshot of the guest session. You can use the action buttons to shutdown the guests @@ -216,7 +216,7 @@ top navigation bar. The template screen looks like: -![Kimchi Template View](/docs/kimchi-templates.png) +![Kimchi Template View](docs/kimchi-templates.png) From this view, you can change the parameters of a template or create a new template using the "+" button in the upper right corner. diff --git a/plugins/kimchi/i18n.py b/plugins/kimchi/i18n.py index 253f00d..a2ea858 100644 --- a/plugins/kimchi/i18n.py +++ b/plugins/kimchi/i18n.py @@ -244,15 +244,6 @@ messages = { "KCHNET0021E": _("Failed to activate interface %(iface)s. Please check the physical link status."), "KCHNET0022E": _("Failed to start network %(name)s. Details: %(err)s"), - "KCHDR0001E": _("Debug report %(name)s does not exist"), - "KCHDR0002E": _("Debug report tool not found in system"), - "KCHDR0003E": _("Unable to create debug report %(name)s. Details: %(err)s."), - "KCHDR0004E": _("Can not find any debug report with the given name %(name)s"), - "KCHDR0005E": _("Unable to generate debug report %(name)s. Details: %(err)s"), - "KCHDR0006E": _("You should give a name for the debug report file."), - "KCHDR0007E": _("Debug report name must be a string. Only letters, digits, underscore ('_') and hyphen ('-') are allowed."), - "KCHDR0008E": _("The debug report with specified name \"%(name)s\" already exists. Please use another one."), - "KCHSR0001E": _("Storage server %(server)s was not used by Kimchi"), "KCHDISTRO0001E": _("Distro '%(name)s' does not exist"), @@ -264,11 +255,6 @@ messages = { "KCHHOST0003E": _("Node device '%(name)s' not found"), "KCHHOST0004E": _("Conflicting flag filters specified."), - "KCHPKGUPD0001E": _("No packages marked for update"), - "KCHPKGUPD0002E": _("Package %(name)s is not marked to be updated."), - "KCHPKGUPD0003E": _("Error while getting packages marked to be updated. Details: %(err)s"), - "KCHPKGUPD0004E": _("There is no compatible package manager for this system."), - "KCHUTILS0003E": _("Unable to choose a virtual machine name"), "KCHVMSTOR0002E": _("Invalid storage type. Types supported: 'cdrom', 'disk'"), @@ -286,38 +272,7 @@ messages = { "KCHVMSTOR0016E": _("Volume already in use by other virtual machine."), "KCHVMSTOR0017E": _("Only one of path or pool/volume can be specified to add a new virtual machine disk"), "KCHVMSTOR0018E": _("Volume chosen with format %(format)s does not fit in the storage type %(type)s"), - - "KCHREPOS0001E": _("YUM Repository ID must be one word only string."), - "KCHREPOS0002E": _("Repository URL must be an http://, ftp:// or file:// URL."), - "KCHREPOS0003E": _("Repository configuration is a dictionary with specific values according to repository type."), - "KCHREPOS0004E": _("Distribution to DEB repository must be a string"), - "KCHREPOS0005E": _("Components to DEB repository must be listed in a array"), - "KCHREPOS0006E": _("Components to DEB repository must be a string"), - "KCHREPOS0007E": _("Mirror list to repository must be a string"), - "KCHREPOS0008E": _("YUM Repository name must be string."), - "KCHREPOS0009E": _("GPG check must be a boolean value."), - "KCHREPOS0010E": _("GPG key must be a URL pointing to the ASCII-armored file."), - "KCHREPOS0011E": _("Could not update repository %(repo_id)s."), - "KCHREPOS0012E": _("Repository %(repo_id)s does not exist."), - "KCHREPOS0013E": _("Specify repository base URL, mirror list or metalink in order to create or update a YUM repository."), - "KCHREPOS0014E": _("Repository management tool was not recognized for your system."), - "KCHREPOS0015E": _("Repository %(repo_id)s is already enabled."), - "KCHREPOS0016E": _("Repository %(repo_id)s is already disabled."), - "KCHREPOS0017E": _("Could not remove repository %(repo_id)s."), - "KCHREPOS0018E": _("Could not write repository configuration file %(repo_file)s"), - "KCHREPOS0019E": _("Specify repository distribution in order to create a DEB repository."), - "KCHREPOS0020E": _("Could not enable repository %(repo_id)s."), - "KCHREPOS0021E": _("Could not disable repository %(repo_id)s."), - "KCHREPOS0022E": _("YUM Repository ID already exists"), - "KCHREPOS0023E": _("YUM Repository name must be a string"), - "KCHREPOS0024E": _("Unable to list repositories. Details: '%(err)s'"), - "KCHREPOS0025E": _("Unable to retrieve repository information. Details: '%(err)s'"), - "KCHREPOS0026E": _("Unable to add repository. Details: '%(err)s'"), - "KCHREPOS0027E": _("Unable to remove repository. Details: '%(err)s'"), - "KCHREPOS0028E": _("Configuration items: '%(items)s' are not supported by repository manager"), - "KCHREPOS0029E": _("Repository metalink must be an http://, ftp:// or file:// URL."), - "KCHREPOS0030E": _("Cannot specify mirrorlist and metalink at the same time."), - + "KCHSNAP0001E": _("Virtual machine '%(vm)s' must be stopped before creating a snapshot of it."), "KCHSNAP0002E": _("Unable to create snapshot '%(name)s' on virtual machine '%(vm)s'. Details: %(err)s"), "KCHSNAP0003E": _("Snapshot '%(name)s' does not exist on virtual machine '%(vm)s'."), diff --git a/plugins/kimchi/kimchi.conf b/plugins/kimchi/kimchi.conf index 1bf78e4..ebf346f 100644 --- a/plugins/kimchi/kimchi.conf +++ b/plugins/kimchi/kimchi.conf @@ -23,13 +23,6 @@ tools.staticdir.on = True tools.staticdir.dir = wok.config.PluginPaths('kimchi').state_dir + '/screenshots' tools.nocache.on = False -[/data/debugreports] -tools.staticdir.on = True -tools.staticdir.dir = wok.config.PluginPaths('kimchi').state_dir + '/debugreports' -tools.nocache.on = False -tools.wokauth.on = True -tools.staticdir.content_types = {'xz': 'application/x-xz'} - [/help] tools.staticdir.on = True tools.staticdir.dir = wok.config.PluginPaths('kimchi').ui_dir + '/pages/help' diff --git a/plugins/kimchi/mockmodel.py b/plugins/kimchi/mockmodel.py index b269c26..4550c26 100644 --- a/plugins/kimchi/mockmodel.py +++ b/plugins/kimchi/mockmodel.py @@ -34,7 +34,6 @@ import config import imageinfo import osinfo from model import cpuinfo -from model.debugreports import DebugReportsModel from model.host import DeviceModel from model.libvirtstoragepool import IscsiPoolDef, NetfsPoolDef from model.libvirtstoragepool import StoragePoolDef @@ -114,7 +113,6 @@ class MockModel(Model): StoragePoolModel._update_lvm_disks = self._update_lvm_disks StorageVolumesModel.get_list = self._mock_storagevolumes_get_list StorageVolumeModel.doUpload = self._mock_storagevolume_doUpload - DebugReportsModel._gen_debugreport_file = self._gen_debugreport_file LibvirtVMTemplate._get_volume_path = self._get_volume_path VMTemplate.get_iso_info = self._probe_image imageinfo.probe_image = self._probe_image @@ -246,22 +244,6 @@ class MockModel(Model): return MockModel._libvirt_get_vol_path(pool, vol) - def _gen_debugreport_file(self, name): - return add_task('/plugins/kimchi/debugreports/%s' % name, - self._create_log, self.objstore, name) - - def _create_log(self, cb, name): - path = config.get_debugreports_path() - tmpf = os.path.join(path, name + '.tmp') - realf = os.path.join(path, name + '.txt') - length = random.randint(1000, 10000) - with open(tmpf, 'w') as fd: - while length: - fd.write('I am logged') - length = length - 1 - os.rename(tmpf, realf) - cb("OK", True) - def _update_lvm_disks(self, pool_name, disks): conn = self.conn.get() pool = conn.storagePoolLookupByName(pool_name.encode('utf-8')) diff --git a/plugins/kimchi/model/config.py b/plugins/kimchi/model/config.py index 464ffae..702943b 100644 --- a/plugins/kimchi/model/config.py +++ b/plugins/kimchi/model/config.py @@ -28,10 +28,7 @@ from wok.utils import check_url_path, run_command, wok_log from ..config import find_qemu_binary from ..distroloader import DistroLoader -from ..repositories import Repositories from ..screenshot import VMScreenshot -from ..swupdate import SoftwareUpdate -from debugreports import DebugReportsModel from featuretests import FeatureTests, FEATURETEST_POOL_NAME from featuretests import FEATURETEST_VM_NAME @@ -116,28 +113,10 @@ class CapabilitiesModel(object): return False def lookup(self, *ident): - report_tool = DebugReportsModel.get_system_report_tool() - try: - SoftwareUpdate() - except Exception: - update_tool = False - else: - update_tool = True - - try: - repo = Repositories() - except Exception: - repo_mngt_tool = None - else: - repo_mngt_tool = repo._pkg_mnger.TYPE - return {'libvirt_stream_protocols': self.libvirt_stream_protocols, 'qemu_spice': self._qemu_support_spice(), 'qemu_stream': self.qemu_stream, 'screenshot': VMScreenshot.get_stream_test_result(), - 'system_report_tool': bool(report_tool), - 'update_tool': update_tool, - 'repo_mngt_tool': repo_mngt_tool, 'federation': kconfig.get("server", "federation"), 'auth': kconfig.get("authentication", "method"), 'kernel_vfio': self.kernel_vfio, diff --git a/plugins/kimchi/model/host.py b/plugins/kimchi/model/host.py index 75d4de0..c8f5d24 100644 --- a/plugins/kimchi/model/host.py +++ b/plugins/kimchi/model/host.py @@ -18,27 +18,15 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import libvirt -import os -import platform import psutil -import time -from cherrypy.process.plugins import BackgroundTask -from collections import defaultdict -from wok.basemodel import Singleton from wok.exception import InvalidOperation, InvalidParameter from wok.exception import NotFoundError, OperationFailed -from wok.utils import add_task, wok_log from wok.xmlutils.utils import xpath_get_text -from wok.model.tasks import TaskModel import hostdev -from .. import disks -from .. import netinfo -from ..repositories import Repositories -from ..swupdate import SoftwareUpdate from config import CapabilitiesModel -from vms import DOM_STATE_MAP +from wok.model.tasks import TaskModel HOST_STATS_INTERVAL = 1 @@ -49,57 +37,6 @@ class HostModel(object): self.conn = kargs['conn'] self.objstore = kargs['objstore'] self.task = TaskModel(**kargs) - self.host_info = self._get_host_info() - - def _get_ppc_cpu_info(self): - res = {} - with open('/proc/cpuinfo') as f: - for line in f.xreadlines(): - # Parse CPU, CPU's revision and CPU's clock information - for key in ['cpu', 'revision', 'clock']: - if key in line: - info = line.split(':')[1].strip() - if key == 'clock': - value = float(info.split('MHz')[0].strip()) / 1000 - else: - value = info.split('(')[0].strip() - res[key] = value - - # Power machines show, for each cpu/core, a block with - # all cpu information. Here we control the scan of the - # necessary information (1st block provides - # everything), skipping the function when find all - # information. - if len(res.keys()) == 3: - return "%(cpu)s (%(revision)s) @ %(clock)s GHz\ - " % res - - return "" - - def _get_host_info(self): - res = {} - if platform.machine().startswith('ppc'): - res['cpu_model'] = self._get_ppc_cpu_info() - else: - with open('/proc/cpuinfo') as f: - for line in f.xreadlines(): - if "model name" in line: - res['cpu_model'] = line.split(':')[1].strip() - break - - res['cpus'] = 0 - res['memory'] = 0L - - # Include IBM PowerKVM name to supported distro names - _sup_distros = platform._supported_dists + ('ibm_powerkvm',) - # 'fedora' '17' 'Beefy Miracle' - distro, version, codename = platform.linux_distribution( - supported_dists=_sup_distros) - res['os_distro'] = distro - res['os_version'] = version - res['os_codename'] = unicode(codename, "utf-8") - - return res def lookup(self, *name): cpus = psutil.NUM_CPUS @@ -121,179 +58,6 @@ class HostModel(object): self.host_info['memory'] = psutil.phymem_usage().total return self.host_info - def swupdate(self, *name): - try: - swupdate = SoftwareUpdate() - except: - raise OperationFailed('KCHPKGUPD0004E') - - pkgs = swupdate.getNumOfUpdates() - if pkgs == 0: - raise OperationFailed('KCHPKGUPD0001E') - - wok_log.debug('Host is going to be updated.') - taskid = add_task('/plugins/kimchi/host/swupdate', swupdate.doUpdate, - self.objstore, None) - return self.task.lookup(taskid) - - def shutdown(self, args=None): - # Check for running vms before shutdown - running_vms = self._get_vms_list_by_state('running') - if len(running_vms) > 0: - raise OperationFailed("KCHHOST0001E") - - wok_log.info('Host is going to shutdown.') - os.system('shutdown -h now') - - def reboot(self, args=None): - # Find running VMs - running_vms = self._get_vms_list_by_state('running') - if len(running_vms) > 0: - raise OperationFailed("KCHHOST0002E") - - wok_log.info('Host is going to reboot.') - os.system('reboot') - - def _get_vms_list_by_state(self, state): - conn = self.conn.get() - return [dom.name().decode('utf-8') - for dom in conn.listAllDomains(0) - if (DOM_STATE_MAP[dom.info()[0]]) == state] - - -class HostStatsModel(object): - __metaclass__ = Singleton - - def __init__(self, **kargs): - self.host_stats = defaultdict(list) - self.host_stats_thread = BackgroundTask(HOST_STATS_INTERVAL, - self._update_host_stats) - self.host_stats_thread.start() - - def lookup(self, *name): - return {'cpu_utilization': self.host_stats['cpu_utilization'][-1], - 'memory': self.host_stats['memory'][-1], - 'disk_read_rate': self.host_stats['disk_read_rate'][-1], - 'disk_write_rate': self.host_stats['disk_write_rate'][-1], - 'net_recv_rate': self.host_stats['net_recv_rate'][-1], - 'net_sent_rate': self.host_stats['net_sent_rate'][-1]} - - def _update_host_stats(self): - preTimeStamp = self.host_stats['timestamp'] - timestamp = time.time() - # FIXME when we upgrade psutil, we can get uptime by psutil.uptime - # we get uptime by float(open("/proc/uptime").readline().split()[0]) - # and calculate the first io_rate after the OS started. - with open("/proc/uptime") as time_f: - seconds = (timestamp - preTimeStamp if preTimeStamp else - float(time_f.readline().split()[0])) - - self.host_stats['timestamp'] = timestamp - self._get_host_disk_io_rate(seconds) - self._get_host_network_io_rate(seconds) - - self._get_percentage_host_cpu_usage() - self._get_host_memory_stats() - - # store only 60 stats (1 min) - for key, value in self.host_stats.iteritems(): - if isinstance(value, list): - if len(value) == 60: - self.host_stats[key] = value[10:] - - def _get_percentage_host_cpu_usage(self): - # This is cpu usage producer. This producer will calculate the usage - # at an interval of HOST_STATS_INTERVAL. - # The psutil.cpu_percent works as non blocking. - # psutil.cpu_percent maintains a cpu time sample. - # It will update the cpu time sample when it is called. - # So only this producer can call psutil.cpu_percent in kimchi. - self.host_stats['cpu_utilization'].append(psutil.cpu_percent(None)) - - def _get_host_memory_stats(self): - virt_mem = psutil.virtual_memory() - # available: - # the actual amount of available memory that can be given - # instantly to processes that request more memory in bytes; this - # is calculated by summing different memory values depending on - # the platform (e.g. free + buffers + cached on Linux) - memory_stats = {'total': virt_mem.total, - 'free': virt_mem.free, - 'cached': virt_mem.cached, - 'buffers': virt_mem.buffers, - 'avail': virt_mem.available} - self.host_stats['memory'].append(memory_stats) - - def _get_host_disk_io_rate(self, seconds): - disk_read_bytes = self.host_stats['disk_read_bytes'] - disk_write_bytes = self.host_stats['disk_write_bytes'] - prev_read_bytes = disk_read_bytes[-1] if disk_read_bytes else 0 - prev_write_bytes = disk_write_bytes[-1] if disk_write_bytes else 0 - - disk_io = psutil.disk_io_counters(False) - read_bytes = disk_io.read_bytes - write_bytes = disk_io.write_bytes - - rd_rate = int(float(read_bytes - prev_read_bytes) / seconds + 0.5) - wr_rate = int(float(write_bytes - prev_write_bytes) / seconds + 0.5) - - self.host_stats['disk_read_rate'].append(rd_rate) - self.host_stats['disk_write_rate'].append(wr_rate) - self.host_stats['disk_read_bytes'].append(read_bytes) - self.host_stats['disk_write_bytes'].append(write_bytes) - - def _get_host_network_io_rate(self, seconds): - net_recv_bytes = self.host_stats['net_recv_bytes'] - net_sent_bytes = self.host_stats['net_sent_bytes'] - prev_recv_bytes = net_recv_bytes[-1] if net_recv_bytes else 0 - prev_sent_bytes = net_sent_bytes[-1] if net_sent_bytes else 0 - - net_ios = psutil.network_io_counters(True) - recv_bytes = 0 - sent_bytes = 0 - for key in set(netinfo.nics() + - netinfo.wlans()) & set(net_ios.iterkeys()): - recv_bytes = recv_bytes + net_ios[key].bytes_recv - sent_bytes = sent_bytes + net_ios[key].bytes_sent - - rx_rate = int(float(recv_bytes - prev_recv_bytes) / seconds + 0.5) - tx_rate = int(float(sent_bytes - prev_sent_bytes) / seconds + 0.5) - - self.host_stats['net_recv_rate'].append(rx_rate) - self.host_stats['net_sent_rate'].append(tx_rate) - self.host_stats['net_recv_bytes'].append(recv_bytes) - self.host_stats['net_sent_bytes'].append(sent_bytes) - - -class HostStatsHistoryModel(object): - def __init__(self, **kargs): - self.history = HostStatsModel(**kargs) - - def lookup(self, *name): - return {'cpu_utilization': self.history.host_stats['cpu_utilization'], - 'memory': self.history.host_stats['memory'], - 'disk_read_rate': self.history.host_stats['disk_read_rate'], - 'disk_write_rate': self.history.host_stats['disk_write_rate'], - 'net_recv_rate': self.history.host_stats['net_recv_rate'], - 'net_sent_rate': self.history.host_stats['net_sent_rate']} - - -class PartitionsModel(object): - def __init__(self, **kargs): - pass - - def get_list(self): - result = disks.get_partitions_names() - return result - - -class PartitionModel(object): - def __init__(self, **kargs): - pass - - def lookup(self, name): - return disks.get_partition_details(name) - class DevicesModel(object): def __init__(self, **kargs): @@ -390,87 +154,3 @@ class DeviceModel(object): raise NotFoundError('KCHHOST0003E', {'name': nodedev_name}) return hostdev.get_dev_info(dev) - -class PackagesUpdateModel(object): - def __init__(self, **kargs): - try: - self.host_swupdate = SoftwareUpdate() - except: - self.host_swupdate = None - - def get_list(self): - if self.host_swupdate is None: - raise OperationFailed('KCHPKGUPD0004E') - - return self.host_swupdate.getUpdates() - - -class PackageUpdateModel(object): - def __init__(self, **kargs): - pass - - def lookup(self, name): - try: - swupdate = SoftwareUpdate() - except Exception: - raise OperationFailed('KCHPKGUPD0004E') - - return swupdate.getUpdate(name) - - -class RepositoriesModel(object): - def __init__(self, **kargs): - try: - self.host_repositories = Repositories() - except: - self.host_repositories = None - - def get_list(self): - if self.host_repositories is None: - raise InvalidOperation('KCHREPOS0014E') - - return sorted(self.host_repositories.getRepositories()) - - def create(self, params): - if self.host_repositories is None: - raise InvalidOperation('KCHREPOS0014E') - - return self.host_repositories.addRepository(params) - - -class RepositoryModel(object): - def __init__(self, **kargs): - try: - self._repositories = Repositories() - except: - self._repositories = None - - def lookup(self, repo_id): - if self._repositories is None: - raise InvalidOperation('KCHREPOS0014E') - - return self._repositories.getRepository(repo_id) - - def enable(self, repo_id): - if self._repositories is None: - raise InvalidOperation('KCHREPOS0014E') - - return self._repositories.enableRepository(repo_id) - - def disable(self, repo_id): - if self._repositories is None: - raise InvalidOperation('KCHREPOS0014E') - - return self._repositories.disableRepository(repo_id) - - def update(self, repo_id, params): - if self._repositories is None: - raise InvalidOperation('KCHREPOS0014E') - - return self._repositories.updateRepository(repo_id, params) - - def delete(self, repo_id): - if self._repositories is None: - raise InvalidOperation('KCHREPOS0014E') - - return self._repositories.removeRepository(repo_id) diff --git a/plugins/kimchi/root.py b/plugins/kimchi/root.py index 1e2bfc7..2c8ac5b 100644 --- a/plugins/kimchi/root.py +++ b/plugins/kimchi/root.py @@ -58,7 +58,6 @@ class KimchiRoot(WokRoot): make_dirs = [ os.path.abspath(config.get_distros_store()), - os.path.abspath(config.get_debugreports_path()), os.path.abspath(config.get_screenshot_path()) ] for directory in make_dirs: diff --git a/plugins/kimchi/tests/test_authorization.py b/plugins/kimchi/tests/test_authorization.py index 87d68ab..fda86a3 100644 --- a/plugins/kimchi/tests/test_authorization.py +++ b/plugins/kimchi/tests/test_authorization.py @@ -77,12 +77,6 @@ class AuthorizationTests(unittest.TestCase): resp = self.request('/plugins/kimchi/host/shutdown', '{}', 'POST') self.assertEquals(403, resp.status) - # Non-root users can not get or debug reports - resp = self.request('/plugins/kimchi/debugreports', '{}', 'GET') - self.assertEquals(403, resp.status) - resp = self.request('/plugins/kimchi/debugreports', '{}', 'POST') - self.assertEquals(403, resp.status) - # Non-root users can not create or delete network (only get) resp = self.request('/plugins/kimchi/networks', '{}', 'GET') self.assertEquals(200, resp.status) diff --git a/plugins/kimchi/tests/test_config.py.in b/plugins/kimchi/tests/test_config.py.in index 4604eb1..b74ba7c 100644 --- a/plugins/kimchi/tests/test_config.py.in +++ b/plugins/kimchi/tests/test_config.py.in @@ -248,13 +248,6 @@ class ConfigTests(unittest.TestCase): 'tools.staticdir.dir': get_screenshot_path(), 'tools.nocache.on': False }, - '/data/debugreports': { - 'tools.staticdir.on': True, - 'tools.staticdir.dir': get_debugreports_path(), - 'tools.nocache.on': False, - 'tools.wokauth.on': True, - 'tools.staticdir.content_types': {'xz': 'application/x-xz'} - }, '/help': { 'tools.staticdir.on': True, 'tools.staticdir.dir': '%s/ui/pages/help' % pluginPrefix, diff --git a/plugins/kimchi/tests/test_host.py b/plugins/kimchi/tests/test_host.py index 6cd0833..530262c 100644 --- a/plugins/kimchi/tests/test_host.py +++ b/plugins/kimchi/tests/test_host.py @@ -20,8 +20,6 @@ import json import os -import platform -import psutil import tempfile import time import unittest @@ -29,7 +27,7 @@ from functools import partial from wok.plugins.kimchi.mockmodel import MockModel -from utils import get_free_port, patch_auth, request, run_server, wait_task +from utils import get_free_port, patch_auth, request, run_server test_server = None @@ -62,98 +60,6 @@ class HostTests(unittest.TestCase): def setUp(self): self.request = partial(request, host, ssl_port) - def test_hostinfo(self): - resp = self.request('/plugins/kimchi/host').read() - info = json.loads(resp) - keys = ['os_distro', 'os_version', 'os_codename', 'cpu_model', - 'memory', 'cpus'] - self.assertEquals(sorted(keys), sorted(info.keys())) - - distro, version, codename = platform.linux_distribution() - self.assertEquals(distro, info['os_distro']) - self.assertEquals(version, info['os_version']) - self.assertEquals(unicode(codename, "utf-8"), info['os_codename']) - self.assertEquals(psutil.TOTAL_PHYMEM, info['memory']) - - def test_hoststats(self): - time.sleep(1) - stats_keys = ['cpu_utilization', 'memory', 'disk_read_rate', - 'disk_write_rate', 'net_recv_rate', 'net_sent_rate'] - resp = self.request('/plugins/kimchi/host/stats').read() - stats = json.loads(resp) - self.assertEquals(sorted(stats_keys), sorted(stats.keys())) - - cpu_utilization = stats['cpu_utilization'] - self.assertIsInstance(cpu_utilization, float) - self.assertGreaterEqual(cpu_utilization, 0.0) - self.assertTrue(cpu_utilization <= 100.0) - - memory_stats = stats['memory'] - self.assertIn('total', memory_stats) - self.assertIn('free', memory_stats) - self.assertIn('cached', memory_stats) - self.assertIn('buffers', memory_stats) - self.assertIn('avail', memory_stats) - - resp = self.request('/plugins/kimchi/host/stats/history').read() - history = json.loads(resp) - self.assertEquals(sorted(stats_keys), sorted(history.keys())) - - def test_host_actions(self): - def _task_lookup(taskid): - return json.loads( - self.request('/plugins/kimchi/tasks/%s' % taskid).read() - ) - - resp = self.request('/plugins/kimchi/host/shutdown', '{}', 'POST') - self.assertEquals(200, resp.status) - resp = self.request('/plugins/kimchi/host/reboot', '{}', 'POST') - self.assertEquals(200, resp.status) - - # Test system update - resp = self.request('/plugins/kimchi/host/packagesupdate', None, 'GET') - pkgs = json.loads(resp.read()) - self.assertEquals(3, len(pkgs)) - - pkg_keys = ['package_name', 'repository', 'arch', 'version'] - for p in pkgs: - name = p['package_name'] - resp = self.request('/plugins/kimchi/host/packagesupdate/' + name, - None, 'GET') - info = json.loads(resp.read()) - self.assertEquals(sorted(pkg_keys), sorted(info.keys())) - - resp = self.request('/plugins/kimchi/host/swupdate', '{}', 'POST') - task = json.loads(resp.read()) - task_params = [u'id', u'message', u'status', u'target_uri'] - self.assertEquals(sorted(task_params), sorted(task.keys())) - - resp = self.request('/tasks/' + task[u'id'], None, - 'GET') - task_info = json.loads(resp.read()) - self.assertEquals(task_info['status'], 'running') - wait_task(_task_lookup, task_info['id']) - resp = self.request('/tasks/' + task[u'id'], None, - 'GET') - task_info = json.loads(resp.read()) - self.assertEquals(task_info['status'], 'finished') - self.assertIn(u'All packages updated', task_info['message']) - pkgs = model.packagesupdate_get_list() - self.assertEquals(0, len(pkgs)) - - def test_host_partitions(self): - resp = self.request('/plugins/kimchi/host/partitions') - self.assertEquals(200, resp.status) - partitions = json.loads(resp.read()) - - keys = ['name', 'path', 'type', 'fstype', 'size', 'mountpoint', - 'available'] - for item in partitions: - resp = self.request('/plugins/kimchi/host/partitions/%s' % - item['name']) - info = json.loads(resp.read()) - self.assertEquals(sorted(info.keys()), sorted(keys)) - def test_host_devices(self): def asset_devices_type(devices, dev_type): for dev in devices: diff --git a/plugins/kimchi/tests/test_model.py b/plugins/kimchi/tests/test_model.py index 7c89048..1a1ca49 100644 --- a/plugins/kimchi/tests/test_model.py +++ b/plugins/kimchi/tests/test_model.py @@ -951,54 +951,6 @@ class ModelTests(unittest.TestCase): self.assertTrue('kimchi-vm' in vms) - @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') - def test_debug_reports(self): - inst = model.Model('test:///default', - objstore_loc=self.tmp_store) - - if not inst.capabilities_lookup()['system_report_tool']: - raise unittest.SkipTest("Without debug report tool") - - try: - timeout = int(os.environ['TEST_REPORT_TIMEOUT']) - except (ValueError, KeyError): - timeout = 120 - - namePrefix = 'unitTestReport' - # sosreport always deletes unsual letters like '-' and '_' in the - # generated report file name. - uuidstr = str(uuid.uuid4()).translate(None, "-_") - reportName = namePrefix + uuidstr - try: - inst.debugreport_delete(namePrefix + '*') - except NotFoundError: - pass - with RollbackContext() as rollback: - report_list = inst.debugreports_get_list() - self.assertFalse(reportName in report_list) - try: - tmp_name = reportName + "_1" - task = inst.debugreports_create({'name': reportName}) - rollback.prependDefer(inst.debugreport_delete, tmp_name) - taskid = task['id'] - inst.task_wait(taskid, timeout) - self.assertEquals('finished', - inst.task_lookup(taskid)['status'], - "It is not necessary an error. " - "You may need to increase the " - "timeout number by " - "TEST_REPORT_TIMEOUT=200 " - "./run_tests.sh test_model") - report_list = inst.debugreports_get_list() - self.assertTrue(reportName in report_list) - name = inst.debugreport_update(reportName, {'name': tmp_name}) - self.assertEquals(name, tmp_name) - report_list = inst.debugreports_get_list() - self.assertTrue(tmp_name in report_list) - except OperationFailed, e: - if 'debugreport tool not found' not in e.message: - raise e - def test_get_distros(self): inst = model.Model('test:///default', objstore_loc=self.tmp_store) @@ -1037,195 +989,6 @@ class ModelTests(unittest.TestCase): volumes = inst.storagevolumes_get_list(args['name']) self.assertEquals(len(volumes), 2) - def test_repository_create(self): - inst = model.Model('test:///default', - objstore_loc=self.tmp_store) - - yum_repos = [{'repo_id': 'fedora-fake', - 'baseurl': 'http://www.fedora.org'}, - {'repo_id': 'fedora-updates-fake', - 'config': - {'mirrorlist': 'http://www.fedoraproject.org'}}] - - deb_repos = [{'baseurl': 'http://archive.ubuntu.com/ubuntu/', - 'config': {'dist': 'quantal'}}, - {'baseurl': 'http://archive.ubuntu.com/ubuntu/', - 'config': {'dist': 'quantal', 'comps': ['main']}}] - - yum_invalid_repos = [] - deb_invalid_repos = [] - - for url in invalid_repository_urls: - wrong_baseurl = {'repo_id': 'wrong-id', 'baseurl': url} - wrong_mirrorlist = {'repo_id': 'wrong-id', - 'baseurl': 'www.example.com', - 'config': {'mirrorlist': url}} - wrong_config_item = { - 'repo_id': 'wrong-id', - 'baseurl': 'www.example.com', - 'config': { - 'gpgkey': 'file:///tmp/KEY-fedora-updates-fake-19'}} - - yum_invalid_repos.append(wrong_baseurl) - yum_invalid_repos.append(wrong_mirrorlist) - yum_invalid_repos.append(wrong_config_item) - - wrong_baseurl['config'] = {'dist': 'tasty'} - wrong_config = {'baseurl': deb_repos[0]['baseurl'], - 'config': { - 'unsupported_item': "a_unsupported_item"}} - deb_invalid_repos.append(wrong_baseurl) - deb_invalid_repos.append(wrong_config) - - repo_type = inst.capabilities_lookup()['repo_mngt_tool'] - if repo_type == 'yum': - test_repos = yum_repos - invalid_repos = yum_invalid_repos - elif repo_type == 'deb': - test_repos = deb_repos - invalid_repos = deb_invalid_repos - else: - # repository management tool was not recognized by Kimchi - # skip test case - return - - # create repositories with invalid data - for repo in invalid_repos: - self.assertRaises(InvalidParameter, inst.repositories_create, repo) - - for repo in test_repos: - system_host_repos = len(inst.repositories_get_list()) - repo_id = inst.repositories_create(repo) - host_repos = inst.repositories_get_list() - self.assertEquals(system_host_repos + 1, len(host_repos)) - - repo_info = inst.repository_lookup(repo_id) - self.assertEquals(repo_id, repo_info['repo_id']) - self.assertEquals(True, repo_info.get('enabled')) - self.assertEquals(repo.get('baseurl', ''), - repo_info.get('baseurl')) - - original_config = repo.get('config', {}) - config_info = repo_info.get('config', {}) - - if repo_type == 'yum': - self.assertEquals(original_config.get('mirrorlist', ''), - config_info.get('mirrorlist', '')) - self.assertEquals(True, config_info['gpgcheck']) - else: - self.assertEquals(original_config['dist'], config_info['dist']) - self.assertEquals(original_config.get('comps', []), - config_info.get('comps', [])) - - inst.repository_delete(repo_id) - self.assertRaises(NotFoundError, inst.repository_lookup, repo_id) - - self.assertRaises(NotFoundError, inst.repository_lookup, 'google') - - def test_repository_update(self): - inst = model.Model('test:///default', - objstore_loc=self.tmp_store) - - yum_repo = {'repo_id': 'fedora-fake', - 'baseurl': 'http://www.fedora.org'} - yum_new_repo = {'baseurl': 'http://www.fedoraproject.org'} - - deb_repo = {'baseurl': 'http://archive.ubuntu.com/ubuntu/', - 'config': {'dist': 'quantal'}} - deb_new_repo = {'baseurl': 'http://br.archive.canonical.com/ubuntu/', - 'config': {'dist': 'utopic'}} - - yum_invalid_repos = [] - deb_invalid_repos = [] - - for url in invalid_repository_urls: - wrong_baseurl = {'baseurl': url} - wrong_mirrorlist = {'baseurl': 'www.example.com', - 'config': {'mirrorlist': url}} - - yum_invalid_repos.append(wrong_baseurl) - yum_invalid_repos.append(wrong_mirrorlist) - - wrong_baseurl['config'] = {'dist': 'tasty'} - deb_invalid_repos.append(wrong_baseurl) - - repo_type = inst.capabilities_lookup()['repo_mngt_tool'] - if repo_type == 'yum': - repo = yum_repo - new_repo = yum_new_repo - invalid_repos = yum_invalid_repos - elif repo_type == 'deb': - repo = deb_repo - new_repo = deb_new_repo - invalid_repos = deb_invalid_repos - else: - # repository management tool was not recognized by Kimchi - # skip test case - return - - system_host_repos = len(inst.repositories_get_list()) - - with RollbackContext() as rollback: - repo_id = inst.repositories_create(repo) - rollback.prependDefer(inst.repository_delete, repo_id) - - host_repos = inst.repositories_get_list() - self.assertEquals(system_host_repos + 1, len(host_repos)) - - # update repositories with invalid data - for tmp_repo in invalid_repos: - self.assertRaises(InvalidParameter, inst.repository_update, - repo_id, tmp_repo) - - new_repo_id = inst.repository_update(repo_id, new_repo) - repo_info = inst.repository_lookup(new_repo_id) - - self.assertEquals(new_repo_id, repo_info['repo_id']) - self.assertEquals(new_repo['baseurl'], repo_info['baseurl']) - self.assertEquals(True, repo_info['enabled']) - inst.repository_update(new_repo_id, repo) - - def test_repository_disable_enable(self): - inst = model.Model('test:///default', - objstore_loc=self.tmp_store) - - yum_repo = {'repo_id': 'fedora-fake', - 'baseurl': 'http://www.fedora.org'} - deb_repo = {'baseurl': 'http://archive.ubuntu.com/ubuntu/', - 'config': {'dist': 'quantal'}} - - repo_type = inst.capabilities_lookup()['repo_mngt_tool'] - if repo_type == 'yum': - repo = yum_repo - elif repo_type == 'deb': - repo = deb_repo - else: - # repository management tool was not recognized by Kimchi - # skip test case - return - - system_host_repos = len(inst.repositories_get_list()) - - repo_id = inst.repositories_create(repo) - - host_repos = inst.repositories_get_list() - self.assertEquals(system_host_repos + 1, len(host_repos)) - - repo_info = inst.repository_lookup(repo_id) - self.assertEquals(True, repo_info['enabled']) - - inst.repository_disable(repo_id) - repo_info = inst.repository_lookup(repo_id) - self.assertEquals(False, repo_info['enabled']) - - inst.repository_enable(repo_id) - repo_info = inst.repository_lookup(repo_id) - self.assertEquals(True, repo_info['enabled']) - - # remove files creates - inst.repository_delete(repo_id) - - class BaseModelTests(unittest.TestCase): class FoosModel(object): def __init__(self): diff --git a/plugins/kimchi/tests/test_rest.py b/plugins/kimchi/tests/test_rest.py index 243074e..c0f219a 100644 --- a/plugins/kimchi/tests/test_rest.py +++ b/plugins/kimchi/tests/test_rest.py @@ -1226,60 +1226,6 @@ class RestTests(unittest.TestCase): # Distro not found error self.assertIn('KCHDISTRO0001E', distro.get('reason')) - def test_debugreports(self): - resp = request(host, ssl_port, '/plugins/kimchi/debugreports') - self.assertEquals(200, resp.status) - - def _report_delete(self, name): - request(host, ssl_port, '/plugins/kimchi/debugreports/%s' % name, '{}', - 'DELETE') - - def test_create_debugreport(self): - req = json.dumps({'name': 'report1'}) - with RollbackContext() as rollback: - resp = request(host, ssl_port, '/plugins/kimchi/debugreports', req, - 'POST') - self.assertEquals(202, resp.status) - task = json.loads(resp.read()) - # make sure the debugreport doesn't exist until the - # the task is finished - wait_task(self._task_lookup, task['id']) - rollback.prependDefer(self._report_delete, 'report2') - resp = request(host, ssl_port, - '/plugins/kimchi/debugreports/report1') - debugreport = json.loads(resp.read()) - self.assertEquals("report1", debugreport['name']) - self.assertEquals(200, resp.status) - req = json.dumps({'name': 'report2'}) - resp = request(host, ssl_port, - '/plugins/kimchi/debugreports/report1', req, 'PUT') - self.assertEquals(303, resp.status) - - def test_debugreport_download(self): - req = json.dumps({'name': 'report1'}) - with RollbackContext() as rollback: - resp = request(host, ssl_port, '/plugins/kimchi/debugreports', req, - 'POST') - self.assertEquals(202, resp.status) - task = json.loads(resp.read()) - # make sure the debugreport doesn't exist until the - # the task is finished - wait_task(self._task_lookup, task['id'], 20) - rollback.prependDefer(self._report_delete, 'report1') - resp = request(host, ssl_port, - '/plugins/kimchi/debugreports/report1') - debugreport = json.loads(resp.read()) - self.assertEquals("report1", debugreport['name']) - self.assertEquals(200, resp.status) - resp = request(host, ssl_port, - '/plugins/kimchi/debugreports/report1/content') - self.assertEquals(200, resp.status) - resp = request(host, ssl_port, - '/plugins/kimchi/debugreports/report1') - debugre = json.loads(resp.read()) - resp = request(host, ssl_port, debugre['uri']) - self.assertEquals(200, resp.status) - def test_repositories(self): def verify_repo(t, res): for field in ('repo_id', 'enabled', 'baseurl', 'config'): diff --git a/plugins/kimchi/ui/config/tab-ext.xml b/plugins/kimchi/ui/config/tab-ext.xml index ee88c88..48ec9ec 100644 --- a/plugins/kimchi/ui/config/tab-ext.xml +++ b/plugins/kimchi/ui/config/tab-ext.xml @@ -2,13 +2,6 @@ <tabs-ext> <tab> <access role="admin" mode="admin"/> - <access role="user" mode="none"/> - - <title>Host</title> - <path>plugins/kimchi/host.html</path> - </tab> - <tab> - <access role="admin" mode="admin"/> <access role="user" mode="byInstance"/> <title>Guests</title> -- 2.1.0

2 1

[PATCH 0/3 V2] Update README and header for Wok files
by pvital＠linux.vnet.ibm.com 30 Sep '15

30 Sep '15

2 5

[PATCH] Fix PYTHONPATH to be able to run "make check"
by Aline Manera 30 Sep '15

30 Sep '15

The PYTHONPATH must point to wok and plugins directories in order to run the tests properly. Fix it. There are a lot of tests failing that will be fixed in a next patch set. Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> --- src/wok/plugins/kimchi/tests/run_tests.sh.in | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/wok/plugins/kimchi/tests/run_tests.sh.in b/src/wok/plugins/kimchi/tests/run_tests.sh.in index beef75e..0e848c8 100644 --- a/src/wok/plugins/kimchi/tests/run_tests.sh.in +++ b/src/wok/plugins/kimchi/tests/run_tests.sh.in @@ -52,4 +52,6 @@ for ((i=0;i<${#LIST[@]};i++)); do fi done -PYTHONPATH=../plugins:../src:../ $CMD $OPTS ${MODEL_LIST[@]} ${MOCK_LIST[@]} +# ../../../../ refers to wok directory +# ../../../ refers to plugins directory +PYTHONPATH=../../../../:../../../ $CMD $OPTS ${MODEL_LIST[@]} ${MOCK_LIST[@]} -- 2.1.0

2 2

[PATCH 2/2] Move KCHASYNC0003E to Wok and update code.
by pvital＠linux.vnet.ibm.com 30 Sep '15

30 Sep '15

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> Move the KCHASYNC0003E ("Timeout of %(seconds)s seconds expired while running task '%(task)s.") message error from Kimchi to Wok and update its code to WOKASYNC0003E. Signed-off-by: Paulo Vital <pvital(a)linux.vnet.ibm.com> --- src/wok/i18n.py | 2 ++ src/wok/model/tasks.py | 2 +- src/wok/plugins/kimchi/i18n.py | 2 -- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/wok/i18n.py b/src/wok/i18n.py index 43a1edf..a6376bd 100644 --- a/src/wok/i18n.py +++ b/src/wok/i18n.py @@ -33,6 +33,8 @@ messages = { "WOKASYNC0001E": _("Datastore is not initiated in the model object."), "WOKASYNC0002E": _("Unable to start task due error: %(err)s"), + "WOKASYNC0003E": _("Timeout of %(seconds)s seconds expired while running task '%(task)s."), + "WOKAUTH0001E": _("Authentication failed for user '%(username)s'. [Error code: %(code)s]"), "WOKAUTH0002E": _("You are not authorized to access Kimchi"), diff --git a/src/wok/model/tasks.py b/src/wok/model/tasks.py index 678fdc2..d57c908 100644 --- a/src/wok/model/tasks.py +++ b/src/wok/model/tasks.py @@ -60,5 +60,5 @@ class TaskModel(object): time.sleep(1) - raise TimeoutExpired('KCHASYNC0003E', {'seconds': timeout, + raise TimeoutExpired('WOKASYNC0003E', {'seconds': timeout, 'task': task['target_uri']}) diff --git a/src/wok/plugins/kimchi/i18n.py b/src/wok/plugins/kimchi/i18n.py index 2274d6f..ea325b8 100644 --- a/src/wok/plugins/kimchi/i18n.py +++ b/src/wok/plugins/kimchi/i18n.py @@ -25,8 +25,6 @@ _ = gettext.gettext messages = { "KCHAPI0001E": _("Unknown parameter %(value)s"), - "KCHASYNC0003E": _("Timeout of %(seconds)s seconds expired while running task '%(task)s."), - "KCHAUTH0004E": _("User %(user_id)s not found with given LDAP settings."), "KCHDEVS0001E": _('Unknown "_cap" specified'), -- 2.4.3

2 1

[PATCH V3] Wok tests
by pvital＠linux.vnet.ibm.com 30 Sep '15

30 Sep '15

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> Moved those tests related to Wok framework from plugins/kimchi to wok structure. Updated build files and tests to make all test structure usable. Signed-off-by: Paulo Vital <pvital(a)linux.vnet.ibm.com> --- Makefile.am | 4 +- configure.ac | 1 + plugins/kimchi/tests/test_exception.py | 123 ------------ plugins/kimchi/tests/test_objectstore.py | 97 --------- plugins/kimchi/tests/test_plugin.py | 126 ------------ plugins/kimchi/tests/test_rollbackcontext.py | 99 --------- plugins/kimchi/tests/test_server.py | 289 --------------------------- plugins/kimchi/tests/test_utils.py | 69 ------- tests/Makefile.am | 50 +++++ tests/run_tests.sh.in | 55 +++++ tests/test_config.py.in | 107 ++++++++++ tests/test_exception.py | 132 ++++++++++++ tests/test_objectstore.py | 99 +++++++++ tests/test_plugin.py | 122 +++++++++++ tests/test_rollbackcontext.py | 101 ++++++++++ tests/test_server.py | 284 ++++++++++++++++++++++++++ tests/test_utils.py | 71 +++++++ tests/utils.py | 261 ++++++++++++++++++++++++ 18 files changed, 1285 insertions(+), 805 deletions(-) delete mode 100644 plugins/kimchi/tests/test_exception.py delete mode 100644 plugins/kimchi/tests/test_objectstore.py delete mode 100644 plugins/kimchi/tests/test_plugin.py delete mode 100644 plugins/kimchi/tests/test_rollbackcontext.py delete mode 100644 plugins/kimchi/tests/test_server.py delete mode 100644 plugins/kimchi/tests/test_utils.py create mode 100644 tests/Makefile.am create mode 100644 tests/run_tests.sh.in create mode 100644 tests/test_config.py.in create mode 100644 tests/test_exception.py create mode 100644 tests/test_objectstore.py create mode 100644 tests/test_plugin.py create mode 100644 tests/test_rollbackcontext.py create mode 100644 tests/test_server.py create mode 100644 tests/test_utils.py create mode 100644 tests/utils.py diff --git a/Makefile.am b/Makefile.am index 337aa24..0eabd3d 100644 --- a/Makefile.am +++ b/Makefile.am @@ -17,7 +17,7 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -SUBDIRS = src ui docs contrib po +SUBDIRS = src ui docs contrib po tests man_MANS = docs/wokd.8 @@ -36,7 +36,7 @@ EXTRA_DIST = \ $(NULL) -PEP8_BLACKLIST = *src/wok/config.py,*src/wok/i18n.py,*src/wok/plugins/kimchi +PEP8_BLACKLIST = *src/wok/config.py,*src/wok/i18n.py,*src/wok/plugins/kimchi,*tests/test_config.py SKIP_PYFLAKES_ERR = "\./src/wok/websocket\.py" diff --git a/configure.ac b/configure.ac index 8b2302f..98ce5ee 100644 --- a/configure.ac +++ b/configure.ac @@ -127,6 +127,7 @@ AC_CONFIG_FILES([ contrib/DEBIAN/control contrib/wok.spec.fedora contrib/wok.spec.suse + tests/Makefile ],[ chmod +x po/gen-pot ]) diff --git a/plugins/kimchi/tests/test_exception.py b/plugins/kimchi/tests/test_exception.py deleted file mode 100644 index 4459aa6..0000000 --- a/plugins/kimchi/tests/test_exception.py +++ /dev/null @@ -1,123 +0,0 @@ -# -# Kimchi -# -# Copyright IBM, Corp. 2013-2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -import json -import os -import unittest - -from wok.plugins.kimchi import mockmodel - -from utils import get_free_port, patch_auth, request, run_server - - -test_server = None -model = None -host = None -port = None -ssl_port = None - - -def setup_server(environment='development'): - global test_server, model, host, port, ssl_port - - patch_auth() - model = mockmodel.MockModel('/tmp/obj-store-test') - host = '127.0.0.1' - port = get_free_port('http') - ssl_port = get_free_port('https') - test_server = run_server(host, port, ssl_port, test_mode=True, model=model, - environment=environment) - - -class ExceptionTests(unittest.TestCase): - def tearDown(self): - test_server.stop() - os.unlink('/tmp/obj-store-test') - - def test_production_env(self): - """ - Test reasons sanitized in production env - """ - setup_server('production') - # test 404 - resp = json.loads( - request(host, ssl_port, '/plugins/kimchi/vms/blah').read() - ) - self.assertEquals('404 Not Found', resp.get('code')) - - # test 405 wrong method - resp = json.loads(request(host, ssl_port, '/', None, 'DELETE').read()) - msg = u'WOKAPI0002E: Delete is not allowed for wokroot' - self.assertEquals('405 Method Not Allowed', resp.get('code')) - self.assertEquals(msg, resp.get('reason')) - - # test 400 parse error - resp = json.loads( - request(host, ssl_port, '/plugins/kimchi/vms', '{', 'POST').read() - ) - msg = u'WOKAPI0006E: Unable to parse JSON request' - self.assertEquals('400 Bad Request', resp.get('code')) - self.assertEquals(msg, resp.get('reason')) - self.assertNotIn('call_stack', resp) - - # test 400 missing required parameter - req = json.dumps({}) - resp = json.loads( - request(host, ssl_port, '/plugins/kimchi/vms', req, 'POST').read() - ) - self.assertEquals('400 Bad Request', resp.get('code')) - m = u"KCHVM0016E: Specify a template to create a virtual machine from" - self.assertEquals(m, resp.get('reason')) - self.assertNotIn('call_stack', resp) - - def test_development_env(self): - """ - Test traceback thrown in development env - """ - setup_server() - # test 404 - resp = json.loads( - request(host, ssl_port, '/plugins/kimchi/vms/blah').read() - ) - self.assertEquals('404 Not Found', resp.get('code')) - - # test 405 wrong method - resp = json.loads(request(host, ssl_port, '/', None, 'DELETE').read()) - msg = u'WOKAPI0002E: Delete is not allowed for wokroot' - self.assertEquals('405 Method Not Allowed', resp.get('code')) - self.assertEquals(msg, resp.get('reason')) - - # test 400 parse error - resp = json.loads( - request(host, ssl_port, '/plugins/kimchi/vms', '{', 'POST').read() - ) - msg = u'WOKAPI0006E: Unable to parse JSON request' - self.assertEquals('400 Bad Request', resp.get('code')) - self.assertEquals(msg, resp.get('reason')) - self.assertIn('call_stack', resp) - - # test 400 missing required parameter - req = json.dumps({}) - resp = json.loads( - request(host, ssl_port, '/plugins/kimchi/vms', req, 'POST').read() - ) - m = u"KCHVM0016E: Specify a template to create a virtual machine from" - self.assertEquals('400 Bad Request', resp.get('code')) - self.assertEquals(m, resp.get('reason')) - self.assertIn('call_stack', resp) diff --git a/plugins/kimchi/tests/test_objectstore.py b/plugins/kimchi/tests/test_objectstore.py deleted file mode 100644 index 632786f..0000000 --- a/plugins/kimchi/tests/test_objectstore.py +++ /dev/null @@ -1,97 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Project Kimchi -# -# Copyright IBM, Corp. 2015 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -import os -import tempfile -import threading -import unittest - -from wok import objectstore -from wok.exception import NotFoundError - - -tmpfile = None - - -def setUpModule(): - global tmpfile - tmpfile = tempfile.mktemp() - - -def tearDownModule(): - os.unlink(tmpfile) - - -class ObjectStoreTests(unittest.TestCase): - def test_objectstore(self): - store = objectstore.ObjectStore(tmpfile) - - with store as session: - # Test create - session.store('fǒǒ', 'těst1', {'α': 1}) - session.store('fǒǒ', 'těst2', {'β': 2}) - - # Test list - items = session.get_list('fǒǒ') - self.assertTrue(u'těst1' in items) - self.assertTrue(u'těst2' in items) - - # Test get - item = session.get('fǒǒ', 'těst1') - self.assertEquals(1, item[u'α']) - - # Test delete - session.delete('fǒǒ', 'těst2') - self.assertEquals(1, len(session.get_list('fǒǒ'))) - - # Test get non-existent item - - self.assertRaises(NotFoundError, session.get, - 'α', 'β') - - # Test delete non-existent item - self.assertRaises(NotFoundError, session.delete, - 'fǒǒ', 'těst2') - - # Test refresh existing item - session.store('fǒǒ', 'těst1', {'α': 2}) - item = session.get('fǒǒ', 'těst1') - self.assertEquals(2, item[u'α']) - - def test_object_store_threaded(self): - def worker(ident): - with store as session: - session.store('foo', ident, {}) - - store = objectstore.ObjectStore(tmpfile) - - threads = [] - for i in xrange(50): - t = threading.Thread(target=worker, args=(i,)) - t.setDaemon(True) - t.start() - threads.append(t) - - for t in threads: - t.join() - - with store as session: - self.assertEquals(50, len(session.get_list('foo'))) - self.assertEquals(10, len(store._connections.keys())) diff --git a/plugins/kimchi/tests/test_plugin.py b/plugins/kimchi/tests/test_plugin.py deleted file mode 100644 index fc8e277..0000000 --- a/plugins/kimchi/tests/test_plugin.py +++ /dev/null @@ -1,126 +0,0 @@ -# -# Project Kimchi -# -# Copyright IBM, Corp. 2013-2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -import json -import os -import unittest -from functools import partial - -from wok.utils import get_enabled_plugins - -from wok.plugins.kimchi import mockmodel - -import utils - - -test_server = None -model = None -host = None -port = None -ssl_port = None - - -def setUpModule(): - global test_server, model, host, port, ssl_port - - utils.patch_auth() - model = mockmodel.MockModel('/tmp/obj-store-test') - host = '127.0.0.1' - port = utils.get_free_port('http') - ssl_port = utils.get_free_port('https') - test_server = utils.run_server(host, port, ssl_port, test_mode=True, - model=model) - - -def tearDownModule(): - test_server.stop() - os.unlink('/tmp/obj-store-test') - - -(a)unittest.skipUnless( - 'sample' in [plugin for plugin, _config in get_enabled_plugins()], - 'sample plugin is not enabled, skip this test!') -class PluginTests(unittest.TestCase): - - def setUp(self): - self.request = partial(utils.request, host, ssl_port) - - def _create_rectangle(self, name, length, width): - req = json.dumps({'name': name, 'length': length, 'width': width}) - resp = self.request('/plugins/sample/rectangles', req, 'POST') - return resp - - def _get_rectangle(self, name): - resp = self.request('/plugins/sample/rectangles/%s' % name) - return json.loads(resp.read()) - - def _create_rectangle_and_assert(self, name, length, width): - resp = self._create_rectangle(name, length, width) - self.assertEquals(201, resp.status) - - rectangle = self._get_rectangle(name) - self.assertEquals(rectangle['name'], name) - self.assertEquals(rectangle['length'], length) - self.assertEquals(rectangle['width'], width) - - def _get_rectangles_list(self): - resp = self.request('/plugins/sample/rectangles') - rectangles = json.loads(resp.read()) - name_list = [rectangle['name'] for rectangle in rectangles] - return name_list - - def test_rectangles(self): - # Create two new rectangles - self._create_rectangle_and_assert('small', 10, 8) - self._create_rectangle_and_assert('big', 20, 16) - - # Verify they're in the list - name_list = self._get_rectangles_list() - self.assertIn('small', name_list) - self.assertIn('big', name_list) - - # Update the big rectangle. - req = json.dumps({'length': 40, 'width': 30}) - resp = self.request('/plugins/sample/rectangles/big', req, 'PUT') - self.assertEquals(200, resp.status) - big = self._get_rectangle('big') - self.assertEquals(big['length'], 40) - self.assertEquals(big['width'], 30) - - # Delete two rectangles - resp = self.request('/plugins/sample/rectangles/big', '{}', 'DELETE') - self.assertEquals(204, resp.status) - resp = self.request('/plugins/sample/rectangles/small', '{}', 'DELETE') - self.assertEquals(204, resp.status) - name_list = self._get_rectangles_list() - self.assertEquals([], name_list) - - def test_bad_params(self): - # Bad name - resp = self._create_rectangle(1.0, 30, 40) - self.assertEquals(400, resp.status) - - # Bad length value - resp = self._create_rectangle('test', -10.0, 40) - self.assertEquals(400, resp.status) - - # Missing param for width - req = json.dumps({'name': 'nowidth', 'length': 40}) - resp = self.request('/plugins/sample/rectangles', req, 'POST') - self.assertEquals(400, resp.status) diff --git a/plugins/kimchi/tests/test_rollbackcontext.py b/plugins/kimchi/tests/test_rollbackcontext.py deleted file mode 100644 index 6eac6d0..0000000 --- a/plugins/kimchi/tests/test_rollbackcontext.py +++ /dev/null @@ -1,99 +0,0 @@ -# -# Project Kimchi -# -# Copyright IBM, Corp. 2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -import unittest - -from wok.rollbackcontext import RollbackContext - - -class FirstError(Exception): - '''A hypothetical exception to be raise in the test firstly.''' - pass - - -class SecondError(Exception): - '''A hypothetical exception to be raise in the test secondly.''' - pass - - -class RollbackContextTests(unittest.TestCase): - - def setUp(self): - self._counter = 0 - - def _inc_counter(self): - self._counter += 1 - - def _raise(self, exception=FirstError): - raise exception() - - def test_rollback(self): - with RollbackContext() as rollback: - rollback.prependDefer(self._inc_counter) - rollback.prependDefer(self._inc_counter) - self.assertEquals(self._counter, 2) - - def test_raise(self): - try: - with RollbackContext() as rollback: - rollback.prependDefer(self._inc_counter) - rollback.prependDefer(self._inc_counter) - raise FirstError() - rollback.prependDefer(self._inc_counter) - except FirstError: - # All undo before the FirstError should be run - self.assertEquals(self._counter, 2) - else: - self.fail('Should have raised FirstError') - - def test_raise_undo(self): - try: - with RollbackContext() as rollback: - rollback.prependDefer(self._inc_counter) - rollback.prependDefer(self._raise) - rollback.prependDefer(self._inc_counter) - except FirstError: - # All undo should be run - self.assertEquals(self._counter, 2) - else: - self.fail('Should have raised FirstError') - - def test_raise_prefer_original(self): - try: - with RollbackContext() as rollback: - rollback.prependDefer(self._raise, SecondError) - raise FirstError() - except FirstError: - pass - except SecondError: - self.fail('Should have preferred FirstError to SecondError') - else: - self.fail('Should have raised FirstError') - - def test_raise_prefer_first_undo(self): - try: - with RollbackContext() as rollback: - rollback.prependDefer(self._raise, SecondError) - rollback.prependDefer(self._raise, FirstError) - except FirstError: - pass - except SecondError: - self.fail('Should have preferred FirstError to SecondError') - else: - self.fail('Should have raised FirstError') diff --git a/plugins/kimchi/tests/test_server.py b/plugins/kimchi/tests/test_server.py deleted file mode 100644 index d5ef565..0000000 --- a/plugins/kimchi/tests/test_server.py +++ /dev/null @@ -1,289 +0,0 @@ -# -# Project Kimchi -# -# Copyright IBM, Corp. 2013-2015 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -import base64 -import cherrypy -import json -import os -import tempfile -import threading -import unittest -from functools import partial - -from wok.control.base import Collection, Resource - -from wok.plugins.kimchi import mockmodel - -import utils - - -test_server = None -model = None -host = None -port = None -ssl_port = None -cherrypy_port = None -tmpfile = None - - -def setUpModule(): - global test_server, model, host, port, ssl_port, cherrypy_port, tmpfile - - utils.patch_auth() - tmpfile = tempfile.mktemp() - model = mockmodel.MockModel(tmpfile) - host = '127.0.0.1' - port = utils.get_free_port('http') - ssl_port = utils.get_free_port('https') - cherrypy_port = utils.get_free_port('cherrypy_port') - test_server = utils.run_server(host, port, ssl_port, test_mode=True, - cherrypy_port=cherrypy_port, model=model) - - -def tearDownModule(): - test_server.stop() - os.unlink(tmpfile) - - -class ServerTests(unittest.TestCase): - def setUp(self): - self.request = partial(utils.request, host, ssl_port) - model.reset() - - def assertValidJSON(self, txt): - try: - json.loads(txt) - except ValueError: - self.fail("Invalid JSON: %s" % txt) - - def test_server_start(self): - """ - Test that we can start a server and receive HTTP:200. - """ - resp = self.request('/') - self.assertEquals(200, resp.status) - - def test_multithreaded_connection(self): - def worker(): - for i in xrange(100): - ret = model.vms_get_list() - self.assertEquals('test', ret[0]) - - threads = [] - for i in xrange(100): - t = threading.Thread(target=worker) - t.setDaemon(True) - t.start() - threads.append(t) - for t in threads: - t.join() - - def test_collection(self): - c = Collection(model) - - # The base Collection is always empty - cherrypy.request.method = 'GET' - cherrypy.request.headers['Accept'] = 'application/json' - self.assertEquals('[]', c.index()) - - # POST and DELETE raise HTTP:405 by default - for method in ('POST', 'DELETE'): - cherrypy.request.method = method - try: - c.index() - except cherrypy.HTTPError, e: - self.assertEquals(405, e.code) - else: - self.fail("Expected exception not raised") - - def test_resource(self): - r = Resource(model) - - # Test the base Resource representation - cherrypy.request.method = 'GET' - cherrypy.request.headers['Accept'] = 'application/json' - self.assertEquals('{}', r.index()) - - # POST and DELETE raise HTTP:405 by default - for method in ('POST', 'DELETE'): - cherrypy.request.method = method - try: - r.index() - except cherrypy.HTTPError, e: - self.assertEquals(405, e.code) - else: - self.fail("Expected exception not raised") - - def test_404(self): - """ - A non-existent path should return HTTP:404 - """ - url_list = ['/plugins/kimchi/doesnotexist', '/plugins/kimchi/vms/blah'] - for url in url_list: - resp = self.request(url) - self.assertEquals(404, resp.status) - - # Verify it works for DELETE too - resp = self.request('/plugins/kimchi/templates/blah', '', 'DELETE') - self.assertEquals(404, resp.status) - - def test_accepts(self): - """ - Verify the following expectations regarding the client Accept header: - If omitted, default to html - If 'application/json', serve the rest api - If 'text/html', serve the UI - If both of the above (in any order), serve the rest api - If neither of the above, HTTP:406 - """ - resp = self.request("/", headers={}) - location = resp.getheader('location') - self.assertTrue(location.endswith("login.html")) - resp = self.request("/login.html", headers={}) - self.assertTrue('<!doctype html>' in resp.read().lower()) - - resp = self.request("/", headers={'Accept': 'application/json'}) - self.assertValidJSON(resp.read()) - - resp = self.request("/", headers={'Accept': 'text/html'}) - location = resp.getheader('location') - self.assertTrue(location.endswith("login.html")) - - resp = self.request("/", headers={'Accept': - 'application/json, text/html'}) - self.assertValidJSON(resp.read()) - - resp = self.request("/", headers={'Accept': - 'text/html, application/json'}) - self.assertValidJSON(resp.read()) - - h = {'Accept': 'text/plain'} - resp = self.request('/', None, 'GET', h) - self.assertEquals(406, resp.status) - - def test_auth_unprotected(self): - hdrs = {'AUTHORIZATION': ''} - uris = ['/plugins/kimchi/js/kimchi.min.js', - '/plugins/kimchi/css/theme-default.min.css', - '/plugins/kimchi/images/icon-vm.png', - '/libs/jquery-1.10.0.min.js', - '/login.html', - '/logout'] - - for uri in uris: - resp = self.request(uri, None, 'HEAD', hdrs) - self.assertEquals(200, resp.status) - - def test_auth_protected(self): - hdrs = {'AUTHORIZATION': ''} - uris = ['/plugins/kimchi/vms', - '/plugins/kimchi/vms/doesnotexist', - '/tasks'] - - for uri in uris: - resp = self.request(uri, None, 'GET', hdrs) - self.assertEquals(401, resp.status) - - def test_auth_bad_creds(self): - # Test HTTPBA - hdrs = {'AUTHORIZATION': "Basic " + base64.b64encode("nouser:badpass")} - resp = self.request('/plugins/kimchi/vms', None, 'GET', hdrs) - self.assertEquals(401, resp.status) - - # Test REST API - hdrs = {'AUTHORIZATION': ''} - req = json.dumps({'username': 'nouser', 'password': 'badpass'}) - resp = self.request('/login', req, 'POST', hdrs) - self.assertEquals(401, resp.status) - - def test_auth_browser_no_httpba(self): - # Kimchi detects REST requests from the browser by looking for a - # specific header - hdrs = {"X-Requested-With": "XMLHttpRequest"} - - # Try our request (Note that request() will add a valid HTTPBA header) - resp = self.request('/plugins/kimchi/vms', None, 'GET', hdrs) - self.assertEquals(401, resp.status) - self.assertEquals(None, resp.getheader('WWW-Authenticate')) - - def test_auth_session(self): - hdrs = {'AUTHORIZATION': '', - 'Content-Type': 'application/json', - 'Accept': 'application/json'} - - # Test we are logged out - resp = self.request('/tasks', None, 'GET', hdrs) - self.assertEquals(401, resp.status) - - # Execute a login call - user, pw = mockmodel.fake_user.items()[0] - req = json.dumps({'username': user, 'password': pw}) - resp = self.request('/login', req, 'POST', hdrs) - self.assertEquals(200, resp.status) - - user_info = json.loads(resp.read()) - self.assertEquals(sorted(user_info.keys()), - ['groups', 'roles', 'username']) - roles = user_info['roles'] - for tab, role in roles.iteritems(): - self.assertEquals(role, u'admin') - - cookie = resp.getheader('set-cookie') - hdrs['Cookie'] = cookie - - # Test we are logged in with the cookie - resp = self.request('/tasks', None, 'GET', hdrs) - self.assertEquals(200, resp.status) - - # Execute a logout call - resp = self.request('/logout', '{}', 'POST', hdrs) - self.assertEquals(200, resp.status) - del hdrs['Cookie'] - - # Test we are logged out - resp = self.request('/tasks', None, 'GET', hdrs) - self.assertEquals(401, resp.status) - - def test_get_param(self): - # Create a mock ISO file - mockiso = '/tmp/mock.iso' - open('/tmp/mock.iso', 'w').close() - - # Create 2 different templates - req = json.dumps({'name': 'test-tmpl1', 'cdrom': mockiso}) - self.request('/plugins/kimchi/templates', req, 'POST') - - req = json.dumps({'name': 'test-tmpl2', 'cdrom': mockiso}) - self.request('/plugins/kimchi/templates', req, 'POST') - - # Remove mock iso - os.unlink(mockiso) - - # Get the templates - resp = self.request('/plugins/kimchi/templates') - self.assertEquals(200, resp.status) - res = json.loads(resp.read()) - self.assertEquals(2, len(res)) - - # Get a specific template - resp = self.request('/plugins/kimchi/templates?name=test-tmpl1') - self.assertEquals(200, resp.status) - res = json.loads(resp.read()) - self.assertEquals(1, len(res)) - self.assertEquals('test-tmpl1', res[0]['name']) diff --git a/plugins/kimchi/tests/test_utils.py b/plugins/kimchi/tests/test_utils.py deleted file mode 100644 index bcb14e2..0000000 --- a/plugins/kimchi/tests/test_utils.py +++ /dev/null @@ -1,69 +0,0 @@ -# -# Project Kimchi -# -# Copyright IBM, Corp. 2015 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - -import unittest - -from wok.exception import InvalidParameter -from wok.utils import convert_data_size - - -class UtilsTests(unittest.TestCase): - def test_convert_data_size(self): - failure_data = [{'val': None, 'from': 'MiB'}, - {'val': self, 'from': 'MiB'}, - {'val': 1, 'from': None}, - {'val': 1, 'from': ''}, - {'val': 1, 'from': 'foo'}, - {'val': 1, 'from': 'kib'}, - {'val': 1, 'from': 'MiB', 'to': None}, - {'val': 1, 'from': 'MiB', 'to': ''}, - {'val': 1, 'from': 'MiB', 'to': 'foo'}, - {'val': 1, 'from': 'MiB', 'to': 'kib'}] - - for d in failure_data: - if 'to' in d: - self.assertRaises(InvalidParameter, convert_data_size, - d['val'], d['from'], d['to']) - else: - self.assertRaises(InvalidParameter, convert_data_size, - d['val'], d['from']) - - success_data = [{'got': convert_data_size(5, 'MiB', 'MiB'), - 'want': 5}, - {'got': convert_data_size(5, 'MiB', 'KiB'), - 'want': 5120}, - {'got': convert_data_size(5, 'MiB', 'M'), - 'want': 5.24288}, - {'got': convert_data_size(5, 'MiB', 'GiB'), - 'want': 0.0048828125}, - {'got': convert_data_size(5, 'MiB', 'Tb'), - 'want': 4.194304e-05}, - {'got': convert_data_size(5, 'KiB', 'MiB'), - 'want': 0.0048828125}, - {'got': convert_data_size(5, 'M', 'MiB'), - 'want': 4.76837158203125}, - {'got': convert_data_size(5, 'GiB', 'MiB'), - 'want': 5120}, - {'got': convert_data_size(5, 'Tb', 'MiB'), - 'want': 596046.4477539062}, - {'got': convert_data_size(5, 'MiB'), - 'want': convert_data_size(5, 'MiB', 'B')}] - - for d in success_data: - self.assertEquals(d['got'], d['want']) diff --git a/tests/Makefile.am b/tests/Makefile.am new file mode 100644 index 0000000..e9800a5 --- /dev/null +++ b/tests/Makefile.am @@ -0,0 +1,50 @@ +# +# Project wok +# +# Copyright IBM Corp, 2015 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +EXTRA_DIST = \ + Makefile.am \ + run_tests.sh.in \ + test_config.py.in \ + $(filter-out test_config.py, $(wildcard *.py)) \ + $(NULL) + +noinst_SCRIPTS = run_tests.sh + +do_substitution = \ + sed -e 's,[@]HAVE_PYMOD_UNITTEST[@],$(HAVE_PYMOD_UNITTEST),g' \ + -e 's,[@]prefix[@],$(prefix),g' \ + -e 's,[@]datadir[@],$(datadir),g' \ + -e 's,[@]PYTHON_VERSION[@],$(PYTHON_VERSION),g' \ + -e 's,[@]wokdir[@],$(pythondir)/wok,g' \ + -e 's,[@]pkgdatadir[@],$(pkgdatadir),g' + + +run_tests.sh: run_tests.sh.in Makefile + $(do_substitution) < $(srcdir)/run_tests.sh.in > run_tests.sh + chmod +x run_tests.sh + +test_config.py: test_config.py.in Makefile + $(do_substitution) < $(srcdir)/test_config.py.in > test_config.py + +check-local: + $(MKDIR_P) $(top_srcdir)/data/screenshots + ./run_tests.sh + +BUILT_SOURCES = test_config.py +CLEANFILES = run_tests.sh test_config.py diff --git a/tests/run_tests.sh.in b/tests/run_tests.sh.in new file mode 100644 index 0000000..d31b194 --- /dev/null +++ b/tests/run_tests.sh.in @@ -0,0 +1,55 @@ +#!/bin/bash +# +# Project Wok +# +# Copyright IBM, Corp. 2015 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +HAVE_UNITTEST=@HAVE_PYMOD_UNITTEST@ +PYTHON_VER=@PYTHON_VERSION@ + +if [ "$1" = "-v" ]; then + OPTS="-v" + shift +else + OPTS="" +fi + +if [ $# -ne 0 ]; then + ARGS="$@" +else + ARGS=`find -name "test_*.py" | xargs -I @ basename @ .py` +fi + +if [ "$HAVE_UNITTEST" != "yes" -o "$PYTHON_VER" == "2.6" ]; then + CMD="unit2" +else + CMD="python -m unittest" +fi + +LIST=($ARGS) +MODEL_LIST=() +MOCK_LIST=() +for ((i=0;i<${#LIST[@]};i++)); do + + if [[ ${LIST[$i]} == test_model* ]]; then + MODEL_LIST+=(${LIST[$i]}) + else + MOCK_LIST+=(${LIST[$i]}) + fi +done + +PYTHONPATH=../src:../ $CMD $OPTS ${MODEL_LIST[@]} ${MOCK_LIST[@]} diff --git a/tests/test_config.py.in b/tests/test_config.py.in new file mode 100644 index 0000000..b4b46e1 --- /dev/null +++ b/tests/test_config.py.in @@ -0,0 +1,107 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2015 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import unittest + +from wok.config import Paths, WokConfig + + +get_prefix = None + + +def setUpModule(): + global get_prefix + get_prefix = Paths.get_prefix + + +def tearDownModule(): + Paths.get_prefix = get_prefix + + +class ConfigTests(unittest.TestCase): + def assertInstalledPath(self, actual, expected): + if '@pkgdatadir@' != '/usr/share/wok': + usr_local = '/usr/local' + if not expected.startswith('/usr'): + expected = usr_local + expected + self.assertEquals(actual, expected) + + def test_installed_paths(self): + Paths.get_prefix = lambda self: '@datadir@/wok' + paths = Paths() + self.assertInstalledPath(paths.state_dir, '/var/lib/wok') + self.assertInstalledPath(paths.log_dir, '/var/log/wok') + self.assertInstalledPath(paths.conf_dir, '/etc/wok') + self.assertInstalledPath(paths.src_dir, '@wokdir@') + self.assertInstalledPath(paths.plugins_dir, '@wokdir@/plugins') + self.assertInstalledPath(paths.ui_dir, '@datadir@/wok/ui') + self.assertInstalledPath(paths.mo_dir, '@prefix@/share/locale') + + def test_uninstalled_paths(self): + Paths.get_prefix = lambda self: '/home/user/wok' + paths = Paths() + self.assertEquals(paths.state_dir, '/home/user/wok/data') + self.assertEquals(paths.log_dir, '/home/user/wok/log') + self.assertEquals(paths.conf_dir, '/home/user/wok/src') + self.assertEquals(paths.src_dir, '/home/user/wok/src/wok') + self.assertEquals(paths.plugins_dir, '/home/user/wok/src/wok/plugins') + self.assertEquals(paths.ui_dir, '/home/user/wok/ui') + self.assertEquals(paths.mo_dir, '/home/user/wok/mo') + + def test_wok_config(self): + Paths.get_prefix = get_prefix + paths = Paths() + SESSIONSTIMEOUT = 10 + configObj = { + '/': { + 'tools.trailing_slash.on': False, + 'request.methods_with_bodies': ('POST', 'PUT'), + 'tools.nocache.on': True, + 'tools.proxy.on': True, + 'tools.sessions.on': True, + 'tools.sessions.name': 'wok', + 'tools.sessions.secure': True, + 'tools.sessions.httponly': True, + 'tools.sessions.locking': 'explicit', + 'tools.sessions.storage_type': 'ram', + 'tools.sessions.timeout': SESSIONSTIMEOUT, + 'tools.wokauth.on': False + }, + '/base64/jquery.base64.js': { + 'tools.staticfile.on': True, + 'tools.staticfile.filename': '%s/base64/jquery.base64.js' % + paths.ui_dir, + 'tools.nocache.on': True, + }, + '/wok-ui.html': { + 'tools.wokauth.on': True + }, + '/favicon.ico': { + 'tools.staticfile.on': True, + 'tools.staticfile.filename': + '%s/images/logo.ico' % paths.ui_dir + }, + '/robots.txt': { + 'tools.staticfile.on': True, + 'tools.staticfile.filename': '%s/robots.txt' % paths.ui_dir + }, + } + + wok_config = WokConfig.wok_config + self.assertEquals(wok_config, configObj) diff --git a/tests/test_exception.py b/tests/test_exception.py new file mode 100644 index 0000000..f012569 --- /dev/null +++ b/tests/test_exception.py @@ -0,0 +1,132 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2013-2015 +# +# Code delivered from Project Kimchi +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import json +import unittest + +from utils import get_free_port, patch_auth, request, run_server + + +test_server = None +model = None +host = None +port = None +ssl_port = None + + +def setup_server(environment='development'): + global test_server, model, host, port, ssl_port + + patch_auth() + host = '127.0.0.1' + port = get_free_port('http') + ssl_port = get_free_port('https') + test_server = run_server(host, port, ssl_port, test_mode=True, + environment=environment) + + +class ExceptionTests(unittest.TestCase): + def tearDown(self): + test_server.stop() + + def test_production_env(self): + """ + Test reasons sanitized in production env + """ + setup_server('production') + + # test 404 + resp = json.loads(request(host, ssl_port, '/tasks/blah').read()) + self.assertEquals('404 Not Found', resp.get('code')) + + # test 405 wrong method + resp = json.loads(request(host, ssl_port, '/', None, 'DELETE').read()) + msg = u'WOKAPI0002E: Delete is not allowed for wokroot' + self.assertEquals('405 Method Not Allowed', resp.get('code')) + self.assertEquals(msg, resp.get('reason')) + + # test 400 parse error + resp = json.loads(request(host, ssl_port, '/tasks', '{', + 'POST').read()) + msg = u'WOKAPI0006E: Unable to parse JSON request' + self.assertEquals('400 Bad Request', resp.get('code')) + self.assertEquals(msg, resp.get('reason')) + self.assertNotIn('call_stack', resp) + + # test 400 missing required parameter + # TODO: need add this test when some REST API from wok accepts POST +# req = json.dumps({}) +# resp = json.loads(request(host, ssl_port, '/tasks', req, +# 'POST').read()) +# self.assertEquals('400 Bad Request', resp.get('code')) +# m = u"KCHVM0016E: Specify a template to create a virtual machine from" +# self.assertEquals(m, resp.get('reason')) +# self.assertNotIn('call_stack', resp) + + # test 405 method not allowed + req = json.dumps({}) + resp = json.loads(request(host, ssl_port, '/tasks', req, + 'POST').read()) + m = u"WOKAPI0005E: Create is not allowed for tasks" + self.assertEquals('405 Method Not Allowed', resp.get('code')) + self.assertEquals(m, resp.get('reason')) + + def test_development_env(self): + """ + Test traceback thrown in development env + """ + setup_server() + # test 404 + resp = json.loads(request(host, ssl_port, '/tasks/blah').read()) + self.assertEquals('404 Not Found', resp.get('code')) + + # test 405 wrong method + resp = json.loads(request(host, ssl_port, '/', None, 'DELETE').read()) + msg = u'WOKAPI0002E: Delete is not allowed for wokroot' + self.assertEquals('405 Method Not Allowed', resp.get('code')) + self.assertEquals(msg, resp.get('reason')) + + # test 400 parse error + resp = json.loads(request(host, ssl_port, '/tasks', '{', + 'POST').read()) + msg = u'WOKAPI0006E: Unable to parse JSON request' + self.assertEquals('400 Bad Request', resp.get('code')) + self.assertEquals(msg, resp.get('reason')) + self.assertIn('call_stack', resp) + + # test 400 missing required parameter + # TODO: need add this test when some REST API from wok accepts POST +# req = json.dumps({}) +# resp = json.loads(request(host, ssl_port, '/tasks', req, +# 'POST').read()) +# m = u"KCHVM0016E: Specify a template to create a virtual machine from" +# self.assertEquals('400 Bad Request', resp.get('code')) +# self.assertEquals(m, resp.get('reason')) +# self.assertIn('call_stack', resp) + + # test 405 method not allowed + req = json.dumps({}) + resp = json.loads(request(host, ssl_port, '/tasks', req, + 'POST').read()) + m = u"WOKAPI0005E: Create is not allowed for tasks" + self.assertEquals('405 Method Not Allowed', resp.get('code')) + self.assertEquals(m, resp.get('reason')) + self.assertIn('call_stack', resp) diff --git a/tests/test_objectstore.py b/tests/test_objectstore.py new file mode 100644 index 0000000..bce125e --- /dev/null +++ b/tests/test_objectstore.py @@ -0,0 +1,99 @@ +# -*- coding: utf-8 -*- +# +# Project Wok +# +# Copyright IBM, Corp. 2015 +# +# Code delivered from Project Kimchi +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import os +import tempfile +import threading +import unittest + +from wok import objectstore +from wok.exception import NotFoundError + + +tmpfile = None + + +def setUpModule(): + global tmpfile + tmpfile = tempfile.mktemp() + + +def tearDownModule(): + os.unlink(tmpfile) + + +class ObjectStoreTests(unittest.TestCase): + def test_objectstore(self): + store = objectstore.ObjectStore(tmpfile) + + with store as session: + # Test create + session.store('fǒǒ', 'těst1', {'α': 1}) + session.store('fǒǒ', 'těst2', {'β': 2}) + + # Test list + items = session.get_list('fǒǒ') + self.assertTrue(u'těst1' in items) + self.assertTrue(u'těst2' in items) + + # Test get + item = session.get('fǒǒ', 'těst1') + self.assertEquals(1, item[u'α']) + + # Test delete + session.delete('fǒǒ', 'těst2') + self.assertEquals(1, len(session.get_list('fǒǒ'))) + + # Test get non-existent item + + self.assertRaises(NotFoundError, session.get, + 'α', 'β') + + # Test delete non-existent item + self.assertRaises(NotFoundError, session.delete, + 'fǒǒ', 'těst2') + + # Test refresh existing item + session.store('fǒǒ', 'těst1', {'α': 2}) + item = session.get('fǒǒ', 'těst1') + self.assertEquals(2, item[u'α']) + + def test_object_store_threaded(self): + def worker(ident): + with store as session: + session.store('foo', ident, {}) + + store = objectstore.ObjectStore(tmpfile) + + threads = [] + for i in xrange(50): + t = threading.Thread(target=worker, args=(i,)) + t.setDaemon(True) + t.start() + threads.append(t) + + for t in threads: + t.join() + + with store as session: + self.assertEquals(50, len(session.get_list('foo'))) + self.assertEquals(10, len(store._connections.keys())) diff --git a/tests/test_plugin.py b/tests/test_plugin.py new file mode 100644 index 0000000..f2e12d3 --- /dev/null +++ b/tests/test_plugin.py @@ -0,0 +1,122 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2013-2015 +# +# Code delivered from Project Kimchi +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import json +import unittest +from functools import partial + +from wok.utils import get_enabled_plugins + +import utils + + +test_server = None +model = None +host = None +port = None +ssl_port = None + + +def setUpModule(): + global test_server, model, host, port, ssl_port + + utils.patch_auth() + host = '127.0.0.1' + port = utils.get_free_port('http') + ssl_port = utils.get_free_port('https') + test_server = utils.run_server(host, port, ssl_port, test_mode=True) + + +def tearDownModule(): + test_server.stop() + + +(a)unittest.skipUnless( + 'sample' in [plugin for plugin, _config in get_enabled_plugins()], + 'sample plugin is not enabled, skip this test!') +class PluginTests(unittest.TestCase): + + def setUp(self): + self.request = partial(utils.request, host, ssl_port) + + def _create_rectangle(self, name, length, width): + req = json.dumps({'name': name, 'length': length, 'width': width}) + resp = self.request('/plugins/sample/rectangles', req, 'POST') + return resp + + def _get_rectangle(self, name): + resp = self.request('/plugins/sample/rectangles/%s' % name) + return json.loads(resp.read()) + + def _create_rectangle_and_assert(self, name, length, width): + resp = self._create_rectangle(name, length, width) + self.assertEquals(201, resp.status) + + rectangle = self._get_rectangle(name) + self.assertEquals(rectangle['name'], name) + self.assertEquals(rectangle['length'], length) + self.assertEquals(rectangle['width'], width) + + def _get_rectangles_list(self): + resp = self.request('/plugins/sample/rectangles') + rectangles = json.loads(resp.read()) + name_list = [rectangle['name'] for rectangle in rectangles] + return name_list + + def test_rectangles(self): + # Create two new rectangles + self._create_rectangle_and_assert('small', 10, 8) + self._create_rectangle_and_assert('big', 20, 16) + + # Verify they're in the list + name_list = self._get_rectangles_list() + self.assertIn('small', name_list) + self.assertIn('big', name_list) + + # Update the big rectangle. + req = json.dumps({'length': 40, 'width': 30}) + resp = self.request('/plugins/sample/rectangles/big', req, 'PUT') + self.assertEquals(200, resp.status) + big = self._get_rectangle('big') + self.assertEquals(big['length'], 40) + self.assertEquals(big['width'], 30) + + # Delete two rectangles + resp = self.request('/plugins/sample/rectangles/big', '{}', 'DELETE') + self.assertEquals(204, resp.status) + resp = self.request('/plugins/sample/rectangles/small', '{}', 'DELETE') + self.assertEquals(204, resp.status) + name_list = self._get_rectangles_list() + self.assertEquals([], name_list) + + def test_bad_params(self): + # Bad name + resp = self._create_rectangle(1.0, 30, 40) + self.assertEquals(400, resp.status) + + # Bad length value + resp = self._create_rectangle('test', -10.0, 40) + self.assertEquals(400, resp.status) + + # Missing param for width + req = json.dumps({'name': 'nowidth', 'length': 40}) + resp = self.request('/plugins/sample/rectangles', req, 'POST') + self.assertEquals(400, resp.status) diff --git a/tests/test_rollbackcontext.py b/tests/test_rollbackcontext.py new file mode 100644 index 0000000..3282be5 --- /dev/null +++ b/tests/test_rollbackcontext.py @@ -0,0 +1,101 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2014-2015 +# +# Code delivered from Project Kimchi +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import unittest + +from wok.rollbackcontext import RollbackContext + + +class FirstError(Exception): + '''A hypothetical exception to be raise in the test firstly.''' + pass + + +class SecondError(Exception): + '''A hypothetical exception to be raise in the test secondly.''' + pass + + +class RollbackContextTests(unittest.TestCase): + + def setUp(self): + self._counter = 0 + + def _inc_counter(self): + self._counter += 1 + + def _raise(self, exception=FirstError): + raise exception() + + def test_rollback(self): + with RollbackContext() as rollback: + rollback.prependDefer(self._inc_counter) + rollback.prependDefer(self._inc_counter) + self.assertEquals(self._counter, 2) + + def test_raise(self): + try: + with RollbackContext() as rollback: + rollback.prependDefer(self._inc_counter) + rollback.prependDefer(self._inc_counter) + raise FirstError() + rollback.prependDefer(self._inc_counter) + except FirstError: + # All undo before the FirstError should be run + self.assertEquals(self._counter, 2) + else: + self.fail('Should have raised FirstError') + + def test_raise_undo(self): + try: + with RollbackContext() as rollback: + rollback.prependDefer(self._inc_counter) + rollback.prependDefer(self._raise) + rollback.prependDefer(self._inc_counter) + except FirstError: + # All undo should be run + self.assertEquals(self._counter, 2) + else: + self.fail('Should have raised FirstError') + + def test_raise_prefer_original(self): + try: + with RollbackContext() as rollback: + rollback.prependDefer(self._raise, SecondError) + raise FirstError() + except FirstError: + pass + except SecondError: + self.fail('Should have preferred FirstError to SecondError') + else: + self.fail('Should have raised FirstError') + + def test_raise_prefer_first_undo(self): + try: + with RollbackContext() as rollback: + rollback.prependDefer(self._raise, SecondError) + rollback.prependDefer(self._raise, FirstError) + except FirstError: + pass + except SecondError: + self.fail('Should have preferred FirstError to SecondError') + else: + self.fail('Should have raised FirstError') diff --git a/tests/test_server.py b/tests/test_server.py new file mode 100644 index 0000000..8ffc388 --- /dev/null +++ b/tests/test_server.py @@ -0,0 +1,284 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2013-2015 +# +# Code delivered from Project Kimchi +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import base64 +import cherrypy +import json +import tempfile +import threading +import unittest +from functools import partial + +from wok.control.base import Collection, Resource + +import utils + + +test_server = None +model = None +host = None +port = None +ssl_port = None +cherrypy_port = None +tmpfile = None + + +def setUpModule(): + global test_server, model, host, port, ssl_port, cherrypy_port, tmpfile + + utils.patch_auth() + tmpfile = tempfile.mktemp() + host = '127.0.0.1' + port = utils.get_free_port('http') + ssl_port = utils.get_free_port('https') + cherrypy_port = utils.get_free_port('cherrypy_port') + test_server = utils.run_server(host, port, ssl_port, test_mode=True, + cherrypy_port=cherrypy_port) + + +def tearDownModule(): + test_server.stop() + + +class ServerTests(unittest.TestCase): + def setUp(self): + self.request = partial(utils.request, host, ssl_port) + + def assertValidJSON(self, txt): + try: + json.loads(txt) + except ValueError: + self.fail("Invalid JSON: %s" % txt) + + def test_server_start(self): + """ + Test that we can start a server and receive HTTP:200. + """ + resp = self.request('/') + self.assertEquals(200, resp.status) + + def test_multithreaded_connection(self): + def worker(): + for i in xrange(100): + ret = ['test'] + self.assertEquals('test', ret[0]) + + threads = [] + for i in xrange(100): + t = threading.Thread(target=worker) + t.setDaemon(True) + t.start() + threads.append(t) + for t in threads: + t.join() + + def test_collection(self): + c = Collection(model) + + # The base Collection is always empty + cherrypy.request.method = 'GET' + cherrypy.request.headers['Accept'] = 'application/json' + self.assertEquals('[]', c.index()) + + # POST and DELETE raise HTTP:405 by default + for method in ('POST', 'DELETE'): + cherrypy.request.method = method + try: + c.index() + except cherrypy.HTTPError, e: + self.assertEquals(405, e.code) + else: + self.fail("Expected exception not raised") + + def test_resource(self): + r = Resource(model) + + # Test the base Resource representation + cherrypy.request.method = 'GET' + cherrypy.request.headers['Accept'] = 'application/json' + self.assertEquals('{}', r.index()) + + # POST and DELETE raise HTTP:405 by default + for method in ('POST', 'DELETE'): + cherrypy.request.method = method + try: + r.index() + except cherrypy.HTTPError, e: + self.assertEquals(405, e.code) + else: + self.fail("Expected exception not raised") + + def test_404(self): + """ + A non-existent path should return HTTP:404 + """ + url_list = ['/doesnotexist', '/tasks/blah'] + for url in url_list: + resp = self.request(url) + self.assertEquals(404, resp.status) + + # Verify it works for DELETE too + resp = self.request('/tasks/blah', '', 'DELETE') + self.assertEquals(404, resp.status) + + def test_accepts(self): + """ + Verify the following expectations regarding the client Accept header: + If omitted, default to html + If 'application/json', serve the rest api + If 'text/html', serve the UI + If both of the above (in any order), serve the rest api + If neither of the above, HTTP:406 + """ + resp = self.request("/", headers={}) + location = resp.getheader('location') + self.assertTrue(location.endswith("login.html")) + resp = self.request("/login.html", headers={}) + self.assertTrue('<!doctype html>' in resp.read().lower()) + + resp = self.request("/", headers={'Accept': 'application/json'}) + self.assertValidJSON(resp.read()) + + resp = self.request("/", headers={'Accept': 'text/html'}) + location = resp.getheader('location') + self.assertTrue(location.endswith("login.html")) + + resp = self.request("/", headers={'Accept': + 'application/json, text/html'}) + self.assertValidJSON(resp.read()) + + resp = self.request("/", headers={'Accept': + 'text/html, application/json'}) + self.assertValidJSON(resp.read()) + + h = {'Accept': 'text/plain'} + resp = self.request('/', None, 'GET', h) + self.assertEquals(406, resp.status) + + def test_auth_unprotected(self): + hdrs = {'AUTHORIZATION': ''} + uris = ['/js/wok.min.js', + '/css/theme-default.min.css', + '/images/favicon.png', + '/libs/jquery/jquery.min.js', + '/login.html', + '/logout'] + + for uri in uris: + resp = self.request(uri, None, 'HEAD', hdrs) + self.assertEquals(200, resp.status) + + def test_auth_protected(self): + hdrs = {'AUTHORIZATION': ''} + uris = ['/tasks'] + + for uri in uris: + resp = self.request(uri, None, 'GET', hdrs) + self.assertEquals(401, resp.status) + + def test_auth_bad_creds(self): + # Test HTTPBA + hdrs = {'AUTHORIZATION': "Basic " + base64.b64encode("nouser:badpass")} + resp = self.request('/tasks', None, 'GET', hdrs) + self.assertEquals(401, resp.status) + + # Test REST API + hdrs = {'AUTHORIZATION': ''} + req = json.dumps({'username': 'nouser', 'password': 'badpass'}) + resp = self.request('/login', req, 'POST', hdrs) + self.assertEquals(401, resp.status) + + def test_auth_browser_no_httpba(self): + # Kimchi detects REST requests from the browser by looking for a + # specific header + hdrs = {"X-Requested-With": "XMLHttpRequest"} + + # Try our request (Note that request() will add a valid HTTPBA header) + resp = self.request('/tasks', None, 'GET', hdrs) + self.assertEquals(401, resp.status) + self.assertEquals(None, resp.getheader('WWW-Authenticate')) + + def test_auth_session(self): + hdrs = {'AUTHORIZATION': '', + 'Content-Type': 'application/json', + 'Accept': 'application/json'} + + # Test we are logged out + resp = self.request('/tasks', None, 'GET', hdrs) + self.assertEquals(401, resp.status) + + # Execute a login call + user, pw = utils.fake_user.items()[0] + req = json.dumps({'username': user, 'password': pw}) + resp = self.request('/login', req, 'POST', hdrs) + self.assertEquals(200, resp.status) + + user_info = json.loads(resp.read()) + self.assertEquals(sorted(user_info.keys()), + ['groups', 'roles', 'username']) + roles = user_info['roles'] + for tab, role in roles.iteritems(): + self.assertEquals(role, u'admin') + + cookie = resp.getheader('set-cookie') + hdrs['Cookie'] = cookie + + # Test we are logged in with the cookie + resp = self.request('/tasks', None, 'GET', hdrs) + self.assertEquals(200, resp.status) + + # Execute a logout call + resp = self.request('/logout', '{}', 'POST', hdrs) + self.assertEquals(200, resp.status) + del hdrs['Cookie'] + + # Test we are logged out + resp = self.request('/tasks', None, 'GET', hdrs) + self.assertEquals(401, resp.status) + + # TODO: uncomment and adapt when some wok API accepts parameters to test +# def test_get_param(self): +# # Create a mock ISO file +# mockiso = '/tmp/mock.iso' +# open('/tmp/mock.iso', 'w').close() +# +# # Create 2 different templates +# req = json.dumps({'name': 'test-tmpl1', 'cdrom': mockiso}) +# self.request('/plugins/kimchi/templates', req, 'POST') +# +# req = json.dumps({'name': 'test-tmpl2', 'cdrom': mockiso}) +# self.request('/plugins/kimchi/templates', req, 'POST') +# +# # Remove mock iso +# os.unlink(mockiso) +# +# # Get the templates +# resp = self.request('/plugins/kimchi/templates') +# self.assertEquals(200, resp.status) +# res = json.loads(resp.read()) +# self.assertEquals(2, len(res)) +# +# # Get a specific template +# resp = self.request('/plugins/kimchi/templates?name=test-tmpl1') +# self.assertEquals(200, resp.status) +# res = json.loads(resp.read()) +# self.assertEquals(1, len(res)) +# self.assertEquals('test-tmpl1', res[0]['name']) diff --git a/tests/test_utils.py b/tests/test_utils.py new file mode 100644 index 0000000..745d3d6 --- /dev/null +++ b/tests/test_utils.py @@ -0,0 +1,71 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2015 +# +# Code delivered from Project Kimchi +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import unittest + +from wok.exception import InvalidParameter +from wok.utils import convert_data_size + + +class UtilsTests(unittest.TestCase): + def test_convert_data_size(self): + failure_data = [{'val': None, 'from': 'MiB'}, + {'val': self, 'from': 'MiB'}, + {'val': 1, 'from': None}, + {'val': 1, 'from': ''}, + {'val': 1, 'from': 'foo'}, + {'val': 1, 'from': 'kib'}, + {'val': 1, 'from': 'MiB', 'to': None}, + {'val': 1, 'from': 'MiB', 'to': ''}, + {'val': 1, 'from': 'MiB', 'to': 'foo'}, + {'val': 1, 'from': 'MiB', 'to': 'kib'}] + + for d in failure_data: + if 'to' in d: + self.assertRaises(InvalidParameter, convert_data_size, + d['val'], d['from'], d['to']) + else: + self.assertRaises(InvalidParameter, convert_data_size, + d['val'], d['from']) + + success_data = [{'got': convert_data_size(5, 'MiB', 'MiB'), + 'want': 5}, + {'got': convert_data_size(5, 'MiB', 'KiB'), + 'want': 5120}, + {'got': convert_data_size(5, 'MiB', 'M'), + 'want': 5.24288}, + {'got': convert_data_size(5, 'MiB', 'GiB'), + 'want': 0.0048828125}, + {'got': convert_data_size(5, 'MiB', 'Tb'), + 'want': 4.194304e-05}, + {'got': convert_data_size(5, 'KiB', 'MiB'), + 'want': 0.0048828125}, + {'got': convert_data_size(5, 'M', 'MiB'), + 'want': 4.76837158203125}, + {'got': convert_data_size(5, 'GiB', 'MiB'), + 'want': 5120}, + {'got': convert_data_size(5, 'Tb', 'MiB'), + 'want': 596046.4477539062}, + {'got': convert_data_size(5, 'MiB'), + 'want': convert_data_size(5, 'MiB', 'B')}] + + for d in success_data: + self.assertEquals(d['got'], d['want']) diff --git a/tests/utils.py b/tests/utils.py new file mode 100644 index 0000000..dcea7d9 --- /dev/null +++ b/tests/utils.py @@ -0,0 +1,261 @@ +# +# Project Wok +# +# Copyright IBM, Corp. 2013-2015 +# +# Code delivered from Project Kimchi +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# + +import base64 +import cherrypy +import grp +import httplib +import inspect +import json +import os +import socket +import ssl +import sys +import threading +import time +import unittest +from contextlib import closing +from lxml import etree + +import wok.server +from wok.config import config, PluginPaths +from wok.auth import User, USER_NAME, USER_GROUPS, USER_ROLES, tabs +from wok.exception import NotFoundError, OperationFailed +from wok.utils import wok_log + + +_ports = {} +fake_user = {'test': 'passw0rd'} + +# provide missing unittest decorators and API for python 2.6; these decorators +# do not actually work, just avoid the syntax failure +if sys.version_info[:2] == (2, 6): + def skipUnless(condition, reason): + if not condition: + sys.stderr.write('[expected failure] ') + raise Exception(reason) + return lambda obj: obj + + unittest.skipUnless = skipUnless + unittest.expectedFailure = lambda obj: obj + + def assertGreater(self, a, b, msg=None): + if not a > b: + self.fail('%s not greater than %s' % (repr(a), repr(b))) + + def assertGreaterEqual(self, a, b, msg=None): + if not a >= b: + self.fail('%s not greater than or equal to %s' + % (repr(a), repr(b))) + + def assertIsInstance(self, obj, cls, msg=None): + if not isinstance(obj, cls): + self.fail('%s is not an instance of %r' % (repr(obj), cls)) + + def assertIn(self, a, b, msg=None): + if a not in b: + self.fail("%s is not in %b" % (repr(a), repr(b))) + + def assertNotIn(self, a, b, msg=None): + if a in b: + self.fail("%s is in %b" % (repr(a), repr(b))) + + unittest.TestCase.assertGreaterEqual = assertGreaterEqual + unittest.TestCase.assertGreater = assertGreater + unittest.TestCase.assertIsInstance = assertIsInstance + unittest.TestCase.assertIn = assertIn + unittest.TestCase.assertNotIn = assertNotIn + + +def get_free_port(name='http'): + global _ports + if _ports.get(name) is not None: + return _ports[name] + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + with closing(sock): + try: + sock.bind(("0.0.0.0", 0)) + except: + raise Exception("Could not find a free port") + _ports[name] = sock.getsockname()[1] + return _ports[name] + + +def run_server(host, port, ssl_port, test_mode, cherrypy_port=None, + model=None, environment='development'): + + if cherrypy_port is None: + cherrypy_port = get_free_port('cherrypy_port') + + if ssl_port is None: + ssl_port = get_free_port('https') + + args = type('_', (object,), + {'host': host, 'port': port, 'ssl_port': ssl_port, + 'cherrypy_port': cherrypy_port, 'max_body_size': '4*1024', + 'ssl_cert': '', 'ssl_key': '', + 'test': test_mode, 'access_log': '/dev/null', + 'error_log': '/dev/null', 'environment': environment, + 'log_level': 'debug'})() + if model is not None: + setattr(args, 'model', model) + + s = wok.server.Server(args) + t = threading.Thread(target=s.start) + t.setDaemon(True) + t.start() + cherrypy.engine.wait(cherrypy.engine.states.STARTED) + return s + + +def silence_server(): + """ + Silence server status messages on stdout + """ + cherrypy.config.update({"environment": "embedded"}) + + +def running_as_root(): + return os.geteuid() == 0 + + +def _request(conn, path, data, method, headers): + if headers is None: + headers = {'Content-Type': 'application/json', + 'Accept': 'application/json'} + if 'AUTHORIZATION' not in headers.keys(): + user, pw = fake_user.items()[0] + hdr = "Basic " + base64.b64encode("%s:%s" % (user, pw)) + headers['AUTHORIZATION'] = hdr + conn.request(method, path, data, headers) + return conn.getresponse() + + +def request(host, port, path, data=None, method='GET', headers=None): + # verify if HTTPSConnection has context parameter + if "context" in inspect.getargspec(httplib.HTTPSConnection.__init__).args: + context = ssl._create_unverified_context() + conn = httplib.HTTPSConnection(host, port, context=context) + else: + conn = httplib.HTTPSConnection(host, port) + + return _request(conn, path, data, method, headers) + + +def get_remote_iso_path(): + """ + Get a remote iso with the right arch from the distro files shipped + with kimchi. + """ + host_arch = os.uname()[4] + remote_path = '' + with open(os.path.join(PluginPaths('kimchi').conf_dir, 'distros.d', + 'fedora.json')) as fedora_isos: + # Get a list of dicts + json_isos_list = json.load(fedora_isos) + for iso in json_isos_list: + if (iso.get('os_arch')) == host_arch: + remote_path = iso.get('path') + break + + return remote_path + + +class FakeUser(User): + auth_type = "fake" + sudo = True + + def __init__(self, username): + self.user = {} + self.user[USER_NAME] = username + self.user[USER_GROUPS] = None + self.user[USER_ROLES] = dict.fromkeys(tabs, 'user') + + def get_groups(self): + return sorted([group.gr_name for group in grp.getgrall()])[0:3] + + def get_roles(self): + if self.sudo: + self.user[USER_ROLES] = dict.fromkeys(tabs, 'admin') + return self.user[USER_ROLES] + + def get_user(self): + return self.user + + @staticmethod + def authenticate(username, password, service="passwd"): + try: + return fake_user[username] == password + except KeyError, e: + raise OperationFailed("WOKAUTH0001E", {'username': 'username', + 'code': e.message}) + + +def patch_auth(sudo=True): + """ + Override the authenticate function with a simple test against an + internal dict of users and passwords. + """ + config.set("authentication", "method", "fake") + FakeUser.sudo = sudo + + +def normalize_xml(xml_str): + return etree.tostring(etree.fromstring(xml_str, + etree.XMLParser(remove_blank_text=True))) + + +def wait_task(task_lookup, taskid, timeout=10): + for i in range(0, timeout): + task_info = task_lookup(taskid) + if task_info['status'] == "running": + wok_log.info("Waiting task %s, message: %s", + taskid, task_info['message']) + time.sleep(1) + else: + return + wok_log.error("Timeout while process long-run task, " + "try to increase timeout value.") + + +# The action functions in model backend raise NotFoundError exception if the +# element is not found. But in some tests, these functions are called after +# the element has been deleted if test finishes correctly, then NofFoundError +# exception is raised and rollback breaks. To avoid it, this wrapper ignores +# the NotFoundError. +def rollback_wrapper(func, resource, *args): + try: + func(resource, *args) + except NotFoundError: + # VM has been deleted already + return + + +# This function is used to test storage volume upload. +# If we use self.request, we may encode multipart formdata by ourselves +# requests lib take care of encode part, so use this lib instead +def fake_auth_header(): + headers = {'Accept': 'application/json'} + user, pw = fake_user.items()[0] + hdr = "Basic " + base64.b64encode("%s:%s" % (user, pw)) + headers['AUTHORIZATION'] = hdr + return headers -- 2.4.3

2 1

[PATCH 0/3] Update README and header for Wok files
by pvital＠linux.vnet.ibm.com 30 Sep '15

30 Sep '15

2 4

[PATCH 0/2 V2] Wok tests
by pvital＠linux.vnet.ibm.com 30 Sep '15

30 Sep '15

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> V1 -> V2: - Rebase with new plugins structure. - Changed headers V1: Patch-set with Wok tests, provided from old Kimchi tests, and modification of one error message to Wok format. Paulo Vital (2): Wok tests Move KCHASYNC0003E to Wok and update code. Makefile.am | 4 +- configure.ac | 1 + plugins/kimchi/tests/test_exception.py | 123 ------------ plugins/kimchi/tests/test_objectstore.py | 97 --------- plugins/kimchi/tests/test_plugin.py | 126 ------------ plugins/kimchi/tests/test_rollbackcontext.py | 99 --------- plugins/kimchi/tests/test_server.py | 289 --------------------------- plugins/kimchi/tests/test_utils.py | 69 ------- src/wok/i18n.py | 2 + src/wok/model/tasks.py | 2 +- src/wok/plugins/kimchi/i18n.py | 2 - tests/Makefile.am | 50 +++++ tests/run_tests.sh.in | 55 +++++ tests/test_config.py.in | 132 ++++++++++++ tests/test_exception.py | 132 ++++++++++++ tests/test_objectstore.py | 99 +++++++++ tests/test_plugin.py | 122 +++++++++++ tests/test_rollbackcontext.py | 101 ++++++++++ tests/test_server.py | 284 ++++++++++++++++++++++++++ tests/test_utils.py | 71 +++++++ tests/utils.py | 261 ++++++++++++++++++++++++ 21 files changed, 1313 insertions(+), 808 deletions(-) delete mode 100644 plugins/kimchi/tests/test_exception.py delete mode 100644 plugins/kimchi/tests/test_objectstore.py delete mode 100644 plugins/kimchi/tests/test_plugin.py delete mode 100644 plugins/kimchi/tests/test_rollbackcontext.py delete mode 100644 plugins/kimchi/tests/test_server.py delete mode 100644 plugins/kimchi/tests/test_utils.py create mode 100644 tests/Makefile.am create mode 100644 tests/run_tests.sh.in create mode 100644 tests/test_config.py.in create mode 100644 tests/test_exception.py create mode 100644 tests/test_objectstore.py create mode 100644 tests/test_plugin.py create mode 100644 tests/test_rollbackcontext.py create mode 100644 tests/test_server.py create mode 100644 tests/test_utils.py create mode 100644 tests/utils.py -- 2.4.3

1 2