[PATCH v2 0/4] Github #329: "YOU SHALL NOT ... run as root!"
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> Changes: *v2: - adressed Ramon's comment about the python path in kimchid.in This patch series revamps the launch architecture to allow kimchi to not run as root while being exposed at an open http/https port. The solution adopted is using a reverse http proxy (nginx) to make the 'bridge' between two distinct cherrypy processes, one running as frontend as a regular user and another running as backend, as root. The communication with the outside will be done through nginx, running as a regular user too. The changes were heavy in the kimchid script, but the startup and usage options still the same. User-wise, there shouldn't be any functional change in the way kimchi works after applying this change. Refer to https://github.com/kimchi-project/kimchi/issues/329 for further information in all the other approaches considered and why they didn't work out. Daniel Henrique Barboza (4): Github #329: kimchid script changes Github #329: new launch script and proxy template Github #329: server, root and utils changes Github #329: config.py.in, spec, readme and makefile changes .gitignore | 2 + contrib/kimchi.spec.fedora.in | 5 +- contrib/kimchi.spec.suse.in | 3 + docs/README.md | 4 +- src/Makefile.am | 9 +- src/kimchi/config.py.in | 9 +- src/kimchi/root.py | 6 +- src/kimchi/server.py | 100 ++++++++++++--------- src/kimchi/utils.py | 42 ++++++++- src/kimchid.in | 202 ++++++++++++++++++++++++++++++++++++------ src/kimchid_server.in | 45 ++++++++++ src/nginx.conf.in | 69 +++++++++++++++ 12 files changed, 418 insertions(+), 78 deletions(-) create mode 100644 src/kimchid_server.in create mode 100644 src/nginx.conf.in -- 1.8.3.1
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> The kimchid script was revamped to launch 3 processes: - a cherrypy process that will act as frontend, running as non-root, providing the html templates; - another cherrypy process running as backend, with root privileges; - a reverse proxy (nginx) that will forward all requests made to the kimchi port to the frontend/backend. The same options that the previous kimchid scripts provided are available in this new version. Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- src/kimchid.in | 200 +++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 174 insertions(+), 26 deletions(-) diff --git a/src/kimchid.in b/src/kimchid.in index 8b63b57..51d4026 100644 --- a/src/kimchid.in +++ b/src/kimchid.in @@ -2,7 +2,7 @@ # # Project Kimchi # -# Copyright IBM, Corp. 2013 +# Copyright IBM, Corp. 2014 # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -16,18 +16,27 @@ # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA import logging import os +import pwd +import re +import signal +import subprocess import sys +import time sys.path.insert(1, '@pythondir@') +from string import Template + import kimchi.server import kimchi.config from kimchi.config import config, paths -from optparse import OptionParser +from kimchi.utils import parse_command_line_options + if not paths.installed: sys.path.append(paths.prefix) @@ -35,30 +44,169 @@ if not paths.installed: ACCESS_LOG = "kimchi-access.log" ERROR_LOG = "kimchi-error.log" + +def create_proxy_config(p_port, f_port, b_port, + p_ssl_port, f_ssl_port, b_ssl_port): + """Create nginx configuration file based on current ports config + + To allow flexibility in which port kimchi runs, we need the same + flexibility with the nginx proxy. This method creates the config + file dynamically by using 'nginx.conf.in' as a template, creating + the file 'nginx_kimchi.config' which will be used to launch the + proxy. + + Arguments: + p_port - proxy port + f_port - frontend port + b_port - backend port + p_ssl_port - proxy SSL port + f_ssl_port - frontend SSL port + b_ssl_port - backend SSL port + """ + + # get SSL paths to be used by the proxy + config_dir = paths.conf_dir + cert = '%s/kimchi-cert.pem' % config_dir + key = '%s/kimchi-key.pem' % config_dir + + with open(os.path.join(paths.conf_dir, "nginx.conf.in")) as myfile: + data = myfile.read() + + data = Template(data) + data = data.safe_substitute(proxy_port=p_port, + frontend_port=f_port, + backend_port=b_port, + proxy_ssl_port=p_ssl_port, + frontend_ssl_port=f_ssl_port, + backend_ssl_port=b_ssl_port, + cert_pem=cert, cert_key=key) + + config_file = open(os.getcwd() + "/nginx_kimchi.conf", "w") + config_file.write(data) + config_file.close() + + +def start_proxy(): + """Start nginx reverse proxy.""" + config_file = os.getcwd() + "/nginx_kimchi.conf" + cmd = ['nginx', '-c', config_file] + subprocess.call(cmd) + + +def terminate_proxy(): + """Stop nginx process.""" + term_proxy_cmd = ['nginx', '-s', 'stop'] + subprocess.call(term_proxy_cmd) + + def main(options): - host = config.get("server", "host") - port = config.get("server", "port") - ssl_port = config.get("server", "ssl_port") - runningEnv = config.get('server', 'environment') - logDir = config.get("logging", "log_dir") - logLevel = config.get("logging", "log_level") - - parser = OptionParser() - parser.add_option('--host', type="string", default=host, help="Hostname to listen on") - parser.add_option('--port', type="int", default=port, help="Port to listen on") - parser.add_option('--ssl-port', type="int", default=ssl_port, help="Enable a SSL server on the given port") - parser.add_option('--log-level', default=logLevel, help="Logging level") - parser.add_option('--access-log', default=os.path.join(logDir,ACCESS_LOG), help="Access log file") - parser.add_option('--error-log', default=os.path.join(logDir,ERROR_LOG), help="Error log file") - parser.add_option('--environment', default=runningEnv, help="Running environment of kimchi server") - parser.add_option('--test', action='store_true', help="Run server in mock model") - (options, args) = parser.parse_args() - - # Add non-option arguments - setattr(options, 'ssl_cert', config.get('server', 'ssl_cert')) - setattr(options, 'ssl_key', config.get('server', 'ssl_key')) - - kimchi.server.main(options) + # Script must run as root or with sudo. + if not os.geteuid() == 0: + sys.exit("\nMust be root to run this script. Exiting ...\n") + + # The following method is used when the process receives a + # SIGINT signal (CTRL+C). + def terminate_kimchi(signal, frame): + terminate_proxy() + sys.exit(0) + signal.signal(signal.SIGINT, terminate_kimchi) + + # User that will execute the frontend will own this + # process temporarily. First we need to detect if the script + # was run with SUDO or from the root directly. If + # run with SUDO, the user that fired the execution will + # take over (this user is registered at the 'SUDO_USER' env + # variable). Otherwise the user kimchi will take over. + # + # The frontend will **NOT** be run as root under any + # circunstance using this script. + if os.getenv("SUDO_USER") is not None: + temp_user = os.getenv("SUDO_USER") + else: + temp_user = 'kimchi' + + # Drop execution privileges temporarily + os.seteuid(pwd.getpwnam(temp_user).pw_uid) + + # Validate command line options before forwarding them + # to the cherrypy processes. + parse_command_line_options() + + # Copy the options (sys.argv[:1]) into 2 other lists. That + # way we can edit the copies with modified parameters to + # use in the cherrypy processes. + argv_frontend = list(options) + argv_backend = list(options) + + ssl_port_match = re.compile("^--ssl-port.*") + port_match = re.compile("^--port.*") + + port_index = None + ssl_port_index = None + + # Default ports + proxy_port = 8000 + proxy_ssl_port = 8001 + + # Search for the '--port=' and '--ssl-port=' arguments to verify + # if the user wants to run in a port other than the default. + for i, value in enumerate(options): + if re.search(port_match, value): + proxy_port = int(value.partition("=")[2]) + port_index = i + elif re.search(ssl_port_match, value): + proxy_ssl_port = int(value.partition("=")[2]) + ssl_port_index = i + + # The max value between port and proxy_port + # will be used to calculate the frontend and + # backend ports. + next_port = max(proxy_port, proxy_ssl_port) + 1 + frontend_port = next_port + next_port += 1 + frontend_ssl_port = next_port + next_port += 1 + backend_port = next_port + next_port += 1 + backend_ssl_port = next_port + + if port_index is not None: + # edit args to be used by the front/backend + argv_frontend[port_index] = "--port="+str(frontend_port) + argv_backend[port_index] = "--port="+str(backend_port) + else: + # add port info to the front/backend options + argv_frontend += ["--port="+str(frontend_port)] + argv_backend += ["--port="+str(backend_port)] + + if ssl_port_index is not None: + # edit args to be used by the front/backend + argv_frontend[ssl_port_index] = "--ssl-port="+str(frontend_ssl_port) + argv_backend[ssl_port_index] = "--ssl-port="+str(backend_ssl_port) + else: + # add ssl-port info to the front/backend options + argv_frontend += ["--ssl-port="+str(frontend_ssl_port)] + argv_backend += ["--ssl-port="+str(backend_ssl_port)] + + # launch frontend with edited options + command = os.getcwd() + '/src/kimchid_server' + cmd = [command] + argv_frontend + frontend = subprocess.Popen(cmd) + + # Retrieve root access before launching the backend. + os.seteuid(0) + + # Launch the cherrypy backend process. + cmd = [command] + argv_backend + ['--backend'] + ['--log_screen'] + backend = subprocess.Popen(cmd, stderr=subprocess.STDOUT) + + # Launch reverse proxy: create config file and start. + create_proxy_config(proxy_port, frontend_port, backend_port, + proxy_ssl_port, frontend_ssl_port, backend_ssl_port) + start_proxy() + + # suspend execution until interruption (SIGINT, SIGKILL) + signal.pause() if __name__ == '__main__': sys.exit(main(sys.argv[1:])) -- 1.8.3.1
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> src/kimchid_server is a launch script similar to the old version of kimchid. nginx.conf.in is a template has is being used by the new kimchid script to generate a customized proxy configuration. Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- src/kimchid_server.in | 45 +++++++++++++++++++++++++++++++++ src/nginx.conf.in | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+) create mode 100644 src/kimchid_server.in create mode 100644 src/nginx.conf.in diff --git a/src/kimchid_server.in b/src/kimchid_server.in new file mode 100644 index 0000000..e6a4827 --- /dev/null +++ b/src/kimchid_server.in @@ -0,0 +1,45 @@ +#!/usr/bin/python +# +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA + +import sys +sys.path.insert(1, '/usr/lib/python2.7/site-packages') + +import kimchi.server +import kimchi.config + +from kimchi.config import config, paths +from kimchi.utils import parse_command_line_options + +if not paths.installed: + sys.path.append(paths.prefix) + + +def main(options): + (options, args) = parse_command_line_options() + + # Add non-option arguments + setattr(options, 'ssl_cert', config.get('server', 'ssl_cert')) + setattr(options, 'ssl_key', config.get('server', 'ssl_key')) + + kimchi.server.main(options) + +if __name__ == '__main__': + sys.exit(main(sys.argv[1:])) diff --git a/src/nginx.conf.in b/src/nginx.conf.in new file mode 100644 index 0000000..3f958b1 --- /dev/null +++ b/src/nginx.conf.in @@ -0,0 +1,69 @@ +#!/usr/bin/python +# +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA + + +# This is a template file to be used to generate a nginx +# proxy config file at kimchid script. + +user nginx; +worker_processes 1; + +error_log /var/log/nginx/error.log; + +events { + worker_connections 1024; +} + + +http { + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + + sendfile on; + + server { + listen $proxy_port; + listen $proxy_ssl_port ssl; + ssl_certificate $cert_pem; + ssl_certificate_key $cert_key; + + location / { + proxy_pass http://localhost:$backend_port; + proxy_set_header Host $host; + } + location /config/ui/tabs.xml { + proxy_pass http://localhost:$frontend_port; + proxy_set_header Host $host; + } + location /help { + proxy_pass http://localhost:$frontend_port; + proxy_set_header Host $host; + } + location /favicon.ico { + proxy_pass http://localhost:$frontend_port; + proxy_set_header Host $host; + } + } +} -- 1.8.3.1
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> In server.py adjustments were need to consider that the cherrypy instance that will run as frontend will not have a model instance and logging. utils.py: the code that verifies and validates the commmand line arguments to launch kimchid was moved here to avoid code repetition, now that the function is called in more than one place. root.py: do not load API.json schema for the frontend instance. Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- src/kimchi/root.py | 6 ++-- src/kimchi/server.py | 100 ++++++++++++++++++++++++++++++--------------------- src/kimchi/utils.py | 42 +++++++++++++++++++++- 3 files changed, 104 insertions(+), 44 deletions(-) diff --git a/src/kimchi/root.py b/src/kimchi/root.py index 9bae34a..4ca28b9 100644 --- a/src/kimchi/root.py +++ b/src/kimchi/root.py @@ -92,8 +92,10 @@ class KimchiRoot(Root): self.default_page = 'kimchi-ui.html' for ident, node in sub_nodes.items(): setattr(self, ident, node(model)) - self.api_schema = json.load(open(os.path.join(paths.src_dir, - 'API.json'))) + # backend only + if model is not None: + self.api_schema = json.load(open(os.path.join(paths.src_dir, + 'API.json'))) self.paths = paths self.domain = 'kimchi' self.messages = messages diff --git a/src/kimchi/server.py b/src/kimchi/server.py index 0d02868..c9512fe 100644 --- a/src/kimchi/server.py +++ b/src/kimchi/server.py @@ -56,69 +56,87 @@ def set_no_cache(): hList = h +def ignore_cookie(): + cherrypy.response.cookie = cherrypy.request.cookie + + class Server(object): def __init__(self, options): - make_dirs = [ - os.path.dirname(os.path.abspath(options.access_log)), - os.path.dirname(os.path.abspath(options.error_log)), - os.path.dirname(os.path.abspath(config.get_object_store())), - os.path.abspath(config.get_screenshot_path()), - os.path.abspath(config.get_debugreports_path()), - os.path.abspath(config.get_distros_store()) - ] - for directory in make_dirs: - if not os.path.isdir(directory): - os.makedirs(directory) + if options.backend: + make_dirs = [ + os.path.dirname(os.path.abspath(options.access_log)), + os.path.dirname(os.path.abspath(options.error_log)), + os.path.dirname(os.path.abspath(config.get_object_store())), + os.path.abspath(config.get_screenshot_path()), + os.path.abspath(config.get_debugreports_path()), + os.path.abspath(config.get_distros_store()) + ] + for directory in make_dirs: + if not os.path.isdir(directory): + os.makedirs(directory) self.configObj = KimchiConfig() cherrypy.tools.nocache = cherrypy.Tool('on_end_resource', set_no_cache) cherrypy.tools.kimchiauth = cherrypy.Tool('before_handler', auth.kimchiauth) + + cherrypy.tools.ignorecookie = cherrypy.Tool('before_finalize', + ignore_cookie) cherrypy.server.socket_host = options.host cherrypy.server.socket_port = options.port - # SSL Server - try: - if options.ssl_port and options.ssl_port > 0: - self._init_ssl(options) - except AttributeError: - pass + # SSL Server operations - backend instance only + if options.backend: + try: + if options.ssl_port and options.ssl_port > 0: + self._init_ssl(options) + except AttributeError: + pass + cherrypy.log.access_file = options.access_log + cherrypy.log.error_file = options.error_log + + logLevel = LOGGING_LEVEL.get(options.log_level, logging.DEBUG) - cherrypy.log.screen = True - cherrypy.log.access_file = options.access_log - cherrypy.log.error_file = options.error_log + # Create handler to rotate access log file + h = logging.handlers.RotatingFileHandler(options.access_log, 'a', + 10000000, 1000) + h.setLevel(logLevel) + h.setFormatter(cherrypy._cplogging.logfmt) - logLevel = LOGGING_LEVEL.get(options.log_level, logging.DEBUG) - dev_env = options.environment != 'production' + # Add access log file to cherrypy configuration + cherrypy.log.access_log.addHandler(h) - # Create handler to rotate access log file - h = logging.handlers.RotatingFileHandler(options.access_log, 'a', - 10000000, 1000) - h.setLevel(logLevel) - h.setFormatter(cherrypy._cplogging.logfmt) + # Create handler to rotate error log file + h = logging.handlers.RotatingFileHandler(options.error_log, 'a', + 10000000, 1000) + h.setLevel(logLevel) + h.setFormatter(cherrypy._cplogging.logfmt) - # Add access log file to cherrypy configuration - cherrypy.log.access_log.addHandler(h) + # Add rotating log file to cherrypy configuration + cherrypy.log.error_log.addHandler(h) - # Create handler to rotate error log file - h = logging.handlers.RotatingFileHandler(options.error_log, 'a', - 10000000, 1000) - h.setLevel(logLevel) - h.setFormatter(cherrypy._cplogging.logfmt) + else: + cherrypy.log.access_file = None + cherrypy.log.error_file = None - # Add rotating log file to cherrypy configuration - cherrypy.log.error_log.addHandler(h) + cherrypy.log.screen = options.log_screen # Handling running mode + dev_env = options.environment != 'production' if not dev_env: cherrypy.config.update({'environment': 'production'}) - if hasattr(options, 'model'): - model_instance = options.model - elif options.test: - model_instance = mockmodel.get_mock_environment() + # model_instance won't be created in the frontend + # instance + if options.backend: + if hasattr(options, 'model'): + model_instance = options.model + elif options.test: + model_instance = mockmodel.get_mock_environment() + else: + model_instance = model.Model() else: - model_instance = model.Model() + model_instance = None if isinstance(model_instance, model.Model): vnc_ws_proxy = vnc.new_ws_proxy() diff --git a/src/kimchi/utils.py b/src/kimchi/utils.py index a30dcfe..3b2223c 100644 --- a/src/kimchi/utils.py +++ b/src/kimchi/utils.py @@ -20,6 +20,7 @@ import cherrypy import grp +import optparse import os import psutil import pwd @@ -28,12 +29,13 @@ import subprocess import traceback import urllib2 from multiprocessing import Process, Queue +from optparse import OptionParser from threading import Timer from cherrypy.lib.reprconf import Parser from kimchi.asynctask import AsyncTask -from kimchi.config import paths, PluginPaths +from kimchi.config import config, paths, PluginPaths from kimchi.exception import InvalidParameter, TimeoutExpired @@ -264,3 +266,41 @@ def probe_file_permission_as_user(file, user): p.start() p.join() return queue.get() + + +def parse_command_line_options(): + ACCESS_LOG = "kimchi-access.log" + ERROR_LOG = "kimchi-error.log" + + host = config.get("server", "host") + port = config.get("server", "port") + ssl_port = config.get("server", "ssl_port") + runningEnv = config.get('server', 'environment') + logDir = config.get("logging", "log_dir") + logLevel = config.get("logging", "log_level") + + parser = OptionParser() + parser.add_option('--host', type="string", default=host, + help="Hostname to listen on") + parser.add_option('--port', type="int", default=port, + help="Port to listen on") + parser.add_option('--ssl-port', type="int", default=ssl_port, + help="Enable a SSL server on the given port") + parser.add_option('--log-level', default=logLevel, + help="Logging level") + parser.add_option('--access-log', + default=os.path.join(logDir, ACCESS_LOG), + help="Access log file") + parser.add_option('--error-log', + default=os.path.join(logDir, ERROR_LOG), + help="Error log file") + parser.add_option('--environment', default=runningEnv, + help="Running environment of kimchi server") + parser.add_option('--test', action='store_true', + help="Run server in mock model") + parser.add_option('--backend', action='store_true', + help=optparse.SUPPRESS_HELP) + parser.add_option('--log_screen', action='store_true', + help=optparse.SUPPRESS_HELP) + + return parser.parse_args() -- 1.8.3.1
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> Makefile now generates the kimchid_server script. Added this new generated binary to .gitignore. config.py.in was edited to include a new cherrypy tool. Spec files were edited to include the new files. README file updated to include nginx dependency. Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> patch 4 - spec file Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> Readme changes --- .gitignore | 2 ++ contrib/kimchi.spec.fedora.in | 5 ++++- contrib/kimchi.spec.suse.in | 3 +++ docs/README.md | 4 ++-- src/Makefile.am | 9 ++++++++- src/kimchi/config.py.in | 9 ++++++--- 6 files changed, 25 insertions(+), 7 deletions(-) diff --git a/.gitignore b/.gitignore index 67878e2..ad6f51d 100644 --- a/.gitignore +++ b/.gitignore @@ -23,9 +23,11 @@ contrib/make-deb.sh *.min.css *.min.js *.gmo +nginx_kimchi.conf stamp-po kimchi-*.tar.gz src/kimchid +src/kimchid_server src/kimchi.conf src/kimchi/config.py tests/run_tests.sh diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index bf80104..aa55491 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -28,6 +28,7 @@ Requires: sos Requires: python-ipaddr Requires: python-lxml Requires: nfs-utils +Requires: nginx Requires: iscsi-initiator-utils BuildRequires: libxslt BuildRequires: libxml2-python @@ -136,6 +137,7 @@ rm -rf $RPM_BUILD_ROOT %files %attr(-,root,root) %{_bindir}/kimchid +%{_bindir}/kimchid_server %{python_sitelib}/kimchi/*.py* %{python_sitelib}/kimchi/control/*.py* %{python_sitelib}/kimchi/control/vm/*.py* @@ -162,7 +164,7 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/ui/js/novnc/*.js %{_datadir}/kimchi/ui/js/spice/*.js %{_datadir}/kimchi/ui/js/novnc/web-socket-js/WebSocketMain.swf -%{_datadir}/kimchi/ui/js/novnc/web-socket-js/swfobject.js +%{_datadir}/kimchi/ui/js/novnc/web-socket-js/swfobject.j %{_datadir}/kimchi/ui/js/novnc/web-socket-js/web_socket.js %{_datadir}/kimchi/ui/libs/jquery-ui-i18n.min.js %{_datadir}/kimchi/ui/libs/jquery-ui.min.js @@ -175,6 +177,7 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/ui/pages/help/*.html %{_datadir}/kimchi/ui/pages/tabs/*.html.tmpl %{_sysconfdir}/kimchi/kimchi.conf +%{_sysconfdir}/kimchi/nginx.conf.in %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in index cba0899..debddad 100644 --- a/contrib/kimchi.spec.suse.in +++ b/contrib/kimchi.spec.suse.in @@ -24,6 +24,7 @@ Requires: python-ipaddr Requires: python-lxml Requires: python-xml Requires: nfs-client +Requires: nginx Requires: open-iscsi BuildRequires: libxslt-tools BuildRequires: python-libxml2 @@ -62,6 +63,7 @@ rm -rf $RPM_BUILD_ROOT %files %attr(-,root,root) %{_bindir}/kimchid +%{_bindir}/kimchid_server %{python_sitelib}/kimchi/*.py* %{python_sitelib}/kimchi/control/*.py* %{python_sitelib}/kimchi/control/vm/*.py* @@ -101,6 +103,7 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/ui/pages/help/*.html %{_datadir}/kimchi/ui/pages/tabs/*.html.tmpl %{_sysconfdir}/kimchi/kimchi.conf +%{_sysconfdir}/kimchi/nginx.conf.in %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json diff --git a/docs/README.md b/docs/README.md index 8b8b181..63ac760 100644 --- a/docs/README.md +++ b/docs/README.md @@ -53,7 +53,7 @@ Install Dependencies PyPAM m2crypto python-jsonschema rpm-build \ qemu-kvm python-psutil python-ethtool sos \ python-ipaddr python-lxml nfs-utils \ - iscsi-initiator-utils libxslt pyparted + iscsi-initiator-utils libxslt pyparted nginx # If using RHEL6, install the following additional packages: $ sudo yum install python-unittest2 python-ordereddict # Restart libvirt to allow configuration changes to take effect @@ -75,7 +75,7 @@ for more information on how to configure your system to access this repository. python-pam python-m2crypto python-jsonschema \ qemu-kvm libtool python-psutil python-ethtool \ sosreport python-ipaddr python-lxml nfs-common \ - open-iscsi lvm2 xsltproc python-parted + open-iscsi lvm2 xsltproc python-parted nginx Packages version requirement: python-jsonschema >= 1.3.0 diff --git a/src/Makefile.am b/src/Makefile.am index 2005f7c..7306558 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -20,11 +20,14 @@ SUBDIRS = kimchi distros.d EXTRA_DIST = kimchid.in \ + kimchid_server.in \ kimchi.conf.in \ firewalld.xml \ $(NULL) -bin_SCRIPTS = kimchid +bin_SCRIPTS = kimchid \ + kimchid_server \ + $(NULL) confdir = $(sysconfdir)/kimchi dist_conf_DATA = kimchi.conf @@ -42,6 +45,10 @@ kimchid: kimchid.in Makefile $(do_substitution) < $(srcdir)/kimchid.in > kimchid chmod +x kimchid +kimchid_server: kimchid_server.in Makefile + $(do_substitution) < $(srcdir)/kimchid_server.in > kimchid_server + chmod +x kimchid_server + kimchi.conf: kimchi.conf.in Makefile $(do_substitution) < kimchi.conf.in > kimchi.conf diff --git a/src/kimchi/config.py.in b/src/kimchi/config.py.in index d15a6b5..f90b2f3 100644 --- a/src/kimchi/config.py.in +++ b/src/kimchi/config.py.in @@ -187,16 +187,19 @@ class KimchiConfig(dict): 'tools.staticfile.on': True, 'tools.staticfile.filename': '%s/config/ui/tabs.xml' % paths.prefix, - 'tools.nocache.on': True + 'tools.nocache.on': True, + 'tools.ignorecookie.on': True }, '/favicon.ico': { 'tools.staticfile.on': True, - 'tools.staticfile.filename': '%s/images/logo.ico' % paths.ui_dir + 'tools.staticfile.filename': '%s/images/logo.ico' % paths.ui_dir, + 'tools.ignorecookie.on': True }, '/help': { 'tools.staticdir.on': True, 'tools.staticdir.dir': '%s/ui/pages/help' % paths.prefix, - 'tools.nocache.on': False + 'tools.nocache.on': False, + 'tools.ignorecookie.on': True } } -- 1.8.3.1
Reviewed-By: Ramon Medeiros <ramonn@br.ibm.com>
on 2014/04/09 04:50, Daniel Barboza wrote:
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
Changes: *v2: - adressed Ramon's comment about the python path in kimchid.in
This patch series revamps the launch architecture to allow kimchi to not run as root while being exposed at an open http/https port.
The solution adopted is using a reverse http proxy (nginx) to make the 'bridge' between two distinct cherrypy processes, one running as frontend as a regular user and another running as backend, as root. The communication with the outside will be done through nginx, running as a regular user too.
The changes were heavy in the kimchid script, but the startup and usage options still the same. User-wise, there shouldn't be any functional change in the way kimchi works after applying this change.
Refer to https://github.com/kimchi-project/kimchi/issues/329 for further information in all the other approaches considered and why they didn't work out.
Excellent investigation work and nice patch! Since the nginx is already run in non-root mode and forward the request to Kimchi, it seems there is no need to separate kimchi-backend and kimchi-frontend. The solution in this patch series does not improve security but makes the internal of kimchi unnecessarily complicated. This is because nginx forwards requests directly to kimchi-backend and kimchi-frontend. The kimchi-backend and kimchi-frontend are in the same level. To truly make it a secure architecture, the kimchi-backend should sit behind kimchi-frontend, which means the following: nginx -> kimchi-frontend -> kimchi-backend(root). However if we had this nginx -> kimchi-frontend -> kimchi-backend(root), there would be no need to use nginx because kimchi-frontend is non-root already. A simpler solution might be that we can just have one kimchi process listening on localhost, and nginx on 0.0.0.0:8000/8001 and forward all the requests to kimchi. As regard to your RPC investigations, I'm fully agree with you on the opinion that objects are complex and not all objects are serialize. Consider the complexity and variance of third-party libraries, it's impossible to have an RPC solution that work seamless and transparently between a so-called kimchi-frontend(root) and kimchi-backend. On the other hand. There is a way that RPC should work if we don't have it run transparently. Suppose all models run in kimchi-backend(root), and it exposes a RPC interface explicitly via JSON RPC, D-BUS or whatever. And the objects passed to and returned from this RPC interfaces should be only of the simple kind such as list, string, integer, dict. Then all the controllers run in kimchi-frontend(non-root) and it listens on 0.0.0.0:8000/8001. The controller objects should make explicit RPC calls to the kimcih-backend(root). From my experience, seamless remote objects style of RPC is always a dead end, explicit style of RPC always works. In all, the RPC way may need more work, but it make perfect layered separation between the models and controllers. If we don't have a layered separation between the kimchi-frontend and kimchi-backend, there is no need to start two kimchi process, just one is enough, then have the nginx run in non-root and forward the requests.
Daniel Henrique Barboza (4): Github #329: kimchid script changes Github #329: new launch script and proxy template Github #329: server, root and utils changes Github #329: config.py.in, spec, readme and makefile changes
.gitignore | 2 + contrib/kimchi.spec.fedora.in | 5 +- contrib/kimchi.spec.suse.in | 3 + docs/README.md | 4 +- src/Makefile.am | 9 +- src/kimchi/config.py.in | 9 +- src/kimchi/root.py | 6 +- src/kimchi/server.py | 100 ++++++++++++--------- src/kimchi/utils.py | 42 ++++++++- src/kimchid.in | 202 ++++++++++++++++++++++++++++++++++++------ src/kimchid_server.in | 45 ++++++++++ src/nginx.conf.in | 69 +++++++++++++++ 12 files changed, 418 insertions(+), 78 deletions(-) create mode 100644 src/kimchid_server.in create mode 100644 src/nginx.conf.in
-- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshzhou@linux.vnet.ibm.com Telephone: 86-10-82454397
On 04/10/2014 01:03 AM, Zhou Zheng Sheng wrote:
on 2014/04/09 04:50, Daniel Barboza wrote:
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
Changes: *v2: - adressed Ramon's comment about the python path in kimchid.in
This patch series revamps the launch architecture to allow kimchi to not run as root while being exposed at an open http/https port.
The solution adopted is using a reverse http proxy (nginx) to make the 'bridge' between two distinct cherrypy processes, one running as frontend as a regular user and another running as backend, as root. The communication with the outside will be done through nginx, running as a regular user too.
The changes were heavy in the kimchid script, but the startup and usage options still the same. User-wise, there shouldn't be any functional change in the way kimchi works after applying this change.
Refer to https://github.com/kimchi-project/kimchi/issues/329 for further information in all the other approaches considered and why they didn't work out.
Excellent investigation work and nice patch!
Since the nginx is already run in non-root mode and forward the request to Kimchi, it seems there is no need to separate kimchi-backend and kimchi-frontend. The solution in this patch series does not improve security but makes the internal of kimchi unnecessarily complicated. This is because nginx forwards requests directly to kimchi-backend and kimchi-frontend. The kimchi-backend and kimchi-frontend are in the same level. To truly make it a secure architecture, the kimchi-backend should sit behind kimchi-frontend, which means the following: nginx -> kimchi-frontend -> kimchi-backend(root). However if we had this nginx -> kimchi-frontend -> kimchi-backend(root), there would be no need to use nginx because kimchi-frontend is non-root already.
A simpler solution might be that we can just have one kimchi process listening on localhost, and nginx on 0.0.0.0:8000/8001 and forward all the requests to kimchi.
I agree that using nginx on top of kimchi is enough to solve the issue. The separation of frontend and backend makes sense in the design point of view and it was something I had to do in my other attempts, but it wouldn't be necessary to solve this particular bug.
As regard to your RPC investigations, I'm fully agree with you on the opinion that objects are complex and not all objects are serialize. Consider the complexity and variance of third-party libraries, it's impossible to have an RPC solution that work seamless and transparently between a so-called kimchi-frontend(root) and kimchi-backend.
On the other hand. There is a way that RPC should work if we don't have it run transparently. Suppose all models run in kimchi-backend(root), and it exposes a RPC interface explicitly via JSON RPC, D-BUS or whatever. And the objects passed to and returned from this RPC interfaces should be only of the simple kind such as list, string, integer, dict. Then all the controllers run in kimchi-frontend(non-root) and it listens on 0.0.0.0:8000/8001. The controller objects should make explicit RPC calls to the kimcih-backend(root). From my experience, seamless remote objects style of RPC is always a dead end, explicit style of RPC always works.
In all, the RPC way may need more work, but it make perfect layered separation between the models and controllers. If we don't have a layered separation between the kimchi-frontend and kimchi-backend, there is no need to start two kimchi process, just one is enough, then have the nginx run in non-root and forward the requests.
I really liked the RPC idea. I still do. But, as you said, it needs more work than it looks. I've made several attempts, some of the were successful (ex: running run_command through RPC as root) but some parts of the code were too complex to put in RPC without reworking. Perhaps we can solve this bug by using the nginx approach (I am ok in firing only one cherrypy process as root) and then work gradually in the RPC approach? Once we have a full RPC framework we can ditch nginx and run kimchi as regular user, firing RPC calls to remote objects that run as root. Sounds good?
Daniel Henrique Barboza (4): Github #329: kimchid script changes Github #329: new launch script and proxy template Github #329: server, root and utils changes Github #329: config.py.in, spec, readme and makefile changes
.gitignore | 2 + contrib/kimchi.spec.fedora.in | 5 +- contrib/kimchi.spec.suse.in | 3 + docs/README.md | 4 +- src/Makefile.am | 9 +- src/kimchi/config.py.in | 9 +- src/kimchi/root.py | 6 +- src/kimchi/server.py | 100 ++++++++++++--------- src/kimchi/utils.py | 42 ++++++++- src/kimchid.in | 202 ++++++++++++++++++++++++++++++++++++------ src/kimchid_server.in | 45 ++++++++++ src/nginx.conf.in | 69 +++++++++++++++ 12 files changed, 418 insertions(+), 78 deletions(-) create mode 100644 src/kimchid_server.in create mode 100644 src/nginx.conf.in
on 2014/04/10 21:32, Daniel H Barboza wrote:
On 04/10/2014 01:03 AM, Zhou Zheng Sheng wrote:
on 2014/04/09 04:50, Daniel Barboza wrote:
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
Changes: *v2: - adressed Ramon's comment about the python path in kimchid.in
This patch series revamps the launch architecture to allow kimchi to not run as root while being exposed at an open http/https port.
The solution adopted is using a reverse http proxy (nginx) to make the 'bridge' between two distinct cherrypy processes, one running as frontend as a regular user and another running as backend, as root. The communication with the outside will be done through nginx, running as a regular user too.
The changes were heavy in the kimchid script, but the startup and usage options still the same. User-wise, there shouldn't be any functional change in the way kimchi works after applying this change.
Refer to https://github.com/kimchi-project/kimchi/issues/329 for further information in all the other approaches considered and why they didn't work out.
Excellent investigation work and nice patch!
Since the nginx is already run in non-root mode and forward the request to Kimchi, it seems there is no need to separate kimchi-backend and kimchi-frontend. The solution in this patch series does not improve security but makes the internal of kimchi unnecessarily complicated. This is because nginx forwards requests directly to kimchi-backend and kimchi-frontend. The kimchi-backend and kimchi-frontend are in the same level. To truly make it a secure architecture, the kimchi-backend should sit behind kimchi-frontend, which means the following: nginx -> kimchi-frontend -> kimchi-backend(root). However if we had this nginx -> kimchi-frontend -> kimchi-backend(root), there would be no need to use nginx because kimchi-frontend is non-root already.
A simpler solution might be that we can just have one kimchi process listening on localhost, and nginx on 0.0.0.0:8000/8001 and forward all the requests to kimchi.
I agree that using nginx on top of kimchi is enough to solve the issue. The separation of frontend and backend makes sense in the design point of view and it was something I had to do in my other attempts, but it wouldn't be necessary to solve this particular bug.
As regard to your RPC investigations, I'm fully agree with you on the opinion that objects are complex and not all objects are serialize. Consider the complexity and variance of third-party libraries, it's impossible to have an RPC solution that work seamless and transparently between a so-called kimchi-frontend(root) and kimchi-backend.
On the other hand. There is a way that RPC should work if we don't have it run transparently. Suppose all models run in kimchi-backend(root), and it exposes a RPC interface explicitly via JSON RPC, D-BUS or whatever. And the objects passed to and returned from this RPC interfaces should be only of the simple kind such as list, string, integer, dict. Then all the controllers run in kimchi-frontend(non-root) and it listens on 0.0.0.0:8000/8001. The controller objects should make explicit RPC calls to the kimcih-backend(root). From my experience, seamless remote objects style of RPC is always a dead end, explicit style of RPC always works.
In all, the RPC way may need more work, but it make perfect layered separation between the models and controllers. If we don't have a layered separation between the kimchi-frontend and kimchi-backend, there is no need to start two kimchi process, just one is enough, then have the nginx run in non-root and forward the requests.
I really liked the RPC idea. I still do. But, as you said, it needs more work than it looks. I've made several attempts, some of the were successful (ex: running run_command through RPC as root) but some parts of the code were too complex to put in RPC without reworking.
Perhaps we can solve this bug by using the nginx approach (I am ok in firing only one cherrypy process as root) and then work gradually in the RPC approach? Once we have a full RPC framework we can ditch nginx and run kimchi as regular user, firing RPC calls to remote objects that run as root. Sounds good?
Yes, I agree. It looks among the most practical ways to solve the non-root problem.
Daniel Henrique Barboza (4): Github #329: kimchid script changes Github #329: new launch script and proxy template Github #329: server, root and utils changes Github #329: config.py.in, spec, readme and makefile changes
.gitignore | 2 + contrib/kimchi.spec.fedora.in | 5 +- contrib/kimchi.spec.suse.in | 3 + docs/README.md | 4 +- src/Makefile.am | 9 +- src/kimchi/config.py.in | 9 +- src/kimchi/root.py | 6 +- src/kimchi/server.py | 100 ++++++++++++--------- src/kimchi/utils.py | 42 ++++++++- src/kimchid.in | 202 ++++++++++++++++++++++++++++++++++++------ src/kimchid_server.in | 45 ++++++++++ src/nginx.conf.in | 69 +++++++++++++++ 12 files changed, 418 insertions(+), 78 deletions(-) create mode 100644 src/kimchid_server.in create mode 100644 src/nginx.conf.in
-- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshzhou@linux.vnet.ibm.com Telephone: 86-10-82454397
participants (4)
-
Daniel Barboza -
Daniel H Barboza
-
Ramon Medeiros -
Zhou Zheng Sheng