2:29 p.m.
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
New in v6:
- Moved the SSL logic from server.py to proxy.py
- server.py is now responsible to launch the proxy
- removed references of 'ssl_port'
New in v5:
- fixes to VNC connection
- fixes to unit tests
- obscuring kimchid port by making it localhost only
- launhing nginx proxy in unit tests
New in v4:
- added a new module with all nginx-related methods
- added DEBIAN entries
- other fixes proposed by Aline
After reading the comments from Zhou Zheng Sheng, I simplified
the work I did in version 2 to run only one cherrypy process
instead of two processes, one for frontend and another for the
backend. Nginx is still being used as a reverse proxy to allow
kimchid to run as root, but not being exposed to the outside.
As Zhou mentioned, it is to little avail to run frontend and
backend separately if the exposed port is running by the
reverse proxy anyway. He mentioned the RPC approach as a best
long-term approach, which I agree. We can solve this issue right
now and the work in a more suitable solution, such as RPC, and
then ditch nginx.
Daniel Henrique Barboza (4):
Github #329: Proxy module and template file
Github #329: Kimchid, config.py.in and server.py changes
Github #329: changes in mockmodel, model/config and tests
Github #329: .gitignore, spec, control.in and readme
.gitignore | 1 +
Makefile.am | 2 +
contrib/DEBIAN/control.in | 3 +-
contrib/kimchi.spec.fedora.in | 2 +
contrib/kimchi.spec.suse.in | 2 +
docs/README.md | 4 +-
src/Makefile.am | 3 +-
src/kimchi/config.py.in | 5 +-
src/kimchi/mockmodel.py | 2 +-
src/kimchi/model/config.py | 2 +-
src/kimchi/proxy.py | 107 ++++++++++++++++++++++++++++++++++++++++++
src/kimchi/server.py | 44 +++++------------
src/kimchid.in | 44 ++++++++++++-----
src/nginx.conf.in | 55 ++++++++++++++++++++++
tests/test_rest.py | 8 ++--
tests/utils.py | 10 +++-
16 files changed, 238 insertions(+), 56 deletions(-)
create mode 100644 src/kimchi/proxy.py
create mode 100644 src/nginx.conf.in
--
1.8.3.1
2:29 p.m.
New subject: [PATCH v6 1/4] Github #329: Proxy module and template file
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
The file src/kimchi/proxy.py is a module that contains all Nginx
related functions - start proxy, terminate proxy and create
proxy config.
src/nginx.conf.in is a template file that is used by the proxy
module to generate a customized proxy configuration.
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
---
src/kimchi/proxy.py | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++
src/nginx.conf.in | 55 +++++++++++++++++++++++++++
2 files changed, 162 insertions(+)
create mode 100644 src/kimchi/proxy.py
create mode 100644 src/nginx.conf.in
diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py
new file mode 100644
index 0000000..23944ea
--- /dev/null
+++ b/src/kimchi/proxy.py
@@ -0,0 +1,107 @@
+#!/usr/bin/python
+#
+# Project Kimchi
+#
+# Copyright IBM, Corp. 2014
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+# MA 02110-1301 USA
+
+# This module contains functions that the manipulate
+# and configure the Nginx proxy.
+
+import os
+import pwd
+import sslcert
+import subprocess
+from string import Template
+
+import kimchi.config
+from kimchi.config import config, paths
+
+
+def _create_proxy_config(p_port, k_port, p_ssl_port, cert, key):
+ """Create nginx configuration file based on current ports config
+
+ To allow flexibility in which port kimchi runs, we need the same
+ flexibility with the nginx proxy. This method creates the config
+ file dynamically by using 'nginx.conf.in' as a template, creating
+ the file 'nginx_kimchi.config' which will be used to launch the
+ proxy.
+
+ Arguments:
+ p_port - proxy port
+ k_port - kimchid port
+ p_ssl_port - proxy SSL port
+ cert - cert file specified by user config
+ key - key file specified by user config
+ """
+
+ # User that will run the worker process of the proxy. Fedora,
+ # RHEL and Suse creates an user called 'nginx' when installing
+ # the proxy. Ubuntu creates an user 'www-data' for it.
+ user_proxy = 'nginx'
+ try:
+ pwd.getpwnam(user_proxy)
+ except KeyError:
+ user_proxy = 'www-data'
+
+ # No certificates specified by the user
+ if not cert or not key:
+ config_dir = paths.conf_dir
+ cert = '%s/kimchi-cert.pem' % config_dir
+ key = '%s/kimchi-key.pem' % config_dir
+ # create cert files if they don't exist
+ if not os.path.exists(cert) or not os.path.exists(key):
+ ssl_gen = sslcert.SSLCert()
+ with open(cert, "w") as f:
+ f.write(ssl_gen.cert_pem())
+ with open(key, "w") as f:
+ f.write(ssl_gen.key_pem())
+
+ # Read template file and create a new config file
+ # with the specified parameters.
+ with open(os.path.join(config_dir, "nginx.conf.in")) as template:
+ data = template.read()
+ data = Template(data)
+ data = data.safe_substitute(user=user_proxy,
+ proxy_port=p_port,
+ kimchid_port=k_port,
+ proxy_ssl_port=p_ssl_port,
+ cert_pem=cert, cert_key=key)
+
+ # Write file to be used for nginx.
+ config_file = open(os.path.join(config_dir, "nginx_kimchi.conf"),
"w")
+ config_file.write(data)
+ config_file.close()
+
+
+def start_proxy(options):
+ """Start nginx reverse proxy."""
+ _create_proxy_config(options.proxy_port,
+ options.port,
+ options.proxy_ssl_port,
+ options.ssl_cert,
+ options.ssl_key)
+ config_dir = paths.conf_dir
+ config_file = "%s/nginx_kimchi.conf" % config_dir
+ cmd = ['nginx', '-c', config_file]
+ subprocess.call(cmd)
+
+
+def terminate_proxy():
+ """Stop nginx process."""
+ term_proxy_cmd = ['nginx', '-s', 'stop']
+ subprocess.call(term_proxy_cmd)
diff --git a/src/nginx.conf.in b/src/nginx.conf.in
new file mode 100644
index 0000000..967b46b
--- /dev/null
+++ b/src/nginx.conf.in
@@ -0,0 +1,55 @@
+# Project Kimchi
+#
+# Copyright IBM, Corp. 2014
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+# 02110-1301 USA
+
+
+# This is a template file to be used to generate a nginx
+# proxy config file at kimchid script.
+
+user $user;
+worker_processes 1;
+
+error_log /var/log/nginx/error.log;
+
+events {
+ worker_connections 1024;
+}
+
+
+http {
+
+ log_format main '$remote_addr - $remote_user [$time_local] "$request"
'
+ '$status $body_bytes_sent "$http_referer" '
+ '"$http_user_agent"
"$http_x_forwarded_for"';
+
+ access_log /var/log/nginx/access.log main;
+
+ sendfile on;
+
+ server {
+ listen $proxy_port;
+ listen $proxy_ssl_port ssl;
+ ssl_certificate $cert_pem;
+ ssl_certificate_key $cert_key;
+
+ location / {
+ proxy_pass http://localhost:$kimchid_port;
+ proxy_set_header Host $host;
+ }
+ }
+}
--
1.8.3.1
2:29 p.m.
New subject: [PATCH v6 2/4] Github #329: Kimchid, config.py.in and server.py changes
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
Two new config options were added in config.py.in - proxy_port
and proxy_ssl_port - to allow more control in the ports that
the proxy will bind. The defaults proxy ports are 8000 and
8001 (ssl) and default kimchid port is 8010. The default proxy
ports were chosen to avoid editing the existing firewall
configuration (now that nginx is process being exposed).
Kimchid creates the proxy configuration according to user-defined
parameters.
server.py registers the terminate_proxy() method to a cherrypy
engine that fires when the server is shut down. The SSL logic
was moved to the proxy module at src/kimchi/proxy.py.
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
---
src/kimchi/config.py.in | 5 +++--
src/kimchi/server.py | 44 ++++++++++++--------------------------------
src/kimchid.in | 44 +++++++++++++++++++++++++++++++++-----------
3 files changed, 48 insertions(+), 45 deletions(-)
diff --git a/src/kimchi/config.py.in b/src/kimchi/config.py.in
index bddcb5c..f8a645a 100644
--- a/src/kimchi/config.py.in
+++ b/src/kimchi/config.py.in
@@ -231,8 +231,9 @@ def _get_config():
config = SafeConfigParser()
config.add_section("server")
config.set("server", "host", "0.0.0.0")
- config.set("server", "port", "8000")
- config.set("server", "ssl_port", "8001")
+ config.set("server", "port", "8010")
+ config.set("server", "proxy_port", "8000")
+ config.set("server", "proxy_ssl_port", "8001")
config.set("server", "ssl_cert", "")
config.set("server", "ssl_key", "")
config.set("server", "environment", "development")
diff --git a/src/kimchi/server.py b/src/kimchi/server.py
index 0d02868..f3c192e 100644
--- a/src/kimchi/server.py
+++ b/src/kimchi/server.py
@@ -31,6 +31,7 @@ from kimchi import mockmodel
from kimchi import vnc
from kimchi.config import paths, KimchiConfig, PluginConfig
from kimchi.control import sub_nodes
+from kimchi.proxy import start_proxy, terminate_proxy
from kimchi.root import KimchiRoot
from kimchi.utils import get_enabled_plugins, import_class
@@ -58,6 +59,9 @@ def set_no_cache():
class Server(object):
def __init__(self, options):
+ # Launch reverse proxy
+ start_proxy(options)
+
make_dirs = [
os.path.dirname(os.path.abspath(options.access_log)),
os.path.dirname(os.path.abspath(options.error_log)),
@@ -74,15 +78,12 @@ class Server(object):
cherrypy.tools.nocache = cherrypy.Tool('on_end_resource', set_no_cache)
cherrypy.tools.kimchiauth = cherrypy.Tool('before_handler',
auth.kimchiauth)
- cherrypy.server.socket_host = options.host
+ # Setting host to 127.0.0.1. This makes kimchi runs
+ # as a localhost app, inaccessible to the outside
+ # directly. You must go through the proxy.
+ cherrypy.server.socket_host = '127.0.0.1'
cherrypy.server.socket_port = options.port
-
- # SSL Server
- try:
- if options.ssl_port and options.ssl_port > 0:
- self._init_ssl(options)
- except AttributeError:
- pass
+ cherrypy.config.nginx_port = options.proxy_port
cherrypy.log.screen = True
cherrypy.log.access_file = options.access_log
@@ -137,6 +138,9 @@ class Server(object):
config=self.configObj)
self._load_plugins()
+ # Terminate proxy when cherrypy server is terminated
+ cherrypy.engine.subscribe('exit', terminate_proxy)
+
cherrypy.lib.sessions.init()
def _load_plugins(self):
@@ -160,30 +164,6 @@ class Server(object):
continue
cherrypy.tree.mount(plugin_app, script_name, plugin_config)
- def _init_ssl(self, options):
- ssl_server = cherrypy._cpserver.Server()
- ssl_server.socket_port = options.ssl_port
- ssl_server._socket_host = options.host
- ssl_server.ssl_module = 'builtin'
-
- cert = options.ssl_cert
- key = options.ssl_key
- if not cert or not key:
- config_dir = paths.conf_dir
- cert = '%s/kimchi-cert.pem' % config_dir
- key = '%s/kimchi-key.pem' % config_dir
-
- if not os.path.exists(cert) or not os.path.exists(key):
- ssl_gen = sslcert.SSLCert()
- with open(cert, "w") as f:
- f.write(ssl_gen.cert_pem())
- with open(key, "w") as f:
- f.write(ssl_gen.key_pem())
-
- ssl_server.ssl_certificate = cert
- ssl_server.ssl_private_key = key
- ssl_server.subscribe()
-
def start(self):
cherrypy.engine.start()
cherrypy.engine.block()
diff --git a/src/kimchid.in b/src/kimchid.in
index 8b63b57..c64ba4a 100644
--- a/src/kimchid.in
+++ b/src/kimchid.in
@@ -16,18 +16,20 @@
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+# MA 02110-1301 USA
import logging
import os
import sys
sys.path.insert(1, '@pythondir@')
+from optparse import OptionParser
+
import kimchi.server
import kimchi.config
from kimchi.config import config, paths
-from optparse import OptionParser
if not paths.installed:
sys.path.append(paths.prefix)
@@ -35,23 +37,43 @@ if not paths.installed:
ACCESS_LOG = "kimchi-access.log"
ERROR_LOG = "kimchi-error.log"
+
def main(options):
+ # Script must run as root or with sudo.
+ if not os.geteuid() == 0:
+ sys.exit("\nMust be root to run this script. Exiting ...\n")
+
host = config.get("server", "host")
port = config.get("server", "port")
- ssl_port = config.get("server", "ssl_port")
+ proxy_port = config.get("server", "proxy_port")
+ proxy_ssl_port = config.get("server", "proxy_ssl_port")
runningEnv = config.get('server', 'environment')
logDir = config.get("logging", "log_dir")
logLevel = config.get("logging", "log_level")
parser = OptionParser()
- parser.add_option('--host', type="string", default=host,
help="Hostname to listen on")
- parser.add_option('--port', type="int", default=port,
help="Port to listen on")
- parser.add_option('--ssl-port', type="int", default=ssl_port,
help="Enable a SSL server on the given port")
- parser.add_option('--log-level', default=logLevel, help="Logging
level")
- parser.add_option('--access-log', default=os.path.join(logDir,ACCESS_LOG),
help="Access log file")
- parser.add_option('--error-log', default=os.path.join(logDir,ERROR_LOG),
help="Error log file")
- parser.add_option('--environment', default=runningEnv, help="Running
environment of kimchi server")
- parser.add_option('--test', action='store_true', help="Run
server in mock model")
+ parser.add_option('--host', type="string", default=host,
+ help="Hostname to listen on")
+ parser.add_option('--port', type="int", default=port,
+ help="Kimchid process listen port (default %s)" % port)
+ parser.add_option('--proxy-port', type="int", default=proxy_port,
+ help="Proxy port to listen on (default %s)" %
+ proxy_port)
+ parser.add_option('--proxy-ssl-port', type="int",
default=proxy_ssl_port,
+ help="Proxy port to enable SSL (default %s)" %
+ proxy_ssl_port)
+ parser.add_option('--log-level', default=logLevel,
+ help="Logging level")
+ parser.add_option('--access-log',
+ default=os.path.join(logDir, ACCESS_LOG),
+ help="Access log file")
+ parser.add_option('--error-log',
+ default=os.path.join(logDir, ERROR_LOG),
+ help="Error log file")
+ parser.add_option('--environment', default=runningEnv,
+ help="Running environment of kimchi server")
+ parser.add_option('--test', action='store_true',
+ help="Run server in mock model")
(options, args) = parser.parse_args()
# Add non-option arguments
--
1.8.3.1
2:29 p.m.
New subject: [PATCH v6 3/4] Github #329: changes in mockmodel, model/config and tests
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
Changed mockmodel and model/config.py to point to the proxy
port instead of the kimchid port, which is not exposed
anymore.
tests/utils was changed to launch a proxy instance
for each run_server call.
tests/test_rest.py changed to reflect the changes in model
and mockmodel.
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
---
src/kimchi/mockmodel.py | 2 +-
src/kimchi/model/config.py | 2 +-
tests/test_rest.py | 8 +++++---
tests/utils.py | 10 ++++++++--
4 files changed, 15 insertions(+), 7 deletions(-)
diff --git a/src/kimchi/mockmodel.py b/src/kimchi/mockmodel.py
index 8c7d7bb..82f75e3 100644
--- a/src/kimchi/mockmodel.py
+++ b/src/kimchi/mockmodel.py
@@ -832,7 +832,7 @@ class MockModel(object):
return disks.get_partition_details(name)
def config_lookup(self, name):
- return {'http_port': cherrypy.server.socket_port,
+ return {'http_port': cherrypy.config.nginx_port,
'display_proxy_port':
kconfig.get('display', 'display_proxy_port'),
'version': config.get_version()}
diff --git a/src/kimchi/model/config.py b/src/kimchi/model/config.py
index 7081373..c9e3e9d 100644
--- a/src/kimchi/model/config.py
+++ b/src/kimchi/model/config.py
@@ -40,7 +40,7 @@ class ConfigModel(object):
def lookup(self, name):
proxy_port = kconfig.get('display', 'display_proxy_port')
- return {'http_port': cherrypy.server.socket_port,
+ return {'http_port': cherrypy.config.nginx_port,
'display_proxy_port': proxy_port,
'version': get_version()}
diff --git a/tests/test_rest.py b/tests/test_rest.py
index a40ba93..ddb0aa9 100644
--- a/tests/test_rest.py
+++ b/tests/test_rest.py
@@ -49,14 +49,16 @@ ssl_port = None
def setUpModule():
- global test_server, model, host, port, ssl_port
+ global test_server, model, host, port, proxy_port, ssl_port
patch_auth()
model = kimchi.mockmodel.MockModel('/tmp/obj-store-test')
host = '127.0.0.1'
port = get_free_port('http')
+ proxy_port = get_free_port('proxy_port')
ssl_port = get_free_port('https')
- test_server = run_server(host, port, ssl_port, test_mode=True, model=model)
+ test_server = run_server(host, port, ssl_port, test_mode=True,
+ proxy_port=proxy_port, model=model)
def tearDownModule():
@@ -1356,7 +1358,7 @@ class RestTests(unittest.TestCase):
def test_config(self):
resp = self.request('/config').read()
conf = json.loads(resp)
- self.assertEquals(port, conf['http_port'])
+ self.assertEquals(proxy_port, conf['http_port'])
def test_capabilities(self):
resp = self.request('/config/capabilities').read()
diff --git a/tests/utils.py b/tests/utils.py
index fe03a1a..77e5d1d 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -93,16 +93,22 @@ def get_free_port(name='http'):
return _ports[name]
-def run_server(host, port, ssl_port, test_mode,
+def run_server(host, port, ssl_port, test_mode, proxy_port=None,
model=None, environment='development'):
+ if proxy_port is None:
+ proxy_port = get_free_port('proxy_port')
+ if ssl_port is None:
+ ssl_port = get_free_port('ssl_port')
args = type('_', (object,),
- {'host': host, 'port': port, 'ssl_port':
ssl_port,
+ {'host': host, 'port': port, 'proxy_ssl_port':
ssl_port,
+ 'proxy_port': proxy_port,
'ssl_cert': '', 'ssl_key': '',
'test': test_mode, 'access_log': '/dev/null',
'error_log': '/dev/null', 'environment':
environment,
'log_level': 'debug'})()
if model is not None:
setattr(args, 'model', model)
+
s = kimchi.server.Server(args)
t = threading.Thread(target=s.start)
t.setDaemon(True)
--
1.8.3.1
2:29 p.m.
New subject: [PATCH v6 4/4] Github #329: .gitignore, spec, control.in and readme
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
- Added src/nginx.conf.in to Fedora and SUSE spec files and nginx
as dependency.
- Added src/nginx.conf.in to Makefile.am.
- Added nginx to contrib/DEBIAN/control.in as a dependency.
- Added nginx to README.md instructions.
- Added src/nginx_kimchi.conf to .gitignore. This file is a product
of the nginx.config.in template tha was used for the last (or
current) kimchid/nginx run. As such, there is no need to put
this file in version control.
- Makefile.am: added nginx.conf.in to dist_conf_DATA and added
src/kimchid.in and src/kimchi/proxy.py in pep8 whitelist.
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
fix patch 4 Makefile.am add whitelist
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
---
.gitignore | 1 +
Makefile.am | 2 ++
contrib/DEBIAN/control.in | 3 ++-
contrib/kimchi.spec.fedora.in | 2 ++
contrib/kimchi.spec.suse.in | 2 ++
docs/README.md | 4 ++--
src/Makefile.am | 3 ++-
7 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/.gitignore b/.gitignore
index 073f46e..a776cb0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -27,6 +27,7 @@ stamp-po
kimchi-*.tar.gz
src/kimchid
src/kimchi.conf
+src/nginx_kimchi.conf
src/kimchi/config.py
tests/run_tests.sh
tests/test_config.py
diff --git a/Makefile.am b/Makefile.am
index ca5623d..28ee39b 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -38,6 +38,7 @@ PEP8_WHITELIST = \
plugins/__init__.py \
plugins/sample/__init__.py \
plugins/sample/model.py \
+ src/kimchid.in \
src/kimchi/asynctask.py \
src/kimchi/auth.py \
src/kimchi/cachebust.py \
@@ -55,6 +56,7 @@ PEP8_WHITELIST = \
src/kimchi/model/*.py \
src/kimchi/objectstore.py \
src/kimchi/osinfo.py \
+ src/kimchi/proxy.py \
src/kimchi/repositories.py \
src/kimchi/rollbackcontext.py \
src/kimchi/root.py \
diff --git a/contrib/DEBIAN/control.in b/contrib/DEBIAN/control.in
index c2b2a40..aac1a24 100644
--- a/contrib/DEBIAN/control.in
+++ b/contrib/DEBIAN/control.in
@@ -22,7 +22,8 @@ Depends: python-cherrypy3 (>= 3.2.0),
python-ipaddr,
python-lxml,
open-iscsi,
- firewalld
+ firewalld,
+ nginx
Build-Depends: libxslt,
python-libxml2
Maintainer: Aline Manera <alinefm(a)br.ibm.com>
diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in
index 1cd99b0..77bf6bf 100644
--- a/contrib/kimchi.spec.fedora.in
+++ b/contrib/kimchi.spec.fedora.in
@@ -28,6 +28,7 @@ Requires: sos
Requires: python-ipaddr
Requires: python-lxml
Requires: nfs-utils
+Requires: nginx
Requires: iscsi-initiator-utils
BuildRequires: libxslt
BuildRequires: libxml2-python
@@ -175,6 +176,7 @@ rm -rf $RPM_BUILD_ROOT
%{_datadir}/kimchi/ui/pages/help/*/*.html
%{_datadir}/kimchi/ui/pages/tabs/*.html.tmpl
%{_sysconfdir}/kimchi/kimchi.conf
+%{_sysconfdir}/kimchi/nginx.conf.in
%{_sysconfdir}/kimchi/distros.d/debian.json
%{_sysconfdir}/kimchi/distros.d/fedora.json
%{_sysconfdir}/kimchi/distros.d/opensuse.json
diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in
index efb2c08..89374e1 100644
--- a/contrib/kimchi.spec.suse.in
+++ b/contrib/kimchi.spec.suse.in
@@ -24,6 +24,7 @@ Requires: python-ipaddr
Requires: python-lxml
Requires: python-xml
Requires: nfs-client
+Requires: nginx
Requires: open-iscsi
BuildRequires: libxslt-tools
BuildRequires: python-libxml2
@@ -101,6 +102,7 @@ rm -rf $RPM_BUILD_ROOT
%{_datadir}/kimchi/ui/pages/help/*/*.html
%{_datadir}/kimchi/ui/pages/tabs/*.html.tmpl
%{_sysconfdir}/kimchi/kimchi.conf
+%{_sysconfdir}/kimchi/nginx.conf.in
%{_sysconfdir}/kimchi/distros.d/debian.json
%{_sysconfdir}/kimchi/distros.d/fedora.json
%{_sysconfdir}/kimchi/distros.d/opensuse.json
diff --git a/docs/README.md b/docs/README.md
index 8b8b181..63ac760 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -53,7 +53,7 @@ Install Dependencies
PyPAM m2crypto python-jsonschema rpm-build \
qemu-kvm python-psutil python-ethtool sos \
python-ipaddr python-lxml nfs-utils \
- iscsi-initiator-utils libxslt pyparted
+ iscsi-initiator-utils libxslt pyparted nginx
# If using RHEL6, install the following additional packages:
$ sudo yum install python-unittest2 python-ordereddict
# Restart libvirt to allow configuration changes to take effect
@@ -75,7 +75,7 @@ for more information on how to configure your system to access this
repository.
python-pam python-m2crypto python-jsonschema \
qemu-kvm libtool python-psutil python-ethtool \
sosreport python-ipaddr python-lxml nfs-common \
- open-iscsi lvm2 xsltproc python-parted
+ open-iscsi lvm2 xsltproc python-parted nginx
Packages version requirement:
python-jsonschema >= 1.3.0
diff --git a/src/Makefile.am b/src/Makefile.am
index 2005f7c..dfeb24e 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -21,13 +21,14 @@ SUBDIRS = kimchi distros.d
EXTRA_DIST = kimchid.in \
kimchi.conf.in \
+ nginx.conf.in \
firewalld.xml \
$(NULL)
bin_SCRIPTS = kimchid
confdir = $(sysconfdir)/kimchi
-dist_conf_DATA = kimchi.conf
+dist_conf_DATA = kimchi.conf nginx.conf.in
BUILT_SOURCES = kimchi.conf
--
1.8.3.1
9:05 p.m.
New subject: [PATCH v6 1/4] Github #329: Proxy module and template file
On 04/16/2014 04:29 PM, Daniel Barboza wrote:
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
The file src/kimchi/proxy.py is a module that contains all Nginx
related functions - start proxy, terminate proxy and create
proxy config.
src/nginx.conf.in is a template file that is used by the proxy
module to generate a customized proxy configuration.
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
---
src/kimchi/proxy.py | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++
src/nginx.conf.in | 55 +++++++++++++++++++++++++++
2 files changed, 162 insertions(+)
create mode 100644 src/kimchi/proxy.py
create mode 100644 src/nginx.conf.in
diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py
new file mode 100644
index 0000000..23944ea
--- /dev/null
+++ b/src/kimchi/proxy.py
@@ -0,0 +1,107 @@
+#!/usr/bin/python
+#
+# Project Kimchi
+#
+# Copyright IBM, Corp. 2014
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+# MA 02110-1301 USA
+
+# This module contains functions that the manipulate
+# and configure the Nginx proxy.
+
+import os
+import pwd
+import sslcert
sslcert is a kimchi module so we should import it as:
from kimchi import sslcert
I can update it before applying
+import subprocess
+from string import Template
+
+import kimchi.config
+from kimchi.config import config, paths
+
+
+def _create_proxy_config(p_port, k_port, p_ssl_port, cert, key):
+ """Create nginx configuration file based on current ports config
+
+ To allow flexibility in which port kimchi runs, we need the same
+ flexibility with the nginx proxy. This method creates the config
+ file dynamically by using 'nginx.conf.in' as a template, creating
+ the file 'nginx_kimchi.config' which will be used to launch the
+ proxy.
+
+ Arguments:
+ p_port - proxy port
+ k_port - kimchid port
+ p_ssl_port - proxy SSL port
+ cert - cert file specified by user config
+ key - key file specified by user config
+ """
+
+ # User that will run the worker process of the proxy. Fedora,
+ # RHEL and Suse creates an user called 'nginx' when installing
+ # the proxy. Ubuntu creates an user 'www-data' for it.
+ user_proxy = 'nginx'
+ try:
+ pwd.getpwnam(user_proxy)
+ except KeyError:
+ user_proxy = 'www-data'
+
+ # No certificates specified by the user
+ if not cert or not key:
+ config_dir = paths.conf_dir
+ cert = '%s/kimchi-cert.pem' % config_dir
+ key = '%s/kimchi-key.pem' % config_dir
+ # create cert files if they don't exist
+ if not os.path.exists(cert) or not os.path.exists(key):
+ ssl_gen = sslcert.SSLCert()
+ with open(cert, "w") as f:
+ f.write(ssl_gen.cert_pem())
+ with open(key, "w") as f:
+ f.write(ssl_gen.key_pem())
+
+ # Read template file and create a new config file
+ # with the specified parameters.
+ with open(os.path.join(config_dir, "nginx.conf.in")) as template:
+ data = template.read()
+ data = Template(data)
+ data = data.safe_substitute(user=user_proxy,
+ proxy_port=p_port,
+ kimchid_port=k_port,
+ proxy_ssl_port=p_ssl_port,
+ cert_pem=cert, cert_key=key)
+
+ # Write file to be used for nginx.
+ config_file = open(os.path.join(config_dir, "nginx_kimchi.conf"),
"w")
+ config_file.write(data)
+ config_file.close()
+
+
+def start_proxy(options):
+ """Start nginx reverse proxy."""
+ _create_proxy_config(options.proxy_port,
+ options.port,
+ options.proxy_ssl_port,
+ options.ssl_cert,
+ options.ssl_key)
+ config_dir = paths.conf_dir
+ config_file = "%s/nginx_kimchi.conf" % config_dir
+ cmd = ['nginx', '-c', config_file]
+ subprocess.call(cmd)
+
+
+def terminate_proxy():
+ """Stop nginx process."""
+ term_proxy_cmd = ['nginx', '-s', 'stop']
+ subprocess.call(term_proxy_cmd)
diff --git a/src/nginx.conf.in b/src/nginx.conf.in
new file mode 100644
index 0000000..967b46b
--- /dev/null
+++ b/src/nginx.conf.in
@@ -0,0 +1,55 @@
+# Project Kimchi
+#
+# Copyright IBM, Corp. 2014
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+# 02110-1301 USA
+
+
+# This is a template file to be used to generate a nginx
+# proxy config file at kimchid script.
+
+user $user;
+worker_processes 1;
+
+error_log /var/log/nginx/error.log;
+
+events {
+ worker_connections 1024;
+}
+
+
+http {
+
+ log_format main '$remote_addr - $remote_user [$time_local]
"$request" '
+ '$status $body_bytes_sent "$http_referer" '
+ '"$http_user_agent"
"$http_x_forwarded_for"';
+
+ access_log /var/log/nginx/access.log main;
+
+ sendfile on;
+
+ server {
+ listen $proxy_port;
+ listen $proxy_ssl_port ssl;
+ ssl_certificate $cert_pem;
+ ssl_certificate_key $cert_key;
+
+ location / {
+ proxy_pass http://localhost:$kimchid_port;
+ proxy_set_header Host $host;
+ }
+ }
+}
9:06 p.m.
Reviewed-by: Aline Manera <alinefm(a)linux.vnet.ibm.com>
Just a minor comment in patch 1 that I can fixed before applying.
On 04/16/2014 04:29 PM, Daniel Barboza wrote:
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
New in v6:
- Moved the SSL logic from server.py to proxy.py
- server.py is now responsible to launch the proxy
- removed references of 'ssl_port'
New in v5:
- fixes to VNC connection
- fixes to unit tests
- obscuring kimchid port by making it localhost only
- launhing nginx proxy in unit tests
New in v4:
- added a new module with all nginx-related methods
- added DEBIAN entries
- other fixes proposed by Aline
After reading the comments from Zhou Zheng Sheng, I simplified
the work I did in version 2 to run only one cherrypy process
instead of two processes, one for frontend and another for the
backend. Nginx is still being used as a reverse proxy to allow
kimchid to run as root, but not being exposed to the outside.
As Zhou mentioned, it is to little avail to run frontend and
backend separately if the exposed port is running by the
reverse proxy anyway. He mentioned the RPC approach as a best
long-term approach, which I agree. We can solve this issue right
now and the work in a more suitable solution, such as RPC, and
then ditch nginx.
Daniel Henrique Barboza (4):
Github #329: Proxy module and template file
Github #329: Kimchid, config.py.in and server.py changes
Github #329: changes in mockmodel, model/config and tests
Github #329: .gitignore, spec, control.in and readme
.gitignore | 1 +
Makefile.am | 2 +
contrib/DEBIAN/control.in | 3 +-
contrib/kimchi.spec.fedora.in | 2 +
contrib/kimchi.spec.suse.in | 2 +
docs/README.md | 4 +-
src/Makefile.am | 3 +-
src/kimchi/config.py.in | 5 +-
src/kimchi/mockmodel.py | 2 +-
src/kimchi/model/config.py | 2 +-
src/kimchi/proxy.py | 107 ++++++++++++++++++++++++++++++++++++++++++
src/kimchi/server.py | 44 +++++------------
src/kimchid.in | 44 ++++++++++++-----
src/nginx.conf.in | 55 ++++++++++++++++++++++
tests/test_rest.py | 8 ++--
tests/utils.py | 10 +++-
16 files changed, 238 insertions(+), 56 deletions(-)
create mode 100644 src/kimchi/proxy.py
create mode 100644 src/nginx.conf.in
9:29 p.m.
I applied this patch set in a separated branch: wip
(https://github.com/kimchi-project/kimchi/tree/wip)
so anyone can have a try and check everything is working fine.
I plan to merge it to master tomorrow.
On 04/16/2014 04:29 PM, Daniel Barboza wrote:
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
New in v6:
- Moved the SSL logic from server.py to proxy.py
- server.py is now responsible to launch the proxy
- removed references of 'ssl_port'
New in v5:
- fixes to VNC connection
- fixes to unit tests
- obscuring kimchid port by making it localhost only
- launhing nginx proxy in unit tests
New in v4:
- added a new module with all nginx-related methods
- added DEBIAN entries
- other fixes proposed by Aline
After reading the comments from Zhou Zheng Sheng, I simplified
the work I did in version 2 to run only one cherrypy process
instead of two processes, one for frontend and another for the
backend. Nginx is still being used as a reverse proxy to allow
kimchid to run as root, but not being exposed to the outside.
As Zhou mentioned, it is to little avail to run frontend and
backend separately if the exposed port is running by the
reverse proxy anyway. He mentioned the RPC approach as a best
long-term approach, which I agree. We can solve this issue right
now and the work in a more suitable solution, such as RPC, and
then ditch nginx.
Daniel Henrique Barboza (4):
Github #329: Proxy module and template file
Github #329: Kimchid, config.py.in and server.py changes
Github #329: changes in mockmodel, model/config and tests
Github #329: .gitignore, spec, control.in and readme
.gitignore | 1 +
Makefile.am | 2 +
contrib/DEBIAN/control.in | 3 +-
contrib/kimchi.spec.fedora.in | 2 +
contrib/kimchi.spec.suse.in | 2 +
docs/README.md | 4 +-
src/Makefile.am | 3 +-
src/kimchi/config.py.in | 5 +-
src/kimchi/mockmodel.py | 2 +-
src/kimchi/model/config.py | 2 +-
src/kimchi/proxy.py | 107 ++++++++++++++++++++++++++++++++++++++++++
src/kimchi/server.py | 44 +++++------------
src/kimchid.in | 44 ++++++++++++-----
src/nginx.conf.in | 55 ++++++++++++++++++++++
tests/test_rest.py | 8 ++--
tests/utils.py | 10 +++-
16 files changed, 238 insertions(+), 56 deletions(-)
create mode 100644 src/kimchi/proxy.py
create mode 100644 src/nginx.conf.in
3657
days inactive
3665
days old
8 comments
2 participants
participants (2)
-
Aline Manera -
Daniel Barboza