[Kimchi-devel] [PATCH] [Wok] Use system's nginx proxy service.
Daniel Henrique Barboza
dhbarboza82 at gmail.com
Mon Jul 11 11:49:32 UTC 2016
On 07/11/2016 08:05 AM, Paulo Ricardo Paz Vital wrote:
> On Jul 08 06:15PM, Daniel Henrique Barboza wrote:
>> make check-local errors found:
>>
>> [danielhb at arthas wok_all_plugins]$ sudo make check-local
>> PYTHONPATH=src contrib/check_i18n.py src/wok/plugins/*/i18n.py
>> src/wok/i18n.py
>> Checking for invalid i18n string...
>> Checking for invalid i18n string successfully
>> ./src/wok/proxy.py:29: 'subprocess' imported but unused
>> Makefile:930: recipe for target 'check-local' failed
>> make: *** [check-local] Error 1
>> [danielhb at arthas wok_all_plugins]$
>>
> Ouch!!! My check-local passed but I'm going to fix this.
>
>> I wasn't able to start wok after applying this patch. The error given is:
>>
>>
>> [danielhb at arthas wok_all_plugins]$ sudo src/wokd
>> [sudo] password for danielhb:
>> WOKPROXY0001E: WOKPROXY0001E
>> Traceback (most recent call last):
>> File "src/wokd", line 106, in <module>
>> sys.exit(main(sys.argv[1:]))
>> File "src/wokd", line 103, in main
>> wok.server.main(options)
>> File "/home/danielhb/kimchi/wok_all_plugins/src/wok/server.py", line 246,
>> in main
>> srv = Server(options)
>> File "/home/danielhb/kimchi/wok_all_plugins/src/wok/server.py", line 77,
>> in __init__
>> start_proxy(options)
>> File "/home/danielhb/kimchi/wok_all_plugins/src/wok/proxy.py", line 132,
>> in start_proxy
>> raise OperationFailed('WOKPROXY0001E', {'error': error})
>> wok.exception.OperationFailed: WOKPROXY0001E: WOKPROXY0001E
>> [danielhb at arthas wok_all_plugins]$
>>
> It's expected. My patch is restarting the nginx.service using systemctl
> command (since all supported distros today, use Systemd as default
> system) by the run_command(). So, I check if retcode is different of
> zero and raise this error.
>
>> I think it's because the nginx service isn't running in the host. However,
>> when trying to start the nginx service I got the following error:
>>
>> [danielhb at arthas conf.d]$ sudo systemctl start nginx
>> Job for nginx.service failed because the control process exited with error
>> code. See "systemctl status nginx.service" and "journalctl -xe" for details.
>>
>> [danielhb at arthas conf.d]$ sudo systemctl status nginx -l
>> ● nginx.service - The nginx HTTP and reverse proxy server
>> Loaded: loaded (/usr/lib/systemd/system/nginx.service; disabled; vendor
>> preset: disabled)
>> Active: failed (Result: exit-code) since Fri 2016-07-08 18:07:48 BRT; 11s
>> ago
>> Process: 18946 ExecStartPre=/usr/sbin/nginx -t (code=exited,
>> status=1/FAILURE)
>> Process: 18937 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited,
>> status=0/SUCCESS)
>>
>> Jul 08 18:07:48 arthas.ltc.br.ibm.com systemd[1]: Starting The nginx HTTP
>> and reverse proxy server...
>> Jul 08 18:07:48 arthas.ltc.br.ibm.com nginx[18946]: nginx: [emerg] open()
>> "/etc/nginx/conf.d/wok.conf" failed (13: Permission denied) in
>> /etc/nginx/nginx.conf:33
>> Jul 08 18:07:48 arthas.ltc.br.ibm.com nginx[18946]: nginx: configuration
>> file /etc/nginx/nginx.conf test failed
>> Jul 08 18:07:48 arthas.ltc.br.ibm.com systemd[1]: nginx.service: Control
>> process exited, code=exited status=1
>> Jul 08 18:07:48 arthas.ltc.br.ibm.com systemd[1]: Failed to start The nginx
>> HTTP and reverse proxy server.
>> Jul 08 18:07:48 arthas.ltc.br.ibm.com systemd[1]: nginx.service: Unit
>> entered failed state.
>> Jul 08 18:07:48 arthas.ltc.br.ibm.com systemd[1]: nginx.service: Failed with
>> result 'exit-code'.
>> [danielhb at arthas conf.d]$
>>
>>
>> Says that it couldn't open /etc/nginx/conf.d/wok.conf. These are the
>> permissions:
>>
>> [danielhb at arthas conf.d]$ ls -lah
>> total 12K
>> drwxr-xr-x. 2 root root 4.0K Jul 8 18:08 .
>> drwxr-xr-x. 4 root root 4.0K Jun 11 09:17 ..
>> lrwxrwxrwx. 1 root root 56 Jul 8 18:08 wok.conf ->
>> /home/danielhb/kimchi/wok_all_plugins/src/nginx/wok.conf
>> -rw-r--r--. 1 root root 3.6K Jun 21 11:48 wok.conf.in
>> [danielhb at arthas conf.d]$
>> [danielhb at arthas conf.d]$ ls -lah
>> /home/danielhb/kimchi/wok_all_plugins/src/nginx/wok.conf
>> -rw-r--r--. 1 root root 3.2K Jul 8 18:08
>> /home/danielhb/kimchi/wok_all_plugins/src/nginx/wok.conf
>> [danielhb at arthas conf.d]$
>> [danielhb at arthas conf.d]$
>>
>>
>>
>> A quick search on the internet mention that SELinux might be on play
>> here, preventing the nginx service from reading the file. I didn't
>> tried to turn off SELinux to see if it works but I can try later. I've found
>> posts in Ubuntu forums mentioning that the file should be owned to the user
>> 'www-data' or something like that. Haven't tried that either.
>>
>> I'll try this patch in other systems I have access to see if the
>> results are the same. There's a chance that this error is specific
>> to my env.
>>
>>
> Yeah, the "failed (13: Permission denied)" error is a SELinux error, but
> in my tests on two Fedora 24 systems it was solved by the following
> command in console (I forgot comment this when submitted the patch):
>
> $ sudo restorecon -R -v /etc/nginx/conf.d/wok.conf
>
> In addition, it's necessary follow the SELinux steps reported on
> https://github.com/kimchi-project/wok/blob/master/docs/troubleshooting.md#selinux
>
> Something important to note, is that you're running from a NOT INSTALLED
> WOK then it's necessary create a symbolic link in /etc/nginx/conf.d/
> pointing to wok.conf file created from the path you're executing wokd. If
> you install wok, and execute from the installed system, the file is
> created if no problems.
>
> This symbolic link is necessary due to errors in nginx.service startup
> related to not able to open the *.perm and *.cert files in the path from
> the executed wokd.
I believe we can add these instructions of how to run WoK from source
in the README.
>
>>
>> Last but not the least, there are a bunch of unit tests failing here
>> (FAILED (errors=7)) but I think they are related to the problem
>> I've described above (nginx not running).
>>
> I'm going to check these error also.
>
>> Daniel
>>
>> On 07/08/2016 01:26 PM, pvital at linux.vnet.ibm.com wrote:
>>> From: Paulo Vital <pvital at linux.vnet.ibm.com>
>>>
>>> This patch removes the code that executes a dedicated nginx proxy, making Wok
>>> to use the system's nginx service. This is a requirement to make Wok acceptable
>>> in community repositories.
>>>
>>> It also make sure that a Wok executed from path different than installed (from
>>> a cloned and built source code, for example) will create a symbolic link in
>>> system's nginx config dir to the running configuration file.
>>>
>>> This patch solves part of issue #25
>>>
>>> Signed-off-by: Paulo Vital <pvital at linux.vnet.ibm.com>
>>> ---
>>> src/nginx/wok.conf.in | 95 +++++++++++++++++++++------------------------------
>>> src/wok/config.py.in | 6 +++-
>>> src/wok/i18n.py | 2 ++
>>> src/wok/proxy.py | 27 +++++++++------
>>> src/wok/server.py | 6 +---
>>> 5 files changed, 63 insertions(+), 73 deletions(-)
>>>
>>> diff --git a/src/nginx/wok.conf.in b/src/nginx/wok.conf.in
>>> index cb05e4d..823d94d 100644
>>> --- a/src/nginx/wok.conf.in
>>> +++ b/src/nginx/wok.conf.in
>>> @@ -22,71 +22,54 @@
>>> # This is a template file to be used to generate a nginx
>>> # proxy config file at wokd script.
>>>
>>> -user ${user};
>>> -worker_processes 1;
>>> +client_max_body_size ${max_body_size}k;
>>>
>>> -error_log /var/log/nginx/error.log;
>>> +# Timeout set to 10 minutes to avoid the 504 Gateway Timeout
>>> +# when Wok is processing a request.
>>> +proxy_connect_timeout 600;
>>> +proxy_send_timeout 600;
>>> +proxy_read_timeout 600;
>>> +send_timeout 600;
>>>
>>> -events {
>>> - worker_connections 1024;
>>> +map $http_upgrade $connection_upgrade {
>>> + default upgrade;
>>> + '' close;
>>> }
>>>
>>> -http {
>>> - log_format main '$remote_addr - $remote_user [$time_local] "$request" '
>>> - '$status $body_bytes_sent "$http_referer" '
>>> - '"$http_user_agent" "$http_x_forwarded_for"';
>>> -
>>> - access_log /var/log/nginx/access.log main;
>>> - sendfile on;
>>> +upstream websocket {
>>> + server 127.0.0.1:${websockets_port};
>>> +}
>>>
>>> - client_max_body_size ${max_body_size}k;
>>> +server {
>>> + listen ${host_addr}:${proxy_ssl_port} ssl;
>>>
>>> - # Timeout set to 10 minutes to avoid the 504 Gateway Timeout
>>> - # when Wok is processing a request.
>>> - proxy_connect_timeout 600;
>>> - proxy_send_timeout 600;
>>> - proxy_read_timeout 600;
>>> - send_timeout 600;
>>> + ssl_certificate ${cert_pem};
>>> + ssl_certificate_key ${cert_key};
>>> + ssl_protocols TLSv1.1 TLSv1.2;
>>> + ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:@STRENGTH';
>>> + ssl_prefer_server_ciphers on;
>>> + ssl_dhparam ${dhparams_pem};
>>> + ssl_session_timeout ${session_timeout}m;
>>>
>>> - map $http_upgrade $connection_upgrade {
>>> - default upgrade;
>>> - '' close;
>>> - }
>>> + add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
>>> + add_header X-Frame-Options DENY;
>>> + add_header X-Content-Type-Options nosniff;
>>> + add_header X-XSS-Protection "1; mode=block";
>>>
>>> - upstream websocket {
>>> - server 127.0.0.1:${websockets_port};
>>> + location / {
>>> + proxy_pass http://127.0.0.1:${cherrypy_port};
>>> + proxy_set_header Host $host;
>>> + proxy_set_header X-Real-IP $remote_addr;
>>> + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>>> + proxy_redirect http://127.0.0.1:${cherrypy_port}/ https://$host:${proxy_ssl_port}/;
>>> }
>>>
>>> - server {
>>> - listen ${host_addr}:${proxy_ssl_port} ssl;
>>> -
>>> - ssl_certificate ${cert_pem};
>>> - ssl_certificate_key ${cert_key};
>>> - ssl_protocols TLSv1.1 TLSv1.2;
>>> - ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:@STRENGTH';
>>> - ssl_prefer_server_ciphers on;
>>> - ssl_dhparam ${dhparams_pem};
>>> - ssl_session_timeout ${session_timeout}m;
>>> -
>>> - add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
>>> - add_header X-Frame-Options DENY;
>>> - add_header X-Content-Type-Options nosniff;
>>> - add_header X-XSS-Protection "1; mode=block";
>>> -
>>> - location / {
>>> - proxy_pass http://127.0.0.1:${cherrypy_port};
>>> - proxy_set_header Host $host;
>>> - proxy_set_header X-Real-IP $remote_addr;
>>> - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>>> - proxy_redirect http://127.0.0.1:${cherrypy_port}/ https://$host:${proxy_ssl_port}/;
>>> - }
>>> -
>>> - location /websockify {
>>> - proxy_pass http://websocket;
>>> - proxy_http_version 1.1;
>>> - proxy_set_header Upgrade $http_upgrade;
>>> - proxy_set_header Connection $connection_upgrade;
>>> - }
>>> + location /websockify {
>>> + proxy_pass http://websocket;
>>> + proxy_http_version 1.1;
>>> + proxy_set_header Upgrade $http_upgrade;
>>> + proxy_set_header Connection $connection_upgrade;
>>> }
>>> - ${http_config}
>>> }
>>> +
>>> +${http_config}
>>> diff --git a/src/wok/config.py.in b/src/wok/config.py.in
>>> index 1ca6f73..d0bca9b 100644
>>> --- a/src/wok/config.py.in
>>> +++ b/src/wok/config.py.in
>>> @@ -76,9 +76,10 @@ class Paths(object):
>>> self.prefix = self.get_prefix()
>>> self.installed = (self.prefix == '@pkgdatadir@')
>>> self.ui_dir = self.add_prefix('ui')
>>> + self.sys_nginx_conf_dir = '@sysconfdir@/nginx/conf.d'
>>>
>>> if self.installed:
>>> - self.nginx_conf_dir = '@sysconfdir@/nginx/conf.d'
>>> + self.nginx_conf_dir = self.sys_nginx_conf_dir
>>> self.state_dir = '@localstatedir@/lib/wok'
>>> self.log_dir = '@localstatedir@/log/wok'
>>> self.conf_dir = '@sysconfdir@/wok'
>>> @@ -119,6 +120,9 @@ class Paths(object):
>>> def get_template_path(self, resource):
>>> return os.path.join(self.ui_dir, 'pages/%s.tmpl' % resource)
>>>
>>> + def is_wok_installed(self):
>>> + return self.installed
>>> +
>>>
>>> paths = Paths()
>>>
>>> diff --git a/src/wok/i18n.py b/src/wok/i18n.py
>>> index d6cb17c..33107ee 100644
>>> --- a/src/wok/i18n.py
>>> +++ b/src/wok/i18n.py
>>> @@ -55,6 +55,8 @@ messages = {
>>> "WOKUTILS0004E": _("Invalid data value '%(value)s'"),
>>> "WOKUTILS0005E": _("Invalid data unit '%(unit)s'"),
>>>
>>> + "WOKPROXY0001E": _("Unable to (re)start system's nginx.service. Details: '%(error)s'"),
>>> +
>>> # These messages (ending with L) are for user log purposes
>>> "WOKCOL0001L": _("Request made on collection"),
>>> "WOKRES0001L": _("Request made on resource"),
>>> diff --git a/src/wok/proxy.py b/src/wok/proxy.py
>>> index a74e88a..ab959e1 100644
>>> --- a/src/wok/proxy.py
>>> +++ b/src/wok/proxy.py
>>> @@ -31,6 +31,8 @@ from string import Template
>>>
>>> from wok import sslcert
>>> from wok.config import paths
>>> +from wok.exception import OperationFailed
>>> +from wok.utils import run_command
>>>
>>>
>>> HTTP_CONFIG = """
>>> @@ -110,18 +112,21 @@ def _create_proxy_config(options):
>>> config_file.write(data)
>>> config_file.close()
>>>
>>> + # If not running from the installed path (from a cloned and builded source
>>> + # code), create a symbolic link in system's dir to prevent errors on read
>>> + # SSL certifications.
>>> + if not paths.is_wok_installed():
>>> + dst = os.path.join(paths.sys_nginx_conf_dir, "wok.conf")
>>> + if os.path.isfile(dst):
>>> + os.remove(dst)
>>> + os.symlink(os.path.join(nginx_config_dir, "wok.conf"), dst)
>>> +
>>>
>>> def start_proxy(options):
>>> """Start nginx reverse proxy."""
>>> _create_proxy_config(options)
>>> - nginx_config_dir = paths.nginx_conf_dir
>>> - config_file = "%s/wok.conf" % nginx_config_dir
>>> - cmd = ['nginx', '-c', config_file]
>>> - subprocess.call(cmd)
>>> -
>>> -
>>> -def terminate_proxy():
>>> - """Stop nginx process."""
>>> - config_file = "%s/wok.conf" % paths.nginx_conf_dir
>>> - term_proxy_cmd = ['nginx', '-s', 'stop', '-c', config_file]
>>> - subprocess.call(term_proxy_cmd)
>>> + # Restart system's nginx service to reload wok configuration
>>> + cmd = ['systemctl', 'restart', 'nginx.service']
>>> + output, error, retcode = run_command(cmd, silent=True)
>>> + if retcode != 0:
>>> + raise OperationFailed('WOKPROXY0001E', {'error': error})
>>> diff --git a/src/wok/server.py b/src/wok/server.py
>>> index 8a02596..b1185e3 100644
>>> --- a/src/wok/server.py
>>> +++ b/src/wok/server.py
>>> @@ -33,7 +33,7 @@ from wok.config import config as configParser
>>> from wok.config import paths, PluginConfig, WokConfig
>>> from wok.control import sub_nodes
>>> from wok.model import model
>>> -from wok.proxy import start_proxy, terminate_proxy
>>> +from wok.proxy import start_proxy
>>> from wok.reqlogger import RequestLogger
>>> from wok.root import WokRoot
>>> from wok.safewatchedfilehandler import SafeWatchedFileHandler
>>> @@ -180,10 +180,6 @@ class Server(object):
>>> self.app = cherrypy.tree.mount(WokRoot(model_instance, dev_env),
>>> config=self.configObj)
>>> self._load_plugins(options)
>>> -
>>> - # Terminate proxy when cherrypy server is terminated
>>> - cherrypy.engine.subscribe('exit', terminate_proxy)
>>> -
>>> cherrypy.lib.sessions.init()
>>>
>>> def _load_plugins(self, options):
>>> --
>>> 2.7.4
>>>
>>> _______________________________________________
>>> Kimchi-devel mailing list
>>> Kimchi-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/kimchi-devel
>> _______________________________________________
>> Kimchi-devel mailing list
>> Kimchi-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/kimchi-devel
More information about the Kimchi-devel
mailing list