[Kimchi-devel] [PATCH] [Wok] Add federation feature to Wok
Daniel Henrique Barboza
dhbarboza82 at gmail.com
Wed May 3 12:37:52 UTC 2017
Reviewed-by: Daniel Barboza <danielhb at linux.vnet.ibm.com>
On 05/02/2017 01:59 PM, Aline Manera wrote:
> It was primarily implemented on Kimchi project. But it makes more sense
> to be on Wok since the idea is to allow user add/select on which server
> Wok should manage.
> For example, the user could have a Wok installation on server A to
> manage server B.
>
> Signed-off-by: Aline Manera <alinefm at linux.vnet.ibm.com>
> ---
> docs/API/config.md | 2 ++
> docs/API/peers.md | 20 +++++++++++
> docs/README-federation.md | 60 ++++++++++++++++++++++++++++++++
> docs/wokd.8.in | 6 +++-
> src/wok.conf.in | 4 +++
> src/wok/config.py.in | 1 +
> src/wok/control/peers.py | 30 ++++++++++++++++
> src/wok/model/config.py | 1 +
> src/wok/model/peers.py | 74 +++++++++++++++++++++++++++++++++++++++
> src/wokd.in | 5 +++
> tests/test_api.py | 6 +++-
> tests/test_config_model.py | 2 +-
> tests/test_server_root.py | 6 ++--
> ui/js/src/wok.api.js | 14 ++++++++
> ui/js/wok.peers.js | 75 ++++++++++++++++++++++++++++++++++++++++
> ui/pages/tabs/settings.html.tmpl | 34 +++++++++++++++++-
> 16 files changed, 333 insertions(+), 7 deletions(-)
> create mode 100644 docs/API/peers.md
> create mode 100644 docs/README-federation.md
> create mode 100644 src/wok/control/peers.py
> create mode 100644 src/wok/model/peers.py
> create mode 100644 ui/js/wok.peers.js
>
> diff --git a/docs/API/config.md b/docs/API/config.md
> index d1d1007..0e47f5f 100644
> --- a/docs/API/config.md
> +++ b/docs/API/config.md
> @@ -12,6 +12,8 @@ Contains information about the application environment and configuration.
> * proxy_port: SSL port to list on
> * websockets_port: Port for websocket proxy to listen on
> * auth: Authentication method used to log in to Wok
> + * server_root: Relative path to Wok server. No value is specified by default, ie, Wok will run on '/'
> + * federation: 'on' if federation feature is enabled, 'off' otherwise.
> * version: Wok version
> * **POST**: *See Task Actions*
>
> diff --git a/docs/API/peers.md b/docs/API/peers.md
> new file mode 100644
> index 0000000..fa5cf41
> --- /dev/null
> +++ b/docs/API/peers.md
> @@ -0,0 +1,20 @@
> +## REST API Specification for Peers
> +
> +### Collection: Peers
> +
> +**URI:** /peers
> +
> +Return a list of Wok peers in the same network.
> +(It uses openSLP for discovering)
> +
> +**Methods:**
> +
> +* **GET**: Retrieve a list peers URLs.
> +
> +#### Examples
> +GET /peers
> +[
> + https://wok-peer0:8001,
> + https://wok-peer1:8001,
> + https://wok-peer2:8001,
> +]
> diff --git a/docs/README-federation.md b/docs/README-federation.md
> new file mode 100644
> index 0000000..18e19c7
> --- /dev/null
> +++ b/docs/README-federation.md
> @@ -0,0 +1,60 @@
> +Wok Project - Federation Feature
> +===================================
> +
> +Federation feature is a mechanism to discover Wok peers in the same
> +network. It uses openSLP tool (http://www.openslp.org/) to register and find Wok
> +servers.
> +
> +By default this feature is disabled on Wok as it is not critical for KVM
> +virtualization and requires additional software installation.
> +
> +To enable it, do the following:
> +
> +1. Install openslp and openslp-server rpm packages,
> + or install slpd and slptool deb packages.
> +
> +2. openSLP uses port 427 (UDP) and port 427 (TCP) so make sure to open those
> + ports in your firewall configuration
> +
> + For system using firewalld, do:
> + sudo firewall-cmd --permanent --add-port=427/udp
> + sudo firewall-cmd --permanent --add-port=427/tcp
> + sudo firewall-cmd --reload
> +
> + For openSUSE systems, do:
> + sudo /sbin/SuSEfirewall2 open EXT TCP 427
> + sudo /sbin/SuSEfirewall2 open EXT UDP 427
> +
> + For system using iptables, do:
> + sudo iptables -A INPUT -p tcp --dport 427 -j ACCEPT
> + sudo iptables -A INPUT -p udp --dport 427 -j ACCEPT
> +
> +3. In addition to the openSLP ports, you also need to allow multicast in the
> + firewall configuration
> +
> + For system using firewalld, do:
> + sudo firewall-cmd --direct --add-rule ipv4 filter INPUT 0 -s <subnet> -j ACCEPT
> +
> + For openSUSE systems, do:
> + Add the subnet to the trusted networks listed on FW_TRUSTED_NETS in
> + /etc/sysconfig/SuSEfirewall2 file.
> + Make sure to restart /sbin/SuSEfirewall2 after modifying /etc/sysconfig/SuSEfirewall2
> +
> + For system using iptables, do:
> + sudo iptables -A INPUT -s <subnet> -j ACCEPT
> +
> +4. Start slpd service and make sure it is up while running Wok
> + sudo service slpd start
> +
> +5. Enable federation on Wok by editing the /etc/wok/wok.conf file:
> +
> + federation = on
> +
> +6. Then start Wok service
> + sudo service wokd start
> +
> +The Wok server will be registered on openSLP on server starting up and will
> +be found by other Wok peers (with federation feature enabled) in the same
> +network.
> +
> +Enjoy!
> diff --git a/docs/wokd.8.in b/docs/wokd.8.in
> index 217d9f6..09b5847 100644
> --- a/docs/wokd.8.in
> +++ b/docs/wokd.8.in
> @@ -7,7 +7,7 @@ wokd - Launch Wok web server
> [\fB--cherrypy_port\fP \fICHERRYPY_PORT\fP] [\fB--websockets_port\fP \fIWEBSOCKETS_PORT\fP]
> [\fB--session_timeout\fP \fISESSION_TIMEOUT\fP] [\fB--log_level\fP \fILOG_LEVEL\fP]
> [\fB--log_dir\fP \fILOG_DIR\fP] [\fB--environment\fP \fIENV\fP]
> -[\fB--server_root\fP \fISERVER_ROOT\fP] [\fB--test\fP]
> +[\fB--server_root\fP \fISERVER_ROOT\fP] [\fB--federation\fP \fIfederation\fP] [\fB--test\fP]
> .SH DESCRIPTION
> \fBWok\fP is a cherrypy-based web framework with HTML5 support originated from Kimchi.
> It can be extended by plugins which expose functionality through REST APIs.
> @@ -55,6 +55,10 @@ Check cherrypy documentation for more details (default \fIproduction\fP).
> \fB\-\-server_root\fP [\fISERVER_ROOT\fP]
> Relative path to Wok server. No value is specified by default, ie, Wok will run on '/'.
> .TP
> +\fB\-\-federation\fP [\fIon\fP | \fIoff\fP]
> +Register and discover Wok peers in the same network using OpenSLP. Check
> +below the \fBREAD-federation\fP for more details (default \fIoff\fP).
> +.TP
> \fB\-\-test\fP
> Run Wok on a mock version that does not affect the system. For testing proposals.
> It depends on how plugins implements the mock environment as well.
> diff --git a/src/wok.conf.in b/src/wok.conf.in
> index 1ebdacf..1ed6310 100644
> --- a/src/wok.conf.in
> +++ b/src/wok.conf.in
> @@ -27,6 +27,10 @@
> # uncomment the following:
> #server_root=/wok
>
> +# Federation feature: register Wok server on openSLP and discover peers
> +# in the same network. Check README-federation for more details.
> +#federation = off
> +
> [logging]
> # Log directory
>
> diff --git a/src/wok/config.py.in b/src/wok/config.py.in
> index bbf2c09..39d7ad2 100644
> --- a/src/wok/config.py.in
> +++ b/src/wok/config.py.in
> @@ -277,6 +277,7 @@ def _get_config():
> config.set("server", "environment", "production")
> config.set('server', 'max_body_size', '4*1024*1024')
> config.set("server", "server_root", "")
> + config.set("server", "federation", "off")
> config.set("server", "test", "")
> config.add_section("authentication")
> config.set("authentication", "method", "pam")
> diff --git a/src/wok/control/peers.py b/src/wok/control/peers.py
> new file mode 100644
> index 0000000..2a14815
> --- /dev/null
> +++ b/src/wok/control/peers.py
> @@ -0,0 +1,30 @@
> +#
> +# Project Wok
> +#
> +# Copyright IBM Corp, 2017
> +#
> +# Code derived from Kimchi Project
> +#
> +# This library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +#
> +# This library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> +# Lesser General Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this library; if not, write to the Free Software
> +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> +
> +from wok.control.base import SimpleCollection
> +from wok.control.utils import UrlSubNode
> +
> +
> + at UrlSubNode("peers", True)
> +class Peers(SimpleCollection):
> + def __init__(self, model):
> + super(Peers, self).__init__(model)
> + self.admin_methods = ['GET']
> diff --git a/src/wok/model/config.py b/src/wok/model/config.py
> index b69f2dd..887ceba 100644
> --- a/src/wok/model/config.py
> +++ b/src/wok/model/config.py
> @@ -34,6 +34,7 @@ class ConfigModel(object):
> 'websockets_port': config.get('server', 'websockets_port'),
> 'auth': config.get('authentication', 'method'),
> 'server_root': config.get('server', 'server_root'),
> + 'federation': config.get('server', 'federation'),
> 'version': get_version()}
>
> def reload(self, name):
> diff --git a/src/wok/model/peers.py b/src/wok/model/peers.py
> new file mode 100644
> index 0000000..02afd59
> --- /dev/null
> +++ b/src/wok/model/peers.py
> @@ -0,0 +1,74 @@
> +#
> +# Project Wok
> +#
> +# Copyright IBM Corp, 2017
> +#
> +# Code derived from Kimchi Project
> +#
> +# This library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +#
> +# This library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> +# Lesser General Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this library; if not, write to the Free Software
> +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> +
> +import cherrypy
> +import re
> +import socket
> +
> +from wok.config import config
> +from wok.utils import run_command, wok_log
> +
> +
> +class PeersModel(object):
> + def __init__(self, **kargs):
> + # check federation feature is enabled on Wok server
> + if not config.get('server', 'federation') == 'off':
> + return
> +
> + # register server on openslp
> + hostname = socket.getfqdn()
> + port = config.get("server", "proxy_port")
> + self.url = hostname + ":" + port
> +
> + cmd = ["slptool", "register",
> + "service:wokd://%s" % self.url]
> + out, error, ret = run_command(cmd)
> + if out and len(out) != 0:
> + wok_log.error("Unable to register server on openSLP."
> + " Details: %s" % out)
> + cherrypy.engine.subscribe('exit', self._peer_deregister)
> +
> + def _peer_deregister(self):
> + cmd = ["slptool", "deregister",
> + "service:wokd://%s" % self.url]
> + out, error, ret = run_command(cmd)
> + if out and len(out) != 0:
> + wok_log.error("Unable to deregister server on openSLP."
> + " Details: %s" % out)
> +
> + def get_list(self):
> + # check federation feature is enabled on Wok server
> + if not config.get('server', 'federation') == 'off':
> + return []
> +
> + cmd = ["slptool", "findsrvs", "service:wokd"]
> + out, error, ret = run_command(cmd)
> + if ret != 0:
> + return []
> +
> + peers = []
> + for server in out.strip().split("\n"):
> + match = re.match("service:wokd://(.*?),.*", server)
> + peer = match.group(1)
> + if peer != self.url:
> + peers.append("https://" + peer)
> +
> + return peers
> diff --git a/src/wokd.in b/src/wokd.in
> index 29586a0..dfe85a8 100644
> --- a/src/wokd.in
> +++ b/src/wokd.in
> @@ -48,6 +48,7 @@ def main(options):
> websockets_port = config.config.get("server", "websockets_port")
> session_timeout = config.config.get("server", "session_timeout")
> runningEnv = config.config.get("server", "environment")
> + federation = config.config.get("server", "federation")
> server_root = config.config.get("server", "server_root")
> logDir = config.config.get("logging", "log_dir")
> logLevel = config.config.get("logging", "log_level")
> @@ -71,6 +72,10 @@ def main(options):
> help="Running environment of wok server")
> parser.add_option('--server_root', type="string", default=server_root,
> help="Relative path to server")
> + parser.add_option('--federation', default=federation,
> + help="Register and discover Wok peers in the same "
> + "network using openSLP. Check README-federation for"
> + " more details.")
> parser.add_option('--test', action='store_true',
> help="Run server in mock model")
> (options, args) = parser.parse_args()
> diff --git a/tests/test_api.py b/tests/test_api.py
> index 6fbee75..f978c93 100644
> --- a/tests/test_api.py
> +++ b/tests/test_api.py
> @@ -49,11 +49,15 @@ class APITests(unittest.TestCase):
> def setUp(self):
> self.request = partial(utils.request)
>
> + def test_peers(self):
> + resp = self.request('/peers').read()
> + self.assertEquals([], json.loads(resp))
> +
> def test_config(self):
> resp = self.request('/config').read()
> conf = json.loads(resp)
> keys = ["auth", "proxy_port", "websockets_port", "version",
> - "server_root"]
> + "server_root", "federation"]
> self.assertEquals(sorted(keys), sorted(conf.keys()))
>
> def test_config_plugins(self):
> diff --git a/tests/test_config_model.py b/tests/test_config_model.py
> index f8b0848..565b4df 100644
> --- a/tests/test_config_model.py
> +++ b/tests/test_config_model.py
> @@ -30,7 +30,7 @@ class ConfigModelTests(unittest.TestCase):
> config = inst.config_lookup('')
> self.assertItemsEqual(
> ['proxy_port', 'websockets_port', 'auth',
> - 'server_root', 'version'],
> + 'server_root', 'version', 'federation'],
> config.keys()
> )
>
> diff --git a/tests/test_server_root.py b/tests/test_server_root.py
> index e95a13b..89f34d3 100644
> --- a/tests/test_server_root.py
> +++ b/tests/test_server_root.py
> @@ -1,7 +1,7 @@
> #
> # Project Wok
> #
> -# Copyright IBM Corp, 2016
> +# Copyright IBM Corp, 2016-2017
> #
> # This library is free software; you can redistribute it and/or
> # modify it under the terms of the GNU Lesser General Public
> @@ -49,7 +49,7 @@ class ServerRootTests(unittest.TestCase):
> # check if server_root in config is the same used to start server
> resp = request(server_root + '/config').read()
> conf = json.loads(resp)
> - self.assertEquals(len(conf), 5)
> + self.assertEquals(len(conf), 6)
>
> def test_development_env(self):
> """
> @@ -61,4 +61,4 @@ class ServerRootTests(unittest.TestCase):
> # check if server_root in config is the same used to start server
> resp = request(server_root + '/config').read()
> conf = json.loads(resp)
> - self.assertEquals(len(conf), 5)
> + self.assertEquals(len(conf), 6)
> diff --git a/ui/js/src/wok.api.js b/ui/js/src/wok.api.js
> index 06b97aa..4e68eee 100644
> --- a/ui/js/src/wok.api.js
> +++ b/ui/js/src/wok.api.js
> @@ -71,6 +71,20 @@ var wok = {
> });
> },
>
> + getPeers: function(suc, err) {
> + wok.requestJSON({
> + url: 'peers',
> + type: 'GET',
> + contentType: 'application/json',
> + dataType: 'json',
> + resend: true,
> + success: suc,
> + error: err ? err : function(data) {
> + wok.message.error(data.responseJSON.reason);
> + }
> + });
> + },
> +
> getNotifications: function (suc, err) {
> wok.requestJSON({
> url: 'notifications',
> diff --git a/ui/js/wok.peers.js b/ui/js/wok.peers.js
> new file mode 100644
> index 0000000..67e58b0
> --- /dev/null
> +++ b/ui/js/wok.peers.js
> @@ -0,0 +1,75 @@
> +/*
> + * Project Wok
> + *
> + * Copyright IBM Corp, 2017
> + *
> + * Code derived from Kimchi Project
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +wok.initPeers = function() {
> + var peersDatatableTable;
> + var peers = new Array();
> +
> + var peersDatatable = function(nwConfigDataSet) {
> + peersDatatableTable = $('#peers-list').DataTable({
> + "processing": true,
> + "data": peers,
> + "language": {
> + "emptyTable": i18n['WOKSETT0010M']
> + },
> + "order": [],
> + "paging": false,
> + "dom": '<"row"<"col-sm-12"t>>',
> + "scrollY": "269px",
> + "scrollCollapse": true,
> + "columnDefs": [{
> + "targets": 0,
> + "searchable": false,
> + "orderable": false,
> + "width": "100%",
> + "className": "tabular-data",
> + "render": function(data, type, full, meta) {
> + return '<a href="' + data + '" target="_blank">' + data + '</a>';
> + }
> + }],
> + "initComplete": function(settings, json) {
> + $('#peers-content-area > .wok-mask').addClass('hidden');
> + }
> + });
> + };
> +
> + var getPeers = function() {
> + wok.getPeers(function(result) {
> + peers.length = 0;
> + for (var i = 0; i < result.length; i++) {
> + var tempArr = [];
> + tempArr.push(result[i]);
> + peers.push(tempArr);
> + }
> + peersDatatable(peers);
> + }, function(err) {
> + wok.message.error(err.responseJSON.reason, '#peers-alert-container', true);
> + });
> + };
> + getPeers();
> +
> +}
> +
> +if (wok.config.federation == 'on') {
> + $("#peers-accordion").removeClass('hidden');
> + wok.initPeers();
> +} else {
> + $("#peers-accordion").addClass('hidden');
> +}
> diff --git a/ui/pages/tabs/settings.html.tmpl b/ui/pages/tabs/settings.html.tmpl
> index ccc5b01..5f52839 100644
> --- a/ui/pages/tabs/settings.html.tmpl
> +++ b/ui/pages/tabs/settings.html.tmpl
> @@ -27,6 +27,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> <head>
> <link rel="stylesheet" type="text/css" href="$href('css/settings.css')">
> <script type="text/javascript" src="$href('js/wok.settings.js')"></script>
> + <script type="text/javascript" src="$href('js/wok.peers.js')"></script>
> <script type="text/javascript" src="$href('js/wok.bootgrid.js')"></script>
> </head>
>
> @@ -34,7 +35,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> <div id="wok-root-container" class="wok">
> <div class="container">
> <div id="wokSettings" class="wok-settings">
> - <!-- Plugins Management -->
> + <!-- Plugins Management -->
> <div class="panel-group accordion" id="plugins-mgmt-accordion" role="tablist" aria-multiselectable="false">
> <h3>
> <a role="button" data-toggle="collapse" data-parent="#plugin-mgmt-accordion" href="#plugins-mgmt-content-area" aria-expanded="true" aria-controls="plugins-mgmt-content-area" class="">
> @@ -65,6 +66,37 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> </div>
> </div>
> <!-- -->
> + <!-- Peers -->
> + <div class='panel-group federation-enabled accordion hidden' id='peers-accordion' role='tablist' aria-multiselectable='true'>
> + <h3>
> + <a role='button' data-toggle='collapse' data-parent='#peers-accordion' href='#peers-content-area' aria-expanded='false' aria-controls='peers-content-area' class=''>
> + <span class='accordion-icon'></span><span class='accordion-text' id='#peers-title'>$_("Peers")</span>
> + </a>
> + </h3>
> + <div id='peers-content-area' class='panel-collapse collapse' role='tabpanel' aria-labelledby='peers-title'>
> + <div id='peers-alert-container'></div>
> + <div class='row'>
> + <div class='col-sm-12'>
> + <table id='peers-list' class='table table-striped' cellspacing='0' width='100%'>
> + <thead>
> + <tr>
> + <th><span class='sr-only'>$_("Peers")</span></th>
> + </tr>
> + </thead>
> + </table>
> + </div>
> + </div>
> + <div class='wok-mask' role='presentation'>
> + <div class='wok-mask-loader-container'>
> + <div class='wok-mask-loading'>
> + <div class='wok-mask-loading-icon'></div>
> + <div class='wok-mask-loading-text'>$_("Loading...")</div>
> + </div>
> + </div>
> + </div>
> + </div>
> + </div>
> + <!-- -->
> </div>
> </div>
> </div>
More information about the Kimchi-devel
mailing list