[Kimchi-devel] [PATCH] [Wok] Add federation feature to Wok

Daniel Henrique Barboza dhbarboza82 at gmail.com
Wed May 3 12:37:52 UTC 2017


Reviewed-by: Daniel Barboza <danielhb at linux.vnet.ibm.com>

On 05/02/2017 01:59 PM, Aline Manera wrote:
> It was primarily implemented on Kimchi project. But it makes more sense
> to be on Wok since the idea is to allow user add/select on which server
> Wok should manage.
> For example, the user could have a Wok installation on server A to
> manage server B.
>
> Signed-off-by: Aline Manera <alinefm at linux.vnet.ibm.com>
> ---
>   docs/API/config.md               |  2 ++
>   docs/API/peers.md                | 20 +++++++++++
>   docs/README-federation.md        | 60 ++++++++++++++++++++++++++++++++
>   docs/wokd.8.in                   |  6 +++-
>   src/wok.conf.in                  |  4 +++
>   src/wok/config.py.in             |  1 +
>   src/wok/control/peers.py         | 30 ++++++++++++++++
>   src/wok/model/config.py          |  1 +
>   src/wok/model/peers.py           | 74 +++++++++++++++++++++++++++++++++++++++
>   src/wokd.in                      |  5 +++
>   tests/test_api.py                |  6 +++-
>   tests/test_config_model.py       |  2 +-
>   tests/test_server_root.py        |  6 ++--
>   ui/js/src/wok.api.js             | 14 ++++++++
>   ui/js/wok.peers.js               | 75 ++++++++++++++++++++++++++++++++++++++++
>   ui/pages/tabs/settings.html.tmpl | 34 +++++++++++++++++-
>   16 files changed, 333 insertions(+), 7 deletions(-)
>   create mode 100644 docs/API/peers.md
>   create mode 100644 docs/README-federation.md
>   create mode 100644 src/wok/control/peers.py
>   create mode 100644 src/wok/model/peers.py
>   create mode 100644 ui/js/wok.peers.js
>
> diff --git a/docs/API/config.md b/docs/API/config.md
> index d1d1007..0e47f5f 100644
> --- a/docs/API/config.md
> +++ b/docs/API/config.md
> @@ -12,6 +12,8 @@ Contains information about the application environment and configuration.
>       * proxy_port: SSL port to list on
>       * websockets_port: Port for websocket proxy to listen on
>       * auth: Authentication method used to log in to Wok
> +    * server_root: Relative path to Wok server. No value is specified by default, ie, Wok will run on '/'
> +    * federation: 'on' if federation feature is enabled, 'off' otherwise.
>       * version: Wok version
>   * **POST**: *See Task Actions*
>   
> diff --git a/docs/API/peers.md b/docs/API/peers.md
> new file mode 100644
> index 0000000..fa5cf41
> --- /dev/null
> +++ b/docs/API/peers.md
> @@ -0,0 +1,20 @@
> +## REST API Specification for Peers
> +
> +### Collection: Peers
> +
> +**URI:** /peers
> +
> +Return a list of Wok peers in the same network.
> +(It uses openSLP for discovering)
> +
> +**Methods:**
> +
> +* **GET**: Retrieve a list peers URLs.
> +
> +#### Examples
> +GET /peers
> +[
> + https://wok-peer0:8001,
> + https://wok-peer1:8001,
> + https://wok-peer2:8001,
> +]
> diff --git a/docs/README-federation.md b/docs/README-federation.md
> new file mode 100644
> index 0000000..18e19c7
> --- /dev/null
> +++ b/docs/README-federation.md
> @@ -0,0 +1,60 @@
> +Wok Project - Federation Feature
> +===================================
> +
> +Federation feature is a mechanism to discover Wok peers in the same
> +network. It uses openSLP tool (http://www.openslp.org/) to register and find Wok
> +servers.
> +
> +By default this feature is disabled on Wok as it is not critical for KVM
> +virtualization and requires additional software installation.
> +
> +To enable it, do the following:
> +
> +1. Install openslp and openslp-server rpm packages,
> +   or install slpd and slptool deb packages.
> +
> +2. openSLP uses port 427 (UDP) and port 427 (TCP) so make sure to open those
> +   ports in your firewall configuration
> +
> +   For system using firewalld, do:
> +   sudo firewall-cmd --permanent --add-port=427/udp
> +   sudo firewall-cmd --permanent --add-port=427/tcp
> +   sudo firewall-cmd --reload
> +
> +   For openSUSE systems, do:
> +   sudo /sbin/SuSEfirewall2 open EXT TCP 427
> +   sudo /sbin/SuSEfirewall2 open EXT UDP 427
> +
> +   For system using iptables, do:
> +   sudo iptables -A INPUT -p tcp --dport 427 -j ACCEPT
> +   sudo iptables -A INPUT -p udp --dport 427 -j ACCEPT
> +
> +3. In addition to the openSLP ports, you also need to allow multicast in the
> +   firewall configuration
> +
> +   For system using firewalld, do:
> +   sudo firewall-cmd --direct --add-rule ipv4 filter INPUT 0 -s <subnet> -j ACCEPT
> +
> +   For openSUSE systems, do:
> +   Add the subnet to the trusted networks listed on FW_TRUSTED_NETS in
> +   /etc/sysconfig/SuSEfirewall2 file.
> +   Make sure to restart /sbin/SuSEfirewall2 after modifying /etc/sysconfig/SuSEfirewall2
> +
> +   For system using iptables, do:
> +   sudo iptables -A INPUT -s <subnet> -j ACCEPT
> +
> +4. Start slpd service and make sure it is up while running Wok
> +   sudo service slpd start
> +
> +5. Enable federation on Wok by editing the /etc/wok/wok.conf file:
> +
> +   federation = on
> +
> +6. Then start Wok service
> +   sudo service wokd start
> +
> +The Wok server will be registered on openSLP on server starting up and will
> +be found by other Wok peers (with federation feature enabled) in the same
> +network.
> +
> +Enjoy!
> diff --git a/docs/wokd.8.in b/docs/wokd.8.in
> index 217d9f6..09b5847 100644
> --- a/docs/wokd.8.in
> +++ b/docs/wokd.8.in
> @@ -7,7 +7,7 @@ wokd - Launch Wok web server
>   [\fB--cherrypy_port\fP \fICHERRYPY_PORT\fP] [\fB--websockets_port\fP \fIWEBSOCKETS_PORT\fP]
>   [\fB--session_timeout\fP \fISESSION_TIMEOUT\fP] [\fB--log_level\fP \fILOG_LEVEL\fP]
>   [\fB--log_dir\fP \fILOG_DIR\fP] [\fB--environment\fP \fIENV\fP]
> -[\fB--server_root\fP \fISERVER_ROOT\fP] [\fB--test\fP]
> +[\fB--server_root\fP \fISERVER_ROOT\fP] [\fB--federation\fP \fIfederation\fP] [\fB--test\fP]
>   .SH DESCRIPTION
>   \fBWok\fP is a cherrypy-based web framework with HTML5 support originated from Kimchi.
>   It can be extended by plugins which expose functionality through REST APIs.
> @@ -55,6 +55,10 @@ Check cherrypy documentation for more details (default \fIproduction\fP).
>   \fB\-\-server_root\fP [\fISERVER_ROOT\fP]
>   Relative path to Wok server. No value is specified by default, ie, Wok will run on '/'.
>   .TP
> +\fB\-\-federation\fP [\fIon\fP | \fIoff\fP]
> +Register and discover Wok peers in the same network using OpenSLP. Check
> +below the \fBREAD-federation\fP for more details (default \fIoff\fP).
> +.TP
>   \fB\-\-test\fP
>   Run Wok on a mock version that does not affect the system. For testing proposals.
>   It depends on how plugins implements the mock environment as well.
> diff --git a/src/wok.conf.in b/src/wok.conf.in
> index 1ebdacf..1ed6310 100644
> --- a/src/wok.conf.in
> +++ b/src/wok.conf.in
> @@ -27,6 +27,10 @@
>   # uncomment the following:
>   #server_root=/wok
>   
> +# Federation feature: register Wok server on openSLP and discover peers
> +# in the same network. Check README-federation for more details.
> +#federation = off
> +
>   [logging]
>   # Log directory
>   
> diff --git a/src/wok/config.py.in b/src/wok/config.py.in
> index bbf2c09..39d7ad2 100644
> --- a/src/wok/config.py.in
> +++ b/src/wok/config.py.in
> @@ -277,6 +277,7 @@ def _get_config():
>       config.set("server", "environment", "production")
>       config.set('server', 'max_body_size', '4*1024*1024')
>       config.set("server", "server_root", "")
> +    config.set("server", "federation", "off")
>       config.set("server", "test", "")
>       config.add_section("authentication")
>       config.set("authentication", "method", "pam")
> diff --git a/src/wok/control/peers.py b/src/wok/control/peers.py
> new file mode 100644
> index 0000000..2a14815
> --- /dev/null
> +++ b/src/wok/control/peers.py
> @@ -0,0 +1,30 @@
> +#
> +# Project Wok
> +#
> +# Copyright IBM Corp, 2017
> +#
> +# Code derived from Kimchi Project
> +#
> +# This library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +#
> +# This library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this library; if not, write to the Free Software
> +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301 USA
> +
> +from wok.control.base import SimpleCollection
> +from wok.control.utils import UrlSubNode
> +
> +
> + at UrlSubNode("peers", True)
> +class Peers(SimpleCollection):
> +    def __init__(self, model):
> +        super(Peers, self).__init__(model)
> +        self.admin_methods = ['GET']
> diff --git a/src/wok/model/config.py b/src/wok/model/config.py
> index b69f2dd..887ceba 100644
> --- a/src/wok/model/config.py
> +++ b/src/wok/model/config.py
> @@ -34,6 +34,7 @@ class ConfigModel(object):
>                   'websockets_port': config.get('server', 'websockets_port'),
>                   'auth': config.get('authentication', 'method'),
>                   'server_root': config.get('server', 'server_root'),
> +                'federation': config.get('server', 'federation'),
>                   'version': get_version()}
>   
>       def reload(self, name):
> diff --git a/src/wok/model/peers.py b/src/wok/model/peers.py
> new file mode 100644
> index 0000000..02afd59
> --- /dev/null
> +++ b/src/wok/model/peers.py
> @@ -0,0 +1,74 @@
> +#
> +# Project Wok
> +#
> +# Copyright IBM Corp, 2017
> +#
> +# Code derived from Kimchi Project
> +#
> +# This library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +#
> +# This library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this library; if not, write to the Free Software
> +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301 USA
> +
> +import cherrypy
> +import re
> +import socket
> +
> +from wok.config import config
> +from wok.utils import run_command, wok_log
> +
> +
> +class PeersModel(object):
> +    def __init__(self, **kargs):
> +        # check federation feature is enabled on Wok server
> +        if not config.get('server', 'federation') == 'off':
> +            return
> +
> +        # register server on openslp
> +        hostname = socket.getfqdn()
> +        port = config.get("server", "proxy_port")
> +        self.url = hostname + ":" + port
> +
> +        cmd = ["slptool", "register",
> +               "service:wokd://%s" % self.url]
> +        out, error, ret = run_command(cmd)
> +        if out and len(out) != 0:
> +            wok_log.error("Unable to register server on openSLP."
> +                          " Details: %s" % out)
> +        cherrypy.engine.subscribe('exit', self._peer_deregister)
> +
> +    def _peer_deregister(self):
> +        cmd = ["slptool", "deregister",
> +               "service:wokd://%s" % self.url]
> +        out, error, ret = run_command(cmd)
> +        if out and len(out) != 0:
> +            wok_log.error("Unable to deregister server on openSLP."
> +                          " Details: %s" % out)
> +
> +    def get_list(self):
> +        # check federation feature is enabled on Wok server
> +        if not config.get('server', 'federation') == 'off':
> +            return []
> +
> +        cmd = ["slptool", "findsrvs", "service:wokd"]
> +        out, error, ret = run_command(cmd)
> +        if ret != 0:
> +            return []
> +
> +        peers = []
> +        for server in out.strip().split("\n"):
> +            match = re.match("service:wokd://(.*?),.*", server)
> +            peer = match.group(1)
> +            if peer != self.url:
> +                peers.append("https://" + peer)
> +
> +        return peers
> diff --git a/src/wokd.in b/src/wokd.in
> index 29586a0..dfe85a8 100644
> --- a/src/wokd.in
> +++ b/src/wokd.in
> @@ -48,6 +48,7 @@ def main(options):
>       websockets_port = config.config.get("server", "websockets_port")
>       session_timeout = config.config.get("server", "session_timeout")
>       runningEnv = config.config.get("server", "environment")
> +    federation = config.config.get("server", "federation")
>       server_root = config.config.get("server", "server_root")
>       logDir = config.config.get("logging", "log_dir")
>       logLevel = config.config.get("logging", "log_level")
> @@ -71,6 +72,10 @@ def main(options):
>                         help="Running environment of wok server")
>       parser.add_option('--server_root', type="string", default=server_root,
>                         help="Relative path to server")
> +    parser.add_option('--federation', default=federation,
> +                      help="Register and discover Wok peers in the same "
> +                           "network using openSLP. Check README-federation for"
> +                           " more details.")
>       parser.add_option('--test', action='store_true',
>                         help="Run server in mock model")
>       (options, args) = parser.parse_args()
> diff --git a/tests/test_api.py b/tests/test_api.py
> index 6fbee75..f978c93 100644
> --- a/tests/test_api.py
> +++ b/tests/test_api.py
> @@ -49,11 +49,15 @@ class APITests(unittest.TestCase):
>       def setUp(self):
>           self.request = partial(utils.request)
>   
> +    def test_peers(self):
> +        resp = self.request('/peers').read()
> +        self.assertEquals([], json.loads(resp))
> +
>       def test_config(self):
>           resp = self.request('/config').read()
>           conf = json.loads(resp)
>           keys = ["auth", "proxy_port", "websockets_port", "version",
> -                "server_root"]
> +                "server_root", "federation"]
>           self.assertEquals(sorted(keys), sorted(conf.keys()))
>   
>       def test_config_plugins(self):
> diff --git a/tests/test_config_model.py b/tests/test_config_model.py
> index f8b0848..565b4df 100644
> --- a/tests/test_config_model.py
> +++ b/tests/test_config_model.py
> @@ -30,7 +30,7 @@ class ConfigModelTests(unittest.TestCase):
>           config = inst.config_lookup('')
>           self.assertItemsEqual(
>               ['proxy_port', 'websockets_port', 'auth',
> -             'server_root', 'version'],
> +             'server_root', 'version', 'federation'],
>               config.keys()
>           )
>   
> diff --git a/tests/test_server_root.py b/tests/test_server_root.py
> index e95a13b..89f34d3 100644
> --- a/tests/test_server_root.py
> +++ b/tests/test_server_root.py
> @@ -1,7 +1,7 @@
>   #
>   # Project Wok
>   #
> -# Copyright IBM Corp, 2016
> +# Copyright IBM Corp, 2016-2017
>   #
>   # This library is free software; you can redistribute it and/or
>   # modify it under the terms of the GNU Lesser General Public
> @@ -49,7 +49,7 @@ class ServerRootTests(unittest.TestCase):
>           # check if server_root in config is the same used to start server
>           resp = request(server_root + '/config').read()
>           conf = json.loads(resp)
> -        self.assertEquals(len(conf), 5)
> +        self.assertEquals(len(conf), 6)
>   
>       def test_development_env(self):
>           """
> @@ -61,4 +61,4 @@ class ServerRootTests(unittest.TestCase):
>           # check if server_root in config is the same used to start server
>           resp = request(server_root + '/config').read()
>           conf = json.loads(resp)
> -        self.assertEquals(len(conf), 5)
> +        self.assertEquals(len(conf), 6)
> diff --git a/ui/js/src/wok.api.js b/ui/js/src/wok.api.js
> index 06b97aa..4e68eee 100644
> --- a/ui/js/src/wok.api.js
> +++ b/ui/js/src/wok.api.js
> @@ -71,6 +71,20 @@ var wok = {
>           });
>       },
>   
> +    getPeers: function(suc, err) {
> +        wok.requestJSON({
> +            url: 'peers',
> +            type: 'GET',
> +            contentType: 'application/json',
> +            dataType: 'json',
> +            resend: true,
> +            success: suc,
> +            error: err ? err : function(data) {
> +                wok.message.error(data.responseJSON.reason);
> +            }
> +        });
> +    },
> +
>       getNotifications: function (suc, err) {
>           wok.requestJSON({
>               url: 'notifications',
> diff --git a/ui/js/wok.peers.js b/ui/js/wok.peers.js
> new file mode 100644
> index 0000000..67e58b0
> --- /dev/null
> +++ b/ui/js/wok.peers.js
> @@ -0,0 +1,75 @@
> +/*
> + * Project Wok
> + *
> + * Copyright IBM Corp, 2017
> + *
> + * Code derived from Kimchi Project
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at
> + *
> + *     http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +wok.initPeers = function() {
> +    var peersDatatableTable;
> +    var peers = new Array();
> +
> +    var peersDatatable = function(nwConfigDataSet) {
> +        peersDatatableTable = $('#peers-list').DataTable({
> +            "processing": true,
> +            "data": peers,
> +            "language": {
> +                "emptyTable": i18n['WOKSETT0010M']
> +            },
> +            "order": [],
> +            "paging": false,
> +            "dom": '<"row"<"col-sm-12"t>>',
> +            "scrollY": "269px",
> +            "scrollCollapse": true,
> +            "columnDefs": [{
> +                "targets": 0,
> +                "searchable": false,
> +                "orderable": false,
> +                "width": "100%",
> +                "className": "tabular-data",
> +                "render": function(data, type, full, meta) {
> +                    return '<a href="' + data + '" target="_blank">' + data + '</a>';
> +                }
> +            }],
> +            "initComplete": function(settings, json) {
> +                $('#peers-content-area > .wok-mask').addClass('hidden');
> +            }
> +        });
> +    };
> +
> +    var getPeers = function() {
> +        wok.getPeers(function(result) {
> +            peers.length = 0;
> +            for (var i = 0; i < result.length; i++) {
> +                var tempArr = [];
> +                tempArr.push(result[i]);
> +                peers.push(tempArr);
> +            }
> +            peersDatatable(peers);
> +        }, function(err) {
> +            wok.message.error(err.responseJSON.reason, '#peers-alert-container', true);
> +        });
> +    };
> +    getPeers();
> +
> +}
> +
> +if (wok.config.federation == 'on') {
> +    $("#peers-accordion").removeClass('hidden');
> +    wok.initPeers();
> +} else {
> +    $("#peers-accordion").addClass('hidden');
> +}
> diff --git a/ui/pages/tabs/settings.html.tmpl b/ui/pages/tabs/settings.html.tmpl
> index ccc5b01..5f52839 100644
> --- a/ui/pages/tabs/settings.html.tmpl
> +++ b/ui/pages/tabs/settings.html.tmpl
> @@ -27,6 +27,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
>   <head>
>     <link rel="stylesheet" type="text/css" href="$href('css/settings.css')">
>     <script type="text/javascript" src="$href('js/wok.settings.js')"></script>
> +  <script type="text/javascript" src="$href('js/wok.peers.js')"></script>
>     <script type="text/javascript" src="$href('js/wok.bootgrid.js')"></script>
>   </head>
>   
> @@ -34,7 +35,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
>     <div id="wok-root-container" class="wok">
>       <div class="container">
>         <div id="wokSettings" class="wok-settings">
> -      <!-- Plugins Management -->
> +        <!-- Plugins Management -->
>           <div class="panel-group accordion" id="plugins-mgmt-accordion" role="tablist" aria-multiselectable="false">
>             <h3>
>               <a role="button" data-toggle="collapse" data-parent="#plugin-mgmt-accordion" href="#plugins-mgmt-content-area" aria-expanded="true" aria-controls="plugins-mgmt-content-area" class="">
> @@ -65,6 +66,37 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
>             </div>
>           </div>
>           <!-- -->
> +        <!-- Peers -->
> +        <div class='panel-group federation-enabled accordion hidden' id='peers-accordion' role='tablist' aria-multiselectable='true'>
> +          <h3>
> +            <a role='button' data-toggle='collapse' data-parent='#peers-accordion' href='#peers-content-area' aria-expanded='false' aria-controls='peers-content-area' class=''>
> +              <span class='accordion-icon'></span><span class='accordion-text' id='#peers-title'>$_("Peers")</span>
> +            </a>
> +          </h3>
> +          <div id='peers-content-area' class='panel-collapse collapse' role='tabpanel' aria-labelledby='peers-title'>
> +            <div id='peers-alert-container'></div>
> +            <div class='row'>
> +              <div class='col-sm-12'>
> +                <table id='peers-list' class='table table-striped' cellspacing='0' width='100%'>
> +                  <thead>
> +                    <tr>
> +                      <th><span class='sr-only'>$_("Peers")</span></th>
> +                    </tr>
> +                  </thead>
> +                </table>
> +              </div>
> +            </div>
> +            <div class='wok-mask' role='presentation'>
> +              <div class='wok-mask-loader-container'>
> +                <div class='wok-mask-loading'>
> +                  <div class='wok-mask-loading-icon'></div>
> +                  <div class='wok-mask-loading-text'>$_("Loading...")</div>
> +                </div>
> +              </div>
> +            </div>
> +          </div>
> +        </div>
> +        <!-- -->
>         </div>
>       </div>
>     </div>



More information about the Kimchi-devel mailing list