[Kimchi-devel] [RFC] asynchronous (websocket based) notification (Wok #16)

[Aline Manera]

On 02/13/2017 02:29 PM, Daniel Henrique Barboza wrote:
>
>
> On 02/13/2017 12:47 PM, Aline Manera wrote:
>> Hi Daniel,
>>
>> On 02/08/2017 05:07 PM, Daniel Henrique Barboza wrote:
>>> Hi everyone,
>>>
>>> I want to start a discussion on 
>>> https://github.com/kimchi-project/wok/issues/16,
>>> "Asynchronous event notification".
>>>
>>> As most of you are aware, WoK does not have any sort of push 
>>> notification. WoK
>>> UI works based on a polling strategy using the Notifications API to 
>>> fetch for any
>>> backend notifications. This polling is on a 3 second interval to not 
>>> overload the server
>>> with these requests. The problems with this approach are obvious: 
>>> the cost of
>>> the polling process for both UI and backend, the interval for a 
>>> backend event to be
>>> delivered to the UI and so forth.
>>>
>>> - The idea
>>>
>>> This work aims to implement an asynchronous strategy to deliver 
>>> server to client
>>> messages. The idea is to use websockets* to establish a socket 
>>> connection between
>>> the UI and the backend. The backend can send any message using this 
>>> socket and
>>> the UI, after receiving it, can act upon immediately.
>>>
>>>
>>> - Push server implementation
>>>
>>> This socket in the backend side would act as a 'push server' that 
>>> will receive the
>>> connections and push the same messages to all of them. Only server 
>>> to client
>>> messages will be sent.
>>>
>>> This push server can be implemented in two ways:
>>>
>>> * from scratch
>>>
>>> * using an external library
>>>
>>> One library that seems to do this asynchronous socket implementation 
>>> is tornado
>>> ( https://github.com/tornadoweb/tornado ). It is present in all 
>>> distros we support
>>> and it has Apache 2.0 licensing. I'll experiment with it and see if 
>>> it helps. I am
>>> opened to any other suggestion of libraries that can be used in the 
>>> push server
>>> implementation. If no library is good enough for us, I'll have to 
>>> implement it from
>>> scratch.
>>>
>>
>> I am OK in using it as it is available for Fedora 25, Ubuntu 16.10, 
>> openSUSE 42.2 and centOS 7.
>>
>> It would be good to confirm it is also available for Debian 8 as 
>> Lucio is working to get the package into the official distro.
>>
>>>
>>> - WoK backend design
>>>
>>> In WoK backend, my idea is to reuse the 'add_notification' method 
>>> from the
>>> existing Notifications API. When adding a notification, fire a 
>>> message to the
>>> push server and notify all the listeners too.
>>>
>>
>> Could you elaborate more on that?
>> For example, in Kimchi, after adding/starting/deleting a VM we will 
>> need to notify all the browsers sessions to update the data in UI.
>> Or on Wok, after enabling/disabling a plugin.
>>
>> How will that be done with the notifications API?
>>
>> From the documentation, the notifications API is as below:
>>
>> * **GET**: Retrieve the full description of the Notification
>>     * code: message ID
>>     * message: message text already translated
>>     * timestamp: first time notification was emitted
>>
>>> This approach has the following advantages:
>>>
>>> - it will work out of the box for all backend messages in all 
>>> plug-ins that
>>> uses the 'add_notification' method;
>>>
>>
>> Today, the notifications API is only shown on UI as a warning message 
>> to the user.
>> So I think much UI changes will be required.
>>
>>> - we can re-use the same JSON message format of the Notifications API,
>>> reducing the amount of UI work we'll have to adapt the existing UIs;
>>>
>>
>> Same I commented above.
>>
>>> - it will be harmless to implement. Given that the push server will send
>>> messages to all connected UI endpoints, if no endpoint is connect
>>> no message will be sent.
>>>
>>>
>>> - WoK frontend design
>>>
>>> For any tab that wants to receive the push notifications, just connect
>>> to the push server via websocket and react to the messages sent - just
>>> like it is done today with the notifications API but without the need of
>>> sending the GET /notifications messages.
>>>
>>> We will need to be careful to not open unnecessary websockets when
>>> tab switching. We will need to pay attention to closing up the 
>>> connections
>>> we don't need anymore.
>>>
>>
>> Why do not open just one on Wok and reuse for all the plugins?
>
> Imagine that a single socket 'wokchannel' is created. The plug-ins would
> need to register in this channel to see the incoming messages. Now, 
> imagine
> a kimchi function "refreshVMsList" that would refresh the VMs in the 
> listing.
> While in the Guest tab the function would work as intended. However, when
> going to any other tab, "refreshVMList" will start to fail because the 
> elements
> the function references will no longer exists.
>
> There is a Ginger bugs I've fixed in the past with this scenario:
> https://github.com/kimchi-project/ginger/issues/482 . In this bug, the
> ginger.getSensorsInput were being called out of the Administration 
> tab, causing
> JS errors.
>
> The solution in this case would be to remove the listeners when 
> switching to
> a new tab. Add the listener when the tab is created, remove it when it is
> no longer needed. This is the exact same behavior I need to implement with
> the Websockets.
>
> Thus, I don't see any obvious gain or simplification by having only 
> one connection
> and add/remove listeners versus one connection for each tab.
>

Hrm... I was thinking in something like:

1) The plugins would register a listener globally

On wok.main.js:

wok.notificationsListeners = {}

On kimchi.main.js:

wok.notificationsListeners['<code-to-list-vms>'] = <functions-to-list-vms>

2) Reuse the notifications listener we have today and use the "code" to 
proper treat it.

Let's call it handleNotification() which will do

{
     // One notification arrived
     // Gets its code and redirect to the proper listener
     func = wok.notificationsListeners.get(<code>)
     if func !== undefined
         // call func
         func();
}

So that way, I don't think the plugins need to unregister a listener 
when switching tabs.
Wok would proper gets the logic to do that avoiding errors.

>
> Daniel
>
>>
>>> I am planning to do a proof of concept of an UI working with this new
>>> push server notifications in the 'User Log' tab, together with this 
>>> backend
>>> work. When a new log entry is created, a push notification is sent 
>>> and the
>>> UI would refresh automatically. This implementation would be used as 
>>> a base
>>> for the other tabs/plug-ins.
>>>
>>>
>>> Let me know what you think!
>>>
>>>
>>> Daniel
>>>
>>>
>>> PS: for the record, before deciding to use websockets I've 
>>> considered using SSE
>>> (Server-side Events), a HTML5 standard, but gave up due to lack of 
>>> SSE support
>>> from Microsoft browsers.
>>>
>>>
>>> _______________________________________________
>>> Kimchi-devel mailing list
>>> Kimchi-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/kimchi-devel
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/kimchi-devel/attachments/20170214/4fa5f611/attachment.html>