[Engine-devel] [rhevm-api] Event notifications in RHEVM 3

[Itamar Heim]

On 11/25/2011 02:48 PM, David Curylo wrote:
> In our use case, we are trying to get real time notifications of events into our own application tier, so there would only be one or two of our application servers connected at a time.  It need not be REST or HTTP - a persistent TCP connection will work just as well for us, the importance is that we are notified as quickly as possible.  We have a similar need for real time notifications throughout our system and our architecture can be summarized as follows:
>
> Application server(s) -->  AMQP topic exchange -->  AMQP queue topic binding -->  Web server(s)
>
> I will share more of our usage scenarios and a little about our architecture as it pertains to the discussion:
>
> When our application tier is notified of events, it performs other actions.  For example, for Red Hat certification, we track the power off and power on events so we can provide a report of OS usage - polling every minute or so would work fine here.  We also need to notify server monitoring systems that a system was powered off so they can stop monitoring rather than alerting of an outage - in this case, we need to be notified ASAP.  Additionally, we need to notify any connected clients that a system has changed state (any event).  Those clients are connected to web servers, which we notify by a message queuing infrastructure.  All events are published to an AMQP topic exchange, to which web servers (and other systems) may subscribe based on their connected client's needs.  The transport and scalability requirements no longer need to be handled by our application tier.
>
> The web servers, in the end, have to support a lot of client scenarios that include polling, long polling, sending HTTP requests, maintaining a TCP channel for streaming events, etc.  We can easily scale up web servers, and we need to provide transports that are suitable for whatever our clients may need.
>
> I hope this helps explain our usage scenarios and suggest an additional technology to consider.

(moving the discussion to the new upstream mailing list)
do you care about all events, or specific ones?
the event notifier is about sending registered events per user.

maybe extend it to send the amqp events directly / extend the list of 
events it cares about?


>
> -----Original Message-----
> From: Yaniv Kaul [mailto:ykaul at redhat.com]
> Sent: Thursday, November 24, 2011 8:56 AM
> To: Geert Jansen
> Cc: Itamar Heim; rhevm-api at lists.fedorahosted.org; David Curylo
> Subject: Re: [rhevm-api] Event notifications in RHEVM 3
>
> On 11/24/2011 03:14 PM, Geert Jansen wrote:
>>
>> On 11/24/2011 01:18 PM, Itamar Heim wrote:
>>> On 11/24/2011 01:37 PM, David Curylo wrote:
>>>> I don't know the capabilities of this daemon. Can it do other notifications besides email, such as execute curl to send a request to my system?  I would like to get notifications as close to real time as possible.
>>> I think it's every 5 minutes right now, but can be configured (in 3.0).
>> I don't believe though, that out notification service suits this use
>> case. The use case is to get almost real-time notifications via an API
>> when a change has been made.
>>
>> In my view, the best way to do this in HTTP/REST, is so-called long
>> polling. It's not perfect, and not terribly scalable (i takes up one
>> TCP connection, and in the simplest implementation, one app server slot).
>> But on the other side, i do not see a need for 100s of clients polling
>> a single RHEV systems. Typically, a customer may have 1-2 ISV packages
>> on top of RHEV that need events. In my view, 10s of clients should be
>> easily doable with long polling.
>
> - A single persistent TCP connection is "cheaper" to everyone than open-request-response-close-open-request-response-close ....
> - This is widely used over HTTP with "100 continue" messages that never actually complete with a 200. It's neat and takes very little bandwidth, actually.
> Y.
>
>>
>> If we want to scale higher than this, probably a different
>> architecture like e.g. a pub/sub architecture using a protocol
>> optimized for binary messaging would make sense. But again, for now, i
>> have not seen this need so for 3.1 i think we need to focus on the
>> simplest solution that will satisfy 95% of the requirements.
>>
>> Regards,
>> Geert
>> _______________________________________________
>> rhevm-api mailing list
>> rhevm-api at lists.fedorahosted.org
>> https://fedorahosted.org/mailman/listinfo/rhevm-api
>