[ovirt-devel] Killing XMLRPC on master

Wed Jun 22 10:13:00 UTC 2016

> It is not about having a topic but listing what should be posted to that
> topic.

That is the whole point of having a hierarchical topic based system.
Engine would listen for anything coming from vdsm, vdsm would listen
for anything that comes from engine using a path wildcard.

Just use a /verb/<verbName> topic for each verb command automatically
and either the same or something like
/response/<verbName>/<correlation id> for the answer.

There is no reason to distinguish verbs and events. Both are events
and the only difference is whether the initiating party is waiting for
a response. You need some kind of correlation id for that of course
(to identify the specific answer you are waiting for).

> I am not sure whether it is good idea for sever to know
> what should be send where.

The sender always decides what the topic it is. At least in the rest
of the broker universe. There is no such thing as a server or client,
just publisher and listener for specific message topic wildcard.

> Sending to multiple topics is there but selecting a subset to send to one of
> them

But you never do that, you send to one topic (full path) only and the
listener decides what is interesting enough. The tree structure allows
that (not sure about our implementation, but all standard brokers
support this):

/verb/runVm will be received by anybody listening for that or for
/verb/* or for /*.

The same for /response/runVm/16846158. MOM or hosted engine does not
care about which exact command initiated the response and can listen
for /response/runVm/*. On the other hand, the engine can be blocked in
the call until the right response arrives.

Martin

On Wed, Jun 22, 2016 at 12:00 PM, Piotr Kliczewski <pkliczew at redhat.com> wrote:
>
>
> On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <msivak at redhat.com> wrote:
>>
>> > If we subscribe to engine response queue we receive all the traffic. At
>> > the
>> > moment there is no way
>> > to define verbs we listen to. This functionality is only for events.
>>
>> Can't we create a new topic /engine/<verb> for each verb and configure
>> vdsm to listen for /engine/*? That would duplicate what we have now,
>> but MOM would be able to subscribe more selectively.
>>
>
> It is not about having a topic but listing what should be posted to that
> topic.
> Sending to multiple topics is there but selecting a subset to send to one of
> them
> would need to be added. I am not sure whether it is good idea for sever to
> know
> what should be send where.
>
>>
>> Martin
>>
>>
>> On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski <pkliczew at redhat.com>
>> wrote:
>> >
>> >
>> > On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek
>> > <michal.skrivanek at redhat.com> wrote:
>> >>
>> >>
>> >> On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew at redhat.com> wrote:
>> >>
>> >>
>> >>
>> >> On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer at redhat.com> wrote:
>> >>>
>> >>> On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak at redhat.com>
>> >>> wrote:
>> >>> >> 1. Mom is still using xmlrpc
>> >>> >>
>> >>> >> Mom must move to jsonrpc.
>> >>> >> Martin: can you update on progress of this work?
>> >>> >
>> >>> > We would like to avoid going through VDSM completely, except from
>> >>> > the
>> >>> > broker part. Is it possible now to "parasitically" listen to vdsm
>> >>> > events and engine commands without having to go through VDSM API?
>> >>>
>> >>> Sounds like a good plan.
>> >>
>> >>
>> >> bypassing vdsm for mom things is a good idea, but we need to make sure
>> >> we’re not killing the rest of the system
>> >>
>> >>>
>> >>> Piotr, how far are we from letting mom listen to engine queue so it
>> >>> get
>> >>> engine
>> >>> events/responses for certain verbs?
>> >>
>> >>
>> >> We can do it now for both events and responses. We need to remember
>> >> that
>> >> we would
>> >> receive anything that the engine is receiving. We could do better but
>> >> that
>> >> it would take
>> >> more time.
>> >>
>> >>
>> >> so it would receive all responses for only the defined verbs?
>> >> It would wok well for getAllVmStats
>> >>
>> >
>> > If we subscribe to engine response queue we receive all the traffic. At
>> > the
>> > moment there is no way
>> > to define verbs we listen to. This functionality is only for events.
>> >
>> >>
>> >>
>> >>>
>> >>>
>> >>> > You can drop XML RPC,. vdsm does not depend on MOM working anymore.
>> >>> > Just the balloon and ksm stats will be missing from data that are
>> >>> > being sent to the engine.
>> >>
>> >>
>> >> and a lot of things stops working then;)
>> >>
>> >
>> > Maybe we can disable [1] it before dropping it. We can slowly fix master
>> > and
>> > if needed we can enable it for
>> > specific envs.
>> >
>> > [1] https://gerrit.ovirt.org/#/c/59172/
>> >>>
>> >>>
>> >>> Balloon and ksm stats are sent today from mom to vdsm, and reported by
>> >>> vdsm to engine?
>> >>>
>> >>> If vdsm is only a middleman  and does not use this info, best to send
>> >>> it directly
>> >>> to engine via the stomp broker part of vdsm.
>> >>>
>> >>> Piotr, how far are we from having engine listening for mom events
>> >>> using
>> >>> the
>> >>> vdsm broker?
>> >>
>> >>
>> >> I don’t think we should move any monitoring pulled from libvirt into
>> >> mom -
>> >> it would need another expensive and problematic call
>> >>
>> >>
>> >> It can be used now but the engine needs to know to listen for them. We
>> >> would
>> >> need to implement engine subscriber (one class).
>> >>
>> >>>
>> >>>
>> >>> > There also were some issues with eventfd in the json library, I
>> >>> > assume
>> >>> > those are fixed now.
>> >>>
>> >>> They should be fixed here:
>> >>> https://gerrit.ovirt.org/57942
>> >>>
>> >>> But suffering from these issue show that the api was not use properly,
>> >>> you should create one client and reuse it for the entire life of the
>> >>> application.
>> >>>
>> >>>
>> >>> Nir
>> >>>
>> >>> >
>> >>> > Martin
>> >>> >
>> >>> > On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer at redhat.com>
>> >>> > wrote:
>> >>> >> Hi all,
>> >>> >>
>> >>> >> We are still wasting time on maintaining both xmlrpc and jsonrpc.
>> >>> >> If
>> >>> >> we kill
>> >>> >> xmlrpc, we can greatly simplify the code base, making it easier to
>> >>> >> mainain
>> >>> >> and add new features.
>> >>> >>
>> >>> >> I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
>> >>> >>
>> >>> >> Currently the we have 3 issues:
>> >>> >>
>> >>> >> 1. Mom is still using xmlrpc
>> >>> >>
>> >>> >> Mom must move to jsonrpc.
>> >>> >> Martin: can you update on progress of this work?
>> >>> >>
>> >>> >> 2. sos plugin using vdsClient
>> >>> >>
>> >>> >> Need to port it to use jsonrpc library, or jsonrpc client
>> >>> >>
>> >>> >> New jsonrpc client:  https://gerrit.ovirt.org/35181
>> >>> >>
>> >>> >> 3. Engine is using xmlrpc server for ovf upload/download
>> >>> >>
>> >>> >> We must support current engine in 4.1, so we cannot remove
>> >>> >> upload/download feature in this version, but we can remove the
>> >>> >> xmlrpc support in this server.
>> >>> >>
>> >>> >> Currently we abuse the xmlrpc server, supporting PUT and GET for
>> >>> >> upload and download (XMLRPC is using only POST). We can disable
>> >>> >> POST requests in protocoldetector, and not register anything with
>> >>> >> the xmlrpc server.
>> >>> >>
>> >>> >> Thoughts?
>> >>> >>
>> >>> >> Nir
>> >>
>> >>
>> >> _______________________________________________
>> >> Devel mailing list
>> >> Devel at ovirt.org
>> >> http://lists.ovirt.org/mailman/listinfo/devel
>> >>
>> >>
>> >
>
>