[ovirt-devel] Killing XMLRPC on master

Wed Jun 22 12:04:01 UTC 2016

On Wed, Jun 22, 2016 at 1:22 PM, Martin Sivak <msivak at redhat.com> wrote:

> > We do not have hierarchical topic system. Please note that we have
> > p2p communication and real broker is not part of the scope anymore.
> > We only have mini broker implemented in vdsm.
>
> But then I can't implement the parasitic way of listening to data.
> This is basic functionality when you want to call something "a
> broker". Even a mini one...
>
>
We were implementing the code base on jsonrpc 2.0 and stomp 1.2 specs.
None of them consider it basic.

> > There is distinction based on historical reasons.
>
> > Please remember that we are not message based but we have 2 logical
> > ways of dealing with network frames:
> > - rpc
> > - events (aka messages)
>
> Which we can easily emulate using events only for anything that needs
> it... everybody does that, because everybody needs blocking calls
> sometimes. But we seem to lack a very basic infrastructure to even
> attempt that. And I seem to remember telling you this same thing last
> year in Seattle :/
>
>
I disagree with: "everybody needs blocking calls sometimes"

>
>
> Knowing all this, I can drop XML-RPC from MOM and issue the exact same
> calls via JSON-RPC, but we will get nothing by that. Except maybe
> delete some protocol detector code... The load and vdsm dependencies
> will still stay the same.
>
>
This is my point we want to drop xmlrpc functionality.

>
> Martin
>
> On Wed, Jun 22, 2016 at 12:47 PM, Piotr Kliczewski <pkliczew at redhat.com>
> wrote:
> >
> >
> > On Wed, Jun 22, 2016 at 12:13 PM, Martin Sivak <msivak at redhat.com>
> wrote:
> >>
> >> > It is not about having a topic but listing what should be posted to
> that
> >> > topic.
> >>
> >> That is the whole point of having a hierarchical topic based system.
> >> Engine would listen for anything coming from vdsm, vdsm would listen
> >> for anything that comes from engine using a path wildcard.
> >>
> >
> > We do not have hierarchical topic system. Please note that we have
> > p2p communication and real broker is not part of the scope anymore.
> > We only have mini broker implemented in vdsm.
> >
> >>
> >> Just use a /verb/<verbName> topic for each verb command automatically
> >> and either the same or something like
> >> /response/<verbName>/<correlation id> for the answer.
> >>
> >> There is no reason to distinguish verbs and events. Both are events
> >> and the only difference is whether the initiating party is waiting for
> >> a response. You need some kind of correlation id for that of course
> >> (to identify the specific answer you are waiting for).
> >>
> >
> > There is distinction based on historical reasons.
> >
> >>
> >> > I am not sure whether it is good idea for sever to know
> >> > what should be send where.
> >>
> >> The sender always decides what the topic it is. At least in the rest
> >> of the broker universe. There is no such thing as a server or client,
> >> just publisher and listener for specific message topic wildcard.
> >>
> >> > Sending to multiple topics is there but selecting a subset to send to
> >> > one of
> >> > them
> >>
> >> But you never do that, you send to one topic (full path) only and the
> >> listener decides what is interesting enough. The tree structure allows
> >> that (not sure about our implementation, but all standard brokers
> >> support this):
> >>
> >> /verb/runVm will be received by anybody listening for that or for
> >> /verb/* or for /*.
> >>
> >> The same for /response/runVm/16846158. MOM or hosted engine does not
> >> care about which exact command initiated the response and can listen
> >> for /response/runVm/*. On the other hand, the engine can be blocked in
> >> the call until the right response arrives.
> >>
> >
> > Please remember that we are not message based but we have 2 logical
> > ways of dealing with network frames:
> > - rpc
> > - events (aka messages)
> >
> >>
> >>
> >> Martin
> >>
> >> On Wed, Jun 22, 2016 at 12:00 PM, Piotr Kliczewski <pkliczew at redhat.com
> >
> >> wrote:
> >> >
> >> >
> >> > On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <msivak at redhat.com>
> >> > wrote:
> >> >>
> >> >> > If we subscribe to engine response queue we receive all the
> traffic.
> >> >> > At
> >> >> > the
> >> >> > moment there is no way
> >> >> > to define verbs we listen to. This functionality is only for
> events.
> >> >>
> >> >> Can't we create a new topic /engine/<verb> for each verb and
> configure
> >> >> vdsm to listen for /engine/*? That would duplicate what we have now,
> >> >> but MOM would be able to subscribe more selectively.
> >> >>
> >> >
> >> > It is not about having a topic but listing what should be posted to
> that
> >> > topic.
> >> > Sending to multiple topics is there but selecting a subset to send to
> >> > one of
> >> > them
> >> > would need to be added. I am not sure whether it is good idea for
> sever
> >> > to
> >> > know
> >> > what should be send where.
> >> >
> >> >>
> >> >> Martin
> >> >>
> >> >>
> >> >> On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski
> >> >> <pkliczew at redhat.com>
> >> >> wrote:
> >> >> >
> >> >> >
> >> >> > On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek
> >> >> > <michal.skrivanek at redhat.com> wrote:
> >> >> >>
> >> >> >>
> >> >> >> On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew at redhat.com>
> >> >> >> wrote:
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer at redhat.com>
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <
> msivak at redhat.com>
> >> >> >>> wrote:
> >> >> >>> >> 1. Mom is still using xmlrpc
> >> >> >>> >>
> >> >> >>> >> Mom must move to jsonrpc.
> >> >> >>> >> Martin: can you update on progress of this work?
> >> >> >>> >
> >> >> >>> > We would like to avoid going through VDSM completely, except
> from
> >> >> >>> > the
> >> >> >>> > broker part. Is it possible now to "parasitically" listen to
> vdsm
> >> >> >>> > events and engine commands without having to go through VDSM
> API?
> >> >> >>>
> >> >> >>> Sounds like a good plan.
> >> >> >>
> >> >> >>
> >> >> >> bypassing vdsm for mom things is a good idea, but we need to make
> >> >> >> sure
> >> >> >> we’re not killing the rest of the system
> >> >> >>
> >> >> >>>
> >> >> >>> Piotr, how far are we from letting mom listen to engine queue so
> it
> >> >> >>> get
> >> >> >>> engine
> >> >> >>> events/responses for certain verbs?
> >> >> >>
> >> >> >>
> >> >> >> We can do it now for both events and responses. We need to
> remember
> >> >> >> that
> >> >> >> we would
> >> >> >> receive anything that the engine is receiving. We could do better
> >> >> >> but
> >> >> >> that
> >> >> >> it would take
> >> >> >> more time.
> >> >> >>
> >> >> >>
> >> >> >> so it would receive all responses for only the defined verbs?
> >> >> >> It would wok well for getAllVmStats
> >> >> >>
> >> >> >
> >> >> > If we subscribe to engine response queue we receive all the
> traffic.
> >> >> > At
> >> >> > the
> >> >> > moment there is no way
> >> >> > to define verbs we listen to. This functionality is only for
> events.
> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>>
> >> >> >>>
> >> >> >>> > You can drop XML RPC,. vdsm does not depend on MOM working
> >> >> >>> > anymore.
> >> >> >>> > Just the balloon and ksm stats will be missing from data that
> are
> >> >> >>> > being sent to the engine.
> >> >> >>
> >> >> >>
> >> >> >> and a lot of things stops working then;)
> >> >> >>
> >> >> >
> >> >> > Maybe we can disable [1] it before dropping it. We can slowly fix
> >> >> > master
> >> >> > and
> >> >> > if needed we can enable it for
> >> >> > specific envs.
> >> >> >
> >> >> > [1] https://gerrit.ovirt.org/#/c/59172/
> >> >> >>>
> >> >> >>>
> >> >> >>> Balloon and ksm stats are sent today from mom to vdsm, and
> reported
> >> >> >>> by
> >> >> >>> vdsm to engine?
> >> >> >>>
> >> >> >>> If vdsm is only a middleman  and does not use this info, best to
> >> >> >>> send
> >> >> >>> it directly
> >> >> >>> to engine via the stomp broker part of vdsm.
> >> >> >>>
> >> >> >>> Piotr, how far are we from having engine listening for mom events
> >> >> >>> using
> >> >> >>> the
> >> >> >>> vdsm broker?
> >> >> >>
> >> >> >>
> >> >> >> I don’t think we should move any monitoring pulled from libvirt
> into
> >> >> >> mom -
> >> >> >> it would need another expensive and problematic call
> >> >> >>
> >> >> >>
> >> >> >> It can be used now but the engine needs to know to listen for
> them.
> >> >> >> We
> >> >> >> would
> >> >> >> need to implement engine subscriber (one class).
> >> >> >>
> >> >> >>>
> >> >> >>>
> >> >> >>> > There also were some issues with eventfd in the json library, I
> >> >> >>> > assume
> >> >> >>> > those are fixed now.
> >> >> >>>
> >> >> >>> They should be fixed here:
> >> >> >>> https://gerrit.ovirt.org/57942
> >> >> >>>
> >> >> >>> But suffering from these issue show that the api was not use
> >> >> >>> properly,
> >> >> >>> you should create one client and reuse it for the entire life of
> >> >> >>> the
> >> >> >>> application.
> >> >> >>>
> >> >> >>>
> >> >> >>> Nir
> >> >> >>>
> >> >> >>> >
> >> >> >>> > Martin
> >> >> >>> >
> >> >> >>> > On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <
> nsoffer at redhat.com>
> >> >> >>> > wrote:
> >> >> >>> >> Hi all,
> >> >> >>> >>
> >> >> >>> >> We are still wasting time on maintaining both xmlrpc and
> >> >> >>> >> jsonrpc.
> >> >> >>> >> If
> >> >> >>> >> we kill
> >> >> >>> >> xmlrpc, we can greatly simplify the code base, making it
> easier
> >> >> >>> >> to
> >> >> >>> >> mainain
> >> >> >>> >> and add new features.
> >> >> >>> >>
> >> >> >>> >> I suggest to kill xmlrpc in 4.1, and disable it *now* on
> master.
> >> >> >>> >>
> >> >> >>> >> Currently the we have 3 issues:
> >> >> >>> >>
> >> >> >>> >> 1. Mom is still using xmlrpc
> >> >> >>> >>
> >> >> >>> >> Mom must move to jsonrpc.
> >> >> >>> >> Martin: can you update on progress of this work?
> >> >> >>> >>
> >> >> >>> >> 2. sos plugin using vdsClient
> >> >> >>> >>
> >> >> >>> >> Need to port it to use jsonrpc library, or jsonrpc client
> >> >> >>> >>
> >> >> >>> >> New jsonrpc client:  https://gerrit.ovirt.org/35181
> >> >> >>> >>
> >> >> >>> >> 3. Engine is using xmlrpc server for ovf upload/download
> >> >> >>> >>
> >> >> >>> >> We must support current engine in 4.1, so we cannot remove
> >> >> >>> >> upload/download feature in this version, but we can remove the
> >> >> >>> >> xmlrpc support in this server.
> >> >> >>> >>
> >> >> >>> >> Currently we abuse the xmlrpc server, supporting PUT and GET
> for
> >> >> >>> >> upload and download (XMLRPC is using only POST). We can
> disable
> >> >> >>> >> POST requests in protocoldetector, and not register anything
> >> >> >>> >> with
> >> >> >>> >> the xmlrpc server.
> >> >> >>> >>
> >> >> >>> >> Thoughts?
> >> >> >>> >>
> >> >> >>> >> Nir
> >> >> >>
> >> >> >>
> >> >> >> _______________________________________________
> >> >> >> Devel mailing list
> >> >> >> Devel at ovirt.org
> >> >> >> http://lists.ovirt.org/mailman/listinfo/devel
> >> >> >>
> >> >> >>
> >> >> >
> >> >
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20160622/4d35272f/attachment-0001.html>