<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 22, 2016 at 1:22 PM, Martin Sivak <span dir="ltr"><<a href="mailto:msivak@redhat.com" target="_blank">msivak@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">> We do not have hierarchical topic system. Please note that we have<br>
> p2p communication and real broker is not part of the scope anymore.<br>
> We only have mini broker implemented in vdsm.<br>
<br>
</span>But then I can't implement the parasitic way of listening to data.<br>
This is basic functionality when you want to call something "a<br>
broker". Even a mini one...<br>
<span class=""><br></span></blockquote><div><br></div><div>We were implementing the code base on jsonrpc 2.0 and stomp 1.2 specs.<br></div><div>None of them consider it basic.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">
> There is distinction based on historical reasons.<br>
<br>
</span><span class="">> Please remember that we are not message based but we have 2 logical<br>
> ways of dealing with network frames:<br>
> - rpc<br>
> - events (aka messages)<br>
<br>
</span>Which we can easily emulate using events only for anything that needs<br>
it... everybody does that, because everybody needs blocking calls<br>
sometimes. But we seem to lack a very basic infrastructure to even<br>
attempt that. And I seem to remember telling you this same thing last<br>
year in Seattle :/<br>
<br></blockquote><div><br></div><div>I disagree with: "everybody needs blocking calls sometimes"</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
<br>
Knowing all this, I can drop XML-RPC from MOM and issue the exact same<br>
calls via JSON-RPC, but we will get nothing by that. Except maybe<br>
delete some protocol detector code... The load and vdsm dependencies<br>
will still stay the same.<br>
<span class=""><font color="#888888"><br></font></span></blockquote><div><br></div><div>This is my point we want to drop xmlrpc functionality.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class=""><font color="#888888">
<br>
Martin<br>
</font></span><div class=""><div class="h5"><br>
On Wed, Jun 22, 2016 at 12:47 PM, Piotr Kliczewski <<a href="mailto:pkliczew@redhat.com">pkliczew@redhat.com</a>> wrote:<br>
><br>
><br>
> On Wed, Jun 22, 2016 at 12:13 PM, Martin Sivak <<a href="mailto:msivak@redhat.com">msivak@redhat.com</a>> wrote:<br>
>><br>
>> > It is not about having a topic but listing what should be posted to that<br>
>> > topic.<br>
>><br>
>> That is the whole point of having a hierarchical topic based system.<br>
>> Engine would listen for anything coming from vdsm, vdsm would listen<br>
>> for anything that comes from engine using a path wildcard.<br>
>><br>
><br>
> We do not have hierarchical topic system. Please note that we have<br>
> p2p communication and real broker is not part of the scope anymore.<br>
> We only have mini broker implemented in vdsm.<br>
><br>
>><br>
>> Just use a /verb/<verbName> topic for each verb command automatically<br>
>> and either the same or something like<br>
>> /response/<verbName>/<correlation id> for the answer.<br>
>><br>
>> There is no reason to distinguish verbs and events. Both are events<br>
>> and the only difference is whether the initiating party is waiting for<br>
>> a response. You need some kind of correlation id for that of course<br>
>> (to identify the specific answer you are waiting for).<br>
>><br>
><br>
> There is distinction based on historical reasons.<br>
><br>
>><br>
>> > I am not sure whether it is good idea for sever to know<br>
>> > what should be send where.<br>
>><br>
>> The sender always decides what the topic it is. At least in the rest<br>
>> of the broker universe. There is no such thing as a server or client,<br>
>> just publisher and listener for specific message topic wildcard.<br>
>><br>
>> > Sending to multiple topics is there but selecting a subset to send to<br>
>> > one of<br>
>> > them<br>
>><br>
>> But you never do that, you send to one topic (full path) only and the<br>
>> listener decides what is interesting enough. The tree structure allows<br>
>> that (not sure about our implementation, but all standard brokers<br>
>> support this):<br>
>><br>
>> /verb/runVm will be received by anybody listening for that or for<br>
>> /verb/* or for /*.<br>
>><br>
>> The same for /response/runVm/16846158. MOM or hosted engine does not<br>
>> care about which exact command initiated the response and can listen<br>
>> for /response/runVm/*. On the other hand, the engine can be blocked in<br>
>> the call until the right response arrives.<br>
>><br>
><br>
> Please remember that we are not message based but we have 2 logical<br>
> ways of dealing with network frames:<br>
> - rpc<br>
> - events (aka messages)<br>
><br>
>><br>
>><br>
>> Martin<br>
>><br>
>> On Wed, Jun 22, 2016 at 12:00 PM, Piotr Kliczewski <<a href="mailto:pkliczew@redhat.com">pkliczew@redhat.com</a>><br>
>> wrote:<br>
>> ><br>
>> ><br>
>> > On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <<a href="mailto:msivak@redhat.com">msivak@redhat.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> > If we subscribe to engine response queue we receive all the traffic.<br>
>> >> > At<br>
>> >> > the<br>
>> >> > moment there is no way<br>
>> >> > to define verbs we listen to. This functionality is only for events.<br>
>> >><br>
>> >> Can't we create a new topic /engine/<verb> for each verb and configure<br>
>> >> vdsm to listen for /engine/*? That would duplicate what we have now,<br>
>> >> but MOM would be able to subscribe more selectively.<br>
>> >><br>
>> ><br>
>> > It is not about having a topic but listing what should be posted to that<br>
>> > topic.<br>
>> > Sending to multiple topics is there but selecting a subset to send to<br>
>> > one of<br>
>> > them<br>
>> > would need to be added. I am not sure whether it is good idea for sever<br>
>> > to<br>
>> > know<br>
>> > what should be send where.<br>
>> ><br>
>> >><br>
>> >> Martin<br>
>> >><br>
>> >><br>
>> >> On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski<br>
>> >> <<a href="mailto:pkliczew@redhat.com">pkliczew@redhat.com</a>><br>
>> >> wrote:<br>
>> >> ><br>
>> >> ><br>
>> >> > On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek<br>
>> >> > <<a href="mailto:michal.skrivanek@redhat.com">michal.skrivanek@redhat.com</a>> wrote:<br>
>> >> >><br>
>> >> >><br>
>> >> >> On 20 Jun 2016, at 18:41, Piotr Kliczewski <<a href="mailto:pkliczew@redhat.com">pkliczew@redhat.com</a>><br>
>> >> >> wrote:<br>
>> >> >><br>
>> >> >><br>
>> >> >><br>
>> >> >> On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>><br>
>> >> >> wrote:<br>
>> >> >>><br>
>> >> >>> On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <<a href="mailto:msivak@redhat.com">msivak@redhat.com</a>><br>
>> >> >>> wrote:<br>
>> >> >>> >> 1. Mom is still using xmlrpc<br>
>> >> >>> >><br>
>> >> >>> >> Mom must move to jsonrpc.<br>
>> >> >>> >> Martin: can you update on progress of this work?<br>
>> >> >>> ><br>
>> >> >>> > We would like to avoid going through VDSM completely, except from<br>
>> >> >>> > the<br>
>> >> >>> > broker part. Is it possible now to "parasitically" listen to vdsm<br>
>> >> >>> > events and engine commands without having to go through VDSM API?<br>
>> >> >>><br>
>> >> >>> Sounds like a good plan.<br>
>> >> >><br>
>> >> >><br>
>> >> >> bypassing vdsm for mom things is a good idea, but we need to make<br>
>> >> >> sure<br>
>> >> >> we’re not killing the rest of the system<br>
>> >> >><br>
>> >> >>><br>
>> >> >>> Piotr, how far are we from letting mom listen to engine queue so it<br>
>> >> >>> get<br>
>> >> >>> engine<br>
>> >> >>> events/responses for certain verbs?<br>
>> >> >><br>
>> >> >><br>
>> >> >> We can do it now for both events and responses. We need to remember<br>
>> >> >> that<br>
>> >> >> we would<br>
>> >> >> receive anything that the engine is receiving. We could do better<br>
>> >> >> but<br>
>> >> >> that<br>
>> >> >> it would take<br>
>> >> >> more time.<br>
>> >> >><br>
>> >> >><br>
>> >> >> so it would receive all responses for only the defined verbs?<br>
>> >> >> It would wok well for getAllVmStats<br>
>> >> >><br>
>> >> ><br>
>> >> > If we subscribe to engine response queue we receive all the traffic.<br>
>> >> > At<br>
>> >> > the<br>
>> >> > moment there is no way<br>
>> >> > to define verbs we listen to. This functionality is only for events.<br>
>> >> ><br>
>> >> >><br>
>> >> >><br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> > You can drop XML RPC,. vdsm does not depend on MOM working<br>
>> >> >>> > anymore.<br>
>> >> >>> > Just the balloon and ksm stats will be missing from data that are<br>
>> >> >>> > being sent to the engine.<br>
>> >> >><br>
>> >> >><br>
>> >> >> and a lot of things stops working then;)<br>
>> >> >><br>
>> >> ><br>
>> >> > Maybe we can disable [1] it before dropping it. We can slowly fix<br>
>> >> > master<br>
>> >> > and<br>
>> >> > if needed we can enable it for<br>
>> >> > specific envs.<br>
>> >> ><br>
>> >> > [1] <a href="https://gerrit.ovirt.org/#/c/59172/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/59172/</a><br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> Balloon and ksm stats are sent today from mom to vdsm, and reported<br>
>> >> >>> by<br>
>> >> >>> vdsm to engine?<br>
>> >> >>><br>
>> >> >>> If vdsm is only a middleman and does not use this info, best to<br>
>> >> >>> send<br>
>> >> >>> it directly<br>
>> >> >>> to engine via the stomp broker part of vdsm.<br>
>> >> >>><br>
>> >> >>> Piotr, how far are we from having engine listening for mom events<br>
>> >> >>> using<br>
>> >> >>> the<br>
>> >> >>> vdsm broker?<br>
>> >> >><br>
>> >> >><br>
>> >> >> I don’t think we should move any monitoring pulled from libvirt into<br>
>> >> >> mom -<br>
>> >> >> it would need another expensive and problematic call<br>
>> >> >><br>
>> >> >><br>
>> >> >> It can be used now but the engine needs to know to listen for them.<br>
>> >> >> We<br>
>> >> >> would<br>
>> >> >> need to implement engine subscriber (one class).<br>
>> >> >><br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> > There also were some issues with eventfd in the json library, I<br>
>> >> >>> > assume<br>
>> >> >>> > those are fixed now.<br>
>> >> >>><br>
>> >> >>> They should be fixed here:<br>
>> >> >>> <a href="https://gerrit.ovirt.org/57942" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/57942</a><br>
>> >> >>><br>
>> >> >>> But suffering from these issue show that the api was not use<br>
>> >> >>> properly,<br>
>> >> >>> you should create one client and reuse it for the entire life of<br>
>> >> >>> the<br>
>> >> >>> application.<br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> Nir<br>
>> >> >>><br>
>> >> >>> ><br>
>> >> >>> > Martin<br>
>> >> >>> ><br>
>> >> >>> > On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>><br>
>> >> >>> > wrote:<br>
>> >> >>> >> Hi all,<br>
>> >> >>> >><br>
>> >> >>> >> We are still wasting time on maintaining both xmlrpc and<br>
>> >> >>> >> jsonrpc.<br>
>> >> >>> >> If<br>
>> >> >>> >> we kill<br>
>> >> >>> >> xmlrpc, we can greatly simplify the code base, making it easier<br>
>> >> >>> >> to<br>
>> >> >>> >> mainain<br>
>> >> >>> >> and add new features.<br>
>> >> >>> >><br>
>> >> >>> >> I suggest to kill xmlrpc in 4.1, and disable it *now* on master.<br>
>> >> >>> >><br>
>> >> >>> >> Currently the we have 3 issues:<br>
>> >> >>> >><br>
>> >> >>> >> 1. Mom is still using xmlrpc<br>
>> >> >>> >><br>
>> >> >>> >> Mom must move to jsonrpc.<br>
>> >> >>> >> Martin: can you update on progress of this work?<br>
>> >> >>> >><br>
>> >> >>> >> 2. sos plugin using vdsClient<br>
>> >> >>> >><br>
>> >> >>> >> Need to port it to use jsonrpc library, or jsonrpc client<br>
>> >> >>> >><br>
>> >> >>> >> New jsonrpc client: <a href="https://gerrit.ovirt.org/35181" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/35181</a><br>
>> >> >>> >><br>
>> >> >>> >> 3. Engine is using xmlrpc server for ovf upload/download<br>
>> >> >>> >><br>
>> >> >>> >> We must support current engine in 4.1, so we cannot remove<br>
>> >> >>> >> upload/download feature in this version, but we can remove the<br>
>> >> >>> >> xmlrpc support in this server.<br>
>> >> >>> >><br>
>> >> >>> >> Currently we abuse the xmlrpc server, supporting PUT and GET for<br>
>> >> >>> >> upload and download (XMLRPC is using only POST). We can disable<br>
>> >> >>> >> POST requests in protocoldetector, and not register anything<br>
>> >> >>> >> with<br>
>> >> >>> >> the xmlrpc server.<br>
>> >> >>> >><br>
>> >> >>> >> Thoughts?<br>
>> >> >>> >><br>
>> >> >>> >> Nir<br>
>> >> >><br>
>> >> >><br>
>> >> >> _______________________________________________<br>
>> >> >> Devel mailing list<br>
>> >> >> <a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
>> >> >> <a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br>
>> >> >><br>
>> >> >><br>
>> >> ><br>
>> ><br>
>> ><br>
><br>
><br>
</div></div></blockquote></div><br></div></div>