Killing XMLRPC on master
Hi all, We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features. I suggest to kill xmlrpc in 4.1, and disable it *now* on master. Currently the we have 3 issues: 1. Mom is still using xmlrpc Mom must move to jsonrpc. Martin: can you update on progress of this work? 2. sos plugin using vdsClient Need to port it to use jsonrpc library, or jsonrpc client New jsonrpc client: https://gerrit.ovirt.org/35181 3. Engine is using xmlrpc server for ovf upload/download We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server. Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server. Thoughts? Nir
On Sun, Jun 19, 2016 at 02:02:23PM +0300, Nir Soffer wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
Irit, that seems like a low-hanging fruit.
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
I'd like to kill xmlrpc just like anyone, but we cannot do that before these 3 known issues are solved. Sandro, have hosted engine dropped all of its xmlrpc dependecies?
On Sun, Jun 19, 2016 at 3:51 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Jun 19, 2016 at 02:02:23PM +0300, Nir Soffer wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
Irit, that seems like a low-hanging fruit.
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
I'd like to kill xmlrpc just like anyone, but we cannot do that before these 3 known issues are solved.
Sandro, have hosted engine dropped all of its xmlrpc dependecies?
I think setup did, but can still find vdscli in -ha. -- Didi
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API? You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine. There also were some issues with eventfd in the json library, I assume those are fixed now. Martin On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir
On Mon, Jun 20, 2016 at 10:33:19AM +0200, Martin Sivak wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Won't we miss all mom's smart policy if we do that?
There also were some issues with eventfd in the json library, I assume those are fixed now.
Martin
On Mon, Jun 20, 2016 at 11:05 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Mon, Jun 20, 2016 at 10:33:19AM +0200, Martin Sivak wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Yes, you can use jsonrpc client and register for specific events assuming that they are sent.
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Won't we miss all mom's smart policy if we do that?
There also were some issues with eventfd in the json library, I assume those are fixed now.
Martin
Won't we miss all mom's smart policy if we do that?
- Mom uses getAllVmStats (and one or two others) for stats collection and it polls for those atm. We can avoid that in the broker mode (VDSM sends the stats to the engine and we can listen). - Engine sends mom configuration through VDSM, but VDSM only converts that to different format and forwards that to MOM. This can be processed directly in MOM if VDSM knows how to ignore calls that are not relevant for it. We will have to call VDSM to set values, but that should be still possible. Martin On Mon, Jun 20, 2016 at 11:05 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Mon, Jun 20, 2016 at 10:33:19AM +0200, Martin Sivak wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Won't we miss all mom's smart policy if we do that?
There also were some issues with eventfd in the json library, I assume those are fixed now.
Martin
On Mon, Jun 20, 2016 at 11:43 AM, Martin Sivak <msivak@redhat.com> wrote:
Won't we miss all mom's smart policy if we do that?
- Mom uses getAllVmStats (and one or two others) for stats collection and it polls for those atm. We can avoid that in the broker mode (VDSM sends the stats to the engine and we can listen).
We do not support this mode but with some work we could trigger an event containing this information so mom can register to get it.
- Engine sends mom configuration through VDSM, but VDSM only converts that to different format and forwards that to MOM. This can be processed directly in MOM if VDSM knows how to ignore calls that are not relevant for it.
We will have to call VDSM to set values, but that should be still possible.
Martin
On Mon, Jun 20, 2016 at 11:05 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Mon, Jun 20, 2016 at 10:33:19AM +0200, Martin Sivak wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Won't we miss all mom's smart policy if we do that?
There also were some issues with eventfd in the json library, I assume those are fixed now.
Martin
We do not support this mode but with some work we could trigger an event containing this information so mom can register to get it.
Wasn't polling for stats removed? I thought we send more using events than just VM status. Martin On Mon, Jun 20, 2016 at 11:48 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:43 AM, Martin Sivak <msivak@redhat.com> wrote:
Won't we miss all mom's smart policy if we do that?
- Mom uses getAllVmStats (and one or two others) for stats collection and it polls for those atm. We can avoid that in the broker mode (VDSM sends the stats to the engine and we can listen).
We do not support this mode but with some work we could trigger an event containing this information so mom can register to get it.
- Engine sends mom configuration through VDSM, but VDSM only converts that to different format and forwards that to MOM. This can be processed directly in MOM if VDSM knows how to ignore calls that are not relevant for it.
We will have to call VDSM to set values, but that should be still possible.
Martin
On Mon, Jun 20, 2016 at 11:05 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Mon, Jun 20, 2016 at 10:33:19AM +0200, Martin Sivak wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Won't we miss all mom's smart policy if we do that?
There also were some issues with eventfd in the json library, I assume those are fixed now.
Martin
On Mon, Jun 20, 2016 at 12:11 PM, Martin Sivak <msivak@redhat.com> wrote:
We do not support this mode but with some work we could trigger an event containing this information so mom can register to get it.
Wasn't polling for stats removed? I thought we send more using events than just VM status.
We send only vm status changes and small bunch of additional data that Francesco added. They are not one to one replacement. If there amount of data is good enough for mom than it is enough to register for this event. The engine still calls to get all vm stats every 15 seconds.
Martin
On Mon, Jun 20, 2016 at 11:48 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:43 AM, Martin Sivak <msivak@redhat.com>
wrote:
Won't we miss all mom's smart policy if we do that?
- Mom uses getAllVmStats (and one or two others) for stats collection and it polls for those atm. We can avoid that in the broker mode (VDSM sends the stats to the engine and we can listen).
We do not support this mode but with some work we could trigger an event containing this information so mom can register to get it.
- Engine sends mom configuration through VDSM, but VDSM only converts that to different format and forwards that to MOM. This can be processed directly in MOM if VDSM knows how to ignore calls that are not relevant for it.
We will have to call VDSM to set values, but that should be still possible.
Martin
On Mon, Jun 20, 2016 at 11:05 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Mon, Jun 20, 2016 at 10:33:19AM +0200, Martin Sivak wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Won't we miss all mom's smart policy if we do that?
There also were some issues with eventfd in the json library, I
assume
those are fixed now.
Martin
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Sounds like a good plan. Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine? If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm. Piotr, how far are we from having engine listening for mom events using the vdsm broker?
There also were some issues with eventfd in the json library, I assume those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942 But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application. Nir
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Sounds like a good plan.
Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but that it would take more time.
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine?
If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm.
Piotr, how far are we from having engine listening for mom events using the vdsm broker?
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
There also were some issues with eventfd in the json library, I assume those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942
But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application.
Nir
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we
xmlrpc, we can greatly simplify the code base, making it easier to
kill mainain
and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> = wrote: =20 =20 =20 On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com = <mailto:nsoffer@redhat.com>> wrote: On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com = <mailto:msivak@redhat.com>> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from =
--Apple-Mail=_16763C7B-934F-44FC-84DF-77685A35781C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 the
broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API? =20 Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure = we=E2=80=99re not killing the rest of the system
=20 Piotr, how far are we from letting mom listen to engine queue so it = get engine events/responses for certain verbs?
=20 We can do it now for both events and responses. We need to remember = that we would receive anything that the engine is receiving. We could do better but = that it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
=20 =20
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
and a lot of things stops working then;)
=20 Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine? =20 If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm. =20 Piotr, how far are we from having engine listening for mom events = using the vdsm broker?
=20 It can be used now but the engine needs to know to listen for them. We = would=20 need to implement engine subscriber (one class).=20 =20 =20
There also were some issues with eventfd in the json library, I = assume those are fixed now. =20 They should be fixed here: https://gerrit.ovirt.org/57942 <https://gerrit.ovirt.org/57942> =20 But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application. =20 =20 Nir =20
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com = <mailto:nsoffer@redhat.com>> wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. = If we kill xmlrpc, we can greatly simplify the code base, making it easier to =
I don=E2=80=99t think we should move any monitoring pulled from libvirt = into mom - it would need another expensive and problematic call mainain
and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181 = <https://gerrit.ovirt.org/35181>
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir =20
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--Apple-Mail=_16763C7B-934F-44FC-84DF-77685A35781C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><br class=3D""><div><blockquote type=3D"cite" class=3D""><div = class=3D"">On 20 Jun 2016, at 18:41, Piotr Kliczewski <<a = href=3D"mailto:pkliczew@redhat.com" class=3D"">pkliczew@redhat.com</a>>= wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = dir=3D"ltr" class=3D""><br class=3D""><div class=3D"gmail_extra"><br = class=3D""><div class=3D"gmail_quote">On Mon, Jun 20, 2016 at 6:22 PM, = Nir Soffer <span dir=3D"ltr" class=3D""><<a = href=3D"mailto:nsoffer@redhat.com" target=3D"_blank" = class=3D"">nsoffer@redhat.com</a>></span> wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=3D"">On = Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <<a = href=3D"mailto:msivak@redhat.com" class=3D"">msivak@redhat.com</a>> = wrote:<br class=3D""> >> 1. Mom is still using xmlrpc<br class=3D""> >><br class=3D""> >> Mom must move to jsonrpc.<br class=3D""> >> Martin: can you update on progress of this work?<br class=3D""> ><br class=3D""> > We would like to avoid going through VDSM completely, except from = the<br class=3D""> > broker part. Is it possible now to "parasitically" listen to = vdsm<br class=3D""> > events and engine commands without having to go through VDSM = API?<br class=3D""> <br class=3D""> </span>Sounds like a good plan.<br = class=3D""></blockquote></div></div></div></div></blockquote><div><br = class=3D""></div><div class=3D"">bypassing vdsm for mom things is a good = idea, but we need to make sure we=E2=80=99re not killing the rest of the = system</div><div class=3D""><br class=3D""></div><blockquote type=3D"cite"= class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"gmail_extra"><div class=3D"gmail_quote"><blockquote = class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"> <br class=3D""> Piotr, how far are we from letting mom listen to engine queue so it get = engine<br class=3D""> events/responses for certain = verbs?</blockquote></div></div></div></div></blockquote></div><div><blockq= uote type=3D"cite" class=3D""><div class=3D""><div dir=3D"ltr" = class=3D""><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div = class=3D""><br class=3D""></div><div class=3D"">We can do it now for = both events and responses. We need to remember that we would<br = class=3D""></div><div class=3D"">receive anything that the engine is = receiving. We could do better but that it would take<br = class=3D""></div><div class=3D"">more time.<br = class=3D""></div></div></div></div></div></blockquote><div><br = class=3D""></div>so it would receive all responses for only the defined = verbs?</div><div>It would wok well for getAllVmStats</div><div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = dir=3D"ltr" class=3D""><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><div class=3D""> </div><blockquote = class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"> <span class=3D""><br class=3D""> > You can drop XML RPC,. vdsm does not depend on MOM working = anymore.<br class=3D""> > Just the balloon and ksm stats will be missing from data that = are<br class=3D""> > being sent to the engine.<br = class=3D""></span></blockquote></div></div></div></div></blockquote><div><= br class=3D""></div>and a lot of things stops working = then;)</div><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D""><div dir=3D"ltr" class=3D""><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 = 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=3D""> <br class=3D""> </span>Balloon and ksm stats are sent today from mom to vdsm, and = reported by<br class=3D""> vdsm to engine?<br class=3D""> <br class=3D""> If vdsm is only a middleman and does not use this info, best to = send<br class=3D""> it directly<br class=3D""> to engine via the stomp broker part of vdsm.<br class=3D""> <br class=3D""> Piotr, how far are we from having engine listening for mom events using = the<br class=3D""> vdsm broker?<br = class=3D""></blockquote></div></div></div></div></blockquote><div><br = class=3D""></div>I don=E2=80=99t think we should move any monitoring = pulled from libvirt into mom - it would need another expensive and = problematic call</div><div><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div dir=3D"ltr" class=3D""><div = class=3D"gmail_extra"><div class=3D"gmail_quote"><div class=3D""><br = class=3D""></div><div class=3D"">It can be used now but the engine needs = to know to listen for them. We would <br class=3D""></div><div = class=3D"">need to implement engine subscriber (one class). <br = class=3D""></div><div class=3D""> </div><blockquote = class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"> <span class=3D""><br class=3D""> > There also were some issues with eventfd in the json library, I = assume<br class=3D""> > those are fixed now.<br class=3D""> <br class=3D""> </span>They should be fixed here:<br class=3D""> <a href=3D"https://gerrit.ovirt.org/57942" rel=3D"noreferrer" = target=3D"_blank" class=3D"">https://gerrit.ovirt.org/57942</a><br = class=3D""> <br class=3D""> But suffering from these issue show that the api was not use = properly,<br class=3D""> you should create one client and reuse it for the entire life of the<br = class=3D""> application.<br class=3D""> <span class=3D"HOEnZb"><font color=3D"#888888" class=3D""><br class=3D""> <br class=3D""> Nir<br class=3D""> </font></span><div class=3D"HOEnZb"><div class=3D"h5"><br class=3D""> ><br class=3D""> > Martin<br class=3D""> ><br class=3D""> > On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <<a = href=3D"mailto:nsoffer@redhat.com" class=3D"">nsoffer@redhat.com</a>> = wrote:<br class=3D""> >> Hi all,<br class=3D""> >><br class=3D""> >> We are still wasting time on maintaining both xmlrpc and = jsonrpc. If we kill<br class=3D""> >> xmlrpc, we can greatly simplify the code base, making it easier = to mainain<br class=3D""> >> and add new features.<br class=3D""> >><br class=3D""> >> I suggest to kill xmlrpc in 4.1, and disable it *now* on = master.<br class=3D""> >><br class=3D""> >> Currently the we have 3 issues:<br class=3D""> >><br class=3D""> >> 1. Mom is still using xmlrpc<br class=3D""> >><br class=3D""> >> Mom must move to jsonrpc.<br class=3D""> >> Martin: can you update on progress of this work?<br class=3D""> >><br class=3D""> >> 2. sos plugin using vdsClient<br class=3D""> >><br class=3D""> >> Need to port it to use jsonrpc library, or jsonrpc client<br = class=3D""> >><br class=3D""> >> New jsonrpc client: <a = href=3D"https://gerrit.ovirt.org/35181" rel=3D"noreferrer" = target=3D"_blank" class=3D"">https://gerrit.ovirt.org/35181</a><br = class=3D""> >><br class=3D""> >> 3. Engine is using xmlrpc server for ovf upload/download<br = class=3D""> >><br class=3D""> >> We must support current engine in 4.1, so we cannot remove<br = class=3D""> >> upload/download feature in this version, but we can remove = the<br class=3D""> >> xmlrpc support in this server.<br class=3D""> >><br class=3D""> >> Currently we abuse the xmlrpc server, supporting PUT and GET = for<br class=3D""> >> upload and download (XMLRPC is using only POST). We can = disable<br class=3D""> >> POST requests in protocoldetector, and not register anything = with<br class=3D""> >> the xmlrpc server.<br class=3D""> >><br class=3D""> >> Thoughts?<br class=3D""> >><br class=3D""> >> Nir<br class=3D""> </div></div></blockquote></div><br class=3D""></div></div> _______________________________________________<br class=3D"">Devel = mailing list<br class=3D""><a href=3D"mailto:Devel@ovirt.org" = class=3D"">Devel@ovirt.org</a><br = class=3D"">http://lists.ovirt.org/mailman/listinfo/devel</div></blockquote=
</div><br class=3D""></body></html>=
--Apple-Mail=_16763C7B-934F-44FC-84DF-77685A35781C--
--rRnowXasBTKV4uiQifXhmi2jWqkoDElUg Content-Type: multipart/mixed; boundary="DPS8NTOpp45nGP9QKGWhKLu2kIfIMueDU" From: Sven Kieske <s.kieske@mittwald.de> To: devel@ovirt.org Message-ID: <576A5446.4040705@mittwald.de> X-Authenticated-mymxserver.com: Yes Subject: Re: [ovirt-devel] Killing XMLRPC on master References: <CAMRbyyuuONNgzOLEZ-XMg7=YDFq9NEw-5WcAm1+25ts9wC5Vyg@mail.gmail.com> <5B86CDB7-4AA3-42A8-B64C-509A571E2173@redhat.com> In-Reply-To: <5B86CDB7-4AA3-42A8-B64C-509A571E2173@redhat.com> --DPS8NTOpp45nGP9QKGWhKLu2kIfIMueDU Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 22/06/16 10:39, Michal Skrivanek wrote:
bypassing vdsm for mom things is a good idea, but we need to make sure = we=92re not killing the rest of the system
Wouldn't that mean you need the whole authentication and communication code duplicated to mom? or is it already in there? from a sysadmin perspective I would like to keep the needed ports to open between engine and virt-host on a minimum. --=20 Mit freundlichen Gr=FC=DFen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG K=F6nigsberger Stra=DFe 6 32339 Espelkamp T: +495772 293100 F: +495772 293333 https://www.mittwald.de Gesch=E4ftsf=FChrer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhause= n Komplement=E4rin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynha= usen --DPS8NTOpp45nGP9QKGWhKLu2kIfIMueDU-- --rRnowXasBTKV4uiQifXhmi2jWqkoDElUg Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJXalRGAAoJEMby9TMDAbQRfrsP/0EIS/2GyxLtkTMVtrpYKh5u JUfhnuVRc7e1CAnuvPN6DSA9eFLNEeS6lMuXNHN8mqRdl5TDPVg7oFJqKN7e+gAC d/Nt2HCJLnrLYX87LZsweztjFFZ1DwnhGnq2p/YwqMvnjJxi4ww4zzG/z94VFbzw E6e+82f/a3yl1N3wfoBcsm9RDQ2KYgwX7tuoIQ99fjYoI6dD+Y2WA/XgkX80y7FD 2H2waPtOmXdRYCogtkB1j/+XLi+f61C5CXQ1BitTbuV2NBhhUkW+8PBmcAOWN9aM 6tm7a3pCco6fSFBIoQdnTvOw3JPzCqtkfzij8BCZJcwFxvFXdjPCZ+u0pUFAafTe QeuQ81LaPVCURNs3OsvBO+dSyh8pCVOfeSA5GANmJPAZ2rhMryp+gLYfxMs3PstE 2up0KcKEj7hySnBkkgV5E98ikp4e0HiCkGxZzMyfjSAyxFdhI78Ib1Ywuda31auS DVmJrZWBoliIc1iIsrjHj2E75g3ZcIHfbUe6yO7IYMj4OdGLH+DckTw4KarfVR+j xPDcd3pTMyZAQtkB26PPWxOKXSD1C97nTIUiWUvSF6yjbPNBdgQlHRXOd23knf9f 7pwKcJlkGEYuOPUoas4GV+WYR2klAQgzfQpyDF4LYIIk2BwL0qUSmRYguU4M6KOZ 8wdIxoUrgSLS74Xfvfr3 =AQlf -----END PGP SIGNATURE----- --rRnowXasBTKV4uiQifXhmi2jWqkoDElUg--
Hi,
Wouldn't that mean you need the whole authentication and communication code duplicated to mom? or is it already in there?
Not really, MOM would connect to the local VDSM (as it does now, just with a different protocol), not to engine directly. All data will still flow using the single engine - vdsm connection. VDSM has a broker part that would then distribute the messages between MOM, VDSM API and hopefully hosted engine too. Regards Martin On Wed, Jun 22, 2016 at 11:03 AM, Sven Kieske <s.kieske@mittwald.de> wrote:
On 22/06/16 10:39, Michal Skrivanek wrote:
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
Wouldn't that mean you need the whole authentication and communication code duplicated to mom? or is it already in there?
from a sysadmin perspective I would like to keep the needed ports to open between engine and virt-host on a minimum.
-- Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +495772 293100 F: +495772 293333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek < michal.skrivanek@redhat.com> wrote:
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but that it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
and a lot of things stops working then;)
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs. [1] https://gerrit.ovirt.org/#/c/59172/
Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine?
If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm.
Piotr, how far are we from having engine listening for mom events using the vdsm broker?
I don’t think we should move any monitoring pulled from libvirt into mom - it would need another expensive and problematic call
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
There also were some issues with eventfd in the json library, I assume those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942
But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application.
Nir
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If
xmlrpc, we can greatly simplify the code base, making it easier to
we kill mainain
and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
Can't we create a new topic /engine/<verb> for each verb and configure vdsm to listen for /engine/*? That would duplicate what we have now, but MOM would be able to subscribe more selectively. Martin On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but that it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
and a lot of things stops working then;)
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs.
[1] https://gerrit.ovirt.org/#/c/59172/
Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine?
If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm.
Piotr, how far are we from having engine listening for mom events using the vdsm broker?
I don’t think we should move any monitoring pulled from libvirt into mom - it would need another expensive and problematic call
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
There also were some issues with eventfd in the json library, I assume those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942
But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application.
Nir
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <msivak@redhat.com> wrote:
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
Can't we create a new topic /engine/<verb> for each verb and configure vdsm to listen for /engine/*? That would duplicate what we have now, but MOM would be able to subscribe more selectively.
It is not about having a topic but listing what should be posted to that topic. Sending to multiple topics is there but selecting a subset to send to one of them would need to be added. I am not sure whether it is good idea for sever to know what should be send where.
Martin
On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com>
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but
it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
If we subscribe to engine response queue we receive all the traffic. At
wrote: that the
moment there is no way to define verbs we listen to. This functionality is only for events.
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
and a lot of things stops working then;)
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs.
[1] https://gerrit.ovirt.org/#/c/59172/
Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine?
If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm.
Piotr, how far are we from having engine listening for mom events using the vdsm broker?
I don’t think we should move any monitoring pulled from libvirt into mom - it would need another expensive and problematic call
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
There also were some issues with eventfd in the json library, I
assume
those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942
But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application.
Nir
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com>
wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
It is not about having a topic but listing what should be posted to that topic.
That is the whole point of having a hierarchical topic based system. Engine would listen for anything coming from vdsm, vdsm would listen for anything that comes from engine using a path wildcard. Just use a /verb/<verbName> topic for each verb command automatically and either the same or something like /response/<verbName>/<correlation id> for the answer. There is no reason to distinguish verbs and events. Both are events and the only difference is whether the initiating party is waiting for a response. You need some kind of correlation id for that of course (to identify the specific answer you are waiting for).
I am not sure whether it is good idea for sever to know what should be send where.
The sender always decides what the topic it is. At least in the rest of the broker universe. There is no such thing as a server or client, just publisher and listener for specific message topic wildcard.
Sending to multiple topics is there but selecting a subset to send to one of them
But you never do that, you send to one topic (full path) only and the listener decides what is interesting enough. The tree structure allows that (not sure about our implementation, but all standard brokers support this): /verb/runVm will be received by anybody listening for that or for /verb/* or for /*. The same for /response/runVm/16846158. MOM or hosted engine does not care about which exact command initiated the response and can listen for /response/runVm/*. On the other hand, the engine can be blocked in the call until the right response arrives. Martin On Wed, Jun 22, 2016 at 12:00 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <msivak@redhat.com> wrote:
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
Can't we create a new topic /engine/<verb> for each verb and configure vdsm to listen for /engine/*? That would duplicate what we have now, but MOM would be able to subscribe more selectively.
It is not about having a topic but listing what should be posted to that topic. Sending to multiple topics is there but selecting a subset to send to one of them would need to be added. I am not sure whether it is good idea for sever to know what should be send where.
Martin
On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
> 1. Mom is still using xmlrpc > > Mom must move to jsonrpc. > Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but that it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
and a lot of things stops working then;)
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs.
[1] https://gerrit.ovirt.org/#/c/59172/
Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine?
If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm.
Piotr, how far are we from having engine listening for mom events using the vdsm broker?
I don’t think we should move any monitoring pulled from libvirt into mom - it would need another expensive and problematic call
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
There also were some issues with eventfd in the json library, I assume those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942
But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application.
Nir
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> wrote: > Hi all, > > We are still wasting time on maintaining both xmlrpc and jsonrpc. > If > we kill > xmlrpc, we can greatly simplify the code base, making it easier to > mainain > and add new features. > > I suggest to kill xmlrpc in 4.1, and disable it *now* on master. > > Currently the we have 3 issues: > > 1. Mom is still using xmlrpc > > Mom must move to jsonrpc. > Martin: can you update on progress of this work? > > 2. sos plugin using vdsClient > > Need to port it to use jsonrpc library, or jsonrpc client > > New jsonrpc client: https://gerrit.ovirt.org/35181 > > 3. Engine is using xmlrpc server for ovf upload/download > > We must support current engine in 4.1, so we cannot remove > upload/download feature in this version, but we can remove the > xmlrpc support in this server. > > Currently we abuse the xmlrpc server, supporting PUT and GET for > upload and download (XMLRPC is using only POST). We can disable > POST requests in protocoldetector, and not register anything with > the xmlrpc server. > > Thoughts? > > Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Jun 22, 2016 at 12:13 PM, Martin Sivak <msivak@redhat.com> wrote:
It is not about having a topic but listing what should be posted to that topic.
That is the whole point of having a hierarchical topic based system. Engine would listen for anything coming from vdsm, vdsm would listen for anything that comes from engine using a path wildcard.
We do not have hierarchical topic system. Please note that we have p2p communication and real broker is not part of the scope anymore. We only have mini broker implemented in vdsm.
Just use a /verb/<verbName> topic for each verb command automatically and either the same or something like /response/<verbName>/<correlation id> for the answer.
There is no reason to distinguish verbs and events. Both are events and the only difference is whether the initiating party is waiting for a response. You need some kind of correlation id for that of course (to identify the specific answer you are waiting for).
There is distinction based on historical reasons.
I am not sure whether it is good idea for sever to know what should be send where.
The sender always decides what the topic it is. At least in the rest of the broker universe. There is no such thing as a server or client, just publisher and listener for specific message topic wildcard.
Sending to multiple topics is there but selecting a subset to send to one of them
But you never do that, you send to one topic (full path) only and the listener decides what is interesting enough. The tree structure allows that (not sure about our implementation, but all standard brokers support this):
/verb/runVm will be received by anybody listening for that or for /verb/* or for /*.
The same for /response/runVm/16846158. MOM or hosted engine does not care about which exact command initiated the response and can listen for /response/runVm/*. On the other hand, the engine can be blocked in the call until the right response arrives.
Please remember that we are not message based but we have 2 logical ways of dealing with network frames: - rpc - events (aka messages)
Martin
On Wed, Jun 22, 2016 at 12:00 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <msivak@redhat.com>
If we subscribe to engine response queue we receive all the traffic.
At
the moment there is no way to define verbs we listen to. This functionality is only for events.
Can't we create a new topic /engine/<verb> for each verb and configure vdsm to listen for /engine/*? That would duplicate what we have now, but MOM would be able to subscribe more selectively.
It is not about having a topic but listing what should be posted to that topic. Sending to multiple topics is there but selecting a subset to send to one of them would need to be added. I am not sure whether it is good idea for sever to know what should be send where.
Martin
On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski <pkliczew@redhat.com
wrote:
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com>
wrote:
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com>
wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote: >> 1. Mom is still using xmlrpc >> >> Mom must move to jsonrpc. >> Martin: can you update on progress of this work? > > We would like to avoid going through VDSM completely, except from > the > broker part. Is it possible now to "parasitically" listen to vdsm > events and engine commands without having to go through VDSM API?
Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but that it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
> You can drop XML RPC,. vdsm does not depend on MOM working
anymore.
> Just the balloon and ksm stats will be missing from data that are > being sent to the engine.
and a lot of things stops working then;)
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs.
[1] https://gerrit.ovirt.org/#/c/59172/
Balloon and ksm stats are sent today from mom to vdsm, and reported
by
vdsm to engine?
If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm.
Piotr, how far are we from having engine listening for mom events using the vdsm broker?
I don’t think we should move any monitoring pulled from libvirt into mom - it would need another expensive and problematic call
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
> There also were some issues with eventfd in the json library, I > assume > those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942
But suffering from these issue show that the api was not use
wrote: properly,
you should create one client and reuse it for the entire life of the application.
Nir
> > Martin > > On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> > wrote: >> Hi all, >> >> We are still wasting time on maintaining both xmlrpc and jsonrpc. >> If >> we kill >> xmlrpc, we can greatly simplify the code base, making it easier to >> mainain >> and add new features. >> >> I suggest to kill xmlrpc in 4.1, and disable it *now* on master. >> >> Currently the we have 3 issues: >> >> 1. Mom is still using xmlrpc >> >> Mom must move to jsonrpc. >> Martin: can you update on progress of this work? >> >> 2. sos plugin using vdsClient >> >> Need to port it to use jsonrpc library, or jsonrpc client >> >> New jsonrpc client: https://gerrit.ovirt.org/35181 >> >> 3. Engine is using xmlrpc server for ovf upload/download >> >> We must support current engine in 4.1, so we cannot remove >> upload/download feature in this version, but we can remove the >> xmlrpc support in this server. >> >> Currently we abuse the xmlrpc server, supporting PUT and GET for >> upload and download (XMLRPC is using only POST). We can disable >> POST requests in protocoldetector, and not register anything with >> the xmlrpc server. >> >> Thoughts? >> >> Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
We do not have hierarchical topic system. Please note that we have p2p communication and real broker is not part of the scope anymore. We only have mini broker implemented in vdsm.
But then I can't implement the parasitic way of listening to data. This is basic functionality when you want to call something "a broker". Even a mini one...
There is distinction based on historical reasons.
Please remember that we are not message based but we have 2 logical ways of dealing with network frames: - rpc - events (aka messages)
Which we can easily emulate using events only for anything that needs it... everybody does that, because everybody needs blocking calls sometimes. But we seem to lack a very basic infrastructure to even attempt that. And I seem to remember telling you this same thing last year in Seattle :/ Knowing all this, I can drop XML-RPC from MOM and issue the exact same calls via JSON-RPC, but we will get nothing by that. Except maybe delete some protocol detector code... The load and vdsm dependencies will still stay the same. Martin On Wed, Jun 22, 2016 at 12:47 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 12:13 PM, Martin Sivak <msivak@redhat.com> wrote:
It is not about having a topic but listing what should be posted to that topic.
That is the whole point of having a hierarchical topic based system. Engine would listen for anything coming from vdsm, vdsm would listen for anything that comes from engine using a path wildcard.
We do not have hierarchical topic system. Please note that we have p2p communication and real broker is not part of the scope anymore. We only have mini broker implemented in vdsm.
Just use a /verb/<verbName> topic for each verb command automatically and either the same or something like /response/<verbName>/<correlation id> for the answer.
There is no reason to distinguish verbs and events. Both are events and the only difference is whether the initiating party is waiting for a response. You need some kind of correlation id for that of course (to identify the specific answer you are waiting for).
There is distinction based on historical reasons.
I am not sure whether it is good idea for sever to know what should be send where.
The sender always decides what the topic it is. At least in the rest of the broker universe. There is no such thing as a server or client, just publisher and listener for specific message topic wildcard.
Sending to multiple topics is there but selecting a subset to send to one of them
But you never do that, you send to one topic (full path) only and the listener decides what is interesting enough. The tree structure allows that (not sure about our implementation, but all standard brokers support this):
/verb/runVm will be received by anybody listening for that or for /verb/* or for /*.
The same for /response/runVm/16846158. MOM or hosted engine does not care about which exact command initiated the response and can listen for /response/runVm/*. On the other hand, the engine can be blocked in the call until the right response arrives.
Please remember that we are not message based but we have 2 logical ways of dealing with network frames: - rpc - events (aka messages)
Martin
On Wed, Jun 22, 2016 at 12:00 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <msivak@redhat.com> wrote:
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
Can't we create a new topic /engine/<verb> for each verb and configure vdsm to listen for /engine/*? That would duplicate what we have now, but MOM would be able to subscribe more selectively.
It is not about having a topic but listing what should be posted to that topic. Sending to multiple topics is there but selecting a subset to send to one of them would need to be added. I am not sure whether it is good idea for sever to know what should be send where.
Martin
On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> wrote: > > On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> > wrote: > >> 1. Mom is still using xmlrpc > >> > >> Mom must move to jsonrpc. > >> Martin: can you update on progress of this work? > > > > We would like to avoid going through VDSM completely, except from > > the > > broker part. Is it possible now to "parasitically" listen to vdsm > > events and engine commands without having to go through VDSM API? > > Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
> > Piotr, how far are we from letting mom listen to engine queue so it > get > engine > events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but that it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
> > > > You can drop XML RPC,. vdsm does not depend on MOM working > > anymore. > > Just the balloon and ksm stats will be missing from data that are > > being sent to the engine.
and a lot of things stops working then;)
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs.
[1] https://gerrit.ovirt.org/#/c/59172/
> > > Balloon and ksm stats are sent today from mom to vdsm, and reported > by > vdsm to engine? > > If vdsm is only a middleman and does not use this info, best to > send > it directly > to engine via the stomp broker part of vdsm. > > Piotr, how far are we from having engine listening for mom events > using > the > vdsm broker?
I don’t think we should move any monitoring pulled from libvirt into mom - it would need another expensive and problematic call
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
> > > > There also were some issues with eventfd in the json library, I > > assume > > those are fixed now. > > They should be fixed here: > https://gerrit.ovirt.org/57942 > > But suffering from these issue show that the api was not use > properly, > you should create one client and reuse it for the entire life of > the > application. > > > Nir > > > > > Martin > > > > On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> > > wrote: > >> Hi all, > >> > >> We are still wasting time on maintaining both xmlrpc and > >> jsonrpc. > >> If > >> we kill > >> xmlrpc, we can greatly simplify the code base, making it easier > >> to > >> mainain > >> and add new features. > >> > >> I suggest to kill xmlrpc in 4.1, and disable it *now* on master. > >> > >> Currently the we have 3 issues: > >> > >> 1. Mom is still using xmlrpc > >> > >> Mom must move to jsonrpc. > >> Martin: can you update on progress of this work? > >> > >> 2. sos plugin using vdsClient > >> > >> Need to port it to use jsonrpc library, or jsonrpc client > >> > >> New jsonrpc client: https://gerrit.ovirt.org/35181 > >> > >> 3. Engine is using xmlrpc server for ovf upload/download > >> > >> We must support current engine in 4.1, so we cannot remove > >> upload/download feature in this version, but we can remove the > >> xmlrpc support in this server. > >> > >> Currently we abuse the xmlrpc server, supporting PUT and GET for > >> upload and download (XMLRPC is using only POST). We can disable > >> POST requests in protocoldetector, and not register anything > >> with > >> the xmlrpc server. > >> > >> Thoughts? > >> > >> Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Jun 22, 2016 at 1:22 PM, Martin Sivak <msivak@redhat.com> wrote:
We do not have hierarchical topic system. Please note that we have p2p communication and real broker is not part of the scope anymore. We only have mini broker implemented in vdsm.
But then I can't implement the parasitic way of listening to data. This is basic functionality when you want to call something "a broker". Even a mini one...
We were implementing the code base on jsonrpc 2.0 and stomp 1.2 specs. None of them consider it basic.
There is distinction based on historical reasons.
Please remember that we are not message based but we have 2 logical ways of dealing with network frames: - rpc - events (aka messages)
Which we can easily emulate using events only for anything that needs it... everybody does that, because everybody needs blocking calls sometimes. But we seem to lack a very basic infrastructure to even attempt that. And I seem to remember telling you this same thing last year in Seattle :/
I disagree with: "everybody needs blocking calls sometimes"
Knowing all this, I can drop XML-RPC from MOM and issue the exact same calls via JSON-RPC, but we will get nothing by that. Except maybe delete some protocol detector code... The load and vdsm dependencies will still stay the same.
This is my point we want to drop xmlrpc functionality.
Martin
On Wed, Jun 22, 2016 at 12:47 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 12:13 PM, Martin Sivak <msivak@redhat.com>
It is not about having a topic but listing what should be posted to
topic.
That is the whole point of having a hierarchical topic based system. Engine would listen for anything coming from vdsm, vdsm would listen for anything that comes from engine using a path wildcard.
We do not have hierarchical topic system. Please note that we have p2p communication and real broker is not part of the scope anymore. We only have mini broker implemented in vdsm.
Just use a /verb/<verbName> topic for each verb command automatically and either the same or something like /response/<verbName>/<correlation id> for the answer.
There is no reason to distinguish verbs and events. Both are events and the only difference is whether the initiating party is waiting for a response. You need some kind of correlation id for that of course (to identify the specific answer you are waiting for).
There is distinction based on historical reasons.
I am not sure whether it is good idea for sever to know what should be send where.
The sender always decides what the topic it is. At least in the rest of the broker universe. There is no such thing as a server or client, just publisher and listener for specific message topic wildcard.
Sending to multiple topics is there but selecting a subset to send to one of them
But you never do that, you send to one topic (full path) only and the listener decides what is interesting enough. The tree structure allows that (not sure about our implementation, but all standard brokers support this):
/verb/runVm will be received by anybody listening for that or for /verb/* or for /*.
The same for /response/runVm/16846158. MOM or hosted engine does not care about which exact command initiated the response and can listen for /response/runVm/*. On the other hand, the engine can be blocked in the call until the right response arrives.
Please remember that we are not message based but we have 2 logical ways of dealing with network frames: - rpc - events (aka messages)
Martin
On Wed, Jun 22, 2016 at 12:00 PM, Piotr Kliczewski <pkliczew@redhat.com
wrote:
On Wed, Jun 22, 2016 at 11:40 AM, Martin Sivak <msivak@redhat.com> wrote:
If we subscribe to engine response queue we receive all the
At the moment there is no way to define verbs we listen to. This functionality is only for events.
Can't we create a new topic /engine/<verb> for each verb and configure vdsm to listen for /engine/*? That would duplicate what we have now, but MOM would be able to subscribe more selectively.
It is not about having a topic but listing what should be posted to
topic. Sending to multiple topics is there but selecting a subset to send to one of them would need to be added. I am not sure whether it is good idea for sever to know what should be send where.
Martin
On Wed, Jun 22, 2016 at 11:24 AM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek <michal.skrivanek@redhat.com> wrote: > > > On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> > wrote: > > > > On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> > wrote: >> >> On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <
msivak@redhat.com>
>> wrote: >> >> 1. Mom is still using xmlrpc >> >> >> >> Mom must move to jsonrpc. >> >> Martin: can you update on progress of this work? >> > >> > We would like to avoid going through VDSM completely, except from >> > the >> > broker part. Is it possible now to "parasitically" listen to vdsm >> > events and engine commands without having to go through VDSM API? >> >> Sounds like a good plan. > > > bypassing vdsm for mom things is a good idea, but we need to make > sure > we’re not killing the rest of the system > >> >> Piotr, how far are we from letting mom listen to engine queue so it >> get >> engine >> events/responses for certain verbs? > > > We can do it now for both events and responses. We need to remember > that > we would > receive anything that the engine is receiving. We could do better > but > that > it would take > more time. > > > so it would receive all responses for only the defined verbs? > It would wok well for getAllVmStats >
If we subscribe to engine response queue we receive all the
At the moment there is no way to define verbs we listen to. This functionality is only for events.
> > >> >> >> > You can drop XML RPC,. vdsm does not depend on MOM working >> > anymore. >> > Just the balloon and ksm stats will be missing from data that are >> > being sent to the engine. > > > and a lot of things stops working then;) >
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs.
[1] https://gerrit.ovirt.org/#/c/59172/ >> >> >> Balloon and ksm stats are sent today from mom to vdsm, and reported >> by >> vdsm to engine? >> >> If vdsm is only a middleman and does not use this info, best to >> send >> it directly >> to engine via the stomp broker part of vdsm. >> >> Piotr, how far are we from having engine listening for mom events >> using >> the >> vdsm broker? > > > I don’t think we should move any monitoring pulled from libvirt into > mom - > it would need another expensive and problematic call > > > It can be used now but the engine needs to know to listen for
wrote: that traffic. that traffic. them.
> We > would > need to implement engine subscriber (one class). > >> >> >> > There also were some issues with eventfd in the json library, I >> > assume >> > those are fixed now. >> >> They should be fixed here: >> https://gerrit.ovirt.org/57942 >> >> But suffering from these issue show that the api was not use >> properly, >> you should create one client and reuse it for the entire life of >> the >> application. >> >> >> Nir >> >> > >> > Martin >> > >> > On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer < nsoffer@redhat.com> >> > wrote: >> >> Hi all, >> >> >> >> We are still wasting time on maintaining both xmlrpc and >> >> jsonrpc. >> >> If >> >> we kill >> >> xmlrpc, we can greatly simplify the code base, making it easier >> >> to >> >> mainain >> >> and add new features. >> >> >> >> I suggest to kill xmlrpc in 4.1, and disable it *now* on master. >> >> >> >> Currently the we have 3 issues: >> >> >> >> 1. Mom is still using xmlrpc >> >> >> >> Mom must move to jsonrpc. >> >> Martin: can you update on progress of this work? >> >> >> >> 2. sos plugin using vdsClient >> >> >> >> Need to port it to use jsonrpc library, or jsonrpc client >> >> >> >> New jsonrpc client: https://gerrit.ovirt.org/35181 >> >> >> >> 3. Engine is using xmlrpc server for ovf upload/download >> >> >> >> We must support current engine in 4.1, so we cannot remove >> >> upload/download feature in this version, but we can remove the >> >> xmlrpc support in this server. >> >> >> >> Currently we abuse the xmlrpc server, supporting PUT and GET for >> >> upload and download (XMLRPC is using only POST). We can disable >> >> POST requests in protocoldetector, and not register anything >> >> with >> >> the xmlrpc server. >> >> >> >> Thoughts? >> >> >> >> Nir > > > _______________________________________________ > Devel mailing list > Devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > >
On Wed, Jun 22, 2016 at 12:24 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Wed, Jun 22, 2016 at 10:39 AM, Michal Skrivanek <michal.skrivanek@redhat.com> wrote:
On 20 Jun 2016, at 18:41, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Mon, Jun 20, 2016 at 6:22 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
Sounds like a good plan.
bypassing vdsm for mom things is a good idea, but we need to make sure we’re not killing the rest of the system
Piotr, how far are we from letting mom listen to engine queue so it get engine events/responses for certain verbs?
We can do it now for both events and responses. We need to remember that we would receive anything that the engine is receiving. We could do better but that it would take more time.
so it would receive all responses for only the defined verbs? It would wok well for getAllVmStats
If we subscribe to engine response queue we receive all the traffic. At the moment there is no way to define verbs we listen to. This functionality is only for events.
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
and a lot of things stops working then;)
Maybe we can disable [1] it before dropping it. We can slowly fix master and if needed we can enable it for specific envs.
Unfortunately this is not possible, as ovf upload/download depends on http server used by xmlrpc. We probably need to support engine 4.0 depending on this for couple of years. Disabling required removal of the xmlrpc verbs, maybe also the xmlrpc dispatcher, and leaving the http server serving GET and POST requests. Use BaseHTTPServer.BaseHTTPRequestHandler here: https://github.com/oVirt/vdsm/blob/master/lib/vdsm/rpc/bindingxmlrpc.py#L99 Use BaseHTTPServer.HTTPServer here: https://github.com/oVirt/vdsm/blob/master/lib/vdsm/rpc/bindingxmlrpc.py#L296 And finally we remove POST here: https://github.com/oVirt/vdsm/blob/master/lib/vdsm/rpc/bindingxmlrpc.py#L128... But until mom migrate to jsonrpc, I think the best would be to remove all the xmlrpc verbs except these used by mom, or the sos plugin. Martin listed the verbs used by mom here: http://lists.ovirt.org/pipermail/devel/2016-June/013298.html - getAllVmStats - vmSetCpuTuneQuota - vmSetCpuTunePeriod - getIoTunePolicy - getIoTune - setIoTune - setKsmTune sos plugin is using these (via vdsClient): - getVdsCapabilities - getVdsStats - getAllVmStats - list - getVGList - getDeviceList - getAllTasksInfo - getAllTasksStatuses - getConnectedStoragePoolsList - getSpmStatus Removing anything else from bindingxmlrpc.py is possible now.
Balloon and ksm stats are sent today from mom to vdsm, and reported by vdsm to engine?
If vdsm is only a middleman and does not use this info, best to send it directly to engine via the stomp broker part of vdsm.
Piotr, how far are we from having engine listening for mom events using the vdsm broker?
I don’t think we should move any monitoring pulled from libvirt into mom - it would need another expensive and problematic call
It can be used now but the engine needs to know to listen for them. We would need to implement engine subscriber (one class).
There also were some issues with eventfd in the json library, I assume those are fixed now.
They should be fixed here: https://gerrit.ovirt.org/57942
But suffering from these issue show that the api was not use properly, you should create one client and reuse it for the entire life of the application.
Nir
Martin
On Sun, Jun 19, 2016 at 1:02 PM, Nir Soffer <nsoffer@redhat.com> wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Thoughts?
Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Martin, can you list the verbs that mom is using? Removing the verbs that nobody is using now can be nice progress. Nir
On Mon, Jun 20, 2016 at 6:26 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Martin, can you list the verbs that mom is using?
Easily: Those below are currently used by MOM directly: getAllVmStats vmSetCpuTuneQuota vmSetCpuTunePeriod getIoTunePolicy getIoTune setIoTune setKsmTune We also have some verbs that are used by the engine and forwarded to MOM: setMOMPolicyParameters setMOMPolicy updateVmPolicy Btw, about stats collection: Balloon stats are collected by VDSM KSM stats are collected by MOM and VDSM grabs them in vdsm.momIF.MomClient#getKsmStats Martin
Removing the verbs that nobody is using now can be nice progress.
Nir
On 21 Jun 2016, at 10:11, Martin Sivak <msivak@redhat.com> wrote:
On Mon, Jun 20, 2016 at 6:26 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Mon, Jun 20, 2016 at 11:33 AM, Martin Sivak <msivak@redhat.com> wrote:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
We would like to avoid going through VDSM completely, except from the broker part. Is it possible now to "parasitically" listen to vdsm events and engine commands without having to go through VDSM API?
You can drop XML RPC,. vdsm does not depend on MOM working anymore. Just the balloon and ksm stats will be missing from data that are being sent to the engine.
Martin, can you list the verbs that mom is using?
Easily:
Those below are currently used by MOM directly:
getAllVmStats vmSetCpuTuneQuota vmSetCpuTunePeriod getIoTunePolicy getIoTune setIoTune setKsmTune
We also have some verbs that are used by the engine and forwarded to MOM:
setMOMPolicyParameters setMOMPolicy updateVmPolicy
Btw, about stats collection:
Balloon stats are collected by VDSM KSM stats are collected by MOM and VDSM grabs them in vdsm.momIF.MomClient#getKsmStats
should probably think about keeping it this way or letting mom to send this data…but not sure it’s worth the troubles of twice as many host stats updates it doesn’t sounds like much benefit to change it right now. What do you think?
Martin
Removing the verbs that nobody is using now can be nice progress.
Nir
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Sun, Jun 19, 2016 at 02:02:23PM +0300, Nir Soffer wrote:
Hi all,
We are still wasting time on maintaining both xmlrpc and jsonrpc. If we kill xmlrpc, we can greatly simplify the code base, making it easier to mainain and add new features.
I suggest to kill xmlrpc in 4.1, and disable it *now* on master.
Currently the we have 3 issues:
1. Mom is still using xmlrpc
Mom must move to jsonrpc. Martin: can you update on progress of this work?
2. sos plugin using vdsClient
Need to port it to use jsonrpc library, or jsonrpc client
New jsonrpc client: https://gerrit.ovirt.org/35181
3. Engine is using xmlrpc server for ovf upload/download
We must support current engine in 4.1, so we cannot remove upload/download feature in this version, but we can remove the xmlrpc support in this server.
Currently we abuse the xmlrpc server, supporting PUT and GET for upload and download (XMLRPC is using only POST). We can disable POST requests in protocoldetector, and not register anything with the xmlrpc server.
Another user is our internal QE, which sometime contact vdsm directly via xmlrpc. They, too, should move to jsonrpccli.
participants (7)
-
Dan Kenigsberg -
Martin Sivak -
Michal Skrivanek -
Nir Soffer -
Piotr Kliczewski -
Sven Kieske -
Yedidyah Bar David