On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote: > Hi everyone. On today's VDSM call we discussed the requirements, design, and > plan for updating the API to include support for QMF and single-host REST API. > All members present arrived at a general consensus on the best way to design the > next-generation API. I have tried to capture this discussion in the oVirt wiki: > > http://ovirt.org/wiki/Vdsm_API > > Please take a look at this page and let's discuss any changes that may be needed > in order to adopt it as a working plan that we can begin to execute. Thanks! > Very nice, I've fixed two bullets about the future of the xml-rpc.

I think that we are missing something here: how do we model Vdsm-to-Vdsm communication, in a binding-blind way? I'm less worried about the storage-based mailbox used for lvextend requests: my problem is with migration command.

Currently, the implementation of the "migrate" verb includes contacting the remote Vdsm over xml-rpc before issuing the libvirt migrateToURI2 command ('migrationCreate' verb). A Vdsm user who choose to use the REST binding, is likely to want this to be implemented this using a REST request to the destination. This means that the implementation of Vdsm depends on the chosen binding. The issue can be mitigating by requiring the binding level to provide a "callback" for migrationCreate (and any other future Vdsm->world requests). This would complicate the beautiful png at http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have another suggestion?

On Tue, Dec 06, 2011 at 08:46:57AM -0600, Adam Litke wrote: > On Tue, Dec 06, 2011 at 02:58:59PM +0200, Dan Kenigsberg wrote: > > On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote: > > > Hi everyone. On today's VDSM call we discussed the > > > requirements, design, and > > > plan for updating the API to include support for QMF and > > > single-host REST API. > > > All members present arrived at a general consensus on the best > > > way to design the > > > next-generation API. I have tried to capture this discussion > > > in the oVirt wiki: > > > > > > http://ovirt.org/wiki/Vdsm_API > > > > > > Please take a look at this page and let's discuss any changes > > > that may be needed > > > in order to adopt it as a working plan that we can begin to > > > execute. Thanks! > > > > > > > Very nice, I've fixed two bullets about the future of the > > xml-rpc. > > Thanks... Updates look good to me. > > > I think that we are missing something here: how do we model > > Vdsm-to-Vdsm > > communication, in a binding-blind way? I'm less worried about the > > storage-based mailbox used for lvextend requests: my problem is > > with > > migration command. > > Ok, interesting... Besides migration, are there other features > (current or > planned) that would involve P2P communication? I want to ensure we > consider the > full problem space. Well, I can imagine we would like a host in distress to migrate VMs to whomever can take them, without central management driving this process. (CAVE split brain) At the momemt I cannot think of something that cannot be implemented by QMF events. Ayal? > > > Currently, the implementation of the "migrate" verb includes > > contacting > > the remote Vdsm over xml-rpc before issuing the libvirt > > migrateToURI2 > > command ('migrationCreate' verb). > > > > A Vdsm user who choose to use the REST binding, is likely to want > > this to > > be implemented this using a REST request to the destination. This > > means > > that the implementation of Vdsm depends on the chosen binding. > > > > The issue can be mitigating by requiring the binding level to > > provide a > > "callback" for migrationCreate (and any other future Vdsm->world > > requests). > > This would complicate the beautiful png at > > http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have > > another > > suggestion? > > Actually, I think you are blending the external API with vdsm > internals. As a > management server or ovirt-engine, I don't care about the protocol > that vdsm > uses to contact the migration recipient. As far as I am concerned > this is a > special case internal function call. For that purpose, I think > xmlrpc is > perfectly well-suited to the task and should be used > unconditionally, regardless > of the bindings used to initiate the migration. > > So I would propose that we modify the design such that we keep an > extremely thin > xmlrpc server active whose sole purpose is to service internal P2P > requests. Interesting. We could avoid even that, if we could register a callback with libvirt, so that destination libvirtd called destination Vdsm to verify that all storage and networking resources are ready, before executing qemu. DanPB, can something like that be done? (I guess it is not realistic since we may need to pass vdsm-specific data from source to dest, and libvirt is not supposed to be a general purpose transport.) Dan.

_______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel

On Thu, Dec 08, 2011 at 04:56:17AM -0500, Ayal Baron wrote: > > > ----- Original Message ----- > > On Tue, Dec 06, 2011 at 08:46:57AM -0600, Adam Litke wrote: > > > On Tue, Dec 06, 2011 at 02:58:59PM +0200, Dan Kenigsberg wrote: > > > > On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote: > > > > > Hi everyone. On today's VDSM call we discussed the > > > > > requirements, design, and > > > > > plan for updating the API to include support for QMF and > > > > > single-host REST API. > > > > > All members present arrived at a general consensus on the > > > > > best > > > > > way to design the > > > > > next-generation API. I have tried to capture this > > > > > discussion > > > > > in the oVirt wiki: > > > > > > > > > > http://ovirt.org/wiki/Vdsm_API > > > > > > > > > > Please take a look at this page and let's discuss any > > > > > changes > > > > > that may be needed > > > > > in order to adopt it as a working plan that we can begin to > > > > > execute. Thanks! > > > > > > > > > > > > > Very nice, I've fixed two bullets about the future of the > > > > xml-rpc. > > > > > > Thanks... Updates look good to me. > > > > > > > I think that we are missing something here: how do we model > > > > Vdsm-to-Vdsm > > > > communication, in a binding-blind way? I'm less worried about > > > > the > > > > storage-based mailbox used for lvextend requests: my problem > > > > is > > > > with > > > > migration command. > > > > > > Ok, interesting... Besides migration, are there other features > > > (current or > > > planned) that would involve P2P communication? I want to > > > ensure we > > > consider the > > > full problem space. > > > > Well, I can imagine we would like a host in distress to migrate > > VMs > > to > > whomever can take them, without central management driving this > > process. > > (CAVE split brain) > > > > At the momemt I cannot think of something that cannot be > > implemented > > by > > QMF events. Ayal? > > > > > > > > > Currently, the implementation of the "migrate" verb includes > > > > contacting > > > > the remote Vdsm over xml-rpc before issuing the libvirt > > > > migrateToURI2 > > > > command ('migrationCreate' verb). > > > > > > > > A Vdsm user who choose to use the REST binding, is likely to > > > > want > > > > this to > > > > be implemented this using a REST request to the destination. > > > > This > > > > means > > > > that the implementation of Vdsm depends on the chosen > > > > binding. > > > > > > > > The issue can be mitigating by requiring the binding level to > > > > provide a > > > > "callback" for migrationCreate (and any other future > > > > Vdsm->world > > > > requests). > > > > This would complicate the beautiful png at > > > > http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have > > > > another > > > > suggestion? > > > > > > Actually, I think you are blending the external API with vdsm > > > internals. As a > > > management server or ovirt-engine, I don't care about the > > > protocol > > > that vdsm > > > uses to contact the migration recipient. As far as I am > > > concerned > > > this is a > > > special case internal function call. For that purpose, I think > > > xmlrpc is > > > perfectly well-suited to the task and should be used > > > unconditionally, regardless > > > of the bindings used to initiate the migration. > > > > > > So I would propose that we modify the design such that we keep > > > an > > > extremely thin > > > xmlrpc server active whose sole purpose is to service internal > > > P2P > > > requests. > > > > Interesting. We could avoid even that, if we could register a > > callback > > with libvirt, so that destination libvirtd called destination > > Vdsm to > > verify that all storage and networking resources are ready, > > before > > executing qemu. DanPB, can something like that be done? (I guess > > it > > is > > not realistic since we may need to pass vdsm-specific data from > > source > > to dest, and libvirt is not supposed to be a general purpose > > transport.) > > > > Dan. > > I don't understand the issue. The whole point of the REST API is > to be an > easily consumable *single* node management API. Once you start > coordinating > among different nodes then you need clustering and management > (either > distributed or centralized), in both cases it is fine to require > having a bus > in which case you have your method of communications between hosts > to replace > current xml-rpc. Implicit in this statement is an assertion that live migration between two vdsm instances will not be supported without orchestration from an ovirt-engine instance. I don't agree with placing such a limitation on vdsm since p2p migration is already well-supported by the underlying components (libvirt and qemu).

> Requiring an additional xml-rpc server sounds wrong to me. The other option is to support a migrateCreate binding in REST and QMF.

-- Adam Litke <agl(a)us.ibm.com&gt; IBM Linux Technology Center

On 12/08/2011 04:34 PM, Ayal Baron wrote: > I just noticed, why are we cross-posting to engine-devel? > > Anyway, comments inline > > ----- Original Message ----- >> On Thu, Dec 08, 2011 at 04:56:17AM -0500, Ayal Baron wrote: >>> >>> ----- Original Message ----- >>>> On Tue, Dec 06, 2011 at 08:46:57AM -0600, Adam Litke wrote: >>>>> On Tue, Dec 06, 2011 at 02:58:59PM +0200, Dan Kenigsberg wrote: >>>>>> On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote: >>>>>>> Hi everyone. On today's VDSM call we discussed the >>>>>>> requirements, design, and >>>>>>> plan for updating the API to include support for QMF and >>>>>>> single-host REST API. >>>>>>> All members present arrived at a general consensus on the >>>>>>> best >>>>>>> way to design the >>>>>>> next-generation API. I have tried to capture this >>>>>>> discussion >>>>>>> in the oVirt wiki: >>>>>>> >>>>>>> http://ovirt.org/wiki/Vdsm_API >>>>>>> >>>>>>> Please take a look at this page and let's discuss any >>>>>>> changes >>>>>>> that may be needed >>>>>>> in order to adopt it as a working plan that we can begin to >>>>>>> execute. Thanks! >>>>>>> >>>>>> Very nice, I've fixed two bullets about the future of the >>>>>> xml-rpc. >>>>> Thanks... Updates look good to me. >>>>> >>>>>> I think that we are missing something here: how do we model >>>>>> Vdsm-to-Vdsm >>>>>> communication, in a binding-blind way? I'm less worried about >>>>>> the >>>>>> storage-based mailbox used for lvextend requests: my problem >>>>>> is >>>>>> with >>>>>> migration command. >>>>> Ok, interesting... Besides migration, are there other features >>>>> (current or >>>>> planned) that would involve P2P communication? I want to >>>>> ensure we >>>>> consider the >>>>> full problem space. >>>> Well, I can imagine we would like a host in distress to migrate >>>> VMs >>>> to >>>> whomever can take them, without central management driving this >>>> process. >>>> (CAVE split brain) >>>> >>>> At the momemt I cannot think of something that cannot be >>>> implemented >>>> by >>>> QMF events. Ayal? >>>> >>>>>> Currently, the implementation of the "migrate" verb includes >>>>>> contacting >>>>>> the remote Vdsm over xml-rpc before issuing the libvirt >>>>>> migrateToURI2 >>>>>> command ('migrationCreate' verb). >>>>>> >>>>>> A Vdsm user who choose to use the REST binding, is likely to >>>>>> want >>>>>> this to >>>>>> be implemented this using a REST request to the destination. >>>>>> This >>>>>> means >>>>>> that the implementation of Vdsm depends on the chosen >>>>>> binding. >>>>>> >>>>>> The issue can be mitigating by requiring the binding level to >>>>>> provide a >>>>>> "callback" for migrationCreate (and any other future >>>>>> Vdsm->world >>>>>> requests). >>>>>> This would complicate the beautiful png at >>>>>> http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have >>>>>> another >>>>>> suggestion? >>>>> Actually, I think you are blending the external API with vdsm >>>>> internals. As a >>>>> management server or ovirt-engine, I don't care about the >>>>> protocol >>>>> that vdsm >>>>> uses to contact the migration recipient. As far as I am >>>>> concerned >>>>> this is a >>>>> special case internal function call. For that purpose, I think >>>>> xmlrpc is >>>>> perfectly well-suited to the task and should be used >>>>> unconditionally, regardless >>>>> of the bindings used to initiate the migration. >>>>> >>>>> So I would propose that we modify the design such that we keep >>>>> an >>>>> extremely thin >>>>> xmlrpc server active whose sole purpose is to service internal >>>>> P2P >>>>> requests. >>>> Interesting. We could avoid even that, if we could register a >>>> callback >>>> with libvirt, so that destination libvirtd called destination >>>> Vdsm to >>>> verify that all storage and networking resources are ready, >>>> before >>>> executing qemu. DanPB, can something like that be done? (I guess >>>> it >>>> is >>>> not realistic since we may need to pass vdsm-specific data from >>>> source >>>> to dest, and libvirt is not supposed to be a general purpose >>>> transport.) >>>> >>>> Dan. >>> I don't understand the issue. The whole point of the REST API is >>> to be an >>> easily consumable *single* node management API. Once you start >>> coordinating >>> among different nodes then you need clustering and management >>> (either >>> distributed or centralized), in both cases it is fine to require >>> having a bus >>> in which case you have your method of communications between hosts >>> to replace >>> current xml-rpc. >> Implicit in this statement is an assertion that live migration >> between two vdsm >> instances will not be supported without orchestration from an >> ovirt-engine >> instance. I don't agree with placing such a limitation on vdsm since >> p2p >> migration is already well-supported by the underlying components >> (libvirt and >> qemu). > proper migration requires a lot of coordination. > e.g. making sure that both source and target have the same networks > (required by guest) available, access to shared storage, access to > same vm configuration, handling split brain scenarios, etc. > This has to be managed somehow... > I don't think that ovirt-engine should be a requirement for this, but > something should take care of this. This entity could just initiate > migrateCreate on destination if not using qmf. > >>> Requiring an additional xml-rpc server sounds wrong to me. >> The other option is to support a migrateCreate binding in REST and >> QMF. > I see no problem with this, I don't view preparing to accept a > migration as an internal flow, it can be directed by either another > vdsm or a management server or anything else. Seeing as it is by any > event run by an external entity, it should be a public API. > > >> -- >> Adam Litke&lt;agl(a)us.ibm.com&gt; >> IBM Linux Technology Center >> >> > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel Is there a bias towards using the mailing list and against the discussion tab on the wiki page when discussing wiki pages? Just want to know. I'm used to commenting on wikis on the discussion page.

From: "Ayal Baron" <abaron(a)redhat.com&gt; To: "Itamar Heim" <iheim(a)redhat.com&gt; Cc: engine-devel(a)ovirt.org, "Jon Choate" <jchoate(a)redhat.com&gt; Sent: Monday, December 12, 2011 4:40:37 PM Subject: Re: [Engine-devel] API design and plan ----- Original Message ----- > On 12/12/2011 08:18 PM, Jon Choate wrote: > > On 12/08/2011 04:34 PM, Ayal Baron wrote: > >> I just noticed, why are we cross-posting to engine-devel? > >> > >> Anyway, comments inline > >> > >> ----- Original Message ----- > >>> On Thu, Dec 08, 2011 at 04:56:17AM -0500, Ayal Baron wrote: > >>>> > >>>> ----- Original Message ----- > >>>>> On Tue, Dec 06, 2011 at 08:46:57AM -0600, Adam Litke wrote: > >>>>>> On Tue, Dec 06, 2011 at 02:58:59PM +0200, Dan Kenigsberg > >>>>>> wrote: > >>>>>>> On Mon, Dec 05, 2011 at 11:34:18AM -0600, Adam Litke wrote: > >>>>>>>> Hi everyone. On today's VDSM call we discussed the > >>>>>>>> requirements, design, and > >>>>>>>> plan for updating the API to include support for QMF and > >>>>>>>> single-host REST API. > >>>>>>>> All members present arrived at a general consensus on the > >>>>>>>> best > >>>>>>>> way to design the > >>>>>>>> next-generation API. I have tried to capture this > >>>>>>>> discussion > >>>>>>>> in the oVirt wiki: > >>>>>>>> > >>>>>>>> http://ovirt.org/wiki/Vdsm_API > >>>>>>>> > >>>>>>>> Please take a look at this page and let's discuss any > >>>>>>>> changes > >>>>>>>> that may be needed > >>>>>>>> in order to adopt it as a working plan that we can begin > >>>>>>>> to > >>>>>>>> execute. Thanks! > >>>>>>>> > >>>>>>> Very nice, I've fixed two bullets about the future of the > >>>>>>> xml-rpc. > >>>>>> Thanks... Updates look good to me. > >>>>>> > >>>>>>> I think that we are missing something here: how do we model > >>>>>>> Vdsm-to-Vdsm > >>>>>>> communication, in a binding-blind way? I'm less worried > >>>>>>> about > >>>>>>> the > >>>>>>> storage-based mailbox used for lvextend requests: my > >>>>>>> problem > >>>>>>> is > >>>>>>> with > >>>>>>> migration command. > >>>>>> Ok, interesting... Besides migration, are there other > >>>>>> features > >>>>>> (current or > >>>>>> planned) that would involve P2P communication? I want to > >>>>>> ensure we > >>>>>> consider the > >>>>>> full problem space. > >>>>> Well, I can imagine we would like a host in distress to > >>>>> migrate > >>>>> VMs > >>>>> to > >>>>> whomever can take them, without central management driving > >>>>> this > >>>>> process. > >>>>> (CAVE split brain) > >>>>> > >>>>> At the momemt I cannot think of something that cannot be > >>>>> implemented > >>>>> by > >>>>> QMF events. Ayal? > >>>>> > >>>>>>> Currently, the implementation of the "migrate" verb > >>>>>>> includes > >>>>>>> contacting > >>>>>>> the remote Vdsm over xml-rpc before issuing the libvirt > >>>>>>> migrateToURI2 > >>>>>>> command ('migrationCreate' verb). > >>>>>>> > >>>>>>> A Vdsm user who choose to use the REST binding, is likely > >>>>>>> to > >>>>>>> want > >>>>>>> this to > >>>>>>> be implemented this using a REST request to the > >>>>>>> destination. > >>>>>>> This > >>>>>>> means > >>>>>>> that the implementation of Vdsm depends on the chosen > >>>>>>> binding. > >>>>>>> > >>>>>>> The issue can be mitigating by requiring the binding level > >>>>>>> to > >>>>>>> provide a > >>>>>>> "callback" for migrationCreate (and any other future > >>>>>>> Vdsm->world > >>>>>>> requests). > >>>>>>> This would complicate the beautiful png at > >>>>>>> http://ovirt.org/wiki/Vdsm_API#Design ... Does anyone have > >>>>>>> another > >>>>>>> suggestion? > >>>>>> Actually, I think you are blending the external API with > >>>>>> vdsm > >>>>>> internals. As a > >>>>>> management server or ovirt-engine, I don't care about the > >>>>>> protocol > >>>>>> that vdsm > >>>>>> uses to contact the migration recipient. As far as I am > >>>>>> concerned > >>>>>> this is a > >>>>>> special case internal function call. For that purpose, I > >>>>>> think > >>>>>> xmlrpc is > >>>>>> perfectly well-suited to the task and should be used > >>>>>> unconditionally, regardless > >>>>>> of the bindings used to initiate the migration. > >>>>>> > >>>>>> So I would propose that we modify the design such that we > >>>>>> keep > >>>>>> an > >>>>>> extremely thin > >>>>>> xmlrpc server active whose sole purpose is to service > >>>>>> internal > >>>>>> P2P > >>>>>> requests. > >>>>> Interesting. We could avoid even that, if we could register a > >>>>> callback > >>>>> with libvirt, so that destination libvirtd called destination > >>>>> Vdsm to > >>>>> verify that all storage and networking resources are ready, > >>>>> before > >>>>> executing qemu. DanPB, can something like that be done? (I > >>>>> guess > >>>>> it > >>>>> is > >>>>> not realistic since we may need to pass vdsm-specific data > >>>>> from > >>>>> source > >>>>> to dest, and libvirt is not supposed to be a general purpose > >>>>> transport.) > >>>>> > >>>>> Dan. > >>>> I don't understand the issue. The whole point of the REST API > >>>> is > >>>> to be an > >>>> easily consumable *single* node management API. Once you start > >>>> coordinating > >>>> among different nodes then you need clustering and management > >>>> (either > >>>> distributed or centralized), in both cases it is fine to > >>>> require > >>>> having a bus > >>>> in which case you have your method of communications between > >>>> hosts > >>>> to replace > >>>> current xml-rpc. > >>> Implicit in this statement is an assertion that live migration > >>> between two vdsm > >>> instances will not be supported without orchestration from an > >>> ovirt-engine > >>> instance. I don't agree with placing such a limitation on vdsm > >>> since > >>> p2p > >>> migration is already well-supported by the underlying > >>> components > >>> (libvirt and > >>> qemu). > >> proper migration requires a lot of coordination. > >> e.g. making sure that both source and target have the same > >> networks > >> (required by guest) available, access to shared storage, access > >> to > >> same vm configuration, handling split brain scenarios, etc. > >> This has to be managed somehow... > >> I don't think that ovirt-engine should be a requirement for > >> this, > >> but > >> something should take care of this. This entity could just > >> initiate > >> migrateCreate on destination if not using qmf. > >> > >>>> Requiring an additional xml-rpc server sounds wrong to me. > >>> The other option is to support a migrateCreate binding in REST > >>> and > >>> QMF. > >> I see no problem with this, I don't view preparing to accept a > >> migration as an internal flow, it can be directed by either > >> another > >> vdsm or a management server or anything else. Seeing as it is by > >> any > >> event run by an external entity, it should be a public API. > >> > >> > >>> -- > >>> Adam Litke&lt;agl(a)us.ibm.com&gt; > >>> IBM Linux Technology Center > >>> > >>> > >> _______________________________________________ > >> Engine-devel mailing list > >> Engine-devel(a)ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/engine-devel > > Is there a bias towards using the mailing list and against the > > discussion tab on the wiki page when discussing wiki pages? Just > > want to > > know. I'm used to commenting on wikis on the discussion page. > > mailing list - shouldn't be. > wiki - may be lost on the larger audience - I think discussing on > the > mailing list makes more sense since many don't subscribe to the > wiki > pages after their first review from when it was published. No offense, but the only thing which is more broken than email for discussing such things is a wiki discussion (mailing list at least pushes changes by default. has clients to manage replies by threads, responses can be sent concurrently etc). > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel >

Hi everyone. On today's VDSM call we discussed the requirements, design, and plan for updating the API to include support for QMF and single-host REST API. All members present arrived at a general consensus on the best way to design the next-generation API. I have tried to capture this discussion in the oVirt wiki: http://ovirt.org/wiki/Vdsm_API Please take a look at this page and let's discuss any changes that may be needed in order to adopt it as a working plan that we can begin to execute. Thanks!