[Engine-devel] Gluster Volume asynchronous tasks

[Eli Mesika]

----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Eli Mesika" <emesika at redhat.com>
> Cc: "Sahina Bose" <sabose at redhat.com>, "engine-devel" <engine-devel at ovirt.org>, arch at ovirt.org
> Sent: Wednesday, August 21, 2013 6:24:21 PM
> Subject: Re: [Engine-devel] Gluster Volume asynchronous tasks
> 
> On 08/21/2013 11:15 AM, Eli Mesika wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Sahina Bose" <sabose at redhat.com>
> >> To: "Itamar Heim" <iheim at redhat.com>
> >> Cc: "engine-devel" <engine-devel at ovirt.org>, arch at ovirt.org
> >> Sent: Wednesday, August 21, 2013 2:19:51 PM
> >> Subject: Re: [Engine-devel] Gluster Volume asynchronous tasks
> >>
> >>
> >> On 08/21/2013 03:54 PM, Itamar Heim wrote:
> >>> On 08/21/2013 12:46 AM, Sahina Bose wrote:
> >>>>
> >>>> On 08/20/2013 04:00 AM, Itamar Heim wrote:
> >>>>> On 08/12/2013 06:09 AM, Sahina Bose wrote:
> >>>>>>
> >>>>>> On 08/12/2013 03:28 PM, Yair Zaslavsky wrote:
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>>> From: "Sahina Bose" <sabose at redhat.com>
> >>>>>>>> To: "Eli Mesika" <emesika at redhat.com>
> >>>>>>>> Cc: "engine-devel" <engine-devel at ovirt.org>, arch at ovirt.org
> >>>>>>>> Sent: Monday, August 12, 2013 11:51:15 AM
> >>>>>>>> Subject: Re: [Engine-devel] Gluster Volume asynchronous tasks
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 08/12/2013 01:21 PM, Eli Mesika wrote:
> >>>>>>>>> ----- Original Message -----
> >>>>>>>>>> From: "Sahina Bose" <sabose at redhat.com>
> >>>>>>>>>> To: "engine-devel" <engine-devel at ovirt.org>, arch at ovirt.org,
> >>>>>>>>>> "Michael
> >>>>>>>>>> Pasternak" <mpastern at redhat.com>
> >>>>>>>>>> Sent: Monday, August 12, 2013 8:41:55 AM
> >>>>>>>>>> Subject: [Engine-devel] Gluster Volume asynchronous tasks
> >>>>>>>>>>
> >>>>>>>>>> Hi all,
> >>>>>>>>>>
> >>>>>>>>>> We are working on a feature to add support to start and monitor
> >>>>>>>>>> gluster
> >>>>>>>>>> volume asynchronous tasks (like rebalancing a gluster volume,
> >>>>>>>>>> removing
> >>>>>>>>>> brick from volume ) from the oVirt engine.
> >>>>>>>>>>
> >>>>>>>>>> The operations can be started from the Volumes tab or the Bricks
> >>>>>>>>>> sub-tab
> >>>>>>>>>> using the Rebalance, Remove options.
> >>>>>>>>>> These are long running operations which can be monitored using a
> >>>>>>>>>> task id
> >>>>>>>>>> returned from Gluster. We are planning to add the monitoring in
> >>>>>>>>>> the
> >>>>>>>>>> existing Task sub tab
> >>>>>>>>>>
> >>>>>>>>>> The feature description and User flows are at
> >>>>>>>>>> http://www.ovirt.org/Features/Gluster_Volume_Asynchronous_Tasks_Management
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> The detailed design (including REST API design) is at
> >>>>>>>>>> http://www.ovirt.org/Features/Detailed_Gluster_Volume_Asynchronous_Tasks_Management.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> I would really appreciate if you could review and provide your
> >>>>>>>>>> valuable
> >>>>>>>>>> feedback.
> >>>>>>>>> I Sahina
> >>>>>>>>> Why not using 6the External Tasks feature introduced for 3.3 for
> >>>>>>>>> those
> >>>>>>>>> Gluster tasks ???
> >>>>>>>>> http://www.ovirt.org/Features/Design/DetailedExternalTasks
> >>>>>>>> Hi Eli,
> >>>>>>>>
> >>>>>>>> We still want to be able to start and stop these operations from the
> >>>>>>>> engine.
> >>>>>>>> So, when a user wants to say, rebalance a volume, they would go
> >>>>>>>> select
> >>>>>>>> the volume and click on Rebalance Start.
> >>>>>>>> This would then call the BLL command to start rebalance which will
> >>>>>>>> invoke the corresponding vdsm verb to start the rebalance on the
> >>>>>>>> volume.
> >>>>>>>> This is the same as existing flow for other commands. The only
> >>>>>>>> difference is the vdsm verb will return the task id from gluster,
> >>>>>>>> for
> >>>>>>>> the rebalance operation that was started. And we will monitor the
> >>>>>>>> progress of the task using the gluster task id (by calling a gluster
> >>>>>>>> command)
> >>>>>>>>
> >>>>>>>> I'm not sure how ExternalTasks would fit in here? I was thinking of
> >>>>>>>> using ExternalTask support for adding Job/Steps to engine when the
> >>>>>>>> operation is started outside of engine, that is, from Gluster CLI.
> >>>>>>>> Please correct me if I'm missing something.
> >>>>>>> Does this mean that from Gluster CLI you will not try and invoke the
> >>>>>>> rebalance command ?
> >>>>>>> (I mean, I should either use Gluster CLI or Engine's REST API?)
> >>>>>> Rebalance volume command could be invoked in any of the following
> >>>>>> ways:
> >>>>>> 1. From the console UI (clicking on Rebalance as shown in
> >>>>>> http://www.ovirt.org/Features/Gluster_Volume_Asynchronous_Tasks_Management#Rebalance_Volume)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> 2. Using REST API
> >>>>>> 3. Outside of engine, from Gluster CLI - In such cases, the engine
> >>>>>> should detect that a user has triggered rebalance operation outside
> >>>>>> the
> >>>>>> engine, and allow the user to monitor progress of this from the
> >>>>>> engine.
> >>>>>> This is where, we need support to add a Job for an operation that was
> >>>>>> started externally, so that it can be seen in the Tasks tab.
> >>>>>
> >>>>> and still, it should be considered an internal task, since the engine
> >>>>> is managing it / detected it.
> >>>>>
> >>>>
> >>>> Itamar, yes, you are right. This would need to be treated as an internal
> >>>> task as the engine needs to be able to stop it and also monitor it. We
> >>>> would probably need a similar mechanism as external task injection, to
> >>>> add a Job for the task started from gluster CLI.
> >>>>
> >>>>
> >>>
> >>> even if it was started from CLI, wouldn't it be better if engine
> >>> detected it, and still treated it as an internal task, allowing to
> >>> cancel it, etc.?
> >>
> >> Yes, but I need to add a Job for this internal task, so that it can be
> >> monitored in the Tasks pane. Any idea if I can use any existing
> >> framework to do it? I was thinking I would use
> >> ExecutionHandler.createJob to do this (similar to what's done in
> >> AddExternalJobCommand)
> >
> > Maybe in order to avoid code duplication we should re-factor having a base
> > AddJobCommand and two descendants AddExternalJobCommand and
> > AddInternalJobCommand when the only diff between those is the external
> > flag value while the execute method of both calls super at first and the
> > ancestor has the original code of ddExternalJobCommand in its execute
> > method...
> 
> and in this case, the code will call AddInternalJobCommand for a job
> detected and not initiated by engine?

Yes, Gluster will use the AddInternalJobCommand
> 
>