[Engine-devel] Gluster Volume asynchronous tasks

[Itamar Heim]

On 08/21/2013 11:15 AM, Eli Mesika wrote:
>
>
> ----- Original Message -----
>> From: "Sahina Bose" <sabose at redhat.com>
>> To: "Itamar Heim" <iheim at redhat.com>
>> Cc: "engine-devel" <engine-devel at ovirt.org>, arch at ovirt.org
>> Sent: Wednesday, August 21, 2013 2:19:51 PM
>> Subject: Re: [Engine-devel] Gluster Volume asynchronous tasks
>>
>>
>> On 08/21/2013 03:54 PM, Itamar Heim wrote:
>>> On 08/21/2013 12:46 AM, Sahina Bose wrote:
>>>>
>>>> On 08/20/2013 04:00 AM, Itamar Heim wrote:
>>>>> On 08/12/2013 06:09 AM, Sahina Bose wrote:
>>>>>>
>>>>>> On 08/12/2013 03:28 PM, Yair Zaslavsky wrote:
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Sahina Bose" <sabose at redhat.com>
>>>>>>>> To: "Eli Mesika" <emesika at redhat.com>
>>>>>>>> Cc: "engine-devel" <engine-devel at ovirt.org>, arch at ovirt.org
>>>>>>>> Sent: Monday, August 12, 2013 11:51:15 AM
>>>>>>>> Subject: Re: [Engine-devel] Gluster Volume asynchronous tasks
>>>>>>>>
>>>>>>>>
>>>>>>>> On 08/12/2013 01:21 PM, Eli Mesika wrote:
>>>>>>>>> ----- Original Message -----
>>>>>>>>>> From: "Sahina Bose" <sabose at redhat.com>
>>>>>>>>>> To: "engine-devel" <engine-devel at ovirt.org>, arch at ovirt.org,
>>>>>>>>>> "Michael
>>>>>>>>>> Pasternak" <mpastern at redhat.com>
>>>>>>>>>> Sent: Monday, August 12, 2013 8:41:55 AM
>>>>>>>>>> Subject: [Engine-devel] Gluster Volume asynchronous tasks
>>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> We are working on a feature to add support to start and monitor
>>>>>>>>>> gluster
>>>>>>>>>> volume asynchronous tasks (like rebalancing a gluster volume,
>>>>>>>>>> removing
>>>>>>>>>> brick from volume ) from the oVirt engine.
>>>>>>>>>>
>>>>>>>>>> The operations can be started from the Volumes tab or the Bricks
>>>>>>>>>> sub-tab
>>>>>>>>>> using the Rebalance, Remove options.
>>>>>>>>>> These are long running operations which can be monitored using a
>>>>>>>>>> task id
>>>>>>>>>> returned from Gluster. We are planning to add the monitoring in
>>>>>>>>>> the
>>>>>>>>>> existing Task sub tab
>>>>>>>>>>
>>>>>>>>>> The feature description and User flows are at
>>>>>>>>>> http://www.ovirt.org/Features/Gluster_Volume_Asynchronous_Tasks_Management
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The detailed design (including REST API design) is at
>>>>>>>>>> http://www.ovirt.org/Features/Detailed_Gluster_Volume_Asynchronous_Tasks_Management.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I would really appreciate if you could review and provide your
>>>>>>>>>> valuable
>>>>>>>>>> feedback.
>>>>>>>>> I Sahina
>>>>>>>>> Why not using 6the External Tasks feature introduced for 3.3 for
>>>>>>>>> those
>>>>>>>>> Gluster tasks ???
>>>>>>>>> http://www.ovirt.org/Features/Design/DetailedExternalTasks
>>>>>>>> Hi Eli,
>>>>>>>>
>>>>>>>> We still want to be able to start and stop these operations from the
>>>>>>>> engine.
>>>>>>>> So, when a user wants to say, rebalance a volume, they would go
>>>>>>>> select
>>>>>>>> the volume and click on Rebalance Start.
>>>>>>>> This would then call the BLL command to start rebalance which will
>>>>>>>> invoke the corresponding vdsm verb to start the rebalance on the
>>>>>>>> volume.
>>>>>>>> This is the same as existing flow for other commands. The only
>>>>>>>> difference is the vdsm verb will return the task id from gluster,
>>>>>>>> for
>>>>>>>> the rebalance operation that was started. And we will monitor the
>>>>>>>> progress of the task using the gluster task id (by calling a gluster
>>>>>>>> command)
>>>>>>>>
>>>>>>>> I'm not sure how ExternalTasks would fit in here? I was thinking of
>>>>>>>> using ExternalTask support for adding Job/Steps to engine when the
>>>>>>>> operation is started outside of engine, that is, from Gluster CLI.
>>>>>>>> Please correct me if I'm missing something.
>>>>>>> Does this mean that from Gluster CLI you will not try and invoke the
>>>>>>> rebalance command ?
>>>>>>> (I mean, I should either use Gluster CLI or Engine's REST API?)
>>>>>> Rebalance volume command could be invoked in any of the following
>>>>>> ways:
>>>>>> 1. From the console UI (clicking on Rebalance as shown in
>>>>>> http://www.ovirt.org/Features/Gluster_Volume_Asynchronous_Tasks_Management#Rebalance_Volume)
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2. Using REST API
>>>>>> 3. Outside of engine, from Gluster CLI - In such cases, the engine
>>>>>> should detect that a user has triggered rebalance operation outside
>>>>>> the
>>>>>> engine, and allow the user to monitor progress of this from the
>>>>>> engine.
>>>>>> This is where, we need support to add a Job for an operation that was
>>>>>> started externally, so that it can be seen in the Tasks tab.
>>>>>
>>>>> and still, it should be considered an internal task, since the engine
>>>>> is managing it / detected it.
>>>>>
>>>>
>>>> Itamar, yes, you are right. This would need to be treated as an internal
>>>> task as the engine needs to be able to stop it and also monitor it. We
>>>> would probably need a similar mechanism as external task injection, to
>>>> add a Job for the task started from gluster CLI.
>>>>
>>>>
>>>
>>> even if it was started from CLI, wouldn't it be better if engine
>>> detected it, and still treated it as an internal task, allowing to
>>> cancel it, etc.?
>>
>> Yes, but I need to add a Job for this internal task, so that it can be
>> monitored in the Tasks pane. Any idea if I can use any existing
>> framework to do it? I was thinking I would use
>> ExecutionHandler.createJob to do this (similar to what's done in
>> AddExternalJobCommand)
>
> Maybe in order to avoid code duplication we should re-factor having a base AddJobCommand and two descendants AddExternalJobCommand and AddInternalJobCommand when the only diff between those is the external flag value while the execute method of both calls super at first and the ancestor has the original code of ddExternalJobCommand in its execute method...

and in this case, the code will call AddInternalJobCommand for a job 
detected and not initiated by engine?