[Engine-devel] Serial Execution of Async Tasks

Thu Aug 16 12:21:41 UTC 2012

On 08/14/2012 05:23 PM, Itamar Heim wrote:
> On 08/14/2012 02:35 PM, Maor Lipchuk wrote:
>> How should we handle the auditLogMessages?
>> Basically when a command ends it print an audit log.
>>
>> When we will start to use multiple tasks I assume user might get a bulk
>> of audit logs which are actually related to the same action (when we
>> fail for example the process will be create and delete).
>> It might be a bit confusing for the user not to know which action is
>> related to the operation
> 
> I thought audit log gets written regardless of the transaction, so audit
> log appears "as they happen"?
That is correct,
The issue that I was referring to, is that now, with multiple tasks
execution, we will get many audit logs which related to the same
transaction but each one will be printed at a different time.

I think that it might be confusing for the user to relate each audit log
to the operation that was started.

For example :
User run an action that executes some tasks of create volumes,
then the engine encounter a problem, and decide to rollback the
operation and delete the volumes, in that case the engine will execute a
delete task for the volumes, so user might see that delete of the volume
(for example a snapshot) was initiated.
Since those are asynchronous tasks, audit log will be printed in a
different period of time and a user might not be aware what is the
relation of those specific delete.
> 
>>
>> Maybe we will need to use the correlation id of the Execution handler as
>> Eli suggested or maybe add new states at CommandActionState?
>>
>> On 08/14/2012 02:10 PM, Allon Mureinik wrote:
>>> Hi guys,
>>>
>>> Thanks for all your comments!
>>> The correct response for many these points is to update the wiki.
>>> I'm enclosing here the quick-and-dirty replies just to keep this
>>> thread alive, and will update the wiki shortly.
>>>
>>> See inline.
>>>
>>> ----- Original Message -----
>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>> To: "Allon Mureinik" <amureini at redhat.com>
>>>> Cc: "Eli Mesika" <emesika at redhat.com>, "Liron Aravot"
>>>> <laravot at redhat.com>, "Federico Simoncelli"
>>>> <fsimonce at redhat.com>, "engine-devel" <engine-devel at ovirt.org>,
>>>> "Eduardo Warszawski" <ewarszaw at redhat.com>, "Yeela
>>>> Kaplan" <ykaplan at redhat.com>
>>>> Sent: Sunday, August 12, 2012 9:39:23 AM
>>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
>>>>
>>>> On 10/08/12 03:40, Eli Mesika wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Allon Mureinik" <amureini at redhat.com>
>>>>>> To: "engine-devel" <engine-devel at ovirt.org>
>>>>>> Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Yeela Kaplan"
>>>>>> <ykaplan at redhat.com>, "Federico Simoncelli"
>>>>>> <fsimonce at redhat.com>, "Liron Aravot" <laravot at redhat.com>
>>>>>> Sent: Thursday, August 9, 2012 6:41:09 PM
>>>>>> Subject: [Engine-devel] Serial Execution of Async Tasks
>>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> As you may know the engine currently has the ability to fire an
>>>>>> SPM
>>>>>> task, and be asynchronously be "woken-up" when it ends.
>>>>>> This is great, but we found the for the Live Storage Migration
>>>>>> feature we need something a bit complex - the ability to have a
>>>>>> series of async tasks in a single control flow.
>>>>>>
>>>>>> Here's my initial design for this, your comments and criticism
>>>>>> would
>>>>>> be welcome:
>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
>>>>>>
>>>>>
>>>>> Apart from the short explanation & flow , since this is a detailed
>>>>> design , I would add
>>>>> 1) Class diagram
>>>>> 2) Flow diagram
>>> Good idea, I'll see if I can jimmy something up.
>>>
>>>>>
>>>>
>>>> +1, it would help understanding the flow.
>>>>
>>>> - It looks like you chose not re-use/extend the ExecutionHandler (the
>>>> entity used for building the tasks view exposed to the users).
>>>> It might be a good idea to keep the separation between the engine
>>>> Jobs
>>>> and the underlying vdsm tasks, but I want to make sure you are
>>>> familiar
>>>> with this mechanism and ruled it out with a reason. If this is the
>>>> case
>>>> please share why you decided not to use it.
>>> As you said Jobs and Steps are pure engine entities - they can
>>> contain no VDSM tasks, one VDSM task, or plausibly, in the future,
>>> several tasks.
>>> Even /today/, AsyncTasks and Jobs/Steps are two different kinds of
>>> animals - I don't see any added value in mixing them together.
>>>
>>>>
>>>>
>>>> - how does this design survives a jboss restart? Can you please a
>>>> section in the wiki to explain that.
>>> Basically, the way as a Command does today - the task is saved with
>>> the executionIndex, and continues when the command is woken up.
>>> I'll clarify this point in the wiki.
>>>
>>>>
>>>> -successful execution -
>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" - when?
>>> This is the new suggested format of executeCommand(). I'll clarify
>>> this too.
>>>
>>>> * If the second task is an HSM command (vs. SPM command), I think you
>>>> should explain in the design how to handle such flows as well.
>>> HSM commands do not create AsyncTasks, as they do today - I will
>>> clarify this.
>>>
>>>> * Why do we need before task? can you give a concrete example of what
>>>> would you do in such a method.
>>> Basically, /today/, command look like this:
>>> executeCommand() {
>>>    doStuffInTheDB();
>>>    runVdsCommand(someCommand);
>>> }
>>>
>>> endSuccessfully() {
>>>    doMoreStuffInTheDB();
>>> }
>>>
>>> endWithFailure() {
>>>    doMoreStuffForFailureInTheDB();
>>> }
>>>
>>> In the new design, the entire doStuffInTheDB() should be moved to a
>>> breforeTask of the (only) SPMAsyncTaskHandler.
>>>
>>>>
>>>> - I see you added SPMAsyncTaskHandler, any reason not to use
>>>> SPMAsyncTasK to manage it own life-cycle?
>>> Conserving today's design - The SPMAsyncTaskHandler is the place to
>>> add additional, non-SPM, logic around the SPM task execution, like
>>> CommandBase allows today.
>>>
>>>>
>>>> - In the life-cycle managed by the SPMAsyncTaskHandler there is a
>>>> step
>>>> 'createTask - how to create the async task' can you please elaborate
>>>> what are the options.
>>> new [any type of async task]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Livnat
>>>>
>>>>>>
>>>>>>
>>>>>> -Allon
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> 
> 