[Engine-devel] Serial Execution of Async Tasks

Itamar Heim iheim at redhat.com
Wed Aug 22 13:29:03 UTC 2012 

On 08/22/2012 12:21 PM, Allon Mureinik wrote:
>
>
> ----- Original Message -----
>> From: "Maor Lipchuk" <mlipchuk at redhat.com>
>> To: "Itamar Heim" <iheim at redhat.com>
>> Cc: "Allon Mureinik" <amureini at redhat.com>, "engine-devel" <engine-devel at ovirt.org>, "Eduardo Warszawski"
>> <ewarszaw at redhat.com>, "Yeela Kaplan" <ykaplan at redhat.com>, "Federico Simoncelli" <fsimonce at redhat.com>, "Liron
>> Aravot" <laravot at redhat.com>
>> Sent: Thursday, August 16, 2012 8:27:33 PM
>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
>>
>> On 08/16/2012 06:51 PM, Itamar Heim wrote:
>>> On 08/16/2012 03:21 PM, Maor Lipchuk wrote:
>>>> On 08/14/2012 05:23 PM, Itamar Heim wrote:
>>>>> On 08/14/2012 02:35 PM, Maor Lipchuk wrote:
>>>>>> How should we handle the auditLogMessages?
>>>>>> Basically when a command ends it print an audit log.
>>>>>>
>>>>>> When we will start to use multiple tasks I assume user might get
>>>>>> a bulk
>>>>>> of audit logs which are actually related to the same action
>>>>>> (when we
>>>>>> fail for example the process will be create and delete).
>>>>>> It might be a bit confusing for the user not to know which
>>>>>> action is
>>>>>> related to the operation
>>>>>
>>>>> I thought audit log gets written regardless of the transaction,
>>>>> so audit
>>>>> log appears "as they happen"?
>>>> That is correct,
>>>> The issue that I was referring to, is that now, with multiple
>>>> tasks
>>>> execution, we will get many audit logs which related to the same
>>>> transaction but each one will be printed at a different time.
>>>>
>>>> I think that it might be confusing for the user to relate each
>>>> audit log
>>>> to the operation that was started.
>>>>
>>>>
>>>> For example :
>>>> User run an action that executes some tasks of create volumes,
>>>> then the engine encounter a problem, and decide to rollback the
>>>> operation and delete the volumes, in that case the engine will
>>>> execute a
>>>> delete task for the volumes, so user might see that delete of the
>>>> volume
>>>> (for example a snapshot) was initiated.
>>>> Since those are asynchronous tasks, audit log will be printed in a
>>>> different period of time and a user might not be aware what is the
>>>> relation of those specific delete.
>>>
>>> async doesn't mean we don't print an audit log when we start it,
>>> and
>>> when we end it.
>>> so user would get the starting audit log when the task failed in
>>> your
>>> example. of course this may happen 2 hours after they started the
>>> task.
>>> as long as we can correlate the audit log to be part of the same
>>> "job",
>>> i don't see the issue.
>> yes, but if I understood correctly, we don't want to correlate the
>> multiple tasks with the execution handler (which AFAIK handle the
>> correlation id).
> I actually didn't mention this, but I don't see why not.
> What's I'd probably like to have is a log with "Correlation ID xyzabc, step #3 starting/executing/ending"
> Does this make any sense?

I think it is important to decide if correlation ID is also a "job id", 
or since correlation id can be controlled by user, for multiple steps 
over several commands (and maybe allowing clients to do multiple actions 
as a single transaction) we want a more formal "job id".

>>
>> I assume this issue can be addressed in a future phase,
>> but maybe it is an issue that might worth to think about.
>>>
>>>>>
>>>>>>
>>>>>> Maybe we will need to use the correlation id of the Execution
>>>>>> handler as
>>>>>> Eli suggested or maybe add new states at CommandActionState?
>>>>>>
>>>>>> On 08/14/2012 02:10 PM, Allon Mureinik wrote:
>>>>>>> Hi guys,
>>>>>>>
>>>>>>> Thanks for all your comments!
>>>>>>> The correct response for many these points is to update the
>>>>>>> wiki.
>>>>>>> I'm enclosing here the quick-and-dirty replies just to keep
>>>>>>> this
>>>>>>> thread alive, and will update the wiki shortly.
>>>>>>>
>>>>>>> See inline.
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>>>>> To: "Allon Mureinik" <amureini at redhat.com>
>>>>>>>> Cc: "Eli Mesika" <emesika at redhat.com>, "Liron Aravot"
>>>>>>>> <laravot at redhat.com>, "Federico Simoncelli"
>>>>>>>> <fsimonce at redhat.com>, "engine-devel"
>>>>>>>> <engine-devel at ovirt.org>,
>>>>>>>> "Eduardo Warszawski" <ewarszaw at redhat.com>, "Yeela
>>>>>>>> Kaplan" <ykaplan at redhat.com>
>>>>>>>> Sent: Sunday, August 12, 2012 9:39:23 AM
>>>>>>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
>>>>>>>>
>>>>>>>> On 10/08/12 03:40, Eli Mesika wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>>> From: "Allon Mureinik" <amureini at redhat.com>
>>>>>>>>>> To: "engine-devel" <engine-devel at ovirt.org>
>>>>>>>>>> Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Yeela
>>>>>>>>>> Kaplan"
>>>>>>>>>> <ykaplan at redhat.com>, "Federico Simoncelli"
>>>>>>>>>> <fsimonce at redhat.com>, "Liron Aravot" <laravot at redhat.com>
>>>>>>>>>> Sent: Thursday, August 9, 2012 6:41:09 PM
>>>>>>>>>> Subject: [Engine-devel] Serial Execution of Async Tasks
>>>>>>>>>>
>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> As you may know the engine currently has the ability to fire
>>>>>>>>>> an
>>>>>>>>>> SPM
>>>>>>>>>> task, and be asynchronously be "woken-up" when it ends.
>>>>>>>>>> This is great, but we found the for the Live Storage
>>>>>>>>>> Migration
>>>>>>>>>> feature we need something a bit complex - the ability to
>>>>>>>>>> have a
>>>>>>>>>> series of async tasks in a single control flow.
>>>>>>>>>>
>>>>>>>>>> Here's my initial design for this, your comments and
>>>>>>>>>> criticism
>>>>>>>>>> would
>>>>>>>>>> be welcome:
>>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Apart from the short explanation & flow , since this is a
>>>>>>>>> detailed
>>>>>>>>> design , I would add
>>>>>>>>> 1) Class diagram
>>>>>>>>> 2) Flow diagram
>>>>>>> Good idea, I'll see if I can jimmy something up.
>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> +1, it would help understanding the flow.
>>>>>>>>
>>>>>>>> - It looks like you chose not re-use/extend the
>>>>>>>> ExecutionHandler (the
>>>>>>>> entity used for building the tasks view exposed to the users).
>>>>>>>> It might be a good idea to keep the separation between the
>>>>>>>> engine
>>>>>>>> Jobs
>>>>>>>> and the underlying vdsm tasks, but I want to make sure you are
>>>>>>>> familiar
>>>>>>>> with this mechanism and ruled it out with a reason. If this is
>>>>>>>> the
>>>>>>>> case
>>>>>>>> please share why you decided not to use it.
>>>>>>> As you said Jobs and Steps are pure engine entities - they can
>>>>>>> contain no VDSM tasks, one VDSM task, or plausibly, in the
>>>>>>> future,
>>>>>>> several tasks.
>>>>>>> Even /today/, AsyncTasks and Jobs/Steps are two different kinds
>>>>>>> of
>>>>>>> animals - I don't see any added value in mixing them together.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> - how does this design survives a jboss restart? Can you
>>>>>>>> please a
>>>>>>>> section in the wiki to explain that.
>>>>>>> Basically, the way as a Command does today - the task is saved
>>>>>>> with
>>>>>>> the executionIndex, and continues when the command is woken up.
>>>>>>> I'll clarify this point in the wiki.
>>>>>>>
>>>>>>>>
>>>>>>>> -successful execution -
>>>>>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" - when?
>>>>>>> This is the new suggested format of executeCommand(). I'll
>>>>>>> clarify
>>>>>>> this too.
>>>>>>>
>>>>>>>> * If the second task is an HSM command (vs. SPM command), I
>>>>>>>> think you
>>>>>>>> should explain in the design how to handle such flows as well.
>>>>>>> HSM commands do not create AsyncTasks, as they do today - I
>>>>>>> will
>>>>>>> clarify this.
>>>>>>>
>>>>>>>> * Why do we need before task? can you give a concrete example
>>>>>>>> of what
>>>>>>>> would you do in such a method.
>>>>>>> Basically, /today/, command look like this:
>>>>>>> executeCommand() {
>>>>>>>      doStuffInTheDB();
>>>>>>>      runVdsCommand(someCommand);
>>>>>>> }
>>>>>>>
>>>>>>> endSuccessfully() {
>>>>>>>      doMoreStuffInTheDB();
>>>>>>> }
>>>>>>>
>>>>>>> endWithFailure() {
>>>>>>>      doMoreStuffForFailureInTheDB();
>>>>>>> }
>>>>>>>
>>>>>>> In the new design, the entire doStuffInTheDB() should be moved
>>>>>>> to a
>>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
>>>>>>>
>>>>>>>>
>>>>>>>> - I see you added SPMAsyncTaskHandler, any reason not to use
>>>>>>>> SPMAsyncTasK to manage it own life-cycle?
>>>>>>> Conserving today's design - The SPMAsyncTaskHandler is the
>>>>>>> place to
>>>>>>> add additional, non-SPM, logic around the SPM task execution,
>>>>>>> like
>>>>>>> CommandBase allows today.
>>>>>>>
>>>>>>>>
>>>>>>>> - In the life-cycle managed by the SPMAsyncTaskHandler there
>>>>>>>> is a
>>>>>>>> step
>>>>>>>> 'createTask - how to create the async task' can you please
>>>>>>>> elaborate
>>>>>>>> what are the options.
>>>>>>> new [any type of async task]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Livnat
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -Allon
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Engine-devel mailing list
>>>>>>>>>> Engine-devel at ovirt.org
>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Engine-devel mailing list
>>>>>>>>> Engine-devel at ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Engine-devel mailing list
>>>>>>> Engine-devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>

More information about the Engine-devel mailing list