Re: [Engine-devel] Serial Execution of Async Tasks

On 08/14/2012 05:23 PM, Itamar Heim wrote: > On 08/14/2012 02:35 PM, Maor Lipchuk wrote: >> How should we handle the auditLogMessages? >> Basically when a command ends it print an audit log. >> >> When we will start to use multiple tasks I assume user might get a bulk >> of audit logs which are actually related to the same action (when we >> fail for example the process will be create and delete). >> It might be a bit confusing for the user not to know which action is >> related to the operation > > I thought audit log gets written regardless of the transaction, so audit > log appears "as they happen"? That is correct, The issue that I was referring to, is that now, with multiple tasks execution, we will get many audit logs which related to the same transaction but each one will be printed at a different time. I think that it might be confusing for the user to relate each audit log to the operation that was started. For example : User run an action that executes some tasks of create volumes, then the engine encounter a problem, and decide to rollback the operation and delete the volumes, in that case the engine will execute a delete task for the volumes, so user might see that delete of the volume (for example a snapshot) was initiated. Since those are asynchronous tasks, audit log will be printed in a different period of time and a user might not be aware what is the relation of those specific delete.

> >> >> Maybe we will need to use the correlation id of the Execution handler as >> Eli suggested or maybe add new states at CommandActionState? >> >> On 08/14/2012 02:10 PM, Allon Mureinik wrote: >>> Hi guys, >>> >>> Thanks for all your comments! >>> The correct response for many these points is to update the wiki. >>> I'm enclosing here the quick-and-dirty replies just to keep this >>> thread alive, and will update the wiki shortly. >>> >>> See inline. >>> >>> ----- Original Message ----- >>>> From: "Livnat Peer" <lpeer(a)redhat.com&gt; >>>> To: "Allon Mureinik" <amureini(a)redhat.com&gt; >>>> Cc: "Eli Mesika" <emesika(a)redhat.com&gt;, "Liron Aravot" >>>> <laravot(a)redhat.com&gt;, "Federico Simoncelli" >>>> <fsimonce(a)redhat.com&gt;, "engine-devel" <engine-devel(a)ovirt.org&gt;, >>>> "Eduardo Warszawski" <ewarszaw(a)redhat.com&gt;, "Yeela >>>> Kaplan" <ykaplan(a)redhat.com&gt; >>>> Sent: Sunday, August 12, 2012 9:39:23 AM >>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks >>>> >>>> On 10/08/12 03:40, Eli Mesika wrote: >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: "Allon Mureinik" <amureini(a)redhat.com&gt; >>>>>> To: "engine-devel" <engine-devel(a)ovirt.org&gt; >>>>>> Cc: "Eduardo Warszawski" <ewarszaw(a)redhat.com&gt;, "Yeela Kaplan" >>>>>> <ykaplan(a)redhat.com&gt;, "Federico Simoncelli" >>>>>> <fsimonce(a)redhat.com&gt;, "Liron Aravot" <laravot(a)redhat.com&gt; >>>>>> Sent: Thursday, August 9, 2012 6:41:09 PM >>>>>> Subject: [Engine-devel] Serial Execution of Async Tasks >>>>>> >>>>>> Hi guys, >>>>>> >>>>>> As you may know the engine currently has the ability to fire an >>>>>> SPM >>>>>> task, and be asynchronously be "woken-up" when it ends. >>>>>> This is great, but we found the for the Live Storage Migration >>>>>> feature we need something a bit complex - the ability to have a >>>>>> series of async tasks in a single control flow. >>>>>> >>>>>> Here's my initial design for this, your comments and criticism >>>>>> would >>>>>> be welcome: >>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Task... >>>>>> >>>>> >>>>> Apart from the short explanation & flow , since this is a detailed >>>>> design , I would add >>>>> 1) Class diagram >>>>> 2) Flow diagram >>> Good idea, I'll see if I can jimmy something up. >>> >>>>> >>>> >>>> +1, it would help understanding the flow. >>>> >>>> - It looks like you chose not re-use/extend the ExecutionHandler (the >>>> entity used for building the tasks view exposed to the users). >>>> It might be a good idea to keep the separation between the engine >>>> Jobs >>>> and the underlying vdsm tasks, but I want to make sure you are >>>> familiar >>>> with this mechanism and ruled it out with a reason. If this is the >>>> case >>>> please share why you decided not to use it. >>> As you said Jobs and Steps are pure engine entities - they can >>> contain no VDSM tasks, one VDSM task, or plausibly, in the future, >>> several tasks. >>> Even /today/, AsyncTasks and Jobs/Steps are two different kinds of >>> animals - I don't see any added value in mixing them together. >>> >>>> >>>> >>>> - how does this design survives a jboss restart? Can you please a >>>> section in the wiki to explain that. >>> Basically, the way as a Command does today - the task is saved with >>> the executionIndex, and continues when the command is woken up. >>> I'll clarify this point in the wiki. >>> >>>> >>>> -successful execution - >>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" - when? >>> This is the new suggested format of executeCommand(). I'll clarify >>> this too. >>> >>>> * If the second task is an HSM command (vs. SPM command), I think you >>>> should explain in the design how to handle such flows as well. >>> HSM commands do not create AsyncTasks, as they do today - I will >>> clarify this. >>> >>>> * Why do we need before task? can you give a concrete example of what >>>> would you do in such a method. >>> Basically, /today/, command look like this: >>> executeCommand() { >>> doStuffInTheDB(); >>> runVdsCommand(someCommand); >>> } >>> >>> endSuccessfully() { >>> doMoreStuffInTheDB(); >>> } >>> >>> endWithFailure() { >>> doMoreStuffForFailureInTheDB(); >>> } >>> >>> In the new design, the entire doStuffInTheDB() should be moved to a >>> breforeTask of the (only) SPMAsyncTaskHandler. >>> >>>> >>>> - I see you added SPMAsyncTaskHandler, any reason not to use >>>> SPMAsyncTasK to manage it own life-cycle? >>> Conserving today's design - The SPMAsyncTaskHandler is the place to >>> add additional, non-SPM, logic around the SPM task execution, like >>> CommandBase allows today. >>> >>>> >>>> - In the life-cycle managed by the SPMAsyncTaskHandler there is a >>>> step >>>> 'createTask - how to create the async task' can you please elaborate >>>> what are the options. >>> new [any type of async task] >>>> >>>> >>>> >>>> >>>> >>>> Livnat >>>> >>>>>> >>>>>> >>>>>> -Allon >>>>>> _______________________________________________ >>>>>> Engine-devel mailing list >>>>>> Engine-devel(a)ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>>> >>>>> _______________________________________________ >>>>> Engine-devel mailing list >>>>> Engine-devel(a)ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>>>> >>>> >>>> >>> _______________________________________________ >>> Engine-devel mailing list >>> Engine-devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>> >> >> >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >> > >