[ovirt-devel] [ OST Failure Report ] [ oVirtMaster ] [ 07-11-2017 ] [007_sd_reattach.deactivate_storage_domain ]

[Yaniv Kaul]

On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz at redhat.com> wrote:

> I think that the maybe the QE can share their methods on how to avoid
> those issues.
> From what I remember, before deactivating storage domain they make sure
> that there are no running tasks related to
> the storage domain.
>

Looks like an easy fix is to wrap it with try, except sdk4.Error and let it
sit within the testlib.assert_true_within_short() loop.
Y.


>
> On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul at redhat.com> wrote:
>
>>
>>
>> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron at redhat.com> wrote:
>>
>>> Maor, I either need to get new glasses or a magnifier glass to read what
>>> you wrote :-P
>>> when you say running tasks - these are actually running tasks that may
>>> be running because of other tests in ost - correct? wouldn't killing or
>>> blocking those can cause other tests to fail?
>>>
>>
>> It might well be the OVF update. How can we, from the API, wait for those
>> tasks to complete? Or should we catch exception and retry?
>> Y.
>>
>>
>>>
>>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>>>
>>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
>>> that we might have tasks which will start running internally using
>>> scheduling (like OVF_UPDATE) and we can't really know how much time every
>>> task will take until it will end.
>>>
>>> Even if we check that there are no running tasks it will not guarantee
>>> that no task will start until you deactivate the storage domain.
>>>
>>> I think that the best solution for it is an engine support to cancel
>>> running tasks or block tasks from running.
>>>
>>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron at redhat.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We had a failure on master basic suite for test
>>>> 007_sd_reattach.deactivate_storage_domain.
>>>>
>>>> The failure was that we failed to deactivate domain due to running
>>>> tasks.
>>>>
>>>> It does not seem to be related to the patch it was testing and I think
>>>> that the test itself needs to be modified to check there are no running
>>>> tasks.
>>>>
>>>> Is there perhaps a way to query if there are running tasks before
>>>> running the command? can you please take a look at the test on OST?
>>>>
>>>> *Link and headline of suspected patches: Not related to error*
>>>>
>>>> *Link to Job:
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>>>
>>>>
>>>> * Link to all logs:
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/>
>>>> (Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05
>>>> WARN
>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
>>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
>>>> admin at internal-authz. Reasons:
>>>> VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>>>> 2017-12-06 20:13:23,167-05 INFO
>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
>>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>>>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>>>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
>>>> 20:13:23,172-05 DEBUG
>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction,
>>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>>> rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>>>> user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
>>>> 20:13:23,176-05 ERROR
>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are
>>>> running tasks on this Storage. -Please wait until tasks will finish and try
>>>> again.] *<error>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
>
> --
> Regards,
> Eyal Shenitzky
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20171207/443f3db6/attachment.html>