[ovirt-devel] [ OST Failure Report ] [ oVirtMaster ] [ 07-11-2017 ] [007_sd_reattach.deactivate_storage_domain ]

[Yaniv Kaul]

On Fri, Dec 8, 2017 at 9:31 PM, Dafna Ron <dron at redhat.com> wrote:

> I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813
>

I'm optimistically hoping https://gerrit.ovirt.org/#/c/85195/ will fix it.
Not sure.
Y.


>
> Allon, can you please assign someone to help fix this test?
> Please let me know if you need help from me.
>
> Thanks!
> Dafna
>
>
>
> On 12/07/2017 11:59 AM, Yaniv Kaul wrote:
>
>
>
> On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz at redhat.com>
> wrote:
>
>> I think that the maybe the QE can share their methods on how to avoid
>> those issues.
>> From what I remember, before deactivating storage domain they make sure
>> that there are no running tasks related to
>> the storage domain.
>>
>
> Looks like an easy fix is to wrap it with try, except sdk4.Error and let
> it sit within the testlib.assert_true_within_short() loop.
> Y.
>
>
>>
>> On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul at redhat.com> wrote:
>>
>>>
>>>
>>> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron at redhat.com> wrote:
>>>
>>>> Maor, I either need to get new glasses or a magnifier glass to read
>>>> what you wrote :-P
>>>> when you say running tasks - these are actually running tasks that may
>>>> be running because of other tests in ost - correct? wouldn't killing or
>>>> blocking those can cause other tests to fail?
>>>>
>>>
>>> It might well be the OVF update. How can we, from the API, wait for
>>> those tasks to complete? Or should we catch exception and retry?
>>> Y.
>>>
>>>
>>>>
>>>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>>>>
>>>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
>>>> that we might have tasks which will start running internally using
>>>> scheduling (like OVF_UPDATE) and we can't really know how much time every
>>>> task will take until it will end.
>>>>
>>>> Even if we check that there are no running tasks it will not guarantee
>>>> that no task will start until you deactivate the storage domain.
>>>>
>>>> I think that the best solution for it is an engine support to cancel
>>>> running tasks or block tasks from running.
>>>>
>>>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron at redhat.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We had a failure on master basic suite for test
>>>>> 007_sd_reattach.deactivate_storage_domain.
>>>>>
>>>>> The failure was that we failed to deactivate domain due to running
>>>>> tasks.
>>>>>
>>>>> It does not seem to be related to the patch it was testing and I think
>>>>> that the test itself needs to be modified to check there are no running
>>>>> tasks.
>>>>>
>>>>> Is there perhaps a way to query if there are running tasks before
>>>>> running the command? can you please take a look at the test on OST?
>>>>>
>>>>> *Link and headline of suspected patches: Not related to error*
>>>>>
>>>>> *Link to Job:
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>>>>
>>>>>
>>>>> * Link to all logs:
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/>
>>>>> (Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05
>>>>> WARN
>>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
>>>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
>>>>> admin at internal-authz. Reasons:
>>>>> VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>>>>> 2017-12-06 20:13:23,167-05 INFO
>>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
>>>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>>>>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>>>>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
>>>>> 20:13:23,172-05 DEBUG
>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction,
>>>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>>>> rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>>>>> user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
>>>>> 20:13:23,176-05 ERROR
>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are
>>>>> running tasks on this Storage. -Please wait until tasks will finish and try
>>>>> again.] *<error>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Devel mailing list
>>>>> Devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>>
>> --
>> Regards,
>> Eyal Shenitzky
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20171208/78080473/attachment-0001.html>