[ovirt-devel] [ OST Failure Report ] [ oVirtMaster ] [ 07-11-2017 ] [007_sd_reattach.deactivate_storage_domain ]

Fri Dec 8 21:20:03 UTC 2017

On Fri, Dec 8, 2017 at 10:39 PM, Yaniv Kaul <ykaul at redhat.com> wrote:

>
>
> On Fri, Dec 8, 2017 at 9:31 PM, Dafna Ron <dron at redhat.com> wrote:
>
>> I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813
>>
>
> I'm optimistically hoping https://gerrit.ovirt.org/#/c/85195/ will fix it.
> Not sure.
>

Keeps failing with:
Operation Failed: [Cannot deactivate Storage. The relevant Storage Domain's
status is Maintenance.]

Which is strange:
1. I do check if the SD is in Maint. mode before trying to deactive and the
test is supposed to be skipped if it is.
2. Why can't I deactive a SD when in Maint. mode?

Probably missing something here.
Y.

Y.
>
>
>>
>> Allon, can you please assign someone to help fix this test?
>> Please let me know if you need help from me.
>>
>> Thanks!
>> Dafna
>>
>>
>>
>> On 12/07/2017 11:59 AM, Yaniv Kaul wrote:
>>
>>
>>
>> On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz at redhat.com>
>> wrote:
>>
>>> I think that the maybe the QE can share their methods on how to avoid
>>> those issues.
>>> From what I remember, before deactivating storage domain they make sure
>>> that there are no running tasks related to
>>> the storage domain.
>>>
>>
>> Looks like an easy fix is to wrap it with try, except sdk4.Error and let
>> it sit within the testlib.assert_true_within_short() loop.
>> Y.
>>
>>
>>>
>>> On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul at redhat.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron at redhat.com> wrote:
>>>>
>>>>> Maor, I either need to get new glasses or a magnifier glass to read
>>>>> what you wrote :-P
>>>>> when you say running tasks - these are actually running tasks that may
>>>>> be running because of other tests in ost - correct? wouldn't killing or
>>>>> blocking those can cause other tests to fail?
>>>>>
>>>>
>>>> It might well be the OVF update. How can we, from the API, wait for
>>>> those tasks to complete? Or should we catch exception and retry?
>>>> Y.
>>>>
>>>>
>>>>>
>>>>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>>>>>
>>>>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
>>>>> that we might have tasks which will start running internally using
>>>>> scheduling (like OVF_UPDATE) and we can't really know how much time every
>>>>> task will take until it will end.
>>>>>
>>>>> Even if we check that there are no running tasks it will not guarantee
>>>>> that no task will start until you deactivate the storage domain.
>>>>>
>>>>> I think that the best solution for it is an engine support to cancel
>>>>> running tasks or block tasks from running.
>>>>>
>>>>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron at redhat.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We had a failure on master basic suite for test
>>>>>> 007_sd_reattach.deactivate_storage_domain.
>>>>>>
>>>>>> The failure was that we failed to deactivate domain due to running
>>>>>> tasks.
>>>>>>
>>>>>> It does not seem to be related to the patch it was testing and I
>>>>>> think that the test itself needs to be modified to check there are no
>>>>>> running tasks.
>>>>>>
>>>>>> Is there perhaps a way to query if there are running tasks before
>>>>>> running the command? can you please take a look at the test on OST?
>>>>>>
>>>>>> *Link and headline of suspected patches: Not related to error*
>>>>>>
>>>>>> *Link to Job:
>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>>>>>
>>>>>>
>>>>>> * Link to all logs:
>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/>
>>>>>> (Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05
>>>>>> WARN
>>>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
>>>>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
>>>>>> admin at internal-authz. Reasons:
>>>>>> VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>>>>>> 2017-12-06 20:13:23,167-05 INFO
>>>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
>>>>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>>>>>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>>>>>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
>>>>>> 20:13:23,172-05 DEBUG
>>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction,
>>>>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>>>>> rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>>>>>> user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
>>>>>> 20:13:23,176-05 ERROR
>>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>>>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are
>>>>>> running tasks on this Storage. -Please wait until tasks will finish and try
>>>>>> again.] *<error>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Devel mailing list
>>>>>> Devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Devel mailing list
>>>>> Devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Eyal Shenitzky
>>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20171208/bfb3300d/attachment-0001.html>