Re: [ovirt-devel] [ OST Failure Report ] [ oVirtMaster ] [ 07-11-2017 ] [007_sd_reattach.deactivate_storage_domain ]

On Fri, Dec 8, 2017 at 9:31 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: > I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813 > I'm optimistically hoping https://gerrit.ovirt.org/#/c/85195/ will fix it. Not sure.

> > Allon, can you please assign someone to help fix this test? > Please let me know if you need help from me. > > Thanks! > Dafna > > > > On 12/07/2017 11:59 AM, Yaniv Kaul wrote: > > > > On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz(a)redhat.com&gt; > wrote: > >> I think that the maybe the QE can share their methods on how to avoid >> those issues. >> From what I remember, before deactivating storage domain they make sure >> that there are no running tasks related to >> the storage domain. >> > > Looks like an easy fix is to wrap it with try, except sdk4.Error and let > it sit within the testlib.assert_true_within_short() loop. > Y. > > >> >> On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: >> >>> >>> >>> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >>> >>>> Maor, I either need to get new glasses or a magnifier glass to read >>>> what you wrote :-P >>>> when you say running tasks - these are actually running tasks that may >>>> be running because of other tests in ost - correct? wouldn't killing or >>>> blocking those can cause other tests to fail? >>>> >>> >>> It might well be the OVF update. How can we, from the API, wait for >>> those tasks to complete? Or should we catch exception and retry? >>> Y. >>> >>> >>>> >>>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote: >>>> >>>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is >>>> that we might have tasks which will start running internally using >>>> scheduling (like OVF_UPDATE) and we can't really know how much time every >>>> task will take until it will end. >>>> >>>> Even if we check that there are no running tasks it will not guarantee >>>> that no task will start until you deactivate the storage domain. >>>> >>>> I think that the best solution for it is an engine support to cancel >>>> running tasks or block tasks from running. >>>> >>>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> >>>>> Hi, >>>>> >>>>> We had a failure on master basic suite for test >>>>> 007_sd_reattach.deactivate_storage_domain. >>>>> >>>>> The failure was that we failed to deactivate domain due to running >>>>> tasks. >>>>> >>>>> It does not seem to be related to the patch it was testing and I >>>>> think that the test itself needs to be modified to check there are no >>>>> running tasks. >>>>> >>>>> Is there perhaps a way to query if there are running tasks before >>>>> running the command? can you please take a look at the test on OST? >>>>> >>>>> *Link and headline of suspected patches: Not related to error* >>>>> >>>>> *Link to Job: >>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/ >>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>* >>>>> >>>>> >>>>> * Link to all logs: >>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/ >>>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa... >>>>> (Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05 >>>>> WARN >>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand] >>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of >>>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user >>>>> admin@internal-authz. Reasons: >>>>> VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS >>>>> 2017-12-06 20:13:23,167-05 INFO >>>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand] >>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to >>>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a >>>>> a4-44fe-aa43-e96754a975ba=STORAGE]', >>>>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06 >>>>> 20:13:23,172-05 DEBUG >>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction, >>>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto >>>>> rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3', >>>>> user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06 >>>>> 20:13:23,176-05 ERROR >>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >>>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are >>>>> running tasks on this Storage. -Please wait until tasks will finish and try >>>>> again.] *<error> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel(a)ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> >> -- >> Regards, >> Eyal Shenitzky >> > > >