Re: [ovirt-devel] [ OST Failure Report ] [ oVirtMaster ] [ 07-11-2017 ] [007_sd_reattach.deactivate_storage_domain ]

I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813

Allon, can you please assign someone to help fix this test? Please let me know if you need help from me. Thanks! Dafna On 12/07/2017 11:59 AM, Yaniv Kaul wrote: On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz(a)redhat.com&gt; wrote: > I think that the maybe the QE can share their methods on how to avoid > those issues. > From what I remember, before deactivating storage domain they make sure > that there are no running tasks related to > the storage domain. > Looks like an easy fix is to wrap it with try, except sdk4.Error and let it sit within the testlib.assert_true_within_short() loop. Y. > > On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com&gt; wrote: > >> >> >> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >> >>> Maor, I either need to get new glasses or a magnifier glass to read >>> what you wrote :-P >>> when you say running tasks - these are actually running tasks that may >>> be running because of other tests in ost - correct? wouldn't killing or >>> blocking those can cause other tests to fail? >>> >> >> It might well be the OVF update. How can we, from the API, wait for >> those tasks to complete? Or should we catch exception and retry? >> Y. >> >> >>> >>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote: >>> >>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is >>> that we might have tasks which will start running internally using >>> scheduling (like OVF_UPDATE) and we can't really know how much time every >>> task will take until it will end. >>> >>> Even if we check that there are no running tasks it will not guarantee >>> that no task will start until you deactivate the storage domain. >>> >>> I think that the best solution for it is an engine support to cancel >>> running tasks or block tasks from running. >>> >>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >>> >>>> Hi, >>>> >>>> We had a failure on master basic suite for test >>>> 007_sd_reattach.deactivate_storage_domain. >>>> >>>> The failure was that we failed to deactivate domain due to running >>>> tasks. >>>> >>>> It does not seem to be related to the patch it was testing and I think >>>> that the test itself needs to be modified to check there are no running >>>> tasks. >>>> >>>> Is there perhaps a way to query if there are running tasks before >>>> running the command? can you please take a look at the test on OST? >>>> >>>> *Link and headline of suspected patches: Not related to error* >>>> >>>> *Link to Job: >>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/ >>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>* >>>> >>>> >>>> * Link to all logs: >>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/ >>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa... >>>> (Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05 >>>> WARN >>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand] >>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of >>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user >>>> admin@internal-authz. Reasons: >>>> VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS >>>> 2017-12-06 20:13:23,167-05 INFO >>>> [org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand] >>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to >>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a >>>> a4-44fe-aa43-e96754a975ba=STORAGE]', >>>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06 >>>> 20:13:23,172-05 DEBUG >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction, >>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto >>>> rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3', >>>> user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06 >>>> 20:13:23,176-05 ERROR >>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are >>>> running tasks on this Storage. -Please wait until tasks will finish and try >>>> again.] *<error> >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>> >>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > Regards, > Eyal Shenitzky >