[ovirt-devel] Re: ovirt-system-tests_compat-4.2-suite-master failing for 3 weeks?

On Mon, May 6, 2019 at 6:18 PM Galit Rosenthal <grosenth(a)redhat.com&gt; wrote: > > The issue is solved now for both compat 4.2 and for compat 4.1 > the suites passed successfully > Great! Can we have more details? what was the root cause? how it was fixed? On Thu, May 2, 2019 at 11:15 AM Shani Leviim <sleviim(a)redhat.com&gt; wrote: > >> It seems that the storage domain's format wasn't properly set by the >> cluster's level (although the cluster level is 4.1 or 4.2, the storage >> format was set as v5). >> Galit and I are working on it. >> >> >> *Regards,* >> >> *Shani Leviim* >> >> >> On Wed, May 1, 2019 at 5:34 PM Tal Nisan <tnisan(a)redhat.com&gt; wrote: >> >>> Shani, please have a look >>> >>> On Wed, May 1, 2019 at 2:34 PM Galit Rosenthal <grosenth(a)redhat.com&gt; >>> wrote: >>> >>>> Hi, >>>> >>>> There is a different issue now (jenkins job [1]): >>>> >>>> Fault reason is "Operation Failed". Fault detail is "[Cannot attach Storage. Storage Domain format V5 is illegal.]". HTTP response code is 409. >>>> >>>> >>>> <http://goog_2029942553> >>>> Can someone take a look? >>>> >>>> *Error from engine.log:* >>>> >>>> 2019-05-01 02:00:54,015-04 DEBUG [org.ovirt.engine.core.bll.Backend] (default task-3) [] Executing command AttachStorageDomainToPool for user admin@internal-authz. >>>> 2019-05-01 02:00:54,032-04 DEBUG [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Checking whether user 'caa94e16-6bd5-11e9-b8ca-5452c0a8ca02' or one of the groups he is member of, have the following permissions: ID: bb8f0934-0325-474d-9f62-f2983972a16b Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN, ID: b2c7ddfb-b9ba-4218-b3f1-4b952031de8f Type: StoragePoolAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >>>> 2019-05-01 02:00:54,032-04 DEBUG [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Found permission 'caa959ec-6bd5-11e9-b8cb-5452c0a8ca02' for user when running 'AttachStorageDomainToPool', on 'Storage' with id 'bb8f0934-0325-474d-9f62-f2983972a16b' >>>> 2019-05-01 02:00:54,033-04 DEBUG [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Found permission 'caa959ec-6bd5-11e9-b8cb-5452c0a8ca02' for user when running 'AttachStorageDomainToPool', on 'Data Center' with id 'b2c7ddfb-b9ba-4218-b3f1-4b952031de8f' >>>> 2019-05-01 02:00:54,033-04 DEBUG [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Before acquiring lock 'EngineLock:{exclusiveLocks='[bb8f0934-0325-474d-9f62-f2983972a16b=STORAGE]', sharedLocks=''}' >>>> 2019-05-01 02:00:54,033-04 DEBUG [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Success acquiring lock 'EngineLock:{exclusiveLocks='[bb8f0934-0325-474d-9f62-f2983972a16b=STORAGE]', sharedLocks=''}' >>>> 2019-05-01 02:00:54,033-04 INFO [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Lock Acquired to object 'EngineLock:{exclusiveLocks='[bb8f0934-0325-474d-9f62-f2983972a16b=STORAGE]', sharedLocks=''}' >>>> 2019-05-01 02:00:54,039-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] method: getAllForStoragePoolAndStatus, params: [b2c7ddfb-b9ba-4218-b3f1-4b952031de8f, Up], timeElapsed: 6ms >>>> 2019-05-01 02:00:54,040-04 WARN [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Validation of action 'AttachStorageDomainToPool' failed for user admin@internal-authz. Reasons: VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ATTACH,ACTION_TYPE_FAILED_STORAGE_DOMAIN_FORMAT_ILLEGAL,$storageFormat V5 >>>> 2019-05-01 02:00:54,041-04 DEBUG [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Before releasing a lock 'EngineLock:{exclusiveLocks='[bb8f0934-0325-474d-9f62-f2983972a16b=STORAGE]', sharedLocks=''}' >>>> 2019-05-01 02:00:54,041-04 DEBUG [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] The exclusive lock for key 'bb8f0934-0325-474d-9f62-f2983972a16bSTORAGE' is released and lock is removed from map >>>> 2019-05-01 02:00:54,041-04 INFO [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] Lock freed to object 'EngineLock:{exclusiveLocks='[bb8f0934-0325-474d-9f62-f2983972a16b=STORAGE]', sharedLocks=''}' >>>> 2019-05-01 02:00:54,045-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-3) [13e92697-ec26-478b-a29e-c652b6006e63] method: runAction, params: [AttachStorageDomainToPool, AttachStorageDomainToPoolParameters:{commandId='07b837fe-6d86-44f6-81b6-9ddcc4f0cc49', user='null', commandType='Unknown'}], timeElapsed: 30ms >>>> 2019-05-01 02:00:54,049-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-3) [] Operation Failed: [Cannot attach Storage. Storage Domain format V5 is illegal.] >>>> 2019-05-01 02:00:54,053-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-3) [] Entered SsoRestApiAuthFilter >>>> >>>> >>>> [1] >>>> http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/561/ >>>> <http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/5... >>>> >>>> Thanks, >>>> Galit >>>> >>>> >>>> On Mon, Apr 29, 2019 at 5:26 PM Fred Rolland <frolland(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Galit is investigating the problem. >>>>> The issue is that before adding the storage domain, we check that the >>>>> hosts are on UP state. >>>>> However, it is not always the case and the root cause is unclear. >>>>> >>>>> On Mon, Apr 29, 2019 at 3:16 PM Doron Fediuck <dfediuck(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> >>>>>> https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.2-suite-master/... >>>>>> >>>>>> The current failure is on adding a storage domain. Did anyone review >>>>>> it? >>>>>> _______________________________________________ >>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: >>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/QRMG6BISWXU... >>>>>> >>>>> _______________________________________________ >>>>> Devel mailing list -- devel(a)ovirt.org >>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YIGTLVYSA5V... >>>>> >>>> >>>> >>>> -- >>>> >>>> GALIT ROSENTHAL >>>> >>>> SOFTWARE ENGINEER >>>> >>>> Red Hat >>>> >>>> <https://www.redhat.com/> >>>> >>>> galit(a)redhat.com T: 972-9-7692230 >>>> <https://red.ht/sig> >>>> _______________________________________________ >>>> Devel mailing list -- devel(a)ovirt.org >>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/T7Y2WW4OBBS... >>>> >>> > > -- > > GALIT ROSENTHAL > > SOFTWARE ENGINEER > > Red Hat > > <https://www.redhat.com/> > > galit(a)redhat.com T: 972-9-7692230 > <https://red.ht/sig> > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CNEA3DP6TJN... >