Re: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ]

Thanks to the reviewers, merged on master now. Working with Dafna on getting it fixed on 4.2 and understanding whether 4.1.10 is affected (probably the most important question as we've already built it and it should be shipped to customers). On Thu, Mar 15, 2018 at 10:11 AM, Tal Nisan <tnisan(a)redhat.com&gt; wrote: > I've reviewed and marked +1, I'll need another reviewer though for this > matter. > I've also based one of my patches on top of it and it passed OST: > http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi > rt-system-tests_manual/2387/ > > Dafna, prior to Eli's patch all OST jobs failed? > > > On Wed, Mar 14, 2018 at 6:46 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: > >> Eli updated the bug with a fix that reverts parts of the reported patch: >> https://gerrit.ovirt.org/#/c/89005/ >> >> waiting for verification and merge. >> >> Thanks! >> Dafna >> >> >> >> >> On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >> >>> >>> >>> On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < >>> michal.skrivanek(a)redhat.com&gt; wrote: >>> >>>> >>>> >>>> On 13 Mar 2018, at 22:24, Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> >>>> >>>> >>>> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >>>> michal.skrivanek(a)redhat.com&gt; wrote: >>>> >>>>> >>>>> >>>>> On 13 Mar 2018, at 09:27, Eyal Edri <eedri(a)redhat.com&gt; wrote: >>>>> >>>>> >>>>> >>>>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg <danken(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >>>>>> > We just had a failure in master 002_bootstrap.add_mac_pool with >>>>>> the same >>>>>> > error on edit cluster. >>>>>> >>>>> >>>>> Does it fail consistently? >>>>> >>>> >>>> yes. >>>> >>>> >>>> that’s good >>>> >>>> >>>> >>>> >>>>> Did you narrow down the commit(s) where it started to happen? >>>>> >>>> >>>> First change reported failed by CQ on this issue is this one: >>>> https://gerrit.ovirt.org/#/c/88738/2 >>>> - db: add func to turn table columns to empty string (this was >>>> reported by Daniel at the beginning of this thread) >>>> >>>> Was there any other update done at that time? >>>>> >>>>> there are always other changes submitted. but CQ tries to isolate the >>>> change that it believes is causing the failure by reducing the change it >>>> tests until it gets to one single change. >>>> >>>> >>>> I meant changes like major update of packages or any other >>>> configuration change >>>> >>> >>> :) the last one I saw was from Eli on Friday but I don't think its >>> related. since this is a big project and there a lot of changes submitted >>> daily, maybe someone more qualified them me can have a look and see if >>> anything catches their eyes? >>> >>>> >>>> >>>> We cannot have OST keep failing for a long time, especially on a big >>>> project like ovirt-engine. if we cannot have a fix on this quickly I think >>>> we should start skipping failed tests to allow changes to pass successfully >>>> until the bug is fixed. >>>> >>>> >>>> sure. But in this case you’re just going to hit the same problem in >>>> the next test. Please enable back the one you commented out, and try to >>>> revert that patch instead. There is a chance it changed the behavior >>>> because somehow the tests using Default cluster somehow rely on undefined >>>> values (not sure if that’s even intentional, but that’s the way it is >>>> written), and that patch may have changed it perhaps. Eli? >>>> >>>> cool. I think that Eyal has reverted my skip test so we can try to >>> revert the change reported. >>> >>> >>>> Thanks, >>>> michal >>>> >>>> >>>> Thanks, >>>> Dafna >>>> >>>> >>>> >>>>> Thanks, >>>>> michal >>>>> >>>>> > >>>>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>>>> > >>>>>> > either I skipped the wrong test or we have a bigger issue. >>>>>> >>>>>> We certainly do. As before, the error pops up on an attempt to update >>>>>> the cluster (this time it is changing only the mac pool of the >>>>>> cluster). CPU is not specified by the command, so it should not have >>>>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>>>> value. >>>>>> >>>>>> cluster_service.update( >>>>>> cluster=sdk4.types.Cluster( >>>>>> mac_pool=sdk4.types.MacPool( >>>>>> id=pool.id, >>>>>> ) >>>>>> ) >>>>>> ) >>>>>> >>>>>> 2018-03-12 13:58:56,263-04 WARN >>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>>>> 'UpdateCluster' failed for user admin@internal-authz. Reasons: >>>>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>>>> 2018-03-12 13:58:56,264-04 INFO >>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>>>> 2018-03-12 13:58:56,264-04 DEBUG >>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInt >>>>>> erceptor] >>>>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>>>> runAction, params: [UpdateCluster, >>>>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>>>> 2018-03-12 13:58:56,269-04 ERROR >>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>>>> chosen CPU is not supported.] >>>>>> >>>>> >>>>> So I guess we can't skip this test as well, and this issue has to be >>>>> fixed right? >>>>> >>>>> >>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> Devel(a)ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Eyal edri >>>>> >>>>> MANAGER >>>>> >>>>> RHV DevOps >>>>> >>>>> EMEA VIRTUALIZATION R&D >>>>> >>>>> >>>>> Red Hat EMEA <https://www.redhat.com/> >>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>>> <https://redhat.com/trusted> >>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel(a)ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>>> >>>>> >>>> >>>> >>> >> >