Re: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ]

And I've started http://jenkins.ovirt.org/job/ovirt-system-tests_manual/ 2392/console to see if it OST passes there. On Thu, Mar 15, 2018 at 11:26 AM, Tal Nisan <tnisan(a)redhat.com&gt; wrote: > I've backported to 4.2, kind reviewers from master please review as well > - https://gerrit.ovirt.org/#/c/89035/ > > > > On Thu, Mar 15, 2018 at 11:18 AM, Tal Nisan <tnisan(a)redhat.com&gt; wrote: > >> Thanks to the reviewers, merged on master now. >> Working with Dafna on getting it fixed on 4.2 and understanding whether >> 4.1.10 is affected (probably the most important question as we've already >> built it and it should be shipped to customers). >> >> >> On Thu, Mar 15, 2018 at 10:11 AM, Tal Nisan <tnisan(a)redhat.com&gt; wrote: >> >>> I've reviewed and marked +1, I'll need another reviewer though for this >>> matter. >>> I've also based one of my patches on top of it and it passed OST: >>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >>> rt-system-tests_manual/2387/ >>> >>> Dafna, prior to Eli's patch all OST jobs failed? >>> >>> >>> On Wed, Mar 14, 2018 at 6:46 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >>> >>>> Eli updated the bug with a fix that reverts parts of the reported >>>> patch: https://gerrit.ovirt.org/#/c/89005/ >>>> >>>> waiting for verification and merge. >>>> >>>> Thanks! >>>> Dafna >>>> >>>> >>>> >>>> >>>> On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> >>>>> >>>>> >>>>> On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < >>>>> michal.skrivanek(a)redhat.com&gt; wrote: >>>>> >>>>>> >>>>>> >>>>>> On 13 Mar 2018, at 22:24, Dafna Ron <dron(a)redhat.com&gt; wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >>>>>> michal.skrivanek(a)redhat.com&gt; wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On 13 Mar 2018, at 09:27, Eyal Edri <eedri(a)redhat.com&gt; wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg <danken(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> >>>>>>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron <dron(a)redhat.com&gt; >>>>>>>> wrote: >>>>>>>> > We just had a failure in master 002_bootstrap.add_mac_pool with >>>>>>>> the same >>>>>>>> > error on edit cluster. >>>>>>>> >>>>>>> >>>>>>> Does it fail consistently? >>>>>>> >>>>>> >>>>>> yes. >>>>>> >>>>>> >>>>>> that’s good >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Did you narrow down the commit(s) where it started to happen? >>>>>>> >>>>>> >>>>>> First change reported failed by CQ on this issue is this one: >>>>>> https://gerrit.ovirt.org/#/c/88738/2 >>>>>> - db: add func to turn table columns to empty string (this was >>>>>> reported by Daniel at the beginning of this thread) >>>>>> >>>>>> Was there any other update done at that time? >>>>>>> >>>>>>> there are always other changes submitted. but CQ tries to isolate >>>>>> the change that it believes is causing the failure by reducing the change >>>>>> it tests until it gets to one single change. >>>>>> >>>>>> >>>>>> I meant changes like major update of packages or any other >>>>>> configuration change >>>>>> >>>>> >>>>> :) the last one I saw was from Eli on Friday but I don't think its >>>>> related. since this is a big project and there a lot of changes submitted >>>>> daily, maybe someone more qualified them me can have a look and see if >>>>> anything catches their eyes? >>>>> >>>>>> >>>>>> >>>>>> We cannot have OST keep failing for a long time, especially on a big >>>>>> project like ovirt-engine. if we cannot have a fix on this quickly I think >>>>>> we should start skipping failed tests to allow changes to pass successfully >>>>>> until the bug is fixed. >>>>>> >>>>>> >>>>>> sure. But in this case you’re just going to hit the same problem in >>>>>> the next test. Please enable back the one you commented out, and try to >>>>>> revert that patch instead. There is a chance it changed the behavior >>>>>> because somehow the tests using Default cluster somehow rely on undefined >>>>>> values (not sure if that’s even intentional, but that’s the way it is >>>>>> written), and that patch may have changed it perhaps. Eli? >>>>>> >>>>>> cool. I think that Eyal has reverted my skip test so we can try to >>>>> revert the change reported. >>>>> >>>>> >>>>>> Thanks, >>>>>> michal >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Dafna >>>>>> >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> michal >>>>>>> >>>>>>> > >>>>>>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>>>>>> > >>>>>>>> > either I skipped the wrong test or we have a bigger issue. >>>>>>>> >>>>>>>> We certainly do. As before, the error pops up on an attempt to >>>>>>>> update >>>>>>>> the cluster (this time it is changing only the mac pool of the >>>>>>>> cluster). CPU is not specified by the command, so it should not >>>>>>>> have >>>>>>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>>>>>> value. >>>>>>>> >>>>>>>> cluster_service.update( >>>>>>>> cluster=sdk4.types.Cluster( >>>>>>>> mac_pool=sdk4.types.MacPool( >>>>>>>> id=pool.id, >>>>>>>> ) >>>>>>>> ) >>>>>>>> ) >>>>>>>> >>>>>>>> 2018-03-12 13:58:56,263-04 WARN >>>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>>>>>> 'UpdateCluster' failed for user admin@internal-authz. Reasons: >>>>>>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>>>>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>>>>>> 2018-03-12 13:58:56,264-04 INFO >>>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>>>>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>>>>>> 2018-03-12 13:58:56,264-04 DEBUG >>>>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInt >>>>>>>> erceptor] >>>>>>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>>>>>> runAction, params: [UpdateCluster, >>>>>>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>>>>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>>>>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>>>>>> 2018-03-12 13:58:56,269-04 ERROR >>>>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>>>>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>>>>>> chosen CPU is not supported.] >>>>>>>> >>>>>>> >>>>>>> So I guess we can't skip this test as well, and this issue has to >>>>>>> be fixed right? >>>>>>> >>>>>>> >>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list >>>>>>>> Devel(a)ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Eyal edri >>>>>>> >>>>>>> MANAGER >>>>>>> >>>>>>> RHV DevOps >>>>>>> >>>>>>> EMEA VIRTUALIZATION R&D >>>>>>> >>>>>>> >>>>>>> Red Hat EMEA <https://www.redhat.com/> >>>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED. >>>>>>> <https://redhat.com/trusted> >>>>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>>> _______________________________________________ >>>>>>> Devel mailing list >>>>>>> Devel(a)ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel >