[ovirt-devel] Re: FYI - Many nightly suites, including Network, HE and HC are failing since 2-3 days ago

the patch was merged on 4.3 but test run [1] has build [2] which is one patch before Ahmed's merge... [1] http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.3/21/ [2] ovirt-engine-4.3.2.2-0.0.master.20190324105929.git8b0969c.el7.noarch.rpm On Mon, Mar 25, 2019 at 8:42 AM Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > but http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.3/21/ > is still failing > Was your patch merged? > > On Sun, Mar 24, 2019 at 10:14 AM Ahmad Khiet <akhiet(a)redhat.com&gt; wrote: > >> patched 4.3! >> >> On Sun, Mar 24, 2019 at 9:06 AM Eitan Raviv <eraviv(a)redhat.com&gt; wrote: >> >>> After some off line discussions it seems that the change that should be >>> implemented in order to solve the original problem (remove host fails due >>> to disconnect storage in progress) is to leave the host in 'preparing for >>> maintenance' until all relevant storage operations are completed. >>> >>> On Sat, Mar 23, 2019 at 11:36 PM Dan Kenigsberg <danken(a)redhat.com&gt; >>> wrote: >>> >>>> Unfortunately, the network suite is still failing on >>>> >>>> Cannot edit Host. Related operation is currently in progress. Please >>>> try again later. >>>> >>>> Can you check if that's the same issue? Did you revert from 4.3 too? >>>> >>>> http://jenkins.ovirt.org/job/ovirt-system-tests_network-suite-4.3/19/ >>>> >>>> >>>> >>>> On Sat, 23 Mar 2019, 21:27 Benny Zlotnik, <bzlotnik(a)redhat.com&gt; wrote: >>>> >>>>> The patch was reverted on Thursday >>>>> >>>>> On Sat, Mar 23, 2019 at 8:48 PM Dan Kenigsberg <danken(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> I was told that intervening in the host state machine is delicate, but >>>>>> I think that this is the only correct approach. >>>>>> >>>>>> Benny, Ahmad, Tal: do you have a plan to resolve this? We are entering >>>>>> a third week with this constant failure. >>>>>> >>>>>> >>>>>> On Wed, Mar 20, 2019 at 2:42 PM Eitan Raviv <eraviv(a)redhat.com&gt; >>>>>> wrote: >>>>>> > >>>>>> > I am not sure that locking both groups would be sufficient, because >>>>>> there is still a chance that the removeNetowrks request will start and >>>>>> acquire the lock before the DisconnectStorage operation starts. >>>>>> > So probably the correct and full proof solution is to not move the >>>>>> host to maintenance until all related storage ops terminate. >>>>>> > >>>>>> > >>>>>> > On Wed, Mar 20, 2019 at 2:07 PM Simone Tiraboschi < >>>>>> stirabos(a)redhat.com&gt; wrote: >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> On Sun, Mar 17, 2019 at 3:04 PM Eyal Edri <eedri(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>> >>>>>> >>> Not sure if all the same issue, but seems to failing around the >>>>>> same time: >>>>>> >>> >>>>>> >>>   ovirt-system-tests_hc-basic-suite-4.2 1 day 12 hr - #824 12 >>>>>> hr - #825 57 min  integ-tests >>>>>> >>>   ovirt-system-tests_hc-basic-suite-master 2 days 12 hr - #1043 >>>>>> 12 hr - #1045 51 min  integ-tests >>>>>> >>>   ovirt-system-tests_he-basic-ansible-suite-4.3 N/A 11 hr - #11 >>>>>> 24 sec  integ-tests >>>>>> >>>   ovirt-system-tests_he-basic-ipv6-suite-master N/A 12 hr - #11 >>>>>> 10 min  integ-tests >>>>>> >>>   ovirt-system-tests_he-basic-iscsi-suite-master 2 days 10 hr - >>>>>> #871 10 hr - #873 1 hr 36 min  integ-tests >>>>>> >>>   ovirt-system-tests_he-basic-suite-master 2 days 11 hr - #1147 >>>>>> 11 hr - #1149 1 hr 26 min  integ-tests >>>>>> >>>   ovirt-system-tests_he-node-ng-suite-master 2 days 12 hr - >>>>>> #727 12 hr - #729 1 hr 54 min  integ-tests >>>>>> >> >>>>>> >> >>>>>> >> It's a NullPointerException on engine side: >>>>>> >> I opened a bug here: >>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1690159 which is not on >>>>>> POST >>>>>> >> >>>>>> >> >>>>>> >>> >>>>>> >>>   ovirt-system-tests_network-suite-master 3 days 12 hr - #926 >>>>>> 12 hr - #929 45 min  integ-tests >>>>>> >>>   ovirt-system-tests_openshift-on-ovirt-suite-4.2 3 days 10 hr >>>>>> - #187 10 hr - #190 45 min  integ-tests >>>>>> >>> >>>>>> >>> Links to jobs can be found here: https://jenkins.ovirt.org/ >>>>>> >>> >>>>>> >>> -- >>>>>> >>> >>>>>> >>> Eyal edri >>>>>> >>> >>>>>> >>> >>>>>> >>> MANAGER >>>>>> >>> >>>>>> >>> RHV/CNV DevOps >>>>>> >>> >>>>>> >>> EMEA VIRTUALIZATION R&D >>>>>> >>> >>>>>> >>> >>>>>> >>> Red Hat EMEA >>>>>> >>> >>>>>> >>> TRIED. TESTED. TRUSTED. >>>>>> >>> phone: +972-9-7692018 >>>>>> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>> >> >>>>>> >> _______________________________________________ >>>>>> >> Devel mailing list -- devel(a)ovirt.org >>>>>> >> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>> >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> >> oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> >> List Archives: >>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/CQP4YDTMT53... >>>>>> > >>>>>> > _______________________________________________ >>>>>> > Devel mailing list -- devel(a)ovirt.org >>>>>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> > oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> > List Archives: >>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/3XP6WDOAL54... >>>>>> >>>>>