[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

On Wed, Jun 12, 2019 at 11:53 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > latest tests are still failing on log collector > I had a typo in my patch, here is the fix: https://gerrit.ovirt.org/#/c/100784/ Currently trying to verify using manual OST to make sure latest VDSM is used (and not latest tested as in check-patch executed from the fix) ... > On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > >> thanks Martin. >> we are failing on missing packages so I have a patch to fix it. >> I will update once we have a vdsm build >> >> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina <mperina(a)redhat.com&gt; >> wrote: >> >>> >>> >>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina <mperina(a)redhat.com&gt; >>> wrote: >>> >>>> >>>> >>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> >>>>> Galit's patch should have solved it. >>>>> Marcin, are you still failing? >>>>> >>>> >>>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's >>>> change so we should know within an hour or so ... >>>> >>> >>> CI finished successfully, please review so we can merge ... >>> >>> >>>> >>>>> >>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri <eedri(a)redhat.com&gt; wrote: >>>>> >>>>>> >>>>>> >>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk <msobczyk(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> basic suite fails for the patch with the following log: >>>>>>> >>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... - STDERR >>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... + yum -y install ovirt-host >>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync) >>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Requires: bind-license = 32:9.9.4-74.el7_6.1 >>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Installed: 32:bind-license-9.9.4-73.el7_6.noarch (installed) >>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... bind-license = 32:9.9.4-73.el7_6 >>>>>>> >>>>>>> >>>>>> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2 >>>>>> hours ago should address this issue. >>>>>> Maybe other reposync files should be updated as well? >>>>>> >>>>>> >>>>>>> Seems unrelated and I'm having the same issue locally on my server. >>>>>>> Dafna, do you have some insight regarding this dependency error? >>>>>>> >>>>>>> Thanks, Marcin >>>>>>> On 6/11/19 1:15 PM, Martin Perina wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 11, 2019 at 11:58 AM Milan Zamazal <mzamazal(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> >>>>>>>> Dafna Ron <dron(a)redhat.com&gt; writes: >>>>>>>> >>>>>>>> > Hi, >>>>>>>> > >>>>>>>> > Please note vdsm has been broken since Fri the 7th >>>>>>>> > >>>>>>>> > to summarize again, vdsm has a patch to remove sos plugin which >>>>>>>> is what >>>>>>>> > metrics is using in its ost tests >>>>>>>> > due to that, vdsm is failing the metrics tests and in order to >>>>>>>> solve it we >>>>>>>> > need to make a choice: >>>>>>>> > 1. fix the metrics tests to not use sos >>>>>>>> > 2. disable the metrics tests >>>>>>>> > 3. revert the sos patch until a decision is made on ^^ >>>>>>>> >>>>>>>> #3 is not an option, it would make Vdsm uninstallable on newer RHEL >>>>>>>> versions. >>>>>>>> >>>>>>> >>>>>>> I've posted a patch https://gerrit.ovirt.org/100716 which is >>>>>>> trying to install vdsm sos plugin if it's not installed either by vdsm nor >>>>>>> sos. >>>>>>> Currenlty waiting for CI, if run is successfull, I will extend the >>>>>>> patch also for 4.3 basic suite. >>>>>>> >>>>>>> >>>>>>>> > Thanks, >>>>>>>> > Dafna >>>>>>>> > >>>>>>>> > >>>>>>>> > ---------- Forwarded message --------- >>>>>>>> > From: Dafna Ron <dron(a)redhat.com&gt; >>>>>>>> > Date: Mon, Jun 10, 2019 at 1:30 PM >>>>>>>> > Subject: Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ >>>>>>>> 07-06-2019 ] >>>>>>>> > [ 003_00_metrics_bootstrap.metrics_and_log_collector ] >>>>>>>> > To: Martin Perina <mperina(a)redhat.com&gt;, Milan Zamazal < >>>>>>>> mzamazal(a)redhat.com&gt;, >>>>>>>> > Shirly Radco <sradco(a)redhat.com&gt; >>>>>>>> > Cc: devel <devel(a)ovirt.org&gt;, infra <infra(a)ovirt.org&gt; >>>>>>>> > >>>>>>>> > >>>>>>>> > Shirly? any update on this? >>>>>>>> > >>>>>>>> > On Fri, Jun 7, 2019 at 11:54 AM Dafna Ron <dron(a)redhat.com&gt; >>>>>>>> wrote: >>>>>>>> > >>>>>>>> >> Hi, >>>>>>>> >> >>>>>>>> >> We have a failure in vdsm project on master. >>>>>>>> >> >>>>>>>> >> The issue is change: >>>>>>>> >> https://gerrit.ovirt.org/#/c/100576/ - Remove SOS VDSM plugin >>>>>>>> >> >>>>>>>> >> which is failing on metrics as metrics is calling >>>>>>>> sos-logcollector. >>>>>>>> >> >>>>>>>> >> The patch cannot be changed as until centos 7.7 when sos-3.7-3, >>>>>>>> which >>>>>>>> >> contains vdsm plugin will come out. >>>>>>>> >> so until then, we are left with no sos plugin, which is causing >>>>>>>> the >>>>>>>> >> metrics test to fail. >>>>>>>> >> >>>>>>>> >> Shirly, can you please take a look and see if we can change the >>>>>>>> test to >>>>>>>> >> not call sos-logcollector? >>>>>>>> >> Please note, that we are expecting 4.3 to fail on same issue >>>>>>>> very soon. >>>>>>>> >> >>>>>>>> >> failed job can be found here: >>>>>>>> >> >>>>>>>> >> >>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14452/ >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> ERROR from test: >>>>>>>> >> >>>>>>>> >> lago.ssh: DEBUG: Command 8626fe70 on >>>>>>>> lago-basic-suite-master-engine errors: >>>>>>>> >> ERROR: Failed to get a sosreport from: >>>>>>>> lago-basic-suite-master-host-1; Could not parse sosreport output >>>>>>>> >> ERROR: Failed to get a sosreport from: >>>>>>>> lago-basic-suite-master-host-0; Could not parse sosreport output >>>>>>>> >> >>>>>>>> >> lago.utils: DEBUG: Error while running thread Thread-3 >>>>>>>> >> Traceback (most recent call last): >>>>>>>> >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line >>>>>>>> 58, in _ret_via_queue >>>>>>>> >> queue.put({'return': func()}) >>>>>>>> >> File >>>>>>>> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/003_00_metrics_bootstrap.py", >>>>>>>> line 97, in run_log_collector >>>>>>>> >> 'log collector failed. Exit code is %s' % result.code >>>>>>>> >> File "/usr/lib64/python2.7/unittest/case.py", line 462, in >>>>>>>> assertTrue >>>>>>>> >> raise self.failureException(msg) >>>>>>>> >> AssertionError: log collector failed. Exit code is 1 >>>>>>>> >> --------------------- >> end captured logging << ---------- >>>>>>>> >> >>>>>>>> >> Thanks, >>>>>>>> >> Dafna >>>>>>>> >> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Martin Perina >>>>>>> Manager, Software Engineering >>>>>>> Red Hat Czech s.r.o. >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/SQJDJHHQR52... >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: >>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: >>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GW3E7ZIE3WI... >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Eyal edri >>>>>> >>>>>> >>>>>> MANAGER >>>>>> >>>>>> CONTINUOUS PRODUCTIZATION >>>>>> >>>>>> SYSTEM ENGINEERING >>>>>> >>>>>> Red Hat <https://www.redhat.com/> >>>>>> <https://red.ht/sig> >>>>>> phone: +972-9-7692018 >>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel) >>>>>> >>>>> _______________________________________________ >>>>> Devel mailing list -- devel(a)ovirt.org >>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/3TLL7C4LF76... >>>>> >>>> >>>> >>>> -- >>>> Martin Perina >>>> Manager, Software Engineering >>>> Red Hat Czech s.r.o. >>>> >>> >>> >>> -- >>> Martin Perina >>> Manager, Software Engineering >>> Red Hat Czech s.r.o. >>> >> -- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o.