Re: [ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

Thanks Shirly and Martin. I see the patch was merged so monitoring and will update On Wed, Jun 12, 2019 at 4:09 PM Martin Perina <mperina(a)redhat.com&gt; wrote: > > > On Wed, Jun 12, 2019 at 11:53 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > >> latest tests are still failing on log collector >> > > I had a typo in my patch, here is the fix: > https://gerrit.ovirt.org/#/c/100784/ > Currently trying to verify using manual OST to make sure latest VDSM is > used (and not latest tested as in check-patch executed from the fix) ... > > >> On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron <dron(a)redhat.com&gt; wrote: >> >>> thanks Martin. >>> we are failing on missing packages so I have a patch to fix it. >>> I will update once we have a vdsm build >>> >>> On Tue, Jun 11, 2019 at 10:08 PM Martin Perina <mperina(a)redhat.com&gt; >>> wrote: >>> >>>> >>>> >>>> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina <mperina(a)redhat.com&gt; >>>> wrote: >>>> >>>>> >>>>> >>>>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>>> >>>>>> Galit's patch should have solved it. >>>>>> Marcin, are you still failing? >>>>>> >>>>> >>>>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's >>>>> change so we should know within an hour or so ... >>>>> >>>> >>>> CI finished successfully, please review so we can merge ... >>>> >>>> >>>>> >>>>>> >>>>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri <eedri(a)redhat.com&gt; wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk <msobczyk(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> basic suite fails for the patch with the following log: >>>>>>>> >>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... - STDERR >>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... + yum -y install ovirt-host >>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync) >>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Requires: bind-license = 32:9.9.4-74.el7_6.1 >>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Installed: 32:bind-license-9.9.4-73.el7_6.noarch (installed) >>>>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... bind-license = 32:9.9.4-73.el7_6 >>>>>>>> >>>>>>>> >>>>>>> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2 >>>>>>> hours ago should address this issue. >>>>>>> Maybe other reposync files should be updated as well? >>>>>>> >>>>>>> >>>>>>>> Seems unrelated and I'm having the same issue locally on my >>>>>>>> server. >>>>>>>> Dafna, do you have some insight regarding this dependency error? >>>>>>>> >>>>>>>> Thanks, Marcin >>>>>>>> On 6/11/19 1:15 PM, Martin Perina wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 11, 2019 at 11:58 AM Milan Zamazal < >>>>>>>> mzamazal(a)redhat.com&gt; wrote: >>>>>>>> >>>>>>>>> Dafna Ron <dron(a)redhat.com&gt; writes: >>>>>>>>> >>>>>>>>> > Hi, >>>>>>>>> > >>>>>>>>> > Please note vdsm has been broken since Fri the 7th >>>>>>>>> > >>>>>>>>> > to summarize again, vdsm has a patch to remove sos plugin >>>>>>>>> which is what >>>>>>>>> > metrics is using in its ost tests >>>>>>>>> > due to that, vdsm is failing the metrics tests and in order to >>>>>>>>> solve it we >>>>>>>>> > need to make a choice: >>>>>>>>> > 1. fix the metrics tests to not use sos >>>>>>>>> > 2. disable the metrics tests >>>>>>>>> > 3. revert the sos patch until a decision is made on ^^ >>>>>>>>> >>>>>>>>> #3 is not an option, it would make Vdsm uninstallable on newer >>>>>>>>> RHEL >>>>>>>>> versions. >>>>>>>>> >>>>>>>> >>>>>>>> I've posted a patch https://gerrit.ovirt.org/100716 which is >>>>>>>> trying to install vdsm sos plugin if it's not installed either by vdsm nor >>>>>>>> sos. >>>>>>>> Currenlty waiting for CI, if run is successfull, I will extend the >>>>>>>> patch also for 4.3 basic suite. >>>>>>>> >>>>>>>> >>>>>>>>> > Thanks, >>>>>>>>> > Dafna >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > ---------- Forwarded message --------- >>>>>>>>> > From: Dafna Ron <dron(a)redhat.com&gt; >>>>>>>>> > Date: Mon, Jun 10, 2019 at 1:30 PM >>>>>>>>> > Subject: Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ >>>>>>>>> 07-06-2019 ] >>>>>>>>> > [ 003_00_metrics_bootstrap.metrics_and_log_collector ] >>>>>>>>> > To: Martin Perina <mperina(a)redhat.com&gt;, Milan Zamazal < >>>>>>>>> mzamazal(a)redhat.com&gt;, >>>>>>>>> > Shirly Radco <sradco(a)redhat.com&gt; >>>>>>>>> > Cc: devel <devel(a)ovirt.org&gt;, infra <infra(a)ovirt.org&gt; >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > Shirly? any update on this? >>>>>>>>> > >>>>>>>>> > On Fri, Jun 7, 2019 at 11:54 AM Dafna Ron <dron(a)redhat.com&gt; >>>>>>>>> wrote: >>>>>>>>> > >>>>>>>>> >> Hi, >>>>>>>>> >> >>>>>>>>> >> We have a failure in vdsm project on master. >>>>>>>>> >> >>>>>>>>> >> The issue is change: >>>>>>>>> >> https://gerrit.ovirt.org/#/c/100576/ - Remove SOS VDSM plugin >>>>>>>>> >> >>>>>>>>> >> which is failing on metrics as metrics is calling >>>>>>>>> sos-logcollector. >>>>>>>>> >> >>>>>>>>> >> The patch cannot be changed as until centos 7.7 when >>>>>>>>> sos-3.7-3, which >>>>>>>>> >> contains vdsm plugin will come out. >>>>>>>>> >> so until then, we are left with no sos plugin, which is >>>>>>>>> causing the >>>>>>>>> >> metrics test to fail. >>>>>>>>> >> >>>>>>>>> >> Shirly, can you please take a look and see if we can change >>>>>>>>> the test to >>>>>>>>> >> not call sos-logcollector? >>>>>>>>> >> Please note, that we are expecting 4.3 to fail on same issue >>>>>>>>> very soon. >>>>>>>>> >> >>>>>>>>> >> failed job can be found here: >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14452/ >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> ERROR from test: >>>>>>>>> >> >>>>>>>>> >> lago.ssh: DEBUG: Command 8626fe70 on >>>>>>>>> lago-basic-suite-master-engine errors: >>>>>>>>> >> ERROR: Failed to get a sosreport from: >>>>>>>>> lago-basic-suite-master-host-1; Could not parse sosreport output >>>>>>>>> >> ERROR: Failed to get a sosreport from: >>>>>>>>> lago-basic-suite-master-host-0; Could not parse sosreport output >>>>>>>>> >> >>>>>>>>> >> lago.utils: DEBUG: Error while running thread Thread-3 >>>>>>>>> >> Traceback (most recent call last): >>>>>>>>> >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line >>>>>>>>> 58, in _ret_via_queue >>>>>>>>> >> queue.put({'return': func()}) >>>>>>>>> >> File >>>>>>>>> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/003_00_metrics_bootstrap.py", >>>>>>>>> line 97, in run_log_collector >>>>>>>>> >> 'log collector failed. Exit code is %s' % result.code >>>>>>>>> >> File "/usr/lib64/python2.7/unittest/case.py", line 462, in >>>>>>>>> assertTrue >>>>>>>>> >> raise self.failureException(msg) >>>>>>>>> >> AssertionError: log collector failed. Exit code is 1 >>>>>>>>> >> --------------------- >> end captured logging << ---------- >>>>>>>>> >> >>>>>>>>> >> Thanks, >>>>>>>>> >> Dafna >>>>>>>>> >> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Martin Perina >>>>>>>> Manager, Software Engineering >>>>>>>> Red Hat Czech s.r.o. >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>> List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/SQJDJHHQR52... >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>> oVirt Code of Conduct: >>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>> List Archives: >>>>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GW3E7ZIE3WI... >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Eyal edri >>>>>>> >>>>>>> >>>>>>> MANAGER >>>>>>> >>>>>>> CONTINUOUS PRODUCTIZATION >>>>>>> >>>>>>> SYSTEM ENGINEERING >>>>>>> >>>>>>> Red Hat <https://www.redhat.com/> >>>>>>> <https://red.ht/sig> >>>>>>> phone: +972-9-7692018 >>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel) >>>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: >>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/3TLL7C4LF76... >>>>>> >>>>> >>>>> >>>>> -- >>>>> Martin Perina >>>>> Manager, Software Engineering >>>>> Red Hat Czech s.r.o. >>>>> >>>> >>>> >>>> -- >>>> Martin Perina >>>> Manager, Software Engineering >>>> Red Hat Czech s.r.o. >>>> >>> > > -- > Martin Perina > Manager, Software Engineering > Red Hat Czech s.r.o. >