[ovirt-devel] Re: [urgent] vdsm broken since Friday (failing CQ)

latest tests are still failing on log collector On Wed, Jun 12, 2019 at 10:18 AM Dafna Ron <dron(a)redhat.com&gt; wrote: > thanks Martin. > we are failing on missing packages so I have a patch to fix it. > I will update once we have a vdsm build > > On Tue, Jun 11, 2019 at 10:08 PM Martin Perina <mperina(a)redhat.com&gt; > wrote: > >> >> >> On Tue, Jun 11, 2019 at 7:41 PM Martin Perina <mperina(a)redhat.com&gt; >> wrote: >> >>> >>> >>> On Tue, Jun 11, 2019 at 6:53 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>> >>>> Galit's patch should have solved it. >>>> Marcin, are you still failing? >>>> >>> >>> I've just rebased https://gerrit.ovirt.org/100716 on top of Galit's >>> change so we should know within an hour or so ... >>> >> >> CI finished successfully, please review so we can merge ... >> >> >>> >>>> >>>> On Tue, Jun 11, 2019 at 2:40 PM Eyal Edri <eedri(a)redhat.com&gt; wrote: >>>> >>>>> >>>>> >>>>> On Tue, Jun 11, 2019 at 4:35 PM Marcin Sobczyk <msobczyk(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> basic suite fails for the patch with the following log: >>>>>> >>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... - STDERR >>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... + yum -y install ovirt-host >>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Error: Package: 32:bind-libs-9.9.4-74.el7_6.1.x86_64 (alocalsync) >>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Requires: bind-license = 32:9.9.4-74.el7_6.1 >>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... Installed: 32:bind-license-9.9.4-73.el7_6.noarch (installed) >>>>>> <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... bind-license = 32:9.9.4-73.el7_6 >>>>>> >>>>>> >>>>> I think https://gerrit.ovirt.org/#/c/100691/ which was merged 2 >>>>> hours ago should address this issue. >>>>> Maybe other reposync files should be updated as well? >>>>> >>>>> >>>>>> Seems unrelated and I'm having the same issue locally on my server. >>>>>> Dafna, do you have some insight regarding this dependency error? >>>>>> >>>>>> Thanks, Marcin >>>>>> On 6/11/19 1:15 PM, Martin Perina wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jun 11, 2019 at 11:58 AM Milan Zamazal <mzamazal(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> Dafna Ron <dron(a)redhat.com&gt; writes: >>>>>>> >>>>>>> > Hi, >>>>>>> > >>>>>>> > Please note vdsm has been broken since Fri the 7th >>>>>>> > >>>>>>> > to summarize again, vdsm has a patch to remove sos plugin which >>>>>>> is what >>>>>>> > metrics is using in its ost tests >>>>>>> > due to that, vdsm is failing the metrics tests and in order to >>>>>>> solve it we >>>>>>> > need to make a choice: >>>>>>> > 1. fix the metrics tests to not use sos >>>>>>> > 2. disable the metrics tests >>>>>>> > 3. revert the sos patch until a decision is made on ^^ >>>>>>> >>>>>>> #3 is not an option, it would make Vdsm uninstallable on newer RHEL >>>>>>> versions. >>>>>>> >>>>>> >>>>>> I've posted a patch https://gerrit.ovirt.org/100716 which is trying >>>>>> to install vdsm sos plugin if it's not installed either by vdsm nor sos. >>>>>> Currenlty waiting for CI, if run is successfull, I will extend the >>>>>> patch also for 4.3 basic suite. >>>>>> >>>>>> >>>>>>> > Thanks, >>>>>>> > Dafna >>>>>>> > >>>>>>> > >>>>>>> > ---------- Forwarded message --------- >>>>>>> > From: Dafna Ron <dron(a)redhat.com&gt; >>>>>>> > Date: Mon, Jun 10, 2019 at 1:30 PM >>>>>>> > Subject: Re: [ OST Failure Report ] [ oVirt Master (vdsm) ] [ >>>>>>> 07-06-2019 ] >>>>>>> > [ 003_00_metrics_bootstrap.metrics_and_log_collector ] >>>>>>> > To: Martin Perina <mperina(a)redhat.com&gt;, Milan Zamazal < >>>>>>> mzamazal(a)redhat.com&gt;, >>>>>>> > Shirly Radco <sradco(a)redhat.com&gt; >>>>>>> > Cc: devel <devel(a)ovirt.org&gt;, infra <infra(a)ovirt.org&gt; >>>>>>> > >>>>>>> > >>>>>>> > Shirly? any update on this? >>>>>>> > >>>>>>> > On Fri, Jun 7, 2019 at 11:54 AM Dafna Ron <dron(a)redhat.com&gt; >>>>>>> wrote: >>>>>>> > >>>>>>> >> Hi, >>>>>>> >> >>>>>>> >> We have a failure in vdsm project on master. >>>>>>> >> >>>>>>> >> The issue is change: >>>>>>> >> https://gerrit.ovirt.org/#/c/100576/ - Remove SOS VDSM plugin >>>>>>> >> >>>>>>> >> which is failing on metrics as metrics is calling >>>>>>> sos-logcollector. >>>>>>> >> >>>>>>> >> The patch cannot be changed as until centos 7.7 when sos-3.7-3, >>>>>>> which >>>>>>> >> contains vdsm plugin will come out. >>>>>>> >> so until then, we are left with no sos plugin, which is causing >>>>>>> the >>>>>>> >> metrics test to fail. >>>>>>> >> >>>>>>> >> Shirly, can you please take a look and see if we can change the >>>>>>> test to >>>>>>> >> not call sos-logcollector? >>>>>>> >> Please note, that we are expecting 4.3 to fail on same issue >>>>>>> very soon. >>>>>>> >> >>>>>>> >> failed job can be found here: >>>>>>> >> >>>>>>> >> >>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/14452/ >>>>>>> >> >>>>>>> >> >>>>>>> >> ERROR from test: >>>>>>> >> >>>>>>> >> lago.ssh: DEBUG: Command 8626fe70 on >>>>>>> lago-basic-suite-master-engine errors: >>>>>>> >> ERROR: Failed to get a sosreport from: >>>>>>> lago-basic-suite-master-host-1; Could not parse sosreport output >>>>>>> >> ERROR: Failed to get a sosreport from: >>>>>>> lago-basic-suite-master-host-0; Could not parse sosreport output >>>>>>> >> >>>>>>> >> lago.utils: DEBUG: Error while running thread Thread-3 >>>>>>> >> Traceback (most recent call last): >>>>>>> >> File "/usr/lib/python2.7/site-packages/lago/utils.py", line >>>>>>> 58, in _ret_via_queue >>>>>>> >> queue.put({'return': func()}) >>>>>>> >> File >>>>>>> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/003_00_metrics_bootstrap.py", >>>>>>> line 97, in run_log_collector >>>>>>> >> 'log collector failed. Exit code is %s' % result.code >>>>>>> >> File "/usr/lib64/python2.7/unittest/case.py", line 462, in >>>>>>> assertTrue >>>>>>> >> raise self.failureException(msg) >>>>>>> >> AssertionError: log collector failed. Exit code is 1 >>>>>>> >> --------------------- >> end captured logging << ---------- >>>>>>> >> >>>>>>> >> Thanks, >>>>>>> >> Dafna >>>>>>> >> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Martin Perina >>>>>> Manager, Software Engineering >>>>>> Red Hat Czech s.r.o. >>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/SQJDJHHQR52... >>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list -- devel(a)ovirt.org >>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: >>>>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/GW3E7ZIE3WI... >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Eyal edri >>>>> >>>>> >>>>> MANAGER >>>>> >>>>> CONTINUOUS PRODUCTIZATION >>>>> >>>>> SYSTEM ENGINEERING >>>>> >>>>> Red Hat <https://www.redhat.com/> >>>>> <https://red.ht/sig> >>>>> phone: +972-9-7692018 >>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel) >>>>> >>>> _______________________________________________ >>>> Devel mailing list -- devel(a)ovirt.org >>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/3TLL7C4LF76... >>>> >>> >>> >>> -- >>> Martin Perina >>> Manager, Software Engineering >>> Red Hat Czech s.r.o. >>> >> >> >> -- >> Martin Perina >> Manager, Software Engineering >> Red Hat Czech s.r.o. >> > _______________________________________________ Infra mailing list -- infra(a)ovirt.org To unsubscribe send an email to infra-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/255DI3N4FIZ...