Re: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.3 - Build # 346 - Still Failing!

On Wed, Feb 12, 2020 at 12:27 PM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Mon, Feb 10, 2020 at 3:19 PM Barak Korren <bkorren(a)redhat.com&gt; wrote: > >> >> >> On Mon, 10 Feb 2020 at 15:11, Dominik Holler <dholler(a)redhat.com&gt; wrote: >> >>> >>> On Mon, Feb 10, 2020 at 10:32 AM Dominik Holler <dholler(a)redhat.com&gt; >>> wrote: >>> >>>> Hello, >>>> is this issue reproducible local for someone, or does this happen only >>>> on jenkins? >>>> >>>> >>> For me it seems to happen only on jenkins. >>> I don't know enough about the environment on jenkins. >>> Might a "yum update" fix the problem? >>> >> >> >> `yum update` actually caused the problem... >> >> >>> I guess that there are incompatible versions of libvirt and firewalld >>> installed. >>> >> >> We have the latest versions for both, currently we suspect its the >> startup order >> >> >>> >>> >>>> On Mon, Feb 10, 2020 at 9:46 AM Galit Rosenthal <grosenth(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Checking this >>>>> Once I have more info I will update. >>>>> >>>>> On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth(a)redhat.com&gt; >>>>> wrote: >>>>> >>>>>> Hey! >>>>>> >>>>>> hc_basic_suite_4.3 is failing with the same error as well >>>>>> >>>>>> Project: >>>>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-te... >>>>>> Build: >>>>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-te... >>>>>> <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-te... >>>>>> l >>>>>> >>>>>> *07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable >>>>>> >>>>>> > I now spent some time and think I at least managed to find what emits > this error: > > https://github.com/firewalld/firewalld/blob/master/src/firewall/core/fw.py > > So perhaps it's an update to firewalld, or to iptables, ip6tables, etc. > I think it is: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/348/c... *03:56:02* ---> Package firewalld.noarch 0:0.5.3-5.el7 will be updated *03:56:02* ---> Package firewalld.noarch 0:0.6.3-2.el7_7.3 will be an update

> >> *07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> On Mon, Feb 10, 2020 at 4:13 AM <jenkins(a)jenkins.phx.ovirt.org&gt; >>>>>>> wrote: >>>>>>> > >>>>>>> > Project: >>>>>>> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/ >>>>>>> > Build: >>>>>>> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/ >>>>>>> >>>>>>> lago.log has: >>>>>>> >>>>>>> 1. some repo issue: >>>>>>> >>>>>>> 2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts >>>>>>> found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing >>>>>>> >>>>>>> Galit - any idea? >>>>>>> >>>>>>> 2. IPv6 issue - after a long series of tracebacks: >>>>>>> >>>>>>> libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid >>>>>>> backend or is unavailable >>>>>>> >>>>>>> Dominik, any idea? Not sure if this is an infra issue or a recent >>>>>>> change to OST (or lago, or libvirt...). >>>>>>> >>>>>>> Thanks and best regards, >>>>>>> -- >>>>>>> Didi >>>>>>> _______________________________________________ >>>>>>> Infra mailing list -- infra(a)ovirt.org >>>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: >>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: >>>>>>> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW... >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> GALIT ROSENTHAL >>>>> >>>>> SOFTWARE ENGINEER >>>>> >>>>> Red Hat >>>>> >>>>> <https://www.redhat.com/> >>>>> >>>>> galit(a)redhat.com T: 972-9-7692230 >>>>> <https://red.ht/sig> >>>>> >>>> _______________________________________________ >>> Infra mailing list -- infra(a)ovirt.org >>> To unsubscribe send an email to infra-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYYL7RVRKNW... >>> >> >> >> -- >> Barak Korren >> RHV DevOps team , RHCE, RHCi >> Red Hat EMEA >> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted >> _______________________________________________ >> Infra mailing list -- infra(a)ovirt.org >> To unsubscribe send an email to infra-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/574TXW4JBJG... >> > > > -- > Didi > -- Didi