Re: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.3 - Build # 346 - Still Failing!
On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/ Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/
lago.log has: 1. some repo issue: 2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing Galit - any idea? 2. IPv6 issue - after a long series of tracebacks: libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable Dominik, any idea? Not sure if this is an infra issue or a recent change to OST (or lago, or libvirt...). Thanks and best regards, -- Didi
Hey! hc_basic_suite_4.3 is failing with the same error as well Project: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Build: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> l *07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/
lago.log has:
1. some repo issue:
2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing
Galit - any idea?
2. IPv6 issue - after a long series of tracebacks:
libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
Dominik, any idea? Not sure if this is an infra issue or a recent change to OST (or lago, or libvirt...).
Thanks and best regards, -- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R...
Checking this Once I have more info I will update. On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth@redhat.com> wrote:
Hey!
hc_basic_suite_4.3 is failing with the same error as well
Project: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Build: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> l
*07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status
On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/
lago.log has:
1. some repo issue:
2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing
Galit - any idea?
2. IPv6 issue - after a long series of tracebacks:
libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
Dominik, any idea? Not sure if this is an infra issue or a recent change to OST (or lago, or libvirt...).
Thanks and best regards, -- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R...
-- GALIT ROSENTHAL SOFTWARE ENGINEER Red Hat <https://www.redhat.com/> galit@redhat.com T: 972-9-7692230 <https://red.ht/sig>
Hello, is this issue reproducible local for someone, or does this happen only on jenkins? On Mon, Feb 10, 2020 at 9:46 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Checking this Once I have more info I will update.
On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth@redhat.com> wrote:
Hey!
hc_basic_suite_4.3 is failing with the same error as well
Project: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Build: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> l
*07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status
On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/
lago.log has:
1. some repo issue:
2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing
Galit - any idea?
2. IPv6 issue - after a long series of tracebacks:
libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
Dominik, any idea? Not sure if this is an infra issue or a recent change to OST (or lago, or libvirt...).
Thanks and best regards, -- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R...
--
GALIT ROSENTHAL
SOFTWARE ENGINEER
Red Hat
galit@redhat.com T: 972-9-7692230 <https://red.ht/sig>
On Mon, Feb 10, 2020 at 10:32 AM Dominik Holler <dholler@redhat.com> wrote:
Hello, is this issue reproducible local for someone, or does this happen only on jenkins?
For me it seems to happen only on jenkins. I don't know enough about the environment on jenkins. Might a "yum update" fix the problem? I guess that there are incompatible versions of libvirt and firewalld installed.
On Mon, Feb 10, 2020 at 9:46 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Checking this Once I have more info I will update.
On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth@redhat.com> wrote:
Hey!
hc_basic_suite_4.3 is failing with the same error as well
Project: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Build: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> l
*07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status
On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/
lago.log has:
1. some repo issue:
2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing
Galit - any idea?
2. IPv6 issue - after a long series of tracebacks:
libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
Dominik, any idea? Not sure if this is an infra issue or a recent change to OST (or lago, or libvirt...).
Thanks and best regards, -- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R...
--
GALIT ROSENTHAL
SOFTWARE ENGINEER
Red Hat
galit@redhat.com T: 972-9-7692230 <https://red.ht/sig>
On Mon, 10 Feb 2020 at 15:11, Dominik Holler <dholler@redhat.com> wrote:
On Mon, Feb 10, 2020 at 10:32 AM Dominik Holler <dholler@redhat.com> wrote:
Hello, is this issue reproducible local for someone, or does this happen only on jenkins?
For me it seems to happen only on jenkins. I don't know enough about the environment on jenkins. Might a "yum update" fix the problem?
`yum update` actually caused the problem...
I guess that there are incompatible versions of libvirt and firewalld installed.
We have the latest versions for both, currently we suspect its the startup order
On Mon, Feb 10, 2020 at 9:46 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Checking this Once I have more info I will update.
On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth@redhat.com> wrote:
Hey!
hc_basic_suite_4.3 is failing with the same error as well
Project: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Build: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> l
*07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status
On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> wrote:
Project:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/
lago.log has:
1. some repo issue:
2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing
Galit - any idea?
2. IPv6 issue - after a long series of tracebacks:
libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
Dominik, any idea? Not sure if this is an infra issue or a recent change to OST (or lago, or libvirt...).
Thanks and best regards, -- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R...
--
GALIT ROSENTHAL
SOFTWARE ENGINEER
Red Hat
galit@redhat.com T: 972-9-7692230 <https://red.ht/sig>
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYYL7RVRKNWHD6...
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Mon, Feb 10, 2020 at 3:19 PM Barak Korren <bkorren@redhat.com> wrote:
On Mon, 10 Feb 2020 at 15:11, Dominik Holler <dholler@redhat.com> wrote:
On Mon, Feb 10, 2020 at 10:32 AM Dominik Holler <dholler@redhat.com> wrote:
Hello, is this issue reproducible local for someone, or does this happen only on jenkins?
For me it seems to happen only on jenkins. I don't know enough about the environment on jenkins. Might a "yum update" fix the problem?
`yum update` actually caused the problem...
I guess that there are incompatible versions of libvirt and firewalld installed.
We have the latest versions for both, currently we suspect its the startup order
On Mon, Feb 10, 2020 at 9:46 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Checking this Once I have more info I will update.
On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth@redhat.com> wrote:
Hey!
hc_basic_suite_4.3 is failing with the same error as well
Project: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Build: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> l
*07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
I now spent some time and think I at least managed to find what emits this error: https://github.com/firewalld/firewalld/blob/master/src/firewall/core/fw.py So perhaps it's an update to firewalld, or to iptables, ip6tables, etc.
*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status
On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/
lago.log has:
1. some repo issue:
2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing
Galit - any idea?
2. IPv6 issue - after a long series of tracebacks:
libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
Dominik, any idea? Not sure if this is an infra issue or a recent change to OST (or lago, or libvirt...).
Thanks and best regards, -- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R...
--
GALIT ROSENTHAL
SOFTWARE ENGINEER
Red Hat
galit@redhat.com T: 972-9-7692230 <https://red.ht/sig>
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYYL7RVRKNWHD6...
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/574TXW4JBJGY53...
-- Didi
On Wed, Feb 12, 2020 at 12:27 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 3:19 PM Barak Korren <bkorren@redhat.com> wrote:
On Mon, 10 Feb 2020 at 15:11, Dominik Holler <dholler@redhat.com> wrote:
On Mon, Feb 10, 2020 at 10:32 AM Dominik Holler <dholler@redhat.com> wrote:
Hello, is this issue reproducible local for someone, or does this happen only on jenkins?
For me it seems to happen only on jenkins. I don't know enough about the environment on jenkins. Might a "yum update" fix the problem?
`yum update` actually caused the problem...
I guess that there are incompatible versions of libvirt and firewalld installed.
We have the latest versions for both, currently we suspect its the startup order
On Mon, Feb 10, 2020 at 9:46 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Checking this Once I have more info I will update.
On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth@redhat.com> wrote:
Hey!
hc_basic_suite_4.3 is failing with the same error as well
Project: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Build: https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> l
*07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
I now spent some time and think I at least managed to find what emits this error:
https://github.com/firewalld/firewalld/blob/master/src/firewall/core/fw.py
So perhaps it's an update to firewalld, or to iptables, ip6tables, etc.
I think it is: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/348/cons... *03:56:02* ---> Package firewalld.noarch 0:0.5.3-5.el7 will be updated *03:56:02* ---> Package firewalld.noarch 0:0.6.3-2.el7_7.3 will be an update
*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status
On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> wrote:
> On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> > wrote: > > > > Project: > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/ > > Build: > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/ > > lago.log has: > > 1. some repo issue: > > 2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts > found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing > > Galit - any idea? > > 2. IPv6 issue - after a long series of tracebacks: > > libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid > backend or is unavailable > > Dominik, any idea? Not sure if this is an infra issue or a recent > change to OST (or lago, or libvirt...). > > Thanks and best regards, > -- > Didi > _______________________________________________ > Infra mailing list -- infra@ovirt.org > To unsubscribe send an email to infra-leave@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R... >
--
GALIT ROSENTHAL
SOFTWARE ENGINEER
Red Hat
galit@redhat.com T: 972-9-7692230 <https://red.ht/sig>
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYYL7RVRKNWHD6...
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/574TXW4JBJGY53...
-- Didi
-- Didi
On Wed, 12 Feb 2020 at 12:47, Yedidyah Bar David <didi@redhat.com> wrote:
On Wed, Feb 12, 2020 at 12:27 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Feb 10, 2020 at 3:19 PM Barak Korren <bkorren@redhat.com> wrote:
On Mon, 10 Feb 2020 at 15:11, Dominik Holler <dholler@redhat.com> wrote:
On Mon, Feb 10, 2020 at 10:32 AM Dominik Holler <dholler@redhat.com> wrote:
Hello, is this issue reproducible local for someone, or does this happen only on jenkins?
For me it seems to happen only on jenkins. I don't know enough about the environment on jenkins. Might a "yum update" fix the problem?
`yum update` actually caused the problem...
I guess that there are incompatible versions of libvirt and firewalld installed.
We have the latest versions for both, currently we suspect its the startup order
On Mon, Feb 10, 2020 at 9:46 AM Galit Rosenthal <grosenth@redhat.com> wrote:
Checking this Once I have more info I will update.
On Mon, Feb 10, 2020 at 9:16 AM Parth Dhanjal <dparth@redhat.com> wrote:
> Hey! > > hc_basic_suite_4.3 is failing with the same error as well > > Project: > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > Build: > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > <https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_hc-basic-suite-4.3/328/consoleFull> > l > > *07:36:24* @ Start Prefix: *07:36:24* # Start nets: *07:36:24* * Create network lago-hc-basic-suite-4-3-net-management: *07:36:30* * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:05)*07:36:30* # Start nets: ERROR (in 0:00:05)*07:36:30* @ Start Prefix: ERROR (in 0:00:05)*07:36:30* Error occured, aborting*07:36:30* Traceback (most recent call last):*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:36:30* cli_plugins[args.verb].do_run(args)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:36:30* self._do_run(**vars(args))*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:36:30* return func(*args, prefix=prefix, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:36:30* prefix.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:36:30* return func(*args, **kwargs)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:36:30* self.virt_env.start(vm_names=vm_names)*07:36:30* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:36:30* net.start()*07:36:30* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:36:30* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:36:30* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:36:30* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:36:30* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable > >
I now spent some time and think I at least managed to find what emits this error:
https://github.com/firewalld/firewalld/blob/master/src/firewall/core/fw.py
So perhaps it's an update to firewalld, or to iptables, ip6tables, etc.
I think it is:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/348/cons...
*03:56:02* ---> Package firewalld.noarch 0:0.5.3-5.el7 will be updated
*03:56:02* ---> Package firewalld.noarch 0:0.6.3-2.el7_7.3 will be an update
This means we actually have an updated package by the time the error occurs. After some research we reached the conclusion this has to do with having firewalld run and get updated inside a privileged container. It seems that during its startup process it removes the IPv6 kernel modules from the running kernel and then fails to re-load them because the module files are not available from inside the container. Galit is working on a patch to remove firewalld from the container since we don't really need it there.
*07:36:30* + on_exit*07:36:30* + [[ 1 -ne 0 ]]*07:36:30* + logger.error 'on_exit: Exiting with a non-zero status'*07:36:30* + logger.log ERROR 'on_exit: Exiting with a non-zero status'*07:36:30* + set +x*07:36:30* 2020-02-10 02:06:30.296966051+0000 run_suite.sh::on_exit::ERROR:: on_exit: Exiting with a non-zero status
> > > > On Mon, Feb 10, 2020 at 12:08 PM Yedidyah Bar David <didi@redhat.com> > wrote: > >> On Mon, Feb 10, 2020 at 4:13 AM <jenkins@jenkins.phx.ovirt.org> >> wrote: >> > >> > Project: >> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/ >> > Build: >> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.3/346/ >> >> lago.log has: >> >> 1. some repo issue: >> >> 2020-02-10 02:11:13,681::ERROR::repoman.common.parser::No artifacts >> found for source /var/lib/lago/ovirt-appliance-4.3-el7:only-missing >> >> Galit - any idea? >> >> 2. IPv6 issue - after a long series of tracebacks: >> >> libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid >> backend or is unavailable >> >> Dominik, any idea? Not sure if this is an infra issue or a recent >> change to OST (or lago, or libvirt...). >> >> Thanks and best regards, >> -- >> Didi >> _______________________________________________ >> Infra mailing list -- infra@ovirt.org >> To unsubscribe send an email to infra-leave@ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/URERUP2VUCW25R... >> >
--
GALIT ROSENTHAL
SOFTWARE ENGINEER
Red Hat
galit@redhat.com T: 972-9-7692230 <https://red.ht/sig>
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYYL7RVRKNWHD6...
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/574TXW4JBJGY53...
-- Didi
-- Didi
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
participants (5)
-
Barak Korren -
Dominik Holler -
Galit Rosenthal -
Parth Dhanjal -
Yedidyah Bar David