On Tue, Jan 21, 2020 at 1:18 AM oVirt Jenkins <jenkins(a)ovirt.org&gt; wrote: > > Change 66276a7 (ovirt-ansible-hosted-engine-setup) is probably the reason > behind recent system test failures in the "ovirt-master" change queue and needs > to be fixed. > > This change had been removed from the testing queue. Artifacts build from this > change will not be released until it is fixed. > > For further details about the change see: > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/66276a7... Above change is unrelated to the failure below. How can I make CQ look at it again? > > For failed test results see: > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/ This failed in basic suite, in 002_bootstrap.verify_add_hosts. engine.log [1] has (e.g.): 2020-01-20 17:56:33,198-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [66755eba] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM lago-basic-suite-master-host-0 command HostSetupNetworksVDS failed: Internal JSON-RPC error: {'reason': "type object 'LinuxBridge' has no attribute 'STP'"} supervdsm log [2] has: MainProcess|jsonrpc/4::INFO::2020-01-20 17:56:32,430::configurator::190::root::(_setup_nmstate) Processing setup through nmstate MainProcess|jsonrpc/4::ERROR::2020-01-20 17:56:32,695::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line 95, in wrapper res = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line 240, in setupNetworks _setup_networks(networks, bondings, options, net_info) File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line 265, in _setup_networks networks, bondings, options, net_info, in_rollback File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", line 154, in setup _setup_nmstate(networks, bondings, options, in_rollback, net_info) File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", line 195, in _setup_nmstate desired_state = nmstate.generate_state(networks, bondings) File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 73, in generate_state networks, rconfig.networks, current_ifaces_state File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 603, in generate_state for netname, netattrs in six.viewitems(networks) File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 603, in <listcomp> for netname, netattrs in six.viewitems(networks) File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 339, in __init__ self._create_interfaces_state() File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 430, in _create_interfaces_state sb_iface, vlan_iface, bridge_iface = self._create_ifaces() File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 444, in _create_ifaces options=self._create_bridge_options(), File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", line 492, in _create_bridge_options LinuxBridge.STP.ENABLED: self._netconf.stp AttributeError: type object 'LinuxBridge' has no attribute 'STP' Perhaps that's related to recent changes adding/updating nmstate? [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... > _______________________________________________ > Infra mailing list -- infra(a)ovirt.org > To unsubscribe send an email to infra-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/JTIBSBDL5IY... -- Didi

OK, I think I understand. STP is probably only in nmstate-0.2, and a vdsm that requires this still didn't pass CQ.

On Wed, Jan 22, 2020 at 11:33 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > > Resending and adding devel. > > This now happened to me again. I suspect this affects other runs. Any clue? > > https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7692/ > > On Tue, Jan 21, 2020 at 12:34 PM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > > > > On Tue, Jan 21, 2020 at 1:18 AM oVirt Jenkins <jenkins(a)ovirt.org&gt; wrote: > > > > > > Change 66276a7 (ovirt-ansible-hosted-engine-setup) is probably the reason > > > behind recent system test failures in the "ovirt-master" change queue and needs > > > to be fixed. > > > > > > This change had been removed from the testing queue. Artifacts build from this > > > change will not be released until it is fixed. > > > > > > For further details about the change see: > > > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/66276a7... > > > > Above change is unrelated to the failure below. How can I make CQ look > > at it again? > > > > > > > > For failed test results see: > > > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/ > > > > This failed in basic suite, in 002_bootstrap.verify_add_hosts. > > > > engine.log [1] has (e.g.): > > > > 2020-01-20 17:56:33,198-05 ERROR > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > (EE-ManagedThreadFactory-engine-Thread-1) [66755eba] EVENT_ID: > > VDS_BROKER_COMMAND_FAILURE(10,802), VDSM > > lago-basic-suite-master-host-0 command HostSetupNetworksVDS failed: > > Internal JSON-RPC error: {'reason': "type object 'LinuxBridge' has no > > attribute 'STP'"} > > > > supervdsm log [2] has: > > > > MainProcess|jsonrpc/4::INFO::2020-01-20 > > 17:56:32,430::configurator::190::root::(_setup_nmstate) Processing > > setup through nmstate > > MainProcess|jsonrpc/4::ERROR::2020-01-20 > > 17:56:32,695::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) > > Error in setupNetworks > > Traceback (most recent call last): > > File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", > > line 95, in wrapper > > res = func(*args, **kwargs) > > File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line > > 240, in setupNetworks > > _setup_networks(networks, bondings, options, net_info) > > File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line > > 265, in _setup_networks > > networks, bondings, options, net_info, in_rollback > > File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", > > line 154, in setup > > _setup_nmstate(networks, bondings, options, in_rollback, net_info) > > File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", > > line 195, in _setup_nmstate > > desired_state = nmstate.generate_state(networks, bondings) > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", > > line 73, in generate_state > > networks, rconfig.networks, current_ifaces_state > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", > > line 603, in generate_state > > for netname, netattrs in six.viewitems(networks) > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", > > line 603, in <listcomp> > > for netname, netattrs in six.viewitems(networks) > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", > > line 339, in __init__ > > self._create_interfaces_state() > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", > > line 430, in _create_interfaces_state > > sb_iface, vlan_iface, bridge_iface = self._create_ifaces() > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", > > line 444, in _create_ifaces > > options=self._create_bridge_options(), > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", > > line 492, in _create_bridge_options > > LinuxBridge.STP.ENABLED: self._netconf.stp > > AttributeError: type object 'LinuxBridge' has no attribute 'STP' > > > > Perhaps that's related to recent changes adding/updating nmstate? > > > > [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... > > [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... > > > > > > > _______________________________________________ > > > Infra mailing list -- infra(a)ovirt.org > > > To unsubscribe send an email to infra-leave(a)ovirt.org > > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > > > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/JTIBSBDL5IY... > > > > > > > > -- > > Didi > > > > -- > Didi -- Didi _______________________________________________ Infra mailing list -- infra(a)ovirt.org To unsubscribe send an email to infra-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYTTA36U7YH...

On Wed, Jan 22, 2020 at 12:10 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > > > > On Wed, Jan 22, 2020, 11:59 Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >> >> OK, I think I understand. STP is probably only in nmstate-0.2, and a >> vdsm that requires this still didn't pass CQ. > > > Same error happen when adding Fedora 30 host with master. > > Adding a host broken now on Fedora. Netwok team need to revert the change causing this issue. No. They fixed it, but the fix didn't pass CQ yet.

[2]: libvirt.libvirtError: the CPU is incompatible with host CPU: Host CPU does not provide required features: hle, rtm Adding Michal and Martin. Not sure if that's a known issue/already fixed somewhere. [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/ [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/arti... > > >> >> On Wed, Jan 22, 2020 at 11:33 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >> > >> > Resending and adding devel. >> > >> > This now happened to me again. I suspect this affects other runs. Any clue? >> > >> > https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7692/ >> > >> > On Tue, Jan 21, 2020 at 12:34 PM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >> > > >> > > On Tue, Jan 21, 2020 at 1:18 AM oVirt Jenkins <jenkins(a)ovirt.org&gt; wrote: >> > > > >> > > > Change 66276a7 (ovirt-ansible-hosted-engine-setup) is probably the reason >> > > > behind recent system test failures in the "ovirt-master" change queue and needs >> > > > to be fixed. >> > > > >> > > > This change had been removed from the testing queue. Artifacts build from this >> > > > change will not be released until it is fixed. >> > > > >> > > > For further details about the change see: >> > > > https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/66276a7... >> > > >> > > Above change is unrelated to the failure below. How can I make CQ look >> > > at it again? >> > > >> > > > >> > > > For failed test results see: >> > > > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/ >> > > >> > > This failed in basic suite, in 002_bootstrap.verify_add_hosts. >> > > >> > > engine.log [1] has (e.g.): >> > > >> > > 2020-01-20 17:56:33,198-05 ERROR >> > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> > > (EE-ManagedThreadFactory-engine-Thread-1) [66755eba] EVENT_ID: >> > > VDS_BROKER_COMMAND_FAILURE(10,802), VDSM >> > > lago-basic-suite-master-host-0 command HostSetupNetworksVDS failed: >> > > Internal JSON-RPC error: {'reason': "type object 'LinuxBridge' has no >> > > attribute 'STP'"} >> > > >> > > supervdsm log [2] has: >> > > >> > > MainProcess|jsonrpc/4::INFO::2020-01-20 >> > > 17:56:32,430::configurator::190::root::(_setup_nmstate) Processing >> > > setup through nmstate >> > > MainProcess|jsonrpc/4::ERROR::2020-01-20 >> > > 17:56:32,695::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) >> > > Error in setupNetworks >> > > Traceback (most recent call last): >> > > File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", >> > > line 95, in wrapper >> > > res = func(*args, **kwargs) >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line >> > > 240, in setupNetworks >> > > _setup_networks(networks, bondings, options, net_info) >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line >> > > 265, in _setup_networks >> > > networks, bondings, options, net_info, in_rollback >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >> > > line 154, in setup >> > > _setup_nmstate(networks, bondings, options, in_rollback, net_info) >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >> > > line 195, in _setup_nmstate >> > > desired_state = nmstate.generate_state(networks, bondings) >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >> > > line 73, in generate_state >> > > networks, rconfig.networks, current_ifaces_state >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >> > > line 603, in generate_state >> > > for netname, netattrs in six.viewitems(networks) >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >> > > line 603, in <listcomp> >> > > for netname, netattrs in six.viewitems(networks) >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >> > > line 339, in __init__ >> > > self._create_interfaces_state() >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >> > > line 430, in _create_interfaces_state >> > > sb_iface, vlan_iface, bridge_iface = self._create_ifaces() >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >> > > line 444, in _create_ifaces >> > > options=self._create_bridge_options(), >> > > File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >> > > line 492, in _create_bridge_options >> > > LinuxBridge.STP.ENABLED: self._netconf.stp >> > > AttributeError: type object 'LinuxBridge' has no attribute 'STP' >> > > >> > > Perhaps that's related to recent changes adding/updating nmstate? >> > > >> > > [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >> > > [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >> > > >> > > >> > > > _______________________________________________ >> > > > Infra mailing list -- infra(a)ovirt.org >> > > > To unsubscribe send an email to infra-leave(a)ovirt.org >> > > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> > > > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> > > > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/JTIBSBDL5IY... >> > > >> > > >> > > >> > > -- >> > > Didi >> > >> > >> > >> > -- >> > Didi >> >> >> >> -- >> Didi >> _______________________________________________ >> Infra mailing list -- infra(a)ovirt.org >> To unsubscribe send an email to infra-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYTTA36U7YH... -- Didi

On 22 Jan 2020, at 11:46, Yedidyah Bar David <didi(a)redhat.com&gt; wrote: On Wed, Jan 22, 2020 at 12:34 PM Nir Soffer <nsoffer(a)redhat.com <mailto:nsoffer@redhat.com>> wrote: > > On Wed, Jan 22, 2020 at 12:25 PM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >> >> On Wed, Jan 22, 2020 at 12:10 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >>> >>> >>> >>> On Wed, Jan 22, 2020, 11:59 Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >>>> >>>> OK, I think I understand. STP is probably only in nmstate-0.2, and a >>>> vdsm that requires this still didn't pass CQ.

>>> >>> >>> Same error happen when adding Fedora 30 host with master. >>> >>> Adding a host broken now on Fedora. Netwok team need to revert the change causing this issue. >> >> No. They fixed it, but the fix didn't pass CQ yet. > > > Not related to change queue, engine/vdsm master are broken. > > Did you try to add Fedora host with current engine and vdsm? No, but the fix seems to deliberately not support Fedora, see also the review comments: https://gerrit.ovirt.org/106413 <https://gerrit.ovirt.org/106413>

> >> Last CQ run that finished [1], failed for a different reason. vdsm.log has [2]: >> >> libvirt.libvirtError: the CPU is incompatible with host CPU: Host CPU >> does not provide required features: hle, rtm >> >> Adding Michal and Martin. Not sure if that's a known issue/already >> fixed somewhere. >> >> [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/ >> [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/arti... >> >>> >>> >>>> >>>> On Wed, Jan 22, 2020 at 11:33 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >>>>> >>>>> Resending and adding devel. >>>>> >>>>> This now happened to me again. I suspect this affects other runs. Any clue? >>>>> >>>>> https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7692/ >>>>> >>>>> On Tue, Jan 21, 2020 at 12:34 PM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >>>>>> >>>>>> On Tue, Jan 21, 2020 at 1:18 AM oVirt Jenkins <jenkins(a)ovirt.org&gt; wrote: >>>>>>> >>>>>>> Change 66276a7 (ovirt-ansible-hosted-engine-setup) is probably the reason >>>>>>> behind recent system test failures in the "ovirt-master" change queue and needs >>>>>>> to be fixed. >>>>>>> >>>>>>> This change had been removed from the testing queue. Artifacts build from this >>>>>>> change will not be released until it is fixed. >>>>>>> >>>>>>> For further details about the change see: >>>>>>> https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/66276a7... >>>>>> >>>>>> Above change is unrelated to the failure below. How can I make CQ look >>>>>> at it again? >>>>>> >>>>>>> >>>>>>> For failed test results see: >>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/ >>>>>> >>>>>> This failed in basic suite, in 002_bootstrap.verify_add_hosts. >>>>>> >>>>>> engine.log [1] has (e.g.): >>>>>> >>>>>> 2020-01-20 17:56:33,198-05 ERROR >>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [66755eba] EVENT_ID: >>>>>> VDS_BROKER_COMMAND_FAILURE(10,802), VDSM >>>>>> lago-basic-suite-master-host-0 command HostSetupNetworksVDS failed: >>>>>> Internal JSON-RPC error: {'reason': "type object 'LinuxBridge' has no >>>>>> attribute 'STP'"} >>>>>> >>>>>> supervdsm log [2] has: >>>>>> >>>>>> MainProcess|jsonrpc/4::INFO::2020-01-20 >>>>>> 17:56:32,430::configurator::190::root::(_setup_nmstate) Processing >>>>>> setup through nmstate >>>>>> MainProcess|jsonrpc/4::ERROR::2020-01-20 >>>>>> 17:56:32,695::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) >>>>>> Error in setupNetworks >>>>>> Traceback (most recent call last): >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", >>>>>> line 95, in wrapper >>>>>> res = func(*args, **kwargs) >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line >>>>>> 240, in setupNetworks >>>>>> _setup_networks(networks, bondings, options, net_info) >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line >>>>>> 265, in _setup_networks >>>>>> networks, bondings, options, net_info, in_rollback >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >>>>>> line 154, in setup >>>>>> _setup_nmstate(networks, bondings, options, in_rollback, net_info) >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >>>>>> line 195, in _setup_nmstate >>>>>> desired_state = nmstate.generate_state(networks, bondings) >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>> line 73, in generate_state >>>>>> networks, rconfig.networks, current_ifaces_state >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>> line 603, in generate_state >>>>>> for netname, netattrs in six.viewitems(networks) >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>> line 603, in <listcomp> >>>>>> for netname, netattrs in six.viewitems(networks) >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>> line 339, in __init__ >>>>>> self._create_interfaces_state() >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>> line 430, in _create_interfaces_state >>>>>> sb_iface, vlan_iface, bridge_iface = self._create_ifaces() >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>> line 444, in _create_ifaces >>>>>> options=self._create_bridge_options(), >>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>> line 492, in _create_bridge_options >>>>>> LinuxBridge.STP.ENABLED: self._netconf.stp >>>>>> AttributeError: type object 'LinuxBridge' has no attribute 'STP' >>>>>> >>>>>> Perhaps that's related to recent changes adding/updating nmstate? >>>>>> >>>>>> [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >>>>>> [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >>>>>> >>>>>> >>>>>>> _______________________________________________ >>>>>>> Infra mailing list -- infra(a)ovirt.org >>>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/JTIBSBDL5IY... >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Didi >>>>> >>>>> >>>>> >>>>> -- >>>>> Didi >>>> >>>> >>>> >>>> -- >>>> Didi >>>> _______________________________________________ >>>> Infra mailing list -- infra(a)ovirt.org >>>> To unsubscribe send an email to infra-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYTTA36U7YH... >> >> >> >> -- >> Didi >> -- Didi

On 22 Jan 2020, at 14:31, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: On Wed, Jan 22, 2020 at 1:41 PM Michal Skrivanek <mskrivan(a)redhat.com <mailto:mskrivan@redhat.com>> wrote: > On 22 Jan 2020, at 11:46, Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: > > On Wed, Jan 22, 2020 at 12:34 PM Nir Soffer <nsoffer(a)redhat.com <mailto:nsoffer@redhat.com>> wrote: >> >> On Wed, Jan 22, 2020 at 12:25 PM Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>> >>> On Wed, Jan 22, 2020 at 12:10 PM Nir Soffer <nsoffer(a)redhat.com <mailto:nsoffer@redhat.com>> wrote: >>>> >>>> >>>> >>>> On Wed, Jan 22, 2020, 11:59 Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>>>> >>>>> OK, I think I understand. STP is probably only in nmstate-0.2, and a >>>>> vdsm that requires this still didn't pass CQ. master “tested” has vdsm witht he correct nmstate requirements. But OST had issue picking it up properly, that was fixed in basic suite yesterday evening by martin perina errors below are older than that >>>> >>>> >>>> Same error happen when adding Fedora 30 host with master. >>>> >>>> Adding a host broken now on Fedora. Netwok team need to revert the change causing this issue. >>> >>> No. They fixed it, but the fix didn't pass CQ yet. >> >> >> Not related to change queue, engine/vdsm master are broken. >> >> Did you try to add Fedora host with current engine and vdsm? > > No, but the fix seems to deliberately not support Fedora, see also the > review comments: > > https://gerrit.ovirt.org/106413 <https://gerrit.ovirt.org/106413> that’s correct. Fro F30 you have to disable nmstate support and use the fallback to ifcfg - set net_nmstate_enabled to false This should be done by vdsm network automatically. We use Fedora 30 for development and we vdsm must continue to work on this distro until we move to Fedora 31.

>> >>> Last CQ run that finished [1], failed for a different reason. vdsm.log has [2]: >>> >>> libvirt.libvirtError: the CPU is incompatible with host CPU: Host CPU >>> does not provide required features: hle, rtm >>> >>> Adding Michal and Martin. Not sure if that's a known issue/already >>> fixed somewhere. >>> >>> [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/ <https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/> >>> [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/arti... <https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/arti... >>> >>>> >>>> >>>>> >>>>> On Wed, Jan 22, 2020 at 11:33 AM Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>>>>> >>>>>> Resending and adding devel. >>>>>> >>>>>> This now happened to me again. I suspect this affects other runs. Any clue? >>>>>> >>>>>> https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7692/ <https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/769... >>>>>> >>>>>> On Tue, Jan 21, 2020 at 12:34 PM Yedidyah Bar David <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>>>>>> >>>>>>> On Tue, Jan 21, 2020 at 1:18 AM oVirt Jenkins <jenkins(a)ovirt.org <mailto:jenkins@ovirt.org>> wrote: >>>>>>>> >>>>>>>> Change 66276a7 (ovirt-ansible-hosted-engine-setup) is probably the reason >>>>>>>> behind recent system test failures in the "ovirt-master" change queue and needs >>>>>>>> to be fixed. >>>>>>>> >>>>>>>> This change had been removed from the testing queue. Artifacts build from this >>>>>>>> change will not be released until it is fixed. >>>>>>>> >>>>>>>> For further details about the change see: >>>>>>>> https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/66276a7... <https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/66276a7... >>>>>>> >>>>>>> Above change is unrelated to the failure below. How can I make CQ look >>>>>>> at it again? >>>>>>> >>>>>>>> >>>>>>>> For failed test results see: >>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/ <https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/> >>>>>>> >>>>>>> This failed in basic suite, in 002_bootstrap.verify_add_hosts. >>>>>>> >>>>>>> engine.log [1] has (e.g.): >>>>>>> >>>>>>> 2020-01-20 17:56:33,198-05 ERROR >>>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [66755eba] EVENT_ID: >>>>>>> VDS_BROKER_COMMAND_FAILURE(10,802), VDSM >>>>>>> lago-basic-suite-master-host-0 command HostSetupNetworksVDS failed: >>>>>>> Internal JSON-RPC error: {'reason': "type object 'LinuxBridge' has no >>>>>>> attribute 'STP'"} >>>>>>> >>>>>>> supervdsm log [2] has: >>>>>>> >>>>>>> MainProcess|jsonrpc/4::INFO::2020-01-20 >>>>>>> 17:56:32,430::configurator::190::root::(_setup_nmstate) Processing >>>>>>> setup through nmstate >>>>>>> MainProcess|jsonrpc/4::ERROR::2020-01-20 >>>>>>> 17:56:32,695::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) >>>>>>> Error in setupNetworks >>>>>>> Traceback (most recent call last): >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", >>>>>>> line 95, in wrapper >>>>>>> res = func(*args, **kwargs) >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line >>>>>>> 240, in setupNetworks >>>>>>> _setup_networks(networks, bondings, options, net_info) >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/api.py", line >>>>>>> 265, in _setup_networks >>>>>>> networks, bondings, options, net_info, in_rollback >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >>>>>>> line 154, in setup >>>>>>> _setup_nmstate(networks, bondings, options, in_rollback, net_info) >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >>>>>>> line 195, in _setup_nmstate >>>>>>> desired_state = nmstate.generate_state(networks, bondings) >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>> line 73, in generate_state >>>>>>> networks, rconfig.networks, current_ifaces_state >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>> line 603, in generate_state >>>>>>> for netname, netattrs in six.viewitems(networks) >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>> line 603, in <listcomp> >>>>>>> for netname, netattrs in six.viewitems(networks) >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>> line 339, in __init__ >>>>>>> self._create_interfaces_state() >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>> line 430, in _create_interfaces_state >>>>>>> sb_iface, vlan_iface, bridge_iface = self._create_ifaces() >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>> line 444, in _create_ifaces >>>>>>> options=self._create_bridge_options(), >>>>>>> File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>> line 492, in _create_bridge_options >>>>>>> LinuxBridge.STP.ENABLED: self._netconf.stp >>>>>>> AttributeError: type object 'LinuxBridge' has no attribute 'STP' >>>>>>> >>>>>>> Perhaps that's related to recent changes adding/updating nmstate? >>>>>>> >>>>>>> [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... <https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >>>>>>> [2] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... <https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >>>>>>> >>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Infra mailing list -- infra(a)ovirt.org <mailto:infra@ovirt.org> >>>>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org <mailto:infra-leave@ovirt.org> >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ <https://www.ovirt.org/site/privacy-policy/> >>>>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >>>>>>>> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/JTIBSBDL5IY... <https://lists.ovirt.org/archives/list/infra@ovirt.org/message/JTIBSBDL5IY... >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Didi >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Didi >>>>> >>>>> >>>>> >>>>> -- >>>>> Didi >>>>> _______________________________________________ >>>>> Infra mailing list -- infra(a)ovirt.org <mailto:infra@ovirt.org> >>>>> To unsubscribe send an email to infra-leave(a)ovirt.org <mailto:infra-leave@ovirt.org> >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ <https://www.ovirt.org/site/privacy-policy/> >>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://www.ovirt.org/community/about/community-guidelines/> >>>>> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYTTA36U7YH... <https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYTTA36U7YH... >>> >>> >>> >>> -- >>> Didi >>> > > > -- > Didi

On Wed, Jan 22, 2020 at 3:38 PM Michal Skrivanek <mskrivan(a)redhat.com <mailto:mskrivan@redhat.com>> wrote: > On 22 Jan 2020, at 14:31, Nir Soffer <nsoffer(a)redhat.com > <mailto:nsoffer@redhat.com>> wrote: > > On Wed, Jan 22, 2020 at 1:41 PM Michal Skrivanek > <mskrivan(a)redhat.com <mailto:mskrivan@redhat.com>> wrote: > > > >> On 22 Jan 2020, at 11:46, Yedidyah Bar David >> <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >> >> On Wed, Jan 22, 2020 at 12:34 PM Nir Soffer >> <nsoffer(a)redhat.com <mailto:nsoffer@redhat.com>> wrote: >>> >>> On Wed, Jan 22, 2020 at 12:25 PM Yedidyah Bar David >>> <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>>> >>>> On Wed, Jan 22, 2020 at 12:10 PM Nir Soffer >>>> <nsoffer(a)redhat.com <mailto:nsoffer@redhat.com>> wrote: >>>>> >>>>> >>>>> >>>>> On Wed, Jan 22, 2020, 11:59 Yedidyah Bar David >>>>> <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>>>>> >>>>>> OK, I think I understand. STP is probably only in >>>>>> nmstate-0.2, and a >>>>>> vdsm that requires this still didn't pass CQ. > > master “tested” has vdsm witht he correct nmstate requirements. > But OST had issue picking it up properly, that was fixed in > basic suite yesterday evening by martin perina > errors below are older than that > >>>>> >>>>> >>>>> Same error happen when adding Fedora 30 host with master. >>>>> >>>>> Adding a host broken now on Fedora. Netwok team need to >>>>> revert the change causing this issue. >>>> >>>> No. They fixed it, but the fix didn't pass CQ yet. >>> >>> >>> Not related to change queue, engine/vdsm master are broken. >>> >>> Did you try to add Fedora host with current engine and vdsm? >> >> No, but the fix seems to deliberately not support Fedora, >> see also the >> review comments: >> >> https://gerrit.ovirt.org/106413 > > that’s correct. Fro F30 you have to disable nmstate support > and use the fallback to ifcfg - set net_nmstate_enabled to false > > > This should be done by vdsm network automatically. > > We use Fedora 30 for development and we vdsm must continue to > work on this distro until we > move to Fedora 31. It seems it would be better to move to F31. This requires effort to move the tests to containers. We have new infrastructure but someone have to spend time on this.

But anyway, for now do you still have a host deployment problem when you disable nmstate on F30? Yes, yesterday it failed in the same way with nmstate enabled or disabled, but I will not have time to investigate this before fosdem. > > >>> >>>> Last CQ run that finished [1], failed for a different >>>> reason. vdsm.log has [2]: >>>> >>>> libvirt.libvirtError: the CPU is incompatible with host >>>> CPU: Host CPU >>>> does not provide required features: hle, rtm >>>> >>>> Adding Michal and Martin. Not sure if that's a known >>>> issue/already >>>> fixed somewhere. >>>> >>>> [1] >>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/ >>>> [2] >>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18225/arti... >>>> >>>>> >>>>> >>>>>> >>>>>> On Wed, Jan 22, 2020 at 11:33 AM Yedidyah Bar David >>>>>> <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>>>>>> >>>>>>> Resending and adding devel. >>>>>>> >>>>>>> This now happened to me again. I suspect this affects >>>>>>> other runs. Any clue? >>>>>>> >>>>>>> https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/7692/ >>>>>>> >>>>>>> On Tue, Jan 21, 2020 at 12:34 PM Yedidyah Bar David >>>>>>> <didi(a)redhat.com <mailto:didi@redhat.com>> wrote: >>>>>>>> >>>>>>>> On Tue, Jan 21, 2020 at 1:18 AM oVirt Jenkins >>>>>>>> <jenkins(a)ovirt.org <mailto:jenkins@ovirt.org>> wrote: >>>>>>>>> >>>>>>>>> Change 66276a7 (ovirt-ansible-hosted-engine-setup) is >>>>>>>>> probably the reason >>>>>>>>> behind recent system test failures in the >>>>>>>>> "ovirt-master" change queue and needs >>>>>>>>> to be fixed. >>>>>>>>> >>>>>>>>> This change had been removed from the testing queue. >>>>>>>>> Artifacts build from this >>>>>>>>> change will not be released until it is fixed. >>>>>>>>> >>>>>>>>> For further details about the change see: >>>>>>>>> https://github.com/oVirt/ovirt-ansible-hosted-engine-setup/commit/66276a7... >>>>>>>> >>>>>>>> Above change is unrelated to the failure below. How >>>>>>>> can I make CQ look >>>>>>>> at it again? >>>>>>>> >>>>>>>>> >>>>>>>>> For failed test results see: >>>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/ >>>>>>>> >>>>>>>> This failed in basic suite, in >>>>>>>> 002_bootstrap.verify_add_hosts. >>>>>>>> >>>>>>>> engine.log [1] has (e.g.): >>>>>>>> >>>>>>>> 2020-01-20 17:56:33,198-05 ERROR >>>>>>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>>>>>> (EE-ManagedThreadFactory-engine-Thread-1) [66755eba] >>>>>>>> EVENT_ID: >>>>>>>> VDS_BROKER_COMMAND_FAILURE(10,802), VDSM >>>>>>>> lago-basic-suite-master-host-0 command >>>>>>>> HostSetupNetworksVDS failed: >>>>>>>> Internal JSON-RPC error: {'reason': "type object >>>>>>>> 'LinuxBridge' has no >>>>>>>> attribute 'STP'"} >>>>>>>> >>>>>>>> supervdsm log [2] has: >>>>>>>> >>>>>>>> MainProcess|jsonrpc/4::INFO::2020-01-20 >>>>>>>> 17:56:32,430::configurator::190::root::(_setup_nmstate) >>>>>>>> Processing >>>>>>>> setup through nmstate >>>>>>>> MainProcess|jsonrpc/4::ERROR::2020-01-20 >>>>>>>> 17:56:32,695::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper) >>>>>>>> Error in setupNetworks >>>>>>>> Traceback (most recent call last): >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", >>>>>>>> line 95, in wrapper >>>>>>>>    res = func(*args, **kwargs) >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/api.py", >>>>>>>> line >>>>>>>> 240, in setupNetworks >>>>>>>>    _setup_networks(networks, bondings, options, net_info) >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/api.py", >>>>>>>> line >>>>>>>> 265, in _setup_networks >>>>>>>>    networks, bondings, options, net_info, in_rollback >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >>>>>>>> line 154, in setup >>>>>>>>    _setup_nmstate(networks, bondings, options, >>>>>>>> in_rollback, net_info) >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py", >>>>>>>> line 195, in _setup_nmstate >>>>>>>>    desired_state = nmstate.generate_state(networks, >>>>>>>> bondings) >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>>> line 73, in generate_state >>>>>>>>    networks, rconfig.networks, current_ifaces_state >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>>> line 603, in generate_state >>>>>>>>    for netname, netattrs in six.viewitems(networks) >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>>> line 603, in <listcomp> >>>>>>>>    for netname, netattrs in six.viewitems(networks) >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>>> line 339, in __init__ >>>>>>>>    self._create_interfaces_state() >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>>> line 430, in _create_interfaces_state >>>>>>>>    sb_iface, vlan_iface, bridge_iface = >>>>>>>> self._create_ifaces() >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>>> line 444, in _create_ifaces >>>>>>>>    options=self._create_bridge_options(), >>>>>>>>  File >>>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py", >>>>>>>> line 492, in _create_bridge_options >>>>>>>>    LinuxBridge.STP.ENABLED: self._netconf.stp >>>>>>>> AttributeError: type object 'LinuxBridge' has no >>>>>>>> attribute 'STP' >>>>>>>> >>>>>>>> Perhaps that's related to recent changes >>>>>>>> adding/updating nmstate? >>>>>>>> >>>>>>>> [1] >>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >>>>>>>> [2] >>>>>>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/18189/arti... >>>>>>>> >>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Infra mailing list -- infra(a)ovirt.org >>>>>>>>> <mailto:infra@ovirt.org> >>>>>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org >>>>>>>>> <mailto:infra-leave@ovirt.org> >>>>>>>>> Privacy Statement: >>>>>>>>> https://www.ovirt.org/site/privacy-policy/ >>>>>>>>> oVirt Code of Conduct: >>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>>> List Archives: >>>>>>>>> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/JTIBSBDL5IY... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Didi >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Didi >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Didi >>>>>> _______________________________________________ >>>>>> Infra mailing list -- infra(a)ovirt.org >>>>>> <mailto:infra@ovirt.org> >>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org >>>>>> <mailto:infra-leave@ovirt.org> >>>>>> Privacy Statement: >>>>>> https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: >>>>>> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SYTTA36U7YH... >>>> >>>> >>>> >>>> -- >>>> Didi >>>> >> >> >> -- >> Didi > _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/JDMSYCUEWVW...