new host addition to OVN cluster fails with Connectivity check failed, rolling back
Hello, I have a a oVirt 4.3 installation with two clusters. One of the cluster has switch type as OVS. I'm trying to add a second host to this cluster. I did a clean install of the OS, configured bond0 and bond0.2306 as the VLAN interface. I was able to add the host to the cluster. When I go to setup networks and drag ovirtmgmt onto bond0, I notice in the host that it was able to create the vdsm and br_int bridges, create ovirtmgmt interface and assign the IP on top of it. However I also notice that the bond0.2306 interface also exists with the ip assigned. Then it rolls back the config removing the bridges. I checked the supervdsm log and see that it's rolling back because "connectivity::48::root::(check) Connectivity check failed, rolling back" I'm pasting the relevant lines from supervdsm below MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:53,381::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return network_caps with {'bridges': {}, 'bondings': {'bond0': {'ipv4addrs': [], 'active_slave': '', 'ad_aggregator_id': '1', 'netmask': '', 'ad_partner_mac': '44:38:39:ff:01:33', 'hwaddr': '7c:d3:0a:60:e9:48', 'speed': 20000, 'gateway': '', 'ipv6autoconf': True, 'addr': '', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '9000', 'dhcpv4': False, 'switch': 'legacy', 'ipv4defaultroute': False, 'slaves': ['eno1', 'eno2'], 'ipv6gateway': '::', 'opts': {'mode': '4'}}}, 'nameservers': ['10.222.0.6', '10.333.0.6'], 'nics': {'eno1': {'permhwaddr': '7c:d3:0a:60:e9:48', 'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'ad_aggregator_id': '1', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:48', 'mtu': '9000', 'ipv6gateway': '::', 'gateway': ''}, 'eno2': {'permhwaddr': '7c:d3:0a:60:e9:49', 'ipv6autoco nf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'ad_aggregator_id': '1', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:48', 'mtu': '9000', 'ipv6gateway': '::', 'gateway': ''}, 'eno3': {'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:4a', 'ipv6gateway': '::', 'gateway': ''}, 'eno4': {'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:4b', 'ipv6gateway': '::', 'gateway': ''}, 'enp0s20f0u1u6': {'ipv6autoconf': True, 'addr': '', 'speed': 0, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7e:d3:0a:60:e9:4f', 'ipv6gateway': ' ::', 'gateway': ''}}, 'supportsIPv6': True, 'vlans': {'bond0.2306': {'iface': 'bond0', 'ipv6autoconf': True, 'addr': '10.119.6.237', 'dhcpv6': False, 'ipv6addrs': [], 'vlanid': 2306, 'mtu': '9000', 'dhcpv4': False, 'netmask': '255.255.255.0', 'ipv4defaultroute': True, 'ipv4addrs': ['10.119.6.237/24'], 'ipv6gateway': '::', 'gateway': '10.119.6.1'}}, 'networks': {}} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,243::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_pti with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,243::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_pti with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_retp with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_retp with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_ibrs with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_ibrs with 1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_ssbd with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_ssbd with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,246::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call check_qemu_conf_contains with ('vnc_tls', '1') {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,250::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return check_qemu_conf_contains with True MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,304::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call setupNetworks with ({u'ovirtmgmt': {u'ipv6autoconf': True, u'vlan': u'2306', u'ipaddr': u'10.119.6.237', u'switch': u'ovs', u'mtu': 9000, u'bonding': u'bond0', u'dhcpv6': False, u'STP': u'no', u'bridged': u'true', u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True}}, {}, {u'connectivityCheck': u'true', u'connectivityTimeout': 120, u'commitOnSuccess': False}) {} MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,305::api::211::root::(setupNetworks) Setting up network according to configuration: networks:{u'ovirtmgmt': {u'ipv6autoconf': True, u'vlan': u'2306', u'ipaddr': u'10.119.6.237', u'bonding': u'bond0', u'mtu': 9000, u'switch': u'ovs', u'dhcpv6': False, u'STP': u'no', u'bridged': u'true', u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True}}, bondings:{}, options:{u'connectivityCheck': u'true', u'connectivityTimeout': 120, u'commitOnSuccess': False} MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,323::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,342::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,408::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,408::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,445::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,459::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': ['eno1', 'eno2'], 'switch': 'legacy', 'options': 'mode=4'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,460::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({u'ipv6autoconf': True, 'nameservers': ['10.222.0.6', '10.333.0.6'], u'vlan': 2306, u'ipaddr': u'10.119.6.237', u'switch': u'ovs', u'mtu': 9000, u'bonding': u'bond0', u'dhcpv6': False, 'stp': False, u'bridged': True, u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True, 'bootproto': 'none'}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,462::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,909::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,909::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err= MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,911::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,911::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,947::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,953::configurator::265::root::(_remove_networks) Removing networks: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,955::setup::41::root::(remove_bonds) Removing bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,958::ifacquire::70::root::(acquire) Acquiring ifaces: set([u'eno1', u'eno2', u'bond0']) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,959::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:56,606::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:56,609::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:57,268::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:57,271::cmdutils::133::root::(exec_cmd) /sbin/ifdown bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,940::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,940::setup::61::root::(edit_bonds) Editing bonds: [u'bond0'] MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:58,946::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': [], 'switch': u'ovs', 'options': {u'mode': '4'}}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:58,949::sysfs_driver::104::root::(set_options) Bond bond0 options set: {u'mode': u'4'}. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,950::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 down (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,956::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,000::sysfs_driver::78::root::(add_slaves) Slave eno1 has been added to bond bond0. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,001::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 down (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,009::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,053::sysfs_driver::78::root::(add_slaves) Slave eno2 has been added to bond bond0. MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,053::netconfpersistence::69::root::(setBonding) Adding bond0({u'hwaddr': u'7c:d3:0a:60:e9:48', u'nics': [u'eno1', u'eno2'], u'switch': u'ovs', u'options': u'mode=4'}) netlink/events::DEBUG::2022-04-21 11:26:59,055::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d610>>, args=(), kwargs={}) netlink/events::DEBUG::2022-04-21 11:26:59,057::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e9550>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,058::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev bond0 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,067::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,071::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,073::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e91d0>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,074::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,081::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,083::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,084::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e91d0>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,085::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,090::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,092::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,093::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,219::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev eno1 scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,225::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,340::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev eno2 scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::setup::92::root::(add_bonds) Creating bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::configurator::274::root::(_add_networks) Adding networks: [u'ovirtmgmt'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,347::ifacquire::70::root::(acquire) Acquiring ifaces: set([u'bond0']) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,348::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,004::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,006::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,657::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,660::cmdutils::133::root::(exec_cmd) /sbin/ifdown bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,360::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,361::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- add-br vdsmbr_TtLDFxN1 -- set Bridge vdsmbr_TtLDFxN1 other-config:hwaddr="7a:b2:a6:5b:96:78" -- add-port vdsmbr_TtLDFxN1 bond0 -- set Port bond0 other_config:vdsm_level=southbound -- add-port vdsmbr_TtLDFxN1 ovirtmgmt -- set Port ovirtmgmt other_config:vdsm_level=northbound -- set Interface ovirtmgmt type=internal -- set Port ovirtmgmt tag=2306 -- set Interface ovirtmgmt mtu_request=9000 -- set Interface bond0 mtu_request=9000 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,361::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- add-br vdsmbr_TtLDFxN1 -- set Bridge vdsmbr_TtLDFxN1 'other-config:hwaddr="7a:b2:a6:5b:96:78"' -- add-port vdsmbr_TtLDFxN1 bond0 -- set Port bond0 other_config:vdsm_level=southbound -- add-port vdsmbr_TtLDFxN1 ovirtmgmt -- set Port ovirtmgmt other_config:vdsm_level=northbound -- set Interface ovirtmgmt type=internal -- set Port ovirtmgmt tag=2306 -- set Interface ovirtmgmt mtu_request=9000 -- set Interface bond0 mtu_request=9000 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,704::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:27:02,705::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({u'ipv6autoconf': True, u'nameservers': [u'10.129.0.60', u'10.229.0.60'], u'vlan': 2306, u'ipaddr': u'10.119.6.237', u'bonding': u'bond0', u'mtu': 9000, u'switch': u'ovs', u'dhcpv6': False, u'stp': False, u'bridged': True, u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True, u'bootproto': u'none'}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,705::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,706::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,745::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,753::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . external-ids:ovn-bridge-mappings="ovirtmgmt:vdsmbr_TtLDFxN1" MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,753::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . 'external-ids:ovn-bridge-mappings="ovirtmgmt:vdsmbr_TtLDFxN1"' (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,794::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,796::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,797::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev ovirtmgmt up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,805::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,807::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,808::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,809::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev bond0 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,816::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,819::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,820::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,821::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,867::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,870::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,871::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,872::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,920::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,922::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,923::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev ovirtmgmt scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,930::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,931::cmdutils::133::root::(exec_cmd) /sbin/ip -4 addr add dev ovirtmgmt 10.119.6.237/255.255.255.0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,938::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,938::cmdutils::133::root::(exec_cmd) /sbin/ip -4 route add default via 10.119.6.1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,945::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,947::connectivity::46::root::(check) Checking connectivity... MainProcess|jsonrpc/3::DEBUG::2022-04-21 11:27:10,538::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call ksmTune with ({u'run': 0, u'merge_across_nodes': 1},) {} MainProcess|jsonrpc/3::DEBUG::2022-04-21 11:27:10,538::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return ksmTune with None MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:04,075::connectivity::48::root::(check) Connectivity check failed, rolling back MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,075::logutils::319::root::(_report_stats) ThreadedHandler is ok in the last 136 seconds (max pending: 3) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,075::ifacquire::76::root::(_rollback) Acquiring transaction failed, reverting ifaces: [u'eno1', u'eno2', u'bond0'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,077::cmdutils::133::root::(exec_cmd) /sbin/ifup eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,485::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,488::cmdutils::133::root::(exec_cmd) /sbin/ifup eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,897::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,900::cmdutils::133::root::(exec_cmd) /sbin/ifup bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,045::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = './network-functions: line 119: .: /etc/sysconfig/network-scripts/: is a directory\n'; <rc> = 0 MainProcess|jsonrpc/2::WARNING::2022-04-21 11:29:06,046::netconfpersistence::294::root::(__exit__) Failed setup transaction,reverting to last known good network. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 249, in _setup_ovs connectivity.check(options) File "/usr/lib/python2.7/site-packages/vdsm/network/connectivity.py", line 50, in check 'connectivity check failed') ConfigNetworkError: (10, 'connectivity check failed') MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,047::api::211::root::(setupNetworks) Setting up network according to configuration: networks:{u'ovirtmgmt': {'remove': True}}, bondings:{u'bond0': {'remove': True}}, options:{'connectivityCheck': 0, 'inRollback': True} MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,061::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,079::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,153::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,153::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,190::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,326::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({'bonding': u'bond0', 'ipv6autoconf': True, 'nameservers': [], 'vlan': 2306, 'ipaddr': '10.119.6.237', 'switch': 'ovs', 'mtu': 9000, 'netmask': '255.255.255.0', 'dhcpv6': False, 'stp': False, 'bridged': True, 'defaultRoute': False, 'bootproto': 'none'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,327::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': ['eno1', 'eno2'], 'switch': 'legacy', 'options': 'mode=4'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,327::netconfpersistence::63::root::(removeNetwork) Removing network ovirtmgmt MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,334::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,911::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,911::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err= MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,918::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,918::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,956::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,091::configurator::265::root::(_remove_networks) Removing networks: [u'ovirtmgmt'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,219::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set Interface bond0 mtu_request=1500 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,220::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set Interface bond0 mtu_request=1500 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,425::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,426::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- del-port ovirtmgmt -- del-port bond0 -- del-br vdsmbr_TtLDFxN1 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,426::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- del-port ovirtmgmt -- del-port bond0 -- del-br vdsmbr_TtLDFxN1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,949::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,950::netconfpersistence::65::root::(removeNetwork) Network ovirtmgmt not found for removal MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,951::setup::41::root::(remove_bonds) Removing bonds: [u'bond0'] MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,165::sysfs_driver::69::root::(destroy) Bond bond0 has been destroyed. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,166::netconfpersistence::76::root::(removeBonding) bond0 not found for removal MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::ifacquire::70::root::(acquire) Acquiring ifaces: set([]) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::setup::61::root::(edit_bonds) Editing bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::setup::92::root::(add_bonds) Creating bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::configurator::274::root::(_add_networks) Adding networks: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::ifacquire::70::root::(acquire) Acquiring ifaces: set([]) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,173::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,173::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,211::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,217::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . external-ids:ovn-bridge-mappings="" MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,217::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . 'external-ids:ovn-bridge-mappings=""' (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,247::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,249::netconfpersistence::231::root::(_clearDisk) Clearing netconf: /var/lib/vdsm/staging/netconf MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,253::netconfpersistence::181::root::(save) Saved new config RunningConfig({}, {}, {}) to [/var/lib/vdsm/staging/netconf/nets,/var/lib/vdsm/staging/netconf/bonds,/var/lib/vdsm/staging/netconf/devices] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,256::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/after_network_setup/30_ethtool_options (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,434::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,434::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/after_network_setup/30_ethtool_options: rc=0 err= MainProcess|jsonrpc/2::ERROR::2022-04-21 11:29:08,436::supervdsm_server::103::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 101, in wrapper res = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 228, in setupNetworks _setup_networks(networks, bondings, options, net_info) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 249, in _setup_networks networks, bondings, options, net_info, in_rollback) File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__ self.gen.throw(type, value, traceback) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 157, in _rollback six.reraise(excType, value, tb) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 140, in _rollback yield File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 249, in _setup_networks networks, bondings, options, net_info, in_rollback) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 139, in setup _setup(networks, bondings, options, in_rollback, net_info) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 159, in _setup _setup_ovs(ovs_nets, ovs_bonds, options, net_info, in_rollback) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 249, in _setup_ovs connectivity.check(options) File "/usr/lib/python2.7/site-packages/vdsm/network/netconfpersistence.py", line 295, in __exit__ raise ne.RollbackIncomplete(config_diff, ex_type, ex_value) ConfigNetworkError: (10, 'connectivity check failed') MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,062::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call network_caps with () {} MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,062::logutils::319::root::(_report_stats) ThreadedHandler is ok in the last 1935 seconds (max pending: 3) MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,076::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,098::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 Regards, Ravi
Hi Ravi, I'm sorry to repeat myself but oVirt 4.3 went EOL 2 years ago and now even 4.4 went EOL 2 days ago. As a project we don't consider reports against 4.3 anymore. Please upgrade as soon as practical and if it still reproduces please let us know and we'll be happy to help getting it fixed. You can try contacting Oracle or Red Hat support as they still provide help for 4.3 (well, for Red Hat Virtualization 4.3 Maintenance support ends in 4 months <https://access.redhat.com/support/policy/updates/rhev> too..., couldn't find any public statement for OLVM on Oracle website). Il giorno gio 21 apr 2022 alle ore 23:15 ravi k <kottapar@gmail.com> ha scritto:
Hello,
I have a a oVirt 4.3 installation with two clusters. One of the cluster has switch type as OVS. I'm trying to add a second host to this cluster. I did a clean install of the OS, configured bond0 and bond0.2306 as the VLAN interface. I was able to add the host to the cluster. When I go to setup networks and drag ovirtmgmt onto bond0, I notice in the host that it was able to create the vdsm and br_int bridges, create ovirtmgmt interface and assign the IP on top of it. However I also notice that the bond0.2306 interface also exists with the ip assigned. Then it rolls back the config removing the bridges. I checked the supervdsm log and see that it's rolling back because "connectivity::48::root::(check) Connectivity check failed, rolling back"
I'm pasting the relevant lines from supervdsm below
MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:53,381::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return network_caps with {'bridges': {}, 'bondings': {'bond0': {'ipv4addrs': [], 'active_slave': '', 'ad_aggregator_id': '1', 'netmask': '', 'ad_partner_mac': '44:38:39:ff:01:33', 'hwaddr': '7c:d3:0a:60:e9:48', 'speed': 20000, 'gateway': '', 'ipv6autoconf': True, 'addr': '', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '9000', 'dhcpv4': False, 'switch': 'legacy', 'ipv4defaultroute': False, 'slaves': ['eno1', 'eno2'], 'ipv6gateway': '::', 'opts': {'mode': '4'}}}, 'nameservers': ['10.222.0.6', '10.333.0.6'], 'nics': {'eno1': {'permhwaddr': '7c:d3:0a:60:e9:48', 'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'ad_aggregator_id': '1', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:48', 'mtu': '9000', 'ipv6gateway': '::', 'gateway': ''}, 'eno2': {'permhwaddr': '7c:d3:0a:60:e9:49', 'ipv6autoco nf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'ad_aggregator_id': '1', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:48', 'mtu': '9000', 'ipv6gateway': '::', 'gateway': ''}, 'eno3': {'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:4a', 'ipv6gateway': '::', 'gateway': ''}, 'eno4': {'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:4b', 'ipv6gateway': '::', 'gateway': ''}, 'enp0s20f0u1u6': {'ipv6autoconf': True, 'addr': '', 'speed': 0, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7e:d3:0a:60:e9:4f', 'ipv6gateway': ' ::', 'gateway': ''}}, 'supportsIPv6': True, 'vlans': {'bond0.2306': {'iface': 'bond0', 'ipv6autoconf': True, 'addr': '10.119.6.237', 'dhcpv6': False, 'ipv6addrs': [], 'vlanid': 2306, 'mtu': '9000', 'dhcpv4': False, 'netmask': '255.255.255.0', 'ipv4defaultroute': True, 'ipv4addrs': [' 10.119.6.237/24'], 'ipv6gateway': '::', 'gateway': '10.119.6.1'}}, 'networks': {}} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,243::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_pti with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,243::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_pti with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_retp with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_retp with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_ibrs with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_ibrs with 1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_ssbd with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_ssbd with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,246::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call check_qemu_conf_contains with ('vnc_tls', '1') {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,250::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return check_qemu_conf_contains with True MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,304::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call setupNetworks with ({u'ovirtmgmt': {u'ipv6autoconf': True, u'vlan': u'2306', u'ipaddr': u'10.119.6.237', u'switch': u'ovs', u'mtu': 9000, u'bonding': u'bond0', u'dhcpv6': False, u'STP': u'no', u'bridged': u'true', u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True}}, {}, {u'connectivityCheck': u'true', u'connectivityTimeout': 120, u'commitOnSuccess': False}) {} MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,305::api::211::root::(setupNetworks) Setting up network according to configuration: networks:{u'ovirtmgmt': {u'ipv6autoconf': True, u'vlan': u'2306', u'ipaddr': u'10.119.6.237', u'bonding': u'bond0', u'mtu': 9000, u'switch': u'ovs', u'dhcpv6': False, u'STP': u'no', u'bridged': u'true', u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True}}, bondings:{}, options:{u'connectivityCheck': u'true', u'connectivityTimeout': 120, u'commitOnSuccess': False} MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,323::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,342::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,408::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,408::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,445::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,459::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': ['eno1', 'eno2'], 'switch': 'legacy', 'options': 'mode=4'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,460::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({u'ipv6autoconf': True, 'nameservers': ['10.222.0.6', '10.333.0.6'], u'vlan': 2306, u'ipaddr': u'10.119.6.237', u'switch': u'ovs', u'mtu': 9000, u'bonding': u'bond0', u'dhcpv6': False, 'stp': False, u'bridged': True, u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True, 'bootproto': 'none'}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,462::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,909::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,909::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err= MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,911::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,911::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,947::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,953::configurator::265::root::(_remove_networks) Removing networks: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,955::setup::41::root::(remove_bonds) Removing bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,958::ifacquire::70::root::(acquire) Acquiring ifaces: set([u'eno1', u'eno2', u'bond0']) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,959::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:56,606::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:56,609::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:57,268::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:57,271::cmdutils::133::root::(exec_cmd) /sbin/ifdown bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,940::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,940::setup::61::root::(edit_bonds) Editing bonds: [u'bond0'] MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:58,946::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': [], 'switch': u'ovs', 'options': {u'mode': '4'}}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:58,949::sysfs_driver::104::root::(set_options) Bond bond0 options set: {u'mode': u'4'}. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,950::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 down (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,956::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,000::sysfs_driver::78::root::(add_slaves) Slave eno1 has been added to bond bond0. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,001::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 down (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,009::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,053::sysfs_driver::78::root::(add_slaves) Slave eno2 has been added to bond bond0. MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,053::netconfpersistence::69::root::(setBonding) Adding bond0({u'hwaddr': u'7c:d3:0a:60:e9:48', u'nics': [u'eno1', u'eno2'], u'switch': u'ovs', u'options': u'mode=4'}) netlink/events::DEBUG::2022-04-21 11:26:59,055::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d610>>, args=(), kwargs={}) netlink/events::DEBUG::2022-04-21 11:26:59,057::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e9550>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,058::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev bond0 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,067::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,071::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,073::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e91d0>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,074::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,081::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,083::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,084::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e91d0>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,085::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,090::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,092::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,093::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,219::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev eno1 scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,225::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,340::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev eno2 scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::setup::92::root::(add_bonds) Creating bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::configurator::274::root::(_add_networks) Adding networks: [u'ovirtmgmt'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,347::ifacquire::70::root::(acquire) Acquiring ifaces: set([u'bond0']) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,348::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,004::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,006::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,657::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,660::cmdutils::133::root::(exec_cmd) /sbin/ifdown bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,360::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,361::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- add-br vdsmbr_TtLDFxN1 -- set Bridge vdsmbr_TtLDFxN1 other-config:hwaddr="7a:b2:a6:5b:96:78" -- add-port vdsmbr_TtLDFxN1 bond0 -- set Port bond0 other_config:vdsm_level=southbound -- add-port vdsmbr_TtLDFxN1 ovirtmgmt -- set Port ovirtmgmt other_config:vdsm_level=northbound -- set Interface ovirtmgmt type=internal -- set Port ovirtmgmt tag=2306 -- set Interface ovirtmgmt mtu_request=9000 -- set Interface bond0 mtu_request=9000 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,361::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- add-br vdsmbr_TtLDFxN1 -- set Bridge vdsmbr_TtLDFxN1 'other-config:hwaddr="7a:b2:a6:5b:96:78"' -- add-port vdsmbr_TtLDFxN1 bond0 -- set Port bond0 other_config:vdsm_level=southbound -- add-port vdsmbr_TtLDFxN1 ovirtmgmt -- set Port ovirtmgmt other_config:vdsm_level=northbound -- set Interface ovirtmgmt type=internal -- set Port ovirtmgmt tag=2306 -- set Interface ovirtmgmt mtu_request=9000 -- set Interface bond0 mtu_request=9000 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,704::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:27:02,705::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({u'ipv6autoconf': True, u'nameservers': [u'10.129.0.60', u'10.229.0.60'], u'vlan': 2306, u'ipaddr': u'10.119.6.237', u'bonding': u'bond0', u'mtu': 9000, u'switch': u'ovs', u'dhcpv6': False, u'stp': False, u'bridged': True, u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True, u'bootproto': u'none'}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,705::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,706::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,745::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,753::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . external-ids:ovn-bridge-mappings="ovirtmgmt:vdsmbr_TtLDFxN1" MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,753::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . 'external-ids:ovn-bridge-mappings="ovirtmgmt:vdsmbr_TtLDFxN1"' (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,794::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,796::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,797::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev ovirtmgmt up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,805::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,807::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,808::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,809::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev bond0 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,816::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,819::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,820::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,821::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,867::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,870::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,871::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,872::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,920::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,922::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,923::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev ovirtmgmt scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,930::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,931::cmdutils::133::root::(exec_cmd) /sbin/ip -4 addr add dev ovirtmgmt 10.119.6.237/255.255.255.0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,938::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,938::cmdutils::133::root::(exec_cmd) /sbin/ip -4 route add default via 10.119.6.1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,945::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,947::connectivity::46::root::(check) Checking connectivity... MainProcess|jsonrpc/3::DEBUG::2022-04-21 11:27:10,538::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call ksmTune with ({u'run': 0, u'merge_across_nodes': 1},) {} MainProcess|jsonrpc/3::DEBUG::2022-04-21 11:27:10,538::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return ksmTune with None MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:04,075::connectivity::48::root::(check) Connectivity check failed, rolling back MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,075::logutils::319::root::(_report_stats) ThreadedHandler is ok in the last 136 seconds (max pending: 3) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,075::ifacquire::76::root::(_rollback) Acquiring transaction failed, reverting ifaces: [u'eno1', u'eno2', u'bond0'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,077::cmdutils::133::root::(exec_cmd) /sbin/ifup eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,485::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,488::cmdutils::133::root::(exec_cmd) /sbin/ifup eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,897::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,900::cmdutils::133::root::(exec_cmd) /sbin/ifup bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,045::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = './network-functions: line 119: .: /etc/sysconfig/network-scripts/: is a directory\n'; <rc> = 0 MainProcess|jsonrpc/2::WARNING::2022-04-21 11:29:06,046::netconfpersistence::294::root::(__exit__) Failed setup transaction,reverting to last known good network. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 249, in _setup_ovs connectivity.check(options) File "/usr/lib/python2.7/site-packages/vdsm/network/connectivity.py", line 50, in check 'connectivity check failed') ConfigNetworkError: (10, 'connectivity check failed') MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,047::api::211::root::(setupNetworks) Setting up network according to configuration: networks:{u'ovirtmgmt': {'remove': True}}, bondings:{u'bond0': {'remove': True}}, options:{'connectivityCheck': 0, 'inRollback': True} MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,061::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,079::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,153::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,153::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,190::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,326::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({'bonding': u'bond0', 'ipv6autoconf': True, 'nameservers': [], 'vlan': 2306, 'ipaddr': '10.119.6.237', 'switch': 'ovs', 'mtu': 9000, 'netmask': '255.255.255.0', 'dhcpv6': False, 'stp': False, 'bridged': True, 'defaultRoute': False, 'bootproto': 'none'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,327::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': ['eno1', 'eno2'], 'switch': 'legacy', 'options': 'mode=4'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,327::netconfpersistence::63::root::(removeNetwork) Removing network ovirtmgmt MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,334::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,911::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,911::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err= MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,918::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,918::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,956::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,091::configurator::265::root::(_remove_networks) Removing networks: [u'ovirtmgmt'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,219::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set Interface bond0 mtu_request=1500 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,220::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set Interface bond0 mtu_request=1500 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,425::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,426::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- del-port ovirtmgmt -- del-port bond0 -- del-br vdsmbr_TtLDFxN1 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,426::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- del-port ovirtmgmt -- del-port bond0 -- del-br vdsmbr_TtLDFxN1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,949::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,950::netconfpersistence::65::root::(removeNetwork) Network ovirtmgmt not found for removal MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,951::setup::41::root::(remove_bonds) Removing bonds: [u'bond0'] MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,165::sysfs_driver::69::root::(destroy) Bond bond0 has been destroyed. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,166::netconfpersistence::76::root::(removeBonding) bond0 not found for removal MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::ifacquire::70::root::(acquire) Acquiring ifaces: set([]) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::setup::61::root::(edit_bonds) Editing bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::setup::92::root::(add_bonds) Creating bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::configurator::274::root::(_add_networks) Adding networks: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::ifacquire::70::root::(acquire) Acquiring ifaces: set([]) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,173::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,173::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,211::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,217::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . external-ids:ovn-bridge-mappings="" MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,217::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . 'external-ids:ovn-bridge-mappings=""' (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,247::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,249::netconfpersistence::231::root::(_clearDisk) Clearing netconf: /var/lib/vdsm/staging/netconf MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,253::netconfpersistence::181::root::(save) Saved new config RunningConfig({}, {}, {}) to [/var/lib/vdsm/staging/netconf/nets,/var/lib/vdsm/staging/netconf/bonds,/var/lib/vdsm/staging/netconf/devices] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,256::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/after_network_setup/30_ethtool_options (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,434::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,434::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/after_network_setup/30_ethtool_options: rc=0 err= MainProcess|jsonrpc/2::ERROR::2022-04-21 11:29:08,436::supervdsm_server::103::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 101, in wrapper res = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 228, in setupNetworks _setup_networks(networks, bondings, options, net_info) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 249, in _setup_networks networks, bondings, options, net_info, in_rollback) File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__ self.gen.throw(type, value, traceback) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 157, in _rollback six.reraise(excType, value, tb) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 140, in _rollback yield File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 249, in _setup_networks networks, bondings, options, net_info, in_rollback) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 139, in setup _setup(networks, bondings, options, in_rollback, net_info) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 159, in _setup _setup_ovs(ovs_nets, ovs_bonds, options, net_info, in_rollback) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 249, in _setup_ovs connectivity.check(options) File "/usr/lib/python2.7/site-packages/vdsm/network/netconfpersistence.py", line 295, in __exit__ raise ne.RollbackIncomplete(config_diff, ex_type, ex_value) ConfigNetworkError: (10, 'connectivity check failed') MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,062::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call network_caps with () {} MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,062::logutils::319::root::(_report_stats) ThreadedHandler is ok in the last 1935 seconds (max pending: 3) MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,076::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,098::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0
Regards, Ravi _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/TBU63MQJDVKGL5...
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
Hi Sandro, Thanks for that. I remember you mentioned this in another post too :) I posted about this as there might be some users using 4.3 and might pitch in with some ideas. We're starting out with a greenfield setup. Would you suggest that we rather go with 4.4 directly? Regards, Ravi
Il giorno mar 26 apr 2022 alle ore 07:33 ravi k <kottapar@gmail.com> ha scritto:
Hi Sandro, Thanks for that. I remember you mentioned this in another post too :) I posted about this as there might be some users using 4.3 and might pitch in with some ideas. We're starting out with a greenfield setup. Would you suggest that we rather go with 4.4 directly?
I would suggest to rather go with 4.5 directly.
Regards, Ravi _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LO42OOBSIL5W5B...
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
What's surprising is we're seeing the same issue in 4.4.8 as well. Sadly even this is also EOL now.
participants (2)
-
ravi k -
Sandro Bonazzola