[ovirt-devel] Re: [ OST Failure Report ] [ oVirt master (ovirt-engine) ] [ 22-03-2019 ] [004_basic_sanity.hotplug_cpu ]

Il giorno mer 27 mar 2019 alle ore 09:54 Dominik Holler < dholler(a)redhat.com&gt; ha scritto: > On Wed, 27 Mar 2019 10:07:16 +0200 > Eyal Edri <eedri(a)redhat.com&gt; wrote: > > > On Wed, Mar 27, 2019 at 3:06 AM Ryan Barry <rbarry(a)redhat.com&gt; wrote: > > > > > On Tue, Mar 26, 2019 at 4:07 PM Dominik Holler <dholler(a)redhat.com&gt; > wrote: > > > > > > > > I added in > > > > https://gerrit.ovirt.org/#/c/98925/ > > > > a ping directly before the ssh. > > > > The ping succeeds, but the ssh fails. > > > > > > > > > > > > On Tue, 26 Mar 2019 17:07:45 +0100 > > > > Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: > > > > > > > > > Il giorno mar 26 mar 2019 alle ore 16:48 Ryan Barry < > rbarry(a)redhat.com&gt; > > > ha > > > > > scritto: > > > > > > > > > > > +1 from me > > > > > > > > > > > > > > > > Merged. I have 2 patches constantly failing on it, rebased them, > you > > > can > > > > > follow on: > > > > > https://gerrit.ovirt.org/#/c/98863/ and > https://gerrit.ovirt.org/98862 > > > > > > > > > > > > > still failing on jenkins, but at least one succeeds locally for me > > > > > > Succeeds locally for me also. > > > > > > Dafna, are we sure there's not an infra issue? > > > > > > > I think since its a race ( and we've seen failures on this test in the > > past, also a race I think ), its probably hard to reproduce locally. > > Also, we probably need to make sure the same Libvirt version is used. > > The upstream servers are quite old, it can also be local run ends up > being > > faster and not hitting the same issues ( as we've seen in the past ) > > > > Could it be a bug in the ssh client ( paramiko? ) > > > > > Probably wrong idea, but worth to ask: > Any ideas which ssh_timeout is used or how to modify? > > If 100 tries including a time.sleep(1) takes 100 seconds, > either the timeout is not the expected 10 seconds, or the guest refuses > the connection. > > I'm looking into a similar failure and found this on host1 logs at the time of the ssh failure: https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/390... Mar 27 03:35:20 lago-basic-suite-master-host-1 kernel: on65322a61b5f14: port 2(vnet1) entered blocking state Mar 27 03:35:20 lago-basic-suite-master-host-1 kernel: on65322a61b5f14: port 2(vnet1) entered disabled state Mar 27 03:35:20 lago-basic-suite-master-host-1 kernel: device vnet1 entered promiscuous mode Mar 27 03:35:20 lago-basic-suite-master-host-1 kernel: on65322a61b5f14: port 2(vnet1) entered blocking state Mar 27 03:35:20 lago-basic-suite-master-host-1 kernel: on65322a61b5f14: port 2(vnet1) entered forwarding state Mar 27 03:35:20 lago-basic-suite-master-host-1 NetworkManager[2667]: <info> [1553672120.9133] manager: (vnet1): new Tun device (/org/freedesktop/NetworkManager/Devices/44) Mar 27 03:35:20 lago-basic-suite-master-host-1 lldpad: recvfrom(Event interface): No buffer space available Mar 27 03:35:20 lago-basic-suite-master-host-1 NetworkManager[2667]: <info> [1553672120.9277] device (vnet1): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external') Mar 27 03:35:20 lago-basic-suite-master-host-1 NetworkManager[2667]: <info> [1553672120.9297] device (vnet1): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'external') Mar 27 03:35:20 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet1 -j libvirt-J-vnet1' failed: Illegal target name 'libvirt-J-vnet1'. Mar 27 03:35:20 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet1 -j libvirt-P-vnet1' failed: Illegal target name 'libvirt-P-vnet1'. Mar 27 03:35:20 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-J-vnet1' failed: Chain 'libvirt-J-vnet1' doesn't exist. Mar 27 03:35:20 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-P-vnet1' failed: Chain 'libvirt-P-vnet1' doesn't exist. Mar 27 03:35:20 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-J-vnet1' failed: Chain 'libvirt-J-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-J-vnet1' failed: Chain 'libvirt-J-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-P-vnet1' failed: Chain 'libvirt-P-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-P-vnet1' failed: Chain 'libvirt-P-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F J-vnet1-mac' failed: Chain 'J-vnet1-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X J-vnet1-mac' failed: Chain 'J-vnet1-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F J-vnet1-arp-mac' failed: Chain 'J-vnet1-arp-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X J-vnet1-arp-mac' failed: Chain 'J-vnet1-arp-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet1 -g FO-vnet1' failed: iptables v1.4.21: goto 'FO-vnet1' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet1 -g FO-vnet1' failed: iptables v1.4.21: goto 'FO-vnet1' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet1 -g FI-vnet1' failed: iptables v1.4.21: goto 'FI-vnet1' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet1 -g HI-vnet1' failed: iptables v1.4.21: goto 'HI-vnet1' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FO-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FO-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FI-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FI-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HI-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HI-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -E FP-vnet1 FO-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -E FJ-vnet1 FI-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -E HJ-vnet1 HI-vnet1' failed: iptables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet1 -g FO-vnet1' failed: ip6tables v1.4.21: goto 'FO-vnet1' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet1 -g FO-vnet1' failed: ip6tables v1.4.21: goto 'FO-vnet1' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet1 -g FI-vnet1' failed: ip6tables v1.4.21: goto 'FI-vnet1' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet1 -g HI-vnet1' failed: ip6tables v1.4.21: goto 'HI-vnet1' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FO-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FO-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FI-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FI-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HI-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HI-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -E FP-vnet1 FO-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -E FJ-vnet1 FI-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -E HJ-vnet1 HI-vnet1' failed: ip6tables: No chain/target/match by that name. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet1 -j libvirt-I-vnet1' failed: Illegal target name 'libvirt-I-vnet1'. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet1 -j libvirt-O-vnet1' failed: Illegal target name 'libvirt-O-vnet1'. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-I-vnet1' failed: Chain 'libvirt-I-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-O-vnet1' failed: Chain 'libvirt-O-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-I-vnet1' failed: Chain 'libvirt-I-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-I-vnet1' failed: Chain 'libvirt-I-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-O-vnet1' failed: Chain 'libvirt-O-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-O-vnet1' failed: Chain 'libvirt-O-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-P-vnet1' failed: Chain 'libvirt-P-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -E libvirt-P-vnet1 libvirt-O-vnet1' failed: Chain 'libvirt-P-vnet1' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F I-vnet1-mac' failed: Chain 'I-vnet1-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X I-vnet1-mac' failed: Chain 'I-vnet1-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F I-vnet1-arp-mac' failed: Chain 'I-vnet1-arp-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 firewalld[2643]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X I-vnet1-arp-mac' failed: Chain 'I-vnet1-arp-mac' doesn't exist. Mar 27 03:35:21 lago-basic-suite-master-host-1 lldpad: recvfrom(Event interface): No buffer space available Mar 27 03:35:21 lago-basic-suite-master-host-1 lldpad: recvfrom(Event interface): No buffer space available Mar 27 03:35:28 lago-basic-suite-master-host-1 systemd: Created slice User Slice of root. On the lago log at corresponding time frame: https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/390... 2019-03-27 07:35:23,801::log_utils.py::end_log_task::670::nose::INFO:: # hotplug_nic: [32mSuccess [0m (in 0:00:05) 2019-03-27 07:35:23,801::log_utils.py::start_log_task::655::nose::INFO:: # hotplug_cpu: [0m [0m Around the same time on host 0: https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/390... Mar 27 03:33:06 lago-basic-suite-master-host-0 kernel: ovirtmgmt: port 2(vnet0) entered disabled state Mar 27 03:33:06 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:33:06 lago-basic-suite-master-host-0 NetworkManager[2671]: <info> [1553671986.5243] device (vnet0): state change: disconnected -> unmanaged (reason 'unmanaged', sys-iface-state: 'removed') Mar 27 03:33:06 lago-basic-suite-master-host-0 NetworkManager[2671]: <info> [1553671986.5251] device (vnet0): released from master device ovirtmgmt Mar 27 03:33:06 lago-basic-suite-master-host-0 lldpad: recvfrom(Event interface): No buffer space available Mar 27 03:33:06 lago-basic-suite-master-host-0 systemd-machined: Machine qemu-2-vm2 terminated. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vnet0' failed: iptables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FP-vnet0' failed: iptables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: iptables v1.4.21: goto 'FJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' failed: iptables v1.4.21: goto 'HJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FP-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FP-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vnet0' failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FP-vnet0' failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:06 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: ip6tables v1.4.21: goto 'FJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' failed: ip6tables v1.4.21: goto 'HJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FP-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FP-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-J-vnet0' failed: Illegal target name 'libvirt-J-vnet0'. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-P-vnet0' failed: Illegal target name 'libvirt-P-vnet0'. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: iptables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: iptables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FO-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FO-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: ip6tables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 systemd: Created slice User Slice of root. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: ip6tables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:07 lago-basic-suite-master-host-0 systemd: Started Session c453 of user root. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' failed: ip6tables: Bad rule (does a matching rule exist in that chain?). Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FO-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FO-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 multipathd: dm-13: remove map (uevent) Mar 27 03:33:07 lago-basic-suite-master-host-0 multipathd: dm-13: devmap not registered, can't remove Mar 27 03:33:07 lago-basic-suite-master-host-0 multipathd: dm-13: remove map (uevent) Mar 27 03:33:07 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:07 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:33:08 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-O-vnet0' failed: Illegal target name 'libvirt-O-vnet0'. Mar 27 03:33:08 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:33:08 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:33:08 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Created slice User Slice of root. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Started Session c454 of user root. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Created slice User Slice of root. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Started Session c455 of user root. Mar 27 03:33:08 lago-basic-suite-master-host-0 multipathd: dm-12: remove map (uevent) Mar 27 03:33:08 lago-basic-suite-master-host-0 multipathd: dm-12: devmap not registered, can't remove Mar 27 03:33:08 lago-basic-suite-master-host-0 multipathd: dm-12: remove map (uevent) Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Created slice User Slice of root. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Started Session c456 of user root. Mar 27 03:33:08 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:33:09 lago-basic-suite-master-host-0 vdsm[22587]: WARN File: /var/lib/libvirt/qemu/channels/fc3c11ae-4b16-4bab-8b5e-aa839b2a42f6.ovirt-guest-agent.0 already removed Mar 27 03:33:09 lago-basic-suite-master-host-0 vdsm[22587]: WARN File: /var/lib/libvirt/qemu/channels/fc3c11ae-4b16-4bab-8b5e-aa839b2a42f6.org.qemu.guest_agent.0 already removed Mar 27 03:33:16 lago-basic-suite-master-host-0 kernel: ovirtmgmt: port 2(vnet0) entered blocking state Mar 27 03:33:16 lago-basic-suite-master-host-0 kernel: ovirtmgmt: port 2(vnet0) entered disabled state Mar 27 03:33:16 lago-basic-suite-master-host-0 kernel: device vnet0 entered promiscuous mode Mar 27 03:33:16 lago-basic-suite-master-host-0 kernel: ovirtmgmt: port 2(vnet0) entered blocking state Mar 27 03:33:16 lago-basic-suite-master-host-0 kernel: ovirtmgmt: port 2(vnet0) entered forwarding state Mar 27 03:33:16 lago-basic-suite-master-host-0 lldpad: recvfrom(Event interface): No buffer space available Mar 27 03:33:16 lago-basic-suite-master-host-0 NetworkManager[2671]: <info> [1553671996.4807] manager: (vnet0): new Tun device (/org/freedesktop/NetworkManager/Devices/49) Mar 27 03:33:16 lago-basic-suite-master-host-0 NetworkManager[2671]: <info> [1553671996.4970] device (vnet0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external') Mar 27 03:33:16 lago-basic-suite-master-host-0 NetworkManager[2671]: <info> [1553671996.4997] device (vnet0): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'external') Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-J-vnet0' failed: Illegal target name 'libvirt-J-vnet0'. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-P-vnet0' failed: Illegal target name 'libvirt-P-vnet0'. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F J-vnet0-mac' failed: Chain 'J-vnet0-mac' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X J-vnet0-mac' failed: Chain 'J-vnet0-mac' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F J-vnet0-arp-mac' failed: Chain 'J-vnet0-arp-mac' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X J-vnet0-arp-mac' failed: Chain 'J-vnet0-arp-mac' doesn't exist. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: iptables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: iptables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FO-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FO-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -E FP-vnet0 FO-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:16 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -E FJ-vnet0 FI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -E HJ-vnet0 HI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: ip6tables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: ip6tables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FO-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FO-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -E FP-vnet0 FO-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -E FJ-vnet0 FI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -E HJ-vnet0 HI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-I-vnet0' failed: Illegal target name 'libvirt-I-vnet0'. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-O-vnet0' failed: Illegal target name 'libvirt-O-vnet0'. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-I-vnet0' failed: Chain 'libvirt-I-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-I-vnet0' failed: Chain 'libvirt-I-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-I-vnet0' failed: Chain 'libvirt-I-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -E libvirt-P-vnet0 libvirt-O-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F I-vnet0-mac' failed: Chain 'I-vnet0-mac' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X I-vnet0-mac' failed: Chain 'I-vnet0-mac' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F I-vnet0-arp-mac' failed: Chain 'I-vnet0-arp-mac' doesn't exist. Mar 27 03:33:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X I-vnet0-arp-mac' failed: Chain 'I-vnet0-arp-mac' doesn't exist. Mar 27 03:33:18 lago-basic-suite-master-host-0 systemd-machined: New machine qemu-3-vm2. Mar 27 03:33:18 lago-basic-suite-master-host-0 systemd: Started Virtual Machine qemu-3-vm2. Mar 27 03:33:18 lago-basic-suite-master-host-0 kvm: 1 guest now active Mar 27 03:33:28 lago-basic-suite-master-host-0 systemd: Created slice User Slice of root. Mar 27 03:33:28 lago-basic-suite-master-host-0 systemd: Started Session c457 of user root. Mar 27 03:33:28 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:33:29 lago-basic-suite-master-host-0 systemd: Created slice User Slice of root. Mar 27 03:33:29 lago-basic-suite-master-host-0 systemd: Started Session c458 of user root. Mar 27 03:33:29 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:33:35 lago-basic-suite-master-host-0 systemd: Created slice User Slice of root. Mar 27 03:33:35 lago-basic-suite-master-host-0 systemd: Started Session c459 of user root. Mar 27 03:33:36 lago-basic-suite-master-host-0 systemd: Removed slice User Slice of root. Mar 27 03:34:16 lago-basic-suite-master-host-0 kernel: ovirtmgmt: port 2(vnet0) entered disabled state Mar 27 03:34:16 lago-basic-suite-master-host-0 kernel: device vnet0 left promiscuous mode Mar 27 03:34:16 lago-basic-suite-master-host-0 kernel: ovirtmgmt: port 2(vnet0) entered disabled state Mar 27 03:34:16 lago-basic-suite-master-host-0 NetworkManager[2671]: <info> [1553672056.8430] device (vnet0): state change: disconnected -> unmanaged (reason 'unmanaged', sys-iface-state: 'removed') Mar 27 03:34:16 lago-basic-suite-master-host-0 NetworkManager[2671]: <info> [1553672056.8437] device (vnet0): released from master device ovirtmgmt Mar 27 03:34:16 lago-basic-suite-master-host-0 lldpad: recvfrom(Event interface): No buffer space available Mar 27 03:34:16 lago-basic-suite-master-host-0 kvm: 0 guests now active Mar 27 03:34:16 lago-basic-suite-master-host-0 systemd-machined: Machine qemu-3-vm2 terminated. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vnet0' failed: iptables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FP-vnet0' failed: iptables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: iptables v1.4.21: goto 'FJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' failed: iptables v1.4.21: goto 'HJ-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FP-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FP-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HJ-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FP-vnet0' failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FP-vnet0' failed: ip6tables v1.4.21: goto 'FP-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FJ-vnet0' failed: ip6tables v1.4.21: goto 'FJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HJ-vnet0' failed: ip6tables v1.4.21: goto 'HJ-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FP-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FP-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HJ-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i vnet0 -j libvirt-J-vnet0' failed: Illegal target name 'libvirt-J-vnet0'. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-P-vnet0' failed: Illegal target name 'libvirt-P-vnet0'. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-J-vnet0' failed: Chain 'libvirt-J-vnet0' doesn't exist. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-P-vnet0' failed: Chain 'libvirt-P-vnet0' doesn't exist. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: iptables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: iptables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: iptables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `iptables -h' or 'iptables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?). Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FO-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FO-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F FI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X FI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -F HI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -w -X HI-vnet0' failed: iptables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0' failed: ip6tables v1.4.21: goto 'FO-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0' failed: ip6tables v1.4.21: goto 'FI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0' failed: ip6tables v1.4.21: goto 'HI-vnet0' is not a chain#012#012Try `ip6tables -h' or 'ip6tables --help' for more information. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT' failed: ip6tables: Bad rule (does a matching rule exist in that chain?). Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FO-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FO-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F FI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X FI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -F HI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ip6tables -w2 -w -X HI-vnet0' failed: ip6tables: No chain/target/match by that name. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o vnet0 -j libvirt-O-vnet0' failed: Illegal target name 'libvirt-O-vnet0'. Mar 27 03:34:17 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -L libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:34:18 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -F libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:34:18 lago-basic-suite-master-host-0 firewalld[2651]: WARNING: COMMAND_FAILED: '/usr/sbin/ebtables --concurrent -t nat -X libvirt-O-vnet0' failed: Chain 'libvirt-O-vnet0' doesn't exist. Mar 27 03:34:18 lago-basic-suite-master-host-0 vdsm[22587]: WARN executor state: count=5 workers=set([<Worker name=periodic/0 waiting task#=207 at 0x7f8fac0b5310>, <Worker name=periodic/3 waiting task#=194 at 0x7f8fac0b5b10>, <Worker name=periodic/4 waiting task#=0 at 0x7f8f5c412190>, <Worker name=periodic/1 waiting task#=200 at 0x7f8fac0b5550>, <Worker name=periodic/2 running <Task discardable <Operation action=<VmDispatcher operation=<class 'vdsm.virt.periodic.DriveWatermarkMonitor'> at 0x7f8fac0c6250> at 0x7f8fac0c6290> timeout=1.0, duration=1.01 at 0x7f8f5c46f650> discarded task#=214 at 0x7f8fac0b5810>]) Mar 27 03:34:19 lago-basic-suite-master-host-0 vdsm[22587]: WARN File: /var/lib/libvirt/qemu/channels/fc3c11ae-4b16-4bab-8b5e-aa839b2a42f6.ovirt-guest-agent.0 already removed Mar 27 03:34:19 lago-basic-suite-master-host-0 vdsm[22587]: WARN File: /var/lib/libvirt/qemu/channels/fc3c11ae-4b16-4bab-8b5e-aa839b2a42f6.org.qemu.guest_agent.0 already removed Mar 27 03:34:20 lago-basic-suite-master-host-0 vdsm[22587]: WARN executor state: count=6 workers=set([<Worker name=periodic/2 running <Task discardable <Operation action=<VmDispatcher operation=<class 'vdsm.virt.periodic.DriveWatermarkMonitor'> at 0x7f8fac0c6250> at 0x7f8fac0c6290> timeout=1.0, duration=3.01 at 0x7f8f5c46f650> discarded task#=214 at 0x7f8fac0b5810>, <Worker name=periodic/3 running <Task discardable <Operation action=<VmDispatcher operation=<class 'vdsm.virt.periodic.DriveWatermarkMonitor'> at 0x7f8fac0c6250> at 0x7f8fac0c6290> timeout=1.0, duration=1.01 at 0x7f8f5c35ba90> discarded task#=194 at 0x7f8fac0b5b10>, <Worker name=periodic/0 waiting task#=208 at 0x7f8fac0b5310>, <Worker name=periodic/1 running <Task discardable <Operation action=<VmDispatcher operation=<class 'vdsm.virt.periodic.BlockjobMonitor'> at 0x7f8fac0c61d0> at 0x7f8fac0c6210> timeout=7.5, duration=0.44 at 0x7f8f5c2dd250> task#=200 at 0x7f8fac0b5550>, <Worker name=periodic/4 waiting task#=0 at 0x7f8f5c412190>, <Worker name=periodic/5 waiting task#=0 at 0x7f8f8c13dfd0>]) > > > Barak,Gal,Galit, Evgheni - any thoughts on something on infra that can > > cause this? ( other than slow servers ) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 26, 2019 at 11:13 AM Dominik Holler < > dholler(a)redhat.com&gt; > > > > > > wrote: > > > > > > > > > > > > > > On Tue, 26 Mar 2019 12:31:36 +0100 > > > > > > > Dominik Holler <dholler(a)redhat.com&gt; wrote: > > > > > > > > > > > > > > > On Tue, 26 Mar 2019 10:58:22 +0000 > > > > > > > > Dafna Ron <dron(a)redhat.com&gt; wrote: > > > > > > > > > > > > > > > > > This is still failing randomly > > > > > > > > > > > > > > > > > > > > > > > > > I created https://gerrit.ovirt.org/#/c/98906/ to help to > > > understand > > > > > > > > which action is crashing the guest. > > > > > > > > > > > > > > > > > > > > > > I was not able to reproduce the failure with the change above. > > > > > > > We could merge the change to have better information on the > next > > > > > > > failure. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 26, 2019 at 8:15 AM Dominik Holler < > > > dholler(a)redhat.com&gt; > > > > > > wrote: > > > > > > > > > > > > > > > > > > > On Mon, 25 Mar 2019 17:30:53 -0400 > > > > > > > > > > Ryan Barry <rbarry(a)redhat.com&gt; wrote: > > > > > > > > > > > > > > > > > > > > > It may be virt, but I'm looking... > > > > > > > > > > > > > > > > > > > > > > I'm very suspicious of this happening immediately > after > > > > > > hotplugging a > > > > > > > > > > NIC, > > > > > > > > > > > especially since the bug attached to > > > > > > https://gerrit.ovirt.org/#/c/98765/ > > > > > > > > > > > talks about dropping packets. Dominik, did anything > else > > > change > > > > > > here? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > No, nothing I am aware of. > > > > > > > > > > > > > > > > > > > > Is there already a pattern in the failed runs detected, > or > > > does it > > > > > > fail > > > > > > > > > > randomly? > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 25, 2019 at 12:42 PM Anton Marchukov < > > > > > > amarchuk(a)redhat.com&gt; > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Which team is it? Is it Virt? Just checking who > should > > > open a > > > > > > bug in > > > > > > > > > > > > libvirt as suggested. > > > > > > > > > > > > > > > > > > > > > > > > > On 22 Mar 2019, at 20:52, Nir Soffer < > > > nsoffer(a)redhat.com&gt; > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Mar 22, 2019 at 7:12 PM Dafna Ron < > > > dron(a)redhat.com&gt; > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > We are failing ovirt-engine master on test > > > > > > > > > > 004_basic_sanity.hotplug_cpu > > > > > > > > > > > > > looking at the logs, we can see that the for some > > > reason, > > > > > > libvirt > > > > > > > > > > > > reports a vm as none responsive which fails the > test. > > > > > > > > > > > > > > > > > > > > > > > > > > CQ first failure was for patch: > > > > > > > > > > > > > https://gerrit.ovirt.org/#/c/98553/ - core: Add > > > > > > display="on" for > > > > > > > > > > mdevs, > > > > > > > > > > > > use nodisplay to override > > > > > > > > > > > > > But I do not think this is the cause of failure. > > > > > > > > > > > > > > > > > > > > > > > > > > Adding Marcin, Milan and Dan as well as I think > it may > > > be > > > > > > netwrok > > > > > > > > > > > > related. > > > > > > > > > > > > > > > > > > > > > > > > > > You can see the libvirt log here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/arti... > > > > > > > > > > > > > > > > > > > > > > > > > > you can see the full logs here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13516/artif... > > > > > > > > > > > > > > > > > > > > > > > > > > Evgheni and I confirmed this is not an infra > issue and > > > the > > > > > > problem is > > > > > > > > > > > > ssh connection to the internal vm > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > Dafna > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > error: > > > > > > > > > > > > > 2019-03-22 15:08:22.658+0000: 22068: warning : > > > > > > > > > > qemuDomainObjTaint:7521 : > > > > > > > > > > > > Domain id=3 name='vm0' > > > > > > uuid=a9443d02-e054-40bb-8ea3-ae346e2d02a7 is > > > > > > > > > > > > tainted: hook-script > > > > > > > > > > > > > > > > > > > > > > > > > > Why our vm is tainted? > > > > > > > > > > > > > > > > > > > > > > > > > > 2019-03-22 15:08:22.693+0000: 22068: error : > > > > > > > > > > > > virProcessRunInMountNamespace:1159 : internal error: > > > child > > > > > > reported: > > > > > > > > > > unable > > > > > > > > > > > > to set security context > > > 'system_u:object_r:virt_content_t:s0' > > > > > > on > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > '/rhev/data-center/mnt/blockSD/91d97292-9ac3-4d77-a152-c7ea3250b065/images/e60dae48-ecc7-4171-8bfe-42bfc2190ffd/40243c76-a384-4497-8a2d-792a5e10d510': > > > > > > > > > > > > No such file or directory > > > > > > > > > > > > > > > > > > > > > > > > > > This should not happen, libvirt is not adding > labels to > > > > > > files in > > > > > > > > > > > > /rhev/data-center. It is using using its own mount > > > > > > > > > > > > > namespace and adding there the devices used by > the VM. > > > Since > > > > > > libvirt > > > > > > > > > > > > create the devices in its namespace > > > > > > > > > > > > > it should not complain about missing paths in > > > > > > /rhev/data-center. > > > > > > > > > > > > > > > > > > > > > > > > > > I think we should file a libvirt bug for this. > > > > > > > > > > > > > > > > > > > > > > > > > > 2019-03-22 15:08:28.168+0000: 22070: error : > > > > > > > > > > > > qemuDomainAgentAvailable:9133 : Guest agent is not > > > responding: > > > > > > QEMU > > > > > > > > > > guest > > > > > > > > > > > > agent is not connected > > > > > > > > > > > > > 2019-03-22 15:08:58.193+0000: 22070: error : > > > > > > > > > > > > qemuDomainAgentAvailable:9133 : Guest agent is not > > > responding: > > > > > > QEMU > > > > > > > > > > guest > > > > > > > > > > > > agent is not connected > > > > > > > > > > > > > 2019-03-22 15:13:58.179+0000: 22071: error : > > > > > > > > > > > > qemuDomainAgentAvailable:9133 : Guest agent is not > > > responding: > > > > > > QEMU > > > > > > > > > > guest > > > > > > > > > > > > agent is not connected > > > > > > > > > > > > > > > > > > > > > > > > > > Do we have guest agent in the test VMs? > > > > > > > > > > > > > > > > > > > > > > > > > > Nir > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Anton Marchukov > > > > > > > > > > > > Associate Manager - RHV DevOps - Red Hat > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > Infra mailing list -- infra(a)ovirt.org > > > > > > > > > > > > To unsubscribe send an email to > infra-leave(a)ovirt.org > > > > > > > > > > > > Privacy Statement: > > > https://www.ovirt.org/site/privacy-policy/ > > > > > > > > > > > > oVirt Code of Conduct: > > > > > > > > > > > > > > > https://www.ovirt.org/community/about/community-guidelines/ > > > > > > > > > > > > List Archives: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.ovirt.org/archives/list/infra@ovirt.org/message/B44Q3AZA7JU... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > Devel mailing list -- devel(a)ovirt.org > > > > > > > > > > To unsubscribe send an email to devel-leave(a)ovirt.org > > > > > > > > > > Privacy Statement: > > > https://www.ovirt.org/site/privacy-policy/ > > > > > > > > > > oVirt Code of Conduct: > > > > > > > > > > > https://www.ovirt.org/community/about/community-guidelines/ > > > > > > > > > > List Archives: > > > > > > > > > > > > > > > > > > > > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7XYIPXZLPHR... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Ryan Barry > > > > > > > > > > > > Associate Manager - RHV Virt/SLA > > > > > > > > > > > > rbarry(a)redhat.com M: +16518159306 IM: rbarry > > > > > > _______________________________________________ > > > > > > Infra mailing list -- infra(a)ovirt.org > > > > > > To unsubscribe send an email to infra-leave(a)ovirt.org > > > > > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > > > > > oVirt Code of Conduct: > > > > > > https://www.ovirt.org/community/about/community-guidelines/ > > > > > > List Archives: > > > > > > > > > > https://lists.ovirt.org/archives/list/infra@ovirt.org/message/K7WPLV4WFLN... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Ryan Barry > > > > > > Associate Manager - RHV Virt/SLA > > > > > > rbarry(a)redhat.com M: +16518159306 IM: rbarry > > > _______________________________________________ > > > Devel mailing list -- devel(a)ovirt.org > > > To unsubscribe send an email to devel-leave(a)ovirt.org > > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > > oVirt Code of Conduct: > > > https://www.ovirt.org/community/about/community-guidelines/ > > > List Archives: > > > > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/J5LSZJVOJV7... > > > > > > > > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NCD6OCOWXSH... > -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>