On Wed, Oct 09, 2013 at 02:42:22PM +0200, Gianluca Cecchi wrote:
On Tue, Oct 8, 2013 at 10:40 AM, Dan Kenigsberg wrote:
But migration still fails
It seems like an unrelated failure. I do not know what's blocking migration traffic. Could you see if libvirtd.log and qemu logs at source and destinaiton have clues?
It seems that on VM.log under qemu of desdt host I have: ... -incoming tcp:[::]:49153: Failed to bind socket: Address already in use
Is that port really taken (`ss -ntp` should tell by whom)?
See all: - In libvirtd.log of source host 2013-10-07 23:20:54.471+0000: 1209: debug : qemuMonitorOpenInternal:751 : QEMU_MONITOR_NEW: mon=0x7fc66412e820 refs=2 fd=30 2013-10-07 23:20:54.472+0000: 1209: warning : qemuDomainObjEnterMonitorInternal:1136 : This thread seems to be the async job owner; entering monitor without asking for a nested job is dangerous 2013-10-07 23:20:54.472+0000: 1209: debug : qemuMonitorSetCapabilities:1145 : mon=0x7fc66412e820 2013-10-07 23:20:54.472+0000: 1209: debug : qemuMonitorSend:887 : QEMU_MONITOR_SEND_MSG: mon=0x7fc66412e820 msg={"execute":"qmp_capabilities","id":"libvirt-1"} fd=-1 2013-10-07 23:20:54.769+0000: 1199: error : qemuMonitorIORead:505 : Unable to read from monitor: Connection reset by peer 2013-10-07 23:20:54.769+0000: 1199: debug : qemuMonitorIO:638 : Error on monitor Unable to read from monitor: Connection reset by peer 2013-10-07 23:20:54.769+0000: 1199: debug : qemuMonitorIO:672 : Triggering error callback 2013-10-07 23:20:54.769+0000: 1199: debug : qemuProcessHandleMonitorError:351 : Received error on 0x7fc664124fb0 'c8again32' 2013-10-07 23:20:54.769+0000: 1209: debug : qemuMonitorSend:899 : Send command resulted in error Unable to read from monitor: Connection reset by peer 2013-10-07 23:20:54.770+0000: 1199: debug : qemuMonitorIO:638 : Error on monitor Unable to read from monitor: Connection reset by peer 2013-10-07 23:20:54.770+0000: 1209: debug : virFileMakePathHelper:1283 : path=/var/run/libvirt/qemu mode=0777 2013-10-07 23:20:54.770+0000: 1199: debug : qemuMonitorIO:661 : Triggering EOF callback 2013-10-07 23:20:54.770+0000: 1199: debug : qemuProcessHandleMonitorEOF:294 : Received EOF on 0x7fc664124fb0 'c8again32' 2013-10-07 23:20:54.770+0000: 1209: debug : qemuProcessStop:3992 : Shutting down VM 'c8again32' pid=18053 flags=0 2013-10-07 23:20:54.771+0000: 1209: error : virNWFilterDHCPSnoopEnd:2135 : internal error ifname "vnet0" not in key map 2013-10-07 23:20:54.782+0000: 1209: debug : virCommandRunAsync:2251 : About to run /bin/sh -c 'IPT="/usr/sbin/iptables" $IPT -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0 $IPT -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0 $IPT -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0 $IPT -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0 $IPT -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT $IPT -F FO-vnet0 $IPT -X FO-vnet0 $IPT -F FI-vnet0 $IPT -X FI-vnet0 $IPT -F HI-vnet0 $IPT -X HI-vnet0 IPT="/usr/sbin/ip6tables" $IPT -D libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet0 -g FO-vnet0 $IPT -D libvirt-out -m physdev --physdev-out vnet0 -g FO-vnet0 $IPT -D libvirt-in -m physdev --physdev-in vnet0 -g FI-vnet0 $IPT -D libvirt-host-in -m physdev --physdev-in vnet0 -g HI-vnet0 $IPT -D libvirt-in-post -m physdev --physdev-in vnet0 -j ACCEPT $IPT -F FO-vnet0 $IPT -X FO-vnet0 $IPT -F FI-vnet0 $IPT -X FI-vnet0 $IPT -F HI-vnet0 $IPT -X HI-vnet0 EBT="/usr/sbin/ebtables" $EBT -t nat -D PREROUTING -i vnet0 -j libvirt-I-vnet0 $EBT -t nat -D POSTROUTING -o vnet0 -j libvirt-O-vnet0 EBT="/usr/sbin/ebtables" collect_chains() { for tmp2 in $*; do for tmp in $($EBT -t nat -L $tmp2 | \ sed -n "/Bridge chain/,\$ s/.*-j \\([IO]-.*\\)/\\1/p"); do echo $tmp collect_chains $tmp done done } rm_chains() { for tmp in $*; do $EBT -t nat -F $tmp; done for tmp in $*; do $EBT -t nat -X $tmp; done } tmp='\'' '\'' IFS='\'' '\'''\'' '\''$tmp chains="$(collect_chains libvirt-I-vnet0 libvirt-O-vnet0)" $EBT -t nat -F libvirt-I-vnet0 $EBT -t nat -F libvirt-O-vnet0 rm_chains $chains $EBT -t nat -F libvirt-I-vnet0 $EBT -t nat -X libvirt-I-vnet0 $EBT -t nat -F libvirt-O-vnet0 $EBT -t nat -X libvirt-O-vnet0 ' 2013-10-07 23:20:54.784+0000: 1209: debug : virCommandRunAsync:2256 : Command result 0, with PID 18076 2013-10-07 23:20:54.863+0000: 1209: debug : virCommandRun:2125 : Result exit status 255, stdout: '' stderr: 'iptables v1.4.18: goto 'FO-vnet0' is not a chain
Try `iptables -h' or 'iptables --help' for more information. iptables v1.4.18: goto 'FO-vnet0' is not a chain
Try `iptables -h' or 'iptables --help' for more information. iptables v1.4.18: goto 'FI-vnet0' is not a chain Try `iptables -h' or 'iptables --help' for more information. iptables v1.4.18: goto 'HI-vnet0' is not a chain
Try `iptables -h' or 'iptables --help' for more information. iptables: Bad rule (does a matching rule exist in that chain?). iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. iptables: No chain/target/match by that name. ip6tables v1.4.18: goto 'FO-vnet0' is not a chain
Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables v1.4.18: goto 'FO-vnet0' is not a chain
Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables v1.4.18: goto 'FI-vnet0' is not a chain
Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables v1.4.18: goto 'HI-vnet0' is not a chain
Try `ip6tables -h' or 'ip6tables --help' for more information. ip6tables: Bad rule (does a matching rule exist in that chain?). ip6tables: No chain/target/match by that name. ip6tables: No chain/target/match by that name. ip6tables: No chain/target/match by that name. ip6tables: No chain/target/match by that name. ip6tables: No chain/target/match by that name. ip6tables: No chain/target/match by that name. Illegal target name 'libvirt-O-vnet0'. Chain 'libvirt-O-vnet0' doesn't exist. Chain 'libvirt-O-vnet0' doesn't exist. Chain 'libvirt-O-vnet0' doesn't exist. Chain 'libvirt-O-vnet0' doesn't exist. ' 2013-10-07 23:20:54.863+0000: 1209: debug : qemuMonitorClose:821 : QEMU_MONITOR_CLOSE: mon=0x7fc66412e820 refs=2 2013-10-07 23:20:54.863+0000: 1209: debug : qemuProcessKill:3951 : vm=c8again32 pid=18053 flags=5 2013-10-07 23:20:54.863+0000: 1209: debug : virProcessKillPainfully:269 : vpid=18053 force=1 2013-10-07 23:20:54.863+0000: 1209: debug : qemuDomainCleanupRun:2132 : driver=0x7fc664024cd0, vm=c8again32 2013-10-07 23:20:54.863+0000: 1209: debug : qemuProcessAutoDestroyRemove:4504 : vm=c8again32 2013-10-07 23:20:54.863+0000: 1209: debug : virQEMUCloseCallbacksUnset:744 : vm=c8again32, uuid=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1, cb=0x7fc66b6fe570 2013-10-07 23:20:54.864+0000: 1209: error : virPortAllocatorRelease:174 : port 0 must be in range (5900, 65535) 2013-10-07 23:20:54.865+0000: 1209: debug : qemuDomainObjEndJob:1070 : Stopping job: none (async=migration in) 2013-10-07 23:20:54.865+0000: 1209: debug : qemuDomainObjEndAsyncJob:1088 : Stopping async job: migration in 2013-10-07 23:20:54.865+0000: 1199: debug : qemuProcessHandleMonitorEOF:306 : Domain 0x7fc664124fb0 is not active, ignoring EOF Caught Segmentation violation dumping internal log buffer:
This line one seems ominous. Can libvir-list help with it?