[ovirt-users] Details about why a live migration failed

Fri Mar 24 14:15:26 UTC 2017

Mmmh this is all I got from libvirt log on receiver host:

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name
guest=druid-co01,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-5-druid-co01/master-key.aes
-machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m
size=16777216k,slots=16,maxmem=4294967296k -realtime mlock=off -smp
4,maxcpus=64,sockets=16,cores=4,threads=1 -numa
node,nodeid=0,cpus=0-3,mem=16384 -uuid 4f627cc1-9b52-4eef-bf3a-c02e8a6303b8
-smbios 'type=1,manufacturer=oVirt,product=oVirt
Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-0037-4C10-8031-B7C04F564232,uuid=4f627cc1-9b52-4eef-bf3a-c02e8a6303b8'
-no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-5-druid-co01.billydoma/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc
base=2017-03-24T10:38:03,driftfix=slew -global
kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device
virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive
if=none,id=drive-ide0-1-0,readonly=on -device
ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
file=/rhev/data-center/00000001-0001-0001-0001-0000000003e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/08b19faa-4b1f-4da8-87a2-2af0700a7906/bdb18a7d-1558-41f9-aa3a-e63407c7881e,format=qcow2,if=none,id=drive-virtio-disk0,serial=08b19faa-4b1f-4da8-87a2-2af0700a7906,cache=none,werror=stop,rerror=stop,aio=threads
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-drive
file=/rhev/data-center/00000001-0001-0001-0001-0000000003e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/987d84da-188b-45c0-99d0-3dde29ddcb6e/51a1b9ee-b0ae-4208-9806-a319d34db06e,format=qcow2,if=none,id=drive-virtio-disk1,serial=987d84da-188b-45c0-99d0-3dde29ddcb6e,cache=none,werror=stop,rerror=stop,aio=threads
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk1,id=virtio-disk1
-netdev tap,fd=34,id=hostnet0,vhost=on,vhostfd=35 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:16:01:93,bus=pci.0,addr=0x3
-chardev
socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4f627cc1-9b52-4eef-bf3a-c02e8a6303b8.com.redhat.rhevm.vdsm,server,nowait
-device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm
-chardev
socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/4f627cc1-9b52-4eef-bf3a-c02e8a6303b8.org.qemu.guest_agent.0,server,nowait
-device
virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
-chardev spicevmc,id=charchannel2,name=vdagent -device
virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0
-spice
tls-port=5904,addr=192.168.10.107,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on
-device
qxl-vga,id=video0,ram_size=67108864,vram_size=8388608,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2
-incoming defer -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
-msg timestamp=on
Domain id=5 is tainted: hook-script
2017-03-24T10:38:03.414396Z qemu-kvm: warning: CPU(s) not present in any
NUMA nodes: 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
52 53 54 55 56 57 58 59 60 61 62 63
2017-03-24T10:38:03.414497Z qemu-kvm: warning: All CPU(s) up to maxcpus
should be described in NUMA config
2017-03-24 10:41:20.982+0000: shutting down
2017-03-24T10:41:20.986633Z qemu-kvm: load of migration failed:
Input/output error

Donating host doesn't say a thing about this VM.
There's an "input/output error" but I can't see to what is related...

2017-03-24 13:38 GMT+01:00 Francesco Romani <fromani at redhat.com>:

>
> On 03/24/2017 11:58 AM, Davide Ferrari wrote:
> > And this is the vdsm log from vmhost04:
> >
> > Thread-6320717::INFO::2017-03-24
> > 11:41:13,019::migration::712::virt.vm::(monitor_migration)
> > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::Migration Progress: 190
> > seconds elapsed, 98% of data processed, total data: 16456MB, processed
> > data: 9842MB, remaining data: 386MB, transfer speed 52MBps, zero
> > pages: 1718676MB, compressed: 0MB, dirty rate: -1, memory iteration: -1
> > libvirtEventLoop::DEBUG::2017-03-24
> > 11:41:21,007::vm::4291::virt.vm::(onLibvirtLifecycleEvent)
> > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::event Suspended detail 0
> > opaque None
> > libvirtEventLoop::INFO::2017-03-24
> > 11:41:21,025::vm::4815::virt.vm::(_logGuestCpuStatus)
> > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::CPU stopped: onSuspend
> > libvirtEventLoop::DEBUG::2017-03-24
> > 11:41:21,069::vm::4291::virt.vm::(onLibvirtLifecycleEvent)
> > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::event Resumed detail 0
> > opaque None
> > libvirtEventLoop::INFO::2017-03-24
> > 11:41:21,069::vm::4815::virt.vm::(_logGuestCpuStatus)
> > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::CPU running: onResume
> > Thread-6320715::DEBUG::2017-03-24
> > 11:41:21,224::migration::715::virt.vm::(stop)
> > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::stopping migration
> > monitor thread
> > Thread-6320715::ERROR::2017-03-24
> > 11:41:21,225::migration::252::virt.vm::(_recover)
> > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::operation failed:
> > migration job: unexpectedly failed
>
> This is surprising (no pun intended)
> With a pretty high chance this comes from libvirt, I'm afraid you need
> to dig in the libvirt logs/journal entries to learn more.
> Vdsm could unfortunately do better than what it is already doing :\
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R&D
> Red Hat
> IRC: fromani github: @fromanirh
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>

-- 
Davide Ferrari
Senior Systems Engineer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170324/ebe94607/attachment.html>