[ovirt-devel] Help with issues with migration
David Caro
dcaroest at redhat.com
Wed Feb 4 19:47:58 UTC 2015
On 02/04, Francesco Romani wrote:
> ----- Original Message -----
> > From: "David Caro" <dcaroest at redhat.com>
> > To: devel at ovirt.org
> > Sent: Wednesday, February 4, 2015 11:21:39 AM
> > Subject: [ovirt-devel] Help with issues with migration
> >
> >
> > Hi!
> >
> > Upstream phoenix lab has stabilized, but after the outages we are finding
> > some
> > issues probably caused by it.
> >
> > One of them is that th vm migration is not working, ew have two hosts srv05
> > and
> > srv06, and we want to migrate vms from 06 to 05 but we find those errors on
> > vdsm on 05:
> >
> >
> > Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
> > vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Unknown type found, device:
> > '{'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address':
> > {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}'
> > found
> > Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
> > vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Unknown type found, device:
> > '{'device': 'unix', 'alias': 'channel1', 'type': 'channel', 'address':
> > {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}'
> > found
> > Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
> > vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Unknown type found, device:
> > '{'device': 'spicevmc', 'alias': 'channel2', 'type': 'channel', 'address':
> > {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}'
> > found
> > Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm ERROR
> > vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Alias not found for device
> > type graphics during migration at destination host
>
> All of these are mostly noise. We have BZs basically to silence them, but they
> are harmless.
>
> > After that sanlock complains:
> > Feb 04 02:56:50 ovirt-srv05 sanlock[1055]: 2015-02-04 02:56:50-0700 1453
> > [1055]: cmd 9 target pid 6479 not found
>
> There is a BZ for this also, but likely not the root cause here
>
> > but it seems that the vm is starting up:
> > Feb 04 02:56:50 ovirt-srv05 systemd[1]: Starting Virtual Machine
> > qemu-el6-vm03-phx-ovirt-org.
> > -- Subject: Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope
> > has begun with start-up
> > -- Defined-By: systemd
> > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> > --
> > -- Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope has begun
> > starting up.
> > Feb 04 02:56:50 ovirt-srv05 systemd-machined[5642]: New machine
> > qemu-el6-vm03-phx-ovirt-org.
> > -- Subject: A virtual machine or container has been started
> > -- Defined-By: systemd
> > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> > --
> > -- The virtual machine qemu-el6-vm03-phx-ovirt-org with its leader PID 6479
> > has been
> > -- started is now ready to use.
> > Feb 04 02:56:50 ovirt-srv05 systemd[1]: Started Virtual Machine
> > qemu-el6-vm03-phx-ovirt-org.
> > -- Subject: Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope
> > has finished start-up
> > -- Defined-By: systemd
> > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> > --
> > -- Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope has
> > finished starting up.
> > --
> > -- The start-up result is done.
> >
> >
> > But it shuts down:
> > Feb 04 02:56:51 ovirt-srv05 systemd-machined[5642]: Machine
> > qemu-el6-vm03-phx-ovirt-org terminated.
> > -- Subject: A virtual machine or container has been terminated
> > -- Defined-By: systemd
> > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> > --
> > -- The virtual machine qemu-el6-vm03-phx-ovirt-org with its leader PID 6479
> > has been
> > -- shut down.
> > Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm vm.Vm ERROR
> > vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Failed to start a migration
> > destination vm
> > Traceback (most recent call last):
> > File "/usr/share/vdsm/virt/vm.py",
> > line 2298, in _startUnderlyingVm
> > self._completeIncomingMigration()
> > File "/usr/share/vdsm/virt/vm.py",
> > line 4107, in
> > _completeIncomingMigration
> > self._incomingMigrationFinished.isSet(),
> > usedTimeout)
> > File "/usr/share/vdsm/virt/vm.py",
> > line 4160, in
> > _attachLibvirtDomainAfterMigration
> > raise
> > MigrationError(e.get_error_message())
> > MigrationError: Domain not found: no
> > domain with matching uuid
> > '43415276-a7bf-4c86-b0e9-70a5f6d39a40'
> > Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm root WARNING File:
> > /var/lib/libvirt/qemu/channels/43415276-a7bf-4c86-b0e9-70a5f6d39a40.com.redhat.rhevm.vdsm
> > already removed
> > Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm root WARNING File:
> > /var/lib/libvirt/qemu/channels/43415276-a7bf-4c86-b0e9-70a5f6d39a40.org.qemu.guest_agent.0
> > already removed
> > Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
> > vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::trying to set state to Down
> > when already Down
>
> This just tells us that QEMU failed to run in the dest host. Any more informations
> in the QEMU logs and/or related to libvirt?
After upgrading all the hosts the migration is working again, I suppose that
there was something in the storage locked after the dirty outage that just
needed a cleanup....
I did not see any extra traces on any libvirt logs then, and I can't reproduce
now :/
>
> Bests,
>
> --
> Francesco Romani
> RedHat Engineering Virtualization R & D
> Phone: 8261328
> IRC: fromani
--
David Caro
Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D
Tel.: +420 532 294 605
Email: dcaro at redhat.com
Web: www.redhat.com
RHT Global #: 82-62605
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20150204/03b4222a/attachment-0001.sig>
More information about the Devel
mailing list