Re: [ovirt-devel] Help with issues with migration

4 Feb 2015


      ----- Original Message -----
...
From: "David Caro" <dcaroest@redhat.com>
To: devel@ovirt.org
Sent: Wednesday, February 4, 2015 11:21:39 AM
Subject: [ovirt-devel] Help with issues with migration
Hi!
Upstream phoenix lab has stabilized, but after the outages we are finding
some
issues probably caused by it.
One of them is that th vm migration is not working, ew have two hosts srv05
and
srv06, and we want to migrate vms from 06 to 05 but we find those errors on
vdsm on 05:
Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
  vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Unknown type found, device:
  '{'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address':
  {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '1'}}'
  found
  Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
  vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Unknown type found, device:
  '{'device': 'unix', 'alias': 'channel1', 'type': 'channel', 'address':
  {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '2'}}'
  found
  Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
  vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Unknown type found, device:
  '{'device': 'spicevmc', 'alias': 'channel2', 'type': 'channel', 'address':
  {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port': '3'}}'
  found
  Feb 04 02:56:50 ovirt-srv05 vdsm[5170]: vdsm vm.Vm ERROR
  vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Alias not found for device
  type graphics during migration at destination host
All of these are mostly noise. We have BZs basically to silence them, but they
are harmless.
...
After that sanlock complains:
  Feb 04 02:56:50 ovirt-srv05 sanlock[1055]: 2015-02-04 02:56:50-0700 1453
  [1055]: cmd 9 target pid 6479 not found
There is a BZ for this also, but likely not the root cause here
...
but it seems that the vm is starting up:
  Feb 04 02:56:50 ovirt-srv05 systemd[1]: Starting Virtual Machine
  qemu-el6-vm03-phx-ovirt-org.
  -- Subject: Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope
  has begun with start-up
  -- Defined-By: systemd
  -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
  --
  -- Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope has begun
  starting up.
  Feb 04 02:56:50 ovirt-srv05 systemd-machined[5642]: New machine
  qemu-el6-vm03-phx-ovirt-org.
  -- Subject: A virtual machine or container has been started
  -- Defined-By: systemd
  -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
  --
  -- The virtual machine qemu-el6-vm03-phx-ovirt-org with its leader PID 6479
  has been
  -- started is now ready to use.
  Feb 04 02:56:50 ovirt-srv05 systemd[1]: Started Virtual Machine
  qemu-el6-vm03-phx-ovirt-org.
  -- Subject: Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope
  has finished start-up
  -- Defined-By: systemd
  -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
  --
  -- Unit machine-qemu\x2del6\x2dvm03\x2dphx\x2dovirt\x2dorg.scope has
  finished starting up.
  --
  -- The start-up result is done.
But it shuts down:
Feb 04 02:56:51 ovirt-srv05 systemd-machined[5642]: Machine
qemu-el6-vm03-phx-ovirt-org terminated.
  -- Subject: A virtual machine or container has been terminated
  -- Defined-By: systemd
  -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
  --
  -- The virtual machine qemu-el6-vm03-phx-ovirt-org with its leader PID 6479
  has been
  -- shut down.
  Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm vm.Vm ERROR
  vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::Failed to start a migration
  destination vm
                                        Traceback (most recent call last):
                                          File "/usr/share/vdsm/virt/vm.py",
                                          line 2298, in _startUnderlyingVm
                                            self._completeIncomingMigration()
                                          File "/usr/share/vdsm/virt/vm.py",
                                          line 4107, in
                                          _completeIncomingMigration
                                            self._incomingMigrationFinished.isSet(),
                                            usedTimeout)
                                          File "/usr/share/vdsm/virt/vm.py",
                                          line 4160, in
                                          _attachLibvirtDomainAfterMigration
                                            raise
                                            MigrationError(e.get_error_message())
                                        MigrationError: Domain not found: no
                                        domain with matching uuid
                                        '43415276-a7bf-4c86-b0e9-70a5f6d39a40'
  Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm root WARNING File:
  /var/lib/libvirt/qemu/channels/43415276-a7bf-4c86-b0e9-70a5f6d39a40.com.redhat.rhevm.vdsm
  already removed
  Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm root WARNING File:
  /var/lib/libvirt/qemu/channels/43415276-a7bf-4c86-b0e9-70a5f6d39a40.org.qemu.guest_agent.0
  already removed
  Feb 04 02:56:51 ovirt-srv05 vdsm[5170]: vdsm vm.Vm WARNING
  vmId=`43415276-a7bf-4c86-b0e9-70a5f6d39a40`::trying to set state to Down
  when already Down
This just tells us that QEMU failed to run in the dest host. Any more informations
in the QEMU logs and/or related to libvirt?

Bests,

-- 
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani