[Users] Migration issues with ovirt 3.3
Gianluca Cecchi
gianluca.cecchi at gmail.com
Mon Oct 7 23:44:47 UTC 2013
On Mon, Oct 7, 2013 at 2:59 AM, Dan Kenigsberg wrote:
>
> Would you please test if http://gerrit.ovirt.org/19906 solves the
> issue? (I haven't. Too late at night.)
>
> Regards,
> Dan.
I can confirm that it resolves
https://bugzilla.redhat.com/show_bug.cgi?id=1007980
so now I'm able to start VM without having to select run once and
attaching a cd iso
(note that is only valid for newly created VMs though)
But migration still fails
On engine:
2013-10-08 01:20:51,459 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-15) START, GlusterV
olumesListVDSCommand(HostName = f18ovn03, HostId =
b67bcfd4-f868-49d5-8704-4936ee922249), log id: 4425a532
2013-10-08 01:20:51,547 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-15) FINISH, Gluster
VolumesListVDSCommand, return:
{ed71a4c2-6205-4aad-9aab-85da086d5ba3=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity at 8defb13f
, c8cbcac7-1d40-4cee-837d-bb97467fb2bd=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity at 91be2b8a},
log id: 4425a532
2013-10-08 01:20:51,697 INFO
[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
(DefaultQuartzScheduler_Worker-39) RefreshVmList vm id d54660a2-45
ed-41ae-ab99-a6f93ebbdbb1 is migrating to vds f18ovn01 ignoring it in
the refresh until migration is done
2013-10-08 01:20:54,765 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServicesListVDSCommand]
(pool-6-thread-38) START, GlusterServicesListVDSC
ommand(HostName = f18ovn01, HostId =
80188ccc-83b2-4bc8-9385-8d07f7458a3c), log id: 41ed75a4
2013-10-08 01:20:54,766 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServicesListVDSCommand]
(pool-6-thread-36) START, GlusterServicesListVDSC
ommand(HostName = f18ovn03, HostId =
b67bcfd4-f868-49d5-8704-4936ee922249), log id: 7eea38bc
2013-10-08 01:20:56,618 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-2) START, GlusterVo
lumesListVDSCommand(HostName = f18ovn01, HostId =
80188ccc-83b2-4bc8-9385-8d07f7458a3c), log id: 34d0827b
2013-10-08 01:20:56,722 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand]
(DefaultQuartzScheduler_Worker-84) Command ListVDS execution failed.
Exception: VDSNetworkException: java.net.SocketException: Connection
reset
2013-10-08 01:20:56,725 ERROR
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServicesListVDSCommand]
(pool-6-thread-36) Command GlusterServicesListVDS execution failed.
Exception: VDSNetworkException: java.net.SocketException: Connection
reset
2013-10-08 01:20:56,725 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServicesListVDSCommand]
(pool-6-thread-36) FINISH, GlusterServicesListVDSCommand, log id:
7eea38bc
2013-10-08 01:20:56,727 WARN
[org.ovirt.engine.core.vdsbroker.VdsManager]
(DefaultQuartzScheduler_Worker-84) Failed to refresh VDS , vds =
b67bcfd4-f868-49d5-8704-4936ee922249 : f18ovn03, VDS Network Error,
continuing.
java.net.SocketException: Connection reset
2013-10-08 01:20:58,583 INFO
[org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand]
(DefaultQuartzScheduler_Worker-48) Running command:
SetStoragePoolStatusCommand internal: true. Entities affected : ID:
eb679feb-4da2-4fd0-a185-abbe459ffa70 Type: StoragePool
2013-10-08 01:20:58,603 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler_Worker-48) Correlation ID: 16f779a3, Call
Stack: null, Custom Event ID: -1, Message: Invalid status on Data
Center Gluster. Setting status to Non Responsive.
2013-10-08 01:20:58,701 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
(DefaultQuartzScheduler_Worker-48) Command
org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand
return value
TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654,
mMessage=Not SPM: ()]]
2013-10-08 01:20:58,714 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
(DefaultQuartzScheduler_Worker-48) HostName = f18ovn03
2013-10-08 01:20:58,714 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
(DefaultQuartzScheduler_Worker-48) Command HSMGetAllTasksStatusesVDS
execution failed. Exception: IRSNonOperationalException:
IRSGenericException: IRSErrorException: IRSNonOperationalException:
Not SPM: ()
2013-10-08 01:20:59,356 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand]
(DefaultQuartzScheduler_Worker-41) Command ListVDS execution failed.
Exception: VDSNetworkException: java.net.SocketException: Connection
reset
On host where VM was running vdsm.log contains:
Thread-4643::DEBUG::2013-10-08
01:20:53,700::BindingXMLRPC::979::vds::(wrapper) client
[10.4.4.60]::call vmMigrate with ({'tunneled': 'false', 'dst':
'10.4.4.58:54321', 'vmId': 'd54660a2-45ed-41ae-ab99-a6f93ebbdbb1',
'method': 'online', 'src': '10.4.4.59'},) {} flowID [7c55a4a9]
Thread-4643::DEBUG::2013-10-08 01:20:53,700::API::462::vds::(migrate)
{'tunneled': 'false', 'dst': '10.4.4.58:54321', 'vmId':
'd54660a2-45ed-41ae-ab99-a6f93ebbdbb1', 'method': 'online', 'src':
'10.4.4.59'}
Thread-4643::DEBUG::2013-10-08
01:20:53,701::BindingXMLRPC::986::vds::(wrapper) return vmMigrate with
{'status': {'message': 'Migration in progress', 'code': 0},
'progress': 0}
Thread-4644::DEBUG::2013-10-08
01:20:53,702::vm::182::vm.Vm::(_setupVdsConnection)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Destination server is:
10.4.4.58:54321
Thread-4644::DEBUG::2013-10-08
01:20:53,703::vm::184::vm.Vm::(_setupVdsConnection)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Initiating connection
with destination
Thread-4644::DEBUG::2013-10-08
01:20:53,718::vm::238::vm.Vm::(_prepareGuest)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Migration started
Thread-4644::DEBUG::2013-10-08 01:20:53,739::vm::302::vm.Vm::(run)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::migration semaphore
acquired
Thread-4644::DEBUG::2013-10-08
01:20:53,933::vm::360::vm.Vm::(_startUnderlyingMigration)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::starting migration to
qemu+tls://10.4.4.58/system with miguri tcp://10.4.4.58
Thread-4645::DEBUG::2013-10-08 01:20:53,934::vm::718::vm.Vm::(run)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::migration downtime thread
started
Thread-4646::DEBUG::2013-10-08 01:20:53,935::vm::756::vm.Vm::(run)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::starting migration
monitor thread
Thread-4648::DEBUG::2013-10-08
01:20:54,321::BindingXMLRPC::979::vds::(wrapper) client
[10.4.4.60]::call volumesList with () {}
Thread-4648::DEBUG::2013-10-08
01:20:54,349::BindingXMLRPC::986::vds::(wrapper) return volumesList
with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'gviso':
{'transportType': ['TCP'], 'uuid':
'c8cbcac7-1d40-4cee-837d-bb97467fb2bd', 'bricks':
['f18ovn01.mydomain:/gluster/ISO_GLUSTER/brick1',
'f18ovn03.mydomain:/gluster/ISO_GLUSTER/brick1'], 'volumeName':
'gviso', 'volumeType': 'REPLICATE', 'replicaCount': '2', 'brickCount':
'2', 'distCount': '2', 'volumeStatus': 'ONLINE', 'stripeCount': '1',
'options': {'storage.owner-gid': '36', 'storage.owner-uid': '36',
'server.allow-insecure': 'on'}}, 'gvdata': {'transportType': ['TCP'],
'uuid': 'ed71a4c2-6205-4aad-9aab-85da086d5ba3', 'bricks':
['f18ovn01.mydomain:/gluster/DATA_GLUSTER/brick1',
'f18ovn03.mydomain:/gluster/DATA_GLUSTER/brick1'], 'volumeName':
'gvdata', 'volumeType': 'REPLICATE', 'replicaCount': '2',
'brickCount': '2', 'distCount': '2', 'volumeStatus': 'ONLINE',
'stripeCount': '1', 'options': {'server.allow-insecure': 'on',
'storage.owner-uid': '36', 'storage.owner-gid': '36'}}}}
Thread-4644::ERROR::2013-10-08
01:20:54,873::libvirtconnection::94::libvirtconnection::(wrapper)
connection to libvirt broken. ecode: 38 edom: 7
Thread-4644::ERROR::2013-10-08
01:20:54,873::libvirtconnection::96::libvirtconnection::(wrapper)
taking calling process down.
MainThread::DEBUG::2013-10-08
01:20:54,874::vdsm::45::vds::(sigtermHandler) Received signal 15
Thread-4644::DEBUG::2013-10-08 01:20:54,874::vm::733::vm.Vm::(cancel)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::canceling migration
downtime thread
Thread-4644::DEBUG::2013-10-08 01:20:54,875::vm::803::vm.Vm::(stop)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::stopping migration
monitor thread
Thread-4645::DEBUG::2013-10-08 01:20:54,875::vm::730::vm.Vm::(run)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::migration downtime thread
exiting
Thread-4644::ERROR::2013-10-08
01:20:54,875::vm::244::vm.Vm::(_recover)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Cannot recv data:
Input/output error
Thread-4644::ERROR::2013-10-08 01:20:55,008::vm::324::vm.Vm::(run)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Failed to migrate
Traceback (most recent call last):
File "/usr/share/vdsm/vm.py", line 311, in run
self._startUnderlyingMigration()
File "/usr/share/vdsm/vm.py", line 388, in _startUnderlyingMigration
None, maxBandwidth)
File "/usr/share/vdsm/vm.py", line 826, in f
ret = attr(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/vdsm/libvirtconnection.py",
line 76, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1253, in
migrateToURI2
if ret == -1: raise libvirtError ('virDomainMigrateToURI2()
failed', dom=self)
libvirtError: Cannot recv data: Input/output error
MainThread::INFO::2013-10-08
01:20:55,321::vmChannels::183::vds::(stop) VM channels listener was
stopped.
MainThread::INFO::2013-10-08 01:20:55,322::momIF::81::MOM::(stop)
Shutting down MOM
MainThread::DEBUG::2013-10-08
01:20:55,322::task::579::TaskManager.Task::(_updateState)
Task=`23df8b46-c916-4c52-98a9-3e6540171467`::moving from state init ->
state preparing
MainThread::INFO::2013-10-08
01:20:55,323::logUtils::44::dispatcher::(wrapper) Run and protect:
prepareForShutdown(options=None)
Thread-11::DEBUG::2013-10-08
01:20:55,323::storageServer::698::ConnectionMonitor::(_monitorConnections)
Monitoring stopped
VM Channels Listener::INFO::2013-10-08
01:20:55,672::vmChannels::178::vds::(run) VM channels listener thread
has ended.
Thread-23::DEBUG::2013-10-08
01:20:55,929::domainMonitor::161::Storage.DomainMonitorThread::(_monitorLoop)
Stopping domain monitor for cc790a86-72e2-4aa2-a0b6-700756c399c3
Thread-24::DEBUG::2013-10-08
01:20:55,930::domainMonitor::161::Storage.DomainMonitorThread::(_monitorLoop)
Stopping domain monitor for d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291
Thread-24::INFO::2013-10-08
01:20:55,930::clusterlock::197::SANLock::(releaseHostId) Releasing
host id for domain d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291 (id: 2)
Thread-24::DEBUG::2013-10-08
01:20:55,930::domainMonitor::170::Storage.DomainMonitorThread::(_monitorLoop)
Unable to release the host id 2 for domain
d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291
Traceback (most recent call last):
File "/usr/share/vdsm/storage/domainMonitor.py", line 167, in _monitorLoop
self.domain.releaseHostId(self.hostId, unused=True)
File "/usr/share/vdsm/storage/sd.py", line 461, in releaseHostId
self._clusterLock.releaseHostId(hostId, async, unused)
File "/usr/share/vdsm/storage/clusterlock.py", line 204, in releaseHostId
raise se.ReleaseHostIdFailure(self._sdUUID, e)
ReleaseHostIdFailure: Cannot release host id:
('d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291', SanlockException(16, 'Sanlock
lockspace remove failure', 'Device or resource busy'))
MainThread::DEBUG::2013-10-08
01:20:55,932::taskManager::80::TaskManager::(prepareForShutdown)
Request to stop all tasks
MainThread::INFO::2013-10-08
01:20:55,932::logUtils::47::dispatcher::(wrapper) Run and protect:
prepareForShutdown, Return response: None
MainThread::DEBUG::2013-10-08
01:20:55,932::task::1168::TaskManager.Task::(prepare)
Task=`23df8b46-c916-4c52-98a9-3e6540171467`::finished: None
MainThread::DEBUG::2013-10-08
01:20:55,933::task::579::TaskManager.Task::(_updateState)
Task=`23df8b46-c916-4c52-98a9-3e6540171467`::moving from state
preparing -> state finished
On target host vdsm.log:
Thread-3393::DEBUG::2013-10-08
01:20:53,900::resourceManager::634::ResourceManager::(releaseResource)
Released resource 'Storage.d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291' (0
active users)
Thread-3393::DEBUG::2013-10-08
01:20:53,900::resourceManager::640::ResourceManager::(releaseResource)
Resource 'Storage.d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291' is free,
finding out if anyone is waiting for it.
Thread-3393::DEBUG::2013-10-08
01:20:53,900::resourceManager::648::ResourceManager::(releaseResource)
No one is waiting for resource
'Storage.d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291', Clearing records.
Thread-3393::DEBUG::2013-10-08
01:20:53,901::task::974::TaskManager.Task::(_decref)
Task=`be76585d-3de3-4520-873e-9eb501e12311`::ref 0 aborting False
Thread-3393::INFO::2013-10-08
01:20:53,901::clientIF::325::vds::(prepareVolumePath) prepared volume
path: /rhev/data-center/eb679feb-4da2-4fd0-a185-abbe459ffa70/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49
Thread-3393::INFO::2013-10-08
01:20:53,901::clientIF::325::vds::(prepareVolumePath) prepared volume
path: gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49
Thread-3393::DEBUG::2013-10-08
01:20:53,907::vm::2036::vm.Vm::(_startUnderlyingVm)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::_ongoingCreations
released
Thread-3393::DEBUG::2013-10-08
01:20:53,908::vm::3347::vm.Vm::(_waitForIncomingMigrationFinish)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Waiting 300 seconds for
end of migration
Thread-3392::ERROR::2013-10-08
01:20:53,907::vm::4789::vm.Vm::(_updateDevicesDomxmlCache)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Alias not found for
device type disk during migration at destination host
Thread-3392::DEBUG::2013-10-08
01:20:53,918::API::510::vds::(migrationCreate) Destination VM creation
succeeded
Thread-3392::DEBUG::2013-10-08
01:20:53,918::BindingXMLRPC::986::vds::(wrapper) return
vmMigrationCreate with {'status': {'message': 'Done', 'code': 0},
'migrationPort': 0, 'params': {'guestFQDN': '', 'acpiEnable': 'true',
'emulatedMachine': 'pc-1.0', 'afterMigrationStatus': 'Up',
'spiceSecureChannels':
'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard',
'pid': '0', 'memGuaranteedSize': 1024, 'transparentHugePages': 'true',
'displaySecurePort': '-1', 'timeOffset': '-114', 'cpuType':
'Opteron_G2', 'smp': '1', 'pauseCode': 'NOERR', 'migrationDest':
'libvirt', 'custom':
{'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104bedevice_1459cc81-2f88-45dd-ae3c-0c030a084d2e':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=1459cc81-2f88-45dd-ae3c-0c030a084d2e, device=unix,
type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0,
controller=0, type=virtio-serial}, managed=false, plugged=true,
readOnly=false, deviceAlias=channel0, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104bedevice_1459cc81-2f88-45dd-ae3c-0c030a084d2edevice_5bea8f60-7de6-4c93-99ff-af67635e2af6device_8d086706-28b6-40fa-a86d-3733c9ebe5c1':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=8d086706-28b6-40fa-a86d-3733c9ebe5c1, device=spicevmc,
type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0,
controller=0, type=virtio-serial}, managed=false, plugged=true,
readOnly=false, deviceAlias=channel2, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19e':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=db19ff20-8f6d-488c-bad8-19e33b01f19e, device=ide,
type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00,
domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false,
plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104be':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=9c0a4779-62bc-414b-abb5-a2fab36104be, device=virtio-serial,
type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00,
domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false,
plugged=true, readOnly=false, deviceAlias=virtio-serial0,
customProperties={}}', 'device_a5fa5a6d-f191-42d2-be55-d6227499c658':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=a5fa5a6d-f191-42d2-be55-d6227499c658, device=scsi,
type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00,
domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false,
plugged=true, readOnly=false, deviceAlias=scsi0,
customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104bedevice_1459cc81-2f88-45dd-ae3c-0c030a084d2edevice_5bea8f60-7de6-4c93-99ff-af67635e2af6':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=5bea8f60-7de6-4c93-99ff-af67635e2af6, device=unix,
type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0,
controller=0, type=virtio-serial}, managed=false, plugged=true,
readOnly=false, deviceAlias=channel1, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=bd099ee2-faee-41c3-9e01-0a01f458abe9, device=disk, type=DISK,
bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000,
type=pci, slot=0x06, function=0x0}, managed=false, plugged=true,
readOnly=false, deviceAlias=virtio-disk0, customProperties={}}'},
'vmType': 'kvm', 'spiceSslCipherSuite': 'DEFAULT', 'memSize': 1024,
'vmName': 'c8again32', 'nice': '0', 'username': 'Unknown', 'vmId':
'd54660a2-45ed-41ae-ab99-a6f93ebbdbb1', 'displayIp': '0',
'keyboardLayout': 'en-us', 'displayPort': '-1', 'smartcardEnable':
'false', 'guestIPs': '', 'nicModel': 'rtl8139,pv',
'smpCoresPerSocket': '1', 'kvmEnable': 'true', 'pitReinjection':
'false', 'devices': [{'device': 'unix', 'alias': 'channel0', 'type':
'channel', 'address': {'bus': '0', 'controller': '0', 'type':
'virtio-serial', 'port': '1'}}, {'device': 'unix', 'alias':
'channel1', 'type': 'channel', 'address': {'bus': '0', 'controller':
'0', 'type': 'virtio-serial', 'port': '2'}}, {'device': 'spicevmc',
'alias': 'channel2', 'type': 'channel', 'address': {'bus': '0',
'controller': '0', 'type': 'virtio-serial', 'port': '3'}}, {'device':
'scsi', 'alias': 'scsi0', 'model': 'virtio-scsi', 'type':
'controller', 'address': {'slot': '0x04', 'bus': '0x00', 'domain':
'0x0000', 'type': 'pci', 'function': '0x0'}}, {'device': 'usb',
'alias': 'usb0', 'type': 'controller', 'address': {'slot': '0x01',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}},
{'device': 'ide', 'alias': 'ide0', 'type': 'controller', 'address':
{'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci',
'function': '0x1'}}, {'device': 'virtio-serial', 'alias':
'virtio-serial0', 'type': 'controller', 'address': {'slot': '0x05',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}},
{'specParams': {'vram': '65536'}, 'alias': 'video0', 'deviceId':
'c4763c67-c69b-4b7c-b0cd-ac5beca82669', 'address': {'slot': '0x02',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'},
'device': 'qxl', 'type': 'video'}, {'nicModel': 'pv', 'macAddr':
'00:1a:4a:8f:04:f7', 'linkActive': True, 'network': 'ovirtmgmt',
'specParams': {}, 'custom': {}, 'filter': 'vdsm-no-mac-spoofing',
'alias': 'net0', 'deviceId': '4478616d-d9d4-4284-8e6b-2cce0d330504',
'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type':
'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface',
'name': 'vnet0'}, {'index': '2', 'iface': 'ide', 'name': 'hdc',
'alias': 'ide0-1-0', 'shared': 'false', 'specParams': {'path': ''},
'readonly': 'True', 'deviceId':
'a3d3a41d-7f47-4a64-b3fe-10f33312f929', 'address': {'bus': '1',
'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'},
'device': 'cdrom', 'path': '', 'type': 'disk'}, {'index': 0, 'iface':
'virtio', 'volumeInfo': {'volPort': '0', 'volType': 'network',
'volfileServer': 'f18ovn01.mydomain', 'volTransport': 'tcp',
'protocol': 'gluster', 'path':
'gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49'},
'format': 'raw', 'shared': 'false', 'optional': 'false', 'volumeID':
'a123801a-0a4d-4a47-a426-99d8480d2e49', 'propagateErrors': 'off',
'imageID': '15f9ca1c-c435-4892-9eb7-0c84583b2a7d', 'specParams': {},
'readonly': 'false', 'domainID':
'd0b96d4a-62aa-4e9f-b50e-f7a0cb5be291', 'deviceId':
'15f9ca1c-c435-4892-9eb7-0c84583b2a7d', 'truesize': '4337233920',
'poolID': 'eb679feb-4da2-4fd0-a185-abbe459ffa70', 'device': 'disk',
'reqsize': '0', 'path':
'/rhev/data-center/eb679feb-4da2-4fd0-a185-abbe459ffa70/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49',
'apparentsize': '10737418240', 'type': 'disk', 'volumeChain':
[{'path': '/rhev/data-center/eb679feb-4da2-4fd0-a185-abbe459ffa70/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49',
'domainID': 'd0b96d4a-62aa-4e9f-b50e-f7a0cb5be291', 'vmVolInfo':
{'volPort': '0', 'volType': 'network', 'volfileServer':
'f18ovn01.mydomain', 'volTransport': 'tcp', 'protocol': 'gluster',
'path': 'gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49'},
'volumeID': 'a123801a-0a4d-4a47-a426-99d8480d2e49', 'imageID':
'15f9ca1c-c435-4892-9eb7-0c84583b2a7d'}]}, {'index': '0', 'iface':
'pci', 'name': 'vda', 'alias': 'virtio-disk0', 'readonly': 'False',
'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type':
'pci', 'function': '0x0'}, 'device': 'disk', 'path':
'gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49',
'type': 'disk'}, {'target': 1048576, 'specParams': {'model':
'virtio'}, 'alias': 'balloon0', 'deviceId':
'216acfca-9672-4390-91b0-feb7aa201484', 'address': {'slot': '0x07',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'},
'device': 'memballoon', 'type': 'balloon'}], 'status': 'Migration
Destination', 'clientIp': '', 'display': 'qxl'}}
Thread-3396::DEBUG::2013-10-08
01:20:54,554::BindingXMLRPC::979::vds::(wrapper) client
[10.4.4.60]::call vmGetStats with
('d54660a2-45ed-41ae-ab99-a6f93ebbdbb1',) {}
Thread-3396::DEBUG::2013-10-08
01:20:54,555::BindingXMLRPC::986::vds::(wrapper) return vmGetStats
with {'status': {'message': 'Done', 'code': 0}, 'statsList':
[{'status': 'Migration Destination', 'hash': '0', 'acpiEnable':
'true', 'pid': '0', 'displayIp': '0', 'displayPort': '-1',
'displaySecurePort': '-1', 'timeOffset': '-114', 'pauseCode': 'NOERR',
'kvmEnable': 'true', 'network': {}, 'vmId':
'd54660a2-45ed-41ae-ab99-a6f93ebbdbb1', 'displayType': 'qxl', 'disks':
{}, 'monitorResponse': '0', 'elapsedTime': '178', 'vmType': 'kvm',
'clientIp': ''}]}
Thread-3397::DEBUG::2013-10-08
01:20:54,885::BindingXMLRPC::979::vds::(wrapper) client
[10.4.4.59]::call vmDestroy with
('d54660a2-45ed-41ae-ab99-a6f93ebbdbb1',) {}
Thread-3397::INFO::2013-10-08 01:20:54,886::API::317::vds::(destroy)
vmContainerLock acquired by vm d54660a2-45ed-41ae-ab99-a6f93ebbdbb1
Thread-3397::DEBUG::2013-10-08
01:20:54,886::vm::4258::vm.Vm::(destroy)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::destroy Called
Thread-3397::INFO::2013-10-08
01:20:54,887::vm::4204::vm.Vm::(releaseVm)
vmId=`d54660a2-45ed-41ae-ab99-a6f93ebbdbb1`::Release VM resources
Thread-3397::WARNING::2013-10-08
01:20:54,887::clientIF::337::vds::(teardownVolumePath) Drive is not a
vdsm image: VOLWM_CHUNK_MB:1024 VOLWM_CHUNK_REPLICATE_MULT:2
VOLWM_FREE_PCT:50 _blockDev:False _checkIoTuneCategories:<bound method
Drive._checkIoTuneCategories of <vm.Drive object at 0x7fae4415bc90>>
_customize:<bound method Drive._customize of <vm.Drive object at
0x7fae4415bc90>> _deviceXML:<disk device="cdrom" type="file">
<driver name="qemu" type="raw"/>
<source startupPolicy="optional"/>
<target bus="ide" dev="hdc"/>
<readonly/>
<serial/>
<alias name="ide0-1-0"/>
<address bus="1" controller="0" target="0" type="drive" unit="0"/>
</disk> _makeName:<bound method Drive._makeName of <vm.Drive
object at 0x7fae4415bc90>> _validateIoTuneParams:<bound method
Drive._validateIoTuneParams of <vm.Drive object at 0x7fae4415bc90>>
address:{'bus': '1', 'controller': '0', 'type': 'drive', 'target':
'0', 'unit': '0'} alias:ide0-1-0 apparentsize:0 blockDev:False
cache:none conf:{'guestFQDN': '', 'acpiEnable': 'true',
'emulatedMachine': 'pc-1.0', 'afterMigrationStatus': 'Up',
'spiceSecureChannels':
'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard',
'pid': '0', 'memGuaranteedSize': 1024, 'transparentHugePages': 'true',
'displaySecurePort': '-1', 'timeOffset': '-114', 'cpuType':
'Opteron_G2', 'smp': '1', 'pauseCode': 'NOERR', 'migrationDest':
'libvirt', 'custom':
{'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104bedevice_1459cc81-2f88-45dd-ae3c-0c030a084d2e':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=1459cc81-2f88-45dd-ae3c-0c030a084d2e, device=unix,
type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0,
controller=0, type=virtio-serial}, managed=false, plugged=true,
readOnly=false, deviceAlias=channel0, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104bedevice_1459cc81-2f88-45dd-ae3c-0c030a084d2edevice_5bea8f60-7de6-4c93-99ff-af67635e2af6device_8d086706-28b6-40fa-a86d-3733c9ebe5c1':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=8d086706-28b6-40fa-a86d-3733c9ebe5c1, device=spicevmc,
type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0,
controller=0, type=virtio-serial}, managed=false, plugged=true,
readOnly=false, deviceAlias=channel2, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19e':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=db19ff20-8f6d-488c-bad8-19e33b01f19e, device=ide,
type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00,
domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false,
plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104be':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=9c0a4779-62bc-414b-abb5-a2fab36104be, device=virtio-serial,
type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00,
domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false,
plugged=true, readOnly=false, deviceAlias=virtio-serial0,
customProperties={}}', 'device_a5fa5a6d-f191-42d2-be55-d6227499c658':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=a5fa5a6d-f191-42d2-be55-d6227499c658, device=scsi,
type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00,
domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false,
plugged=true, readOnly=false, deviceAlias=scsi0,
customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9device_db19ff20-8f6d-488c-bad8-19e33b01f19edevice_9c0a4779-62bc-414b-abb5-a2fab36104bedevice_1459cc81-2f88-45dd-ae3c-0c030a084d2edevice_5bea8f60-7de6-4c93-99ff-af67635e2af6':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=5bea8f60-7de6-4c93-99ff-af67635e2af6, device=unix,
type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0,
controller=0, type=virtio-serial}, managed=false, plugged=true,
readOnly=false, deviceAlias=channel1, customProperties={}}',
'device_a5fa5a6d-f191-42d2-be55-d6227499c658device_bd099ee2-faee-41c3-9e01-0a01f458abe9':
'VmDevice {vmId=d54660a2-45ed-41ae-ab99-a6f93ebbdbb1,
deviceId=bd099ee2-faee-41c3-9e01-0a01f458abe9, device=disk, type=DISK,
bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000,
type=pci, slot=0x06, function=0x0}, managed=false, plugged=true,
readOnly=false, deviceAlias=virtio-disk0, customProperties={}}'},
'vmType': 'kvm', 'spiceSslCipherSuite': 'DEFAULT', 'memSize': 1024,
'vmName': 'c8again32', 'nice': '0', 'username': 'Unknown', 'vmId':
'd54660a2-45ed-41ae-ab99-a6f93ebbdbb1', 'displayIp': '0',
'keyboardLayout': 'en-us', 'displayPort': '-1', 'smartcardEnable':
'false', 'guestIPs': '', 'nicModel': 'rtl8139,pv',
'smpCoresPerSocket': '1', 'kvmEnable': 'true', 'pitReinjection':
'false', 'devices': [{'device': 'unix', 'alias': 'channel0', 'type':
'channel', 'address': {'bus': '0', 'controller': '0', 'type':
'virtio-serial', 'port': '1'}}, {'device': 'unix', 'alias':
'channel1', 'type': 'channel', 'address': {'bus': '0', 'controller':
'0', 'type': 'virtio-serial', 'port': '2'}}, {'device': 'spicevmc',
'alias': 'channel2', 'type': 'channel', 'address': {'bus': '0',
'controller': '0', 'type': 'virtio-serial', 'port': '3'}}, {'device':
'scsi', 'alias': 'scsi0', 'model': 'virtio-scsi', 'type':
'controller', 'address': {'slot': '0x04', 'bus': '0x00', 'domain':
'0x0000', 'type': 'pci', 'function': '0x0'}}, {'device': 'usb',
'alias': 'usb0', 'type': 'controller', 'address': {'slot': '0x01',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x2'}},
{'device': 'ide', 'alias': 'ide0', 'type': 'controller', 'address':
{'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci',
'function': '0x1'}}, {'device': 'virtio-serial', 'alias':
'virtio-serial0', 'type': 'controller', 'address': {'slot': '0x05',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}},
{'specParams': {'vram': '65536'}, 'alias': 'video0', 'deviceId':
'c4763c67-c69b-4b7c-b0cd-ac5beca82669', 'address': {'slot': '0x02',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'},
'device': 'qxl', 'type': 'video'}, {'nicModel': 'pv', 'macAddr':
'00:1a:4a:8f:04:f7', 'linkActive': True, 'network': 'ovirtmgmt',
'specParams': {}, 'custom': {}, 'filter': 'vdsm-no-mac-spoofing',
'alias': 'net0', 'deviceId': '4478616d-d9d4-4284-8e6b-2cce0d330504',
'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type':
'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface',
'name': 'vnet0'}, {'index': '2', 'iface': 'ide', 'name': 'hdc',
'alias': 'ide0-1-0', 'shared': 'false', 'specParams': {'path': ''},
'readonly': 'True', 'deviceId':
'a3d3a41d-7f47-4a64-b3fe-10f33312f929', 'address': {'bus': '1',
'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'},
'device': 'cdrom', 'path': '', 'type': 'disk'}, {'index': 0, 'iface':
'virtio', 'volumeInfo': {'volPort': '0', 'volType': 'network',
'volfileServer': 'f18ovn01.mydomain', 'volTransport': 'tcp',
'protocol': 'gluster', 'path':
'gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49'},
'format': 'raw', 'shared': 'false', 'optional': 'false', 'volumeID':
'a123801a-0a4d-4a47-a426-99d8480d2e49', 'propagateErrors': 'off',
'imageID': '15f9ca1c-c435-4892-9eb7-0c84583b2a7d', 'specParams': {},
'readonly': 'false', 'domainID':
'd0b96d4a-62aa-4e9f-b50e-f7a0cb5be291', 'deviceId':
'15f9ca1c-c435-4892-9eb7-0c84583b2a7d', 'truesize': '4337233920',
'poolID': 'eb679feb-4da2-4fd0-a185-abbe459ffa70', 'device': 'disk',
'reqsize': '0', 'path':
'/rhev/data-center/eb679feb-4da2-4fd0-a185-abbe459ffa70/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49',
'apparentsize': '10737418240', 'type': 'disk', 'volumeChain':
[{'path': '/rhev/data-center/eb679feb-4da2-4fd0-a185-abbe459ffa70/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49',
'domainID': 'd0b96d4a-62aa-4e9f-b50e-f7a0cb5be291', 'vmVolInfo':
{'volPort': '0', 'volType': 'network', 'volfileServer':
'f18ovn01.mydomain', 'volTransport': 'tcp', 'protocol': 'gluster',
'path': 'gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49'},
'volumeID': 'a123801a-0a4d-4a47-a426-99d8480d2e49', 'imageID':
'15f9ca1c-c435-4892-9eb7-0c84583b2a7d'}]}, {'index': '0', 'iface':
'pci', 'name': 'vda', 'alias': 'virtio-disk0', 'readonly': 'False',
'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type':
'pci', 'function': '0x0'}, 'device': 'disk', 'path':
'gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/15f9ca1c-c435-4892-9eb7-0c84583b2a7d/a123801a-0a4d-4a47-a426-99d8480d2e49',
'type': 'disk'}, {'target': 1048576, 'specParams': {'model':
'virtio'}, 'alias': 'balloon0', 'deviceId':
'216acfca-9672-4390-91b0-feb7aa201484', 'address': {'slot': '0x07',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'},
'device': 'memballoon', 'type': 'balloon'}], 'status': 'Migration
Destination', 'clientIp': '', 'display': 'qxl'} createXmlElem:<bound
method Drive.createXmlElem of <vm.Drive object at 0x7fae4415bc90>>
device:cdrom deviceId:a3d3a41d-7f47-4a64-b3fe-10f33312f929
getNextVolumeSize:<bound method Drive.getNextVolumeSize of <vm.Drive
object at 0x7fae4415bc90>> getXML:<bound method Drive.getXML of
<vm.Drive object at 0x7fae4415bc90>> iface:ide index:2
isDiskReplicationInProgress:<bound method
Drive.isDiskReplicationInProgress of <vm.Drive object at
0x7fae4415bc90>> isVdsmImage:<bound method Drive.isVdsmImage of
<vm.Drive object at 0x7fae4415bc90>> log:<logUtils.SimpleLogAdapter
object at 0x7fae2058f1d0> name:hdc networkDev:False path:
readonly:True reqsize:0 serial: shared:false specParams:{'path': ''}
truesize:0 type:disk volExtensionChunk:1024 watermarkLimit:536870912
Traceback (most recent call last):
File "/usr/share/vdsm/clientIF.py", line 331, in teardownVolumePath
res = self.irs.teardownImage(drive['domainID'],
File "/usr/share/vdsm/vm.py", line 1344, in __getitem__
raise KeyError(key)
KeyError: 'domainID'
Thread-3393::ERROR::2013-10-08
01:20:54,888::libvirtconnection::94::libvirtconnection::(wrapper)
connection to libvirt broken. ecode: 1 edom: 7
Thread-3393::ERROR::2013-10-08
01:20:54,892::libvirtconnection::96::libvirtconnection::(wrapper)
taking calling process down.
firewall on destination seems ok:
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 192.168.3.3 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state
RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:54321
...
More information about the Users
mailing list