Re: [Users] Migration Failed

Hi Dafna, Thanks for the reply. Attached is the log from the source server (node03). I'll reply to your other questions as soon as I'm back in the office this afternoon, have to run off to a meeting. Regards. Neil Wilson. On Tue, Jan 7, 2014 at 8:13 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: > Ok... several things :) > > 1. for migration we need to see vdsm logs from both src and dst. > > 2. Is it possible that the vm has an iso attached? because I see that you > are having problems with the iso domain: > > 2014-01-07 14:26:27,714 ERROR > [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] > (pool-6-thread-48) Domain e9ab725d-69c1-4a59-b225-b995d095c289:bla-iso was > reported with error code 358 > > > Thread-1165153::DEBUG::2014-01-07 > 13:39:42,460::libvirtconnection::108::libvirtconnection::(wrapper) Unknown > libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no > domain with matching uuid '63da7faa-f92a-4652-90f2-b6660a4fb7b3' > > hread-19::ERROR::2014-01-07 > 13:01:02,621::sdc::143::Storage.StorageDomainCache::(_findDomain) domain > e9ab725d-69c1-4a59-b225-b995d095c289 not found > Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) > StorageDomainDoesNotExist: Storage domain does not exist: > (u'e9ab725d-69c1-4a59-b225-b995d095c289',) > Thread-19::ERROR::2014-01-07 > 13:01:02,622::domainMonitor::225::Storage.DomainMonitorThread::(_monitorDomain) > Error while collecting domain e9ab725d-69c1-4a59-b225-b995d095c289 > monitoring information > Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 190, in > _monitorDomain > self.domain = sdCache.produce(self.sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 98, in produce > domain.getRealDomain() > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) > StorageDomainDoesNotExist: Storage domain does not exist: > (u'e9ab725d-69c1-4a59-b225-b995d095c289',) > Dummy-29013::DEBUG::2014-01-07 > 13:01:03,507::storage_mailbox::733::Storage.Misc.excCmd::(_checkForMail) 'dd > if=/rhev/data-center/28adaf38-a4f6-11e1-a859-cb68949043e4/mastersd/dom_md/inbox > iflag=direct,fullblock count=1 bs=1024000' (cwd N > one) > > 3. The migration fails with libvirt error but we need the trace from the > second log: > > Thread-1165153::DEBUG::2014-01-07 13:39:42,451::sampling::292::vm.Vm::(stop) > vmId=`63da7faa-f92a-4652-90f2-b6660a4fb7b3`::Stop statistics collection > Thread-1163583::DEBUG::2014-01-07 13:39:42,452::sampling::323::vm.Vm::(run) > vmId=`63da7faa-f92a-4652-90f2-b6660a4fb7b3`::Stats thread finished > Thread-1165153::DEBUG::2014-01-07 > 13:39:42,460::libvirtconnection::108::libvirtconnection::(wrapper) Unknown > libvirterror: ecode: 42 edom: 10 level: 2 message: Domain not found: no > domain with matching uuid '63da7faa-f92a-4652-90f2-b6660 > a4fb7b3' > > > 4. But I am worried about this and would more info about this vm... > > Thread-247::ERROR::2014-01-07 15:35:14,868::sampling::355::vm.Vm::(collect) > vmId=`63da7faa-f92a-4652-90f2-b6660a4fb7b3`::Stats function failed: > <AdvancedStatsFunction _highWrite at 0x2ce0998> > Traceback (most recent call last): > File "/usr/share/vdsm/sampling.py", line 351, in collect > statsFunction() > File "/usr/share/vdsm/sampling.py", line 226, in __call__ > retValue = self._function(*args, **kwargs) > File "/usr/share/vdsm/vm.py", line 509, in _highWrite > if not vmDrive.blockDev or vmDrive.format != 'cow': > AttributeError: 'Drive' object has no attribute 'format' > > How did you create this vm? was it from the UI? was it from a script? what > are the parameters you used? > > Thanks, > > Dafna > > > > On 01/07/2014 04:34 PM, Neil wrote: >> Hi Elad, >> >> Thanks for assisting me, yes the same condition exists, if I try to >> migrate Tux it says "The VM Tux is being migrated". >> >> >> Below are the details requested. >> >> >> [root@node01 ~]# virsh -r list >> Id Name State >> ---------------------------------------------------- >> 1 adam running >> >> [root@node01 ~]# pgrep qemu >> 11232 >> [root@node01 ~]# vdsClient -s 0 list table >> 63da7faa-f92a-4652-90f2-b6660a4fb7b3 11232 adam Up >> >> >> [root@node03 ~]# virsh -r list >> Id Name State >> ---------------------------------------------------- >> 7 tux running >> >> [root@node03 ~]# pgrep qemu >> 32333 >> [root@node03 ~]# vdsClient -s 0 list table >> 2736197b-6dc3-4155-9a29-9306ca64881d 32333 tux Up >> >> Thanks. >> >> Regards. >> >> Neil Wilson. >> >> >> On Tue, Jan 7, 2014 at 4:43 PM, Elad Ben Aharon <ebenahar(a)redhat.com&gt; >> wrote: >>> Is it still in the same condition? >>> If yes, please add the outputs from both hosts for: >>> >>> #virsh -r list >>> #pgrep qemu >>> #vdsClient -s 0 list table (or 'vdsClient 0 list table' if you are >>> working in insecure mode) >>> >>> >>> Thnaks, >>> >>> Elad Ben Aharon >>> RHEV-QE storage team >>> >>> >>> >>> >>> ----- Original Message ----- >>> From: "Neil" <nwilson123(a)gmail.com&gt; >>> To: users(a)ovirt.org >>> Sent: Tuesday, January 7, 2014 4:21:43 PM >>> Subject: [Users] Migration Failed >>> >>> Hi guys, >>> >>> I've tried to migrate a VM from one host(node03) to another(node01), >>> and it failed to migrate, and the VM(tux) remained on the original >>> host. I've now tried to migrate the same VM again, and it picks up >>> that the previous migration is still in progress and refuses to >>> migrate. >>> >>> I've checked for the KVM process on each of the hosts and the VM is >>> definitely still running on node03 so there doesn't appear to be any >>> chance of the VM trying to run on both hosts (which I've had before >>> which is very scary). >>> >>> These are my versions... and attached are my engine.log and my vdsm.log >>> >>> Centos 6.5 >>> ovirt-iso-uploader-3.3.1-1.el6.noarch >>> ovirt-host-deploy-1.1.2-1.el6.noarch >>> ovirt-release-el6-9-1.noarch >>> ovirt-engine-setup-3.3.1-2.el6.noarch >>> ovirt-engine-3.3.1-2.el6.noarch >>> ovirt-host-deploy-java-1.1.2-1.el6.noarch >>> ovirt-image-uploader-3.3.1-1.el6.noarch >>> ovirt-engine-dbscripts-3.3.1-2.el6.noarch >>> ovirt-engine-cli-3.3.0.6-1.el6.noarch >>> ovirt-engine-websocket-proxy-3.3.1-2.el6.noarch >>> ovirt-engine-userportal-3.3.1-2.el6.noarch >>> ovirt-log-collector-3.3.1-1.el6.noarch >>> ovirt-engine-tools-3.3.1-2.el6.noarch >>> ovirt-engine-lib-3.3.1-2.el6.noarch >>> ovirt-engine-webadmin-portal-3.3.1-2.el6.noarch >>> ovirt-engine-backend-3.3.1-2.el6.noarch >>> ovirt-engine-sdk-python-3.3.0.8-1.el6.noarch >>> ovirt-engine-restapi-3.3.1-2.el6.noarch >>> >>> >>> vdsm-python-4.13.0-11.el6.x86_64 >>> vdsm-cli-4.13.0-11.el6.noarch >>> vdsm-xmlrpc-4.13.0-11.el6.noarch >>> vdsm-4.13.0-11.el6.x86_64 >>> vdsm-python-cpopen-4.13.0-11.el6.x86_64 >>> >>> I've had a few issues with this particular installation in the past, >>> as it's from a very old pre release of ovirt, then upgrading to the >>> dreyou repo, then finally moving to the official Centos ovirt repo. >>> >>> Thanks, any help is greatly appreciated. >>> >>> Regards. >>> >>> Neil Wilson. >>> >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users > > > -- > Dafna Ron