Re: [Users] All VMs disappeared

----- Original Message ----- > From: "Itamar Heim" <iheim(a)redhat.com&gt; > To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; > Cc: "Michal Skrivanek" <mskrivan(a)redhat.com&gt;, users(a)ovirt.org > Sent: Wednesday, August 14, 2013 4:27:17 PM > Subject: Re: [Users] All VMs disappeared > > On 08/14/2013 05:19 PM, Jakub Bittner wrote: > > Dne 14.8.2013 10:48, Jakub Bittner napsal(a): > >> Dne 14.8.2013 09:57, Liron Aravot napsal(a): > >>> Jakub, are only your vms disappear? what about the disks? > >>> > >>> ----- Original Message ----- > >>>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; > >>>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; > >>>> Cc: users(a)ovirt.org > >>>> Sent: Wednesday, August 14, 2013 10:39:48 AM > >>>> Subject: Re: [Users] All VMs disappeared > >>>> > >>>> > >>>> > >>>> ----- Original Message ----- > >>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; > >>>>> To: users(a)ovirt.org > >>>>> Sent: Wednesday, August 14, 2013 8:09:45 AM > >>>>> Subject: Re: [Users] All VMs disappeared > >>>>> > >>>>> Dne 13.8.2013 18:29, Laszlo Hornyak napsal(a): > >>>>>> Ahoj Jakub, > >>>>>> > >>>>>> Just one more idea: could you turn statement logging on in postgresql > >>>>>> before you try again? > >>>>>> It is in /var/lib/pgsql/data/postgresql.conf > >>>>>> log_statement = 'all' > >>>>>> > >>>>>> > >>>>>> ----- Original Message ----- > >>>>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; > >>>>>>> To: "Itamar Heim" <iheim(a)redhat.com&gt; > >>>>>>> Cc: users(a)ovirt.org > >>>>>>> Sent: Tuesday, August 13, 2013 5:20:29 PM > >>>>>>> Subject: Re: [Users] All VMs disappeared > >>>>>>> > >>>>>>> Dne 13.8.2013 17:02, Itamar Heim napsal(a): > >>>>>>>> On 08/13/2013 05:31 PM, Jakub Bittner wrote: > >>>>>>>>> Dne 13.8.2013 15:39, Itamar Heim napsal(a): > >>>>>>>>>> On 08/13/2013 08:50 AM, Jakub Bittner wrote: > >>>>>>>>>>> Dne 12.8.2013 22:44, Itamar Heim napsal(a): > >>>>>>>>>>>> On 08/12/2013 06:59 PM, Jakub Bittner wrote: > >>>>>>>>>>>>> Dne 12.8.2013 14:54, Jakub Bittner napsal(a): > >>>>>>>>>>>>>> Dne 12.8.2013 14:29, Laszlo Hornyak napsal(a): > >>>>>>>>>>>>>>> I looked around Noam's patch and that should not cause such > >>>>>>>>>>>>>>> behavior. > >>>>>>>>>>>>>>> I am wondering how that lost VM's could happen. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Jakub, can you give a more detailed description what you > >>>>>>>>>>>>>>> were > >>>>>>>>>>>>>>> doing > >>>>>>>>>>>>>>> with oVirt when this happened? Maybe the bug is still there. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thank you, > >>>>>>>>>>>>>>> Laszlo > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> ----- Original Message ----- > >>>>>>>>>>>>>>>> From: "Greg Sheremeta" <gshereme(a)redhat.com&gt; > >>>>>>>>>>>>>>>> To: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; > >>>>>>>>>>>>>>>> Cc: "Jakub Bittner" <j.bittner(a)nbu.cz&gt;, "Noam Slomianko" > >>>>>>>>>>>>>>>> <nslomian(a)redhat.com&gt;, users(a)ovirt.org > >>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 1:23:40 PM > >>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Not the one I fixed, 987907. It was a simple UI > >>>>>>>>>>>>>>>> NullPointerException. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Greg > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> ----- Original Message ----- > >>>>>>>>>>>>>>>>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; > >>>>>>>>>>>>>>>>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt;, "Noam Slomianko" > >>>>>>>>>>>>>>>>> <nslomian(a)redhat.com&gt;, "Greg Sheremeta" > >>>>>>>>>>>>>>>>> <gshereme(a)redhat.com&gt; > >>>>>>>>>>>>>>>>> Cc: users(a)ovirt.org > >>>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 7:21:26 AM > >>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Well if they are no longer in DB then that explains why > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> exception no > >>>>>>>>>>>>>>>>> longer occurs, but at the cost of database corruption. > >>>>>>>>>>>>>>>>> Noam and Greg, can these bugs cause data corruption? > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thank you, > >>>>>>>>>>>>>>>>> Laszlo > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> ----- Original Message ----- > >>>>>>>>>>>>>>>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; > >>>>>>>>>>>>>>>>>> To: "Greg Sheremeta" <gshereme(a)redhat.com&gt; > >>>>>>>>>>>>>>>>>> Cc: users(a)ovirt.org, "Laszlo Hornyak" > >>>>>>>>>>>>>>>>>> <lhornyak(a)redhat.com&gt; > >>>>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 9:05:09 AM > >>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Dne 10.8.2013 01:54, Greg Sheremeta napsal(a): > >>>>>>>>>>>>>>>>>>> It could also be this bug[1], for which I just > >>>>>>>>>>>>>>>>>>> submitted a > >>>>>>>>>>>>>>>>>>> fix. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=987907 > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> You can work around it by typing just "VMs:" (without > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> quotes) in > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> search bar. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Greg > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> ----- Original Message ----- > >>>>>>>>>>>>>>>>>>>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; > >>>>>>>>>>>>>>>>>>>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; > >>>>>>>>>>>>>>>>>>>> Cc: users(a)ovirt.org > >>>>>>>>>>>>>>>>>>>> Sent: Friday, August 9, 2013 11:21:26 AM > >>>>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Hi Jakub, > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Could you check through DB or REST-API if the VM's > >>>>>>>>>>>>>>>>>>>> are in > >>>>>>>>>>>>>>>>>>>> your > >>>>>>>>>>>>>>>>>>>> DB? > >>>>>>>>>>>>>>>>>>>> select * from vm_static; > >>>>>>>>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>> curl -u admin@internal:blablabla [engine-url]api/vms > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> It seems Noam fixed this issue already in > >>>>>>>>>>>>>>>>>>>> c2295c31fa645e1ba1b94cd557bd1fecb40c8829. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thank you, > >>>>>>>>>>>>>>>>>>>> Laszlo > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> ----- Original Message ----- > >>>>>>>>>>>>>>>>>>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; > >>>>>>>>>>>>>>>>>>>>> To: users(a)ovirt.org > >>>>>>>>>>>>>>>>>>>>> Sent: Friday, August 9, 2013 1:06:53 PM > >>>>>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Dne 9.8.2013 09:48, Jakub Bittner napsal(a): > >>>>>>>>>>>>>>>>>>>>>> Hello, > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Iam running ovirt 3.3.0.beta1 on centos 6,4 and > >>>>>>>>>>>>>>>>>>>>>> all our > >>>>>>>>>>>>>>>>>>>>>> VMs > >>>>>>>>>>>>>>>>>>>>>> disappeared from VMs tab. Nodes running on centos > >>>>>>>>>>>>>>>>>>>>>> too. > >>>>>>>>>>>>>>>>>>>>>> Repeating > >>>>>>>>>>>>>>>>>>>>>> problem in log is: > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> 2013-08-09 09:44:24,203 WARN > >>>>>>>>>>>>>>>>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager] > >>>>>>>>>>>>>>>>>>>>>> (DefaultQuartzScheduler_Worker-45) Failed to > >>>>>>>>>>>>>>>>>>>>>> refresh VDS > >>>>>>>>>>>>>>>>>>>>>> , > >>>>>>>>>>>>>>>>>>>>>> vds = > >>>>>>>>>>>>>>>>>>>>>> 7cb6aedf-47bc-40b0-877f-2a537fca5c64 : > >>>>>>>>>>>>>>>>>>>>>> node2.x.com, error > >>>>>>>>>>>>>>>>>>>>>> = > >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException, continuing.: > >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown > >>>>>>>>>>>>>>>>>>>>>> Source) [:1.7.0_25] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> java.lang.reflect.Method.invoke(Method.java:606) > >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [scheduler.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> org.quartz.core.JobRunShell.run(JobRunShell.java:213) > >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> 2013-08-09 09:44:27,039 WARN > >>>>>>>>>>>>>>>>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager] > >>>>>>>>>>>>>>>>>>>>>> (DefaultQuartzScheduler_Worker-47) Failed to > >>>>>>>>>>>>>>>>>>>>>> refresh VDS > >>>>>>>>>>>>>>>>>>>>>> , > >>>>>>>>>>>>>>>>>>>>>> vds = > >>>>>>>>>>>>>>>>>>>>>> 653e34a6-6b7e-4757-9042-dc02e7a0cddf : > >>>>>>>>>>>>>>>>>>>>>> node3.x.com, error > >>>>>>>>>>>>>>>>>>>>>> = > >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException, continuing.: > >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown > >>>>>>>>>>>>>>>>>>>>>> Source) [:1.7.0_25] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> java.lang.reflect.Method.invoke(Method.java:606) > >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [scheduler.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> org.quartz.core.JobRunShell.run(JobRunShell.java:213) > >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] > >>>>>>>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>>>>> > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] > >>>>>>>>>>>>>>>>>>>>>> _______________________________________________ > >>>>>>>>>>>>>>>>>>>>>> Users mailing list > >>>>>>>>>>>>>>>>>>>>>> Users(a)ovirt.org > >>>>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>>>>>>>>>>>>>>>>>> I restarted management node and both nodes and this > >>>>>>>>>>>>>>>>>>>>> problem > >>>>>>>>>>>>>>>>>>>>> disappeared. > >>>>>>>>>>>>>>>>>>>>> But I still can not see any VMs. > >>>>>>>>>>>>>>>>>>>>> _______________________________________________ > >>>>>>>>>>>>>>>>>>>>> Users mailing list > >>>>>>>>>>>>>>>>>>>>> Users(a)ovirt.org > >>>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> _______________________________________________ > >>>>>>>>>>>>>>>>>>>> Users mailing list > >>>>>>>>>>>>>>>>>>>> Users(a)ovirt.org > >>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I do not have them in database. Both commands shows > >>>>>>>>>>>>>>>>>> nothing > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hello, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> we run ovirt 3.2.1 (or 2). I upgraded to RC1, everything was > >>>>>>>>>>>>>> great > >>>>>>>>>>>>>> (except SPICE. VNC and creating new WM due to serialization > >>>>>>>>>>>>>> bug.). > >>>>>>>>>>>>>> Because those bugs were irritating, we switched to > >>>>>>>>>>>>>> nightly. Still > >>>>>>>>>>>>>> works fine (but bug with VMs is not fixed there), than I > >>>>>>>>>>>>>> installed > >>>>>>>>>>>>>> new > >>>>>>>>>>>>>> ovirt engine management server in KVM virtualization (using > >>>>>>>>>>>>>> libvirtd) > >>>>>>>>>>>>>> due to taking snapshots before deploying nightly builds. New > >>>>>>>>>>>>>> virtualized engine works great and I delete old one. Next > >>>>>>>>>>>>>> day I > >>>>>>>>>>>>>> came > >>>>>>>>>>>>>> to work and engine web page was very very slow, every > >>>>>>>>>>>>>> operation > >>>>>>>>>>>>>> takes > >>>>>>>>>>>>>> ages, that time I found there were no VMs, no HDDs and so. I > >>>>>>>>>>>>>> converted > >>>>>>>>>>>>>> its qcow2 image to raw, because I think it has slow DB > >>>>>>>>>>>>>> because of > >>>>>>>>>>>>>> running i qcow2. Did not helped, than I take a look in nodes > >>>>>>>>>>>>>> logs and > >>>>>>>>>>>>>> found this problem, which I posted in first email (VMs still > >>>>>>>>>>>>>> running, > >>>>>>>>>>>>>> but not visible in engine VMs tab, but on host I can see > >>>>>>>>>>>>>> number > >>>>>>>>>>>>>> of > >>>>>>>>>>>>>> running VMs). After restart bug disappeared, but VMs too. > >>>>>>>>>>>>>> Today I reinstalled engine with oVirt 3.3.0rc1. But all my > >>>>>>>>>>>>>> VMs > >>>>>>>>>>>>>> are > >>>>>>>>>>>>>> gone. Even from storage. (but disappearing from storage > >>>>>>>>>>>>>> may be my > >>>>>>>>>>>>>> fault during recovery). > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I can provide logs from both two nodes and (hopefully) from > >>>>>>>>>>>>>> engine > >>>>>>>>>>>>>> too. I can provide you with those logs, but not publicly. > >>>>>>>>>>>>>> Sorry. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thank you, > >>>>>>>>>>>>>> Jakub Bittner, NSA Czech > >>>>>>>>>>>>>> _______________________________________________ > >>>>>>>>>>>>>> Users mailing list > >>>>>>>>>>>>>> Users(a)ovirt.org > >>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>>>>>>>>>> Well I reinstalled to 3.3.0 RC1, rebooted server and my VMs > >>>>>>>>>>>>> are > >>>>>>>>>>>>> gone > >>>>>>>>>>>>> again. > >>>>>>>>>>>>> _______________________________________________ > >>>>>>>>>>>>> Users mailing list > >>>>>>>>>>>>> Users(a)ovirt.org > >>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>>>>>>>>> can you verify the db from before the upgrade has the VMs? > >>>>>>>>>>> I dont know what you mean by verify database. Please, could you > >>>>>>>>>>> point me > >>>>>>>>>>> how to do it? Thank you > >>>>>>>>>>> _______________________________________________ > >>>>>>>>>>> Users mailing list > >>>>>>>>>>> Users(a)ovirt.org > >>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>>>>>>> I assume from your expectation to see the VMs from the previous > >>>>>>>>>> install that you upgraded your system rather than do a clean > >>>>>>>>>> install? > >>>>>>>>>> in an upgrade, there should be a backup of the db from before the > >>>>>>>>>> upgrade. > >>>>>>>>>> (in a clean install, engine is not aware of VMs running on > >>>>>>>>>> hosts from > >>>>>>>>>> previous installs - we need to cover "import existing storage > >>>>>>>>>> domain" > >>>>>>>>>> in a future version to close this gap. current alternatives > >>>>>>>>>> around > >>>>>>>>>> this are to convert the storage domain to an export domain > >>>>>>>>>> (for an > >>>>>>>>>> nfs > >>>>>>>>>> data domain) an re-import them to the system or a few other > >>>>>>>>>> options. > >>>>>>>>>> > >>>>>>>>>> how many VMs are there? > >>>>>>>>> They(VMs) are deleted. So, my situation is this: > >>>>>>>>> > >>>>>>>>> I have clean install of ovirt 3.3.0beta with imported VMs from > >>>>>>>>> export > >>>>>>>>> domain. Everything works except snapshots. But when I restart > >>>>>>>>> ovirt > >>>>>>>>> mgmt > >>>>>>>>> server, VMs will disappear. Right now I am doing hourly backup > >>>>>>>>> of DB. > >>>>>>>> this is very fishy... > >>>>>>>> can you arrange for a remote access session for someone to take > >>>>>>>> a look? > >>>>>>>> > >>>>>>>>> I can not use workaround with converting domain to export domain, > >>>>>>>>> because we use iscsi. It would be perfect if I can retake existing > >>>>>>>>> domain. Another gap is if I dont detach export domain I can not > >>>>>>>>> attach > >>>>>>>>> it to new datacenter and I have to manually edit some files. It > >>>>>>>>> would > >>>>>>>>> be > >>>>>>>>> absolutely perfect to do it from ovirt web gui (for example by > >>>>>>>>> "force > >>>>>>>>> override button") > >>>>>>>> i thought 3.3 allowed for that, but i could be wrong. > >>>>>>> Well, I will backup all VMs to export domain, than create > >>>>>>> snapshot of > >>>>>>> ovirt and I will try to restart that server if its going to > >>>>>>> delete VMs > >>>>>>> from DB. Than I will report back. I think, that remote session > >>>>>>> will not > >>>>>>> be possible, but I can be wrong (I hope). > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> Users mailing list > >>>>>>> Users(a)ovirt.org > >>>>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>>>> > >>>>> I change it. But I can not restart postgres. > >>>> You can restart it only when the engine is down, it will not restart > >>>> with > >>>> database connections open. > >>>> > >>>>> _______________________________________________ > >>>>> Users mailing list > >>>>> Users(a)ovirt.org > >>>>> http://lists.ovirt.org/mailman/listinfo/users > >>>>> > >>>> _______________________________________________ > >>>> Users mailing list > >>>> Users(a)ovirt.org > >>>> http://lists.ovirt.org/mailman/listinfo/users > >>>> > >> I think, that disks did not disappeared, but new instance of ovirt > >> deleted them as I thought that it can import them back. > >> _______________________________________________ > >> Users mailing list > >> Users(a)ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/users > > > > > > hello, > > > > right now I restarted ovirt-engine and postgresql and my VMs disappeared > > again, but not all of them! Some of them persists. (two) And I get > > messages about failed export and import of those who disappeared. Than I > > restored postgresql from backup and everything is OK. Restarting > > postgresql is like using way back machine ;-) > > Laszlo hinted this may be around some compnesation logic - detecting an > incomplete import, thus deleting them. yair - what should the logs > indicate for this? I thought of the compensation because it runs after engine start and does changes in the DB if there is something to compensate in business_entity_snapshot. So for example having records there could undo the import of VM's and at the same time delete their disks from the storage. However, when last time I had anything to do with the compensation, if it failed, it did not allow the engine to start. I don't know if it is still like that, I just noticed last week that some services may fail to configure and the engine ignores them in such cases. Jakub, do you have anything in the business_entity_snapshot table? > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users >