Re: [Users] All VMs disappeared

Dne 14.8.2013 18:04, j.bittner(a)nbu.cz napsal(a): > Cituji "Laszlo Hornyak" <lhornyak(a)redhat.com&gt;: > >> >> >> ----- Original Message ----- >>> From: "Itamar Heim" <iheim(a)redhat.com&gt; >>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>> Cc: "Michal Skrivanek" <mskrivan(a)redhat.com&gt;, users(a)ovirt.org >>> Sent: Wednesday, August 14, 2013 4:27:17 PM >>> Subject: Re: [Users] All VMs disappeared >>> >>> On 08/14/2013 05:19 PM, Jakub Bittner wrote: >>> > Dne 14.8.2013 10:48, Jakub Bittner napsal(a): >>> >> Dne 14.8.2013 09:57, Liron Aravot napsal(a): >>> >>> Jakub, are only your vms disappear? what about the disks? >>> >>> >>> >>> ----- Original Message ----- >>> >>>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >>> >>>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>> >>>> Cc: users(a)ovirt.org >>> >>>> Sent: Wednesday, August 14, 2013 10:39:48 AM >>> >>>> Subject: Re: [Users] All VMs disappeared >>> >>>> >>> >>>> >>> >>>> >>> >>>> ----- Original Message ----- >>> >>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>> >>>>> To: users(a)ovirt.org >>> >>>>> Sent: Wednesday, August 14, 2013 8:09:45 AM >>> >>>>> Subject: Re: [Users] All VMs disappeared >>> >>>>> >>> >>>>> Dne 13.8.2013 18:29, Laszlo Hornyak napsal(a): >>> >>>>>> Ahoj Jakub, >>> >>>>>> >>> >>>>>> Just one more idea: could you turn statement logging on in >>> postgresql >>> >>>>>> before you try again? >>> >>>>>> It is in /var/lib/pgsql/data/postgresql.conf >>> >>>>>> log_statement = 'all' >>> >>>>>> >>> >>>>>> >>> >>>>>> ----- Original Message ----- >>> >>>>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>> >>>>>>> To: "Itamar Heim" <iheim(a)redhat.com&gt; >>> >>>>>>> Cc: users(a)ovirt.org >>> >>>>>>> Sent: Tuesday, August 13, 2013 5:20:29 PM >>> >>>>>>> Subject: Re: [Users] All VMs disappeared >>> >>>>>>> >>> >>>>>>> Dne 13.8.2013 17:02, Itamar Heim napsal(a): >>> >>>>>>>> On 08/13/2013 05:31 PM, Jakub Bittner wrote: >>> >>>>>>>>> Dne 13.8.2013 15:39, Itamar Heim napsal(a): >>> >>>>>>>>>> On 08/13/2013 08:50 AM, Jakub Bittner wrote: >>> >>>>>>>>>>> Dne 12.8.2013 22:44, Itamar Heim napsal(a): >>> >>>>>>>>>>>> On 08/12/2013 06:59 PM, Jakub Bittner wrote: >>> >>>>>>>>>>>>> Dne 12.8.2013 14:54, Jakub Bittner napsal(a): >>> >>>>>>>>>>>>>> Dne 12.8.2013 14:29, Laszlo Hornyak napsal(a): >>> >>>>>>>>>>>>>>> I looked around Noam's patch and that should not >>> cause such >>> >>>>>>>>>>>>>>> behavior. >>> >>>>>>>>>>>>>>> I am wondering how that lost VM's could happen. >>> >>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>> Jakub, can you give a more detailed description >>> what you >>> >>>>>>>>>>>>>>> were >>> >>>>>>>>>>>>>>> doing >>> >>>>>>>>>>>>>>> with oVirt when this happened? Maybe the bug is >>> still there. >>> >>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>> Thank you, >>> >>>>>>>>>>>>>>> Laszlo >>> >>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>> ----- Original Message ----- >>> >>>>>>>>>>>>>>>> From: "Greg Sheremeta" <gshereme(a)redhat.com&gt; >>> >>>>>>>>>>>>>>>> To: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >>> >>>>>>>>>>>>>>>> Cc: "Jakub Bittner" <j.bittner(a)nbu.cz&gt;, "Noam >>> Slomianko" >>> >>>>>>>>>>>>>>>> <nslomian(a)redhat.com&gt;, users(a)ovirt.org >>> >>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 1:23:40 PM >>> >>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared >>> >>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>> Not the one I fixed, 987907. It was a simple UI >>> >>>>>>>>>>>>>>>> NullPointerException. >>> >>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>> Greg >>> >>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>> ----- Original Message ----- >>> >>>>>>>>>>>>>>>>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >>> >>>>>>>>>>>>>>>>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt;, "Noam >>> Slomianko" >>> >>>>>>>>>>>>>>>>> <nslomian(a)redhat.com&gt;, "Greg Sheremeta" >>> >>>>>>>>>>>>>>>>> <gshereme(a)redhat.com&gt; >>> >>>>>>>>>>>>>>>>> Cc: users(a)ovirt.org >>> >>>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 7:21:26 AM >>> >>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> Well if they are no longer in DB then that >>> explains why >>> >>>>>>>>>>>>>>>>> the >>> >>>>>>>>>>>>>>>>> exception no >>> >>>>>>>>>>>>>>>>> longer occurs, but at the cost of database >>> corruption. >>> >>>>>>>>>>>>>>>>> Noam and Greg, can these bugs cause data corruption? >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> Thank you, >>> >>>>>>>>>>>>>>>>> Laszlo >>> >>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>> ----- Original Message ----- >>> >>>>>>>>>>>>>>>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>> >>>>>>>>>>>>>>>>>> To: "Greg Sheremeta" <gshereme(a)redhat.com&gt; >>> >>>>>>>>>>>>>>>>>> Cc: users(a)ovirt.org, "Laszlo Hornyak" >>> >>>>>>>>>>>>>>>>>> <lhornyak(a)redhat.com&gt; >>> >>>>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 9:05:09 AM >>> >>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared >>> >>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>> Dne 10.8.2013 01:54, Greg Sheremeta napsal(a): >>> >>>>>>>>>>>>>>>>>>> It could also be this bug[1], for which I just >>> >>>>>>>>>>>>>>>>>>> submitted a >>> >>>>>>>>>>>>>>>>>>> fix. >>> >>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>> [1] >>> https://bugzilla.redhat.com/show_bug.cgi?id=987907 >>> >>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>> You can work around it by typing just "VMs:" >>> (without >>> >>>>>>>>>>>>>>>>>>> the >>> >>>>>>>>>>>>>>>>>>> quotes) in >>> >>>>>>>>>>>>>>>>>>> the >>> >>>>>>>>>>>>>>>>>>> search bar. >>> >>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>> Greg >>> >>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>> ----- Original Message ----- >>> >>>>>>>>>>>>>>>>>>>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >>> >>>>>>>>>>>>>>>>>>>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>> >>>>>>>>>>>>>>>>>>>> Cc: users(a)ovirt.org >>> >>>>>>>>>>>>>>>>>>>> Sent: Friday, August 9, 2013 11:21:26 AM >>> >>>>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared >>> >>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>> Hi Jakub, >>> >>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>> Could you check through DB or REST-API if the >>> VM's >>> >>>>>>>>>>>>>>>>>>>> are in >>> >>>>>>>>>>>>>>>>>>>> your >>> >>>>>>>>>>>>>>>>>>>> DB? >>> >>>>>>>>>>>>>>>>>>>> select * from vm_static; >>> >>>>>>>>>>>>>>>>>>>> or >>> >>>>>>>>>>>>>>>>>>>> curl -u admin@internal:blablabla >>> [engine-url]api/vms >>> >>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>> It seems Noam fixed this issue already in >>> >>>>>>>>>>>>>>>>>>>> c2295c31fa645e1ba1b94cd557bd1fecb40c8829. >>> >>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>> Thank you, >>> >>>>>>>>>>>>>>>>>>>> Laszlo >>> >>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>> ----- Original Message ----- >>> >>>>>>>>>>>>>>>>>>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>> >>>>>>>>>>>>>>>>>>>>> To: users(a)ovirt.org >>> >>>>>>>>>>>>>>>>>>>>> Sent: Friday, August 9, 2013 1:06:53 PM >>> >>>>>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared >>> >>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>> Dne 9.8.2013 09:48, Jakub Bittner napsal(a): >>> >>>>>>>>>>>>>>>>>>>>>> Hello, >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> Iam running ovirt 3.3.0.beta1 on centos 6,4 and >>> >>>>>>>>>>>>>>>>>>>>>> all our >>> >>>>>>>>>>>>>>>>>>>>>> VMs >>> >>>>>>>>>>>>>>>>>>>>>> disappeared from VMs tab. Nodes running on >>> centos >>> >>>>>>>>>>>>>>>>>>>>>> too. >>> >>>>>>>>>>>>>>>>>>>>>> Repeating >>> >>>>>>>>>>>>>>>>>>>>>> problem in log is: >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> 2013-08-09 09:44:24,203 WARN >>> >>>>>>>>>>>>>>>>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager] >>> >>>>>>>>>>>>>>>>>>>>>> (DefaultQuartzScheduler_Worker-45) Failed to >>> >>>>>>>>>>>>>>>>>>>>>> refresh VDS >>> >>>>>>>>>>>>>>>>>>>>>> , >>> >>>>>>>>>>>>>>>>>>>>>> vds = >>> >>>>>>>>>>>>>>>>>>>>>> 7cb6aedf-47bc-40b0-877f-2a537fca5c64 : >>> >>>>>>>>>>>>>>>>>>>>>> node2.x.com, error >>> >>>>>>>>>>>>>>>>>>>>>> = >>> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException, continuing.: >>> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown >>> >>>>>>>>>>>>>>>>>>>>>> Source) [:1.7.0_25] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> java.lang.reflect.Method.invoke(Method.java:606) >>> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [scheduler.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.quartz.core.JobRunShell.run(JobRunShell.java:213) >>> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> 2013-08-09 09:44:27,039 WARN >>> >>>>>>>>>>>>>>>>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager] >>> >>>>>>>>>>>>>>>>>>>>>> (DefaultQuartzScheduler_Worker-47) Failed to >>> >>>>>>>>>>>>>>>>>>>>>> refresh VDS >>> >>>>>>>>>>>>>>>>>>>>>> , >>> >>>>>>>>>>>>>>>>>>>>>> vds = >>> >>>>>>>>>>>>>>>>>>>>>> 653e34a6-6b7e-4757-9042-dc02e7a0cddf : >>> >>>>>>>>>>>>>>>>>>>>>> node3.x.com, error >>> >>>>>>>>>>>>>>>>>>>>>> = >>> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException, continuing.: >>> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown >>> >>>>>>>>>>>>>>>>>>>>>> Source) [:1.7.0_25] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> java.lang.reflect.Method.invoke(Method.java:606) >>> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [scheduler.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.quartz.core.JobRunShell.run(JobRunShell.java:213) >>> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> at >>> >>>>>>>>>>>>>>>>>>>>>> >>> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:] >>> >>>>>>>>>>>>>>>>>>>>>> _______________________________________________ >>> >>>>>>>>>>>>>>>>>>>>>> Users mailing list >>> >>>>>>>>>>>>>>>>>>>>>> Users(a)ovirt.org >>> >>>>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>>>>>>>>>>>>>>>>>> I restarted management node and both nodes >>> and this >>> >>>>>>>>>>>>>>>>>>>>> problem >>> >>>>>>>>>>>>>>>>>>>>> disappeared. >>> >>>>>>>>>>>>>>>>>>>>> But I still can not see any VMs. >>> >>>>>>>>>>>>>>>>>>>>> _______________________________________________ >>> >>>>>>>>>>>>>>>>>>>>> Users mailing list >>> >>>>>>>>>>>>>>>>>>>>> Users(a)ovirt.org >>> >>>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>>>> _______________________________________________ >>> >>>>>>>>>>>>>>>>>>>> Users mailing list >>> >>>>>>>>>>>>>>>>>>>> Users(a)ovirt.org >>> >>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>> Hi, >>> >>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>>>>>> I do not have them in database. Both commands shows >>> >>>>>>>>>>>>>>>>>> nothing >>> >>>>>>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> Hello, >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> we run ovirt 3.2.1 (or 2). I upgraded to RC1, >>> everything was >>> >>>>>>>>>>>>>> great >>> >>>>>>>>>>>>>> (except SPICE. VNC and creating new WM due to >>> serialization >>> >>>>>>>>>>>>>> bug.). >>> >>>>>>>>>>>>>> Because those bugs were irritating, we switched to >>> >>>>>>>>>>>>>> nightly. Still >>> >>>>>>>>>>>>>> works fine (but bug with VMs is not fixed there), >>> than I >>> >>>>>>>>>>>>>> installed >>> >>>>>>>>>>>>>> new >>> >>>>>>>>>>>>>> ovirt engine management server in KVM virtualization >>> (using >>> >>>>>>>>>>>>>> libvirtd) >>> >>>>>>>>>>>>>> due to taking snapshots before deploying nightly >>> builds. New >>> >>>>>>>>>>>>>> virtualized engine works great and I delete old one. >>> Next >>> >>>>>>>>>>>>>> day I >>> >>>>>>>>>>>>>> came >>> >>>>>>>>>>>>>> to work and engine web page was very very slow, every >>> >>>>>>>>>>>>>> operation >>> >>>>>>>>>>>>>> takes >>> >>>>>>>>>>>>>> ages, that time I found there were no VMs, no HDDs >>> and so. I >>> >>>>>>>>>>>>>> converted >>> >>>>>>>>>>>>>> its qcow2 image to raw, because I think it has slow DB >>> >>>>>>>>>>>>>> because of >>> >>>>>>>>>>>>>> running i qcow2. Did not helped, than I take a look >>> in nodes >>> >>>>>>>>>>>>>> logs and >>> >>>>>>>>>>>>>> found this problem, which I posted in first email >>> (VMs still >>> >>>>>>>>>>>>>> running, >>> >>>>>>>>>>>>>> but not visible in engine VMs tab, but on host I can >>> see >>> >>>>>>>>>>>>>> number >>> >>>>>>>>>>>>>> of >>> >>>>>>>>>>>>>> running VMs). After restart bug disappeared, but VMs >>> too. >>> >>>>>>>>>>>>>> Today I reinstalled engine with oVirt 3.3.0rc1. But >>> all my >>> >>>>>>>>>>>>>> VMs >>> >>>>>>>>>>>>>> are >>> >>>>>>>>>>>>>> gone. Even from storage. (but disappearing from storage >>> >>>>>>>>>>>>>> may be my >>> >>>>>>>>>>>>>> fault during recovery). >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> I can provide logs from both two nodes and >>> (hopefully) from >>> >>>>>>>>>>>>>> engine >>> >>>>>>>>>>>>>> too. I can provide you with those logs, but not >>> publicly. >>> >>>>>>>>>>>>>> Sorry. >>> >>>>>>>>>>>>>> >>> >>>>>>>>>>>>>> Thank you, >>> >>>>>>>>>>>>>> Jakub Bittner, NSA Czech >>> >>>>>>>>>>>>>> _______________________________________________ >>> >>>>>>>>>>>>>> Users mailing list >>> >>>>>>>>>>>>>> Users(a)ovirt.org >>> >>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>>>>>>>>>> Well I reinstalled to 3.3.0 RC1, rebooted server and >>> my VMs >>> >>>>>>>>>>>>> are >>> >>>>>>>>>>>>> gone >>> >>>>>>>>>>>>> again. >>> >>>>>>>>>>>>> _______________________________________________ >>> >>>>>>>>>>>>> Users mailing list >>> >>>>>>>>>>>>> Users(a)ovirt.org >>> >>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>>>>>>>>> can you verify the db from before the upgrade has the >>> VMs? >>> >>>>>>>>>>> I dont know what you mean by verify database. Please, >>> could you >>> >>>>>>>>>>> point me >>> >>>>>>>>>>> how to do it? Thank you >>> >>>>>>>>>>> _______________________________________________ >>> >>>>>>>>>>> Users mailing list >>> >>>>>>>>>>> Users(a)ovirt.org >>> >>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>>>>>>> I assume from your expectation to see the VMs from the >>> previous >>> >>>>>>>>>> install that you upgraded your system rather than do a >>> clean >>> >>>>>>>>>> install? >>> >>>>>>>>>> in an upgrade, there should be a backup of the db from >>> before the >>> >>>>>>>>>> upgrade. >>> >>>>>>>>>> (in a clean install, engine is not aware of VMs running on >>> >>>>>>>>>> hosts from >>> >>>>>>>>>> previous installs - we need to cover "import existing >>> storage >>> >>>>>>>>>> domain" >>> >>>>>>>>>> in a future version to close this gap. current alternatives >>> >>>>>>>>>> around >>> >>>>>>>>>> this are to convert the storage domain to an export domain >>> >>>>>>>>>> (for an >>> >>>>>>>>>> nfs >>> >>>>>>>>>> data domain) an re-import them to the system or a few other >>> >>>>>>>>>> options. >>> >>>>>>>>>> >>> >>>>>>>>>> how many VMs are there? >>> >>>>>>>>> They(VMs) are deleted. So, my situation is this: >>> >>>>>>>>> >>> >>>>>>>>> I have clean install of ovirt 3.3.0beta with imported VMs >>> from >>> >>>>>>>>> export >>> >>>>>>>>> domain. Everything works except snapshots. But when I >>> restart >>> >>>>>>>>> ovirt >>> >>>>>>>>> mgmt >>> >>>>>>>>> server, VMs will disappear. Right now I am doing hourly >>> backup >>> >>>>>>>>> of DB. >>> >>>>>>>> this is very fishy... >>> >>>>>>>> can you arrange for a remote access session for someone to >>> take >>> >>>>>>>> a look? >>> >>>>>>>> >>> >>>>>>>>> I can not use workaround with converting domain to export >>> domain, >>> >>>>>>>>> because we use iscsi. It would be perfect if I can retake >>> existing >>> >>>>>>>>> domain. Another gap is if I dont detach export domain I >>> can not >>> >>>>>>>>> attach >>> >>>>>>>>> it to new datacenter and I have to manually edit some >>> files. It >>> >>>>>>>>> would >>> >>>>>>>>> be >>> >>>>>>>>> absolutely perfect to do it from ovirt web gui (for >>> example by >>> >>>>>>>>> "force >>> >>>>>>>>> override button") >>> >>>>>>>> i thought 3.3 allowed for that, but i could be wrong. >>> >>>>>>> Well, I will backup all VMs to export domain, than create >>> >>>>>>> snapshot of >>> >>>>>>> ovirt and I will try to restart that server if its going to >>> >>>>>>> delete VMs >>> >>>>>>> from DB. Than I will report back. I think, that remote session >>> >>>>>>> will not >>> >>>>>>> be possible, but I can be wrong (I hope). >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> _______________________________________________ >>> >>>>>>> Users mailing list >>> >>>>>>> Users(a)ovirt.org >>> >>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>>>> >>> >>>>> I change it. But I can not restart postgres. >>> >>>> You can restart it only when the engine is down, it will not >>> restart >>> >>>> with >>> >>>> database connections open. >>> >>>> >>> >>>>> _______________________________________________ >>> >>>>> Users mailing list >>> >>>>> Users(a)ovirt.org >>> >>>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>>> >>> >>>> _______________________________________________ >>> >>>> Users mailing list >>> >>>> Users(a)ovirt.org >>> >>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>>> >>> >> I think, that disks did not disappeared, but new instance of ovirt >>> >> deleted them as I thought that it can import them back. >>> >> _______________________________________________ >>> >> Users mailing list >>> >> Users(a)ovirt.org >>> >> http://lists.ovirt.org/mailman/listinfo/users >>> > >>> > >>> > hello, >>> > >>> > right now I restarted ovirt-engine and postgresql and my VMs >>> disappeared >>> > again, but not all of them! Some of them persists. (two) And I get >>> > messages about failed export and import of those who disappeared. >>> Than I >>> > restored postgresql from backup and everything is OK. Restarting >>> > postgresql is like using way back machine ;-) >>> >>> Laszlo hinted this may be around some compnesation logic - >>> detecting an >>> incomplete import, thus deleting them. yair - what should the logs >>> indicate for this? >> >> I thought of the compensation because it runs after engine start and >> does changes in the DB if there is something to compensate in >> business_entity_snapshot. >> So for example having records there could undo the import of VM's >> and at the same time delete their disks from the storage. >> However, when last time I had anything to do with the compensation, >> if it failed, it did not allow the engine to start. I don't know if >> it is still like that, I just noticed last week that some services >> may fail to configure and the engine ignores them in such cases. >> >> Jakub, do you have anything in the business_entity_snapshot table? >> >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >>> >> > > I do appologies, but I will be at work on Friday 23 and than I take a > look. Sorry. > But thank you for hints ;-) > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users May I ask where to find engine db password or how to get to pg? _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users