Re: [Users] All VMs disappeared

Dne 12.8.2013 14:29, Laszlo Hornyak napsal(a): > I looked around Noam's patch and that should not cause such behavior. > I am wondering how that lost VM's could happen. > > Jakub, can you give a more detailed description what you were doing > with oVirt when this happened? Maybe the bug is still there. > > Thank you, > Laszlo > > ----- Original Message ----- >> From: "Greg Sheremeta" <gshereme(a)redhat.com&gt; >> To: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >> Cc: "Jakub Bittner" <j.bittner(a)nbu.cz&gt;, "Noam Slomianko" >> <nslomian(a)redhat.com&gt;, users(a)ovirt.org >> Sent: Monday, August 12, 2013 1:23:40 PM >> Subject: Re: [Users] All VMs disappeared >> >> Not the one I fixed, 987907. It was a simple UI NullPointerException. >> >> Greg >> >> >> ----- Original Message ----- >>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt;, "Noam Slomianko" >>> <nslomian(a)redhat.com&gt;, "Greg Sheremeta" >>> <gshereme(a)redhat.com&gt; >>> Cc: users(a)ovirt.org >>> Sent: Monday, August 12, 2013 7:21:26 AM >>> Subject: Re: [Users] All VMs disappeared >>> >>> Well if they are no longer in DB then that explains why the >>> exception no >>> longer occurs, but at the cost of database corruption. >>> Noam and Greg, can these bugs cause data corruption? >>> >>> Thank you, >>> Laszlo >>> >>> ----- Original Message ----- >>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>>> To: "Greg Sheremeta" <gshereme(a)redhat.com&gt; >>>> Cc: users(a)ovirt.org, "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >>>> Sent: Monday, August 12, 2013 9:05:09 AM >>>> Subject: Re: [Users] All VMs disappeared >>>> >>>> Dne 10.8.2013 01:54, Greg Sheremeta napsal(a): >>>>> It could also be this bug[1], for which I just submitted a fix. >>>>> >>>>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=987907 >>>>> >>>>> You can work around it by typing just "VMs:" (without the quotes) in >>>>> the >>>>> search bar. >>>>> >>>>> Greg >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: "Laszlo Hornyak" <lhornyak(a)redhat.com&gt; >>>>>> To: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>>>>> Cc: users(a)ovirt.org >>>>>> Sent: Friday, August 9, 2013 11:21:26 AM >>>>>> Subject: Re: [Users] All VMs disappeared >>>>>> >>>>>> Hi Jakub, >>>>>> >>>>>> Could you check through DB or REST-API if the VM's are in your DB? >>>>>> select * from vm_static; >>>>>> or >>>>>> curl -u admin@internal:blablabla [engine-url]api/vms >>>>>> >>>>>> It seems Noam fixed this issue already in >>>>>> c2295c31fa645e1ba1b94cd557bd1fecb40c8829. >>>>>> >>>>>> Thank you, >>>>>> Laszlo >>>>>> >>>>>> ----- Original Message ----- >>>>>>> From: "Jakub Bittner" <j.bittner(a)nbu.cz&gt; >>>>>>> To: users(a)ovirt.org >>>>>>> Sent: Friday, August 9, 2013 1:06:53 PM >>>>>>> Subject: Re: [Users] All VMs disappeared >>>>>>> >>>>>>> Dne 9.8.2013 09:48, Jakub Bittner napsal(a): >>>>>>>> Hello, >>>>>>>> >>>>>>>> Iam running ovirt 3.3.0.beta1 on centos 6,4 and all our VMs >>>>>>>> disappeared from VMs tab. Nodes running on centos too. Repeating >>>>>>>> problem in log is: >>>>>>>> >>>>>>>> >>>>>>>> 2013-08-09 09:44:24,203 WARN >>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager] >>>>>>>> (DefaultQuartzScheduler_Worker-45) Failed to refresh VDS , vds = >>>>>>>> 7cb6aedf-47bc-40b0-877f-2a537fca5c64 : node2.x.com, error = >>>>>>>> java.lang.NullPointerException, continuing.: >>>>>>>> java.lang.NullPointerException >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> >>>>>>>> at >>>>>>>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown >>>>>>>> Source) [:1.7.0_25] >>>>>>>> at >>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>> >>>>>>>> [rt.jar:1.7.0_25] >>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>>> [rt.jar:1.7.0_25] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) >>>>>>>> >>>>>>>> [scheduler.jar:] >>>>>>>> at >>>>>>>> org.quartz.core.JobRunShell.run(JobRunShell.java:213) >>>>>>>> [quartz.jar:] >>>>>>>> at >>>>>>>> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >>>>>>>> >>>>>>>> [quartz.jar:] >>>>>>>> >>>>>>>> 2013-08-09 09:44:27,039 WARN >>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager] >>>>>>>> (DefaultQuartzScheduler_Worker-47) Failed to refresh VDS , vds = >>>>>>>> 653e34a6-6b7e-4757-9042-dc02e7a0cddf : node3.x.com, error = >>>>>>>> java.lang.NullPointerException, continuing.: >>>>>>>> java.lang.NullPointerException >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) >>>>>>>> >>>>>>>> [vdsbroker.jar:] >>>>>>>> >>>>>>>> at >>>>>>>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown >>>>>>>> Source) [:1.7.0_25] >>>>>>>> at >>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>> >>>>>>>> [rt.jar:1.7.0_25] >>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>>> [rt.jar:1.7.0_25] >>>>>>>> at >>>>>>>> org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) >>>>>>>> >>>>>>>> [scheduler.jar:] >>>>>>>> at >>>>>>>> org.quartz.core.JobRunShell.run(JobRunShell.java:213) >>>>>>>> [quartz.jar:] >>>>>>>> at >>>>>>>> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >>>>>>>> >>>>>>>> [quartz.jar:] >>>>>>>> _______________________________________________ >>>>>>>> Users mailing list >>>>>>>> Users(a)ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>>> I restarted management node and both nodes and this problem >>>>>>> disappeared. >>>>>>> But I still can not see any VMs. >>>>>>> _______________________________________________ >>>>>>> Users mailing list >>>>>>> Users(a)ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>>> >>>>>> _______________________________________________ >>>>>> Users mailing list >>>>>> Users(a)ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>> >>>> Hi, >>>> >>>> I do not have them in database. Both commands shows nothing >>>> Hello, we run ovirt 3.2.1 (or 2). I upgraded to RC1, everything was great (except SPICE. VNC and creating new WM due to serialization bug.). Because those bugs were irritating, we switched to nightly. Still works fine (but bug with VMs is not fixed there), than I installed new ovirt engine management server in KVM virtualization (using libvirtd) due to taking snapshots before deploying nightly builds. New virtualized engine works great and I delete old one. Next day I came to work and engine web page was very very slow, every operation takes ages, that time I found there were no VMs, no HDDs and so. I converted its qcow2 image to raw, because I think it has slow DB because of running i qcow2. Did not helped, than I take a look in nodes logs and found this problem, which I posted in first email (VMs still running, but not visible in engine VMs tab, but on host I can see number of running VMs). After restart bug disappeared, but VMs too. Today I reinstalled engine with oVirt 3.3.0rc1. But all my VMs are gone. Even from storage. (but disappearing from storage may be my fault during recovery). I can provide logs from both two nodes and (hopefully) from engine too. I can provide you with those logs, but not publicly. Sorry. Thank you, Jakub Bittner, NSA Czech _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users