[Users] All VMs disappeared
Laszlo Hornyak
lhornyak at redhat.com
Wed Aug 14 15:03:53 UTC 2013
----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Jakub Bittner" <j.bittner at nbu.cz>
> Cc: "Michal Skrivanek" <mskrivan at redhat.com>, users at ovirt.org
> Sent: Wednesday, August 14, 2013 4:27:17 PM
> Subject: Re: [Users] All VMs disappeared
>
> On 08/14/2013 05:19 PM, Jakub Bittner wrote:
> > Dne 14.8.2013 10:48, Jakub Bittner napsal(a):
> >> Dne 14.8.2013 09:57, Liron Aravot napsal(a):
> >>> Jakub, are only your vms disappear? what about the disks?
> >>>
> >>> ----- Original Message -----
> >>>> From: "Laszlo Hornyak" <lhornyak at redhat.com>
> >>>> To: "Jakub Bittner" <j.bittner at nbu.cz>
> >>>> Cc: users at ovirt.org
> >>>> Sent: Wednesday, August 14, 2013 10:39:48 AM
> >>>> Subject: Re: [Users] All VMs disappeared
> >>>>
> >>>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Jakub Bittner" <j.bittner at nbu.cz>
> >>>>> To: users at ovirt.org
> >>>>> Sent: Wednesday, August 14, 2013 8:09:45 AM
> >>>>> Subject: Re: [Users] All VMs disappeared
> >>>>>
> >>>>> Dne 13.8.2013 18:29, Laszlo Hornyak napsal(a):
> >>>>>> Ahoj Jakub,
> >>>>>>
> >>>>>> Just one more idea: could you turn statement logging on in postgresql
> >>>>>> before you try again?
> >>>>>> It is in /var/lib/pgsql/data/postgresql.conf
> >>>>>> log_statement = 'all'
> >>>>>>
> >>>>>>
> >>>>>> ----- Original Message -----
> >>>>>>> From: "Jakub Bittner" <j.bittner at nbu.cz>
> >>>>>>> To: "Itamar Heim" <iheim at redhat.com>
> >>>>>>> Cc: users at ovirt.org
> >>>>>>> Sent: Tuesday, August 13, 2013 5:20:29 PM
> >>>>>>> Subject: Re: [Users] All VMs disappeared
> >>>>>>>
> >>>>>>> Dne 13.8.2013 17:02, Itamar Heim napsal(a):
> >>>>>>>> On 08/13/2013 05:31 PM, Jakub Bittner wrote:
> >>>>>>>>> Dne 13.8.2013 15:39, Itamar Heim napsal(a):
> >>>>>>>>>> On 08/13/2013 08:50 AM, Jakub Bittner wrote:
> >>>>>>>>>>> Dne 12.8.2013 22:44, Itamar Heim napsal(a):
> >>>>>>>>>>>> On 08/12/2013 06:59 PM, Jakub Bittner wrote:
> >>>>>>>>>>>>> Dne 12.8.2013 14:54, Jakub Bittner napsal(a):
> >>>>>>>>>>>>>> Dne 12.8.2013 14:29, Laszlo Hornyak napsal(a):
> >>>>>>>>>>>>>>> I looked around Noam's patch and that should not cause such
> >>>>>>>>>>>>>>> behavior.
> >>>>>>>>>>>>>>> I am wondering how that lost VM's could happen.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Jakub, can you give a more detailed description what you
> >>>>>>>>>>>>>>> were
> >>>>>>>>>>>>>>> doing
> >>>>>>>>>>>>>>> with oVirt when this happened? Maybe the bug is still there.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thank you,
> >>>>>>>>>>>>>>> Laszlo
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>> From: "Greg Sheremeta" <gshereme at redhat.com>
> >>>>>>>>>>>>>>>> To: "Laszlo Hornyak" <lhornyak at redhat.com>
> >>>>>>>>>>>>>>>> Cc: "Jakub Bittner" <j.bittner at nbu.cz>, "Noam Slomianko"
> >>>>>>>>>>>>>>>> <nslomian at redhat.com>, users at ovirt.org
> >>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 1:23:40 PM
> >>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Not the one I fixed, 987907. It was a simple UI
> >>>>>>>>>>>>>>>> NullPointerException.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Greg
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>> From: "Laszlo Hornyak" <lhornyak at redhat.com>
> >>>>>>>>>>>>>>>>> To: "Jakub Bittner" <j.bittner at nbu.cz>, "Noam Slomianko"
> >>>>>>>>>>>>>>>>> <nslomian at redhat.com>, "Greg Sheremeta"
> >>>>>>>>>>>>>>>>> <gshereme at redhat.com>
> >>>>>>>>>>>>>>>>> Cc: users at ovirt.org
> >>>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 7:21:26 AM
> >>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Well if they are no longer in DB then that explains why
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> exception no
> >>>>>>>>>>>>>>>>> longer occurs, but at the cost of database corruption.
> >>>>>>>>>>>>>>>>> Noam and Greg, can these bugs cause data corruption?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thank you,
> >>>>>>>>>>>>>>>>> Laszlo
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>> From: "Jakub Bittner" <j.bittner at nbu.cz>
> >>>>>>>>>>>>>>>>>> To: "Greg Sheremeta" <gshereme at redhat.com>
> >>>>>>>>>>>>>>>>>> Cc: users at ovirt.org, "Laszlo Hornyak"
> >>>>>>>>>>>>>>>>>> <lhornyak at redhat.com>
> >>>>>>>>>>>>>>>>>> Sent: Monday, August 12, 2013 9:05:09 AM
> >>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Dne 10.8.2013 01:54, Greg Sheremeta napsal(a):
> >>>>>>>>>>>>>>>>>>> It could also be this bug[1], for which I just
> >>>>>>>>>>>>>>>>>>> submitted a
> >>>>>>>>>>>>>>>>>>> fix.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=987907
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> You can work around it by typing just "VMs:" (without
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> quotes) in
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>> search bar.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Greg
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>> From: "Laszlo Hornyak" <lhornyak at redhat.com>
> >>>>>>>>>>>>>>>>>>>> To: "Jakub Bittner" <j.bittner at nbu.cz>
> >>>>>>>>>>>>>>>>>>>> Cc: users at ovirt.org
> >>>>>>>>>>>>>>>>>>>> Sent: Friday, August 9, 2013 11:21:26 AM
> >>>>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi Jakub,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Could you check through DB or REST-API if the VM's
> >>>>>>>>>>>>>>>>>>>> are in
> >>>>>>>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>> DB?
> >>>>>>>>>>>>>>>>>>>> select * from vm_static;
> >>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>> curl -u admin at internal:blablabla [engine-url]api/vms
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> It seems Noam fixed this issue already in
> >>>>>>>>>>>>>>>>>>>> c2295c31fa645e1ba1b94cd557bd1fecb40c8829.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thank you,
> >>>>>>>>>>>>>>>>>>>> Laszlo
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>>>>>> From: "Jakub Bittner" <j.bittner at nbu.cz>
> >>>>>>>>>>>>>>>>>>>>> To: users at ovirt.org
> >>>>>>>>>>>>>>>>>>>>> Sent: Friday, August 9, 2013 1:06:53 PM
> >>>>>>>>>>>>>>>>>>>>> Subject: Re: [Users] All VMs disappeared
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Dne 9.8.2013 09:48, Jakub Bittner napsal(a):
> >>>>>>>>>>>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Iam running ovirt 3.3.0.beta1 on centos 6,4 and
> >>>>>>>>>>>>>>>>>>>>>> all our
> >>>>>>>>>>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>>>>>>>>>> disappeared from VMs tab. Nodes running on centos
> >>>>>>>>>>>>>>>>>>>>>> too.
> >>>>>>>>>>>>>>>>>>>>>> Repeating
> >>>>>>>>>>>>>>>>>>>>>> problem in log is:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> 2013-08-09 09:44:24,203 WARN
> >>>>>>>>>>>>>>>>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager]
> >>>>>>>>>>>>>>>>>>>>>> (DefaultQuartzScheduler_Worker-45) Failed to
> >>>>>>>>>>>>>>>>>>>>>> refresh VDS
> >>>>>>>>>>>>>>>>>>>>>> ,
> >>>>>>>>>>>>>>>>>>>>>> vds =
> >>>>>>>>>>>>>>>>>>>>>> 7cb6aedf-47bc-40b0-877f-2a537fca5c64 :
> >>>>>>>>>>>>>>>>>>>>>> node2.x.com, error
> >>>>>>>>>>>>>>>>>>>>>> =
> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException, continuing.:
> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown
> >>>>>>>>>>>>>>>>>>>>>> Source) [:1.7.0_25]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> java.lang.reflect.Method.invoke(Method.java:606)
> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [scheduler.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:]
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> 2013-08-09 09:44:27,039 WARN
> >>>>>>>>>>>>>>>>>>>>>> [org.ovirt.engine.core.vdsbroker.VdsManager]
> >>>>>>>>>>>>>>>>>>>>>> (DefaultQuartzScheduler_Worker-47) Failed to
> >>>>>>>>>>>>>>>>>>>>>> refresh VDS
> >>>>>>>>>>>>>>>>>>>>>> ,
> >>>>>>>>>>>>>>>>>>>>>> vds =
> >>>>>>>>>>>>>>>>>>>>>> 653e34a6-6b7e-4757-9042-dc02e7a0cddf :
> >>>>>>>>>>>>>>>>>>>>>> node3.x.com, error
> >>>>>>>>>>>>>>>>>>>>>> =
> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException, continuing.:
> >>>>>>>>>>>>>>>>>>>>>> java.lang.NullPointerException
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.proceedGuaranteedMemoryCheck(VdsUpdateRunTimeInfo.java:1313)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:968)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:542)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:383)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [vdsbroker.jar:]
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> sun.reflect.GeneratedMethodAccessor171.invoke(Unknown
> >>>>>>>>>>>>>>>>>>>>>> Source) [:1.7.0_25]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> java.lang.reflect.Method.invoke(Method.java:606)
> >>>>>>>>>>>>>>>>>>>>>> [rt.jar:1.7.0_25]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [scheduler.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:]
> >>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [quartz.jar:]
> >>>>>>>>>>>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>>>>>>>>>>> Users mailing list
> >>>>>>>>>>>>>>>>>>>>>> Users at ovirt.org
> >>>>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>>>>>>>>>>>>>>> I restarted management node and both nodes and this
> >>>>>>>>>>>>>>>>>>>>> problem
> >>>>>>>>>>>>>>>>>>>>> disappeared.
> >>>>>>>>>>>>>>>>>>>>> But I still can not see any VMs.
> >>>>>>>>>>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>>>>>>>>>> Users mailing list
> >>>>>>>>>>>>>>>>>>>>> Users at ovirt.org
> >>>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>>>>>>>>> Users mailing list
> >>>>>>>>>>>>>>>>>>>> Users at ovirt.org
> >>>>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I do not have them in database. Both commands shows
> >>>>>>>>>>>>>>>>>> nothing
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> we run ovirt 3.2.1 (or 2). I upgraded to RC1, everything was
> >>>>>>>>>>>>>> great
> >>>>>>>>>>>>>> (except SPICE. VNC and creating new WM due to serialization
> >>>>>>>>>>>>>> bug.).
> >>>>>>>>>>>>>> Because those bugs were irritating, we switched to
> >>>>>>>>>>>>>> nightly. Still
> >>>>>>>>>>>>>> works fine (but bug with VMs is not fixed there), than I
> >>>>>>>>>>>>>> installed
> >>>>>>>>>>>>>> new
> >>>>>>>>>>>>>> ovirt engine management server in KVM virtualization (using
> >>>>>>>>>>>>>> libvirtd)
> >>>>>>>>>>>>>> due to taking snapshots before deploying nightly builds. New
> >>>>>>>>>>>>>> virtualized engine works great and I delete old one. Next
> >>>>>>>>>>>>>> day I
> >>>>>>>>>>>>>> came
> >>>>>>>>>>>>>> to work and engine web page was very very slow, every
> >>>>>>>>>>>>>> operation
> >>>>>>>>>>>>>> takes
> >>>>>>>>>>>>>> ages, that time I found there were no VMs, no HDDs and so. I
> >>>>>>>>>>>>>> converted
> >>>>>>>>>>>>>> its qcow2 image to raw, because I think it has slow DB
> >>>>>>>>>>>>>> because of
> >>>>>>>>>>>>>> running i qcow2. Did not helped, than I take a look in nodes
> >>>>>>>>>>>>>> logs and
> >>>>>>>>>>>>>> found this problem, which I posted in first email (VMs still
> >>>>>>>>>>>>>> running,
> >>>>>>>>>>>>>> but not visible in engine VMs tab, but on host I can see
> >>>>>>>>>>>>>> number
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>> running VMs). After restart bug disappeared, but VMs too.
> >>>>>>>>>>>>>> Today I reinstalled engine with oVirt 3.3.0rc1. But all my
> >>>>>>>>>>>>>> VMs
> >>>>>>>>>>>>>> are
> >>>>>>>>>>>>>> gone. Even from storage. (but disappearing from storage
> >>>>>>>>>>>>>> may be my
> >>>>>>>>>>>>>> fault during recovery).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I can provide logs from both two nodes and (hopefully) from
> >>>>>>>>>>>>>> engine
> >>>>>>>>>>>>>> too. I can provide you with those logs, but not publicly.
> >>>>>>>>>>>>>> Sorry.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thank you,
> >>>>>>>>>>>>>> Jakub Bittner, NSA Czech
> >>>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>>> Users mailing list
> >>>>>>>>>>>>>> Users at ovirt.org
> >>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>>>>>>> Well I reinstalled to 3.3.0 RC1, rebooted server and my VMs
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>> gone
> >>>>>>>>>>>>> again.
> >>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>> Users mailing list
> >>>>>>>>>>>>> Users at ovirt.org
> >>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>>>>>> can you verify the db from before the upgrade has the VMs?
> >>>>>>>>>>> I dont know what you mean by verify database. Please, could you
> >>>>>>>>>>> point me
> >>>>>>>>>>> how to do it? Thank you
> >>>>>>>>>>> _______________________________________________
> >>>>>>>>>>> Users mailing list
> >>>>>>>>>>> Users at ovirt.org
> >>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>>>> I assume from your expectation to see the VMs from the previous
> >>>>>>>>>> install that you upgraded your system rather than do a clean
> >>>>>>>>>> install?
> >>>>>>>>>> in an upgrade, there should be a backup of the db from before the
> >>>>>>>>>> upgrade.
> >>>>>>>>>> (in a clean install, engine is not aware of VMs running on
> >>>>>>>>>> hosts from
> >>>>>>>>>> previous installs - we need to cover "import existing storage
> >>>>>>>>>> domain"
> >>>>>>>>>> in a future version to close this gap. current alternatives
> >>>>>>>>>> around
> >>>>>>>>>> this are to convert the storage domain to an export domain
> >>>>>>>>>> (for an
> >>>>>>>>>> nfs
> >>>>>>>>>> data domain) an re-import them to the system or a few other
> >>>>>>>>>> options.
> >>>>>>>>>>
> >>>>>>>>>> how many VMs are there?
> >>>>>>>>> They(VMs) are deleted. So, my situation is this:
> >>>>>>>>>
> >>>>>>>>> I have clean install of ovirt 3.3.0beta with imported VMs from
> >>>>>>>>> export
> >>>>>>>>> domain. Everything works except snapshots. But when I restart
> >>>>>>>>> ovirt
> >>>>>>>>> mgmt
> >>>>>>>>> server, VMs will disappear. Right now I am doing hourly backup
> >>>>>>>>> of DB.
> >>>>>>>> this is very fishy...
> >>>>>>>> can you arrange for a remote access session for someone to take
> >>>>>>>> a look?
> >>>>>>>>
> >>>>>>>>> I can not use workaround with converting domain to export domain,
> >>>>>>>>> because we use iscsi. It would be perfect if I can retake existing
> >>>>>>>>> domain. Another gap is if I dont detach export domain I can not
> >>>>>>>>> attach
> >>>>>>>>> it to new datacenter and I have to manually edit some files. It
> >>>>>>>>> would
> >>>>>>>>> be
> >>>>>>>>> absolutely perfect to do it from ovirt web gui (for example by
> >>>>>>>>> "force
> >>>>>>>>> override button")
> >>>>>>>> i thought 3.3 allowed for that, but i could be wrong.
> >>>>>>> Well, I will backup all VMs to export domain, than create
> >>>>>>> snapshot of
> >>>>>>> ovirt and I will try to restart that server if its going to
> >>>>>>> delete VMs
> >>>>>>> from DB. Than I will report back. I think, that remote session
> >>>>>>> will not
> >>>>>>> be possible, but I can be wrong (I hope).
> >>>>>>>
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> Users mailing list
> >>>>>>> Users at ovirt.org
> >>>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>>>
> >>>>> I change it. But I can not restart postgres.
> >>>> You can restart it only when the engine is down, it will not restart
> >>>> with
> >>>> database connections open.
> >>>>
> >>>>> _______________________________________________
> >>>>> Users mailing list
> >>>>> Users at ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>
> >>>> _______________________________________________
> >>>> Users mailing list
> >>>> Users at ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>
> >> I think, that disks did not disappeared, but new instance of ovirt
> >> deleted them as I thought that it can import them back.
> >> _______________________________________________
> >> Users mailing list
> >> Users at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/users
> >
> >
> > hello,
> >
> > right now I restarted ovirt-engine and postgresql and my VMs disappeared
> > again, but not all of them! Some of them persists. (two) And I get
> > messages about failed export and import of those who disappeared. Than I
> > restored postgresql from backup and everything is OK. Restarting
> > postgresql is like using way back machine ;-)
>
> Laszlo hinted this may be around some compnesation logic - detecting an
> incomplete import, thus deleting them. yair - what should the logs
> indicate for this?
I thought of the compensation because it runs after engine start and does changes in the DB if there is something to compensate in business_entity_snapshot.
So for example having records there could undo the import of VM's and at the same time delete their disks from the storage.
However, when last time I had anything to do with the compensation, if it failed, it did not allow the engine to start. I don't know if it is still like that, I just noticed last week that some services may fail to configure and the engine ignores them in such cases.
Jakub, do you have anything in the business_entity_snapshot table?
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
More information about the Users
mailing list