19 Nov
2012
19 Nov
'12
3:48 p.m.
----- Original Message ----- > From: "Liron Aravot" <laravot@redhat.com> > To: "Michael Kublin" <mkublin@redhat.com> > Cc: engine-devel@ovirt.org > Sent: Monday, November 19, 2012 3:58:55 PM > Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater > > > > ----- Original Message ----- > > From: "Michael Kublin" <mkublin@redhat.com> > > To: "Liron Aravot" <laravot@redhat.com> > > Cc: engine-devel@ovirt.org > > Sent: Monday, November 19, 2012 3:44:41 PM > > Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater > > > > > > > > ----- Original Message ----- > > > From: "Liron Aravot" <laravot@redhat.com> > > > To: "Michael Kublin" <mkublin@redhat.com> > > > Cc: engine-devel@ovirt.org > > > Sent: Monday, November 19, 2012 3:34:11 PM > > > Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Michael Kublin" <mkublin@redhat.com> > > > > To: "Liron Aravot" <laravot@redhat.com> > > > > Cc: engine-devel@ovirt.org > > > > Sent: Monday, November 19, 2012 3:28:06 PM > > > > Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > From: "Liron Aravot" <laravot@redhat.com> > > > > > To: "Michael Kublin" <mkublin@redhat.com> > > > > > Cc: engine-devel@ovirt.org > > > > > Sent: Monday, November 19, 2012 3:17:04 PM > > > > > Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > From: "Michael Kublin" <mkublin@redhat.com> > > > > > > To: "Liron Aravot" <laravot@redhat.com> > > > > > > Cc: engine-devel@ovirt.org > > > > > > Sent: Monday, November 19, 2012 3:03:23 PM > > > > > > Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater > > > > > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > From: "Liron Aravot" <laravot@redhat.com> > > > > > > > To: "Mike Kolesnik" <mkolesni@redhat.com> > > > > > > > Cc: engine-devel@ovirt.org, "Michael Kublin" > > > > > > > <mkublin@redhat.com> > > > > > > > Sent: Monday, November 19, 2012 2:52:56 PM > > > > > > > Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater > > > > > > > > > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > > From: "Mike Kolesnik" <mkolesni@redhat.com> > > > > > > > > To: "Liron Aravot" <laravot@redhat.com> > > > > > > > > Cc: engine-devel@ovirt.org, "Michael Kublin" > > > > > > > > <mkublin@redhat.com> > > > > > > > > Sent: Monday, November 19, 2012 2:44:56 PM > > > > > > > > Subject: Re: [Engine-devel] [Engine-Devel] > > > > > > > > OvfDataUpdater > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > > > > From: "Liron Aravot" <laravot@redhat.com> > > > > > > > > > > To: engine-devel@ovirt.org > > > > > > > > > > Sent: Monday, November 19, 2012 2:03:57 PM > > > > > > > > > > Subject: [Engine-devel] [Engine-Devel] > > > > > > > > > > OvfDataUpdater > > > > > > > > > > > > > > > > > > > > starting a new discussion thread. > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > > > > > From: "Mike Kolesnik" <mkolesni@redhat.com> > > > > > > > > > > > To: "Liron Aravot" <laravot@redhat.com> > > > > > > > > > > > Sent: Monday, November 19, 2012 12:42:10 PM > > > > > > > > > > > Subject: Re: [Engine-devel] OvfAutoUpdater > > > > > > > > > > > > > > > > > > > > > > I think 'version' is a more standard term for the > > > > > > > > > > > column > > > > > > > > > > > names. > > > > > > > > > > > > > > > > > > > > > > Also in: 4. If succesfull - for each vm update > > > > > > > > > > > the > > > > > > > > > > > ovf_generation > > > > > > > > > > > to > > > > > > > > > > > equal the db_generation. > > > > > > > > > > > I think you mean that the update should be to the > > > > > > > > > > > entity > > > > > > > > > > > version > > > > > > > > > > > you > > > > > > > > > > > initially selected. > > > > > > > > > > > > > > > > > > > > 1. I think should be version = version +1; > > > > > > > the version will be the db_generation that was loaded > > > > > > > when > > > > > > > we > > > > > > > loaded > > > > > > > the vm/template. > > > > > > > for example - if the vm/template was updated twice > > > > > > > between > > > > > > > OvfUpdater > > > > > > > runs, we will need to increment twice - so incrementing > > > > > > > by > > > > > > > one > > > > > > > will > > > > > > > cause another unneeded ovf update for that vm. > > > > > > I am talking about ovfVersion or how you called it. > > > > > me too, > > > > > if we update the vm three times between OvfAutoUpdater runs , > > > > > we > > > > > will > > > > > have the following values in the loaded vm for example: > > > > > db_generation 8 > > > > > ovf_generation 5 > > > > > > > > > > when we perform the ovf update , ovf_generation should be set > > > > > to > > > > > 8 > > > > > and not to 6, as version '8' is the version that we wrote the > > > > > ovf > > > > > metadata of. > > > > These should be written in wiki, with remarks that values which > > > > were > > > > retrieved > > > > from DB together. > > > > > > > > > 2. Now the quartz is automatic process, updateVmInSpm > > > > > > > > > it > > > > > > > > > is > > > > > > > > > spm > > > > > > > > > operation -it means it can trigger a reconstruct > > > > > > > > > and spm election. > > > > > > > 2. updateVmInSpm command would be changed so it won't > > > > > > > trigger > > > > > > > failover, will add that to the wiki. > > > > > > > > > 3. How your solution is handling a case of > > > > > > > > > hotPlug/hotUnplug/remove/add of vm disks. vm_static > > > > > > > > > is > > > > > > > > > not > > > > > > > > > usually > > > > > > > > > updated. > > > > > > > Whenever will be a command that locks a vm/template, you > > > > > > > will > > > > > > > have > > > > > > > an > > > > > > > option during the unlock to specify if the version need > > > > > > > to > > > > > > > be > > > > > > > incremented, you'll be able to increment it also > > > > > > > yourself. > > > > > > > The HotUnPlug command does lock the vm and during > > > > > > > endSuccesfully > > > > > > > attempt to update the vm in spm, so it will increment the > > > > > > > db_version/generation instead. > > > > > > > > > > > > > > addDisk, removeDisk, hotPlug/unPlug not locking a vm. > > > > > > Also, as far as I know LockVm/UnLockVm it is operation on > > > > > > vm_dynamic > > > > > > and not > > > > > > vm_static. > > > > > > > > > > hotplug acquires memory lock on the vm and has no async tasks > > > > > if > > > > > I > > > > > recall correctly. > > > > Yes, correct and what? These what I said, vm_static is not > > > > updated. > > > you will just call the dao method for incrementing the > > > db_generation, > > > you don't have to perform vm_static update. > > > > > adddisk acquires shared lock on the vm which is problematic > > > > > with > > > > > today flow as well - as you have a race during updateVmInSpm, > > > > > it > > > > > should be fixed regardless of the OvfAutoUpdater. > > > > It is not answer on my question. How ovf will be updated? How > > > > you > > > > understand > > > > that ovf of vm should be updated? > > > > > > the ovf will be updated when you increment the db_generation > > > value > > > instead of calling updateVmInSpm(). > > > you will be able to do that from the dao directly when you want, > > > or > > > when you unlock a vm/template. > > > when you will update the db_generation, the vm/template would be > > > picked up by OvfUpdater as it's db_generation is different than > > > its > > > ovf_generation. > > 1. Update of ovfs is not related to locks and it should not be, I > > don't understand why > > you link them. > > 2. Races - your automatic process and couple of add disk commands. > > You will miss update > > of ovf. > > 3. Also when you will add increase of generation it is not clear > > from > > your design > > 1+2 The OvfAutoUpdater is not related and doesn't acquire any locks. > > as of today, if you call updateVmInSpm without performing the > operations with lock on the vm you'll have a race - for example, in Liron, updateVmInSpm is called at endSuccessfully, you can not build on locks, especially on in memory locks. > addDisk (shared calls, two updateVmInSpm might be executed in the > same time and you can't know with with ovf you'll end). > > therefore, when updating the ovf_generation you should have a lock > somewhere otherwise you might miss updates because of races. > if today updatevminspm is called without locks and have races, it > needs to be fixed unrelated to it. Liron, you should fix these. These is a new development, it should fix all known bugs and problems and not introduce a new ones > 3. you will increase the generation whenever you made updateVmInSpm. > > > > > > > > > > > > > > Also consider snapshots & vNICs which affect the VM > > > > > > > > state. > > > > > > > Whatever affect the vm state and triggers updateVmInSpm > > > > > > > today > > > > > > > will > > > > > > > continue to, it can be added to flows which don't do that > > > > > > > today. > > > > > > > > > > > > > > > > > 4. Reconstruct/Recovery - updateVmInSpm should be > > > > > > > > > called > > > > > > > > > on > > > > > > > > > all > > > > > > > > > vms, > > > > > > > > > no matter if they were updated. > > > > > > > > > > > > > > Ofcourse, it's being taken care of. > > > > > > How? > > > > > we don't have ovf's of the vms when we reconstruct as we > > > > > don't > > > > > have > > > > > master domain - so we should set the ovf_generation of the > > > > > vms > > > > > to > > > > > be > > > > > 0 - which will cause ovfautoupdater to copy the vm metadata. > > > > I did not understand that statement and how it is related. > > > OvfAutoUpdater will get those vms for update as db_generation is > > > different than ovf_generation. > > > > > a performance improvement may be introduced later on in case > > > > > of > > > > > wrong > > > > > master version, but that's not related to the ovfautoupdater. > > > > I don't understand connection to performance. > > > > Now you are replacing a calls to VmCommand.updateVmInSpm() one > > > > of > > > > them it > > > > is at reconstruct flow, and you should handle it also and add > > > > it > > > > to > > > > wiki. > > > answered below, OvfAutoUpdater will get those vms for update as > > At reconstruct all vms are updated. So it is not related to > > generation. > > How you handle these case? > > when doing reconstruct, the ovf_generation of all vms is set to 0 as > we don't have them on the master domain. > ovfupdater will get this vms for update as ovf_generation < > db_generation, will copy the vm ovf and set db_genertion to > ovf_generation. > > > > db_generation is different than ovf_generation. > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > > > > > > http://wiki.ovirt.org/wiki/Feature/OvfAutoUpdater > > > > > > > > > > > > > > > > > > > > > > > > Hi all, i'll be glad if you could review the > > > > > > > > > > > > wiki > > > > > > > > > > > > page > > > > > > > > > > > > of > > > > > > > > > > > > OvfAutoUpdater, if you have any suggestions or > > > > > > > > > > > > ideas > > > > > > > > > > > > please > > > > > > > > > > > > let > > > > > > > > > > > > me > > > > > > > > > > > > know. > > > > > > > > > > > > > > > > > > > > > > > > http://wiki.ovirt.org/wiki/Feature/OvfAutoUpdater > > > > > > > > > > > > > > > > > > > > > > > > short preview from the wiki: > > > > > > > > > > > > vm/template configurations (including disks > > > > > > > > > > > > info) > > > > > > > > > > > > are > > > > > > > > > > > > stored > > > > > > > > > > > > on > > > > > > > > > > > > the > > > > > > > > > > > > master storage domain for backup purposes, > > > > > > > > > > > > import/export > > > > > > > > > > > > and > > > > > > > > > > > > also > > > > > > > > > > > > to > > > > > > > > > > > > provide the abillity to run VMs without having > > > > > > > > > > > > a > > > > > > > > > > > > running > > > > > > > > > > > > engine/db. > > > > > > > > > > > > Currently ovf update is done synchronously when > > > > > > > > > > > > performing > > > > > > > > > > > > various > > > > > > > > > > > > operations on vms/templates - update, save, > > > > > > > > > > > > adding/removing > > > > > > > > > > > > a > > > > > > > > > > > > disk, > > > > > > > > > > > > etc. What's more, currently updating the ovf > > > > > > > > > > > > (updateVM > > > > > > > > > > > > vdsm > > > > > > > > > > > > call) > > > > > > > > > > > > is > > > > > > > > > > > > usually done within a transcation. > > > > > > > > > > > > > > > > > > > > > > > > The idea behined OvfAutoUpdater is to perform > > > > > > > > > > > > batch > > > > > > > > > > > > ovf > > > > > > > > > > > > update > > > > > > > > > > > > operations that aggregate all outstanding > > > > > > > > > > > > updates > > > > > > > > > > > > per > > > > > > > > > > > > data > > > > > > > > > > > > center. > > > > > > > > > > > > These updates will be done in specifed time > > > > > > > > > > > > intervals > > > > > > > > > > > > which > > > > > > > > > > > > will > > > > > > > > > > > > reduce XML-RPC calls and will enable the > > > > > > > > > > > > removal > > > > > > > > > > > > of > > > > > > > > > > > > this > > > > > > > > > > > > syncronous > > > > > > > > > > > > vdsm call from all over the code. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Liron Aravot. > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > Engine-devel mailing list > > > > > > > > > > > > Engine-devel@ovirt.org > > > > > > > > > > > > http://lists.ovirt.org/mailman/listinfo/engine-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > Engine-devel mailing list > > > > > > > > > > Engine-devel@ovirt.org > > > > > > > > > > http://lists.ovirt.org/mailman/listinfo/engine-devel > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > Engine-devel mailing list > > > > > > > > > Engine-devel@ovirt.org > > > > > > > > > http://lists.ovirt.org/mailman/listinfo/engine-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >