[Engine-devel] [Engine-Devel] OvfDataUpdater

Yair Zaslavsky yzaslavs at redhat.com
Mon Nov 19 13:44:12 UTC 2012 


On 11/19/2012 03:38 PM, Yair Zaslavsky wrote:
>
>
> On 11/19/2012 03:34 PM, Liron Aravot wrote:
>>
>>
>> ----- Original Message -----
>>> From: "Michael Kublin" <mkublin at redhat.com>
>>> To: "Liron Aravot" <laravot at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Monday, November 19, 2012 3:28:06 PM
>>> Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Liron Aravot" <laravot at redhat.com>
>>>> To: "Michael Kublin" <mkublin at redhat.com>
>>>> Cc: engine-devel at ovirt.org
>>>> Sent: Monday, November 19, 2012 3:17:04 PM
>>>> Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Michael Kublin" <mkublin at redhat.com>
>>>>> To: "Liron Aravot" <laravot at redhat.com>
>>>>> Cc: engine-devel at ovirt.org
>>>>> Sent: Monday, November 19, 2012 3:03:23 PM
>>>>> Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Liron Aravot" <laravot at redhat.com>
>>>>>> To: "Mike Kolesnik" <mkolesni at redhat.com>
>>>>>> Cc: engine-devel at ovirt.org, "Michael Kublin"
>>>>>> <mkublin at redhat.com>
>>>>>> Sent: Monday, November 19, 2012 2:52:56 PM
>>>>>> Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Mike Kolesnik" <mkolesni at redhat.com>
>>>>>>> To: "Liron Aravot" <laravot at redhat.com>
>>>>>>> Cc: engine-devel at ovirt.org, "Michael Kublin"
>>>>>>> <mkublin at redhat.com>
>>>>>>> Sent: Monday, November 19, 2012 2:44:56 PM
>>>>>>> Subject: Re: [Engine-devel] [Engine-Devel] OvfDataUpdater
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Liron Aravot" <laravot at redhat.com>
>>>>>>>>> To: engine-devel at ovirt.org
>>>>>>>>> Sent: Monday, November 19, 2012 2:03:57 PM
>>>>>>>>> Subject: [Engine-devel] [Engine-Devel] OvfDataUpdater
>>>>>>>>>
>>>>>>>>> starting a new discussion thread.
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>>> From: "Mike Kolesnik" <mkolesni at redhat.com>
>>>>>>>>>> To: "Liron Aravot" <laravot at redhat.com>
>>>>>>>>>> Sent: Monday, November 19, 2012 12:42:10 PM
>>>>>>>>>> Subject: Re: [Engine-devel]  OvfAutoUpdater
>>>>>>>>>>
>>>>>>>>>> I think 'version' is a more standard term for the
>>>>>>>>>> column
>>>>>>>>>> names.
>>>>>>>>>>
>>>>>>>>>> Also in:  4. If succesfull - for each vm update the
>>>>>>>>>> ovf_generation
>>>>>>>>>> to
>>>>>>>>>> equal the db_generation.
>>>>>>>>>> I think you mean that the update should be to the
>>>>>>>>>> entity
>>>>>>>>>> version
>>>>>>>>>> you
>>>>>>>>>> initially selected.
>>>>>>>>>>
>>>>>>>> 1. I think should be version = version +1;
>>>>>> the version will be the db_generation that was loaded when we
>>>>>> loaded
>>>>>> the vm/template.
>>>>>> for example - if the vm/template was updated twice between
>>>>>> OvfUpdater
>>>>>> runs, we will need to increment twice - so incrementing by one
>>>>>> will
>>>>>> cause another unneeded ovf update for that vm.
>>>>> I am talking about ovfVersion or how you called it.
>>>> me too,
>>>> if we update the vm three times between OvfAutoUpdater runs , we
>>>> will
>>>> have the following values in the loaded vm for example:
>>>> db_generation 8
>>>> ovf_generation 5
>>>>
>>>> when we perform the ovf update , ovf_generation should be set to 8
>>>> and not to 6, as version '8' is the version that we wrote the ovf
>>>> metadata of.
>>> These should be written in wiki, with remarks that values which were
>>> retrieved
>>> from DB together.
>>>>>>>> 2. Now the quartz is automatic process, updateVmInSpm it is
>>>>>>>> spm
>>>>>>>> operation -it means it can trigger a reconstruct
>>>>>>>> and spm election.
>>>>>> 2. updateVmInSpm command would be changed so it won't trigger
>>>>>> failover, will add that to the wiki.
>>>>>>>> 3. How your solution is handling a case of
>>>>>>>> hotPlug/hotUnplug/remove/add of vm disks. vm_static is not
>>>>>>>> usually
>>>>>>>> updated.
>>>>>> Whenever will be a command that locks a vm/template, you will
>>>>>> have
>>>>>> an
>>>>>> option during the unlock to specify if the version need to be
>>>>>> incremented, you'll be able to increment it also yourself.
>>>>>> The HotUnPlug command does lock the vm and during
>>>>>> endSuccesfully
>>>>>> attempt to update the vm in spm, so it will increment the
>>>>>> db_version/generation instead.
>>>>>>>
>>>>> addDisk, removeDisk, hotPlug/unPlug not locking a vm.
>>>>> Also, as far as I know LockVm/UnLockVm it is operation on
>>>>> vm_dynamic
>>>>> and not
>>>>> vm_static.
>>>>
>>>> hotplug acquires memory lock on the vm and has no async tasks if I
>>>> recall correctly.
>>> Yes, correct and what? These what I said, vm_static is not updated.
>> you will just call the dao method for incrementing the db_generation,
>> you don't have to perform vm_static update.
>>>> adddisk acquires shared lock on the vm which is problematic with
>>>> today flow as well - as you have a race during updateVmInSpm, it
>>>> should be fixed regardless of the OvfAutoUpdater.
>>> It is not answer on my question. How ovf will be updated? How you
>>> understand
>>> that ovf of vm should be updated?
>>
>> the ovf will be updated when you increment the db_generation value
>> instead of calling updateVmInSpm().
>> you will be able to do that from the dao directly when you want, or
>> when you unlock a vm/template.
>> when you will update the db_generation, the vm/template would be
>> picked up by OvfUpdater as it's db_generation is different than its
>> ovf_generation.
>>
>>>>>
>>>>>>> Also consider snapshots & vNICs which affect the VM state.
>>>>>> Whatever affect the vm state and triggers updateVmInSpm today
>>>>>> will
>>>>>> continue to, it can be added to flows which don't do that
>>>>>> today.
>>>>>>>
>>>>>>>> 4. Reconstruct/Recovery - updateVmInSpm should be called on
>>>>>>>> all
>>>>>>>> vms,
>>>>>>>> no matter if they were updated.
>>>>>>
>>>>>> Ofcourse, it's being taken care of.
>>>>> How?
>>>> we don't have ovf's of the vms when we reconstruct as we don't have
>>>> master domain - so we should set the ovf_generation of the vms to
>>>> be
>>>> 0 - which will cause ovfautoupdater to copy the vm metadata.
>>> I did not understand that statement and how it is related.
>> OvfAutoUpdater will get those vms for update as db_generation is
>> different than ovf_generation.
>>>> a performance improvement may be introduced later on in case of
>>>> wrong
>>>> master version, but that's not related to the ovfautoupdater.
>>> I don't understand connection to performance.
>>> Now you are replacing a calls to VmCommand.updateVmInSpm() one of
>>> them it
>>> is at reconstruct flow, and you should handle it also and add it to
>>> wiki.
>> answered below, OvfAutoUpdater will get those vms for update as
>> db_generation is different than ovf_generation.
>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Mike
>>>>>>>>>>
>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>> http://wiki.ovirt.org/wiki/Feature/OvfAutoUpdater
>>>>>>>>>>>
>>>>>>>>>>> Hi all, i'll be glad if you could review the wiki
>>>>>>>>>>> page
>>>>>>>>>>> of
>>>>>>>>>>> OvfAutoUpdater, if you have any suggestions or ideas
>>>>>>>>>>> please
>>>>>>>>>>> let
>>>>>>>>>>> me
>>>>>>>>>>> know.
>>>>>>>>>>>
>>>>>>>>>>> http://wiki.ovirt.org/wiki/Feature/OvfAutoUpdater
>>>>>>>>>>>
>>>>>>>>>>> short preview from the wiki:
>>>>>>>>>>> vm/template configurations (including disks info) are
>>>>>>>>>>> stored
>>>>>>>>>>> on
>>>>>>>>>>> the
>>>>>>>>>>> master storage domain for backup purposes,
>>>>>>>>>>> import/export
>>>>>>>>>>> and
>>>>>>>>>>> also
>>>>>>>>>>> to
>>>>>>>>>>> provide the abillity to run VMs without having a
>>>>>>>>>>> running
>>>>>>>>>>> engine/db.
>>>>>>>>>>> Currently ovf update is done synchronously when
>>>>>>>>>>> performing
>>>>>>>>>>> various
>>>>>>>>>>> operations on vms/templates - update, save,
>>>>>>>>>>> adding/removing
>>>>>>>>>>> a
>>>>>>>>>>> disk,
>>>>>>>>>>> etc. What's more, currently updating the ovf
>>>>>>>>>>> (updateVM
>>>>>>>>>>> vdsm
>>>>>>>>>>> call)
>>>>>>>>>>> is
>>>>>>>>>>> usually done within a transcation.
>>>>>>>>>>>
>>>>>>>>>>> The idea behined OvfAutoUpdater is to perform batch
>>>>>>>>>>> ovf
>>>>>>>>>>> update
>>>>>>>>>>> operations that aggregate all outstanding updates per
>>>>>>>>>>> data
>>>>>>>>>>> center.
>>>>>>>>>>> These updates will be done in specifed time intervals
>>>>>>>>>>> which
>>>>>>>>>>> will
>>>>>>>>>>> reduce XML-RPC calls and will enable the removal of
>>>>>>>>>>> this
>>>>>>>>>>> syncronous
>>>>>>>>>>> vdsm call from all over the code.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Liron Aravot.
>
> Are you going to expose the config value via engine-config?

Another question -

Not sure i understand the part of

basically a corner case which we can take in order to avoid using locks 
which will affect the whole system and will prevent user initiated 
operations.

What do you mean with "we can take" - do you mean that we can live with 
that?
Why do we want to do that?

>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Engine-devel mailing list
>>>>>>>>>>> Engine-devel at ovirt.org
>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Engine-devel mailing list
>>>>>>>>> Engine-devel at ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Engine-devel mailing list
>>>>>>>> Engine-devel at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

More information about the Devel mailing list