[ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Arik Hadas]

----- Original Message -----
> On 23 November 2016 at 19:27, Yaniv Dary <ydary at redhat.com> wrote:
> 
> > Does this mean we will be able to move the hooks to the engine side at
> > some point? That would be much better than needing it on VDSM.
> >
> > Yaniv Dary
> > Technical Product Manager
> > Red Hat Israel Ltd.
> > 34 Jerusalem Road
> > Building A, 4th floor
> > Ra'anana, Israel 4350109
> >
> > Tel : +972 (9) 7692306
> >         8272306
> > Email: ydary at redhat.com
> > IRC : ydary
> >
> >
> > On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas <ahadas at redhat.com> wrote:
> >
> >>
> >>
> >> ----- Original Message -----
> >> > Hi,
> >> >
> >> > Arik there is also custom metadata section for MOM and QoS in the
> >> > libvirt domain XML and those values are now created as XML and later
> >> > manipulated using libvirt API. VDSM will still need to know at least
> >> > some basic stuff about the XML to be able to process it correctly (the
> >> > metadata descriptor and the structure for example).
> >>
> >> Besides the parsing of the devices that we would love to get rid of, I
> >> believe the parsing of Libvirt XML done by VDSM would remain as is.
> >>
> >> >
> >> > Will the engine assume anything about the XML post-migration (for
> >> > example to speed-up restarts for highly available VMs)? Because the
> >> > hooks can change it significantly. Start hooks will be used anyway,
> >> > but migration hook changes might not be reflected during the restart.
> >> >
> >>
> >> Not sure that I fully understand.
> >> You have a VM with configuration conf1 running on host1. While being
> >> migrated to host2 the configuration is changed to conf2.
> >> Do you suggest that the next time the VM restarts it would be created
> >> based on conf2?
> >>
> >> Changes in the devices would be reflected on the next run (assuming the
> >> engine got the updated devices) - as today.
> >> Other than the devices, no configuration that is used on VM creation
> >> (static data) is read back by the engine so if it is changed it won't be
> >> reflected.
> >> And at the moment we have no plan to change the content of the data that
> >> is passed between the engine and VDSM, only its representation.
> >>
> >> Btw, I'm taking back my statement about migration:
> >> "5. Not to re-generate the XML on each rerun attempt of VM run/migration."
> >> VM migration flow is irrelevant since the engine doesn't pass the VM
> >> configuration. It will remain as-is.
> >>
> >> >
> >> > Martin
> >> >
> >> >
> >> >
> >> > On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas at redhat.com> wrote:
> >> > > Hi All,
> >> > >
> >> > > We are working on something that is expected to have a big impact,
> >> hence
> >> > > this heads-up.
> >> > > First, we want you to be aware of this change and provide your
> >> feedback to
> >> > > make it as good as possible.
> >> > > Second, until the proposed mechanism is fully merged there will be a
> >> chase
> >> > > to cover all features unless new features are also implemented with
> >> the
> >> > > new mechanism. So please, if you are working on something that
> >> > > adds/changes something in the Libvirt's domain xml, do it with this
> >> new
> >> > > mechanism as well (first version would be merged soon).
> >> > >
> >> > > * Goal
> >> > > Creating Libvirt XML in the engine rather than in VDSM.
> >> > > ** Today's flow
> >> > > Engine: VM business entity -> VM properties map
> >> > > VDSM:   VM properties map  -> Libvirt XML
> >> > > ** Desired flow
> >> > > Engine: VM business entity -> Libvirt XML
> >> > >
> >> > > * Potential Benefits
> >> > > 1. Reduce the number of conversions from 2 to 1, reducing chances for
> >> > > mistakes in the process.
> >> > > 2. Reduce the amount of code in VDSM.
> >> > > 3. Make VM related changes easier - today many of these changes need
> >> to be
> >> > > reviewed in 2 projects, this will eliminate the one that tends to take
> >> > > longer.
> >> > > 4. Prevent shortcuts in the form of VDSM-only changes that should be
> >> better
> >> > > reflected in the engine.
> >> > > 5. Not to re-generate the XML on each rerun attempt of VM
> >> run/migration.
> >> > > 6. Future - not to re-generate the XML on each attempt to auto-start
> >> HA VM
> >> > > when using vm-leases (need to make sure we're using the up-to-date VM
> >> > > configuration though).
> >> > > 7. We already found improvements and cleanups that could be made while
> >> > > touching this area (e.g., remove the boot order from devices in the
> >> > > database).
> >> > >
> >> > > * Challenges
> >> > > 1. Not to move host-specific information to the engine. For example,
> >> path
> >> > > to storage domain or sockets of channels.
> >> > >    The solution is to use place-holders that will be replaced by VDSM.
> >> > > 2. Backward compatibility.
> >> > > 3. The more challenging part is the other direction - that will be
> >> the next
> >> > > phase.
> >> > >
> >> > > * Status
> >> > > As a first step, we began with producing the Libvirt XML in the
> >> engine by
> >> > > converting the VM properties map to XML in the engine [1]
> >> > > And using the XML that is received as an input in VDSM [2]
> >> > >
> >> > >
> >> > > [1] https://gerrit.ovirt.org/#/c/64473/
> >> > > [2] https://gerrit.ovirt.org/#/c/65182/
> >> > >
> >>
> >
> +1 This simplifies the process of implementing features. We always have to
> encode the engine structures to vdsm and then
> libvirt, and get it wrong... hate it.
> 
> - Do you know of limitations when passing data types? numbers floats dicts
> . XML always hides a surprise with data types.

1. We have no real alternative for generating XML, right? that's the API chosen for Libvirt that we are using..
2. The goal is to generate exactly the same XML that VDSM produces (and that's possible since we are using the same data) and to pass it as a text - so this solution is expected to be as good as what we have today.

> 
> - I saw you used xml writer. Please look into JAXB.
> 
> - If the schema is well maintained, another JAXB advantage would be that
> the XML we will emit will always be valid.
> 

Thanks but I believe that at this stage, although it is tempting for developers to talk about the implementation, the discussion should concentrate on more high-level/conceptual details. The discussion on lower-level/implementation details could (and should) be done later on, and separately.

> 
> > > Regards,
> >> > > Arik
> >> > > _______________________________________________
> >> > > Devel mailing list
> >> > > Devel at ovirt.org
> >> > > http://lists.ovirt.org/mailman/listinfo/devel
> >> >
> >> _______________________________________________
> >> Devel mailing list
> >> Devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/devel
> >>
> >
> >
> > _______________________________________________
> > Devel mailing list
> > Devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> >
>