[ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

Wed Jan 4 19:00:53 UTC 2017

I agree with Piotr&Edward:

   - having engine uses libvirt xml would impose coupling between engine
   and libvirt (a cross-layers dependency), which currently does not exist.
   - moreover, engine would be coupled to all versions of libvirt api that
   it manages - currently each VDSM version knows how to interact with libvirt
   it uses (de-centralized approach)
   - currently we do not have a mechanism that ensures libvirt version
   based on cluster level (IMHO). Ensuring that would imply requesting a very
   specific version of libvirt by VDSM.
   - cluster version means a minimum level of compatibility, but what if
   cluster of version (x.y) contains a host of a newer version (x+1.y+2). What
   would be able to tell about libvirt it uses?

Regards,
Yevgeny

On Fri, Dec 9, 2016 at 12:58 PM, Martin Sivak <msivak at redhat.com> wrote:

> > I like transport abstractions (DTOs) and here we make the libvirt xml
> > (its structure) part of our communication interface.
>
> I agree, but only partially. We can have a DTO for this, but we need a
> library to convert from/to it. Hosted engine has to duplicate the
> current Java code as it has to convert the OVF to the format vdsm is
> expecting. And that is very error prone. Getting the domain XML from
> the engine and using it directly would solve a lot of bugs we have. Or
> alternatively the ability to send the OVF to vdsm without
> preprocessing.
>
> On the other hand, the logic we have in vdsm is not that complicated
> and we could move it to the engine, because we have (almost) all the
> information there as well. The (complicated) logic needed to use vdsm
> verbs is part of engine anyway and I am constantly under pressure to
> not use (for example from virt) the verbs directly. It does not make
> sense to maintain the current separation in the situation where there
> is tight coupling anyway.
>
> This whole discussion will boil to a simple point eventually: What is
> the role of vdsm and how dumb is it supposed to be in the future (it
> is pretty dumb currently.. just a conversion proxy + monitoring, at
> least in areas I am concerned about). I would like slightly smarter
> vdsm with engine acting only as the high level boss, without the
> micromanaging of storage verbs for example.
>
> TLDR; I would like to get data from storage (currently OVF) and pass
> them to vdsm without any preprocessing. Separation is not a bad idea,
> but we currently have three different formats with no clear rules for
> conversion. I would also like to have slightly more high level API in
> vdsm ("prepareImage" should make sure domain monitor is up, domain is
> mounted and so on.. I do not want to micromanage that).
>
> Martin
>
>
> On Fri, Dec 9, 2016 at 11:39 AM, Piotr Kliczewski
> <piotr.kliczewski at gmail.com> wrote:
> > I like transport abstractions (DTOs) and here we make the libvirt xml
> > (its structure) part of our communication interface.
> > We were never good at evolving our api. Here is [1] recent example of it.
> >
> > I am not against this change but I would like us to think about
> > potential changes to this xml. With this approach when change
> > happens we would need to change the engine and vdsm code and make sure
> > that supported engine/vdsm matrix still work.
> >
> > Will it be possible to have different libvirt versions (different xml
> > structure) between version clusters? If so, how are we going to
> > handle different xml structure per cluster?
> >
> > Thanks,
> > Piotr
> >
> > [1] https://gerrit.ovirt.org/#/c/67596
> >
> >
> > On Thu, Dec 8, 2016 at 11:30 PM, Edward Haas <ehaas at redhat.com> wrote:
> >>
> >>
> >> On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas at redhat.com> wrote:
> >>>
> >>> Hi All,
> >>>
> >>> We are working on something that is expected to have a big impact,
> hence
> >>> this heads-up.
> >>> First, we want you to be aware of this change and provide your
> feedback to
> >>> make it as good as possible.
> >>> Second, until the proposed mechanism is fully merged there will be a
> chase
> >>> to cover all features unless new features are also implemented with
> the new
> >>> mechanism. So please, if you are working on something that adds/changes
> >>> something in the Libvirt's domain xml, do it with this new mechanism
> as well
> >>> (first version would be merged soon).
> >>>
> >>> * Goal
> >>> Creating Libvirt XML in the engine rather than in VDSM.
> >>> ** Today's flow
> >>> Engine: VM business entity -> VM properties map
> >>> VDSM:   VM properties map  -> Libvirt XML
> >>> ** Desired flow
> >>> Engine: VM business entity -> Libvirt XML
> >>>
> >>> * Potential Benefits
> >>> 1. Reduce the number of conversions from 2 to 1, reducing chances for
> >>> mistakes in the process.
> >>> 2. Reduce the amount of code in VDSM.
> >>> 3. Make VM related changes easier - today many of these changes need
> to be
> >>> reviewed in 2 projects, this will eliminate the one that tends to take
> >>> longer.
> >>> 4. Prevent shortcuts in the form of VDSM-only changes that should be
> >>> better reflected in the engine.
> >>> 5. Not to re-generate the XML on each rerun attempt of VM
> run/migration.
> >>> 6. Future - not to re-generate the XML on each attempt to auto-start
> HA VM
> >>> when using vm-leases (need to make sure we're using the up-to-date VM
> >>> configuration though).
> >>> 7. We already found improvements and cleanups that could be made while
> >>> touching this area (e.g., remove the boot order from devices in the
> >>> database).
> >>>
> >>> * Challenges
> >>> 1. Not to move host-specific information to the engine. For example,
> path
> >>> to storage domain or sockets of channels.
> >>>    The solution is to use place-holders that will be replaced by VDSM.
> >>> 2. Backward compatibility.
> >>> 3. The more challenging part is the other direction - that will be the
> >>> next phase.
> >>>
> >>> * Status
> >>> As a first step, we began with producing the Libvirt XML in the engine
> by
> >>> converting the VM properties map to XML in the engine [1]
> >>> And using the XML that is received as an input in VDSM [2]
> >>>
> >>>
> >>> [1] https://gerrit.ovirt.org/#/c/64473/
> >>> [2] https://gerrit.ovirt.org/#/c/65182/
> >>>
> >>> Regards,
> >>> Arik
> >>
> >>
> >> This is an interesting path to take, but centralizing the logic to a
> single
> >> component often limits and does not allow scaling.
> >> A large amount of solutions these days attempt to distribute work,
> reducing
> >> central work to a minimum, but this approach suggests the opposite.
> >>
> >> In the networking area, from my limited experience, changes are pushed
> >> faster to VDSM compared to Engine.
> >> In many cases it is just logically simpler: Engine needs to handle and
> be
> >> concern about all the system as a whole, while VDSM just takes care of
> the
> >> host.
> >> Therefore, in my mind, the goal is to distribute as much as possible to
> the
> >> edges, keeping in the centre the minimum required to connect then all
> >> together.
> >>
> >> This approach will remove a conversion and with it an abstraction
> layer. I
> >> find abstraction useful, decoupling components and increasing
> modularity.
> >> As an example from the OvS integration work, changing the underlying
> >> networking implementation should not concern the upper business logic
> >> components,
> >> it should be well hidden in the hypervisor, exposing only capabilities
> and
> >> nothing more that hints about the what and how.
> >>
> >> Thanks,
> >> Edy.
> >>
> >>
> >>
> >> _______________________________________________
> >> Devel mailing list
> >> Devel at ovirt.org
> >> http://lists.phx.ovirt.org/mailman/listinfo/devel
> > _______________________________________________
> > Devel mailing list
> > Devel at ovirt.org
> > http://lists.phx.ovirt.org/mailman/listinfo/devel
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.phx.ovirt.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170104/57d4779f/attachment-0001.html>