[ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Dan Kenigsberg]

On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
> Hi All,
> 
> We are working on something that is expected to have a big impact, hence this heads-up.
> First, we want you to be aware of this change and provide your feedback to make it as good as possible.
> Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
> 
> * Goal
> Creating Libvirt XML in the engine rather than in VDSM.
> ** Today's flow
> Engine: VM business entity -> VM properties map
> VDSM:   VM properties map  -> Libvirt XML
> ** Desired flow
> Engine: VM business entity -> Libvirt XML
> 
> * Potential Benefits
> 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process.
> 2. Reduce the amount of code in VDSM.
> 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer.
> 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine.
> 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
> 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though).
> 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
> 
> * Challenges
> 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels.
>    The solution is to use place-holders that will be replaced by VDSM.
> 2. Backward compatibility.
> 3. The more challenging part is the other direction - that will be the next phase.
> 
> * Status
> As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1]
> And using the XML that is received as an input in VDSM [2]
> 
> 
> [1] https://gerrit.ovirt.org/#/c/64473/
> [2] https://gerrit.ovirt.org/#/c/65182/

I should start by saying that I love libvirt's domxml standard. Unlike
Vdsm's API, it is a real *standard* for defining VMs. In this regards,
you are suggesting a positive step.

However, Engine is much more complex than Vdsm. It is also our
single-point-of-failure, and where CPU is the most scarce. I am worried
that in the foreseeable future it would only make Engine bigger, without
reducing the size and complexity of Vdsm.

Before taking this move, we must map what Vdsm does, because that logic
would have to be copied into Engine. Few things pop up to mind:

- pci addresses. would Vdsm report back the libvirt-assigned addresses
  in XML format, or would it keep parsing them?

- hot plug. Device xml should be generated by Engine, much like in the
  vm cteate flow

- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
  that is connected no-where. Engine would need to care about what was
  up until now a vdsm-side implementation detail.

- storage path. this was mentioned above, but actually, the paths are
  the same on all hosts. We inteded to have an abstraction layer there,
  but we never ever used it. All volumes sit under
  /rhev/data-center/poolID/domainID/imageID/volumeID
  Basically, Engine can hard-code this in the domxml, and no one would
  notice.

- OvS. Recently, we have changed how VMs can be connected to their
  network. It is possible (albeit not recommended yet!) to connect a VM
  to an OvS instead of Linux bridges. This is done without Engine really
  caring, or knowing how the domxml is modified.

- minor tweaks. exposing a new feature into Engine's UI is hard. Over
  the years, few tweaks have been pushed as custom properties.
  there are not many (I see now only sndbuf, queues, viodiskcache,
  vhost) but the implementation should make sure they are not forgotten.

Maybe, Vdsm should consider Engine's domxml only as a "recomendation"
and modify it based on its hooks and custom properties. This can
surprise Engine, a defies the pupose of having xml-building logic moved
away from Vdsm.