[ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

[Nir Soffer]

On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas <ahadas at redhat.com> wrote:
>
>
> ----- Original Message -----
>> On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
>> > Hi All,
>> >
>> > We are working on something that is expected to have a big impact, hence
>> > this heads-up.
>> > First, we want you to be aware of this change and provide your feedback to
>> > make it as good as possible.
>> > Second, until the proposed mechanism is fully merged there will be a chase
>> > to cover all features unless new features are also implemented with the
>> > new mechanism. So please, if you are working on something that
>> > adds/changes something in the Libvirt's domain xml, do it with this new
>> > mechanism as well (first version would be merged soon).
>> >
>> > * Goal
>> > Creating Libvirt XML in the engine rather than in VDSM.
>> > ** Today's flow
>> > Engine: VM business entity -> VM properties map
>> > VDSM:   VM properties map  -> Libvirt XML
>> > ** Desired flow
>> > Engine: VM business entity -> Libvirt XML
>> >
>> > * Potential Benefits
>> > 1. Reduce the number of conversions from 2 to 1, reducing chances for
>> > mistakes in the process.
>> > 2. Reduce the amount of code in VDSM.
>> > 3. Make VM related changes easier - today many of these changes need to be
>> > reviewed in 2 projects, this will eliminate the one that tends to take
>> > longer.
>> > 4. Prevent shortcuts in the form of VDSM-only changes that should be better
>> > reflected in the engine.
>> > 5. Not to re-generate the XML on each rerun attempt of VM run/migration.
>> > 6. Future - not to re-generate the XML on each attempt to auto-start HA VM
>> > when using vm-leases (need to make sure we're using the up-to-date VM
>> > configuration though).
>> > 7. We already found improvements and cleanups that could be made while
>> > touching this area (e.g., remove the boot order from devices in the
>> > database).
>> >
>> > * Challenges
>> > 1. Not to move host-specific information to the engine. For example, path
>> > to storage domain or sockets of channels.
>> >    The solution is to use place-holders that will be replaced by VDSM.
>> > 2. Backward compatibility.
>> > 3. The more challenging part is the other direction - that will be the next
>> > phase.
>> >
>> > * Status
>> > As a first step, we began with producing the Libvirt XML in the engine by
>> > converting the VM properties map to XML in the engine [1]
>> > And using the XML that is received as an input in VDSM [2]
>> >
>> >
>> > [1] https://gerrit.ovirt.org/#/c/64473/
>> > [2] https://gerrit.ovirt.org/#/c/65182/
>>
>> I should start by saying that I love libvirt's domxml standard. Unlike
>> Vdsm's API, it is a real *standard* for defining VMs. In this regards,
>> you are suggesting a positive step.
>>
>> However, Engine is much more complex than Vdsm. It is also our
>> single-point-of-failure, and where CPU is the most scarce. I am worried
>> that in the foreseeable future it would only make Engine bigger, without
>> reducing the size and complexity of Vdsm.
>>
>> Before taking this move, we must map what Vdsm does, because that logic
>> would have to be copied into Engine. Few things pop up to mind:
>>
>> - pci addresses. would Vdsm report back the libvirt-assigned addresses
>>   in XML format, or would it keep parsing them?
>
> Ideally, VDSM will report back the devices in XML format.
> The engine will then add the unmanaged devices and update the pci addresses.
> Need to put some more thoughts into this, though.
>
>>
>> - hot plug. Device xml should be generated by Engine, much like in the
>>   vm cteate flow
>
> Good point, I didn't think of hot plugs - right, they could be changed as well later on.
>
>>
>> - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC
>>   that is connected no-where. Engine would need to care about what was
>>   up until now a vdsm-side implementation detail.
>
> Right, I almost finished to copy the creation of the network interfaces to the engine.
> This knowledge that you refer to will only be in the module that creates the XML, it doesn't seem to be much of an issue.
>
>>
>> - storage path. this was mentioned above, but actually, the paths are
>>   the same on all hosts. We inteded to have an abstraction layer there,
>>   but we never ever used it. All volumes sit under
>>   /rhev/data-center/poolID/domainID/imageID/volumeID
>>   Basically, Engine can hard-code this in the domxml, and no one would
>>   notice.

This is wrong, and engine cannot hard code this or anything else.

Engine should can describe only what is knows about disks, only vdsm
can add the disk xml.

>
> But I see that LUN and cinder disks are represented differently (not as PDIV) - I'll check this.

Of course, and even disks using DIV can modified in by vdsm, for
example glusterfs
using libgfapi.

>
>>
>> - OvS. Recently, we have changed how VMs can be connected to their
>>   network. It is possible (albeit not recommended yet!) to connect a VM
>>   to an OvS instead of Linux bridges. This is done without Engine really
>>   caring, or knowing how the domxml is modified.
>
> Yeah, I saw that. The only complication I see at this point is that for OvS we create more elements than only the 'source' element.
> I believe that we could use a place-holder that contains the network name and replace it with the tags that are needed for SR-IOV, OvS and ordinary interfaces, no?
> This seems to be the only thing that is difficult to generate on the engine's side (related to the network interfaces).
>
>>
>> - minor tweaks. exposing a new feature into Engine's UI is hard. Over
>>   the years, few tweaks have been pushed as custom properties.
>>   there are not many (I see now only sndbuf, queues, viodiskcache,
>>   vhost) but the implementation should make sure they are not forgotten.
>
> Sure.
>
>>
>> Maybe, Vdsm should consider Engine's domxml only as a "recomendation"
>> and modify it based on its hooks and custom properties. This can
>> surprise Engine, a defies the pupose of having xml-building logic moved
>> away from Vdsm.
>>
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel