Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine

> I like transport abstractions (DTOs) and here we make the libvirt xml > (its structure) part of our communication interface. I agree, but only partially. We can have a DTO for this, but we need a library to convert from/to it. Hosted engine has to duplicate the current Java code as it has to convert the OVF to the format vdsm is expecting. And that is very error prone. Getting the domain XML from the engine and using it directly would solve a lot of bugs we have. Or alternatively the ability to send the OVF to vdsm without preprocessing. On the other hand, the logic we have in vdsm is not that complicated and we could move it to the engine, because we have (almost) all the information there as well. The (complicated) logic needed to use vdsm verbs is part of engine anyway and I am constantly under pressure to not use (for example from virt) the verbs directly. It does not make sense to maintain the current separation in the situation where there is tight coupling anyway. This whole discussion will boil to a simple point eventually: What is the role of vdsm and how dumb is it supposed to be in the future (it is pretty dumb currently.. just a conversion proxy + monitoring, at least in areas I am concerned about). I would like slightly smarter vdsm with engine acting only as the high level boss, without the micromanaging of storage verbs for example. TLDR; I would like to get data from storage (currently OVF) and pass them to vdsm without any preprocessing. Separation is not a bad idea, but we currently have three different formats with no clear rules for conversion. I would also like to have slightly more high level API in vdsm ("prepareImage" should make sure domain monitor is up, domain is mounted and so on.. I do not want to micromanage that). Martin On Fri, Dec 9, 2016 at 11:39 AM, Piotr Kliczewski <piotr.kliczewski(a)gmail.com&gt; wrote: > I like transport abstractions (DTOs) and here we make the libvirt xml > (its structure) part of our communication interface. > We were never good at evolving our api. Here is [1] recent example of it. > > I am not against this change but I would like us to think about > potential changes to this xml. With this approach when change > happens we would need to change the engine and vdsm code and make sure > that supported engine/vdsm matrix still work. > > Will it be possible to have different libvirt versions (different xml > structure) between version clusters? If so, how are we going to > handle different xml structure per cluster? > > Thanks, > Piotr > > [1] https://gerrit.ovirt.org/#/c/67596 > > > On Thu, Dec 8, 2016 at 11:30 PM, Edward Haas <ehaas(a)redhat.com&gt; wrote: >> >> >> On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas(a)redhat.com&gt; wrote: >>> >>> Hi All, >>> >>> We are working on something that is expected to have a big impact, hence >>> this heads-up. >>> First, we want you to be aware of this change and provide your feedback to >>> make it as good as possible. >>> Second, until the proposed mechanism is fully merged there will be a chase >>> to cover all features unless new features are also implemented with the new >>> mechanism. So please, if you are working on something that adds/changes >>> something in the Libvirt's domain xml, do it with this new mechanism as well >>> (first version would be merged soon). >>> >>> * Goal >>> Creating Libvirt XML in the engine rather than in VDSM. >>> ** Today's flow >>> Engine: VM business entity -> VM properties map >>> VDSM: VM properties map -> Libvirt XML >>> ** Desired flow >>> Engine: VM business entity -> Libvirt XML >>> >>> * Potential Benefits >>> 1. Reduce the number of conversions from 2 to 1, reducing chances for >>> mistakes in the process. >>> 2. Reduce the amount of code in VDSM. >>> 3. Make VM related changes easier - today many of these changes need to be >>> reviewed in 2 projects, this will eliminate the one that tends to take >>> longer. >>> 4. Prevent shortcuts in the form of VDSM-only changes that should be >>> better reflected in the engine. >>> 5. Not to re-generate the XML on each rerun attempt of VM run/migration. >>> 6. Future - not to re-generate the XML on each attempt to auto-start HA VM >>> when using vm-leases (need to make sure we're using the up-to-date VM >>> configuration though). >>> 7. We already found improvements and cleanups that could be made while >>> touching this area (e.g., remove the boot order from devices in the >>> database). >>> >>> * Challenges >>> 1. Not to move host-specific information to the engine. For example, path >>> to storage domain or sockets of channels. >>> The solution is to use place-holders that will be replaced by VDSM. >>> 2. Backward compatibility. >>> 3. The more challenging part is the other direction - that will be the >>> next phase. >>> >>> * Status >>> As a first step, we began with producing the Libvirt XML in the engine by >>> converting the VM properties map to XML in the engine [1] >>> And using the XML that is received as an input in VDSM [2] >>> >>> >>> [1] https://gerrit.ovirt.org/#/c/64473/ >>> [2] https://gerrit.ovirt.org/#/c/65182/ >>> >>> Regards, >>> Arik >> >> >> This is an interesting path to take, but centralizing the logic to a single >> component often limits and does not allow scaling. >> A large amount of solutions these days attempt to distribute work, reducing >> central work to a minimum, but this approach suggests the opposite. >> >> In the networking area, from my limited experience, changes are pushed >> faster to VDSM compared to Engine. >> In many cases it is just logically simpler: Engine needs to handle and be >> concern about all the system as a whole, while VDSM just takes care of the >> host. >> Therefore, in my mind, the goal is to distribute as much as possible to the >> edges, keeping in the centre the minimum required to connect then all >> together. >> >> This approach will remove a conversion and with it an abstraction layer. I >> find abstraction useful, decoupling components and increasing modularity. >> As an example from the OvS integration work, changing the underlying >> networking implementation should not concern the upper business logic >> components, >> it should be well hidden in the hypervisor, exposing only capabilities and >> nothing more that hints about the what and how. >> >> Thanks, >> Edy. >> >> >> >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.phx.ovirt.org/mailman/listinfo/devel > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.phx.ovirt.org/mailman/listinfo/devel _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel