Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine
I agree with Piotr&Edward: - having engine uses libvirt xml would impose coupling between engine and libvirt (a cross-layers dependency), which currently does not exist. - moreover, engine would be coupled to all versions of libvirt api that it manages - currently each VDSM version knows how to interact with libvirt it uses (de-centralized approach) - currently we do not have a mechanism that ensures libvirt version based on cluster level (IMHO). Ensuring that would imply requesting a very specific version of libvirt by VDSM. - cluster version means a minimum level of compatibility, but what if cluster of version (x.y) contains a host of a newer version (x+1.y+2). What would be able to tell about libvirt it uses? Regards, Yevgeny On Fri, Dec 9, 2016 at 12:58 PM, Martin Sivak <msivak@redhat.com> wrote:
I like transport abstractions (DTOs) and here we make the libvirt xml (its structure) part of our communication interface.
I agree, but only partially. We can have a DTO for this, but we need a library to convert from/to it. Hosted engine has to duplicate the current Java code as it has to convert the OVF to the format vdsm is expecting. And that is very error prone. Getting the domain XML from the engine and using it directly would solve a lot of bugs we have. Or alternatively the ability to send the OVF to vdsm without preprocessing.
On the other hand, the logic we have in vdsm is not that complicated and we could move it to the engine, because we have (almost) all the information there as well. The (complicated) logic needed to use vdsm verbs is part of engine anyway and I am constantly under pressure to not use (for example from virt) the verbs directly. It does not make sense to maintain the current separation in the situation where there is tight coupling anyway.
This whole discussion will boil to a simple point eventually: What is the role of vdsm and how dumb is it supposed to be in the future (it is pretty dumb currently.. just a conversion proxy + monitoring, at least in areas I am concerned about). I would like slightly smarter vdsm with engine acting only as the high level boss, without the micromanaging of storage verbs for example.
TLDR; I would like to get data from storage (currently OVF) and pass them to vdsm without any preprocessing. Separation is not a bad idea, but we currently have three different formats with no clear rules for conversion. I would also like to have slightly more high level API in vdsm ("prepareImage" should make sure domain monitor is up, domain is mounted and so on.. I do not want to micromanage that).
Martin
I like transport abstractions (DTOs) and here we make the libvirt xml (its structure) part of our communication interface. We were never good at evolving our api. Here is [1] recent example of it.
I am not against this change but I would like us to think about potential changes to this xml. With this approach when change happens we would need to change the engine and vdsm code and make sure that supported engine/vdsm matrix still work.
Will it be possible to have different libvirt versions (different xml structure) between version clusters? If so, how are we going to handle different xml structure per cluster?
Thanks, Piotr
[1] https://gerrit.ovirt.org/#/c/67596
On Thu, Dec 8, 2016 at 11:30 PM, Edward Haas <ehaas@redhat.com> wrote:
On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact,
hence
this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with
mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example,
to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik
This is an interesting path to take, but centralizing the logic to a single component often limits and does not allow scaling. A large amount of solutions these days attempt to distribute work, reducing central work to a minimum, but this approach suggests the opposite.
In the networking area, from my limited experience, changes are pushed faster to VDSM compared to Engine. In many cases it is just logically simpler: Engine needs to handle and be concern about all the system as a whole, while VDSM just takes care of
host. Therefore, in my mind, the goal is to distribute as much as possible to
On Fri, Dec 9, 2016 at 11:39 AM, Piotr Kliczewski <piotr.kliczewski@gmail.com> wrote: the new path the the
edges, keeping in the centre the minimum required to connect then all together.
This approach will remove a conversion and with it an abstraction layer. I find abstraction useful, decoupling components and increasing modularity. As an example from the OvS integration work, changing the underlying networking implementation should not concern the upper business logic components, it should be well hidden in the hypervisor, exposing only capabilities and nothing more that hints about the what and how.
Thanks, Edy.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel
Thanks for feedback, this is WIP, early stage, so your comments and your time to think about it is much appreciated. few comments/arguments/thoughts below On 04 Jan 2017, at 20:01, Yevgeny Zaspitsky <yzaspits@redhat.com> wrote: I agree with Piotr&Edward: - having engine uses libvirt xml would impose coupling between engine and libvirt (a cross-layers dependency), which currently does not exist. Yes, it creates a coupling. We do have such coupling already though, just more of it. I would refer you to the religious wars on "is vdsm api a general purpose api" Obviously there is another close coupling in vdsm<->libvirt as well - moreover, engine would be coupled to all versions of libvirt api that it manages - currently each VDSM version knows how to interact with libvirt it uses (de-centralized approach) Yes, same as engine is coupled to all versions of vdsms. - currently we do not have a mechanism that ensures libvirt version based on cluster level (IMHO). That is correct. OTOH we do have that thing in engine. This is effort is removing one layer of translation, but it doesn't magically solve/remove such mechanism - Ensuring that would imply requesting a very specific version of libvirt by VDSM. No, similarly as all vdsms are backward compatible, libvirt has the same (to certain degree, but actually in general they do a better job than vdsm due to wider exposure of the project) - cluster version means a minimum level of compatibility, but what if cluster of version (x.y) contains a host of a newer version (x+1.y+2). What would be able to tell about libvirt it uses? Same as today with engine and vdsm Regards, Yevgeny On Fri, Dec 9, 2016 at 12:58 PM, Martin Sivak <msivak@redhat.com> wrote:
I like transport abstractions (DTOs) and here we make the libvirt xml (its structure) part of our communication interface.
It's not only communication, it is a design change, to model entities and features close to how they are modeled in libvirt, participate on that _there_, contribute It does look quite when you look deep at vdsm minus storage and network code.
I agree, but only partially. We can have a DTO for this, but we need a library to convert from/to it. Hosted engine has to duplicate the current Java code as it has to convert the OVF to the format vdsm is
Yeah, this could have been avoided, but it was not, so you are right. expecting. And that is very error prone. Getting the domain XML from
the engine and using it directly would solve a lot of bugs we have. Or alternatively the ability to send the OVF to vdsm without preprocessing.
On the other hand, the logic we have in vdsm is not that complicated and we could move it to the engine, because we have (almost) all the information there as well. The (complicated) logic needed to use vdsm verbs is part of engine anyway and I am constantly under pressure to not use (for example from virt) the verbs directly. It does not make sense to maintain the current separation in the situation where there is tight coupling anyway.
Yep, I do see the pain of SLA/HE teams around that.
This whole discussion will boil to a simple point eventually: What is the role of vdsm and how dumb is it supposed to be in the future (it is pretty dumb currently.. just a conversion proxy + monitoring, at
Yep, especially in "virt" it's a dumb 1:1 proxy with its quirks and tedious boilerplate not helping anyone. All that on top of dealing with libvirt quirks, boilerplate, etc. least in areas I am concerned about). I would like slightly smarter
vdsm with engine acting only as the high level boss, without the micromanaging of storage verbs for example.
TLDR; I would like to get data from storage (currently OVF) and pass them to vdsm without any preprocessing. Separation is not a bad idea, but we currently have three different formats with no clear rules for conversion. I would also like to have slightly more high level API in vdsm ("prepareImage" should make sure domain monitor is up, domain is mounted and so on.. I do not want to micromanage that).
It all does make a whole lot more sense when you do not think only within our current central, engine, dc, cluster, host, vdsm design/constraints
Martin
On Fri, Dec 9, 2016 at 11:39 AM, Piotr Kliczewski <piotr.kliczewski@gmail.com> wrote:
I like transport abstractions (DTOs) and here we make the libvirt xml (its structure) part of our communication interface. We were never good at evolving our api. Here is [1] recent example of it.
I am not against this change but I would like us to think about potential changes to this xml. With this approach when change happens we would need to change the engine and vdsm code and make sure that supported engine/vdsm matrix still work.
I do want to get to a point when there is no vdsm(in current sense) involved in elementary changes - those which are modeled and handled by libvirt (e.g. vm config updates, lifecycle changes, etc)
Will it be possible to have different libvirt versions (different xml structure) between version clusters? If so, how are we going to handle different xml structure per cluster?
Why a different structure? IOW how is that different from today where you need to be very careful with modification of vdsm verbs to not break any combination of supported cluster versions I would see same constraints, same level of effort, roughly. But within a wider community, and with one layer removed Thanks, michal
Thanks, Piotr
[1] https://gerrit.ovirt.org/#/c/67596
On Thu, Dec 8, 2016 at 11:30 PM, Edward Haas <ehaas@redhat.com> wrote:
On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact,
this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with
mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example,
to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik
This is an interesting path to take, but centralizing the logic to a single component often limits and does not allow scaling. A large amount of solutions these days attempt to distribute work, reducing central work to a minimum, but this approach suggests the opposite.
In the networking area, from my limited experience, changes are pushed faster to VDSM compared to Engine. In many cases it is just logically simpler: Engine needs to handle and be concern about all the system as a whole, while VDSM just takes care of
host. Therefore, in my mind, the goal is to distribute as much as possible to
hence the new path the the
edges, keeping in the centre the minimum required to connect then all together.
This approach will remove a conversion and with it an abstraction layer. I find abstraction useful, decoupling components and increasing modularity. As an example from the OvS integration work, changing the underlying networking implementation should not concern the upper business logic components, it should be well hidden in the hypervisor, exposing only capabilities and nothing more that hints about the what and how.
Thanks, Edy.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.phx.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (2)
-
Michal Skrivanek -
Yevgeny Zaspitsky