Heads-up: moving Libvirt xml creation to the engine
Hi All, We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon). * Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML * Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database). * Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase. * Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2] [1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/ Regards, Arik
--IO7NNQOluoOKbJ3SiH5OQ2qAtQXtRVcvw Content-Type: multipart/mixed; boundary="fNDvEC3nBnSjb0l71fCnA3e548ooCxqfJ"; protected-headers="v1" From: Sven Kieske <s.kieske@mittwald.de> To: devel@ovirt.org Message-ID: <a9c99f7a-82b6-bc93-6e01-ce36666d3b4d@mittwald.de> Subject: Re: [ovirt-devel] Heads-up: moving Libvirt xml creation to the engine References: <929621053.971772.1479905980625.JavaMail.zimbra@redhat.com> In-Reply-To: <929621053.971772.1479905980625.JavaMail.zimbra@redhat.com> --fNDvEC3nBnSjb0l71fCnA3e548ooCxqfJ Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, very first question which comes to mind is: will vdsm hooks which alter the libvirt xml still work? I believe this is a much used feature and if it would not work anymore out of the box this would be a huge pain in upgrading existing systems. furthermore I don't really see the benfits when you generate the xml in engine: you still need to alter it in vdsm as you write, to fill in host specific things like storagepaths anyway, meaning you have to maintain twice the codebase _and_ coordinate between the two of them, so nothing goes wrong if one project changes something in the xml. but this are just my 2 cent :-) --=20 Mit freundlichen Gr=FC=DFen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG K=F6nigsberger Stra=DFe 6 32339 Espelkamp T: +495772 293100 F: +495772 293333 https://www.mittwald.de Gesch=E4ftsf=FChrer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhause= n Komplement=E4rin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynha= usen --fNDvEC3nBnSjb0l71fCnA3e548ooCxqfJ-- --IO7NNQOluoOKbJ3SiH5OQ2qAtQXtRVcvw Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJYNZdbAAoJEMby9TMDAbQR/9AP/2vV10htQJ451J34OG0IdaVz UDHpVAAwlDp167ijmCYi5j8/cWhpu8bVl19p9N1euWRuNMv/8Z425IaNPp5kqM7u kL5ytNpcpTh36yhRbTT1tXl4gdttk+2vsHJ6KnKroIDSAVKHWQEfqJcOAfwMUJh5 5IBMT5f34y0x8DItEGA8sak26fP/UeaH71ZuKbKB2DUpNPWcebeId91XjTj0VjPf HkVbcvac8G96AooaTaNcs8w2U9ZRIEDBtlDNpJ99tjq6vCGa/ND4HiwPuYjY017M 1/knq0zSSnSZpMamcvgQ7YXY3puLCyiR6spOtqHdk6Xxk3xGlnrdt3dR3NLMZkjL PYXi1oUthGj5kSqegQB+kz2YrQ3tKwZvArtWNi6f701jtNRzFlnOouiKYTdhT/AM vvtRDmLaBhJB9jObDvoVht+Mwlr0znVzwH18Oh6R3PSlnX47RluMEU0eG99JA0vQ 6kUyyaAAPBtarFqWIHialE7dJXfnEOjfdxDPS1azyRowv6T1hvtXUUflJ24bh5HK XP1PdJlttdBDxWJ6wFiQlsaEyEFjPBvNKwf88NLQ9TP7MYhRnCXeSNwVJG1vb6ua pIltWX/PhSnVKO15PUcnlgXpb49yVxuxahYRA7YpQL/psmZBmVf2lrCNmfr7NNVt RaYR6mM2SLTdHWYJlKIi =uOPB -----END PGP SIGNATURE----- --IO7NNQOluoOKbJ3SiH5OQ2qAtQXtRVcvw--
----- Original Message -----
Hi,
very first question which comes to mind is: will vdsm hooks which alter the libvirt xml still work?
I believe this is a much used feature and if it would not work anymore out of the box this would be a huge pain in upgrading existing systems.
Yes, the hooks will still work. Please look at my response to didi regarding VDSM hooks.
furthermore I don't really see the benfits when you generate the xml in engine:
you still need to alter it in vdsm as you write, to fill in host specific things like storagepaths anyway, meaning you have to maintain twice the codebase _and_ coordinate between the two of them, so nothing goes wrong if one project changes something in the xml.
The removal of the code that is being executed before the hooks are called which creates the XML from the VM properties map would justify the effort. Once VDSM will support only the new API (of getting Libvirt XML as an input) this code could be removed.
but this are just my 2 cent :-)
-- Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +495772 293100 F: +495772 293333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
Thanks for the heads up! How does this, if at all, affect: 1. vdsm hooks 2. The interaction of engine, hosted-engine HA agent, engine vm configuration saved in shared storage, etc.? Will this require a change in ha-agent and the saved configuration? Best,
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Didi
----- Original Message -----
On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
Thanks for the heads up!
How does this, if at all, affect: 1. vdsm hooks
VDSM hooks will remain unaffected - they will get the same XML. The only difference is that this XML will be one that was produced by the engine after the replacement of its place-holders rather than one that is produced by VDSM from the properties map.
2. The interaction of engine, hosted-engine HA agent, engine vm configuration saved in shared storage, etc.? Will this require a change in ha-agent and the saved configuration?
In the foreseeable future VDSM should support creating the VM from VM properties map as well - so legacy code will keep working, including the creation of the hosted-engine. However, at some point the creation of the hosted-engine would also need to change I assume. The rest, other than the creation flow, will also remain unaffected.
Best,
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Didi
On 23 Nov 2016, at 14:53, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
Thanks for the heads up!
How does this, if at all, affect: 1. vdsm hooks
VDSM hooks will remain unaffected - they will get the same XML. The only difference is that this XML will be one that was produced by the engine after the replacement of its place-holders rather than one that is produced by VDSM from the properties map.
2. The interaction of engine, hosted-engine HA agent, engine vm configuration saved in shared storage, etc.? Will this require a change in ha-agent and the saved configuration?
In the foreseeable future VDSM should support creating the VM from VM properties map as well - so legacy code will keep working, including the creation of the hosted-engine. However, at some point the creation of the hosted-engine would also need to change I assume.
it would be a bit more straightforward than today, instead of the fragile code creating vdsm internal API properties you would be able to use a regular domain xml definition which is easier to define and maintain than internal representation of features/parameters of vdsm
The rest, other than the creation flow, will also remain unaffected.
Best,
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Didi
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Hi, Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example). Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart. Martin On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
----- Original Message -----
Hi,
Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example).
Besides the parsing of the devices that we would love to get rid of, I believe the parsing of Libvirt XML done by VDSM would remain as is.
Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart.
Not sure that I fully understand. You have a VM with configuration conf1 running on host1. While being migrated to host2 the configuration is changed to conf2. Do you suggest that the next time the VM restarts it would be created based on conf2? Changes in the devices would be reflected on the next run (assuming the engine got the updated devices) - as today. Other than the devices, no configuration that is used on VM creation (static data) is read back by the engine so if it is changed it won't be reflected. And at the moment we have no plan to change the content of the data that is passed between the engine and VDSM, only its representation. Btw, I'm taking back my statement about migration: "5. Not to re-generate the XML on each rerun attempt of VM run/migration." VM migration flow is irrelevant since the engine doesn't pass the VM configuration. It will remain as-is.
Martin
On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Ok, great. Yeah, you confused me by the migration sentence. Martin On Wed, Nov 23, 2016 at 5:12 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
Hi,
Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example).
Besides the parsing of the devices that we would love to get rid of, I believe the parsing of Libvirt XML done by VDSM would remain as is.
Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart.
Not sure that I fully understand. You have a VM with configuration conf1 running on host1. While being migrated to host2 the configuration is changed to conf2. Do you suggest that the next time the VM restarts it would be created based on conf2?
Changes in the devices would be reflected on the next run (assuming the engine got the updated devices) - as today. Other than the devices, no configuration that is used on VM creation (static data) is read back by the engine so if it is changed it won't be reflected. And at the moment we have no plan to change the content of the data that is passed between the engine and VDSM, only its representation.
Btw, I'm taking back my statement about migration: "5. Not to re-generate the XML on each rerun attempt of VM run/migration." VM migration flow is irrelevant since the engine doesn't pass the VM configuration. It will remain as-is.
Martin
On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Does this mean we will be able to move the hooks to the engine side at some point? That would be much better than needing it on VDSM. Yaniv Dary Technical Product Manager Red Hat Israel Ltd. 34 Jerusalem Road Building A, 4th floor Ra'anana, Israel 4350109 Tel : +972 (9) 7692306 8272306 Email: ydary@redhat.com IRC : ydary On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
Hi,
Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example).
Besides the parsing of the devices that we would love to get rid of, I believe the parsing of Libvirt XML done by VDSM would remain as is.
Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart.
Not sure that I fully understand. You have a VM with configuration conf1 running on host1. While being migrated to host2 the configuration is changed to conf2. Do you suggest that the next time the VM restarts it would be created based on conf2?
Changes in the devices would be reflected on the next run (assuming the engine got the updated devices) - as today. Other than the devices, no configuration that is used on VM creation (static data) is read back by the engine so if it is changed it won't be reflected. And at the moment we have no plan to change the content of the data that is passed between the engine and VDSM, only its representation.
Btw, I'm taking back my statement about migration: "5. Not to re-generate the XML on each rerun attempt of VM run/migration." VM migration flow is irrelevant since the engine doesn't pass the VM configuration. It will remain as-is.
Martin
On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact,
this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example,
hence path
to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 23 November 2016 at 19:27, Yaniv Dary <ydary@redhat.com> wrote:
Does this mean we will be able to move the hooks to the engine side at some point? That would be much better than needing it on VDSM.
Yaniv Dary Technical Product Manager Red Hat Israel Ltd. 34 Jerusalem Road Building A, 4th floor Ra'anana, Israel 4350109
Tel : +972 (9) 7692306 8272306 Email: ydary@redhat.com IRC : ydary
On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
Hi,
Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example).
Besides the parsing of the devices that we would love to get rid of, I believe the parsing of Libvirt XML done by VDSM would remain as is.
Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart.
Not sure that I fully understand. You have a VM with configuration conf1 running on host1. While being migrated to host2 the configuration is changed to conf2. Do you suggest that the next time the VM restarts it would be created based on conf2?
Changes in the devices would be reflected on the next run (assuming the engine got the updated devices) - as today. Other than the devices, no configuration that is used on VM creation (static data) is read back by the engine so if it is changed it won't be reflected. And at the moment we have no plan to change the content of the data that is passed between the engine and VDSM, only its representation.
Btw, I'm taking back my statement about migration: "5. Not to re-generate the XML on each rerun attempt of VM run/migration." VM migration flow is irrelevant since the engine doesn't pass the VM configuration. It will remain as-is.
Martin
On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact,
this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with
new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example,
to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be
hence the path the next
phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
+1 This simplifies the process of implementing features. We always have to encode the engine structures to vdsm and then libvirt, and get it wrong... hate it. - Do you know of limitations when passing data types? numbers floats dicts . XML always hides a surprise with data types. - I saw you used xml writer. Please look into JAXB. - If the schema is well maintained, another JAXB advantage would be that the XML we will emit will always be valid.
Regards,
Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
----- Original Message -----
On 23 November 2016 at 19:27, Yaniv Dary <ydary@redhat.com> wrote:
Does this mean we will be able to move the hooks to the engine side at some point? That would be much better than needing it on VDSM.
Yaniv Dary Technical Product Manager Red Hat Israel Ltd. 34 Jerusalem Road Building A, 4th floor Ra'anana, Israel 4350109
Tel : +972 (9) 7692306 8272306 Email: ydary@redhat.com IRC : ydary
On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
Hi,
Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example).
Besides the parsing of the devices that we would love to get rid of, I believe the parsing of Libvirt XML done by VDSM would remain as is.
Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart.
Not sure that I fully understand. You have a VM with configuration conf1 running on host1. While being migrated to host2 the configuration is changed to conf2. Do you suggest that the next time the VM restarts it would be created based on conf2?
Changes in the devices would be reflected on the next run (assuming the engine got the updated devices) - as today. Other than the devices, no configuration that is used on VM creation (static data) is read back by the engine so if it is changed it won't be reflected. And at the moment we have no plan to change the content of the data that is passed between the engine and VDSM, only its representation.
Btw, I'm taking back my statement about migration: "5. Not to re-generate the XML on each rerun attempt of VM run/migration." VM migration flow is irrelevant since the engine doesn't pass the VM configuration. It will remain as-is.
Martin
On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact,
this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with
new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example,
to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be
hence the path the next
phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
+1 This simplifies the process of implementing features. We always have to encode the engine structures to vdsm and then libvirt, and get it wrong... hate it.
- Do you know of limitations when passing data types? numbers floats dicts . XML always hides a surprise with data types.
1. We have no real alternative for generating XML, right? that's the API chosen for Libvirt that we are using.. 2. The goal is to generate exactly the same XML that VDSM produces (and that's possible since we are using the same data) and to pass it as a text - so this solution is expected to be as good as what we have today.
- I saw you used xml writer. Please look into JAXB.
- If the schema is well maintained, another JAXB advantage would be that the XML we will emit will always be valid.
Thanks but I believe that at this stage, although it is tempting for developers to talk about the implementation, the discussion should concentrate on more high-level/conceptual details. The discussion on lower-level/implementation details could (and should) be done later on, and separately.
Regards,
Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
----- Original Message -----
Does this mean we will be able to move the hooks to the engine side at some point? That would be much better than needing it on VDSM.
Well, it sounds feasible to provide a hooking mechanism that enables to alter the domain XML in the engine, at some point. I'm not sure though that it could actually replace the hooks in VDSM since: 1. The XML won't be complete (host-specific configuration will be missing). 2. If the user wants to do some operation on the host with the hook, he'll need to send a request from the engine to the host, which is more complicated.
Yaniv Dary Technical Product Manager Red Hat Israel Ltd. 34 Jerusalem Road Building A, 4th floor Ra'anana, Israel 4350109
Tel : +972 (9) 7692306 8272306 Email: ydary@redhat.com IRC : ydary
On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
Hi,
Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example).
Besides the parsing of the devices that we would love to get rid of, I believe the parsing of Libvirt XML done by VDSM would remain as is.
Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart.
Not sure that I fully understand. You have a VM with configuration conf1 running on host1. While being migrated to host2 the configuration is changed to conf2. Do you suggest that the next time the VM restarts it would be created based on conf2?
Changes in the devices would be reflected on the next run (assuming the engine got the updated devices) - as today. Other than the devices, no configuration that is used on VM creation (static data) is read back by the engine so if it is changed it won't be reflected. And at the moment we have no plan to change the content of the data that is passed between the engine and VDSM, only its representation.
Btw, I'm taking back my statement about migration: "5. Not to re-generate the XML on each rerun attempt of VM run/migration." VM migration flow is irrelevant since the engine doesn't pass the VM configuration. It will remain as-is.
Martin
On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact,
this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example,
hence path
to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Thu, Nov 24, 2016 at 02:23:11AM -0500, Arik Hadas wrote:
----- Original Message -----
Does this mean we will be able to move the hooks to the engine side at some point? That would be much better than needing it on VDSM.
Well, it sounds feasible to provide a hooking mechanism that enables to alter the domain XML in the engine, at some point. I'm not sure though that it could actually replace the hooks in VDSM since: 1. The XML won't be complete (host-specific configuration will be missing). 2. If the user wants to do some operation on the host with the hook, he'll need to send a request from the engine to the host, which is more complicated.
Still, many hooks are intended only for feature velocity - to make use of a libvirt feature without disturbing Engine or Vdsm. It makes a lot of sense to add a hooking point on Engine side, where people can plug their code.
Yaniv, we can probably provide engine side hooks as Arik said, but the current hook mechanism is provided by libvirt, vdsm "just" packages the scripts and puts them in the proper place. https://libvirt.org/hooks.html Martin On Wed, Nov 23, 2016 at 6:27 PM, Yaniv Dary <ydary@redhat.com> wrote:
Does this mean we will be able to move the hooks to the engine side at some point? That would be much better than needing it on VDSM.
Yaniv Dary Technical Product Manager Red Hat Israel Ltd. 34 Jerusalem Road Building A, 4th floor Ra'anana, Israel 4350109
Tel : +972 (9) 7692306 8272306 Email: ydary@redhat.com IRC : ydary
On Wed, Nov 23, 2016 at 6:12 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
Hi,
Arik there is also custom metadata section for MOM and QoS in the libvirt domain XML and those values are now created as XML and later manipulated using libvirt API. VDSM will still need to know at least some basic stuff about the XML to be able to process it correctly (the metadata descriptor and the structure for example).
Besides the parsing of the devices that we would love to get rid of, I believe the parsing of Libvirt XML done by VDSM would remain as is.
Will the engine assume anything about the XML post-migration (for example to speed-up restarts for highly available VMs)? Because the hooks can change it significantly. Start hooks will be used anyway, but migration hook changes might not be reflected during the restart.
Not sure that I fully understand. You have a VM with configuration conf1 running on host1. While being migrated to host2 the configuration is changed to conf2. Do you suggest that the next time the VM restarts it would be created based on conf2?
Changes in the devices would be reflected on the next run (assuming the engine got the updated devices) - as today. Other than the devices, no configuration that is used on VM creation (static data) is read back by the engine so if it is changed it won't be reflected. And at the moment we have no plan to change the content of the data that is passed between the engine and VDSM, only its representation.
Btw, I'm taking back my statement about migration: "5. Not to re-generate the XML on each rerun attempt of VM run/migration." VM migration flow is irrelevant since the engine doesn't pass the VM configuration. It will remain as-is.
Martin
On Wed, Nov 23, 2016 at 1:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Thu, Nov 24, 2016 at 09:47:53AM +0100, Martin Sivak wrote:
Yaniv, we can probably provide engine side hooks as Arik said, but the current hook mechanism is provided by libvirt, vdsm "just" packages the scripts and puts them in the proper place.
That's not precide. oVirt uses libvirt hooks only mariginally (the migration hook was used to clean the domxml from stuff unsupported on the destination). Most hooks modify domxml before it gets into libvirt.
On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
I should start by saying that I love libvirt's domxml standard. Unlike Vdsm's API, it is a real *standard* for defining VMs. In this regards, you are suggesting a positive step. However, Engine is much more complex than Vdsm. It is also our single-point-of-failure, and where CPU is the most scarce. I am worried that in the foreseeable future it would only make Engine bigger, without reducing the size and complexity of Vdsm. Before taking this move, we must map what Vdsm does, because that logic would have to be copied into Engine. Few things pop up to mind: - pci addresses. would Vdsm report back the libvirt-assigned addresses in XML format, or would it keep parsing them? - hot plug. Device xml should be generated by Engine, much like in the vm cteate flow - network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC that is connected no-where. Engine would need to care about what was up until now a vdsm-side implementation detail. - storage path. this was mentioned above, but actually, the paths are the same on all hosts. We inteded to have an abstraction layer there, but we never ever used it. All volumes sit under /rhev/data-center/poolID/domainID/imageID/volumeID Basically, Engine can hard-code this in the domxml, and no one would notice. - OvS. Recently, we have changed how VMs can be connected to their network. It is possible (albeit not recommended yet!) to connect a VM to an OvS instead of Linux bridges. This is done without Engine really caring, or knowing how the domxml is modified. - minor tweaks. exposing a new feature into Engine's UI is hard. Over the years, few tweaks have been pushed as custom properties. there are not many (I see now only sndbuf, queues, viodiskcache, vhost) but the implementation should make sure they are not forgotten. Maybe, Vdsm should consider Engine's domxml only as a "recomendation" and modify it based on its hooks and custom properties. This can surprise Engine, a defies the pupose of having xml-building logic moved away from Vdsm.
----- Original Message -----
On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
I should start by saying that I love libvirt's domxml standard. Unlike Vdsm's API, it is a real *standard* for defining VMs. In this regards, you are suggesting a positive step.
However, Engine is much more complex than Vdsm. It is also our single-point-of-failure, and where CPU is the most scarce. I am worried that in the foreseeable future it would only make Engine bigger, without reducing the size and complexity of Vdsm.
Before taking this move, we must map what Vdsm does, because that logic would have to be copied into Engine. Few things pop up to mind:
- pci addresses. would Vdsm report back the libvirt-assigned addresses in XML format, or would it keep parsing them?
Ideally, VDSM will report back the devices in XML format. The engine will then add the unmanaged devices and update the pci addresses. Need to put some more thoughts into this, though.
- hot plug. Device xml should be generated by Engine, much like in the vm cteate flow
Good point, I didn't think of hot plugs - right, they could be changed as well later on.
- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC that is connected no-where. Engine would need to care about what was up until now a vdsm-side implementation detail.
Right, I almost finished to copy the creation of the network interfaces to the engine. This knowledge that you refer to will only be in the module that creates the XML, it doesn't seem to be much of an issue.
- storage path. this was mentioned above, but actually, the paths are the same on all hosts. We inteded to have an abstraction layer there, but we never ever used it. All volumes sit under /rhev/data-center/poolID/domainID/imageID/volumeID Basically, Engine can hard-code this in the domxml, and no one would notice.
But I see that LUN and cinder disks are represented differently (not as PDIV) - I'll check this.
- OvS. Recently, we have changed how VMs can be connected to their network. It is possible (albeit not recommended yet!) to connect a VM to an OvS instead of Linux bridges. This is done without Engine really caring, or knowing how the domxml is modified.
Yeah, I saw that. The only complication I see at this point is that for OvS we create more elements than only the 'source' element. I believe that we could use a place-holder that contains the network name and replace it with the tags that are needed for SR-IOV, OvS and ordinary interfaces, no? This seems to be the only thing that is difficult to generate on the engine's side (related to the network interfaces).
- minor tweaks. exposing a new feature into Engine's UI is hard. Over the years, few tweaks have been pushed as custom properties. there are not many (I see now only sndbuf, queues, viodiskcache, vhost) but the implementation should make sure they are not forgotten.
Sure.
Maybe, Vdsm should consider Engine's domxml only as a "recomendation" and modify it based on its hooks and custom properties. This can surprise Engine, a defies the pupose of having xml-building logic moved away from Vdsm.
On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
I should start by saying that I love libvirt's domxml standard. Unlike Vdsm's API, it is a real *standard* for defining VMs. In this regards, you are suggesting a positive step.
However, Engine is much more complex than Vdsm. It is also our single-point-of-failure, and where CPU is the most scarce. I am worried that in the foreseeable future it would only make Engine bigger, without reducing the size and complexity of Vdsm.
Before taking this move, we must map what Vdsm does, because that logic would have to be copied into Engine. Few things pop up to mind:
- pci addresses. would Vdsm report back the libvirt-assigned addresses in XML format, or would it keep parsing them?
Ideally, VDSM will report back the devices in XML format. The engine will then add the unmanaged devices and update the pci addresses. Need to put some more thoughts into this, though.
- hot plug. Device xml should be generated by Engine, much like in the vm cteate flow
Good point, I didn't think of hot plugs - right, they could be changed as well later on.
- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC that is connected no-where. Engine would need to care about what was up until now a vdsm-side implementation detail.
Right, I almost finished to copy the creation of the network interfaces to the engine. This knowledge that you refer to will only be in the module that creates the XML, it doesn't seem to be much of an issue.
- storage path. this was mentioned above, but actually, the paths are the same on all hosts. We inteded to have an abstraction layer there, but we never ever used it. All volumes sit under /rhev/data-center/poolID/domainID/imageID/volumeID Basically, Engine can hard-code this in the domxml, and no one would notice.
This is wrong, and engine cannot hard code this or anything else. Engine should can describe only what is knows about disks, only vdsm can add the disk xml.
But I see that LUN and cinder disks are represented differently (not as PDIV) - I'll check this.
Of course, and even disks using DIV can modified in by vdsm, for example glusterfs using libgfapi.
- OvS. Recently, we have changed how VMs can be connected to their network. It is possible (albeit not recommended yet!) to connect a VM to an OvS instead of Linux bridges. This is done without Engine really caring, or knowing how the domxml is modified.
Yeah, I saw that. The only complication I see at this point is that for OvS we create more elements than only the 'source' element. I believe that we could use a place-holder that contains the network name and replace it with the tags that are needed for SR-IOV, OvS and ordinary interfaces, no? This seems to be the only thing that is difficult to generate on the engine's side (related to the network interfaces).
- minor tweaks. exposing a new feature into Engine's UI is hard. Over the years, few tweaks have been pushed as custom properties. there are not many (I see now only sndbuf, queues, viodiskcache, vhost) but the implementation should make sure they are not forgotten.
Sure.
Maybe, Vdsm should consider Engine's domxml only as a "recomendation" and modify it based on its hooks and custom properties. This can surprise Engine, a defies the pupose of having xml-building logic moved away from Vdsm.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
----- Original Message -----
On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
I should start by saying that I love libvirt's domxml standard. Unlike Vdsm's API, it is a real *standard* for defining VMs. In this regards, you are suggesting a positive step.
However, Engine is much more complex than Vdsm. It is also our single-point-of-failure, and where CPU is the most scarce. I am worried that in the foreseeable future it would only make Engine bigger, without reducing the size and complexity of Vdsm.
Before taking this move, we must map what Vdsm does, because that logic would have to be copied into Engine. Few things pop up to mind:
- pci addresses. would Vdsm report back the libvirt-assigned addresses in XML format, or would it keep parsing them?
Ideally, VDSM will report back the devices in XML format. The engine will then add the unmanaged devices and update the pci addresses. Need to put some more thoughts into this, though.
- hot plug. Device xml should be generated by Engine, much like in the vm cteate flow
Good point, I didn't think of hot plugs - right, they could be changed as well later on.
- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC that is connected no-where. Engine would need to care about what was up until now a vdsm-side implementation detail.
Right, I almost finished to copy the creation of the network interfaces to the engine. This knowledge that you refer to will only be in the module that creates the XML, it doesn't seem to be much of an issue.
- storage path. this was mentioned above, but actually, the paths are the same on all hosts. We inteded to have an abstraction layer there, but we never ever used it. All volumes sit under /rhev/data-center/poolID/domainID/imageID/volumeID Basically, Engine can hard-code this in the domxml, and no one would notice.
This is wrong, and engine cannot hard code this or anything else.
Engine should can describe only what is knows about disks, only vdsm can add the disk xml.
Of course, the engine will describe only the information it knows, but that seems to be most of the disk's related data. Let's say that the engine is managed to generate something like: <disk type='file' device='disk' snapshot='no'> <driver name='qemu' type='qcow2' cache='none'/> <source file='$PATH:PDIV or other representations$'/> <target dev='hda' bus='ide'/> <serial>54-a672-23e5b495a9ea</serial> </disk> (and similar structures for the other kind of disks). Then, VDSM can simply replace the placeholder $PATH...$ with the concrete path of the disk. For VDSM it would save a lot of code (most of the code in vdsm/virt/vmdevices/storage.py I suppose) What else doesn't the engine know? And couldn't this data be set by VDSM by replacing place-holders as in the example above?
But I see that LUN and cinder disks are represented differently (not as PDIV) - I'll check this.
Of course, and even disks using DIV can modified in by vdsm, for example glusterfs using libgfapi.
- OvS. Recently, we have changed how VMs can be connected to their network. It is possible (albeit not recommended yet!) to connect a VM to an OvS instead of Linux bridges. This is done without Engine really caring, or knowing how the domxml is modified.
Yeah, I saw that. The only complication I see at this point is that for OvS we create more elements than only the 'source' element. I believe that we could use a place-holder that contains the network name and replace it with the tags that are needed for SR-IOV, OvS and ordinary interfaces, no? This seems to be the only thing that is difficult to generate on the engine's side (related to the network interfaces).
- minor tweaks. exposing a new feature into Engine's UI is hard. Over the years, few tweaks have been pushed as custom properties. there are not many (I see now only sndbuf, queues, viodiskcache, vhost) but the implementation should make sure they are not forgotten.
Sure.
Maybe, Vdsm should consider Engine's domxml only as a "recomendation" and modify it based on its hooks and custom properties. This can surprise Engine, a defies the pupose of having xml-building logic moved away from Vdsm.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Tue, Nov 29, 2016 at 4:30 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
I should start by saying that I love libvirt's domxml standard. Unlike Vdsm's API, it is a real *standard* for defining VMs. In this regards, you are suggesting a positive step.
However, Engine is much more complex than Vdsm. It is also our single-point-of-failure, and where CPU is the most scarce. I am worried that in the foreseeable future it would only make Engine bigger, without reducing the size and complexity of Vdsm.
Before taking this move, we must map what Vdsm does, because that logic would have to be copied into Engine. Few things pop up to mind:
- pci addresses. would Vdsm report back the libvirt-assigned addresses in XML format, or would it keep parsing them?
Ideally, VDSM will report back the devices in XML format. The engine will then add the unmanaged devices and update the pci addresses. Need to put some more thoughts into this, though.
- hot plug. Device xml should be generated by Engine, much like in the vm cteate flow
Good point, I didn't think of hot plugs - right, they could be changed as well later on.
- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC that is connected no-where. Engine would need to care about what was up until now a vdsm-side implementation detail.
Right, I almost finished to copy the creation of the network interfaces to the engine. This knowledge that you refer to will only be in the module that creates the XML, it doesn't seem to be much of an issue.
- storage path. this was mentioned above, but actually, the paths are the same on all hosts. We inteded to have an abstraction layer there, but we never ever used it. All volumes sit under /rhev/data-center/poolID/domainID/imageID/volumeID Basically, Engine can hard-code this in the domxml, and no one would notice.
This is wrong, and engine cannot hard code this or anything else.
Engine should can describe only what is knows about disks, only vdsm can add the disk xml.
Of course, the engine will describe only the information it knows, but that seems to be most of the disk's related data. Let's say that the engine is managed to generate something like: <disk type='file' device='disk' snapshot='no'>
file may be "block" or "network", in glusterfs engine may send "file" and vdsm will replace it to "network".
<driver name='qemu' type='qcow2' cache='none'/> <source file='$PATH:PDIV or other representations$'/>
file may need to be "dev"
<target dev='hda' bus='ide'/> <serial>54-a672-23e5b495a9ea</serial> </disk>
Basically you must inspect the code in vdsm to understand the difference and what engine can do and cannot. Also, how will vdsm fix the xml without the pool, domin, image and volume ids? How do you want to send them if we send xml?
(and similar structures for the other kind of disks).
Then, VDSM can simply replace the placeholder $PATH...$ with the concrete path of the disk. For VDSM it would save a lot of code (most of the code in vdsm/virt/vmdevices/storage.py I suppose)
What else doesn't the engine know? And couldn't this data be set by VDSM by replacing place-holders as in the example above?
But I see that LUN and cinder disks are represented differently (not as PDIV) - I'll check this.
Of course, and even disks using DIV can modified in by vdsm, for example glusterfs using libgfapi.
- OvS. Recently, we have changed how VMs can be connected to their network. It is possible (albeit not recommended yet!) to connect a VM to an OvS instead of Linux bridges. This is done without Engine really caring, or knowing how the domxml is modified.
Yeah, I saw that. The only complication I see at this point is that for OvS we create more elements than only the 'source' element. I believe that we could use a place-holder that contains the network name and replace it with the tags that are needed for SR-IOV, OvS and ordinary interfaces, no? This seems to be the only thing that is difficult to generate on the engine's side (related to the network interfaces).
- minor tweaks. exposing a new feature into Engine's UI is hard. Over the years, few tweaks have been pushed as custom properties. there are not many (I see now only sndbuf, queues, viodiskcache, vhost) but the implementation should make sure they are not forgotten.
Sure.
Maybe, Vdsm should consider Engine's domxml only as a "recomendation" and modify it based on its hooks and custom properties. This can surprise Engine, a defies the pupose of having xml-building logic moved away from Vdsm.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
----- Original Message -----
On Tue, Nov 29, 2016 at 4:30 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
I should start by saying that I love libvirt's domxml standard. Unlike Vdsm's API, it is a real *standard* for defining VMs. In this regards, you are suggesting a positive step.
However, Engine is much more complex than Vdsm. It is also our single-point-of-failure, and where CPU is the most scarce. I am worried that in the foreseeable future it would only make Engine bigger, without reducing the size and complexity of Vdsm.
Before taking this move, we must map what Vdsm does, because that logic would have to be copied into Engine. Few things pop up to mind:
- pci addresses. would Vdsm report back the libvirt-assigned addresses in XML format, or would it keep parsing them?
Ideally, VDSM will report back the devices in XML format. The engine will then add the unmanaged devices and update the pci addresses. Need to put some more thoughts into this, though.
- hot plug. Device xml should be generated by Engine, much like in the vm cteate flow
Good point, I didn't think of hot plugs - right, they could be changed as well later on.
- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC that is connected no-where. Engine would need to care about what was up until now a vdsm-side implementation detail.
Right, I almost finished to copy the creation of the network interfaces to the engine. This knowledge that you refer to will only be in the module that creates the XML, it doesn't seem to be much of an issue.
- storage path. this was mentioned above, but actually, the paths are the same on all hosts. We inteded to have an abstraction layer there, but we never ever used it. All volumes sit under /rhev/data-center/poolID/domainID/imageID/volumeID Basically, Engine can hard-code this in the domxml, and no one would notice.
This is wrong, and engine cannot hard code this or anything else.
Engine should can describe only what is knows about disks, only vdsm can add the disk xml.
Of course, the engine will describe only the information it knows, but that seems to be most of the disk's related data. Let's say that the engine is managed to generate something like: <disk type='file' device='disk' snapshot='no'>
file may be "block" or "network", in glusterfs engine may send "file" and vdsm will replace it to "network".
Right right, that's what I meant by "and similar structures for the other kind of disks" - basically we'll move the same logic that determines this to the engine in order to produce the right elements.
<driver name='qemu' type='qcow2' cache='none'/> <source file='$PATH:PDIV or other representations$'/>
file may need to be "dev"
Same, we'll check the diskType in the engine in order to determine the right attributes.
<target dev='hda' bus='ide'/> <serial>54-a672-23e5b495a9ea</serial> </disk>
Basically you must inspect the code in vdsm to understand the difference and what engine can do and cannot.
Also, how will vdsm fix the xml without the pool, domin, image and volume ids?
How do you want to send them if we send xml?
Note that the placeholder contains this information ($PATH:PDIV or other representations$) It will be the engine's responsibility to provide all the data that VDSM needs in order to produce the right replacements within the placeholders
(and similar structures for the other kind of disks).
Then, VDSM can simply replace the placeholder $PATH...$ with the concrete path of the disk. For VDSM it would save a lot of code (most of the code in vdsm/virt/vmdevices/storage.py I suppose)
What else doesn't the engine know? And couldn't this data be set by VDSM by replacing place-holders as in the example above?
But I see that LUN and cinder disks are represented differently (not as PDIV) - I'll check this.
Of course, and even disks using DIV can modified in by vdsm, for example glusterfs using libgfapi.
- OvS. Recently, we have changed how VMs can be connected to their network. It is possible (albeit not recommended yet!) to connect a VM to an OvS instead of Linux bridges. This is done without Engine really caring, or knowing how the domxml is modified.
Yeah, I saw that. The only complication I see at this point is that for OvS we create more elements than only the 'source' element. I believe that we could use a place-holder that contains the network name and replace it with the tags that are needed for SR-IOV, OvS and ordinary interfaces, no? This seems to be the only thing that is difficult to generate on the engine's side (related to the network interfaces).
- minor tweaks. exposing a new feature into Engine's UI is hard. Over the years, few tweaks have been pushed as custom properties. there are not many (I see now only sndbuf, queues, viodiskcache, vhost) but the implementation should make sure they are not forgotten.
Sure.
Maybe, Vdsm should consider Engine's domxml only as a "recomendation" and modify it based on its hooks and custom properties. This can surprise Engine, a defies the pupose of having xml-building logic moved away from Vdsm.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Tue, Nov 29, 2016 at 5:09 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Tue, Nov 29, 2016 at 4:30 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Tue, Nov 29, 2016 at 2:21 PM, Arik Hadas <ahadas@redhat.com> wrote:
----- Original Message -----
On Wed, Nov 23, 2016 at 07:59:40AM -0500, Arik Hadas wrote: > Hi All, > > We are working on something that is expected to have a big impact, > hence > this heads-up. > First, we want you to be aware of this change and provide your > feedback > to > make it as good as possible. > Second, until the proposed mechanism is fully merged there will be a > chase > to cover all features unless new features are also implemented with > the > new mechanism. So please, if you are working on something that > adds/changes something in the Libvirt's domain xml, do it with this > new > mechanism as well (first version would be merged soon). > > * Goal > Creating Libvirt XML in the engine rather than in VDSM. > ** Today's flow > Engine: VM business entity -> VM properties map > VDSM: VM properties map -> Libvirt XML > ** Desired flow > Engine: VM business entity -> Libvirt XML > > * Potential Benefits > 1. Reduce the number of conversions from 2 to 1, reducing chances for > mistakes in the process. > 2. Reduce the amount of code in VDSM. > 3. Make VM related changes easier - today many of these changes need > to > be > reviewed in 2 projects, this will eliminate the one that tends to > take > longer. > 4. Prevent shortcuts in the form of VDSM-only changes that should be > better > reflected in the engine. > 5. Not to re-generate the XML on each rerun attempt of VM > run/migration. > 6. Future - not to re-generate the XML on each attempt to auto-start > HA > VM > when using vm-leases (need to make sure we're using the up-to-date VM > configuration though). > 7. We already found improvements and cleanups that could be made > while > touching this area (e.g., remove the boot order from devices in the > database). > > * Challenges > 1. Not to move host-specific information to the engine. For example, > path > to storage domain or sockets of channels. > The solution is to use place-holders that will be replaced by > VDSM. > 2. Backward compatibility. > 3. The more challenging part is the other direction - that will be > the > next > phase. > > * Status > As a first step, we began with producing the Libvirt XML in the > engine > by > converting the VM properties map to XML in the engine [1] > And using the XML that is received as an input in VDSM [2] > > > [1] https://gerrit.ovirt.org/#/c/64473/ > [2] https://gerrit.ovirt.org/#/c/65182/
I should start by saying that I love libvirt's domxml standard. Unlike Vdsm's API, it is a real *standard* for defining VMs. In this regards, you are suggesting a positive step.
However, Engine is much more complex than Vdsm. It is also our single-point-of-failure, and where CPU is the most scarce. I am worried that in the foreseeable future it would only make Engine bigger, without reducing the size and complexity of Vdsm.
Before taking this move, we must map what Vdsm does, because that logic would have to be copied into Engine. Few things pop up to mind:
- pci addresses. would Vdsm report back the libvirt-assigned addresses in XML format, or would it keep parsing them?
Ideally, VDSM will report back the devices in XML format. The engine will then add the unmanaged devices and update the pci addresses. Need to put some more thoughts into this, though.
- hot plug. Device xml should be generated by Engine, much like in the vm cteate flow
Good point, I didn't think of hot plugs - right, they could be changed as well later on.
- network rewiring. Vdsm uses the "dummy bridge" to implement a vNIC that is connected no-where. Engine would need to care about what was up until now a vdsm-side implementation detail.
Right, I almost finished to copy the creation of the network interfaces to the engine. This knowledge that you refer to will only be in the module that creates the XML, it doesn't seem to be much of an issue.
- storage path. this was mentioned above, but actually, the paths are the same on all hosts. We inteded to have an abstraction layer there, but we never ever used it. All volumes sit under /rhev/data-center/poolID/domainID/imageID/volumeID Basically, Engine can hard-code this in the domxml, and no one would notice.
This is wrong, and engine cannot hard code this or anything else.
Engine should can describe only what is knows about disks, only vdsm can add the disk xml.
Of course, the engine will describe only the information it knows, but that seems to be most of the disk's related data. Let's say that the engine is managed to generate something like: <disk type='file' device='disk' snapshot='no'>
file may be "block" or "network", in glusterfs engine may send "file" and vdsm will replace it to "network".
Right right, that's what I meant by "and similar structures for the other kind of disks" - basically we'll move the same logic that determines this to the engine in order to produce the right elements.
<driver name='qemu' type='qcow2' cache='none'/> <source file='$PATH:PDIV or other representations$'/>
file may need to be "dev"
Same, we'll check the diskType in the engine in order to determine the right attributes.
diskType that engine send is wrong, it is correct only for network disks (cpeh, glustefs), engine does send incorrect type for file and block based disks.
<target dev='hda' bus='ide'/> <serial>54-a672-23e5b495a9ea</serial> </disk>
Basically you must inspect the code in vdsm to understand the difference and what engine can do and cannot.
Also, how will vdsm fix the xml without the pool, domin, image and volume ids?
How do you want to send them if we send xml?
Note that the placeholder contains this information ($PATH:PDIV or other representations$)
Lets not invent such placeholders. If vdsm needs a list of drives, it must get them in properly, for example as list of dicts, documented in the schema, or add the data inside the xml: <vdsm_data> <sd_id>xxxyyy</sd_id> </vdsm_data> Or move the needed logic to engine. If vdsm reports to engine the storage repo path (/rhev/data-center or /run/vdsm/data-cetner), engine can build the paths. This path must be the same on all the hosts otherwise you could not migrate vms from host to host. Same for LUNs, vdsm reports today the unique name of the device, but it could report the full path (/dev/mapper/xxxyyy). This path is always the same on all the hosts in the cluster (by multipath configuration). The reason why glusterfs disk may change from file to network is we do not manage glusterfs properly on engine. Instead, we have code in vdsm contacting gluster server and fetching the information needed to create the network disk xml. If this was managed properly on engine, like other storage types, engine would have all the information needed to create the xml. Same for other details, like using file or dev - engine does not send the type to vdsm although engine knows the storage domain type, so vdsm have to go and check if the path is a file or block device. This makes no sense file do not become block devices, so there is no need to perform this check on vdsm.
It will be the engine's responsibility to provide all the data that VDSM needs in order to produce the right replacements within the placeholders
(and similar structures for the other kind of disks).
Then, VDSM can simply replace the placeholder $PATH...$ with the concrete path of the disk. For VDSM it would save a lot of code (most of the code in vdsm/virt/vmdevices/storage.py I suppose)
What else doesn't the engine know? And couldn't this data be set by VDSM by replacing place-holders as in the example above?
But I see that LUN and cinder disks are represented differently (not as PDIV) - I'll check this.
Of course, and even disks using DIV can modified in by vdsm, for example glusterfs using libgfapi.
- OvS. Recently, we have changed how VMs can be connected to their network. It is possible (albeit not recommended yet!) to connect a VM to an OvS instead of Linux bridges. This is done without Engine really caring, or knowing how the domxml is modified.
Yeah, I saw that. The only complication I see at this point is that for OvS we create more elements than only the 'source' element. I believe that we could use a place-holder that contains the network name and replace it with the tags that are needed for SR-IOV, OvS and ordinary interfaces, no? This seems to be the only thing that is difficult to generate on the engine's side (related to the network interfaces).
- minor tweaks. exposing a new feature into Engine's UI is hard. Over the years, few tweaks have been pushed as custom properties. there are not many (I see now only sndbuf, queues, viodiskcache, vhost) but the implementation should make sure they are not forgotten.
Sure.
Maybe, Vdsm should consider Engine's domxml only as a "recomendation" and modify it based on its hooks and custom properties. This can surprise Engine, a defies the pupose of having xml-building logic moved away from Vdsm.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 11/23/2016 07:59 AM, Arik Hadas wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
This would be interesting in several aspects: - parsing it to learn what we expect. - but not throwing an exception if a hook changed the expected result/device-type to something we don't know/expect in the engine... - to make "external VMs" better integrated - so we can at some point allow external manipulation/creation of a VM.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Wed, Nov 23, 2016 at 2:59 PM, Arik Hadas <ahadas@redhat.com> wrote:
Hi All,
We are working on something that is expected to have a big impact, hence this heads-up. First, we want you to be aware of this change and provide your feedback to make it as good as possible. Second, until the proposed mechanism is fully merged there will be a chase to cover all features unless new features are also implemented with the new mechanism. So please, if you are working on something that adds/changes something in the Libvirt's domain xml, do it with this new mechanism as well (first version would be merged soon).
* Goal Creating Libvirt XML in the engine rather than in VDSM. ** Today's flow Engine: VM business entity -> VM properties map VDSM: VM properties map -> Libvirt XML ** Desired flow Engine: VM business entity -> Libvirt XML
* Potential Benefits 1. Reduce the number of conversions from 2 to 1, reducing chances for mistakes in the process. 2. Reduce the amount of code in VDSM. 3. Make VM related changes easier - today many of these changes need to be reviewed in 2 projects, this will eliminate the one that tends to take longer. 4. Prevent shortcuts in the form of VDSM-only changes that should be better reflected in the engine. 5. Not to re-generate the XML on each rerun attempt of VM run/migration. 6. Future - not to re-generate the XML on each attempt to auto-start HA VM when using vm-leases (need to make sure we're using the up-to-date VM configuration though). 7. We already found improvements and cleanups that could be made while touching this area (e.g., remove the boot order from devices in the database).
* Challenges 1. Not to move host-specific information to the engine. For example, path to storage domain or sockets of channels. The solution is to use place-holders that will be replaced by VDSM. 2. Backward compatibility. 3. The more challenging part is the other direction - that will be the next phase.
* Status As a first step, we began with producing the Libvirt XML in the engine by converting the VM properties map to XML in the engine [1] And using the XML that is received as an input in VDSM [2]
[1] https://gerrit.ovirt.org/#/c/64473/ [2] https://gerrit.ovirt.org/#/c/65182/
Regards, Arik
This is an interesting path to take, but centralizing the logic to a single component often limits and does not allow scaling. A large amount of solutions these days attempt to distribute work, reducing central work to a minimum, but this approach suggests the opposite. In the networking area, from my limited experience, changes are pushed faster to VDSM compared to Engine. In many cases it is just logically simpler: Engine needs to handle and be concern about all the system as a whole, while VDSM just takes care of the host. Therefore, in my mind, the goal is to distribute as much as possible to the edges, keeping in the centre the minimum required to connect then all together. This approach will remove a conversion and with it an abstraction layer. I find abstraction useful, decoupling components and increasing modularity. As an example from the OvS integration work, changing the underlying networking implementation should not concern the upper business logic components, it should be well hidden in the hypervisor, exposing only capabilities and nothing more that hints about the what and how. Thanks, Edy.
participants (11)
-
Arik Hadas -
Dan Kenigsberg -
Edward Haas -
Itamar Heim -
Martin Sivak -
Michal Skrivanek -
Nir Soffer -
Roy Golan -
Sven Kieske -
Yaniv Dary -
Yedidyah Bar David