Begin forwarded message: =20 From: Martin Kletzander <mkletzan@redhat.com> Subject: [kubevirt-dev] Project for profiles and defaults for libvirt = domains Date: 20 March 2018 at 15:20:31 CET To: libvir-list@redhat.com, openstack-dev@lists.openstack.org, = ovirt-devel@redhat.com, kubevirt-dev@googlegroups.com, = virt-tools-list@redhat.com, cockpit-devel@lists.fedorahosted.org =20 Hi everyone! =20 First of all sorry for such wide distribution, but apparently that's =

best way to make sure we cooperate nicely. So please be considerate = as this is a cross-post between huge amount of mailing lists. =20 After some discussions with developers from different projects that = work with libvirt one cannot but notice some common patterns and = workarounds. So I set off to see how can we make all our lives better and our = coding more effective (and maybe more fun as well). If all goes well we will create a project that will accommodate most of the defaulting, =

workarounds and other common algorithms around libvirt domain definitions. And since early design gets you half way, I would like = to know your feedback on several key points as well as on the general = idea. Also correct me brutally in case I'm wrong. =20 In order to not get confused in the following descriptions, I will = refer to this project idea using the name `virtuned`, but there is really no name for it yet (although an abbreviation for "Virtualization Abstraction Definition and Hypervisor Delegation" would suit well, IMHO). =20 Here are some common problems and use cases that virtuned could solve (or help with). Don't take it as something that's impossible to solve on your own, but rather something that could be de-duplicated from multiple projects or "done right" instead of various hack-ish = solutions. =20 1) Default devices/values =20 Libvirt itself must default to whatever values there were before any particular element was introduced due to the fact that it strives to keep the guest ABI stable. That means, for example, that it can't = just add -vmcoreinfo option (for KASLR support) or magically add the =

device to all QEMU machines, even though it would be useful, as that would change the guest ABI. =20 For default values this is even more obvious. Let's say someone =

out some "pretty good" default values for various HyperV enlightenment feature tunables. Libvirt can't magically change them, but each one = of the projects building on top of it doesn't want to keep that list updated and take care of setting them in every new XML. Some projects don't even expose those to the end user as a knob, while others might. =20 One more thing could be automatically figuring out best values based = on libosinfo-provided data. =20 2) Policies =20 Lot of the time there are parts of the domain definition that need to = be added, but nobody really cares about them. Sometimes it's enough to have few templates, another time you might want to have a policy per-scenario and want to combine them in various ways. For example = with the data provided by point 1). =20 For example if you want PCI-Express, you need the q35 machine type, = but you don't really want to care about the machine type. Or you want to use SPICE, but you don't want to care about adding QXL. =20 What if some of these policies could be specified once (using some DSL for example), and used by virtuned to merge them in a unified and predictable way? =20 3) Abstracting the XML =20 This is probably just usable for stateless apps, but it might happen that some apps don't really want to care about the XML at all. They just want an abstract view of the domain, possibly add/remove a device and that's it. We could do that as well. I can't really tell how = much of a demand there is for it, though. =20 4) Identifying devices properly =20 In contrast to the previous point, stateful apps might have a problem identifying devices after hotplug. For example, let's say you don't care about the addresses and leave that up to libvirt. You hotplug a device into the domain and dump the new XML of it. Depending on what type of device it was, you might need to identify it based on = different values. It could be <target dev=3D''/> for disks, <mac address=3D''/> = for interfaces etc. For some devices it might not even be possible and = you need to remember the addresses of all the previous devices and then parse them just to identify that one device and then throw them away. =20 With new enough libvirt you could use the user aliases for that, but turns out it's not that easy to use them properly anyway. Also the aliases won't help users identify that device inside the guest. =20 <rant> We really should've gone with new attribute for the user alias instead of using an existing one, given how many problems that is causing. </rant> =20 5) Generating the right XML snippet for device hot-(un)plug =20 This is kind of related to some previous points. =20 When hot-plugging a device and creating an XML snippet for it, you = want to keep the defaults from point 1) and policies from 2) in mind. Or something related to the already existing domain which you can = describe systematically. And adding something for identification (see previous point). =20 Doing the hot-unplug is easy depending on how much information about that device is saved by your application. The less you save about the device (or show to the user in a GUI, if applicable) the harder it = might be to generate an XML that libvirt will accept. Again, some problems with this should be fixed in libvirt, some of them are easy to workaround. But having a common ground that takes care of this should help some projects. =20 Hot-unplug could be implemented just based on the alias. This is something that would fit into libvirt as well. =20 = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 To mention some pre-existing solutions: =20 - I understand OpenStack has some really sensible and wisely chosen and/or tested default values. =20 - I know KubeVirt has VirtualMachinePresets. That is something = closely related to points 1) and 2). Also their abstraction of the XML might be usable for point 3). =20 - There was an effort on creating policy based configuration of =

--Apple-Mail=_6D569541-1CA3-4944-AFE7-86FD31122B7D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii cross-posting to the right list, by mistake a wrong ovirt list was used = originally the policies, pvpanic figures libvirt

objects called libvirt-designer. This is closely related to points 2) and 3). Unfortunately there was no much going on lately and part of virt-manager repository has currently more features implemented with the same ideas in mind, just not exported for public use. =20 We could utilize some of the above to various extents. =20 Let me know what you think and have a nice day. Martin =20 --=20 You received this message because you are subscribed to the Google = Groups "kubevirt-dev" group. To unsubscribe from this group and stop receiving emails from it, send = an email to kubevirt-dev+unsubscribe@googlegroups.com. To post to this group, send email to kubevirt-dev@googlegroups.com. To view this discussion on the web visit = https://groups.google.com/d/msgid/kubevirt-dev/20180320142031.GB23007%40wh= eatley. For more options, visit https://groups.google.com/d/optout.

--Apple-Mail=_6D569541-1CA3-4944-AFE7-86FD31122B7D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D"">cross-posting to the right list, by mistake a wrong ovirt = list was used originally<div class=3D""><div><br class=3D""><blockquote = type=3D"cite" class=3D""><div class=3D"">Begin forwarded = message:</div><br class=3D"Apple-interchange-newline"><div = style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; = margin-left: 0px;" class=3D""><span style=3D"font-family: = -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; = color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">From: = </b></span><span style=3D"font-family: -webkit-system-font, Helvetica = Neue, Helvetica, sans-serif;" class=3D"">Martin Kletzander <<a = href=3D"mailto:mkletzan@redhat.com" = class=3D"">mkletzan@redhat.com</a>><br class=3D""></span></div><div = style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; = margin-left: 0px;" class=3D""><span style=3D"font-family: = -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; = color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Subject: = </b></span><span style=3D"font-family: -webkit-system-font, Helvetica = Neue, Helvetica, sans-serif;" class=3D""><b class=3D"">[kubevirt-dev] = Project for profiles and defaults for libvirt domains</b><br = class=3D""></span></div><div style=3D"margin-top: 0px; margin-right: = 0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span = style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, = sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">Date: = </b></span><span style=3D"font-family: -webkit-system-font, Helvetica = Neue, Helvetica, sans-serif;" class=3D"">20 March 2018 at 15:20:31 = CET<br class=3D""></span></div><div style=3D"margin-top: 0px; = margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=3D""><span= style=3D"font-family: -webkit-system-font, Helvetica Neue, Helvetica, = sans-serif; color:rgba(0, 0, 0, 1.0);" class=3D""><b class=3D"">To: = </b></span><span style=3D"font-family: -webkit-system-font, Helvetica = Neue, Helvetica, sans-serif;" class=3D""><a = href=3D"mailto:libvir-list@redhat.com" = class=3D"">libvir-list@redhat.com</a>, <a = href=3D"mailto:openstack-dev@lists.openstack.org" = class=3D"">openstack-dev@lists.openstack.org</a>, <a = href=3D"mailto:ovirt-devel@redhat.com" = class=3D"">ovirt-devel@redhat.com</a>, <a = href=3D"mailto:kubevirt-dev@googlegroups.com" = class=3D"">kubevirt-dev@googlegroups.com</a>, <a = href=3D"mailto:virt-tools-list@redhat.com" = class=3D"">virt-tools-list@redhat.com</a>, <a = href=3D"mailto:cockpit-devel@lists.fedorahosted.org" = class=3D"">cockpit-devel@lists.fedorahosted.org</a><br = class=3D""></span></div><br class=3D""><div class=3D""><div class=3D"">Hi = everyone!<br class=3D""><br class=3D"">First of all sorry for such wide = distribution, but apparently that's the<br class=3D"">best way to make = sure we cooperate nicely.  So please be considerate as<br = class=3D"">this is a cross-post between huge amount of mailing lists.<br = class=3D""><br class=3D"">After some discussions with developers from = different projects that work<br class=3D"">with libvirt one cannot but = notice some common patterns and workarounds.<br class=3D"">So I set off = to see how can we make all our lives better and our coding<br = class=3D"">more effective (and maybe more fun as well).  If all = goes well we will<br class=3D"">create a project that will accommodate = most of the defaulting, policies,<br class=3D"">workarounds and other = common algorithms around libvirt domain<br class=3D"">definitions. =  And since early design gets you half way, I would like to<br = class=3D"">know your feedback on several key points as well as on the = general idea.<br class=3D"">Also correct me brutally in case I'm = wrong.<br class=3D""><br class=3D"">In order to not get confused in the = following descriptions, I will refer<br class=3D"">to this project idea = using the name `virtuned`, but there is really no<br class=3D"">name for = it yet (although an abbreviation for "Virtualization<br = class=3D"">Abstraction Definition and Hypervisor Delegation" would suit = well,<br class=3D"">IMHO).<br class=3D""><br class=3D"">Here are some = common problems and use cases that virtuned could solve<br class=3D"">(or = help with).  Don't take it as something that's impossible to = solve<br class=3D"">on your own, but rather something that could be = de-duplicated from<br class=3D"">multiple projects or "done right" = instead of various hack-ish solutions.<br class=3D""><br class=3D"">1) = Default devices/values<br class=3D""><br class=3D"">Libvirt itself must = default to whatever values there were before any<br class=3D"">particular = element was introduced due to the fact that it strives to<br = class=3D"">keep the guest ABI stable.  That means, for example, = that it can't just<br class=3D"">add -vmcoreinfo option (for KASLR = support) or magically add the pvpanic<br class=3D"">device to all QEMU = machines, even though it would be useful, as that<br class=3D"">would = change the guest ABI.<br class=3D""><br class=3D"">For default values = this is even more obvious.  Let's say someone figures<br = class=3D"">out some "pretty good" default values for various HyperV = enlightenment<br class=3D"">feature tunables.  Libvirt can't = magically change them, but each one of<br class=3D"">the projects = building on top of it doesn't want to keep that list<br class=3D"">updated= and take care of setting them in every new XML.  Some projects<br = class=3D"">don't even expose those to the end user as a knob, while = others might.<br class=3D""><br class=3D"">One more thing could be = automatically figuring out best values based on<br = class=3D"">libosinfo-provided data.<br class=3D""><br class=3D"">2) = Policies<br class=3D""><br class=3D"">Lot of the time there are parts of = the domain definition that need to be<br class=3D"">added, but nobody = really cares about them.  Sometimes it's enough to<br class=3D"">have= few templates, another time you might want to have a policy<br = class=3D"">per-scenario and want to combine them in various ways. =  For example with<br class=3D"">the data provided by point 1).<br = class=3D""><br class=3D"">For example if you want PCI-Express, you need = the q35 machine type, but<br class=3D"">you don't really want to care = about the machine type.  Or you want to<br class=3D"">use SPICE, = but you don't want to care about adding QXL.<br class=3D""><br = class=3D"">What if some of these policies could be specified once (using = some DSL<br class=3D"">for example), and used by virtuned to merge them = in a unified and<br class=3D"">predictable way?<br class=3D""><br = class=3D"">3) Abstracting the XML<br class=3D""><br class=3D"">This is = probably just usable for stateless apps, but it might happen<br = class=3D"">that some apps don't really want to care about the XML at = all.  They<br class=3D"">just want an abstract view of the domain, = possibly add/remove a device<br class=3D"">and that's it.  We could = do that as well.  I can't really tell how much<br class=3D"">of a = demand there is for it, though.<br class=3D""><br class=3D"">4) = Identifying devices properly<br class=3D""><br class=3D"">In contrast to = the previous point, stateful apps might have a problem<br = class=3D"">identifying devices after hotplug.  For example, let's = say you don't<br class=3D"">care about the addresses and leave that up = to libvirt.  You hotplug a<br class=3D"">device into the domain and = dump the new XML of it.  Depending on what<br class=3D"">type of = device it was, you might need to identify it based on different<br = class=3D"">values.  It could be <target dev=3D''/> for disks, = <mac address=3D''/> for<br class=3D"">interfaces etc.  For = some devices it might not even be possible and you<br class=3D"">need to = remember the addresses of all the previous devices and then<br = class=3D"">parse them just to identify that one device and then throw = them away.<br class=3D""><br class=3D"">With new enough libvirt you = could use the user aliases for that, but<br class=3D"">turns out it's = not that easy to use them properly anyway.  Also the<br = class=3D"">aliases won't help users identify that device inside the = guest.<br class=3D""><br class=3D""><rant><br class=3D"">We really = should've gone with new attribute for the user alias instead<br = class=3D"">of using an existing one, given how many problems that is = causing.<br class=3D""></rant><br class=3D""><br class=3D"">5) = Generating the right XML snippet for device hot-(un)plug<br class=3D""><br= class=3D"">This is kind of related to some previous points.<br = class=3D""><br class=3D"">When hot-plugging a device and creating an XML = snippet for it, you want<br class=3D"">to keep the defaults from point = 1) and policies from 2) in mind.  Or<br class=3D"">something = related to the already existing domain which you can describe<br = class=3D"">systematically.  And adding something for identification = (see previous<br class=3D"">point).<br class=3D""><br class=3D"">Doing = the hot-unplug is easy depending on how much information about<br = class=3D"">that device is saved by your application.  The less you = save about the<br class=3D"">device (or show to the user in a GUI, if = applicable) the harder it might<br class=3D"">be to generate an XML that = libvirt will accept.  Again, some problems<br class=3D"">with this = should be fixed in libvirt, some of them are easy to<br = class=3D"">workaround.  But having a common ground that takes care = of this should<br class=3D"">help some projects.<br class=3D""><br = class=3D"">Hot-unplug could be implemented just based on the alias. =  This is<br class=3D"">something that would fit into libvirt as = well.<br class=3D""><br = class=3D"">=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D<br class=3D""><br class=3D"">To mention some pre-existing = solutions:<br class=3D""><br class=3D"">- I understand OpenStack has = some really sensible and wisely chosen<br class=3D""> and/or tested = default values.<br class=3D""><br class=3D"">- I know KubeVirt has = VirtualMachinePresets.  That is something closely<br class=3D""> = related to points 1) and 2).  Also their abstraction of the XML = might<br class=3D""> be usable for point 3).<br class=3D""><br = class=3D"">- There was an effort on creating policy based configuration = of libvirt<br class=3D""> objects called libvirt-designer.  This is = closely related to points 2)<br class=3D""> and 3).  Unfortunately = there was no much going on lately and part of<br class=3D""> = virt-manager repository has currently more features implemented with<br = class=3D""> the same ideas in mind, just not exported for public use.<br = class=3D""><br class=3D"">We could utilize some of the above to various = extents.<br class=3D""><br class=3D"">Let me know what you think and = have a nice day.<br class=3D"">Martin<br class=3D""><br class=3D"">-- = <br class=3D"">You received this message because you are subscribed to = the Google Groups "kubevirt-dev" group.<br class=3D"">To unsubscribe = from this group and stop receiving emails from it, send an email to <a = href=3D"mailto:kubevirt-dev+unsubscribe@googlegroups.com" = class=3D"">kubevirt-dev+unsubscribe@googlegroups.com</a>.<br class=3D"">To= post to this group, send email to <a = href=3D"mailto:kubevirt-dev@googlegroups.com" = class=3D"">kubevirt-dev@googlegroups.com</a>.<br class=3D"">To view this = discussion on the web visit <a = href=3D"https://groups.google.com/d/msgid/kubevirt-dev/20180320142031.GB23= 007%40wheatley" = class=3D"">https://groups.google.com/d/msgid/kubevirt-dev/20180320142031.G= B23007%40wheatley</a>.<br class=3D"">For more options, visit <a = href=3D"https://groups.google.com/d/optout" = class=3D"">https://groups.google.com/d/optout</a>.<br = class=3D""></div></div></blockquote></div><br = class=3D""></div></body></html>= --Apple-Mail=_6D569541-1CA3-4944-AFE7-86FD31122B7D--