From dann.frazier at canonical.com Sat Nov 5 16:45:58 2011 From: dann.frazier at canonical.com (dann frazier) Date: Sat, 5 Nov 2011 10:45:58 -0600 Subject: notes from "rethinking the architecture" breakout Message-ID: <20111105164558.GD11158@dannf.org> These are some notes I took from the Thursday afternoon breakout session initiated by Alexander Graf. I've also posted them here: http://ovirt.org/wiki/Rethinking_the_Architecture_breakout_-_oVirt_workshop_November_2011 libvirt is designed to be a hypervisor agnostic layer. For lots of hypervisors this is pretty straightforward as they offer APIs for vm ops (create/destroy/ etc). No such layer exists for kvm, so libvirt has a lot of kvm-specific code to supplement. VDSM sits on top of libvirt to make use of functionality which would normally be in this mid-layer. This is a layering design defect. Perhaps we could create a "kvmd" layer which manipulates qemu directly. libvirt would then be free to rebase upon this new midlayer, if desired. Alternatively, maybe the "extra" code in libvirt can be extracted to create this midlayer. People like the virtualbox UI - that too could be ported on top of "kvmd". VDSM maybe well suited to become the "kvmd" components. What interface would "kvmd" present? What about using the ESXi API vs. reinventing the wheel? RHEV-M team looked at it in the past - the API is very big, they didn't understand all of it, lots of ESX-specific stuff. Karl suggests that we should concentrate on the API to the host - we can switch out/relayer components internally - but we want an API that people can start using now and maintain. Internals can change organically. We still want to have a data definition file that is generic; don't want to always require usage of oVirt engine to have a self-contained VM description. An OVF for KVM. Why not use the existing ovirt OVF as a standard? SuSE studio would like to use such a format; they have comparable formats for other hypervisor types. If kvm VM creation builds an OVF file (vs. just exporting it) it might help increase the adoption of this as a standard. Let's not forget there are advantages that libvirt brings to oVirt; hooks rely heavily on libvirt modules for example. how does ovirt-engine relate to nova? similar; but different goals. what about subservices of nova? e.g. placement service? per-user kvmd, don't run as root, a lot of benefits - security, selinux, quotas, SLAs-per-user From cctrieloff at redhat.com Mon Nov 7 16:51:55 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Mon, 07 Nov 2011 11:51:55 -0500 Subject: First community release Message-ID: <4EB80CAB.3030908@redhat.com> At the workshop in the release / roadmap working session we agreed that around November 16th we see how all the distro's are making out in creating a release for oVirt and set the date for the first community release. Please have your comments / feedback for that time period, at which point I will start the release discussions. regards Carl. From cctrieloff at redhat.com Wed Nov 9 15:43:18 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Wed, 09 Nov 2011 10:43:18 -0500 Subject: Workshop notes Message-ID: <4EBA9F96.3080306@redhat.com> http://ovirt.org/wiki/Workshop_November_2011 From cctrieloff at redhat.com Wed Nov 9 19:16:45 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Wed, 09 Nov 2011 14:16:45 -0500 Subject: related to community building Message-ID: <4EBAD19D.4040309@redhat.com> http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/index.html From pmyers at redhat.com Sat Nov 12 00:44:41 2011 From: pmyers at redhat.com (Perry Myers) Date: Fri, 11 Nov 2011 19:44:41 -0500 Subject: Some guidelines for bugzilla usage Message-ID: <4EBDC179.5010809@redhat.com> oVirt is utilizing the bugzilla instance at bugzilla.redhat.com for all oVirt sub-projects. The bug list can be seen here: https://bugzilla.redhat.com/buglist.cgi?product=oVirt Using Red Hat's bugzilla server just means one less infrastructure component to maintain, which is a good thing. But I wanted to set up some guidelines for how we use bugzilla to make sure we keep the project as open as possible. In Red Hat bugzilla, Red Hat employees can make comments private and also set flags that restrict visibility of bugs to certain internal groups. Because we're using this bugzilla instance for an open source project, we should _never_ be using the capability to set private comments or set internal private groups. No one is doing this, to my knowledge, but I figured it wouldn't hurt to make sure the ground-rules are laid down clearly for everyone. Cheers, Perry From bazulay at redhat.com Tue Nov 15 17:24:40 2011 From: bazulay at redhat.com (Barak Azulay) Date: Tue, 15 Nov 2011 19:24:40 +0200 Subject: converging around a single guest agent Message-ID: <201111151924.41357.bazulay@redhat.com> Hi, One of the breakout sessions during the ovirt workshop [1] was about the guest tools, and focused mainly on the ovirt-guest-agent [2]. One of the issues discussed there, was the various existing guest agents out there, and the need to converge the efforts to a single agent that will serve all. while 4 agents were mentioned (Matahari, vdagent, qemu-ga & ovirt-guest-agent) during that discussion, we narrowed it down to 2 candidates: qemu-ga (aka virt-agent): ------------------------- - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest I/O) - Communicates directly with qemu (not implemented yet) - Supports ? - So far linux only - written in C Ovirt-guest-agent: ------------------ - Has been around for a long time (~5 years) - considered stable - Started as rhevm specific but evolved a lot since then - Currently the only fully functional guest agent available for ovirt - Written in python - Some VDI related sub components are written in C & C++ - Supports a well defined list of message types / protocol [3] - Supports the folowing guest OSs Linux: RHEL5, RHEL6 F15, F16(soon) Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) The need to converge is obvious, and now that ovirt-guest-agent is opensourced under the ovirt stack, and since it already produces value for enterprise installations, and is cross platform, I offer to join hands around ovirt- guest-agent and formalize a single code base that will serve us all. git @ git://gerrit.ovirt.org/ovirt-guest-agent Thoughts ? Thanks Barak Azulay [1] http://www.ovirt.org/news-and-events/workshop [2] http://www.ovirt.org/wiki/File:Ovirt-guest-agent.odp [3] http://www.ovirt.org/wiki/Ovirt_guest_agent From sghosh at redhat.com Tue Nov 15 18:08:32 2011 From: sghosh at redhat.com (Subhendu Ghosh) Date: Tue, 15 Nov 2011 13:08:32 -0500 Subject: converging around a single guest agent In-Reply-To: <4EC2A8F6.2020406@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC2A8F6.2020406@redhat.com> Message-ID: <4EC2AAA0.5010205@redhat.com> On 11/15/2011 01:01 PM, Perry Myers wrote: > On 11/15/2011 12:24 PM, Barak Azulay wrote: >> Hi, >> >> One of the breakout sessions during the ovirt workshop [1] was about the guest >> tools, and focused mainly on the ovirt-guest-agent [2]. >> >> One of the issues discussed there, was the various existing guest agents out >> there, and the need to converge the efforts to a single agent that will serve >> all. >> >> while 4 agents were mentioned (Matahari, vdagent, qemu-ga& ovirt-guest-agent) >> during that discussion, we narrowed it down to 2 candidates: >> >> qemu-ga (aka virt-agent): >> ------------------------- >> - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest >> I/O) >> - Communicates directly with qemu (not implemented yet) >> - Supports ? >> - So far linux only >> - written in C >> >> Ovirt-guest-agent: >> ------------------ >> - Has been around for a long time (~5 years) - considered stable >> - Started as rhevm specific but evolved a lot since then >> - Currently the only fully functional guest agent available for ovirt >> - Written in python >> - Some VDI related sub components are written in C& C++ >> - Supports a well defined list of message types / protocol [3] >> - Supports the folowing guest OSs >> Linux: RHEL5, RHEL6 F15, F16(soon) >> Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) >> >> >> The need to converge is obvious, and now that ovirt-guest-agent is opensourced >> under the ovirt stack, and since it already produces value for enterprise >> installations, and is cross platform, I offer to join hands around ovirt- >> guest-agent and formalize a single code base that will serve us all. >> >> git @ git://gerrit.ovirt.org/ovirt-guest-agent >> >> Thoughts ? > > +1 > > The only downside that I concretely heard from folks re: > ovirt-guest-agent was that it is written in Python. Two thoughts there: > > 1. On Windows it is compiled to an executable, so no separate python > stack needed > > 2. ovirt-guest-agent is not very large and does not bring in a lot > (any?) additional python class dependencies above/beyond the core > language and interpreter. Given this, the chances of dealing with > python stack issues are probably minimal and also the overhead of > including _just_ the base python interpreter in a given guest OS is > very lightweight. Core python RPM in F16 is about 80k. > > Perry If you needed WMI enablement on Windows - could you support that with this arch? From anthony at codemonkey.ws Tue Nov 15 19:08:18 2011 From: anthony at codemonkey.ws (Anthony Liguori) Date: Tue, 15 Nov 2011 13:08:18 -0600 Subject: converging around a single guest agent In-Reply-To: <201111151924.41357.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> Message-ID: <4EC2B8A2.8040709@codemonkey.ws> On 11/15/2011 11:24 AM, Barak Azulay wrote: > Hi, > > One of the breakout sessions during the ovirt workshop [1] was about the guest > tools, and focused mainly on the ovirt-guest-agent [2]. > > One of the issues discussed there, was the various existing guest agents out > there, and the need to converge the efforts to a single agent that will serve > all. > > while 4 agents were mentioned (Matahari, vdagent, qemu-ga& ovirt-guest-agent) > during that discussion, we narrowed it down to 2 candidates: > > qemu-ga (aka virt-agent): > ------------------------- > - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest > I/O) > - Communicates directly with qemu (not implemented yet) > - Supports ? > - So far linux only But very easy to port. It also should work on just about any Unix since its only dependency is glib. Also: - exists in the QEMU repository > - written in C > > Ovirt-guest-agent: > ------------------ > - Has been around for a long time (~5 years) - considered stable > - Started as rhevm specific but evolved a lot since then > - Currently the only fully functional guest agent available for ovirt > - Written in python > - Some VDI related sub components are written in C& C++ > - Supports a well defined list of message types / protocol [3] > - Supports the folowing guest OSs > Linux: RHEL5, RHEL6 F15, F16(soon) > Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) The guest agent we use in QEMU exists to implement QEMU specific functionality. I think one challenge that comes up here is that the ovirt guest agent has a very different scope than the QEMU agent. The ovirt guest agent has a very ovirt-engine centric scope. > The need to converge is obvious, and now that ovirt-guest-agent is opensourced > under the ovirt stack, and since it already produces value for enterprise > installations, and is cross platform, I offer to join hands around ovirt- > guest-agent and formalize a single code base that will serve us all. You are basically saying, stop what you guys are doing and work on our code because it's better. That's not really convergence. If you want to talk about convergence, the discussion should start around collecting requirements. We can then figure out if the two sets of requirements are strictly overlapping or if there are any requirements that are fundamentally in opposition. Regards, Anthony Liguori From anthony at codemonkey.ws Tue Nov 15 19:09:12 2011 From: anthony at codemonkey.ws (Anthony Liguori) Date: Tue, 15 Nov 2011 13:09:12 -0600 Subject: converging around a single guest agent In-Reply-To: <201111151924.41357.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> Message-ID: <4EC2B8D8.9030101@codemonkey.ws> On 11/15/2011 11:24 AM, Barak Azulay wrote: > Hi, > > One of the breakout sessions during the ovirt workshop [1] was about the guest > tools, and focused mainly on the ovirt-guest-agent [2]. > > One of the issues discussed there, was the various existing guest agents out > there, and the need to converge the efforts to a single agent that will serve > all. > > while 4 agents were mentioned (Matahari, vdagent, qemu-ga& ovirt-guest-agent) > during that discussion, we narrowed it down to 2 candidates: > > qemu-ga (aka virt-agent): > ------------------------- > - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest > I/O) > - Communicates directly with qemu (not implemented yet) > - Supports ? > - So far linux only But very easy to port. It also should work on just about any Unix since its only dependency is glib. Also: - exists in the QEMU repository > - written in C > > Ovirt-guest-agent: > ------------------ > - Has been around for a long time (~5 years) - considered stable > - Started as rhevm specific but evolved a lot since then > - Currently the only fully functional guest agent available for ovirt > - Written in python > - Some VDI related sub components are written in C& C++ > - Supports a well defined list of message types / protocol [3] > - Supports the folowing guest OSs > Linux: RHEL5, RHEL6 F15, F16(soon) > Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) The guest agent we use in QEMU exists to implement QEMU specific functionality. I think one challenge that comes up here is that the ovirt guest agent has a very different scope than the QEMU agent. The ovirt guest agent has a very ovirt-engine centric scope. > The need to converge is obvious, and now that ovirt-guest-agent is opensourced > under the ovirt stack, and since it already produces value for enterprise > installations, and is cross platform, I offer to join hands around ovirt- > guest-agent and formalize a single code base that will serve us all. You are basically saying, stop what you guys are doing and work on our code because it's better. That's not really convergence. If you want to talk about convergence, the discussion should start around collecting requirements. We can then figure out if the two sets of requirements are strictly overlapping or if there are any requirements that are fundamentally in opposition. Regards, Anthony Liguori From cctrieloff at redhat.com Tue Nov 15 19:13:47 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Tue, 15 Nov 2011 14:13:47 -0500 Subject: converging around a single guest agent In-Reply-To: <201111151924.41357.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> Message-ID: <4EC2B9EB.601@redhat.com> Matahari is a can be bound to any agent, i.e. it adds transport, security etc etc so is not an exclusive or discussion. It can be seen as an encapsulation. Carl. On 11/15/2011 12:24 PM, Barak Azulay wrote: > Hi, > > One of the breakout sessions during the ovirt workshop [1] was about the guest > tools, and focused mainly on the ovirt-guest-agent [2]. > > One of the issues discussed there, was the various existing guest agents out > there, and the need to converge the efforts to a single agent that will serve > all. > > while 4 agents were mentioned (Matahari, vdagent, qemu-ga & ovirt-guest-agent) > during that discussion, we narrowed it down to 2 candidates: > > qemu-ga (aka virt-agent): > ------------------------- > - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest > I/O) > - Communicates directly with qemu (not implemented yet) > - Supports ? > - So far linux only > - written in C > > Ovirt-guest-agent: > ------------------ > - Has been around for a long time (~5 years) - considered stable > - Started as rhevm specific but evolved a lot since then > - Currently the only fully functional guest agent available for ovirt > - Written in python > - Some VDI related sub components are written in C & C++ > - Supports a well defined list of message types / protocol [3] > - Supports the folowing guest OSs > Linux: RHEL5, RHEL6 F15, F16(soon) > Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) > > > The need to converge is obvious, and now that ovirt-guest-agent is opensourced > under the ovirt stack, and since it already produces value for enterprise > installations, and is cross platform, I offer to join hands around ovirt- > guest-agent and formalize a single code base that will serve us all. > > git @ git://gerrit.ovirt.org/ovirt-guest-agent > > Thoughts ? > > Thanks > Barak Azulay > > [1] http://www.ovirt.org/news-and-events/workshop > [2] http://www.ovirt.org/wiki/File:Ovirt-guest-agent.odp > [3] http://www.ovirt.org/wiki/Ovirt_guest_agent > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From cctrieloff at redhat.com Tue Nov 15 19:17:03 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Tue, 15 Nov 2011 14:17:03 -0500 Subject: converging around a single guest agent In-Reply-To: <4EC2B8D8.9030101@codemonkey.ws> References: <201111151924.41357.bazulay@redhat.com> <4EC2B8D8.9030101@codemonkey.ws> Message-ID: <4EC2BAAF.90108@redhat.com> On 11/15/2011 02:09 PM, Anthony Liguori wrote: > On 11/15/2011 11:24 AM, Barak Azulay wrote: >> Hi, >> >> One of the breakout sessions during the ovirt workshop [1] was about >> the guest >> tools, and focused mainly on the ovirt-guest-agent [2]. >> >> One of the issues discussed there, was the various existing guest >> agents out >> there, and the need to converge the efforts to a single agent that >> will serve >> all. >> >> while 4 agents were mentioned (Matahari, vdagent, qemu-ga& >> ovirt-guest-agent) >> during that discussion, we narrowed it down to 2 candidates: >> >> qemu-ga (aka virt-agent): >> ------------------------- >> - Qemu specific - it was aimed for specific qemu needs (mainly >> quiesce guest >> I/O) >> - Communicates directly with qemu (not implemented yet) >> - Supports ? >> - So far linux only > > But very easy to port. It also should work on just about any Unix > since its only dependency is glib. Also: > > - exists in the QEMU repository > >> - written in C >> >> Ovirt-guest-agent: >> ------------------ >> - Has been around for a long time (~5 years) - considered stable >> - Started as rhevm specific but evolved a lot since then >> - Currently the only fully functional guest agent available for ovirt >> - Written in python >> - Some VDI related sub components are written in C& C++ >> - Supports a well defined list of message types / protocol [3] >> - Supports the folowing guest OSs >> Linux: RHEL5, RHEL6 F15, F16(soon) >> Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) > > The guest agent we use in QEMU exists to implement QEMU specific > functionality. I think one challenge that comes up here is that the > ovirt guest agent has a very different scope than the QEMU agent. The > ovirt guest agent has a very ovirt-engine centric scope. > >> The need to converge is obvious, and now that ovirt-guest-agent is >> opensourced >> under the ovirt stack, and since it already produces value for >> enterprise >> installations, and is cross platform, I offer to join hands around >> ovirt- >> guest-agent and formalize a single code base that will serve us all. > > You are basically saying, stop what you guys are doing and work on our > code because it's better. That's not really convergence. > > If you want to talk about convergence, the discussion should start > around collecting requirements. We can then figure out if the two > sets of requirements are strictly overlapping or if there are any > requirements that are fundamentally in opposition. > What are the goals of each agent? if the goal are the same we should try converge. If the goals are not the same then then they probably should not be converged. Let's start by getting the goals of each agent articulated. Carl. From pmyers at redhat.com Tue Nov 15 19:45:42 2011 From: pmyers at redhat.com (Perry Myers) Date: Tue, 15 Nov 2011 14:45:42 -0500 Subject: converging around a single guest agent In-Reply-To: <4EC2AAA0.5010205@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC2A8F6.2020406@redhat.com> <4EC2AAA0.5010205@redhat.com> Message-ID: <4EC2C166.8030403@redhat.com> On 11/15/2011 01:08 PM, Subhendu Ghosh wrote: > On 11/15/2011 01:01 PM, Perry Myers wrote: >> On 11/15/2011 12:24 PM, Barak Azulay wrote: >>> Hi, >>> >>> One of the breakout sessions during the ovirt workshop [1] was about >>> the guest >>> tools, and focused mainly on the ovirt-guest-agent [2]. >>> >>> One of the issues discussed there, was the various existing guest >>> agents out >>> there, and the need to converge the efforts to a single agent that >>> will serve >>> all. >>> >>> while 4 agents were mentioned (Matahari, vdagent, qemu-ga& >>> ovirt-guest-agent) >>> during that discussion, we narrowed it down to 2 candidates: >>> >>> qemu-ga (aka virt-agent): >>> ------------------------- >>> - Qemu specific - it was aimed for specific qemu needs (mainly >>> quiesce guest >>> I/O) >>> - Communicates directly with qemu (not implemented yet) >>> - Supports ? >>> - So far linux only >>> - written in C >>> >>> Ovirt-guest-agent: >>> ------------------ >>> - Has been around for a long time (~5 years) - considered stable >>> - Started as rhevm specific but evolved a lot since then >>> - Currently the only fully functional guest agent available for ovirt >>> - Written in python >>> - Some VDI related sub components are written in C& C++ >>> - Supports a well defined list of message types / protocol [3] >>> - Supports the folowing guest OSs >>> Linux: RHEL5, RHEL6 F15, F16(soon) >>> Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) >>> >>> >>> The need to converge is obvious, and now that ovirt-guest-agent is >>> opensourced >>> under the ovirt stack, and since it already produces value for >>> enterprise >>> installations, and is cross platform, I offer to join hands around >>> ovirt- >>> guest-agent and formalize a single code base that will serve us all. >>> >>> git @ git://gerrit.ovirt.org/ovirt-guest-agent >>> >>> Thoughts ? >> >> +1 >> >> The only downside that I concretely heard from folks re: >> ovirt-guest-agent was that it is written in Python. Two thoughts there: >> >> 1. On Windows it is compiled to an executable, so no separate python >> stack needed >> >> 2. ovirt-guest-agent is not very large and does not bring in a lot >> (any?) additional python class dependencies above/beyond the core >> language and interpreter. Given this, the chances of dealing with >> python stack issues are probably minimal and also the overhead of >> including _just_ the base python interpreter in a given guest OS is >> very lightweight. Core python RPM in F16 is about 80k. >> >> Perry > > If you needed WMI enablement on Windows - could you support that with > this arch? I'm not a WMI expert, but google search first result on 'python WMI' turned up: http://timgolden.me.uk/python/wmi/index.html From pmyers at redhat.com Tue Nov 15 19:55:59 2011 From: pmyers at redhat.com (Perry Myers) Date: Tue, 15 Nov 2011 14:55:59 -0500 Subject: converging around a single guest agent In-Reply-To: <4EC2B9EB.601@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC2B9EB.601@redhat.com> Message-ID: <4EC2C3CF.7060500@redhat.com> On 11/15/2011 02:13 PM, Carl Trieloff wrote: > > Matahari is a can be bound to any agent, i.e. it adds transport, > security etc etc so is not an exclusive or discussion. It can be seen as > an encapsulation. Yes, but when we discussed matahari in the context of a ubiquitous lightweight virt agent (for both libvirt/qemu/oVirt) it was rejected due to complexity and dependencies. Our thinking was that matahari should not be required for core functionality for qemu/libvirt/oVirt, but can instead be used in conjunction with the lightweight guest agent/transport to provide higher level mgmt functionality Here's a quick flow of the Matahari path from guest to host, and why it's too complex to be suitable for these use cases: QMF Guest Agent(s) Guest qpid Broker Guest VIOS Proxy Host VIOS Proxy Host qpid Broker QMF Host Console Compare that to: ovirt-guest-agent vdsm qemu-guest agent qemu libvirtd Perry > Carl. > > > On 11/15/2011 12:24 PM, Barak Azulay wrote: >> Hi, >> >> One of the breakout sessions during the ovirt workshop [1] was about the guest >> tools, and focused mainly on the ovirt-guest-agent [2]. >> >> One of the issues discussed there, was the various existing guest agents out >> there, and the need to converge the efforts to a single agent that will serve >> all. >> >> while 4 agents were mentioned (Matahari, vdagent, qemu-ga & ovirt-guest-agent) >> during that discussion, we narrowed it down to 2 candidates: >> >> qemu-ga (aka virt-agent): >> ------------------------- >> - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest >> I/O) >> - Communicates directly with qemu (not implemented yet) >> - Supports ? >> - So far linux only >> - written in C >> >> Ovirt-guest-agent: >> ------------------ >> - Has been around for a long time (~5 years) - considered stable >> - Started as rhevm specific but evolved a lot since then >> - Currently the only fully functional guest agent available for ovirt >> - Written in python >> - Some VDI related sub components are written in C & C++ >> - Supports a well defined list of message types / protocol [3] >> - Supports the folowing guest OSs >> Linux: RHEL5, RHEL6 F15, F16(soon) >> Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) >> >> >> The need to converge is obvious, and now that ovirt-guest-agent is opensourced >> under the ovirt stack, and since it already produces value for enterprise >> installations, and is cross platform, I offer to join hands around ovirt- >> guest-agent and formalize a single code base that will serve us all. >> >> git @ git://gerrit.ovirt.org/ovirt-guest-agent >> >> Thoughts ? >> >> Thanks >> Barak Azulay >> >> [1] http://www.ovirt.org/news-and-events/workshop >> [2] http://www.ovirt.org/wiki/File:Ovirt-guest-agent.odp >> [3] http://www.ovirt.org/wiki/Ovirt_guest_agent >> _______________________________________________ >> Arch mailing list >> Arch at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/arch > > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From pmyers at redhat.com Tue Nov 15 18:01:26 2011 From: pmyers at redhat.com (Perry Myers) Date: Tue, 15 Nov 2011 13:01:26 -0500 Subject: converging around a single guest agent In-Reply-To: <201111151924.41357.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> Message-ID: <4EC2A8F6.2020406@redhat.com> On 11/15/2011 12:24 PM, Barak Azulay wrote: > Hi, > > One of the breakout sessions during the ovirt workshop [1] was about the guest > tools, and focused mainly on the ovirt-guest-agent [2]. > > One of the issues discussed there, was the various existing guest agents out > there, and the need to converge the efforts to a single agent that will serve > all. > > while 4 agents were mentioned (Matahari, vdagent, qemu-ga & ovirt-guest-agent) > during that discussion, we narrowed it down to 2 candidates: > > qemu-ga (aka virt-agent): > ------------------------- > - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest > I/O) > - Communicates directly with qemu (not implemented yet) > - Supports ? > - So far linux only > - written in C > > Ovirt-guest-agent: > ------------------ > - Has been around for a long time (~5 years) - considered stable > - Started as rhevm specific but evolved a lot since then > - Currently the only fully functional guest agent available for ovirt > - Written in python > - Some VDI related sub components are written in C & C++ > - Supports a well defined list of message types / protocol [3] > - Supports the folowing guest OSs > Linux: RHEL5, RHEL6 F15, F16(soon) > Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) > > > The need to converge is obvious, and now that ovirt-guest-agent is opensourced > under the ovirt stack, and since it already produces value for enterprise > installations, and is cross platform, I offer to join hands around ovirt- > guest-agent and formalize a single code base that will serve us all. > > git @ git://gerrit.ovirt.org/ovirt-guest-agent > > Thoughts ? +1 The only downside that I concretely heard from folks re: ovirt-guest-agent was that it is written in Python. Two thoughts there: 1. On Windows it is compiled to an executable, so no separate python stack needed 2. ovirt-guest-agent is not very large and does not bring in a lot (any?) additional python class dependencies above/beyond the core language and interpreter. Given this, the chances of dealing with python stack issues are probably minimal and also the overhead of including _just_ the base python interpreter in a given guest OS is very lightweight. Core python RPM in F16 is about 80k. Perry From abaron at redhat.com Tue Nov 15 22:39:09 2011 From: abaron at redhat.com (Ayal Baron) Date: Tue, 15 Nov 2011 17:39:09 -0500 (EST) Subject: converging around a single guest agent In-Reply-To: <4EC2B8A2.8040709@codemonkey.ws> Message-ID: ----- Original Message ----- > On 11/15/2011 11:24 AM, Barak Azulay wrote: > > Hi, > > > > One of the breakout sessions during the ovirt workshop [1] was > > about the guest > > tools, and focused mainly on the ovirt-guest-agent [2]. > > > > One of the issues discussed there, was the various existing guest > > agents out > > there, and the need to converge the efforts to a single agent that > > will serve > > all. > > > > while 4 agents were mentioned (Matahari, vdagent, qemu-ga& > > ovirt-guest-agent) > > during that discussion, we narrowed it down to 2 candidates: > > > > qemu-ga (aka virt-agent): > > ------------------------- > > - Qemu specific - it was aimed for specific qemu needs (mainly > > quiesce guest > > I/O) > > - Communicates directly with qemu (not implemented yet) > > - Supports ? > > - So far linux only > > But very easy to port. It also should work on just about any Unix > since its > only dependency is glib. Also: > > - exists in the QEMU repository > > > - written in C > > > > Ovirt-guest-agent: > > ------------------ > > - Has been around for a long time (~5 years) - considered stable > > - Started as rhevm specific but evolved a lot since then > > - Currently the only fully functional guest agent available for > > ovirt > > - Written in python > > - Some VDI related sub components are written in C& C++ > > - Supports a well defined list of message types / protocol [3] > > - Supports the folowing guest OSs > > Linux: RHEL5, RHEL6 F15, F16(soon) > > Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) > > The guest agent we use in QEMU exists to implement QEMU specific > functionality. > I think one challenge that comes up here is that the ovirt guest > agent has a > very different scope than the QEMU agent. The ovirt guest agent has > a very > ovirt-engine centric scope. > > > The need to converge is obvious, and now that ovirt-guest-agent is > > opensourced > > under the ovirt stack, and since it already produces value for > > enterprise > > installations, and is cross platform, I offer to join hands around > > ovirt- > > guest-agent and formalize a single code base that will serve us > > all. > > You are basically saying, stop what you guys are doing and work on > our code > because it's better. That's not really convergence. > > If you want to talk about convergence, the discussion should start > around > collecting requirements. We can then figure out if the two sets of > requirements > are strictly overlapping or if there are any requirements that are > fundamentally > in opposition. Agreed. So vdsm guest agent goal is to ease administration of VMs. This is not saying much as it is quite broad so I will list what is provided today and some things we need to add: Assistance in VM life-cycle: "desktopShutdown" - Shuts the VM down gracefully from within the guest. "quiesce" - does not exist today. This is definitely a requirement for us. SSO support for spice sessions (automatically login into guest OS using provided credentials): "desktopLock" - lock current session, used when spice session gets disconnected / before giving a new user access to spice session "desktopLogin" "desktopLogoff" In addition, guest reports relevant info (currently active user, session state) Monitoring and inventory: currently agent sends info periodically, which includes a lot of info which should probably be broken down and served upon request. Info includes - - memory usage - NICs info (name, hw, inet, inet6) - appslist (list of installed apps / rpms) - OS type - guest hostname - internal file systems info (path, fs type, total space, used space) Personally I think the above should become more generic and support user defined counters (using WMI or collectd in the guest to collect the info and passing it via the guest agent), but that might be a different discussion. >From qemu wiki, the following info about qemu guest agent: It's purpose: "Implement support for QMP commands and events that terminate and originate respectively within the guest using an agent built as part of QEMU. " - ties it directly to qemu, but not to specific functionality. ovirt guest agent definitely would need to support this In general, I would say ovirt-guest-agent is scoped to do everything the qemu-guest-agent is and then some, so there is definitely a lot of overlap. > > Regards, > > Anthony Liguori > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch > From mdroth at linux.vnet.ibm.com Tue Nov 15 23:01:00 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Tue, 15 Nov 2011 17:01:00 -0600 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <201111151924.41357.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> Message-ID: <4EC2EF2C.7010100@linux.vnet.ibm.com> On 11/15/2011 11:24 AM, Barak Azulay wrote: > Hi, > > One of the breakout sessions during the ovirt workshop [1] was about the guest > tools, and focused mainly on the ovirt-guest-agent [2]. > > One of the issues discussed there, was the various existing guest agents out > there, and the need to converge the efforts to a single agent that will serve > all. > > while 4 agents were mentioned (Matahari, vdagent, qemu-ga& ovirt-guest-agent) > during that discussion, we narrowed it down to 2 candidates: > > qemu-ga (aka virt-agent): > ------------------------- > - Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest > I/O) > - Communicates directly with qemu (not implemented yet) Won't be implemented until we finish the QMP conversion to QAPI since that's a prereq to implementing a QMP server with proper async support (needed for communication with a potentially non-responsive guest agent). But once it's in guest extensions are completely transparent to QMP/libvirt, which is highly desirable from a management perspective since you're coding against a single API/transport. > - Supports ? Documented in qemu.git/qapi-schema.json: mdroth at illuin:~/w/qemu.git$ grep command qapi-schema-guest.json # Such clients should also preceed this command { 'command': 'guest-sync' { 'command': 'guest-ping' } { 'command': 'guest-info', { 'command': 'guest-shutdown', 'data': { '*mode': 'str' } } { 'command': 'guest-file-open', { 'command': 'guest-file-close', { 'command': 'guest-file-read', { 'command': 'guest-file-write', { 'command': 'guest-file-seek', { 'command': 'guest-file-flush', { 'command': 'guest-fsfreeze-status', { 'command': 'guest-fsfreeze-freeze', { 'command': 'guest-fsfreeze-thaw', There's a wiki page with additional details: > - So far linux only Windows port is *almost* there. I have patches to build/run it on windows but there are some remaining bugs, and guest-fsfreeze* and guest-shutdown* are gonna need windows-specific command implementations. > - written in C > > Ovirt-guest-agent: > ------------------ > - Has been around for a long time (~5 years) - considered stable > - Started as rhevm specific but evolved a lot since then > - Currently the only fully functional guest agent available for ovirt > - Written in python > - Some VDI related sub components are written in C& C++ > - Supports a well defined list of message types / protocol [3] > - Supports the folowing guest OSs > Linux: RHEL5, RHEL6 F15, F16(soon) > Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) > In terms of completeness/support for ovirt (and node/cluster-level management frameworks in general), the ovirt-guest-agent certainly seems like a win. The main problem with convergence within the qemu space is that we can't have guest extensions to qemu-driven guest events like, say, hotplug or shutdown be managed by an external agent: the functionality, specificity, and assurances required are not things that are necessarily suited for ovirt-guest-agent; the scope is very different. And the same holds true in the other direction: I don't, personally at least, foresee qemu-ga ever doing things like providing a list of logged in users on a system or handling SSO, though if there are those who'd like to take qemu-ga in that direction I think that's fine. But practically-speaking, it's unavoidable that qemu-specific management tooling will need to communicate with qemu (via QMP/libqmp/HMP/etc, or by proxy via libvirt). It's through those same channels that the qemu-ga interfaces will ultimately be exposed, so the problem of qemu-ga vs. ovirt-guest-agent isn't really any different than the problem of QMP's system_powerdown/info_balloon/etc vs. ovirt-guest-agent's Shutdown/Available_Ram/etc: it's a policy decision rather than argument for choosing one project over another. > > The need to converge is obvious, and now that ovirt-guest-agent is opensourced > under the ovirt stack, and since it already produces value for enterprise > installations, and is cross platform, I offer to join hands around ovirt- > guest-agent and formalize a single code base that will serve us all. > > git @ git://gerrit.ovirt.org/ovirt-guest-agent > > Thoughts ? > > Thanks > Barak Azulay > > [1] http://www.ovirt.org/news-and-events/workshop > [2] http://www.ovirt.org/wiki/File:Ovirt-guest-agent.odp > [3] http://www.ovirt.org/wiki/Ovirt_guest_agent > From agraf at suse.de Wed Nov 16 00:42:30 2011 From: agraf at suse.de (Alexander Graf) Date: Wed, 16 Nov 2011 01:42:30 +0100 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC2EF2C.7010100@linux.vnet.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <4EC2EF2C.7010100@linux.vnet.ibm.com> Message-ID: <2954A154-7DB5-4F82-82C0-88289606B76A@suse.de> On 16.11.2011, at 00:01, Michael Roth wrote: > But practically-speaking, it's unavoidable that qemu-specific management tooling will need to communicate with qemu (via QMP/libqmp/HMP/etc, or by proxy via libvirt). It's through those same channels that the qemu-ga interfaces will ultimately be exposed, so the problem of qemu-ga vs. ovirt-guest-agent isn't really any different than the problem of QMP's system_powerdown/info_balloon/etc vs. ovirt-guest-agent's Shutdown/Available_Ram/etc: it's a policy decision rather than argument for choosing one project over another. I don't see why we shouldn't be able to just proxy whatever communication happens between the guest agent and the management tool through qemu. At that point qemu could talk to the guest agent just as well as the management tool and everyone's happy. Alex From bazulay at redhat.com Wed Nov 16 06:48:13 2011 From: bazulay at redhat.com (Barak Azulay) Date: Wed, 16 Nov 2011 08:48:13 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC2C166.8030403@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC2AAA0.5010205@redhat.com> <4EC2C166.8030403@redhat.com> Message-ID: <201111160848.13569.bazulay@redhat.com> On Tuesday 15 November 2011 21:45:42 Perry Myers wrote: > On 11/15/2011 01:08 PM, Subhendu Ghosh wrote: > > On 11/15/2011 01:01 PM, Perry Myers wrote: > >> On 11/15/2011 12:24 PM, Barak Azulay wrote: > >>> Hi, > >>> > >>> One of the breakout sessions during the ovirt workshop [1] was about > >>> the guest > >>> tools, and focused mainly on the ovirt-guest-agent [2]. > >>> > >>> One of the issues discussed there, was the various existing guest > >>> agents out > >>> there, and the need to converge the efforts to a single agent that > >>> will serve > >>> all. > >>> > >>> while 4 agents were mentioned (Matahari, vdagent, qemu-ga& > >>> ovirt-guest-agent) > >>> during that discussion, we narrowed it down to 2 candidates: > >>> > >>> qemu-ga (aka virt-agent): > >>> ------------------------- > >>> - Qemu specific - it was aimed for specific qemu needs (mainly > >>> quiesce guest > >>> I/O) > >>> - Communicates directly with qemu (not implemented yet) > >>> - Supports ? > >>> - So far linux only > >>> - written in C > >>> > >>> Ovirt-guest-agent: > >>> ------------------ > >>> - Has been around for a long time (~5 years) - considered stable > >>> - Started as rhevm specific but evolved a lot since then > >>> - Currently the only fully functional guest agent available for ovirt > >>> - Written in python > >>> - Some VDI related sub components are written in C& C++ > >>> - Supports a well defined list of message types / protocol [3] > >>> - Supports the folowing guest OSs > >>> > >>> Linux: RHEL5, RHEL6 F15, F16(soon) > >>> Windows: xp, 2k3 (32/64), w7 (32/64), 2k8 (32/64/R2) > >>> > >>> The need to converge is obvious, and now that ovirt-guest-agent is > >>> opensourced > >>> under the ovirt stack, and since it already produces value for > >>> enterprise > >>> installations, and is cross platform, I offer to join hands around > >>> ovirt- > >>> guest-agent and formalize a single code base that will serve us all. > >>> > >>> git @ git://gerrit.ovirt.org/ovirt-guest-agent > >>> > >>> Thoughts ? > >> > >> +1 > >> > >> The only downside that I concretely heard from folks re: > >> ovirt-guest-agent was that it is written in Python. Two thoughts there: > >> > >> 1. On Windows it is compiled to an executable, so no separate python > >> > >> stack needed > >> > >> 2. ovirt-guest-agent is not very large and does not bring in a lot > >> > >> (any?) additional python class dependencies above/beyond the core > >> language and interpreter. Given this, the chances of dealing with > >> python stack issues are probably minimal and also the overhead of > >> including _just_ the base python interpreter in a given guest OS is > >> very lightweight. Core python RPM in F16 is about 80k. The ovirt-guest-agent also depends on pywin32 package (http://sourceforge.net/projects/pywin32/files/) for windows platforms > >> > >> Perry > > > > If you needed WMI enablement on Windows - could you support that with > > this arch? > > I'm not a WMI expert, but google search first result on 'python WMI' > turned up: > > http://timgolden.me.uk/python/wmi/index.html As the ovirt-guest-agent the above package uses also pywin32. From bazulay at redhat.com Wed Nov 16 07:05:22 2011 From: bazulay at redhat.com (Barak Azulay) Date: Wed, 16 Nov 2011 09:05:22 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <2954A154-7DB5-4F82-82C0-88289606B76A@suse.de> References: <201111151924.41357.bazulay@redhat.com> <4EC2EF2C.7010100@linux.vnet.ibm.com> <2954A154-7DB5-4F82-82C0-88289606B76A@suse.de> Message-ID: <201111160905.23316.bazulay@redhat.com> On Wednesday 16 November 2011 02:42:30 Alexander Graf wrote: > On 16.11.2011, at 00:01, Michael Roth wrote: > > But practically-speaking, it's unavoidable that qemu-specific management > > tooling will need to communicate with qemu (via QMP/libqmp/HMP/etc, or > > by proxy via libvirt). It's through those same channels that the qemu-ga > > interfaces will ultimately be exposed, so the problem of qemu-ga vs. > > ovirt-guest-agent isn't really any different than the problem of QMP's > > system_powerdown/info_balloon/etc vs. ovirt-guest-agent's > > Shutdown/Available_Ram/etc: it's a policy decision rather than argument > > for choosing one project over another. > > I don't see why we shouldn't be able to just proxy whatever communication > happens between the guest agent and the management tool through qemu. At > that point qemu could talk to the guest agent just as well as the > management tool and everyone's happy. I'm not sure proxying all the requests to the guset through qemu is desirable, other than having single point of management, most of the calls will be pass throgh and has no interest to qemu (MITM?). There is a big advantage on direct communication (VDSM <-> agent), that way features can be added to the ovirt stack without the need to add it to the qemu. I envision the agent will have 2 separate ports to listen to, one to communicate to qemu and one for VDSM. Barak > > Alex From abaron at redhat.com Wed Nov 16 08:16:38 2011 From: abaron at redhat.com (Ayal Baron) Date: Wed, 16 Nov 2011 03:16:38 -0500 (EST) Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC36C09.5060805@redhat.com> Message-ID: <488c9fd7-8a5b-4f1c-a3bd-cb750f5769c4@zmail07.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > Hi, > > On 11/15/2011 11:39 PM, Ayal Baron wrote: > > > > > > >> If you want to talk about convergence, the discussion should start > >> around > >> collecting requirements. We can then figure out if the two sets > >> of > >> requirements > >> are strictly overlapping or if there are any requirements that are > >> fundamentally > >> in opposition. > > > > Agreed. > > > > So vdsm guest agent goal is to ease administration of VMs. This is > > not saying much as it is quite broad so I will list what is > > provided today and some things we need to add: > > > > Assistance in VM life-cycle: > > "desktopShutdown" - Shuts the VM down gracefully from within the > > guest. > > "quiesce" - does not exist today. This is definitely a requirement > > for us. > > > > SSO support for spice sessions (automatically login into guest OS > > using provided credentials): > > "desktopLock" - lock current session, used when spice session gets > > disconnected / before giving a new user access to spice session > > "desktopLogin" > > "desktopLogoff" > > In addition, guest reports relevant info (currently active user, > > session state) > > > > Monitoring and inventory: > > currently agent sends info periodically, which includes a lot of > > info which should probably be broken down and served upon request. > > Info includes - > > - memory usage > > - NICs info (name, hw, inet, inet6) > > - appslist (list of installed apps / rpms) > > - OS type > > - guest hostname > > - internal file systems info (path, fs type, total space, used > > space) > > > > > > If we're gathering requirements and trying to come up with one agent > to rule them all, don't forget I don't think we're trying to come up with one agent to rule them all, just avoid duplication of efforts if most of what the 2 agents are doing overlaps. I think we can safely say that seeing as oVirt is KVM centric, ovirt-guest-agent wants to leverage qemu/kvm to the fullest which aligns with what qemu-guest-agent is doing. However, ovirt-guest-agent is required to do a lot more, so we need to see if and how we resolve this. > about VDI and the Spice agent. Currently the spice agent handles the > following: > > 1) Paravirtual mouse (needed to get mouse coordinates right with > multi monitor setups) > 2) Send client monitor configuration, so that the guest os can adjust > its resolution > (and number and place of monitors) to match the client > 3) Copy and paste in a platform neutral manner, if anyone wishes to > add this to another agent > please, please contact us (me) first. This is easy to get wrong > (we went through 2 revisions > of the protocol for this). > 4) Allow the client to request the guest to tone down the bling (for > low spec clients) > > Notes: > 1) All of these are client <-> guest communication, rather then the > host <-> guest communication > which the other agents seem to focus on. > > 2) Getting copy paste right requires a system level guest agent > process as well as a per user > session agent process. Neither qemu-guest-agent nor ovirt-guest-agent is aligned with doing any of the above, so I'm not sure there is any justification in uniting the spice agent with the rest. > > Regards, > > Hans > From agraf at suse.de Wed Nov 16 08:16:57 2011 From: agraf at suse.de (Alexander Graf) Date: Wed, 16 Nov 2011 09:16:57 +0100 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <201111160905.23316.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC2EF2C.7010100@linux.vnet.ibm.com> <2954A154-7DB5-4F82-82C0-88289606B76A@suse.de> <201111160905.23316.bazulay@redhat.com> Message-ID: <724C1350-38D1-4EE2-AC7B-27666B7710A1@suse.de> On 16.11.2011, at 08:05, Barak Azulay wrote: > On Wednesday 16 November 2011 02:42:30 Alexander Graf wrote: >> On 16.11.2011, at 00:01, Michael Roth wrote: >>> But practically-speaking, it's unavoidable that qemu-specific management >>> tooling will need to communicate with qemu (via QMP/libqmp/HMP/etc, or >>> by proxy via libvirt). It's through those same channels that the qemu-ga >>> interfaces will ultimately be exposed, so the problem of qemu-ga vs. >>> ovirt-guest-agent isn't really any different than the problem of QMP's >>> system_powerdown/info_balloon/etc vs. ovirt-guest-agent's >>> Shutdown/Available_Ram/etc: it's a policy decision rather than argument >>> for choosing one project over another. >> >> I don't see why we shouldn't be able to just proxy whatever communication >> happens between the guest agent and the management tool through qemu. At >> that point qemu could talk to the guest agent just as well as the >> management tool and everyone's happy. > > I'm not sure proxying all the requests to the guset through qemu is desirable, > other than having single point of management, most of the calls will be pass > throgh and has no interest to qemu (MITM?). > > There is a big advantage on direct communication (VDSM <-> agent), that way > features can be added to the ovirt stack without the need to add it to the > qemu. If we keep the protocol well-defined, we can get that for free. Just have every command carry its own size and a request id shich the reply also contains and suddenly you get asynchronous proxyable communication. > > I envision the agent will have 2 separate ports to listen to, one to > communicate to qemu and one for VDSM. Ugh, no, I'd much prefer a single 'bus' everyone attaches to. Alex > > Barak > >> >> Alex From alevy at redhat.com Wed Nov 16 12:07:22 2011 From: alevy at redhat.com (Alon Levy) Date: Wed, 16 Nov 2011 14:07:22 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC36C09.5060805@redhat.com> References: <4EC36C09.5060805@redhat.com> Message-ID: <20111116120722.GZ7140@garlic.redhat.com> On Wed, Nov 16, 2011 at 08:53:45AM +0100, Hans de Goede wrote: > Hi, > > On 11/15/2011 11:39 PM, Ayal Baron wrote: > > > > > > >>If you want to talk about convergence, the discussion should start > >>around > >>collecting requirements. We can then figure out if the two sets of > >>requirements > >>are strictly overlapping or if there are any requirements that are > >>fundamentally > >>in opposition. > > > >Agreed. > > > >So vdsm guest agent goal is to ease administration of VMs. This is not saying much as it is quite broad so I will list what is provided today and some things we need to add: > > > >Assistance in VM life-cycle: > >"desktopShutdown" - Shuts the VM down gracefully from within the guest. > >"quiesce" - does not exist today. This is definitely a requirement for us. > > > >SSO support for spice sessions (automatically login into guest OS using provided credentials): > >"desktopLock" - lock current session, used when spice session gets disconnected / before giving a new user access to spice session > >"desktopLogin" > >"desktopLogoff" > >In addition, guest reports relevant info (currently active user, session state) > > > >Monitoring and inventory: > >currently agent sends info periodically, which includes a lot of info which should probably be broken down and served upon request. Info includes - > >- memory usage > >- NICs info (name, hw, inet, inet6) > >- appslist (list of installed apps / rpms) > >- OS type > >- guest hostname > >- internal file systems info (path, fs type, total space, used space) > > > > > > If we're gathering requirements and trying to come up with one agent to rule them all, don't forget > about VDI and the Spice agent. Currently the spice agent handles the following: > > 1) Paravirtual mouse (needed to get mouse coordinates right with multi monitor setups) > 2) Send client monitor configuration, so that the guest os can adjust its resolution > (and number and place of monitors) to match the client > 3) Copy and paste in a platform neutral manner, if anyone wishes to add this to another agent > please, please contact us (me) first. This is easy to get wrong (we went through 2 revisions > of the protocol for this). > 4) Allow the client to request the guest to tone down the bling (for low spec clients) > As long as we are collecting requirements, even if as Ayal said merging spice requirements is not the OP's intent: 5) Window management - Agent can set location of windows, report existing running applications and locations, get notified when a new window is created. For exposing individual applications, this is a future requirement. > Notes: > 1) All of these are client <-> guest communication, rather then the host <-> guest communication > which the other agents seem to focus on. > > 2) Getting copy paste right requires a system level guest agent process as well as a per user > session agent process. > > Regards, > > Hans > From bazulay at redhat.com Wed Nov 16 12:13:20 2011 From: bazulay at redhat.com (Barak Azulay) Date: Wed, 16 Nov 2011 14:13:20 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <724C1350-38D1-4EE2-AC7B-27666B7710A1@suse.de> References: <201111151924.41357.bazulay@redhat.com> <201111160905.23316.bazulay@redhat.com> <724C1350-38D1-4EE2-AC7B-27666B7710A1@suse.de> Message-ID: <201111161413.21026.bazulay@redhat.com> On Wednesday 16 November 2011 10:16:57 Alexander Graf wrote: > On 16.11.2011, at 08:05, Barak Azulay wrote: > > On Wednesday 16 November 2011 02:42:30 Alexander Graf wrote: > >> On 16.11.2011, at 00:01, Michael Roth wrote: > >>> But practically-speaking, it's unavoidable that qemu-specific > >>> management tooling will need to communicate with qemu (via > >>> QMP/libqmp/HMP/etc, or by proxy via libvirt). It's through those same > >>> channels that the qemu-ga interfaces will ultimately be exposed, so > >>> the problem of qemu-ga vs. ovirt-guest-agent isn't really any > >>> different than the problem of QMP's system_powerdown/info_balloon/etc > >>> vs. ovirt-guest-agent's > >>> Shutdown/Available_Ram/etc: it's a policy decision rather than argument > >>> for choosing one project over another. > >> > >> I don't see why we shouldn't be able to just proxy whatever > >> communication happens between the guest agent and the management tool > >> through qemu. At that point qemu could talk to the guest agent just as > >> well as the management tool and everyone's happy. > > > > I'm not sure proxying all the requests to the guset through qemu is > > desirable, other than having single point of management, most of the > > calls will be pass throgh and has no interest to qemu (MITM?). > > > > There is a big advantage on direct communication (VDSM <-> agent), that > > way features can be added to the ovirt stack without the need to add it > > to the qemu. > > If we keep the protocol well-defined, we can get that for free. Just have > every command carry its own size and a request id shich the reply also > contains and suddenly you get asynchronous proxyable communication. > Sure we can keep commands synchronized in various ways the question is do we want that, there are a few down sides for that: 1 - VDSM will have to pass through 2 proxies (libvirt & qemu) in order to deliver a message to the guest, this byiself is not such a big disadvantage but will force us to handle much more corner-cases. 2 - looking at the qemu-ga functionality (read & write ...) do we really want to let a big chunk of data through both qemu & libvirt rather than directtly to the comsumer (VDSM) 3 - When events are fired from the guest agent, the delay of passing it through a double proxy will have it's latency penalty (as we have experianced in the client disconnect spice event) > > I envision the agent will have 2 separate ports to listen to, one to > > communicate to qemu and one for VDSM. > > Ugh, no, I'd much prefer a single 'bus' everyone attaches to. why? I'm thinking on situation we'll need to priorities commands arriving from qemu over "management standard commands" & info gathering, sure there are number of mechanisms to do that but it seems to me that a separation is the best way. e.g. I think we need to priorities a quiesce command from qemu over any other info/command from VDSM. > > Alex > > > Barak > > > >> Alex From hdegoede at redhat.com Wed Nov 16 07:53:45 2011 From: hdegoede at redhat.com (Hans de Goede) Date: Wed, 16 Nov 2011 08:53:45 +0100 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: References: Message-ID: <4EC36C09.5060805@redhat.com> Hi, On 11/15/2011 11:39 PM, Ayal Baron wrote: > >> If you want to talk about convergence, the discussion should start >> around >> collecting requirements. We can then figure out if the two sets of >> requirements >> are strictly overlapping or if there are any requirements that are >> fundamentally >> in opposition. > > Agreed. > > So vdsm guest agent goal is to ease administration of VMs. This is not saying much as it is quite broad so I will list what is provided today and some things we need to add: > > Assistance in VM life-cycle: > "desktopShutdown" - Shuts the VM down gracefully from within the guest. > "quiesce" - does not exist today. This is definitely a requirement for us. > > SSO support for spice sessions (automatically login into guest OS using provided credentials): > "desktopLock" - lock current session, used when spice session gets disconnected / before giving a new user access to spice session > "desktopLogin" > "desktopLogoff" > In addition, guest reports relevant info (currently active user, session state) > > Monitoring and inventory: > currently agent sends info periodically, which includes a lot of info which should probably be broken down and served upon request. Info includes - > - memory usage > - NICs info (name, hw, inet, inet6) > - appslist (list of installed apps / rpms) > - OS type > - guest hostname > - internal file systems info (path, fs type, total space, used space) > If we're gathering requirements and trying to come up with one agent to rule them all, don't forget about VDI and the Spice agent. Currently the spice agent handles the following: 1) Paravirtual mouse (needed to get mouse coordinates right with multi monitor setups) 2) Send client monitor configuration, so that the guest os can adjust its resolution (and number and place of monitors) to match the client 3) Copy and paste in a platform neutral manner, if anyone wishes to add this to another agent please, please contact us (me) first. This is easy to get wrong (we went through 2 revisions of the protocol for this). 4) Allow the client to request the guest to tone down the bling (for low spec clients) Notes: 1) All of these are client <-> guest communication, rather then the host <-> guest communication which the other agents seem to focus on. 2) Getting copy paste right requires a system level guest agent process as well as a per user session agent process. Regards, Hans From berrange at redhat.com Wed Nov 16 10:18:15 2011 From: berrange at redhat.com (Daniel P. Berrange) Date: Wed, 16 Nov 2011 10:18:15 +0000 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC2EF2C.7010100@linux.vnet.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <4EC2EF2C.7010100@linux.vnet.ibm.com> Message-ID: <20111116101815.GC25332@redhat.com> On Tue, Nov 15, 2011 at 05:01:00PM -0600, Michael Roth wrote: > On 11/15/2011 11:24 AM, Barak Azulay wrote: > >Hi, > > > >One of the breakout sessions during the ovirt workshop [1] was about the guest > >tools, and focused mainly on the ovirt-guest-agent [2]. > > > >One of the issues discussed there, was the various existing guest agents out > >there, and the need to converge the efforts to a single agent that will serve > >all. > > > >while 4 agents were mentioned (Matahari, vdagent, qemu-ga& ovirt-guest-agent) > >during that discussion, we narrowed it down to 2 candidates: > > > >qemu-ga (aka virt-agent): > >------------------------- > >- Qemu specific - it was aimed for specific qemu needs (mainly quiesce guest > >I/O) > >- Communicates directly with qemu (not implemented yet) > > Won't be implemented until we finish the QMP conversion to QAPI > since that's a prereq to implementing a QMP server with proper async > support (needed for communication with a potentially non-responsive > guest agent). But once it's in guest extensions are completely > transparent to QMP/libvirt, which is highly desirable from a > management perspective since you're coding against a single > API/transport. FYI, I already have patches for libvirt to make it directly talk to a QEMU guest agent for a guest. We just configure the guest agent with a UNIX socket on the host, and livirt talks QMP to that directly. It was trivial, since the agent uses the same basic protocol as the QMP monitor, so from that POV I really like the QEMU guest agent for libvirt integration, and loathe to have to integrate with something that has invented yet another comms protocol. I expect we'll merge these patches in libvirt 0.9.8 and my intent is for Fedora 17 guests to have qemu-ga installed by default, so we get reliable guest shutdown/reboot support at last. https://www.redhat.com/archives/libvir-list/2011-October/msg00135.html Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| From ghammer at redhat.com Wed Nov 16 13:08:22 2011 From: ghammer at redhat.com (Gal Hammer) Date: Wed, 16 Nov 2011 15:08:22 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <20111115173331.GS7140@garlic.redhat.com> References: <201111151924.41357.bazulay@redhat.com> <20111115173331.GS7140@garlic.redhat.com> Message-ID: <4EC3B5C6.3080909@redhat.com> On 15/11/2011 19:33, Alon Levy wrote: > Does it have a seperate system level and user level part in Linux? It No. The ovirt-guest-agent have only one instance running in the system level. Gal. From dlaor at redhat.com Wed Nov 16 13:39:30 2011 From: dlaor at redhat.com (Dor Laor) Date: Wed, 16 Nov 2011 15:39:30 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3BC67.6030005@codemonkey.ws> References: <4EC3BC67.6030005@codemonkey.ws> Message-ID: <4EC3BD12.9000508@redhat.com> On 11/16/2011 03:36 PM, Anthony Liguori wrote: > On 11/15/2011 04:39 PM, Ayal Baron wrote: >>> If you want to talk about convergence, the discussion should start >>> around >>> collecting requirements. We can then figure out if the two sets of >>> requirements >>> are strictly overlapping or if there are any requirements that are >>> fundamentally >>> in opposition. >> >> Agreed. >> >> So vdsm guest agent goal is to ease administration of VMs. This is not >> saying much as it is quite broad so I will list what is provided today >> and some things we need to add: >> >> Assistance in VM life-cycle: >> "desktopShutdown" - Shuts the VM down gracefully from within the guest. >> "quiesce" - does not exist today. This is definitely a requirement for >> us. >> >> SSO support for spice sessions (automatically login into guest OS >> using provided credentials): >> "desktopLock" - lock current session, used when spice session gets >> disconnected / before giving a new user access to spice session >> "desktopLogin" >> "desktopLogoff" >> In addition, guest reports relevant info (currently active user, >> session state) >> >> Monitoring and inventory: >> currently agent sends info periodically, which includes a lot of info >> which should probably be broken down and served upon request. Info >> includes - >> - memory usage >> - NICs info (name, hw, inet, inet6) >> - appslist (list of installed apps / rpms) >> - OS type >> - guest hostname >> - internal file systems info (path, fs type, total space, used space) >> >> Personally I think the above should become more generic and support >> user defined counters (using WMI or collectd in the guest to collect >> the info and passing it via the guest agent), but that might be a >> different discussion. >> >> >> From qemu wiki, the following info about qemu guest agent: >> >> It's purpose: "Implement support for QMP commands and events that >> terminate and originate respectively within the guest using an agent >> built as part of QEMU. " >> - ties it directly to qemu, but not to specific functionality. ovirt >> guest agent definitely would need to support this >> >> In general, I would say ovirt-guest-agent is scoped to do everything >> the qemu-guest-agent is and then some, so there is definitely a lot of >> overlap. > > We have another requirement. We need to embed the source for the guest > agent in the QEMU release tarball. This is for GPL compliance since we > want to include an ISO (eventually) that contains binaries. > > This could be as simple as doing a git submodule but it would mean that > the guest agent would have to live in its own git repository. Do you all > see a problem with this? Not that I object of placing the code within qemu but I doubt this is a requirement, we can settle w/ GPL license for the code. A requirement of having the agent code reside within qemu is similar to some neighbors idea about kvm-tool and the kernel ... > > Regards, > > Anthony Liguori > >> >> >>> >>> Regards, >>> >>> Anthony Liguori >>> _______________________________________________ >>> Arch mailing list >>> Arch at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/arch >>> >> > > From dlaor at redhat.com Wed Nov 16 13:45:12 2011 From: dlaor at redhat.com (Dor Laor) Date: Wed, 16 Nov 2011 15:45:12 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <20111116120722.GZ7140@garlic.redhat.com> References: <4EC36C09.5060805@redhat.com> <20111116120722.GZ7140@garlic.redhat.com> Message-ID: <4EC3BE68.6090602@redhat.com> On 11/16/2011 02:07 PM, Alon Levy wrote: > On Wed, Nov 16, 2011 at 08:53:45AM +0100, Hans de Goede wrote: >> Hi, >> >> On 11/15/2011 11:39 PM, Ayal Baron wrote: >>> >> >> >> >>>> If you want to talk about convergence, the discussion should start >>>> around >>>> collecting requirements. We can then figure out if the two sets of >>>> requirements >>>> are strictly overlapping or if there are any requirements that are >>>> fundamentally >>>> in opposition. >>> >>> Agreed. >>> >>> So vdsm guest agent goal is to ease administration of VMs. This is not saying much as it is quite broad so I will list what is provided today and some things we need to add: >>> >>> Assistance in VM life-cycle: >>> "desktopShutdown" - Shuts the VM down gracefully from within the guest. >>> "quiesce" - does not exist today. This is definitely a requirement for us. >>> >>> SSO support for spice sessions (automatically login into guest OS using provided credentials): >>> "desktopLock" - lock current session, used when spice session gets disconnected / before giving a new user access to spice session >>> "desktopLogin" >>> "desktopLogoff" >>> In addition, guest reports relevant info (currently active user, session state) >>> >>> Monitoring and inventory: >>> currently agent sends info periodically, which includes a lot of info which should probably be broken down and served upon request. Info includes - >>> - memory usage >>> - NICs info (name, hw, inet, inet6) >>> - appslist (list of installed apps / rpms) >>> - OS type >>> - guest hostname >>> - internal file systems info (path, fs type, total space, used space) >>> >> >> >> >> If we're gathering requirements and trying to come up with one agent to rule them all, don't forget >> about VDI and the Spice agent. Currently the spice agent handles the following: >> >> 1) Paravirtual mouse (needed to get mouse coordinates right with multi monitor setups) >> 2) Send client monitor configuration, so that the guest os can adjust its resolution >> (and number and place of monitors) to match the client >> 3) Copy and paste in a platform neutral manner, if anyone wishes to add this to another agent >> please, please contact us (me) first. This is easy to get wrong (we went through 2 revisions >> of the protocol for this). >> 4) Allow the client to request the guest to tone down the bling (for low spec clients) I suggest we'll have a single protocol and serialization means (JSON) plus a common framework that manages permissions within the guest. Beyond that, the agent should have various plugins for example the above spice related ones (could be also vnc ones) can be implemented as plugins that the ovirt agent will invoke. >> > As long as we are collecting requirements, even if as Ayal said merging > spice requirements is not the OP's intent: > > 5) Window management - Agent can set location of windows, report > existing running applications and locations, get notified when a new > window is created. For exposing individual applications, this is a > future requirement. > >> Notes: >> 1) All of these are client<-> guest communication, rather then the host<-> guest communication >> which the other agents seem to focus on. >> >> 2) Getting copy paste right requires a system level guest agent process as well as a per user >> session agent process. >> > >> Regards, >> >> Hans >> > From abaron at redhat.com Wed Nov 16 14:10:00 2011 From: abaron at redhat.com (Ayal Baron) Date: Wed, 16 Nov 2011 09:10:00 -0500 (EST) Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3BDCC.3050102@codemonkey.ws> Message-ID: ----- Original Message ----- > On 11/16/2011 07:39 AM, Dor Laor wrote: > > On 11/16/2011 03:36 PM, Anthony Liguori wrote: > >> We have another requirement. We need to embed the source for the > >> guest > >> agent in the QEMU release tarball. This is for GPL compliance > >> since we > >> want to include an ISO (eventually) that contains binaries. > >> > >> This could be as simple as doing a git submodule but it would mean > >> that > >> the guest agent would have to live in its own git repository. Do > >> you all > >> see a problem with this? > > > > Not that I object of placing the code within qemu but I doubt this > > is a > > requirement, we can settle w/ GPL license for the code. > > > > A requirement of having the agent code reside within qemu is > > similar to some > > neighbors idea about kvm-tool and the kernel ... > > It can just be a submodule (like we do with SeaBIOS, etc.). The only > request is > that we split guest agent out of vdsm so we don't have to also > include all of > vdsm in the release tarballs. That would make the guest agent an > independent > git repository and presumably project. > > We can't ship a binary without including the source in the release. > The way we > handle this for things that are external to QEMU (SeaBIOS, OpenBIOS, > etc.) are > git submodules. It is already a separate git so that would not be a problem... > > Regards, > > Anthony Liguori > From mdroth at linux.vnet.ibm.com Wed Nov 16 14:59:35 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Wed, 16 Nov 2011 08:59:35 -0600 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <488c9fd7-8a5b-4f1c-a3bd-cb750f5769c4@zmail07.collab.prod.int.phx2.redhat.com> References: <488c9fd7-8a5b-4f1c-a3bd-cb750f5769c4@zmail07.collab.prod.int.phx2.redhat.com> Message-ID: <4EC3CFD7.7070504@linux.vnet.ibm.com> On 11/16/2011 02:16 AM, Ayal Baron wrote: > > > ----- Original Message ----- >> Hi, >> >> On 11/15/2011 11:39 PM, Ayal Baron wrote: >>> >> >> >> >>>> If you want to talk about convergence, the discussion should start >>>> around >>>> collecting requirements. We can then figure out if the two sets >>>> of >>>> requirements >>>> are strictly overlapping or if there are any requirements that are >>>> fundamentally >>>> in opposition. >>> >>> Agreed. >>> >>> So vdsm guest agent goal is to ease administration of VMs. This is >>> not saying much as it is quite broad so I will list what is >>> provided today and some things we need to add: >>> >>> Assistance in VM life-cycle: >>> "desktopShutdown" - Shuts the VM down gracefully from within the >>> guest. >>> "quiesce" - does not exist today. This is definitely a requirement >>> for us. >>> >>> SSO support for spice sessions (automatically login into guest OS >>> using provided credentials): >>> "desktopLock" - lock current session, used when spice session gets >>> disconnected / before giving a new user access to spice session >>> "desktopLogin" >>> "desktopLogoff" >>> In addition, guest reports relevant info (currently active user, >>> session state) >>> >>> Monitoring and inventory: >>> currently agent sends info periodically, which includes a lot of >>> info which should probably be broken down and served upon request. >>> Info includes - >>> - memory usage >>> - NICs info (name, hw, inet, inet6) >>> - appslist (list of installed apps / rpms) >>> - OS type >>> - guest hostname >>> - internal file systems info (path, fs type, total space, used >>> space) >>> >> >> >> >> If we're gathering requirements and trying to come up with one agent >> to rule them all, don't forget > > I don't think we're trying to come up with one agent to rule them all, just avoid duplication of efforts if most of what the 2 agents are doing overlaps. > I think we can safely say that seeing as oVirt is KVM centric, ovirt-guest-agent wants to leverage qemu/kvm to the fullest which aligns with what qemu-guest-agent is doing. > However, ovirt-guest-agent is required to do a lot more, so we need to see if and how we resolve this. > >> about VDI and the Spice agent. Currently the spice agent handles the >> following: >> >> 1) Paravirtual mouse (needed to get mouse coordinates right with >> multi monitor setups) >> 2) Send client monitor configuration, so that the guest os can adjust >> its resolution >> (and number and place of monitors) to match the client >> 3) Copy and paste in a platform neutral manner, if anyone wishes to >> add this to another agent >> please, please contact us (me) first. This is easy to get wrong >> (we went through 2 revisions >> of the protocol for this). >> 4) Allow the client to request the guest to tone down the bling (for >> low spec clients) >> >> Notes: >> 1) All of these are client<-> guest communication, rather then the >> host<-> guest communication >> which the other agents seem to focus on. >> >> 2) Getting copy paste right requires a system level guest agent >> process as well as a per user >> session agent process. > > Neither qemu-guest-agent nor ovirt-guest-agent is aligned with doing any of the above, so I'm not sure there is any justification in uniting the spice agent with the rest. > copy/paste was actually one of the initial use cases motivating qemu-ga; it's just that the requirements (system+user-level agents) were so different from the more pressing use cases of things like reliable shutdown/reboot that it's been put off for now. At some point we had a basic plan on how to approach it, but that needs to be re-assessed. >> >> Regards, >> >> Hans >> > From mdroth at linux.vnet.ibm.com Wed Nov 16 15:28:16 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Wed, 16 Nov 2011 09:28:16 -0600 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <201111161413.21026.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <201111160905.23316.bazulay@redhat.com> <724C1350-38D1-4EE2-AC7B-27666B7710A1@suse.de> <201111161413.21026.bazulay@redhat.com> Message-ID: <4EC3D690.2020609@linux.vnet.ibm.com> On 11/16/2011 06:13 AM, Barak Azulay wrote: > On Wednesday 16 November 2011 10:16:57 Alexander Graf wrote: >> On 16.11.2011, at 08:05, Barak Azulay wrote: >>> On Wednesday 16 November 2011 02:42:30 Alexander Graf wrote: >>>> On 16.11.2011, at 00:01, Michael Roth wrote: >>>>> But practically-speaking, it's unavoidable that qemu-specific >>>>> management tooling will need to communicate with qemu (via >>>>> QMP/libqmp/HMP/etc, or by proxy via libvirt). It's through those same >>>>> channels that the qemu-ga interfaces will ultimately be exposed, so >>>>> the problem of qemu-ga vs. ovirt-guest-agent isn't really any >>>>> different than the problem of QMP's system_powerdown/info_balloon/etc >>>>> vs. ovirt-guest-agent's >>>>> Shutdown/Available_Ram/etc: it's a policy decision rather than argument >>>>> for choosing one project over another. >>>> >>>> I don't see why we shouldn't be able to just proxy whatever >>>> communication happens between the guest agent and the management tool >>>> through qemu. At that point qemu could talk to the guest agent just as >>>> well as the management tool and everyone's happy. >>> >>> I'm not sure proxying all the requests to the guset through qemu is >>> desirable, other than having single point of management, most of the >>> calls will be pass throgh and has no interest to qemu (MITM?). >>> >>> There is a big advantage on direct communication (VDSM<-> agent), that >>> way features can be added to the ovirt stack without the need to add it >>> to the qemu. >> >> If we keep the protocol well-defined, we can get that for free. Just have >> every command carry its own size and a request id shich the reply also >> contains and suddenly you get asynchronous proxyable communication. >> > > > Sure we can keep commands synchronized in various ways the question is do we > want that, there are a few down sides for that: > 1 - VDSM will have to pass through 2 proxies (libvirt& qemu) in order to > deliver a message to the guest, this byiself is not such a big disadvantage > but will force us to handle much more corner-cases. Can't rule out the possibility of corner-cases resulting from this, but the practical way to look at it is VDSM will need handle libvirt/QMP protocols well. The implementation of the proxying mechanism is where the extra challenge comes into play, but this should be transparent to the protocols VDSM speaks. Implementation-wise, just to give you an idea of the work involved if we took this route: 1) ovirt-guest-agent would need to convert request/response payloads from/to QMP payloads on the guest-side, which are JSON and should, theoretically, mesh well with a python-based agent. 2) You'd also need a schema, similar to qemu.git/qapi-schema-guest.json, to describe the calls you're proxying. The existing infrastructure in QEMU will handle all the work of marshalling/unmarshalling responses back to the QMP client on the host-side. It's a bit of extra work, but the benefit is unifying the qemu/guest-level management interface into a single place that's easy for QMP/libvirt to consume. > 2 - looking at the qemu-ga functionality (read& write ...) do we really want > to let a big chunk of data through both qemu& libvirt rather than directtly > to the comsumer (VDSM) VDSM isn't the only consumer however, HMP/QMP and libvirt are consumers in and of themselves. > 3 - When events are fired from the guest agent, the delay of passing it > through a double proxy will have it's latency penalty (as we have experianced > in the client disconnect spice event) > Getting them out of the guest is probably the biggest factor, delivering them between processes on the host is likely a small hit in comparison. > >>> I envision the agent will have 2 separate ports to listen to, one to >>> communicate to qemu and one for VDSM. >> >> Ugh, no, I'd much prefer a single 'bus' everyone attaches to. > > why? > > I'm thinking on situation we'll need to priorities commands arriving from qemu > over "management standard commands"& info gathering, sure there are number of > mechanisms to do that but it seems to me that a separation is the best way. > > e.g. I think we need to priorities a quiesce command from qemu over any other > info/command from VDSM. Do you mean prioritize in terms of order of delivery? Best way to do that is a single protocol with state-tracking, otherwise we're just racing. > > > >> >> Alex >> >>> Barak >>> >>>> Alex > From sgordon at redhat.com Wed Nov 16 16:05:48 2011 From: sgordon at redhat.com (Steve Gordon) Date: Wed, 16 Nov 2011 11:05:48 -0500 (EST) Subject: Call for list moderators. In-Reply-To: <4EC3BEB3.70200@redhat.com> Message-ID: Hi all, The issue of list moderation, particularly moderation of posts from unsubscribed users (usually their first post) vs spam, was raised at today's meeting. For now we have agreed that we should continue with the status quo whereby posts from unsubscribed users require moderation, as always there is a 'but' though. We need more moderators! The task is not too onerous at this time but ideally we'd like a few moderators for each list to ensure that any posts awaiting moderation are always processed in a timely manner. If you'd like to nominate yourself as a volunteer please just respond to this message, include the @ovirt.org list(s) you'd be happy to moderate in your message. Thanks, Steve ----- Original Message ----- > From: "Dor Laor" > To: board at ovirt.org > Sent: Wednesday, November 16, 2011 8:46:27 AM > Subject: Fwd: Your message to Arch awaits moderator approval > > How about allowing non registered members to send unapproved emails? > > -------- Original Message -------- > Subject: Your message to Arch awaits moderator approval > Date: Wed, 16 Nov 2011 08:45:21 -0500 > From: arch-bounces at ovirt.org > To: dlaor at redhat.com > > Your mail to 'Arch' with the subject > > Re: [Qemu-devel] converging around a single guest agent > > Is being held until the list moderator can review it for approval. > > The reason it is being held: > > Post by non-member to a members-only list > > Either the message will get posted to the list, or you will receive > notification of the moderator's decision. If you would like to > cancel > this posting, please visit the following URL: > > > http://lists.ovirt.org/mailman/confirm/arch/458a765d8cab62007fc55c038948e23e11608a56 > > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board > From pmyers at redhat.com Wed Nov 16 18:58:46 2011 From: pmyers at redhat.com (Perry Myers) Date: Wed, 16 Nov 2011 13:58:46 -0500 Subject: Release Managers for oVirt Message-ID: <4EC407E6.3040707@redhat.com> On the board call today, we discussed getting a release schedule nailed down (in the works) and also the need to nominate someone to own the release management. It was suggested that there be the following roles for oVirt release mgmt: * overall oVirt project release manager, responsible for coordinate with all of the sub-projects to drive a unified release of the entire stack for a given point release * per project release manager/owner that is responsible for putting out updates for a given sub-project and will work closely with the global release manager on overall releases If folks think that idea is sound, then the next step is to get volunteers/nominations for who should be the release manager (at least for the time being, this position might shift from person to person over time) The second thing is for each sub-project to nominate a point person for sub-project release mgmt. We can then put on a wiki page who the overall release manager is for a given release. The release managers for each sub-project probably can be called out on the projects page on ovirt.org Nominations/Volunteers? Perry From jchoate at redhat.com Wed Nov 16 19:18:38 2011 From: jchoate at redhat.com (Jon Choate) Date: Wed, 16 Nov 2011 14:18:38 -0500 (EST) Subject: Release Managers for oVirt In-Reply-To: <4EC407E6.3040707@redhat.com> Message-ID: ----- Original Message ----- > From: "Perry Myers" > To: arch at ovirt.org > Sent: Wednesday, November 16, 2011 1:58:46 PM > Subject: Release Managers for oVirt > > On the board call today, we discussed getting a release schedule > nailed > down (in the works) and also the need to nominate someone to own the > release management. > > It was suggested that there be the following roles for oVirt release > mgmt: > > * overall oVirt project release manager, responsible for coordinate > with all of the sub-projects to drive a unified release of the > entire > stack for a given point release > > * per project release manager/owner that is responsible for putting > out > updates for a given sub-project and will work closely with the > global > release manager on overall releases > > If folks think that idea is sound, then the next step is to get > volunteers/nominations for who should be the release manager (at > least > for the time being, this position might shift from person to person > over > time) > > The second thing is for each sub-project to nominate a point person > for > sub-project release mgmt. We can then put on a wiki page who the > overall release manager is for a given release. The release managers > for each sub-project probably can be called out on the projects page > on > ovirt.org > > Nominations/Volunteers? > > Perry > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch > "The second thing is for each sub-project to nominate a point person for sub-project release mgmt." Its a little unclear from this wording but I'm assuming that anyone in the community can nominate anyone for any subproject. For instance, you don't have to be a maintainer of or even a contributor to a subproject to nominate or be nominated. Is that what is intended? From bazulay at redhat.com Wed Nov 16 17:53:11 2011 From: bazulay at redhat.com (Barak Azulay) Date: Wed, 16 Nov 2011 19:53:11 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3D690.2020609@linux.vnet.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <201111161413.21026.bazulay@redhat.com> <4EC3D690.2020609@linux.vnet.ibm.com> Message-ID: <201111161953.12503.bazulay@redhat.com> On Wednesday 16 November 2011 17:28:16 Michael Roth wrote: > On 11/16/2011 06:13 AM, Barak Azulay wrote: > > On Wednesday 16 November 2011 10:16:57 Alexander Graf wrote: > >> On 16.11.2011, at 08:05, Barak Azulay wrote: > >>> On Wednesday 16 November 2011 02:42:30 Alexander Graf wrote: > >>>> On 16.11.2011, at 00:01, Michael Roth wrote: > >>>>> But practically-speaking, it's unavoidable that qemu-specific > >>>>> management tooling will need to communicate with qemu (via > >>>>> QMP/libqmp/HMP/etc, or by proxy via libvirt). It's through those same > >>>>> channels that the qemu-ga interfaces will ultimately be exposed, so > >>>>> the problem of qemu-ga vs. ovirt-guest-agent isn't really any > >>>>> different than the problem of QMP's system_powerdown/info_balloon/etc > >>>>> vs. ovirt-guest-agent's > >>>>> Shutdown/Available_Ram/etc: it's a policy decision rather than > >>>>> argument for choosing one project over another. > >>>> > >>>> I don't see why we shouldn't be able to just proxy whatever > >>>> communication happens between the guest agent and the management tool > >>>> through qemu. At that point qemu could talk to the guest agent just as > >>>> well as the management tool and everyone's happy. > >>> > >>> I'm not sure proxying all the requests to the guset through qemu is > >>> desirable, other than having single point of management, most of the > >>> calls will be pass throgh and has no interest to qemu (MITM?). > >>> > >>> There is a big advantage on direct communication (VDSM<-> agent), that > >>> way features can be added to the ovirt stack without the need to add it > >>> to the qemu. > >> > >> If we keep the protocol well-defined, we can get that for free. Just > >> have every command carry its own size and a request id shich the reply > >> also contains and suddenly you get asynchronous proxyable > >> communication. > > > > Sure we can keep commands synchronized in various ways the question is do > > we want that, there are a few down sides for that: > > 1 - VDSM will have to pass through 2 proxies (libvirt& qemu) in order to > > deliver a message to the guest, this byiself is not such a big > > disadvantage but will force us to handle much more corner-cases. > > Can't rule out the possibility of corner-cases resulting from this, but > the practical way to look at it is VDSM will need handle libvirt/QMP > protocols well. The implementation of the proxying mechanism is where > the extra challenge comes into play, but this should be transparent to > the protocols VDSM speaks. > > Implementation-wise, just to give you an idea of the work involved if we > took this route: > > 1) ovirt-guest-agent would need to convert request/response payloads > from/to QMP payloads on the guest-side, which are JSON and should, > theoretically, mesh well with a python-based agent. ovirt-guest-agent already uses JSON for communication with VDSM > > 2) You'd also need a schema, similar to qemu.git/qapi-schema-guest.json, > to describe the calls you're proxying. The existing infrastructure in > QEMU will handle all the work of marshalling/unmarshalling responses > back to the QMP client on the host-side. > > It's a bit of extra work, but the benefit is unifying the > qemu/guest-level management interface into a single place that's easy > for QMP/libvirt to consume. > The issue is not whether it's possible or not or the amount of efforts need to be done for that to happen, either for qemu-ga or ovirt-guest-agent this work needs to be done. the question is whether all comminication should go through the monitor (hence double proxy) or ... only a subset of the commands that are closly related to hypervisor functionality and separate it from general management-system related actions (e.g. ovirt or any other management system that wants to communicate to the guest). > > 2 - looking at the qemu-ga functionality (read& write ...) do we really > > want to let a big chunk of data through both qemu& libvirt rather than > > directtly to the comsumer (VDSM) > > VDSM isn't the only consumer however, HMP/QMP and libvirt are consumers > in and of themselves. This is not the claim it was just an example, it may as well be any other management system. The question still remains, if we want to pass a big chunk of data, do we want it to be passed through this double proxy layer ? personally I think it does not belong to core hypervisor management interface. > > > 3 - When events are fired from the guest agent, the delay of passing it > > through a double proxy will have it's latency penalty (as we have > > experianced in the client disconnect spice event) > > Getting them out of the guest is probably the biggest factor, delivering > them between processes on the host is likely a small hit in comparison. > From our experiance in the last 3 years this latency (passing a single proxy layer, libvirt) , affected the overall functionality & behaviour. However using libvirt as a monitor proxy also gave us some advantages, the question is what are the advantages of passing ALL guest-agent protocol through qemu ? > >>> I envision the agent will have 2 separate ports to listen to, one to > >>> communicate to qemu and one for VDSM. > >> > >> Ugh, no, I'd much prefer a single 'bus' everyone attaches to. > > > > why? > > > > I'm thinking on situation we'll need to priorities commands arriving from > > qemu over "management standard commands"& info gathering, sure there > > are number of mechanisms to do that but it seems to me that a separation > > is the best way. > > > > e.g. I think we need to priorities a quiesce command from qemu over any > > other info/command from VDSM. > > Do you mean prioritize in terms of order of delivery? Best way to do > that is a single protocol with state-tracking, otherwise we're just racing. > > >> Alex > >> > >>> Barak > >>> > >>>> Alex From hdegoede at redhat.com Wed Nov 16 17:55:06 2011 From: hdegoede at redhat.com (Hans de Goede) Date: Wed, 16 Nov 2011 18:55:06 +0100 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3BEF2.9060805@codemonkey.ws> References: <4EC36C09.5060805@redhat.com> <20111116120722.GZ7140@garlic.redhat.com> <4EC3BEF2.9060805@codemonkey.ws> Message-ID: <4EC3F8FA.7010100@redhat.com> Hi, On 11/16/2011 02:47 PM, Anthony Liguori wrote: > On 11/16/2011 06:07 AM, Alon Levy wrote: >> On Wed, Nov 16, 2011 at 08:53:45AM +0100, Hans de Goede wrote: >>> Hi, >>> >>> On 11/15/2011 11:39 PM, Ayal Baron wrote: >>>> >>> >>> >>> >>>>> If you want to talk about convergence, the discussion should start >>>>> around >>>>> collecting requirements. We can then figure out if the two sets of >>>>> requirements >>>>> are strictly overlapping or if there are any requirements that are >>>>> fundamentally >>>>> in opposition. >>>> >>>> Agreed. >>>> >>>> So vdsm guest agent goal is to ease administration of VMs. This is not saying much as it is quite broad so I will list what is provided today and some things we need to add: >>>> >>>> Assistance in VM life-cycle: >>>> "desktopShutdown" - Shuts the VM down gracefully from within the guest. >>>> "quiesce" - does not exist today. This is definitely a requirement for us. >>>> >>>> SSO support for spice sessions (automatically login into guest OS using provided credentials): >>>> "desktopLock" - lock current session, used when spice session gets disconnected / before giving a new user access to spice session >>>> "desktopLogin" >>>> "desktopLogoff" >>>> In addition, guest reports relevant info (currently active user, session state) >>>> >>>> Monitoring and inventory: >>>> currently agent sends info periodically, which includes a lot of info which should probably be broken down and served upon request. Info includes - >>>> - memory usage >>>> - NICs info (name, hw, inet, inet6) >>>> - appslist (list of installed apps / rpms) >>>> - OS type >>>> - guest hostname >>>> - internal file systems info (path, fs type, total space, used space) >>>> >>> >>> >>> >>> If we're gathering requirements and trying to come up with one agent to rule them all, don't forget >>> about VDI and the Spice agent. Currently the spice agent handles the following: >>> >>> 1) Paravirtual mouse (needed to get mouse coordinates right with multi monitor setups) > > I thought there was wide agreement that pv mouse should be extracted from the guest agent into its own driver. Yes AFAIK there is, but no-one has done that yet. I was merely listening what the spice agent is doing today, hopefully tomorrow > >>> 2) Send client monitor configuration, so that the guest os can adjust its resolution >>> (and number and place of monitors) to match the client > > I also wonder if this should be part of QXL? That is not really practically since this is something between the client and the guest, where as the QXL device does communication between the hypervisor (qemu) and the guest. Also there is a 1 head per QXL device relation, so that would mean that a single qxl dev needs to be aware of other QXL devices in order to communicate the relative position of its head to the other heads. Regards, Hans From pmyers at redhat.com Wed Nov 16 19:45:08 2011 From: pmyers at redhat.com (Perry Myers) Date: Wed, 16 Nov 2011 14:45:08 -0500 Subject: Release Managers for oVirt In-Reply-To: References: Message-ID: <4EC412C4.8070701@redhat.com> > "The second thing is for each sub-project to nominate a point person > for sub-project release mgmt." > > Its a little unclear from this wording but I'm assuming that anyone > in the community can nominate anyone for any subproject. For > instance, you don't have to be a maintainer of or even a contributor > to a subproject to nominate or be nominated. Is that what is > intended? Not sure why that wording is ambiguous. It says 'each sub-project to nominate', that means the sub-project team itself (i.e. oVirt Node team) would nominate someone internally to be release manager for that sub-project I don't think it makes sense to have the community at large nominate release managers for the subprojects From jchoate at redhat.com Wed Nov 16 19:51:12 2011 From: jchoate at redhat.com (Jon Choate) Date: Wed, 16 Nov 2011 14:51:12 -0500 (EST) Subject: Release Managers for oVirt In-Reply-To: <4EC412C4.8070701@redhat.com> Message-ID: <759b885e-b780-45bf-9810-c34ee8cd6245@zmail15.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > From: "Perry Myers" > To: "Jon Choate" > Cc: arch at ovirt.org > Sent: Wednesday, November 16, 2011 2:45:08 PM > Subject: Re: Release Managers for oVirt > > > "The second thing is for each sub-project to nominate a point > > person > > for sub-project release mgmt." > > > > Its a little unclear from this wording but I'm assuming that anyone > > in the community can nominate anyone for any subproject. For > > instance, you don't have to be a maintainer of or even a > > contributor > > to a subproject to nominate or be nominated. Is that what is > > intended? > > Not sure why that wording is ambiguous. It says 'each sub-project to > nominate', that means the sub-project team itself (i.e. oVirt Node > team) > would nominate someone internally to be release manager for that > sub-project > > I don't think it makes sense to have the community at large nominate > release managers for the subprojects > Well then I guess my follow-up question is what constitutes team membership? Does that mean maintainers only or maintainers and board? From pmyers at redhat.com Wed Nov 16 19:54:39 2011 From: pmyers at redhat.com (Perry Myers) Date: Wed, 16 Nov 2011 14:54:39 -0500 Subject: Release Managers for oVirt In-Reply-To: <759b885e-b780-45bf-9810-c34ee8cd6245@zmail15.collab.prod.int.phx2.redhat.com> References: <759b885e-b780-45bf-9810-c34ee8cd6245@zmail15.collab.prod.int.phx2.redhat.com> Message-ID: <4EC414FF.3010904@redhat.com> >> I don't think it makes sense to have the community at large nominate >> release managers for the subprojects >> > Well then I guess my follow-up question is what constitutes team membership? Does that mean maintainers only or maintainers and board? sub-project maintainers are listed here: http://www.ovirt.org/project/subprojects/ Those list define who gets to choose sub-project release managers So for example, right now the maintainers of oVirt Node are jboggs, apevec and mburns. They'll nominate one of themselves to be the oVirt Node release manager This shouldn't be a contentious topic, is there something you're driving at here? From mburns at redhat.com Wed Nov 16 19:55:07 2011 From: mburns at redhat.com (Mike Burns) Date: Wed, 16 Nov 2011 14:55:07 -0500 Subject: Release Managers for oVirt In-Reply-To: <759b885e-b780-45bf-9810-c34ee8cd6245@zmail15.collab.prod.int.phx2.redhat.com> References: <759b885e-b780-45bf-9810-c34ee8cd6245@zmail15.collab.prod.int.phx2.redhat.com> Message-ID: <1321473307.2936.45.camel@mburns-laptop> On Wed, 2011-11-16 at 14:51 -0500, Jon Choate wrote: > > ----- Original Message ----- > > From: "Perry Myers" > > To: "Jon Choate" > > Cc: arch at ovirt.org > > Sent: Wednesday, November 16, 2011 2:45:08 PM > > Subject: Re: Release Managers for oVirt > > > > > "The second thing is for each sub-project to nominate a point > > > person > > > for sub-project release mgmt." > > > > > > Its a little unclear from this wording but I'm assuming that anyone > > > in the community can nominate anyone for any subproject. For > > > instance, you don't have to be a maintainer of or even a > > > contributor > > > to a subproject to nominate or be nominated. Is that what is > > > intended? > > > > Not sure why that wording is ambiguous. It says 'each sub-project to > > nominate', that means the sub-project team itself (i.e. oVirt Node > > team) > > would nominate someone internally to be release manager for that > > sub-project > > > > I don't think it makes sense to have the community at large nominate > > release managers for the subprojects > > > Well then I guess my follow-up question is what constitutes team membership? Does that mean maintainers only or maintainers and board? Seems to me that it would be someone nominated by the maintainers of the sub-project. It can then be anyone that works on that sub-project be they a maintainer, or just a contributor. Mike > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From jchoate at redhat.com Wed Nov 16 19:56:26 2011 From: jchoate at redhat.com (Jon Choate) Date: Wed, 16 Nov 2011 14:56:26 -0500 (EST) Subject: Release Managers for oVirt In-Reply-To: <4EC414FF.3010904@redhat.com> Message-ID: <5ef86e74-8e62-4457-9599-361c53919a0e@zmail15.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > From: "Perry Myers" > To: "Jon Choate" > Cc: arch at ovirt.org > Sent: Wednesday, November 16, 2011 2:54:39 PM > Subject: Re: Release Managers for oVirt > > >> I don't think it makes sense to have the community at large > >> nominate > >> release managers for the subprojects > >> > > Well then I guess my follow-up question is what constitutes team > > membership? Does that mean maintainers only or maintainers and > > board? > > sub-project maintainers are listed here: > > http://www.ovirt.org/project/subprojects/ > > Those list define who gets to choose sub-project release managers > > So for example, right now the maintainers of oVirt Node are jboggs, > apevec and mburns. They'll nominate one of themselves to be the > oVirt > Node release manager > > This shouldn't be a contentious topic, is there something you're > driving > at here? > No not at all. I am still new to how open source projects manage themselves so I just wanted to understand who is involved in these decisions. I apologize if my ignorance came across as argumentative. From pmyers at redhat.com Wed Nov 16 19:57:10 2011 From: pmyers at redhat.com (Perry Myers) Date: Wed, 16 Nov 2011 14:57:10 -0500 Subject: Release Managers for oVirt In-Reply-To: <5ef86e74-8e62-4457-9599-361c53919a0e@zmail15.collab.prod.int.phx2.redhat.com> References: <5ef86e74-8e62-4457-9599-361c53919a0e@zmail15.collab.prod.int.phx2.redhat.com> Message-ID: <4EC41596.6030206@redhat.com> > No not at all. I am still new to how open source projects manage > themselves so I just wanted to understand who is involved in these > decisions. I apologize if my ignorance came across as argumentative. Ah ok, no worries From agl at us.ibm.com Wed Nov 16 20:24:51 2011 From: agl at us.ibm.com (Adam Litke) Date: Wed, 16 Nov 2011 14:24:51 -0600 Subject: converging around a single guest agent In-Reply-To: <201111151924.41357.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> Message-ID: <20111116202451.GI2726@us.ibm.com> I have been following this thread pretty closely and the one sentence summary of the current argument is: ovirt-guest-agent is already featureful and tested, so let's drop qemu-ga and have everyone adopt ovirt-guest-agent. Unfortunately, this track strays completely away from the stated goal of convergence. I have at least two examples of why the greater KVM community can never adopt ovirt-guest-agent as-is. To address this, I would like to counter with an example on how qemu-ga can enable the deployment of ovirt-guest-agent features and satisfy the needs of the whole community at the same time. 1) Scope: The ovirt-guest-agent contains functionality that is incredibly useful within the context of oVirt. Single Sign-on is very handy but KVM users outside the scope of oVirt will not want this extra complexity in their agent. For simplicity they will probably just write something small that does what they need (and we have failed to provide a ubiquitous KVM agent). 1) Deployment complexity: The more complex the guest agent is, the more often it will need to be updated (bug/security fixes, distro compatibility, new features). Rolling out guest agent updates does not scale well in large environments (especially when the guest and host administrators are not the same person). For these reasons (and many others), I support having an agent with very basic primitives that can be orchestrated by the host to provide needed functionality. This agent would present a low-level, stable, extensible API that everyone can use. Today qemu-ga supports the following verbs: sync ping info shutdown file-open file-close file-read file-write file-seek file-flush fsfreeze-status fsfreeze-freeze fsfreeze-thaw. If we add a generic execute mechanism, then the agent can provide everything needed by oVirt to deploy SSO. Let's assume that we have already agreed on some sort of security policy for the write-file and exec primitives. Consensus is possible on this issue but I don't want to get bogged down with that here. With the above primitives, SSO could be deployed automatically to a guest with the following sequence of commands: file-open "/sso-package.bin" "w" file-write file-close file-open "/sso-package.bin" "x" file-exec file-close At this point, the package is installed. It can contain whatever existing logic exists in the ovirt-guest-agent today. To perform a user login, we'll assume that sso-package.bin contains an executable 'sso/do-user-sso': file-open "/sso/do-user-sso" "x" exec file-close At this point the user would be logged in as before. Obviously, this type of approach could be made easier by providing a well designed exec API that returns command exit codes and (optionally) command output. We could also formalize the install of additional components into some sort of plugin interface. These are all relatively easy problems to solve. If we go in this direction, we would have a simple, general-purpose agent with low-level primitives that everyone can use. We would also be able to easily extend the agent based on the needs of individual deployments (not the least of which is an oVirt environment). If certain plugins become popular enough, they can always be promoted to first-order API calls in future versions of the API. What are your thoughts on this approach? -- Adam Litke IBM Linux Technology Center From mdroth at linux.vnet.ibm.com Wed Nov 16 21:44:58 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Wed, 16 Nov 2011 15:44:58 -0600 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <201111161953.12503.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <201111161413.21026.bazulay@redhat.com> <4EC3D690.2020609@linux.vnet.ibm.com> <201111161953.12503.bazulay@redhat.com> Message-ID: <4EC42EDA.7070809@linux.vnet.ibm.com> On 11/16/2011 11:53 AM, Barak Azulay wrote: > On Wednesday 16 November 2011 17:28:16 Michael Roth wrote: >> On 11/16/2011 06:13 AM, Barak Azulay wrote: >>> On Wednesday 16 November 2011 10:16:57 Alexander Graf wrote: >>>> On 16.11.2011, at 08:05, Barak Azulay wrote: >>>>> On Wednesday 16 November 2011 02:42:30 Alexander Graf wrote: >>>>>> On 16.11.2011, at 00:01, Michael Roth wrote: >>>>>>> But practically-speaking, it's unavoidable that qemu-specific >>>>>>> management tooling will need to communicate with qemu (via >>>>>>> QMP/libqmp/HMP/etc, or by proxy via libvirt). It's through those same >>>>>>> channels that the qemu-ga interfaces will ultimately be exposed, so >>>>>>> the problem of qemu-ga vs. ovirt-guest-agent isn't really any >>>>>>> different than the problem of QMP's system_powerdown/info_balloon/etc >>>>>>> vs. ovirt-guest-agent's >>>>>>> Shutdown/Available_Ram/etc: it's a policy decision rather than >>>>>>> argument for choosing one project over another. >>>>>> >>>>>> I don't see why we shouldn't be able to just proxy whatever >>>>>> communication happens between the guest agent and the management tool >>>>>> through qemu. At that point qemu could talk to the guest agent just as >>>>>> well as the management tool and everyone's happy. >>>>> >>>>> I'm not sure proxying all the requests to the guset through qemu is >>>>> desirable, other than having single point of management, most of the >>>>> calls will be pass throgh and has no interest to qemu (MITM?). >>>>> >>>>> There is a big advantage on direct communication (VDSM<-> agent), that >>>>> way features can be added to the ovirt stack without the need to add it >>>>> to the qemu. >>>> >>>> If we keep the protocol well-defined, we can get that for free. Just >>>> have every command carry its own size and a request id shich the reply >>>> also contains and suddenly you get asynchronous proxyable >>>> communication. >>> >>> Sure we can keep commands synchronized in various ways the question is do >>> we want that, there are a few down sides for that: >>> 1 - VDSM will have to pass through 2 proxies (libvirt& qemu) in order to >>> deliver a message to the guest, this byiself is not such a big >>> disadvantage but will force us to handle much more corner-cases. >> >> Can't rule out the possibility of corner-cases resulting from this, but >> the practical way to look at it is VDSM will need handle libvirt/QMP >> protocols well. The implementation of the proxying mechanism is where >> the extra challenge comes into play, but this should be transparent to >> the protocols VDSM speaks. >> >> Implementation-wise, just to give you an idea of the work involved if we >> took this route: >> >> 1) ovirt-guest-agent would need to convert request/response payloads >> from/to QMP payloads on the guest-side, which are JSON and should, >> theoretically, mesh well with a python-based agent. > > > > ovirt-guest-agent already uses JSON for communication with VDSM > > > >> >> 2) You'd also need a schema, similar to qemu.git/qapi-schema-guest.json, >> to describe the calls you're proxying. The existing infrastructure in >> QEMU will handle all the work of marshalling/unmarshalling responses >> back to the QMP client on the host-side. >> >> It's a bit of extra work, but the benefit is unifying the >> qemu/guest-level management interface into a single place that's easy >> for QMP/libvirt to consume. >> > > The issue is not whether it's possible or not or the amount of efforts need to > be done for that to happen, either for qemu-ga or ovirt-guest-agent this work > needs to be done. Right, just trying to flesh out the areas where the extra corner-cases argument to this approach would be relevant. On the host-side the corner cases are something that already needs to be handled in the context of QMP/libvirt in general, it's in the implementation of 1) and 2) on the agent side where they'd potentially arise. > > the question is whether all comminication should go through the monitor (hence > double proxy) or ... only a subset of the commands that are closly related to > hypervisor functionality and separate it from general management-system > related actions (e.g. ovirt or any other management system that wants to > communicate to the guest). > > > >>> 2 - looking at the qemu-ga functionality (read& write ...) do we really >>> want to let a big chunk of data through both qemu& libvirt rather than >>> directtly to the comsumer (VDSM) >> >> VDSM isn't the only consumer however, HMP/QMP and libvirt are consumers >> in and of themselves. > > > This is not the claim it was just an example, it may as well be any other > management system. > > The question still remains, if we want to pass a big chunk of data, do we want > it to be passed through this double proxy layer ? personally I think it does > not belong to core hypervisor management interface. The common use case in practice has been reading things like /proc/meminfo, which ties into higher-level guest statistics collection. I agree that for large data transfers this is not the ideal interface, but we can opt to not use the interface for such purposes at that level and avoid the potential for bad performance entirely. > >> >>> 3 - When events are fired from the guest agent, the delay of passing it >>> through a double proxy will have it's latency penalty (as we have >>> experianced in the client disconnect spice event) >> >> Getting them out of the guest is probably the biggest factor, delivering >> them between processes on the host is likely a small hit in comparison. >> > > From our experiance in the last 3 years this latency (passing a single proxy > layer, libvirt) , affected the overall functionality& behaviour. However > using libvirt as a monitor proxy also gave us some advantages, the question is > what are the advantages of passing ALL guest-agent protocol through qemu ? > There's the existing argument of coding against a single transport/interface which I think is generally applicable. Beyond that, events are a particularly good example actually: QEMU already has a state machine to aid lifecycle management and handle device callbacks, this data is then made available at the QMP-layer. A management interface at all levels of the stack will want to know things like whether the guest is in the middle of a migration, or paused, or in an error state, which you get via QMP. QEMU itself would benefit from additional guest-driven events like reboots, copy(/paste) notifications, etc, since these events can drive the state machine, or trigger device callbacks. We can then propagate the events up the stack via the existing (or a new) notification system for state changes. If we bypass QEMU in reporting these events we complicate things at the management level: we're augmenting events reported by QMP with racy out-of-band events, and taking events away from QEMU that could be useful internally. > > >>>>> I envision the agent will have 2 separate ports to listen to, one to >>>>> communicate to qemu and one for VDSM. >>>> >>>> Ugh, no, I'd much prefer a single 'bus' everyone attaches to. >>> >>> why? >>> >>> I'm thinking on situation we'll need to priorities commands arriving from >>> qemu over "management standard commands"& info gathering, sure there >>> are number of mechanisms to do that but it seems to me that a separation >>> is the best way. >>> >>> e.g. I think we need to priorities a quiesce command from qemu over any >>> other info/command from VDSM. >> >> Do you mean prioritize in terms of order of delivery? Best way to do >> that is a single protocol with state-tracking, otherwise we're just racing. >> >>>> Alex >>>> >>>>> Barak >>>>> >>>>>> Alex > From mdroth at linux.vnet.ibm.com Thu Nov 17 00:48:50 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Wed, 16 Nov 2011 18:48:50 -0600 Subject: [Qemu-devel] wiki summary In-Reply-To: <201111151924.41357.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> Message-ID: <4EC459F2.5010701@linux.vnet.ibm.com> I've tried to summarize the pros/cons, points, and proposals outlined in this thread at the following wiki: http://www.ovirt.org/wiki/Guest_agent_proposals Please feel free to add/edit as needed. If you don't have an account on ovirt.org let me know. Thanks! From mdroth at linux.vnet.ibm.com Thu Nov 17 02:09:46 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Wed, 16 Nov 2011 20:09:46 -0600 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <20111116202451.GI2726@us.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <20111116202451.GI2726@us.ibm.com> Message-ID: <4EC46CEA.4030009@linux.vnet.ibm.com> On 11/16/2011 02:24 PM, Adam Litke wrote: > I have been following this thread pretty closely and the one sentence summary of > the current argument is: ovirt-guest-agent is already featureful and tested, so > let's drop qemu-ga and have everyone adopt ovirt-guest-agent. Unfortunately, > this track strays completely away from the stated goal of convergence. I have > at least two examples of why the greater KVM community can never adopt > ovirt-guest-agent as-is. To address this, I would like to counter with an > example on how qemu-ga can enable the deployment of ovirt-guest-agent features > and satisfy the needs of the whole community at the same time. > > 1) Scope: The ovirt-guest-agent contains functionality that is incredibly > useful within the context of oVirt. Single Sign-on is very handy but KVM users > outside the scope of oVirt will not want this extra complexity in their agent. > For simplicity they will probably just write something small that does what they > need (and we have failed to provide a ubiquitous KVM agent). > > 1) Deployment complexity: The more complex the guest agent is, the more often it > will need to be updated (bug/security fixes, distro compatibility, new > features). Rolling out guest agent updates does not scale well in large > environments (especially when the guest and host administrators are not the same > person). > > For these reasons (and many others), I support having an agent with very basic > primitives that can be orchestrated by the host to provide needed functionality. > This agent would present a low-level, stable, extensible API that everyone can > use. Today qemu-ga supports the following verbs: sync ping info shutdown > file-open file-close file-read file-write file-seek file-flush fsfreeze-status > fsfreeze-freeze fsfreeze-thaw. If we add a generic execute mechanism, then the > agent can provide everything needed by oVirt to deploy SSO. > > Let's assume that we have already agreed on some sort of security policy for the > write-file and exec primitives. Consensus is possible on this issue but I > don't want to get bogged down with that here. > > With the above primitives, SSO could be deployed automatically to a guest with > the following sequence of commands: > > file-open "/sso-package.bin" "w" > file-write > file-close > file-open "/sso-package.bin" "x" > file-exec > file-close > > At this point, the package is installed. It can contain whatever existing logic > exists in the ovirt-guest-agent today. To perform a user login, we'll assume > that sso-package.bin contains an executable 'sso/do-user-sso': > > file-open "/sso/do-user-sso" "x" > exec > file-close > > At this point the user would be logged in as before. > > Obviously, this type of approach could be made easier by providing a well > designed exec API that returns command exit codes and (optionally) command > output. We could also formalize the install of additional components into some > sort of plugin interface. These are all relatively easy problems to solve. > > If we go in this direction, we would have a simple, general-purpose agent with > low-level primitives that everyone can use. We would also be able to easily > extend the agent based on the needs of individual deployments (not the least of > which is an oVirt environment). If certain plugins become popular enough, they > can always be promoted to first-order API calls in future versions of the API. > > What are your thoughts on this approach? > Another possibility, for functionality that may be more suited for a daemon that needs to maintain a lot of state, would be modifying the ovirt-guest-agent code to read/write to a (guest-local) named pipe. We can could then deploy the daemon via file-write+exec (assuming we provide a fork/detach flag), and the management tool could do request/response via file-write/file-read. It's almost equivalent to reading/writing directly to a virtio-serial channel, except there'd need to be a translation (base64decode(qmp_json_response.payload)->oga_json_response, and vice-versa) at the ovirt management layer. And we still reduce the deployment complexity since we can deploy/upgrade via a hypervisor push. There's actually so many ways this could be done with exec support... What's being lost in both approaches are ovirt-guest-agent-provided events, however. We'd either need to subsume those into qemu-ga, or provide a proxying mechanism on the guest-side for event reporting, which is something we've discussed in the past with the Spice folks with regard to support for session-level agents. From pbonzini at redhat.com Wed Nov 16 14:20:51 2011 From: pbonzini at redhat.com (Paolo Bonzini) Date: Wed, 16 Nov 2011 15:20:51 +0100 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3BDCC.3050102@codemonkey.ws> References: <4EC3BC67.6030005@codemonkey.ws> <4EC3BD12.9000508@redhat.com> <4EC3BDCC.3050102@codemonkey.ws> Message-ID: <4EC3C6C3.1020208@redhat.com> On 11/16/2011 02:42 PM, Anthony Liguori wrote: > On 11/16/2011 07:39 AM, Dor Laor wrote: >> On 11/16/2011 03:36 PM, Anthony Liguori wrote: >>> We have another requirement. We need to embed the source for the guest >>> agent in the QEMU release tarball. This is for GPL compliance since we >>> want to include an ISO (eventually) that contains binaries. >>> >>> This could be as simple as doing a git submodule but it would mean that >>> the guest agent would have to live in its own git repository. Do you all >>> see a problem with this? >> >> Not that I object of placing the code within qemu but I doubt this is a >> requirement, we can settle w/ GPL license for the code. >> >> A requirement of having the agent code reside within qemu is similar to some >> neighbors idea about kvm-tool and the kernel ... > > It can just be a submodule (like we do with SeaBIOS, etc.). The only request is > that we split guest agent out of vdsm so we don't have to also include all of > vdsm in the release tarballs. That would make the guest agent an independent > git repository and presumably project. It is already (git://gerrit.ovirt.org/ovirt-guest-agent). Barak, is there a gitweb/cgit instance? > We can't ship a binary without including the source in the release. The way we > handle this for things that are external to QEMU (SeaBIOS, OpenBIOS, etc.) are > git submodules. ovirt-guest-agent is licensed under GPLv3, so you do not need to; the options in GPLv3 include this one: d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. Of course having a separate repo, and mirroring to qemu.org both remain nice things to have. Paolo From abaron at redhat.com Thu Nov 17 08:46:37 2011 From: abaron at redhat.com (Ayal Baron) Date: Thu, 17 Nov 2011 03:46:37 -0500 (EST) Subject: converging around a single guest agent In-Reply-To: <20111116202451.GI2726@us.ibm.com> Message-ID: <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > I have been following this thread pretty closely and the one sentence > summary of > the current argument is: ovirt-guest-agent is already featureful and > tested, so > let's drop qemu-ga and have everyone adopt ovirt-guest-agent. What we're suggesting is let's drop *one* of the two agents (obviously it would be easier for us to drop qemu-ga, but we'd rather reach consensus and unite behind one agent regardless of which agent it is). > Unfortunately, > this track strays completely away from the stated goal of > convergence. I have > at least two examples of why the greater KVM community can never > adopt > ovirt-guest-agent as-is. To address this, I would like to counter > with an > example on how qemu-ga can enable the deployment of ovirt-guest-agent > features > and satisfy the needs of the whole community at the same time. > > 1) Scope: The ovirt-guest-agent contains functionality that is > incredibly > useful within the context of oVirt. Single Sign-on is very handy but > KVM users > outside the scope of oVirt will not want this extra complexity in > their agent. > For simplicity they will probably just write something small that > does what they > need (and we have failed to provide a ubiquitous KVM agent). I totally agree, but that could easily be resolved using the plugin architecture suggested before. > > 1) Deployment complexity: The more complex the guest agent is, the > more often it > will need to be updated (bug/security fixes, distro compatibility, > new > features). Rolling out guest agent updates does not scale well in > large > environments (especially when the guest and host administrators are > not the same > person). Using plugins, you just deploy the ones you need, keeping the attack surface / #bugs / need to update lower > > For these reasons (and many others), I support having an agent with > very basic > primitives that can be orchestrated by the host to provide needed > functionality. > This agent would present a low-level, stable, extensible API that > everyone can > use. Today qemu-ga supports the following verbs: sync ping info > shutdown > file-open file-close file-read file-write file-seek file-flush > fsfreeze-status > fsfreeze-freeze fsfreeze-thaw. If we add a generic execute > mechanism, then the > agent can provide everything needed by oVirt to deploy SSO. > > Let's assume that we have already agreed on some sort of security > policy for the > write-file and exec primitives. Consensus is possible on this issue > but I > don't want to get bogged down with that here. > > With the above primitives, SSO could be deployed automatically to a > guest with > the following sequence of commands: > > file-open "/sso-package.bin" "w" > file-write > file-close > file-open "/sso-package.bin" "x" > file-exec > file-close The guest can run on any number of hosts. currently, the guest tools contain all the relevant logic installed (specifically for the guest os version). What you're suggesting here is that we keep all the relevant guest-agent variants code on the host, automatically detect the guest os version and inject the correct file (e.g. SSO on winXP and on win2k8 is totally different). In addition, there might be things requiring boot for example. So to solve that we would instead need to install a set of tools on the guest like we do the guest agent today (it would be a separate package because it's management specific). And then we would tell the guest-agent to run tools from that set? Sounds overly complex to me. > > At this point, the package is installed. It can contain whatever > existing logic > exists in the ovirt-guest-agent today. To perform a user login, > we'll assume > that sso-package.bin contains an executable 'sso/do-user-sso': > > file-open "/sso/do-user-sso" "x" > exec > file-close > > At this point the user would be logged in as before. > > Obviously, this type of approach could be made easier by providing a > well > designed exec API that returns command exit codes and (optionally) > command > output. We could also formalize the install of additional components > into some > sort of plugin interface. These are all relatively easy problems to > solve. > > If we go in this direction, we would have a simple, general-purpose > agent with > low-level primitives that everyone can use. We would also be able to > easily > extend the agent based on the needs of individual deployments (not > the least of > which is an oVirt environment). If certain plugins become popular > enough, they > can always be promoted to first-order API calls in future versions of > the API. > > What are your thoughts on this approach? > > -- > Adam Litke > IBM Linux Technology Center > > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch > From abaron at redhat.com Thu Nov 17 08:59:44 2011 From: abaron at redhat.com (Ayal Baron) Date: Thu, 17 Nov 2011 03:59:44 -0500 (EST) Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC44F3F.5060602@codemonkey.ws> Message-ID: <030b182c-2851-40e5-b0e8-682725258fac@zmail07.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > On 11/16/2011 11:53 AM, Barak Azulay wrote: > > On Wednesday 16 November 2011 17:28:16 Michael Roth wrote: > >> 2) You'd also need a schema, similar to > >> qemu.git/qapi-schema-guest.json, > >> to describe the calls you're proxying. The existing infrastructure > >> in > >> QEMU will handle all the work of marshalling/unmarshalling > >> responses > >> back to the QMP client on the host-side. > >> > >> It's a bit of extra work, but the benefit is unifying the > >> qemu/guest-level management interface into a single place that's > >> easy > >> for QMP/libvirt to consume. > >> > > > > The issue is not whether it's possible or not or the amount of > > efforts need to > > be done for that to happen, either for qemu-ga or ovirt-guest-agent > > this work > > needs to be done. > > > > the question is whether all comminication should go through the > > monitor (hence > > double proxy) or ... only a subset of the commands that are closly > > related to > > hypervisor functionality and separate it from general > > management-system > > related actions (e.g. ovirt or any other management system that > > wants to > > communicate to the guest). > > Yes, all guest interaction should be funnelled through QEMU. QEMU > has one job > in life--to expose an interface to guests and turn it into something > more useful > to the host. QEMU expose an emulated AHCI controller and turns that > into VFS > operations. > > Likewise, QEMU should expose a paravirtual "agent" device to a guest, > and then > turn that into higher level management interfaces. Exposing higher level management interfaces means that qemu would have to do policy. I have no problem with this, but note that this is counter to what you've been advocating to up to now (e.g. high watermark event for disks). Also, you would still have to have low level interfaces to accomplish things that qemu has not implemented yet or is not interested in implementing (the use case is too narrow). > > QEMU's job is to sanitize information from the guest and try to turn > that into > something that is safer for the broader world to consume. QEMU also > deals with > isolating state in order to support things like live migration. This So are you suggesting that when a user reads a file you would automatically encode the contents? > ends up > being non trivial when it comes to guest agents as it turns out. > > When you bypass QEMU and have higher level components talk directly > to the > guest, you effectively skip through many layers of security and > potentially > break things like migration by spreading state beyond QEMU. It's of > course > fixable given enough hacking but it makes for a brittle architecture. > > VDSM runs as root, right? That means that a guest driven attack that No, vdsm runs as user vdsm. Operations that need root privileges are in a separate process with root privileges and this process exposes a limited API which vdsm is allowed to invoke. > exploits > an issue with guest-agent protocol handling is going to compromise > VDSM and gain > root access. OTOH, QEMU runs with greatly reduced privileges > isolating the > effect of such a compromise. > > VDSM really shouldn't be talking directly to the guest. libvirt > shouldn't be > either although it is now because we haven't properly plumbed the > guest agent > protocol through QMP. > > Regards, > > Anthony Liguori > _______________________________________________ > vdsm-devel mailing list > vdsm-devel at lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/vdsm-devel > From alevy at redhat.com Thu Nov 17 10:16:33 2011 From: alevy at redhat.com (Alon Levy) Date: Thu, 17 Nov 2011 12:16:33 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3F8FA.7010100@redhat.com> References: <4EC36C09.5060805@redhat.com> <20111116120722.GZ7140@garlic.redhat.com> <4EC3BEF2.9060805@codemonkey.ws> <4EC3F8FA.7010100@redhat.com> Message-ID: <20111117101633.GC12180@garlic.tlv.redhat.com> On Wed, Nov 16, 2011 at 06:55:06PM +0100, Hans de Goede wrote: > Hi, > > On 11/16/2011 02:47 PM, Anthony Liguori wrote: > >On 11/16/2011 06:07 AM, Alon Levy wrote: > >>On Wed, Nov 16, 2011 at 08:53:45AM +0100, Hans de Goede wrote: > >>>Hi, > >>> > >>>On 11/15/2011 11:39 PM, Ayal Baron wrote: > >>>> > >>> > >>> > >>> > >>>>>If you want to talk about convergence, the discussion should start > >>>>>around > >>>>>collecting requirements. We can then figure out if the two sets of > >>>>>requirements > >>>>>are strictly overlapping or if there are any requirements that are > >>>>>fundamentally > >>>>>in opposition. > >>>> > >>>>Agreed. > >>>> > >>>>So vdsm guest agent goal is to ease administration of VMs. This is not saying much as it is quite broad so I will list what is provided today and some things we need to add: > >>>> > >>>>Assistance in VM life-cycle: > >>>>"desktopShutdown" - Shuts the VM down gracefully from within the guest. > >>>>"quiesce" - does not exist today. This is definitely a requirement for us. > >>>> > >>>>SSO support for spice sessions (automatically login into guest OS using provided credentials): > >>>>"desktopLock" - lock current session, used when spice session gets disconnected / before giving a new user access to spice session > >>>>"desktopLogin" > >>>>"desktopLogoff" > >>>>In addition, guest reports relevant info (currently active user, session state) > >>>> > >>>>Monitoring and inventory: > >>>>currently agent sends info periodically, which includes a lot of info which should probably be broken down and served upon request. Info includes - > >>>>- memory usage > >>>>- NICs info (name, hw, inet, inet6) > >>>>- appslist (list of installed apps / rpms) > >>>>- OS type > >>>>- guest hostname > >>>>- internal file systems info (path, fs type, total space, used space) > >>>> > >>> > >>> > >>> > >>>If we're gathering requirements and trying to come up with one agent to rule them all, don't forget > >>>about VDI and the Spice agent. Currently the spice agent handles the following: > >>> > >>>1) Paravirtual mouse (needed to get mouse coordinates right with multi monitor setups) > > > >I thought there was wide agreement that pv mouse should be extracted from the guest agent into its own driver. > > Yes AFAIK there is, but no-one has done that yet. I was merely listening what the spice > agent is doing today, hopefully tomorrow > > > > >>>2) Send client monitor configuration, so that the guest os can adjust its resolution > >>>(and number and place of monitors) to match the client > > > >I also wonder if this should be part of QXL? > > That is not really practically since this is something between the client and the guest, > where as the QXL device does communication between the hypervisor (qemu) and the guest. > Also there is a 1 head per QXL device relation, so that would mean that a single qxl dev > needs to be aware of other QXL devices in order to communicate the relative position of > its head to the other heads. We do want to allow multiple heads on a single qxl device, since it would make RandR work. This only relates to the second point, Hans first point is still valid. > > Regards, > > Hans > From iheim at redhat.com Thu Nov 17 07:17:47 2011 From: iheim at redhat.com (Itamar Heim) Date: Thu, 17 Nov 2011 09:17:47 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3C6C3.1020208@redhat.com> References: <4EC3BC67.6030005@codemonkey.ws> <4EC3BD12.9000508@redhat.com> <4EC3BDCC.3050102@codemonkey.ws> <4EC3C6C3.1020208@redhat.com> Message-ID: <4EC4B51B.6040208@redhat.com> On 11/16/2011 04:20 PM, Paolo Bonzini wrote: > ... >> >> It can just be a submodule (like we do with SeaBIOS, etc.). The only >> request is >> that we split guest agent out of vdsm so we don't have to also include >> all of >> vdsm in the release tarballs. That would make the guest agent an >> independent >> git repository and presumably project. > > It is already (git://gerrit.ovirt.org/ovirt-guest-agent). Barak, is > there a gitweb/cgit instance? http://gerrit.ovirt.org/gitweb?p=ovirt-guest-agent.git From cctrieloff at redhat.com Thu Nov 17 14:54:50 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Thu, 17 Nov 2011 09:54:50 -0500 Subject: First community release In-Reply-To: <4EC41596.6030206@redhat.com> References: <5ef86e74-8e62-4457-9599-361c53919a0e@zmail15.collab.prod.int.phx2.redhat.com> <4EC41596.6030206@redhat.com> Message-ID: <4EC5203A.2060401@redhat.com> 1.) Can we see if we can back into a first community RC for Dec 15? 2.) Looking for a release manager nomination. Carl. From mdroth at linux.vnet.ibm.com Thu Nov 17 14:58:16 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Thu, 17 Nov 2011 08:58:16 -0600 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> References: <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> Message-ID: <4EC52108.7040606@linux.vnet.ibm.com> On 11/17/2011 02:46 AM, Ayal Baron wrote: > > > ----- Original Message ----- >> I have been following this thread pretty closely and the one sentence >> summary of >> the current argument is: ovirt-guest-agent is already featureful and >> tested, so >> let's drop qemu-ga and have everyone adopt ovirt-guest-agent. > > What we're suggesting is let's drop *one* of the two agents (obviously it would be easier for us to drop qemu-ga, but we'd rather reach consensus and unite behind one agent regardless of which agent it is). > >> Unfortunately, >> this track strays completely away from the stated goal of >> convergence. I have >> at least two examples of why the greater KVM community can never >> adopt >> ovirt-guest-agent as-is. To address this, I would like to counter >> with an >> example on how qemu-ga can enable the deployment of ovirt-guest-agent >> features >> and satisfy the needs of the whole community at the same time. >> >> 1) Scope: The ovirt-guest-agent contains functionality that is >> incredibly >> useful within the context of oVirt. Single Sign-on is very handy but >> KVM users >> outside the scope of oVirt will not want this extra complexity in >> their agent. >> For simplicity they will probably just write something small that >> does what they >> need (and we have failed to provide a ubiquitous KVM agent). > > I totally agree, but that could easily be resolved using the plugin architecture suggested before. > >> >> 1) Deployment complexity: The more complex the guest agent is, the >> more often it >> will need to be updated (bug/security fixes, distro compatibility, >> new >> features). Rolling out guest agent updates does not scale well in >> large >> environments (especially when the guest and host administrators are >> not the same >> person). > > Using plugins, you just deploy the ones you need, keeping the attack surface / #bugs / need to update lower But you still need to deploy those plugins somehow, so the logistics of distributing this code to multiple types/levels of guests remains, and plugins are insufficient to handle security fixes in the core code (however small that attack surface may be). Eventually you'll need a newer version of the guest agent installed. qemu-ga could be the vehicle for delivering those ovirt plugins/updates, and qemu-ga can upgrade itself to handle it's own security fixes/updates. With this model you can keep your agent functionality closely tied to the high-level management infrastructure, take liberties in what features/changes you need to add/make, and push-deploy those changes through qemu-ga. Low-level primitives to build high-level interfaces higher up the stack has always been a primary design goal so this all fits together fairly well from a QEMU perspective. The extra orchestration required is worth it, IMO, as the alternative is limiting customers to a particular distro, installing a similar backend, or shooting out emails to everyone asking them to update their guest agent so you can leverage feature X. > >> >> For these reasons (and many others), I support having an agent with >> very basic >> primitives that can be orchestrated by the host to provide needed >> functionality. >> This agent would present a low-level, stable, extensible API that >> everyone can >> use. Today qemu-ga supports the following verbs: sync ping info >> shutdown >> file-open file-close file-read file-write file-seek file-flush >> fsfreeze-status >> fsfreeze-freeze fsfreeze-thaw. If we add a generic execute >> mechanism, then the >> agent can provide everything needed by oVirt to deploy SSO. >> >> Let's assume that we have already agreed on some sort of security >> policy for the >> write-file and exec primitives. Consensus is possible on this issue >> but I >> don't want to get bogged down with that here. >> >> With the above primitives, SSO could be deployed automatically to a >> guest with >> the following sequence of commands: >> >> file-open "/sso-package.bin" "w" >> file-write >> file-close >> file-open "/sso-package.bin" "x" >> file-exec >> file-close > > The guest can run on any number of hosts. currently, the guest tools contain all the relevant logic installed (specifically for the guest os version). > What you're suggesting here is that we keep all the relevant guest-agent variants code on the host, automatically detect the guest os version and inject the correct file (e.g. SSO on winXP and on win2k8 is totally different). > In addition, there might be things requiring boot for example. So to solve that we would instead need to install a set of tools on the guest like we do the guest agent today (it would be a separate package because it's management specific). And then we would tell the guest-agent to run tools from that set? Sounds overly complex to me. > The nature of the tools is more an implementation detail. It could also be distributed the same way it is now, except with a CLI interface or something rather than via virtio-serial. Going even further, I posted another approach where ovirt-guest-agent just speaks to a local pipe, and qemu-ga execs ovirt-guest-agent and proxies RPCs via it's existing file-read/file-write interfaces. With a small amount work we could even provide an ovirt-exec command that automatically does the setup if required and takes "native" ovirt-guest-guest agent JSON requests/responses and nests them with a qemu-ga JSON request/response. So you get instant all the benefits of using the same transport as QMP, and QMP users get easy access to ovirt-guest-agent features. Not saying that's a better approach than deploying sets of scripts, but there's a lot of flexibility here with at least a couple that have virtually no negative impact to how extensible or consumable ovirt-guest-agent is at the high-level management level. >> >> At this point, the package is installed. It can contain whatever >> existing logic >> exists in the ovirt-guest-agent today. To perform a user login, >> we'll assume >> that sso-package.bin contains an executable 'sso/do-user-sso': >> >> file-open "/sso/do-user-sso" "x" >> exec >> file-close >> >> At this point the user would be logged in as before. >> >> Obviously, this type of approach could be made easier by providing a >> well >> designed exec API that returns command exit codes and (optionally) >> command >> output. We could also formalize the install of additional components >> into some >> sort of plugin interface. These are all relatively easy problems to >> solve. >> >> If we go in this direction, we would have a simple, general-purpose >> agent with >> low-level primitives that everyone can use. We would also be able to >> easily >> extend the agent based on the needs of individual deployments (not >> the least of >> which is an oVirt environment). If certain plugins become popular >> enough, they >> can always be promoted to first-order API calls in future versions of >> the API. >> >> What are your thoughts on this approach? >> >> -- >> Adam Litke >> IBM Linux Technology Center >> >> _______________________________________________ >> Arch mailing list >> Arch at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/arch >> > From cctrieloff at redhat.com Thu Nov 17 15:04:24 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Thu, 17 Nov 2011 10:04:24 -0500 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: References: Message-ID: <4EC52278.6040508@redhat.com> forwarding and added to subscription list. Carl. On 11/17/2011 09:43 AM, arch-bounces at ovirt.org wrote: > Re: [Qemu-devel] converging around a single guest agent.eml > > Subject: > Re: [Qemu-devel] converging around a single guest agent > From: > Anthony Liguori > Date: > 11/17/2011 09:42 AM > > To: > Ayal Baron > CC: > Barak Azulay , Michael Roth > , Gal Hammer , > vdsm-devel at lists.fedorahosted.org, qemu-devel at nongnu.org, arch at ovirt.org > > > On 11/17/2011 02:59 AM, Ayal Baron wrote: >> >> >> ----- Original Message ----- >>> On 11/16/2011 11:53 AM, Barak Azulay wrote: >>>> On Wednesday 16 November 2011 17:28:16 Michael Roth wrote: >>>>> 2) You'd also need a schema, similar to >>>>> qemu.git/qapi-schema-guest.json, >>>>> to describe the calls you're proxying. The existing infrastructure >>>>> in >>>>> QEMU will handle all the work of marshalling/unmarshalling >>>>> responses >>>>> back to the QMP client on the host-side. >>>>> >>>>> It's a bit of extra work, but the benefit is unifying the >>>>> qemu/guest-level management interface into a single place that's >>>>> easy >>>>> for QMP/libvirt to consume. >>>>> >>>> >>>> The issue is not whether it's possible or not or the amount of >>>> efforts need to >>>> be done for that to happen, either for qemu-ga or ovirt-guest-agent >>>> this work >>>> needs to be done. >>>> >>>> the question is whether all comminication should go through the >>>> monitor (hence >>>> double proxy) or ... only a subset of the commands that are closly >>>> related to >>>> hypervisor functionality and separate it from general >>>> management-system >>>> related actions (e.g. ovirt or any other management system that >>>> wants to >>>> communicate to the guest). >>> >>> Yes, all guest interaction should be funnelled through QEMU. QEMU >>> has one job >>> in life--to expose an interface to guests and turn it into something >>> more useful >>> to the host. QEMU expose an emulated AHCI controller and turns that >>> into VFS >>> operations. >>> >>> Likewise, QEMU should expose a paravirtual "agent" device to a guest, >>> and then >>> turn that into higher level management interfaces. >> >> Exposing higher level management interfaces means that qemu would >> have to do policy. > > No, the way we plan on doing this is having a guest agent schema > (qapi-schema-guest.json) that we can use to (1) white list valid > operations and (2) decode and re-encode operations. > > (1) let's us validate that guest state isn't escaping which keeps > migration safe > > (2) let's us scrub any potentially malicious input from the guest > before we hand it off to the management tool. > > Otherwise, we don't get in the middle and don't really care what the > verbs are. > >>> >>> QEMU's job is to sanitize information from the guest and try to turn >>> that into >>> something that is safer for the broader world to consume. QEMU also >>> deals with >>> isolating state in order to support things like live migration. This >> >> So are you suggesting that when a user reads a file you would >> automatically encode the contents? > > I'm not sure I understand what you're suggesting. > > Here's another way to think of this. In a typical enterprise > environment, you would secure your network infrastructure using > isolated zones. You may have a red zone (guest networking), a yellow > zone (management network), and a green zone (broader intranet). The > zones are physically separate with very few things that exist on two > zones. > > You pay special attention to anything that crosses zones and try to > minimize them as much as possible. You never allow something to live > on more than two zones. > > The guest is the red zone and the rest of the host environment is the > yellow zone. QEMU bridges between the red and yellow zone. That is > fundamentally its job in the stack. > > Other than the guest agent, VDSM lives purely in the yellow zone. In > fact, VDSM bridges from the yellow zone to the green zone (broader > management infrastructure). > > It may be easier to skip QEMU and have VDSM also stride into the red > zone. It's always easier to cross zones. But it's not good security > practice. There is tremendous value in having clean security layers. > > Regards, > > Anthony Liguori -------------- next part -------------- An HTML attachment was scrubbed... URL: From alevy at redhat.com Thu Nov 17 15:11:01 2011 From: alevy at redhat.com (Alon Levy) Date: Thu, 17 Nov 2011 17:11:01 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3CFD7.7070504@linux.vnet.ibm.com> References: <488c9fd7-8a5b-4f1c-a3bd-cb750f5769c4@zmail07.collab.prod.int.phx2.redhat.com> <4EC3CFD7.7070504@linux.vnet.ibm.com> Message-ID: <20111117151101.GF12180@garlic.tlv.redhat.com> On Wed, Nov 16, 2011 at 08:59:35AM -0600, Michael Roth wrote: > On 11/16/2011 02:16 AM, Ayal Baron wrote: > > > > > >----- Original Message ----- > >>Hi, > >> > >>On 11/15/2011 11:39 PM, Ayal Baron wrote: > >>> > >> > >> > >> > >>>>If you want to talk about convergence, the discussion should start > >>>>around > >>>>collecting requirements. We can then figure out if the two sets > >>>>of > >>>>requirements > >>>>are strictly overlapping or if there are any requirements that are > >>>>fundamentally > >>>>in opposition. > >>> > >>>Agreed. > >>> > >>>So vdsm guest agent goal is to ease administration of VMs. This is > >>>not saying much as it is quite broad so I will list what is > >>>provided today and some things we need to add: > >>> > >>>Assistance in VM life-cycle: > >>>"desktopShutdown" - Shuts the VM down gracefully from within the > >>>guest. > >>>"quiesce" - does not exist today. This is definitely a requirement > >>>for us. > >>> > >>>SSO support for spice sessions (automatically login into guest OS > >>>using provided credentials): > >>>"desktopLock" - lock current session, used when spice session gets > >>>disconnected / before giving a new user access to spice session > >>>"desktopLogin" > >>>"desktopLogoff" > >>>In addition, guest reports relevant info (currently active user, > >>>session state) > >>> > >>>Monitoring and inventory: > >>>currently agent sends info periodically, which includes a lot of > >>>info which should probably be broken down and served upon request. > >>>Info includes - > >>>- memory usage > >>>- NICs info (name, hw, inet, inet6) > >>>- appslist (list of installed apps / rpms) > >>>- OS type > >>>- guest hostname > >>>- internal file systems info (path, fs type, total space, used > >>>space) > >>> > >> > >> > >> > >>If we're gathering requirements and trying to come up with one agent > >>to rule them all, don't forget > > > >I don't think we're trying to come up with one agent to rule them all, just avoid duplication of efforts if most of what the 2 agents are doing overlaps. > >I think we can safely say that seeing as oVirt is KVM centric, ovirt-guest-agent wants to leverage qemu/kvm to the fullest which aligns with what qemu-guest-agent is doing. > >However, ovirt-guest-agent is required to do a lot more, so we need to see if and how we resolve this. > > > >>about VDI and the Spice agent. Currently the spice agent handles the > >>following: > >> > >>1) Paravirtual mouse (needed to get mouse coordinates right with > >>multi monitor setups) > >>2) Send client monitor configuration, so that the guest os can adjust > >>its resolution > >> (and number and place of monitors) to match the client > >>3) Copy and paste in a platform neutral manner, if anyone wishes to > >>add this to another agent > >> please, please contact us (me) first. This is easy to get wrong > >> (we went through 2 revisions > >> of the protocol for this). > >>4) Allow the client to request the guest to tone down the bling (for > >>low spec clients) > >> > >>Notes: > >>1) All of these are client<-> guest communication, rather then the > >>host<-> guest communication > >>which the other agents seem to focus on. > >> > >>2) Getting copy paste right requires a system level guest agent > >>process as well as a per user > >>session agent process. > > > >Neither qemu-guest-agent nor ovirt-guest-agent is aligned with doing any of the above, so I'm not sure there is any justification in uniting the spice agent with the rest. > > > > copy/paste was actually one of the initial use cases motivating > qemu-ga; it's just that the requirements (system+user-level agents) > were so different from the more pressing use cases of things like > reliable shutdown/reboot that it's been put off for now. At some > point we had a basic plan on how to approach it, but that needs to > be re-assessed. > I think for large opaque copies in/out to the guest, like image copy paste, or guest<->guest copy paste (word OLE) it would be nice to implemant a side channel scheme: message to allocate a channel message to deallocate a channel and signal successfull completion or error the channel is just another virtio-serial that is used for this communication only The benefits would be: no need to slow down other operations no base64 conversion (both sides) This of course means that this data is not being parsed by qemu, so it can't benefit from any whitelisting / schema description. That's why it should only be used for data that is undescribable - like the aformentioned image/guest copy case (for instance for text copy it makes possibly less sense - although again that's completely unstructured text, so perhaps it makes sense as well). > >> > >>Regards, > >> > >>Hans > >> > > > > From ofrenkel at redhat.com Thu Nov 17 15:43:01 2011 From: ofrenkel at redhat.com (Omer Frenkel) Date: Thu, 17 Nov 2011 10:43:01 -0500 (EST) Subject: Call for list moderators. In-Reply-To: Message-ID: Hi, i don't mind volunteer to help with engine-devel at ovirt.org (maybe even more later on if needed) Omer. ----- Original Message ----- > From: "Steve Gordon" > To: dlaor at redhat.com > Cc: arch at ovirt.org, board at ovirt.org > Sent: Wednesday, November 16, 2011 6:05:48 PM > Subject: Call for list moderators. > > Hi all, > > The issue of list moderation, particularly moderation of posts from > unsubscribed users (usually their first post) vs spam, was raised at > today's meeting. For now we have agreed that we should continue with > the status quo whereby posts from unsubscribed users require > moderation, as always there is a 'but' though. We need more > moderators! > > The task is not too onerous at this time but ideally we'd like a few > moderators for each list to ensure that any posts awaiting > moderation are always processed in a timely manner. If you'd like to > nominate yourself as a volunteer please just respond to this > message, include the @ovirt.org list(s) you'd be happy to moderate > in your message. > > Thanks, > > Steve > > ----- Original Message ----- > > From: "Dor Laor" > > To: board at ovirt.org > > Sent: Wednesday, November 16, 2011 8:46:27 AM > > Subject: Fwd: Your message to Arch awaits moderator approval > > > > How about allowing non registered members to send unapproved > > emails? > > > > -------- Original Message -------- > > Subject: Your message to Arch awaits moderator approval > > Date: Wed, 16 Nov 2011 08:45:21 -0500 > > From: arch-bounces at ovirt.org > > To: dlaor at redhat.com > > > > Your mail to 'Arch' with the subject > > > > Re: [Qemu-devel] converging around a single guest agent > > > > Is being held until the list moderator can review it for approval. > > > > The reason it is being held: > > > > Post by non-member to a members-only list > > > > Either the message will get posted to the list, or you will receive > > notification of the moderator's decision. If you would like to > > cancel > > this posting, please visit the following URL: > > > > > > http://lists.ovirt.org/mailman/confirm/arch/458a765d8cab62007fc55c038948e23e11608a56 > > > > _______________________________________________ > > Board mailing list > > Board at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/board > > > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board > From agl at us.ibm.com Thu Nov 17 15:58:33 2011 From: agl at us.ibm.com (Adam Litke) Date: Thu, 17 Nov 2011 09:58:33 -0600 Subject: converging around a single guest agent In-Reply-To: <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> References: <20111116202451.GI2726@us.ibm.com> <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> Message-ID: <20111117155833.GN2726@us.ibm.com> On Thu, Nov 17, 2011 at 03:46:37AM -0500, Ayal Baron wrote: > > > ----- Original Message ----- > > I have been following this thread pretty closely and the one sentence > > summary of the current argument is: ovirt-guest-agent is already featureful > > and tested, so let's drop qemu-ga and have everyone adopt ovirt-guest-agent. > > What we're suggesting is let's drop *one* of the two agents (obviously it > would be easier for us to drop qemu-ga, but we'd rather reach consensus and > unite behind one agent regardless of which agent it is). > > > Unfortunately, this track strays completely away from the stated goal of > > convergence. I have at least two examples of why the greater KVM community > > can never adopt ovirt-guest-agent as-is. To address this, I would like to > > counter with an example on how qemu-ga can enable the deployment of > > ovirt-guest-agent features and satisfy the needs of the whole community at > > the same time. > > > > 1) Scope: The ovirt-guest-agent contains functionality that is incredibly > > useful within the context of oVirt. Single Sign-on is very handy but KVM > > users outside the scope of oVirt will not want this extra complexity in > > their agent. For simplicity they will probably just write something small > > that does what they need (and we have failed to provide a ubiquitous KVM > > agent). > > I totally agree, but that could easily be resolved using the plugin > architecture suggested before. > > > > > 1) Deployment complexity: The more complex the guest agent is, the more > > often it will need to be updated (bug/security fixes, distro compatibility, > > new features). Rolling out guest agent updates does not scale well in large > > environments (especially when the guest and host administrators are not the > > same person). > > Using plugins, you just deploy the ones you need, keeping the attack surface / > #bugs / need to update lower In order for any KVM guest agent to become ubiquitous, I think the code _must_ live in the qemu repository. This includes the base infrastructure and a core set of plugins to provide the current set of qemu-ga APIs. This way, both endpoints (host/guest) can evolve together. How easy would it be to extract this basic infrastructure from the ovirt-guest-agent? Is the qemu project opposed to a Python agent? > > For these reasons (and many others), I support having an agent with very > > basic primitives that can be orchestrated by the host to provide needed > > functionality. This agent would present a low-level, stable, extensible API > > that everyone can use. Today qemu-ga supports the following verbs: sync > > ping info shutdown file-open file-close file-read file-write file-seek > > file-flush fsfreeze-status fsfreeze-freeze fsfreeze-thaw. If we add a > > generic execute mechanism, then the agent can provide everything needed by > > oVirt to deploy SSO. > > > > Let's assume that we have already agreed on some sort of security policy for > > the write-file and exec primitives. Consensus is possible on this issue but > > I don't want to get bogged down with that here. > > > > With the above primitives, SSO could be deployed automatically to a guest > > with the following sequence of commands: > > > > file-open "/sso-package.bin" "w" file-write file-close > > file-open "/sso-package.bin" "x" file-exec > > file-close > > The guest can run on any number of hosts. currently, the guest tools contain > all the relevant logic installed (specifically for the guest os version). > What you're suggesting here is that we keep all the relevant guest-agent > variants code on the host, automatically detect the guest os version and > inject the correct file (e.g. SSO on winXP and on win2k8 is totally > different). In addition, there might be things requiring boot for example. So > to solve that we would instead need to install a set of tools on the guest > like we do the guest agent today (it would be a separate package because it's > management specific). And then we would tell the guest-agent to run tools > from that set? Sounds overly complex to me. We already have that packaging complexity today. You must already maintain the various Windows packages somewhere. You'd just be pushing them from the host instead. Could you provide examples of the things required for boot? If you are talking virtio drivers, I think this is a separate problem. I would argue that vdsm should have a hardware "safe-mode" when the guest tools are not installed. This would be a set of hardware exposed that is known to work with all guests. Then, when the guest tools are installed, the hardware can be "upgraded" since we will know the guest can support paravirt hw. > > At this point, the package is installed. It can contain whatever existing > > logic exists in the ovirt-guest-agent today. To perform a user login, we'll > > assume that sso-package.bin contains an executable 'sso/do-user-sso': > > > > file-open "/sso/do-user-sso" "x" exec file-close > > > > At this point the user would be logged in as before. > > > > Obviously, this type of approach could be made easier by providing a well > > designed exec API that returns command exit codes and (optionally) command > > output. We could also formalize the install of additional components into > > some sort of plugin interface. These are all relatively easy problems to > > solve. > > > > If we go in this direction, we would have a simple, general-purpose agent > > with low-level primitives that everyone can use. We would also be able to > > easily extend the agent based on the needs of individual deployments (not > > the least of which is an oVirt environment). If certain plugins become > > popular enough, they can always be promoted to first-order API calls in > > future versions of the API. > > > > What are your thoughts on this approach? > > > > -- Adam Litke IBM Linux Technology Center > > > > _______________________________________________ Arch mailing list > > Arch at ovirt.org http://lists.ovirt.org/mailman/listinfo/arch > > > -- Adam Litke IBM Linux Technology Center From berrange at redhat.com Thu Nov 17 16:14:42 2011 From: berrange at redhat.com (Daniel P. Berrange) Date: Thu, 17 Nov 2011 16:14:42 +0000 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <20111117155833.GN2726@us.ibm.com> References: <20111116202451.GI2726@us.ibm.com> <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> <20111117155833.GN2726@us.ibm.com> Message-ID: <20111117161442.GF16420@redhat.com> On Thu, Nov 17, 2011 at 09:58:33AM -0600, Adam Litke wrote: > On Thu, Nov 17, 2011 at 03:46:37AM -0500, Ayal Baron wrote: > > > > > > ----- Original Message ----- > > > I have been following this thread pretty closely and the one sentence > > > summary of the current argument is: ovirt-guest-agent is already featureful > > > and tested, so let's drop qemu-ga and have everyone adopt ovirt-guest-agent. > > > > What we're suggesting is let's drop *one* of the two agents (obviously it > > would be easier for us to drop qemu-ga, but we'd rather reach consensus and > > unite behind one agent regardless of which agent it is). > > > > > Unfortunately, this track strays completely away from the stated goal of > > > convergence. I have at least two examples of why the greater KVM community > > > can never adopt ovirt-guest-agent as-is. To address this, I would like to > > > counter with an example on how qemu-ga can enable the deployment of > > > ovirt-guest-agent features and satisfy the needs of the whole community at > > > the same time. > > > > > > 1) Scope: The ovirt-guest-agent contains functionality that is incredibly > > > useful within the context of oVirt. Single Sign-on is very handy but KVM > > > users outside the scope of oVirt will not want this extra complexity in > > > their agent. For simplicity they will probably just write something small > > > that does what they need (and we have failed to provide a ubiquitous KVM > > > agent). > > > > I totally agree, but that could easily be resolved using the plugin > > architecture suggested before. > > > > > > > > 1) Deployment complexity: The more complex the guest agent is, the more > > > often it will need to be updated (bug/security fixes, distro compatibility, > > > new features). Rolling out guest agent updates does not scale well in large > > > environments (especially when the guest and host administrators are not the > > > same person). > > > > Using plugins, you just deploy the ones you need, keeping the attack surface / > > #bugs / need to update lower > > In order for any KVM guest agent to become ubiquitous, I think the code _must_ live > in the qemu repository. This includes the base infrastructure and a core set of > plugins to provide the current set of qemu-ga APIs. This way, both endpoints > (host/guest) can evolve together. How easy would it be to extract this basic > infrastructure from the ovirt-guest-agent? Is the qemu project opposed to a > Python agent? IMHO Python would be a really bad choice for the agent. An agent wants to be maximally portable to any guest OS, regardless of its vintage. The changes between each python release, even within the 2.x stream, let alone between 2.x and 3.x would cause us endless compatibility problems upon deployment. And while python is common on Linux, we don't really want to get into the business of installing the python runtime on Windows or other OS, simply to run an agent. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| From bazulay at redhat.com Thu Nov 17 16:34:47 2011 From: bazulay at redhat.com (Barak Azulay) Date: Thu, 17 Nov 2011 18:34:47 +0200 Subject: [Qemu-devel] wiki summary In-Reply-To: <4EC459F2.5010701@linux.vnet.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <4EC459F2.5010701@linux.vnet.ibm.com> Message-ID: <201111171834.48040.bazulay@redhat.com> On Thursday 17 November 2011 02:48:50 Michael Roth wrote: > I've tried to summarize the pros/cons, points, and proposals outlined in > this thread at the following wiki: > > http://www.ovirt.org/wiki/Guest_agent_proposals > > Please feel free to add/edit as needed. If you don't have an account on > ovirt.org let me know. > Thanks Michael, it's a good start. A few questions about the qemu-ga's requirements: #1 - same repo ? why is this a requirement ? - distributable via ISO - can you elaborate? - upgradeable via hypervisor push - by the title it sounds like it belongs to deployment, which sounds to me like it belongs to a higher management level #3 a few questions come up when I read it: - some may consider those primitives as a security breach - I understand the motivation of being able to do everything on the guest (exe) but we need to keep in mind it's various guest OSs, and it means that there should be a script for every OS type. to me the option of having a well defined interface is much more appealing Thanks Barak > Thanks! From pmyers at redhat.com Thu Nov 17 16:38:29 2011 From: pmyers at redhat.com (Perry Myers) Date: Thu, 17 Nov 2011 11:38:29 -0500 Subject: First community release In-Reply-To: <4EC5203A.2060401@redhat.com> References: <5ef86e74-8e62-4457-9599-361c53919a0e@zmail15.collab.prod.int.phx2.redhat.com> <4EC41596.6030206@redhat.com> <4EC5203A.2060401@redhat.com> Message-ID: <4EC53885.9030005@redhat.com> On 11/17/2011 09:54 AM, Carl Trieloff wrote: > > > 1.) Can we see if we can back into a first community RC for Dec 15? >From an oVirt Node (based on Fedora 16) perspective, I think we are good with Dec 15th alignment. We just have the one vdsm bug left to deal with before we should have mostly functional Fedora based oVirt Node. Other distros will need to comment on how their Node efforts are going and if Dec 15th is reasonable for them Perry From iheim at redhat.com Thu Nov 17 16:47:55 2011 From: iheim at redhat.com (Itamar Heim) Date: Thu, 17 Nov 2011 18:47:55 +0200 Subject: First community release In-Reply-To: <4EC53885.9030005@redhat.com> References: <5ef86e74-8e62-4457-9599-361c53919a0e@zmail15.collab.prod.int.phx2.redhat.com> <4EC41596.6030206@redhat.com> <4EC5203A.2060401@redhat.com> <4EC53885.9030005@redhat.com> Message-ID: <4EC53ABB.3070502@redhat.com> On 11/17/2011 06:38 PM, Perry Myers wrote: > On 11/17/2011 09:54 AM, Carl Trieloff wrote: >> >> >> 1.) Can we see if we can back into a first community RC for Dec 15? > > From an oVirt Node (based on Fedora 16) perspective, I think we are good > with Dec 15th alignment. We just have the one vdsm bug left to deal > with before we should have mostly functional Fedora based oVirt Node. > > Other distros will need to comment on how their Node efforts are going > and if Dec 15th is reasonable for them engine-wise - I would also like to know if any other distro is going to do an engine for this date (we'll take care of creating a fedora 16 compatible repo and a tar). From eric.gaulin at gmail.com Thu Nov 17 16:53:51 2011 From: eric.gaulin at gmail.com (Eric Gaulin) Date: Thu, 17 Nov 2011 11:53:51 -0500 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <20111117161442.GF16420@redhat.com> References: <20111116202451.GI2726@us.ibm.com> <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> <20111117155833.GN2726@us.ibm.com> <20111117161442.GF16420@redhat.com> Message-ID: On Thu, Nov 17, 2011 at 11:14 AM, Daniel P. Berrange wrote: > On Thu, Nov 17, 2011 at 09:58:33AM -0600, Adam Litke wrote: >> On Thu, Nov 17, 2011 at 03:46:37AM -0500, Ayal Baron wrote: >> > >> > >> > ----- Original Message ----- >> > > I have been following this thread pretty closely and the one sentence >> > > summary of the current argument is: ovirt-guest-agent is already featureful >> > > and tested, so let's drop qemu-ga and have everyone adopt ovirt-guest-agent. >> > >> > What we're suggesting is let's drop *one* of the two agents (obviously it >> > would be easier for us to drop qemu-ga, but we'd rather reach consensus and >> > unite behind one agent regardless of which agent it is). >> > >> > > ?Unfortunately, this track strays completely away from the stated goal of >> > > ?convergence. ?I have at least two examples of why the greater KVM community >> > > ?can never adopt ovirt-guest-agent as-is. ?To address this, I would like to >> > > ?counter with an example on how qemu-ga can enable the deployment of >> > > ?ovirt-guest-agent features and satisfy the needs of the whole community at >> > > ?the same time. >> > > >> > > 1) Scope: ?The ovirt-guest-agent contains functionality that is incredibly >> > > useful within the context of oVirt. ?Single Sign-on is very handy but KVM >> > > users outside the scope of oVirt will not want this extra complexity in >> > > their agent. ?For simplicity they will probably just write something small >> > > that does what they need (and we have failed to provide a ubiquitous KVM >> > > agent). >> > >> > I totally agree, but that could easily be resolved using the plugin >> > architecture suggested before. >> > >> > > >> > > 1) Deployment complexity: The more complex the guest agent is, the more >> > > often it will need to be updated (bug/security fixes, distro compatibility, >> > > new features). ?Rolling out guest agent updates does not scale well in large >> > > environments (especially when the guest and host administrators are not the >> > > same person). >> > >> > Using plugins, you just deploy the ones you need, keeping the attack surface / >> > #bugs / need to update lower >> >> In order for any KVM guest agent to become ubiquitous, I think the code _must_ live >> in the qemu repository. ?This includes the base infrastructure and a core set of >> plugins to provide the current set of qemu-ga APIs. ?This way, both endpoints >> (host/guest) can evolve together. ?How easy would it be to extract this basic >> infrastructure from the ovirt-guest-agent? ?Is the qemu project opposed to a >> Python agent? > > IMHO Python would be a really bad choice for the agent. An agent wants to be > maximally portable to any guest OS, regardless of its vintage. The changes > between each python release, even within the 2.x stream, let alone between > 2.x and 3.x would cause us endless compatibility problems upon deployment. > And while python is common on Linux, we don't really want to get into the > business of installing the python runtime on Windows or other OS, simply to > run an agent. > > Regards, > Daniel > -- I agree with Daniel, A good example to get inspired from is the ZABBIX agent. A single C source tree that can be compiled to many Unix and Windows binaries. Eric Gaulin ___ From bazulay at redhat.com Thu Nov 17 17:09:10 2011 From: bazulay at redhat.com (Barak Azulay) Date: Thu, 17 Nov 2011 19:09:10 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <20111116202451.GI2726@us.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <20111116202451.GI2726@us.ibm.com> Message-ID: <201111171909.11021.bazulay@redhat.com> On Wednesday 16 November 2011 22:24:51 Adam Litke wrote: > I have been following this thread pretty closely and the one sentence > summary of the current argument is: ovirt-guest-agent is already > featureful and tested, so let's drop qemu-ga and have everyone adopt > ovirt-guest-agent. Not exactly what I meant, but the above is also a factor that needs to be taken into account. to be on the safe side I'll add: 1 - true we want the ovirt-guest-agent to be widly adopted. 2 - we are willing to do major changes in it to fit all (let's discuss it in details) 3 - one can not ignore the: * maturity of the 2 components * capabilities when considering a convergence into a single agent > Unfortunately, this track strays completely away from > the stated goal of convergence. I have at least two examples of why the > greater KVM community can never adopt ovirt-guest-agent as-is. To address > this, I would like to counter with an example on how qemu-ga can enable > the deployment of ovirt-guest-agent features and satisfy the needs of the > whole community at the same time. > > 1) Scope: The ovirt-guest-agent contains functionality that is incredibly > useful within the context of oVirt. Single Sign-on is very handy but KVM > users outside the scope of oVirt will not want this extra complexity in > their agent. For simplicity they will probably just write something small > that does what they need (and we have failed to provide a ubiquitous KVM > agent). the ovirt-guest-agent functions well when the SSO if not installed, so no need to install it if one does not wish to enjoy it > > 1) Deployment complexity: The more complex the guest agent is, the more > often it will need to be updated (bug/security fixes, distro > compatibility, new features). Rolling out guest agent updates does not > scale well in large environments (especially when the guest and host > administrators are not the same person). This is what we have been doing for the last few years (with the entire ovirt stack). so we are basing our approach on that experiance. In the session I gave I tried to touch the way we handle deployment of the packages (both linux & windows guests), but it was a too large subject to contain in that session. deploying and upgrading the agent was one of the issues handled. I can elaborate more on the isse if required. I'm not sure that the deployment/upgrade of guest agent should be done through the monitor, I think it should be left for a higher management layer. > > For these reasons (and many others), I support having an agent with very > basic primitives that can be orchestrated by the host to provide needed > functionality. This agent would present a low-level, stable, extensible > API that everyone can use. Today qemu-ga supports the following verbs: > sync ping info shutdown file-open file-close file-read file-write > file-seek file-flush fsfreeze-status fsfreeze-freeze fsfreeze-thaw. If we > add a generic execute mechanism, then the agent can provide everything > needed by oVirt to deploy SSO. I'm not sure this approach will pass common-criteria review. > > Let's assume that we have already agreed on some sort of security policy > for the write-file and exec primitives. Consensus is possible on this > issue but I don't want to get bogged down with that here. > > With the above primitives, SSO could be deployed automatically to a guest > with the following sequence of commands: > > file-open "/sso-package.bin" "w" > file-write > file-close > file-open "/sso-package.bin" "x" > file-exec > file-close > > At this point, the package is installed. It can contain whatever existing > logic exists in the ovirt-guest-agent today. To perform a user login, > we'll assume that sso-package.bin contains an executable > 'sso/do-user-sso': > > file-open "/sso/do-user-sso" "x" > exec > file-close > > At this point the user would be logged in as before. > > Obviously, this type of approach could be made easier by providing a well > designed exec API that returns command exit codes and (optionally) command > output. We could also formalize the install of additional components into > some sort of plugin interface. These are all relatively easy problems to > solve. > > If we go in this direction, we would have a simple, general-purpose agent > with low-level primitives that everyone can use. We would also be able to > easily extend the agent based on the needs of individual deployments (not > the least of which is an oVirt environment). If certain plugins become > popular enough, they can always be promoted to first-order API calls in > future versions of the API. > > What are your thoughts on this approach? When thinking on various guest OSs, I think the approach of a well defined API is much more appealing. The reason is that the implementation is contained within the specific OS implementation of the guest-agent, and not left to a person coding this from the high level management system. e.g. on the SSO example you gave all the coding of the seaquence of login needs to be done on the management system and needs to first check what OS it is and only than execute the relevant code (which is much harder for error handling in that situation), while when using a well defined interface all you need to do is call the "Login()" API. I agree that from development point of view the flexible approach of "I can do anything in the guest" is much more appiling, I don't think this is the case for enterprise virtualization system that needs to manage thousands of VMs with different OS types Thanks Barak Azulay From johnmark at redhat.com Thu Nov 17 18:22:20 2011 From: johnmark at redhat.com (John Mark Walker) Date: Thu, 17 Nov 2011 13:22:20 -0500 (EST) Subject: First community release In-Reply-To: <4EC5203A.2060401@redhat.com> Message-ID: <44935a16-e3c4-42f7-a1bc-395f16f52e2c@zmail01.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > > > 1.) Can we see if we can back into a first community RC for Dec 15? >From a packaging in Fedora perspective for ovirt-engine, there's the matter of the webadmin and UserPortal apps depending on JBoss, which isn't in Fedora yet. There is an attempt to get it packaged for Fedora 17 - would one of you like to lend a hand? I'm happy to put you in touch with Marek, if you're not already - http://fedoraproject.org/wiki/JBossAS7 There's also the matter of not being depending on a specific version of JBoss, or even on JBoss at all. What about those who want to run Tomcat? Not being a java guy, I haven't the faintest idea how difficult it is to port apps to other servlet containers or app servers. -JM From mdroth at linux.vnet.ibm.com Thu Nov 17 19:58:03 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Thu, 17 Nov 2011 13:58:03 -0600 Subject: [Qemu-devel] wiki summary In-Reply-To: <201111171834.48040.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC459F2.5010701@linux.vnet.ibm.com> <201111171834.48040.bazulay@redhat.com> Message-ID: <4EC5674B.30807@linux.vnet.ibm.com> On 11/17/2011 10:34 AM, Barak Azulay wrote: > On Thursday 17 November 2011 02:48:50 Michael Roth wrote: >> I've tried to summarize the pros/cons, points, and proposals outlined in >> this thread at the following wiki: >> >> http://www.ovirt.org/wiki/Guest_agent_proposals >> >> Please feel free to add/edit as needed. If you don't have an account on >> ovirt.org let me know. >> > > Thanks Michael, it's a good start. > > > A few questions about the qemu-ga's requirements: > > #1 > - same repo ? why is this a requirement ? Or git submodule. Main reasons are that integration with QMP requires that qemu be able to generate marshaling code from a guest agent schema definition of commands/parameters, and that qemu needs to be able to consume guest agent extensions internally. A few examples that came up in this thread were opening new virtio-serial channel via agent calls, and registering device callbacks/driving state machine changes for guest agent events. Since we'd like to pursue a push-deployment model where QEMU can deploy a specific, compatible version of the agent to a bootstrapped guest (qemu-ga pre-installed via guest distro or ISO package), having code changes in-sync with repo would be necessary. VMware has a similar model for handling guest tools upgrades, where the hypervisor pushes upgrades based on host hypervisor level: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008907 The alternative is strict APIs with backward-compatibility with down-level agents, which complicates things tremendously on the QEMU side, and pretty much everywhere in the stack. Just keeping libvirt in sync with QMP has proven difficult and that's just on the host, with a common distro and fairly close development communities. Extending this kind of synchronization out to multiple guest distros with varying levels of guest agents makes this far harder. > - distributable via ISO - can you elaborate? We'd eventually like to have an analogue to virtualbox/vmware guest tools, which ship with the hypervisor and can be deployed in a guest via an ISO made available in the guest as a cdrom when push-deployment isn't an option (guest doesnt already have some version of an agent with upgrade support installed). This is to avoid limiting support to specific distros due to lack of available packages in guest repo. > - upgradeable via hypervisor push - by the title it sounds like it belongs > to deployment, which sounds to me like it belongs to a higher management > level We'd like ability to push to be available all throughout the stack. If device X has a callback for event Y, which is only available via version Z of the guest agent, we're now reliant on layers far higher up the stack to enable low-level functionality that's beneficial at all levels. > > #3 a few questions come up when I read it: > - some may consider those primitives as a security breach s/some/virtually everyone/ :) Yes, this is a problem that'll need to be addressed. But at the end of the day, QEMU/host *must* be trusted if there's so be any pretense of security, since we have access to everything at the end of the day. Additionally, VMware has been successfully leveraging guest file access, automatic upgrades of guest tools, and exec functionality for quite some time now. That's not to say we don't need to examine the implications closely, but there's precedence. > - I understand the motivation of being able to do everything on the guest > (exe) but we need to keep in mind it's various guest OSs, and it means > that there should be a script for every OS type. to me the option of > having a well defined interface is much more appealing Agreed, and we should strive for that. But rarely is an interface designed so well that it never needs to change, and however well-defined it may be, it will grow with time and that growth entails deploying new guest code. > > Thanks > Barak > >> Thanks! > > From iheim at redhat.com Thu Nov 17 23:05:44 2011 From: iheim at redhat.com (Itamar Heim) Date: Fri, 18 Nov 2011 01:05:44 +0200 Subject: First community release In-Reply-To: <44935a16-e3c4-42f7-a1bc-395f16f52e2c@zmail01.collab.prod.int.phx2.redhat.com> References: <44935a16-e3c4-42f7-a1bc-395f16f52e2c@zmail01.collab.prod.int.phx2.redhat.com> Message-ID: <4EC59348.2000909@redhat.com> On 11/17/2011 08:22 PM, John Mark Walker wrote: > ----- Original Message ----- >> >> >> 1.) Can we see if we can back into a first community RC for Dec 15? > > From a packaging in Fedora perspective for ovirt-engine, there's the matter of the webadmin and UserPortal apps depending on JBoss, which isn't in Fedora yet. There is an attempt to get it packaged for Fedora 17 - would one of you like to lend a hand? I'm happy to put you in touch with Marek, if you're not already - http://fedoraproject.org/wiki/JBossAS7 ovirt-engine is dependent on jboss, not just the webadmin or user portal (and we are discussing it with Marek). the first release won't be a fedora feature, just an external repo and it comes with a ovirt-engine-jbossas-5 rpm which is based on the jboss zipped version > > There's also the matter of not being depending on a specific version of JBoss, or even on JBoss at all. What about those who want to run Tomcat? Not being a java guy, I haven't the faintest idea how difficult it is to port apps to other servlet containers or app servers. > > -JM > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From bazulay at redhat.com Fri Nov 18 11:25:04 2011 From: bazulay at redhat.com (Barak Azulay) Date: Fri, 18 Nov 2011 13:25:04 +0200 Subject: [Qemu-devel] wiki summary In-Reply-To: <4EC5674B.30807@linux.vnet.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <201111171834.48040.bazulay@redhat.com> <4EC5674B.30807@linux.vnet.ibm.com> Message-ID: <201111181325.05439.bazulay@redhat.com> On Thursday 17 November 2011 21:58:03 Michael Roth wrote: > On 11/17/2011 10:34 AM, Barak Azulay wrote: > > On Thursday 17 November 2011 02:48:50 Michael Roth wrote: > >> I've tried to summarize the pros/cons, points, and proposals outlined in > >> this thread at the following wiki: > >> > >> http://www.ovirt.org/wiki/Guest_agent_proposals > >> > >> Please feel free to add/edit as needed. If you don't have an account on > >> ovirt.org let me know. > > > > Thanks Michael, it's a good start. > > > > > > A few questions about the qemu-ga's requirements: > > > > #1 > > > > - same repo ? why is this a requirement ? > > Or git submodule. Main reasons are that integration with QMP requires > that qemu be able to generate marshaling code from a guest agent schema > definition of commands/parameters, and that qemu needs to be able to > consume guest agent extensions internally. A few examples that came up > in this thread were opening new virtio-serial channel via agent calls, > and registering device callbacks/driving state machine changes for guest > agent events. Since we'd like to pursue a push-deployment model where > QEMU can deploy a specific, compatible version of the agent to a > bootstrapped guest (qemu-ga pre-installed via guest distro or ISO > package), having code changes in-sync with repo would be necessary. > Does it mean that every time we need to add a new feature to ovirt (which may require new API call), we'll have to wait for the appropriate qemu & libvirt release? > VMware has a similar model for handling guest tools upgrades, where the > hypervisor pushes upgrades based on host hypervisor level: > > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=di > splayKC&externalId=1008907 > This is a very good feature (which we have discussed in past many times), but I don't think it has something to do with guest-agent being in the same repo with qemu. > The alternative is strict APIs with backward-compatibility with > down-level agents, which complicates things tremendously on the QEMU > side, and pretty much everywhere in the stack. Just keeping libvirt in > sync with QMP has proven difficult and that's just on the host, with a > common distro and fairly close development communities. Extending this > kind of synchronization out to multiple guest distros with varying > levels of guest agents makes this far harder. > This is exactly my concern about having to pass everything through libvirt & qemu. I'm sure we will not catch all the things we need right from the start, there for we'll have to delay features till they are in qemu & libvirt. > > - distributable via ISO - can you elaborate? > > We'd eventually like to have an analogue to virtualbox/vmware guest > tools, which ship with the hypervisor and can be deployed in a guest via > an ISO made available in the guest as a cdrom when push-deployment isn't > an option (guest doesnt already have some version of an agent with > upgrade support installed). This is to avoid limiting support to > specific distros due to lack of available packages in guest repo. > Actually we have this solution already active in ovirt for windows guests, for linux guests we had assumed that every distro has it's own updates mechanism (network dependant), but adding support for various linux distros is very easy. I'll be more than happt to elaborate if needed. Again I don't think it's a requirement from the guest agent (or qemu) but from a much higher level management system > > - upgradeable via hypervisor push - by the title it sounds like it > > belongs > > > > to deployment, which sounds to me like it belongs to a higher > > management level > > We'd like ability to push to be available all throughout the stack. If > device X has a callback for event Y, which is only available via version > Z of the guest agent, we're now reliant on layers far higher up the > stack to enable low-level functionality that's beneficial at all levels. > > > #3 a few questions come up when I read it: > > - some may consider those primitives as a security breach > > s/some/virtually everyone/ :) Yes, this is a problem that'll need to be > addressed. But at the end of the day, QEMU/host *must* be trusted if > there's so be any pretense of security, since we have access to > everything at the end of the day. Additionally, VMware has been > successfully leveraging guest file access, automatic upgrades of guest > tools, and exec functionality for quite some time now. > > That's not to say we don't need to examine the implications closely, but > there's precedence. 1 - We have had such functionality in the ovirt-guest-agent and removed it becuase of security (BTW it's very easy to add it back) 2 - it's not about trusting qemu, it's about trusting who ever use such an API, meaning: that eventually there is a management system with lots of users and permissions that allow to use this api, so the exposure is much much bigger than just to qemu itself. keep in mind that I qemu only supply the APIs, i find it hard to believe that it will acually do some upgrade logic on it's own. > > > - I understand the motivation of being able to do everything on the > > guest > > > > (exe) but we need to keep in mind it's various guest OSs, and it > > means that there should be a script for every OS type. to me the > > option of having a well defined interface is much more appealing > > Agreed, and we should strive for that. But rarely is an interface > designed so well that it never needs to change, and however well-defined > it may be, it will grow with time and that growth entails deploying new > guest code. Hence my concern above, about having to pass every new API through qemu & libvirt will slow down features drastically. > > > Thanks > > Barak > > > >> Thanks! From agl at us.ibm.com Fri Nov 18 14:10:09 2011 From: agl at us.ibm.com (Adam Litke) Date: Fri, 18 Nov 2011 08:10:09 -0600 Subject: [Qemu-devel] wiki summary In-Reply-To: <201111181325.05439.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <201111171834.48040.bazulay@redhat.com> <4EC5674B.30807@linux.vnet.ibm.com> <201111181325.05439.bazulay@redhat.com> Message-ID: <20111118141009.GQ2726@us.ibm.com> On Fri, Nov 18, 2011 at 01:25:04PM +0200, Barak Azulay wrote: > On Thursday 17 November 2011 21:58:03 Michael Roth wrote: > > On 11/17/2011 10:34 AM, Barak Azulay wrote: > > > On Thursday 17 November 2011 02:48:50 Michael Roth wrote: > > >> I've tried to summarize the pros/cons, points, and proposals outlined in > > >> this thread at the following wiki: > > >> > > >> http://www.ovirt.org/wiki/Guest_agent_proposals > > >> > > >> Please feel free to add/edit as needed. If you don't have an account on > > >> ovirt.org let me know. > > > > > > Thanks Michael, it's a good start. > > > > > > > > > A few questions about the qemu-ga's requirements: > > > > > > #1 > > > > > > - same repo ? why is this a requirement ? > > > > Or git submodule. Main reasons are that integration with QMP requires > > that qemu be able to generate marshaling code from a guest agent schema > > definition of commands/parameters, and that qemu needs to be able to > > consume guest agent extensions internally. A few examples that came up > > in this thread were opening new virtio-serial channel via agent calls, > > and registering device callbacks/driving state machine changes for guest > > agent events. Since we'd like to pursue a push-deployment model where > > QEMU can deploy a specific, compatible version of the agent to a > > bootstrapped guest (qemu-ga pre-installed via guest distro or ISO > > package), having code changes in-sync with repo would be necessary. > > > > Does it mean that every time we need to add a new feature to ovirt (which may > require new API call), we'll have to wait for the appropriate qemu & libvirt > release? No, since qemu-ga is built around primitives you will be able to build nearly anything you want on top of the basic read/write/exec (or plugin) architecture. > > VMware has a similar model for handling guest tools upgrades, where the > > hypervisor pushes upgrades based on host hypervisor level: > > > > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=di > > splayKC&externalId=1008907 > > > > > This is a very good feature (which we have discussed in past many times), but > I don't think it has something to do with guest-agent being in the same repo > with qemu. > > > > The alternative is strict APIs with backward-compatibility with > > down-level agents, which complicates things tremendously on the QEMU > > side, and pretty much everywhere in the stack. Just keeping libvirt in > > sync with QMP has proven difficult and that's just on the host, with a > > common distro and fairly close development communities. Extending this > > kind of synchronization out to multiple guest distros with varying > > levels of guest agents makes this far harder. > > > > > This is exactly my concern about having to pass everything through libvirt & > qemu. > I'm sure we will not catch all the things we need right from the start, there > for we'll have to delay features till they are in qemu & libvirt. See above. This underscores the need for an agent that implements a low level API. > > > - distributable via ISO - can you elaborate? > > > > We'd eventually like to have an analogue to virtualbox/vmware guest > > tools, which ship with the hypervisor and can be deployed in a guest via > > an ISO made available in the guest as a cdrom when push-deployment isn't > > an option (guest doesnt already have some version of an agent with > > upgrade support installed). This is to avoid limiting support to > > specific distros due to lack of available packages in guest repo. > > > > > Actually we have this solution already active in ovirt for windows guests, for > linux guests we had assumed that every distro has it's own updates mechanism > (network dependant), but adding support for various linux distros is very > easy. I'll be more than happt to elaborate if needed. > > Again I don't think it's a requirement from the guest agent (or qemu) but from > a much higher level management system I disagree. Many people use KVM today outside the realm of a "much higher level management system". I run VMs on my laptop and in this environment we still need a way to deploy guest tools easily. I would like to use a mechanism that is the same for all of my guests. This means using the tried and tested model employed by other prominent, easy to use hypervisors -- a host-supplied guest tools ISO. > > > - upgradeable via hypervisor push - by the title it sounds like it > > > belongs > > > > > > to deployment, which sounds to me like it belongs to a higher > > > management level > > > > We'd like ability to push to be available all throughout the stack. If > > device X has a callback for event Y, which is only available via version > > Z of the guest agent, we're now reliant on layers far higher up the > > stack to enable low-level functionality that's beneficial at all levels. > > > > > #3 a few questions come up when I read it: > > > - some may consider those primitives as a security breach > > > > s/some/virtually everyone/ :) Yes, this is a problem that'll need to be > > addressed. But at the end of the day, QEMU/host *must* be trusted if > > there's so be any pretense of security, since we have access to > > everything at the end of the day. Additionally, VMware has been > > successfully leveraging guest file access, automatic upgrades of guest > > tools, and exec functionality for quite some time now. > > > > That's not to say we don't need to examine the implications closely, but > > there's precedence. > > > 1 - We have had such functionality in the ovirt-guest-agent and removed it > becuase of security (BTW it's very easy to add it back) > > 2 - it's not about trusting qemu, it's about trusting who ever use such an > API, meaning: that eventually there is a management system with lots of users > and permissions that allow to use this api, so the exposure is much much > bigger than just to qemu itself. keep in mind that I qemu only supply the > APIs, i find it hard to believe that it will acually do some upgrade logic on > it's own. The security problems are addressable (via auditing, guest and host side controls, etc). And as far as upgrade goes, making the agent a part of qemu will actually help. The monitor will have two APIs: one to check if a guest agent as installed and query capabilities/version (already present), and another to present a guest-tools ISO to the guest for installation/upgrade. With these two host-side APIs in place, it will be possible to provide a trivial guest-tools-upgrader that could be run when the guest tools iso is updated on the host. > > > > > > - I understand the motivation of being able to do everything on the > > > guest > > > > > > (exe) but we need to keep in mind it's various guest OSs, and it > > > means that there should be a script for every OS type. to me the > > > option of having a well defined interface is much more appealing > > > > Agreed, and we should strive for that. But rarely is an interface > > designed so well that it never needs to change, and however well-defined > > it may be, it will grow with time and that growth entails deploying new > > guest code. > > Hence my concern above, about having to pass every new API through qemu & > libvirt will slow down features drastically. I am sure your sentiment is shared by non-oVirt users who would now need to implement low-level guest agent features in an unrelated software stack. I think we need a separation of responsibility. Low-level general purpose agent functionality should be built into a hypervisor (qemu) API which should be consumable by higher level management systems in a natural way. -- Adam Litke IBM Linux Technology Center From mdroth at linux.vnet.ibm.com Fri Nov 18 14:21:29 2011 From: mdroth at linux.vnet.ibm.com (Michael Roth) Date: Fri, 18 Nov 2011 08:21:29 -0600 Subject: [Qemu-devel] wiki summary In-Reply-To: <201111181325.05439.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <201111171834.48040.bazulay@redhat.com> <4EC5674B.30807@linux.vnet.ibm.com> <201111181325.05439.bazulay@redhat.com> Message-ID: <4EC669E9.8070402@linux.vnet.ibm.com> On 11/18/2011 05:25 AM, Barak Azulay wrote: > On Thursday 17 November 2011 21:58:03 Michael Roth wrote: >> On 11/17/2011 10:34 AM, Barak Azulay wrote: >>> On Thursday 17 November 2011 02:48:50 Michael Roth wrote: >>>> I've tried to summarize the pros/cons, points, and proposals outlined in >>>> this thread at the following wiki: >>>> >>>> http://www.ovirt.org/wiki/Guest_agent_proposals >>>> >>>> Please feel free to add/edit as needed. If you don't have an account on >>>> ovirt.org let me know. >>> >>> Thanks Michael, it's a good start. >>> >>> >>> A few questions about the qemu-ga's requirements: >>> >>> #1 >>> >>> - same repo ? why is this a requirement ? >> >> Or git submodule. Main reasons are that integration with QMP requires >> that qemu be able to generate marshaling code from a guest agent schema >> definition of commands/parameters, and that qemu needs to be able to >> consume guest agent extensions internally. A few examples that came up >> in this thread were opening new virtio-serial channel via agent calls, >> and registering device callbacks/driving state machine changes for guest >> agent events. Since we'd like to pursue a push-deployment model where >> QEMU can deploy a specific, compatible version of the agent to a >> bootstrapped guest (qemu-ga pre-installed via guest distro or ISO >> package), having code changes in-sync with repo would be necessary. >> > > Does it mean that every time we need to add a new feature to ovirt (which may > require new API call), we'll have to wait for the appropriate qemu& libvirt > release? Exactly the opposite if you deploy ovirt functionality using Adam's proposal (use qemu-ga to deploy/exec ovirt scripts in guest via ovirt management layer, we could potentially provide a higher-level mechanism to simplify the process: for instance: `guest-script-exec script_name ` for instance, that'll push the latest version of the script to some standard location in the guest if it's not present in the guest, or the md5sum doesn't match the host version, then exec it. I think, since you're already using JSON, we could embed the JSON requests/responses via using a special case `guest-script-exec-json` call so the parsing is virtually transparent on your end. libvirt/qemu/qmp would only need to be aware of the primitives, you can extend your guest agent functionality simply by updating the scripts available to your management tooling on the host. > > >> VMware has a similar model for handling guest tools upgrades, where the >> hypervisor pushes upgrades based on host hypervisor level: >> >> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=di >> splayKC&externalId=1008907 >> > > > This is a very good feature (which we have discussed in past many times), but > I don't think it has something to do with guest-agent being in the same repo > with qemu. To be beneficial from a development/integration perspective, the host level would need to be kept in sync with the guest agent level, same repo or git submodule is the best way to handle this. > > >> The alternative is strict APIs with backward-compatibility with >> down-level agents, which complicates things tremendously on the QEMU >> side, and pretty much everywhere in the stack. Just keeping libvirt in >> sync with QMP has proven difficult and that's just on the host, with a >> common distro and fairly close development communities. Extending this >> kind of synchronization out to multiple guest distros with varying >> levels of guest agents makes this far harder. >> > > > This is exactly my concern about having to pass everything through libvirt& > qemu. > I'm sure we will not catch all the things we need right from the start, there > for we'll have to delay features till they are in qemu& libvirt. Not with Adam's proposal. It's something we're specifically trying to avoid as well and was a core design goal for qemu-ga: make QMP/libvirt aware of low-level primitives, let higher level tooling build higher-level interfaces/functionality from those. QMP/libvirt do not need to be made aware of your changes, though with some work, we could potentionally utilize your agent functionality at those levels as well (something like the guest-script-exec-json I mentioned above) > > >>> - distributable via ISO - can you elaborate? >> >> We'd eventually like to have an analogue to virtualbox/vmware guest >> tools, which ship with the hypervisor and can be deployed in a guest via >> an ISO made available in the guest as a cdrom when push-deployment isn't >> an option (guest doesnt already have some version of an agent with >> upgrade support installed). This is to avoid limiting support to >> specific distros due to lack of available packages in guest repo. >> > > > Actually we have this solution already active in ovirt for windows guests, for > linux guests we had assumed that every distro has it's own updates mechanism > (network dependant), but adding support for various linux distros is very > easy. I'll be more than happt to elaborate if needed. > > Again I don't think it's a requirement from the guest agent (or qemu) but from > a much higher level management system We need to consider low-level requirements: virtio drivers for instance, and guest extensions that integrate closely with QEMU itself. QEMU needs to be useable in and of itself: relying on high-level management tools to package/deploy low-level drivers does nothing for someone who's trying to fire up QEMU via command-line and poke at it via QMP. Ideally we'd have a qemu-guest-tools ISO that a stack can be built on from QEMU on up rather than an ovirt-guest-tools ISO that leaves QEMU crippled when used in isolation. We've shown how a qemu-guest-tools ISO can be used to build arbitrarily complex functionality higher up the stack while still servicing low-level, QEMU-internal functionality. That's not to say it couldn't be done with ovirt-guest-tools, but to me it seems like a much more natural/extensible approach than having QEMU internally reliant on something way higher up the stack. > > >>> - upgradeable via hypervisor push - by the title it sounds like it >>> belongs >>> >>> to deployment, which sounds to me like it belongs to a higher >>> management level >> >> We'd like ability to push to be available all throughout the stack. If >> device X has a callback for event Y, which is only available via version >> Z of the guest agent, we're now reliant on layers far higher up the >> stack to enable low-level functionality that's beneficial at all levels. >> >>> #3 a few questions come up when I read it: >>> - some may consider those primitives as a security breach >> >> s/some/virtually everyone/ :) Yes, this is a problem that'll need to be >> addressed. But at the end of the day, QEMU/host *must* be trusted if >> there's so be any pretense of security, since we have access to >> everything at the end of the day. Additionally, VMware has been >> successfully leveraging guest file access, automatic upgrades of guest >> tools, and exec functionality for quite some time now. >> >> That's not to say we don't need to examine the implications closely, but >> there's precedence. > > > 1 - We have had such functionality in the ovirt-guest-agent and removed it > becuase of security (BTW it's very easy to add it back) > > 2 - it's not about trusting qemu, it's about trusting who ever use such an > API, meaning: that eventually there is a management system with lots of users > and permissions that allow to use this api, so the exposure is much much > bigger than just to qemu itself. keep in mind that I qemu only supply the Agreed. But QMP access in general should be considered root access to a guest: we can reboot, poke at registers, memory, etc, so guest-exec support is not a particularly special case in that regard. What matters is how you expose that functionality higher up the stack. And that's what will need to be assessed. > APIs, i find it hard to believe that it will acually do some upgrade logic on > it's own. A particular version of QEMU will know what particular version of the guest agent it was coded against (ideally, the version it shipped with). The only logic required beyond that it someone typing "guest-update-agent" at a QMP console, or perhaps some lower-level image authoring tool injecting the code via kickstart/libguestfs/etc. > >> >>> - I understand the motivation of being able to do everything on the >>> guest >>> >>> (exe) but we need to keep in mind it's various guest OSs, and it >>> means that there should be a script for every OS type. to me the >>> option of having a well defined interface is much more appealing >> >> Agreed, and we should strive for that. But rarely is an interface >> designed so well that it never needs to change, and however well-defined >> it may be, it will grow with time and that growth entails deploying new >> guest code. > > Hence my concern above, about having to pass every new API through qemu& > libvirt will slow down features drastically. We've shared the same concerns from day one: qemu-ga is actually a mechanism to avoid this: keep the default agent low-level and simple, build APIs on top of it that, above the libvirt/qemu-ga/QMP layers. If it's a common, generally-useful interface, pull it into qemu-ga proper, and expose it at the QMP layer. > > >> >>> Thanks >>> Barak >>> >>>> Thanks! > From yihherng at us.ibm.com Fri Nov 18 17:08:51 2011 From: yihherng at us.ibm.com (Yih-Herng Chuang) Date: Fri, 18 Nov 2011 10:08:51 -0700 Subject: AUTO: Yih-Herng Chuang is out of the office (returning 11/21/2011) Message-ID: I am out of the office until 11/21/2011. I will respond to emails when return back to office. If there is an emergency please contact Doug Jans for technical issues or my manager Carol Roth. Note: This is an automated response to your message "Arch Digest, Vol 1, Issue 15" sent on 11/18/2011 7:10:31. This is the only notification you will receive while this person is away. From bazulay at redhat.com Sun Nov 20 23:53:46 2011 From: bazulay at redhat.com (Barak Azulay) Date: Mon, 21 Nov 2011 01:53:46 +0200 Subject: Release Managers for oVirt In-Reply-To: <4EC407E6.3040707@redhat.com> References: <4EC407E6.3040707@redhat.com> Message-ID: <4EC9930A.8070702@redhat.com> On 11/16/2011 08:58 PM, Perry Myers wrote: > On the board call today, we discussed getting a release schedule nailed > down (in the works) and also the need to nominate someone to own the > release management. > > It was suggested that there be the following roles for oVirt release mgmt: > > * overall oVirt project release manager, responsible for coordinate > with all of the sub-projects to drive a unified release of the entire > stack for a given point release > > * per project release manager/owner that is responsible for putting out > updates for a given sub-project and will work closely with the global > release manager on overall releases > > If folks think that idea is sound, then the next step is to get > volunteers/nominations for who should be the release manager (at least > for the time being, this position might shift from person to person over > time) > > The second thing is for each sub-project to nominate a point person for > sub-project release mgmt. We can then put on a wiki page who the > overall release manager is for a given release. The release managers > for each sub-project probably can be called out on the projects page on > ovirt.org > > Nominations/Volunteers? I would like to suggest nominateing Ofer Schreiber for the overall oVirt project release manager. He is familiar with all the subpackages/projects, and had proven himself as the main orchestrator during 3.0 rhevm builds. Thanks Barak Azulay > Perry > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From pmyers at redhat.com Sun Nov 20 21:26:40 2011 From: pmyers at redhat.com (Perry Myers) Date: Sun, 20 Nov 2011 16:26:40 -0500 Subject: Release Managers for oVirt In-Reply-To: <4EC9930A.8070702@redhat.com> References: <4EC407E6.3040707@redhat.com> <4EC9930A.8070702@redhat.com> Message-ID: <4EC97090.8050609@redhat.com> >> Nominations/Volunteers? > > > I would like to suggest nominateing Ofer Schreiber for the overall > oVirt project release manager. > > He is familiar with all the subpackages/projects, and had proven himself > as the main orchestrator during 3.0 rhevm builds. +1 from me assuming Ofer wants the job :) From ranglust at redhat.com Sun Nov 20 22:09:12 2011 From: ranglust at redhat.com (Ronen Angluster) Date: Sun, 20 Nov 2011 17:09:12 -0500 (EST) Subject: =?utf-8?B?UmU6IFJlbGVhc2UgTWFuYWdlcnMgZm9yIG9WaXJ0?= Message-ID: <788695567.235556.1321826952170.JavaMail.root@zmail16.collab.prod.int.phx2.redhat.com> -----Original Message----- From: Barak Azulay [bazulay at redhat.com] Received: Sunday, 20 Nov 2011, 22:53 To: Perry Myers [pmyers at redhat.com] CC: arch at ovirt.org Subject: Re: Release Managers for oVirt On 11/16/2011 08:58 PM, Perry Myers wrote: > On the board call today, we discussed getting a release schedule nailed > down (in the works) and also the need to nominate someone to own the > release management. > > It was suggested that there be the following roles for oVirt release mgmt: > > * overall oVirt project release manager, responsible for coordinate > with all of the sub-projects to drive a unified release of the entire > stack for a given point release > > * per project release manager/owner that is responsible for putting out > updates for a given sub-project and will work closely with the global > release manager on overall releases > > If folks think that idea is sound, then the next step is to get > volunteers/nominations for who should be the release manager (at least > for the time being, this position might shift from person to person over > time) > > The second thing is for each sub-project to nominate a point person for > sub-project release mgmt. We can then put on a wiki page who the > overall release manager is for a given release. The release managers > for each sub-project probably can be called out on the projects page on > ovirt.org > > Nominations/Volunteers? I would like to suggest nominateing Ofer Schreiber for the overall oVirt project release manager. He is familiar with all the subpackages/projects, and had proven himself as the main orchestrator during 3.0 rhevm builds. Thanks Barak Azulay +1 > Perry > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch _______________________________________________ Arch mailing list Arch at ovirt.org http://lists.ovirt.org/mailman/listinfo/arch From oschreib at redhat.com Mon Nov 21 06:17:39 2011 From: oschreib at redhat.com (Ofer Schreiber) Date: Mon, 21 Nov 2011 01:17:39 -0500 (EST) Subject: Release Managers for oVirt In-Reply-To: <4EC97090.8050609@redhat.com> References: <4EC407E6.3040707@redhat.com> <4EC9930A.8070702@redhat.com> <4EC97090.8050609@redhat.com> Message-ID: <023d01cca815$44763980$cd62ac80$@redhat.com> > >> Nominations/Volunteers? > > > > > > I would like to suggest nominateing Ofer Schreiber for the overall > > oVirt project release manager. > > > > He is familiar with all the subpackages/projects, and had proven himself > > as the main orchestrator during 3.0 rhevm builds. > > +1 from me assuming Ofer wants the job :) No worries, I would be happy to do the job. > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From dlaor at redhat.com Thu Nov 24 12:40:07 2011 From: dlaor at redhat.com (Dor Laor) Date: Thu, 24 Nov 2011 14:40:07 +0200 Subject: [Qemu-devel] wiki summary In-Reply-To: <4EC5674B.30807@linux.vnet.ibm.com> References: <201111151924.41357.bazulay@redhat.com> <4EC459F2.5010701@linux.vnet.ibm.com> <201111171834.48040.bazulay@redhat.com> <4EC5674B.30807@linux.vnet.ibm.com> Message-ID: <4ECE3B27.3020708@redhat.com> On 11/17/2011 09:58 PM, Michael Roth wrote: > On 11/17/2011 10:34 AM, Barak Azulay wrote: >> On Thursday 17 November 2011 02:48:50 Michael Roth wrote: >>> I've tried to summarize the pros/cons, points, and proposals outlined in >>> this thread at the following wiki: >>> >>> http://www.ovirt.org/wiki/Guest_agent_proposals >>> >>> Please feel free to add/edit as needed. If you don't have an account on >>> ovirt.org let me know. >>> >> >> Thanks Michael, it's a good start. >> >> >> A few questions about the qemu-ga's requirements: >> >> #1 >> - same repo ? why is this a requirement ? > > Or git submodule. Main reasons are that integration with QMP requires > that qemu be able to generate marshaling code from a guest agent schema > definition of commands/parameters, and that qemu needs to be able to > consume guest agent extensions internally. A few examples that came up > in this thread were opening new virtio-serial channel via agent calls, > and registering device callbacks/driving state machine changes for guest > agent events. Since we'd like to pursue a push-deployment model where > QEMU can deploy a specific, compatible version of the agent to a > bootstrapped guest (qemu-ga pre-installed via guest distro or ISO > package), having code changes in-sync with repo would be necessary. > > VMware has a similar model for handling guest tools upgrades, where the > hypervisor pushes upgrades based on host hypervisor level: > > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008907 > > The alternative is strict APIs with backward-compatibility with > down-level agents, which complicates things tremendously on the QEMU > side, and pretty much everywhere in the stack. Just keeping libvirt in > sync with QMP has proven difficult and that's just on the host, with a > common distro and fairly close development communities. Extending this > kind of synchronization out to multiple guest distros with varying > levels of guest agents makes this far harder. Using QMP is an advantage, I agree. However it can be used by another option - move the QMP schema out of qemu.git so all projects like libvirt, agents, vdsm, etc will be able to consume it directly. This way, adding a new (agent) command will immediately propagate to all of the stack instead of each component to implement it differently (while it would still be possible). That's what libguestfs scheme do today. In addition, it can answer one of (justified) Barak's concerns that it takes too long a time to have all the stack implements commands. Regardless, we'll want to support kind of path through verbs so we'll be able to execute remote WMI or Matahari commands. > >> - distributable via ISO - can you elaborate? > > We'd eventually like to have an analogue to virtualbox/vmware guest Hmmm, those type-f hypervisors do not have an OS to back them up. Unlike them, we can easily cover Linux and the ovirt project is supported by the major distributions. There is inherited advantage of relaying on the distribution for updating the agents, especially when it comes to security updates (will iso distribution be part of embargo infosec procedure now?). Imagine that bugs/flaws might be in one of your libraries. > tools, which ship with the hypervisor and can be deployed in a guest via > an ISO made available in the guest as a cdrom when push-deployment isn't > an option (guest doesnt already have some version of an agent with > upgrade support installed). This is to avoid limiting support to > specific distros due to lack of available packages in guest repo. Having the (mine) above said, the ovirt-agent get pushed today by iso for windows, so can we call this issue a draw and remove it out from the discussion? > >> - upgradeable via hypervisor push - by the title it sounds like it belongs >> to deployment, which sounds to me like it belongs to a higher management >> level > > We'd like ability to push to be available all throughout the stack. If > device X has a callback for event Y, which is only available via version > Z of the guest agent, we're now reliant on layers far higher up the > stack to enable low-level functionality that's beneficial at all levels. > >> >> #3 a few questions come up when I read it: >> - some may consider those primitives as a security breach > > s/some/virtually everyone/ :) Yes, this is a problem that'll need to be > addressed. But at the end of the day, QEMU/host *must* be trusted if > there's so be any pretense of security, since we have access to > everything at the end of the day. Additionally, VMware has been > successfully leveraging guest file access, automatic upgrades of guest > tools, and exec functionality for quite some time now. > > That's not to say we don't need to examine the implications closely, but > there's precedence. > >> - I understand the motivation of being able to do everything on the guest >> (exe) but we need to keep in mind it's various guest OSs, and it means >> that there should be a script for every OS type. to me the option of >> having a well defined interface is much more appealing > > Agreed, and we should strive for that. But rarely is an interface > designed so well that it never needs to change, and however well-defined > it may be, it will grow with time and that growth entails deploying new > guest code. > >> >> Thanks >> Barak >> >>> Thanks! >> >> > > _______________________________________________ > vdsm-devel mailing list > vdsm-devel at lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/vdsm-devel From berrange at redhat.com Fri Nov 25 10:07:27 2011 From: berrange at redhat.com (Daniel P. Berrange) Date: Fri, 25 Nov 2011 10:07:27 +0000 Subject: [Qemu-devel] wiki summary In-Reply-To: <4ECE3B27.3020708@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC459F2.5010701@linux.vnet.ibm.com> <201111171834.48040.bazulay@redhat.com> <4EC5674B.30807@linux.vnet.ibm.com> <4ECE3B27.3020708@redhat.com> Message-ID: <20111125100727.GB23652@redhat.com> On Thu, Nov 24, 2011 at 02:40:07PM +0200, Dor Laor wrote: > On 11/17/2011 09:58 PM, Michael Roth wrote: > >On 11/17/2011 10:34 AM, Barak Azulay wrote: > >>On Thursday 17 November 2011 02:48:50 Michael Roth wrote: > >>>I've tried to summarize the pros/cons, points, and proposals outlined in > >>>this thread at the following wiki: > >>> > >>>http://www.ovirt.org/wiki/Guest_agent_proposals > >>> > >>>Please feel free to add/edit as needed. If you don't have an account on > >>>ovirt.org let me know. > >>> > >> > >>Thanks Michael, it's a good start. > >> > >> > >>A few questions about the qemu-ga's requirements: > >> > >>#1 > >> - same repo ? why is this a requirement ? > > > >Or git submodule. Main reasons are that integration with QMP requires > >that qemu be able to generate marshaling code from a guest agent schema > >definition of commands/parameters, and that qemu needs to be able to > >consume guest agent extensions internally. A few examples that came up > >in this thread were opening new virtio-serial channel via agent calls, > >and registering device callbacks/driving state machine changes for guest > >agent events. Since we'd like to pursue a push-deployment model where > >QEMU can deploy a specific, compatible version of the agent to a > >bootstrapped guest (qemu-ga pre-installed via guest distro or ISO > >package), having code changes in-sync with repo would be necessary. > > > >VMware has a similar model for handling guest tools upgrades, where the > >hypervisor pushes upgrades based on host hypervisor level: > > > >http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008907 > > > >The alternative is strict APIs with backward-compatibility with > >down-level agents, which complicates things tremendously on the QEMU > >side, and pretty much everywhere in the stack. Just keeping libvirt in > >sync with QMP has proven difficult and that's just on the host, with a > >common distro and fairly close development communities. Extending this > >kind of synchronization out to multiple guest distros with varying > >levels of guest agents makes this far harder. > > Using QMP is an advantage, I agree. > However it can be used by another option - move the QMP schema out > of qemu.git so all projects like libvirt, agents, vdsm, etc will be > able to consume it directly. > > This way, adding a new (agent) command will immediately propagate to > all of the stack instead of each component to implement it > differently (while it would still be possible). > That's what libguestfs scheme do today. That kind of idea doesn't really help much in practice, because it is presuming that the API you want to expose is identical at every level in the stack. If that were really the case, then you wouldn't really have different levels in the stack in the first place. An API exposed at the libvirt level may well map to multiple API calls at the QEMU level, and and API call at the VDSM level may map to multiple API calls at the libvirt level, etc. libguestfs is doing something different here - they have one API and they are exposed that API, at the same level of granularity in other languages. That is very different from stacking APIs with different levels of abstraction. When you're stacking APIs you cannot avoid the fact that you need to create a design that is suitable for the level of abstraction you need at each layer. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| From bazulay at redhat.com Fri Nov 25 19:33:51 2011 From: bazulay at redhat.com (Barak Azulay) Date: Fri, 25 Nov 2011 21:33:51 +0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <20111117161442.GF16420@redhat.com> References: <20111116202451.GI2726@us.ibm.com> <78db1bdf-6440-4808-9658-19fe0e7043ea@zmail07.collab.prod.int.phx2.redhat.com> <20111117155833.GN2726@us.ibm.com> <20111117161442.GF16420@redhat.com> Message-ID: <4ECFED9F.7@redhat.com> On 11/17/2011 06:14 PM, Daniel P. Berrange wrote: > On Thu, Nov 17, 2011 at 09:58:33AM -0600, Adam Litke wrote: >> On Thu, Nov 17, 2011 at 03:46:37AM -0500, Ayal Baron wrote: >>> >>> >>> ----- Original Message ----- >>>> I have been following this thread pretty closely and the one sentence >>>> summary of the current argument is: ovirt-guest-agent is already featureful >>>> and tested, so let's drop qemu-ga and have everyone adopt ovirt-guest-agent. >>> >>> What we're suggesting is let's drop *one* of the two agents (obviously it >>> would be easier for us to drop qemu-ga, but we'd rather reach consensus and >>> unite behind one agent regardless of which agent it is). >>> >>>> Unfortunately, this track strays completely away from the stated goal of >>>> convergence. I have at least two examples of why the greater KVM community >>>> can never adopt ovirt-guest-agent as-is. To address this, I would like to >>>> counter with an example on how qemu-ga can enable the deployment of >>>> ovirt-guest-agent features and satisfy the needs of the whole community at >>>> the same time. >>>> >>>> 1) Scope: The ovirt-guest-agent contains functionality that is incredibly >>>> useful within the context of oVirt. Single Sign-on is very handy but KVM >>>> users outside the scope of oVirt will not want this extra complexity in >>>> their agent. For simplicity they will probably just write something small >>>> that does what they need (and we have failed to provide a ubiquitous KVM >>>> agent). >>> >>> I totally agree, but that could easily be resolved using the plugin >>> architecture suggested before. >>> >>>> >>>> 1) Deployment complexity: The more complex the guest agent is, the more >>>> often it will need to be updated (bug/security fixes, distro compatibility, >>>> new features). Rolling out guest agent updates does not scale well in large >>>> environments (especially when the guest and host administrators are not the >>>> same person). >>> >>> Using plugins, you just deploy the ones you need, keeping the attack surface / >>> #bugs / need to update lower >> >> In order for any KVM guest agent to become ubiquitous, I think the code _must_ live >> in the qemu repository. This includes the base infrastructure and a core set of >> plugins to provide the current set of qemu-ga APIs. This way, both endpoints >> (host/guest) can evolve together. How easy would it be to extract this basic >> infrastructure from the ovirt-guest-agent? Is the qemu project opposed to a >> Python agent? > > IMHO Python would be a really bad choice for the agent. An agent wants to be > maximally portable to any guest OS, regardless of its vintage. The changes > between each python release, even within the 2.x stream, let alone between > 2.x and 3.x would cause us endless compatibility problems upon deployment. > And while python is common on Linux, we don't really want to get into the > business of installing the python runtime on Windows or other OS, simply to > run an agent. And still the ovirt-guest-agent: - is written in python - Supports many guest OSs (f15, f16, RHEL6.X RHEL5.X, winXP, W7 (32&64), 2k3 (32/64) 2k8 (32/64/R2)) - Deployed in binary format on win (py2exe ... no need to install python and no compatibility problems) > > Regards, > Daniel From dlaor at redhat.com Sun Nov 27 12:19:29 2011 From: dlaor at redhat.com (Dor Laor) Date: Sun, 27 Nov 2011 14:19:29 +0200 Subject: [Qemu-devel] wiki summary In-Reply-To: <20111125100727.GB23652@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC459F2.5010701@linux.vnet.ibm.com> <201111171834.48040.bazulay@redhat.com> <4EC5674B.30807@linux.vnet.ibm.com> <4ECE3B27.3020708@redhat.com> <20111125100727.GB23652@redhat.com> Message-ID: <4ED22AD1.3030205@redhat.com> On 11/25/2011 12:07 PM, Daniel P. Berrange wrote: > On Thu, Nov 24, 2011 at 02:40:07PM +0200, Dor Laor wrote: >> On 11/17/2011 09:58 PM, Michael Roth wrote: >>> On 11/17/2011 10:34 AM, Barak Azulay wrote: >>>> On Thursday 17 November 2011 02:48:50 Michael Roth wrote: >>>>> I've tried to summarize the pros/cons, points, and proposals outlined in >>>>> this thread at the following wiki: >>>>> >>>>> http://www.ovirt.org/wiki/Guest_agent_proposals >>>>> >>>>> Please feel free to add/edit as needed. If you don't have an account on >>>>> ovirt.org let me know. >>>>> >>>> >>>> Thanks Michael, it's a good start. >>>> >>>> >>>> A few questions about the qemu-ga's requirements: >>>> >>>> #1 >>>> - same repo ? why is this a requirement ? >>> >>> Or git submodule. Main reasons are that integration with QMP requires >>> that qemu be able to generate marshaling code from a guest agent schema >>> definition of commands/parameters, and that qemu needs to be able to >>> consume guest agent extensions internally. A few examples that came up >>> in this thread were opening new virtio-serial channel via agent calls, >>> and registering device callbacks/driving state machine changes for guest >>> agent events. Since we'd like to pursue a push-deployment model where >>> QEMU can deploy a specific, compatible version of the agent to a >>> bootstrapped guest (qemu-ga pre-installed via guest distro or ISO >>> package), having code changes in-sync with repo would be necessary. >>> >>> VMware has a similar model for handling guest tools upgrades, where the >>> hypervisor pushes upgrades based on host hypervisor level: >>> >>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008907 >>> >>> The alternative is strict APIs with backward-compatibility with >>> down-level agents, which complicates things tremendously on the QEMU >>> side, and pretty much everywhere in the stack. Just keeping libvirt in >>> sync with QMP has proven difficult and that's just on the host, with a >>> common distro and fairly close development communities. Extending this >>> kind of synchronization out to multiple guest distros with varying >>> levels of guest agents makes this far harder. >> >> Using QMP is an advantage, I agree. >> However it can be used by another option - move the QMP schema out >> of qemu.git so all projects like libvirt, agents, vdsm, etc will be >> able to consume it directly. >> >> This way, adding a new (agent) command will immediately propagate to >> all of the stack instead of each component to implement it >> differently (while it would still be possible). >> That's what libguestfs scheme do today. > > That kind of idea doesn't really help much in practice, because it is > presuming that the API you want to expose is identical at every level > in the stack. If that were really the case, then you wouldn't really > have different levels in the stack in the first place. An API exposed > at the libvirt level may well map to multiple API calls at the QEMU > level, and and API call at the VDSM level may map to multiple API > calls at the libvirt level, etc. > > libguestfs is doing something different here - they have one API and > they are exposed that API, at the same level of granularity in other > languages. That is very different from stacking APIs with different > levels of abstraction. When you're stacking APIs you cannot avoid the > fact that you need to create a design that is suitable for the level > of abstraction you need at each layer. You're right, but Barak's concern was that each new command will now need to be represented differently by each API in the stack. That's cause a significant slow down of the development cycle. While we should keep have a rich and different API in the various layers (agent, qmp, libvirt, vdsm,..), we can use a QMP pass through mode for others. Some times this is tricky, and one command generated by layer x might contradict another created by layer y. In the agent case, it can utilized in most cases since most commands are read-only. QMP/qAPI is a stable api and all layers in the stack can relay on it. Libvirt passes it through as well as you know :) Again, some commands like fs-freeze shouldn't be pass through-ed but plenty of others like mem info, fs content, app list, etc can be used. > > Regards, > Daniel From oschreib at redhat.com Tue Nov 29 16:11:17 2011 From: oschreib at redhat.com (Ofer Schreiber) Date: Tue, 29 Nov 2011 11:11:17 -0500 (EST) Subject: First community release In-Reply-To: <4ED50144.60406@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> Message-ID: <023301ccaeb1$85dcb460$91961d20$@redhat.com> > -----Original Message----- > From: board-bounces at ovirt.org [mailto:board-bounces at ovirt.org] On Behalf Of Livnat Peer > Sent: 29 November 2011 17:59 > To: cctrieloff at redhat.com > Cc: arch at ovirt.org; board at ovirt.org > Subject: Re: First community release > > On 11/07/2011 06:51 PM, Carl Trieloff wrote: > > > > At the workshop in the release / roadmap working session we agreed that > > around November 16th we see how all the distro's are making out in > > creating a release for oVirt and set the date for the first community > > release. > > > > Please have your comments / feedback for that time period, at which > > point I will start the release discussions. > > > > regards > > Carl. > > _______________________________________________ > > Board mailing list > > Board at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/board > > Hi All, > Not sure where we stand with the first release but I think we should > take into account the jboss version we use before releasing. > > We currently deploy on jboss as 5, but we started working on deployment > on jboss as 7. > Releasing on AS 5 and then moving to AS 7 might require some upgrade > path and it is a time consuming task to do that. I am not sure we MUST support upgrade path, specifically in such early version. > > Since we already started working on the deployment on jboss AS 7, I > think it might worth holding back first 'formal' release until it is > done. It will help us avoiding upgrade support and get all the benefits > of using as7 (startup time, memory/cpu foot-print etc.) - get better > reviews? > > Livnat > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board From bazulay at redhat.com Tue Nov 29 17:58:23 2011 From: bazulay at redhat.com (Barak Azulay) Date: Tue, 29 Nov 2011 19:58:23 +0200 Subject: First community release In-Reply-To: <4ED50144.60406@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> Message-ID: <4ED51D3F.7050203@redhat.com> On 11/29/2011 05:59 PM, Livnat Peer wrote: > On 11/07/2011 06:51 PM, Carl Trieloff wrote: >> >> At the workshop in the release / roadmap working session we agreed that >> around November 16th we see how all the distro's are making out in >> creating a release for oVirt and set the date for the first community >> release. >> >> Please have your comments / feedback for that time period, at which >> point I will start the release discussions. >> >> regards >> Carl. >> _______________________________________________ >> Board mailing list >> Board at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/board > > Hi All, > Not sure where we stand with the first release but I think we should > take into account the jboss version we use before releasing. > > We currently deploy on jboss as 5, but we started working on deployment > on jboss as 7. > Releasing on AS 5 and then moving to AS 7 might require some upgrade > path and it is a time consuming task to do that. > > Since we already started working on the deployment on jboss AS 7, I > think it might worth holding back first 'formal' release until it is > done. It will help us avoiding upgrade support and get all the benefits > of using as7 (startup time, memory/cpu foot-print etc.) - get better > reviews? > > Livnat I think we should aim for a fast first release, meaning ASAP on jboss 5 The questions are: 1 - what is a release? a tag in all the repos to be picked up by each distro ? 2 - do we have any criteria on what is good enough for a release ? or any point in time that everybody feels ready? Thanks Barak Azulay > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board From cctrieloff at redhat.com Tue Nov 29 18:14:48 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Tue, 29 Nov 2011 13:14:48 -0500 Subject: First community release In-Reply-To: <4ED51D3F.7050203@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> Message-ID: <4ED52118.6050706@redhat.com> On 11/29/2011 12:58 PM, Barak Azulay wrote: > > I think we should aim for a fast first release, meaning ASAP on jboss 5 > > The questions are: > > 1 - what is a release? a tag in all the repos to be picked up by each > distro ? > 2 - do we have any criteria on what is good enough for a release ? or > any point in time that everybody feels ready? I would say a release needs to be at least. 1. a tag in the repos 2. tars posted on the site for easy download, build and install, with instructions. 3. these tars have basically been tested. 4. optional binary builds for different distros linked from the release page 5. a set of any relevant release notes. Carl. From jchoate at redhat.com Tue Nov 29 18:32:27 2011 From: jchoate at redhat.com (Jon Choate) Date: Tue, 29 Nov 2011 13:32:27 -0500 Subject: First community release In-Reply-To: <4ED51D3F.7050203@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> Message-ID: <4ED5253B.7060600@redhat.com> On 11/29/2011 12:58 PM, Barak Azulay wrote: > On 11/29/2011 05:59 PM, Livnat Peer wrote: >> On 11/07/2011 06:51 PM, Carl Trieloff wrote: >>> >>> At the workshop in the release / roadmap working session we agreed that >>> around November 16th we see how all the distro's are making out in >>> creating a release for oVirt and set the date for the first community >>> release. >>> >>> Please have your comments / feedback for that time period, at which >>> point I will start the release discussions. >>> >>> regards >>> Carl. >>> _______________________________________________ >>> Board mailing list >>> Board at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/board >> >> Hi All, >> Not sure where we stand with the first release but I think we should >> take into account the jboss version we use before releasing. >> >> We currently deploy on jboss as 5, but we started working on deployment >> on jboss as 7. >> Releasing on AS 5 and then moving to AS 7 might require some upgrade >> path and it is a time consuming task to do that. >> >> Since we already started working on the deployment on jboss AS 7, I >> think it might worth holding back first 'formal' release until it is >> done. It will help us avoiding upgrade support and get all the benefits >> of using as7 (startup time, memory/cpu foot-print etc.) - get better >> reviews? >> >> Livnat > > > I think we should aim for a fast first release, meaning ASAP on jboss 5 > > The questions are: > > 1 - what is a release? a tag in all the repos to be picked up by each > distro ? > 2 - do we have any criteria on what is good enough for a release ? or > any point in time that everybody feels ready? > I like to approach the idea in reverse: 1 - What is preventing us from releasing today? 2- Are these really worth holding up the release? 3 - What can we do today so that the issues in #1 are gone tomorrow? I too am unsure what a release really means when people can get the code any time that they want and we could declare a release whenever and as often as we want. > Thanks > Barak Azulay > > >> _______________________________________________ >> Board mailing list >> Board at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/board > > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From acathrow at redhat.com Tue Nov 29 18:38:05 2011 From: acathrow at redhat.com (Andrew Cathrow) Date: Tue, 29 Nov 2011 13:38:05 -0500 (EST) Subject: First community release In-Reply-To: <4ED5253B.7060600@redhat.com> Message-ID: ----- Original Message ----- > From: "Jon Choate" > To: arch at ovirt.org > Sent: Tuesday, November 29, 2011 1:32:27 PM > Subject: Re: First community release > > On 11/29/2011 12:58 PM, Barak Azulay wrote: > > On 11/29/2011 05:59 PM, Livnat Peer wrote: > >> On 11/07/2011 06:51 PM, Carl Trieloff wrote: > >>> > >>> At the workshop in the release / roadmap working session we > >>> agreed that > >>> around November 16th we see how all the distro's are making out > >>> in > >>> creating a release for oVirt and set the date for the first > >>> community > >>> release. > >>> > >>> Please have your comments / feedback for that time period, at > >>> which > >>> point I will start the release discussions. > >>> > >>> regards > >>> Carl. > >>> _______________________________________________ > >>> Board mailing list > >>> Board at ovirt.org > >>> http://lists.ovirt.org/mailman/listinfo/board > >> > >> Hi All, > >> Not sure where we stand with the first release but I think we > >> should > >> take into account the jboss version we use before releasing. > >> > >> We currently deploy on jboss as 5, but we started working on > >> deployment > >> on jboss as 7. > >> Releasing on AS 5 and then moving to AS 7 might require some > >> upgrade > >> path and it is a time consuming task to do that. > >> > >> Since we already started working on the deployment on jboss AS 7, > >> I > >> think it might worth holding back first 'formal' release until it > >> is > >> done. It will help us avoiding upgrade support and get all the > >> benefits > >> of using as7 (startup time, memory/cpu foot-print etc.) - get > >> better > >> reviews? > >> > >> Livnat > > > > > > I think we should aim for a fast first release, meaning ASAP on > > jboss 5 > > > > The questions are: > > > > 1 - what is a release? a tag in all the repos to be picked up by > > each > > distro ? > > 2 - do we have any criteria on what is good enough for a release ? > > or > > any point in time that everybody feels ready? > > > > I like to approach the idea in reverse: > > 1 - What is preventing us from releasing today? > 2- Are these really worth holding up the release? > 3 - What can we do today so that the issues in #1 are gone tomorrow? > > I too am unsure what a release really means when people can get the > code > any time that they want and we could declare a release whenever and > as > often as we want. A release should be easily installable by a non-developer. So in the case of RPM packaged content for Fedora that means yum install ovirt. There's a lot of people who want to kick the tires but all the extra command line hackery that they have to do scares them off and they may not return. Once we can do # yum install ovirt # ovirt-setup then it's ready to go. > > > Thanks > > Barak Azulay > > > > > >> _______________________________________________ > >> Board mailing list > >> Board at ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/board > > > > _______________________________________________ > > Arch mailing list > > Arch at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/arch > > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch > From cctrieloff at redhat.com Tue Nov 29 18:37:19 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Tue, 29 Nov 2011 13:37:19 -0500 Subject: First community release In-Reply-To: <4ED5253B.7060600@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED5253B.7060600@redhat.com> Message-ID: <4ED5265F.1090201@redhat.com> On 11/29/2011 01:32 PM, Jon Choate wrote: > > I too am unsure what a release really means when people can get the > code any time that they want and we could declare a release whenever > and as often as we want. if the repos are always golden, i.e. never broken then you can release anytime. A release is basically a bundled up set we like, that we point users to. 90% of users don't like to pull from GIT, they just want a tar with instructions Carl. From iheim at redhat.com Tue Nov 29 19:02:31 2011 From: iheim at redhat.com (Itamar Heim) Date: Tue, 29 Nov 2011 14:02:31 -0500 (EST) Subject: First community release In-Reply-To: References: <4ED5253B.7060600@redhat.com> Message-ID: <9d1401ccaec9$12dec2b0$389c4810$@com> > -----Original Message----- > From: arch-bounces at ovirt.org [mailto:arch-bounces at ovirt.org] On Behalf Of Andrew Cathrow > Sent: Tuesday, November 29, 2011 20:38 PM > To: arch at ovirt.org > Subject: Re: First community release > > > > ----- Original Message ----- > > From: "Jon Choate" > > To: arch at ovirt.org > > Sent: Tuesday, November 29, 2011 1:32:27 PM > > Subject: Re: First community release > > > > On 11/29/2011 12:58 PM, Barak Azulay wrote: > > > On 11/29/2011 05:59 PM, Livnat Peer wrote: > > >> On 11/07/2011 06:51 PM, Carl Trieloff wrote: > > >>> > > >>> At the workshop in the release / roadmap working session we > > >>> agreed that > > >>> around November 16th we see how all the distro's are making out > > >>> in > > >>> creating a release for oVirt and set the date for the first > > >>> community > > >>> release. > > >>> > > >>> Please have your comments / feedback for that time period, at > > >>> which > > >>> point I will start the release discussions. > > >>> > > >>> regards > > >>> Carl. > > >>> _______________________________________________ > > >>> Board mailing list > > >>> Board at ovirt.org > > >>> http://lists.ovirt.org/mailman/listinfo/board > > >> > > >> Hi All, > > >> Not sure where we stand with the first release but I think we > > >> should > > >> take into account the jboss version we use before releasing. > > >> > > >> We currently deploy on jboss as 5, but we started working on > > >> deployment > > >> on jboss as 7. > > >> Releasing on AS 5 and then moving to AS 7 might require some > > >> upgrade > > >> path and it is a time consuming task to do that. > > >> > > >> Since we already started working on the deployment on jboss AS 7, > > >> I > > >> think it might worth holding back first 'formal' release until it > > >> is > > >> done. It will help us avoiding upgrade support and get all the > > >> benefits > > >> of using as7 (startup time, memory/cpu foot-print etc.) - get > > >> better > > >> reviews? > > >> > > >> Livnat > > > > > > > > > I think we should aim for a fast first release, meaning ASAP on > > > jboss 5 > > > > > > The questions are: > > > > > > 1 - what is a release? a tag in all the repos to be picked up by > > > each > > > distro ? > > > 2 - do we have any criteria on what is good enough for a release ? > > > or > > > any point in time that everybody feels ready? > > > > > > > I like to approach the idea in reverse: > > > > 1 - What is preventing us from releasing today? > > 2- Are these really worth holding up the release? > > 3 - What can we do today so that the issues in #1 are gone tomorrow? > > > > I too am unsure what a release really means when people can get the > > code > > any time that they want and we could declare a release whenever and > > as > > often as we want. > > A release should be easily installable by a non-developer. > So in the case of RPM packaged content for Fedora that means yum install ovirt. > There's a lot of people who want to kick the tires but all the extra command line hackery that they have to > do scares them off and they may not return. > > Once we can do > # yum install ovirt > # ovirt-setup > > then it's ready to go. I'd like to try and avoid the ovirt-setup step if possible, either via the rpm or a service doing them. Per Jon's question, there are two main config items: a. create/upgrade the db this can be done manually today, but need to remember the next release will need to be simple to upgrade to as well. otherwise a user doing yum update will get a non-working system until they will upgrade their db. b. configure the certificate this can be done by the rpm, but will be based on the hostname, which probably isn't always right. I think we can live with this if we let the user to easily change it afterwards. From pmyers at redhat.com Tue Nov 29 19:36:33 2011 From: pmyers at redhat.com (Perry Myers) Date: Tue, 29 Nov 2011 14:36:33 -0500 Subject: First community release In-Reply-To: <4ED5253B.7060600@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED5253B.7060600@redhat.com> Message-ID: <4ED53441.5050702@redhat.com> > 1 - What is preventing us from releasing today? oVirt Node still can't get vdsm to register yet. dougsland is working on this from the vdsm side, and mburns is assisting with providing him with Node builds as he needs them > 2 - Are these really worth holding up the release? Yes > 3 - What can we do today so that the issues in #1 are gone tomorrow? It's being worked on, and dougsland is making progress. Not sure what the ETA is though. Perry From dougsland at redhat.com Tue Nov 29 23:15:05 2011 From: dougsland at redhat.com (Douglas Landgraf) Date: Tue, 29 Nov 2011 18:15:05 -0500 Subject: First community release In-Reply-To: <4ED53441.5050702@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED5253B.7060600@redhat.com> <4ED53441.5050702@redhat.com> Message-ID: <4ED56779.9040605@redhat.com> Hi, On 11/29/2011 02:36 PM, Perry Myers wrote: >> 1 - What is preventing us from releasing today? > oVirt Node still can't get vdsm to register yet. dougsland is working > on this from the vdsm side, and mburns is assisting with providing him > with Node builds as he needs them > >> 2 - Are these really worth holding up the release? > Yes > >> 3 - What can we do today so that the issues in #1 are gone tomorrow? > It's being worked on, and dougsland is making progress. Not sure what > the ETA is though. > Currently, I can make vdsm register ovirt Node into ovirt GUI and still have some adjusts to execute and test, like add new storage. I will have more news in the end of this week. Here a few comments: - As Itamar said in the previous email, we will need to automate this steps: http://ovirt.org/wiki/Engine_Node_Integration#Engine_core_machine - vdsm will require this change into ovirt: https://bugzilla.redhat.com/show_bug.cgi?id=756136 - This BZ might be important to resolve before we launch: https://bugzilla.redhat.com/show_bug.cgi?id=755749 Thanks! -- Cheers Douglas From sgordon at redhat.com Tue Nov 29 20:26:25 2011 From: sgordon at redhat.com (Steve Gordon) Date: Tue, 29 Nov 2011 15:26:25 -0500 (EST) Subject: ovirt-release package In-Reply-To: Message-ID: <32cd1bd2-daa2-4bec-994a-fe0a8a9a53e9@zmail15.collab.prod.int.phx2.redhat.com> Hi all, In last week's sync up meeting we again discussed the possibility of creating an RPM to deliver the yum repository file (which I have had a crack at providing, but more on that in a moment). A few questions came up which I'd like to also put to the list with my own thoughts, hopefully others can provide theirs. ------ Q: Why provide an RPM for this? The user still has to download the RPM to install it! A: The benefit of providing the repository file wrapped in an RPM as I see it is that the user only has to manually download the RPM once, if (more likely when in my opinion) we need to change the repo file and/or add GPG keys etc they pick up the updated ovirt-release RPM up via yum update. If we don't provide it in an RPM then they will have to manually wget the updated repo file, and keys if applicable, and we would need a way to communicate to users that it was in fact necessary to do this. ------ Q: Would we need to get this RPM into Fedora? A: No, this repo file (and RPM) is for users wishing to track releases and nightly builds as they are put up by the oVirt Project. As I understand it the various oVirt components are not in any way tied to a particular distributions release schedule. As such it seems probable that there will be times when the release available at ovirt.org is more up to date than the version that is packaged in Fedora, obviously with nightly builds this will almost always be the case. ------ Q: What does the repo file actually look like? A: There are two variations at the moment, the one initially linked from the wiki: http://www.ovirt.org/releases/nightly/fedora/16/ovirt-engine.repo And one that Karsten I think had been working on as a result of the previous meeting: http://ovirt.org/wiki/Yum_repo_file The later is probably closer to what we need long term, in particular supporting both nightly and stable releases and using variables for the release version and architecture. At the moment however the repository doesn't support this level of granularity as we seem to have lumped x86_64 and noarch packages in the same directory? Is this something we intend to change long term? Where are we planning to place SRPMs? ------ Q: Where would this RPM exist in the repository? Where would the spec file live? A: This was the open question that was posed at the last meeting and probably the bit I need help with. I created a spec file based on the one used for fedora-release which I put on the wiki page for want of a better place: http://www.ovirt.org/wiki/Yum_repo_file#Spec_File To get the benefits I talked about earlier in this message the resultant RPM really needs to both: 1) Exist in the repository itself so that once it has installed itself users get updates to it. 2) Easily discoverable, probably linked from whatever 'Get oVirt!' style page we have under Fedora. The question really I guess is how does it get into the repository? ------ What do other people think? I realise that the above could easily be construed as bike shedding but I think that even though these are relatively simple matters it is important to flesh them out now particularly if we are looking to make a "release" (whatever that entails) in the near future. -Steve From ykaul at redhat.com Tue Nov 29 21:39:31 2011 From: ykaul at redhat.com (Yaniv Kaul) Date: Tue, 29 Nov 2011 23:39:31 +0200 Subject: First community release In-Reply-To: <9d1401ccaec9$12dec2b0$389c4810$@com> References: <4ED5253B.7060600@redhat.com> <9d1401ccaec9$12dec2b0$389c4810$@com> Message-ID: <4ED55113.7040906@redhat.com> On 11/29/2011 09:02 PM, Itamar Heim wrote: > >> -----Original Message----- >> From: arch-bounces at ovirt.org [mailto:arch-bounces at ovirt.org] On Behalf > Of Andrew Cathrow >> Sent: Tuesday, November 29, 2011 20:38 PM >> To: arch at ovirt.org >> Subject: Re: First community release >> >> >> >> ----- Original Message ----- >>> From: "Jon Choate" >>> To: arch at ovirt.org >>> Sent: Tuesday, November 29, 2011 1:32:27 PM >>> Subject: Re: First community release >>> >>> On 11/29/2011 12:58 PM, Barak Azulay wrote: >>>> On 11/29/2011 05:59 PM, Livnat Peer wrote: >>>>> On 11/07/2011 06:51 PM, Carl Trieloff wrote: >>>>>> At the workshop in the release / roadmap working session we >>>>>> agreed that >>>>>> around November 16th we see how all the distro's are making out >>>>>> in >>>>>> creating a release for oVirt and set the date for the first >>>>>> community >>>>>> release. >>>>>> >>>>>> Please have your comments / feedback for that time period, at >>>>>> which >>>>>> point I will start the release discussions. >>>>>> >>>>>> regards >>>>>> Carl. >>>>>> _______________________________________________ >>>>>> Board mailing list >>>>>> Board at ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/board >>>>> Hi All, >>>>> Not sure where we stand with the first release but I think we >>>>> should >>>>> take into account the jboss version we use before releasing. >>>>> >>>>> We currently deploy on jboss as 5, but we started working on >>>>> deployment >>>>> on jboss as 7. >>>>> Releasing on AS 5 and then moving to AS 7 might require some >>>>> upgrade >>>>> path and it is a time consuming task to do that. >>>>> >>>>> Since we already started working on the deployment on jboss AS 7, >>>>> I >>>>> think it might worth holding back first 'formal' release until it >>>>> is >>>>> done. It will help us avoiding upgrade support and get all the >>>>> benefits >>>>> of using as7 (startup time, memory/cpu foot-print etc.) - get >>>>> better >>>>> reviews? >>>>> >>>>> Livnat >>>> >>>> I think we should aim for a fast first release, meaning ASAP on >>>> jboss 5 >>>> >>>> The questions are: >>>> >>>> 1 - what is a release? a tag in all the repos to be picked up by >>>> each >>>> distro ? >>>> 2 - do we have any criteria on what is good enough for a release ? >>>> or >>>> any point in time that everybody feels ready? >>>> >>> I like to approach the idea in reverse: >>> >>> 1 - What is preventing us from releasing today? >>> 2- Are these really worth holding up the release? >>> 3 - What can we do today so that the issues in #1 are gone tomorrow? >>> >>> I too am unsure what a release really means when people can get the >>> code >>> any time that they want and we could declare a release whenever and >>> as >>> often as we want. >> A release should be easily installable by a non-developer. >> So in the case of RPM packaged content for Fedora that means yum install > ovirt. >> There's a lot of people who want to kick the tires but all the extra > command line hackery that they have to >> do scares them off and they may not return. >> >> Once we can do >> # yum install ovirt >> # ovirt-setup >> >> then it's ready to go. > I'd like to try and avoid the ovirt-setup step if possible, either via the > rpm or a service doing them. +1 The whole simplified 'yum install ovirt' instead of 'yum install ovirt ovirt-backend ovirt-fronted ...' made few complications wrt package dependencies. I think we should revise this. > Per Jon's question, there are two main config items: > a. create/upgrade the db > this can be done manually today, but need to remember the next release > will need to be simple to upgrade to as well. > otherwise a user doing yum update will get a non-working system until they > will upgrade their db. > > b. configure the certificate > this can be done by the rpm, but will be based on the hostname, which > probably isn't always right. I think we can live with this if we let the > user to easily change it afterwards. > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From ykaul at redhat.com Tue Nov 29 21:40:53 2011 From: ykaul at redhat.com (Yaniv Kaul) Date: Tue, 29 Nov 2011 23:40:53 +0200 Subject: First community release In-Reply-To: <4ED52118.6050706@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED52118.6050706@redhat.com> Message-ID: <4ED55165.2080202@redhat.com> On 11/29/2011 08:14 PM, Carl Trieloff wrote: > On 11/29/2011 12:58 PM, Barak Azulay wrote: >> I think we should aim for a fast first release, meaning ASAP on jboss 5 >> >> The questions are: >> >> 1 - what is a release? a tag in all the repos to be picked up by each >> distro ? >> 2 - do we have any criteria on what is good enough for a release ? or >> any point in time that everybody feels ready? > I would say a release needs to be at least. > > 1. a tag in the repos > 2. tars posted on the site for easy download, build and install, with > instructions. > 3. these tars have basically been tested. Not sure what the definition of 'tested' is, probably worth expanding it a bit. Y. > 4. optional binary builds for different distros linked from the release page > 5. a set of any relevant release notes. > > Carl. > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From ykaul at redhat.com Wed Nov 30 05:34:45 2011 From: ykaul at redhat.com (Yaniv Kaul) Date: Wed, 30 Nov 2011 07:34:45 +0200 Subject: First community release In-Reply-To: References: Message-ID: <4ED5C075.7080701@redhat.com> On 11/30/2011 02:03 AM, David Jorm wrote: >>> 3. these tars have basically been tested. >> Not sure what the definition of 'tested' is, probably worth expanding >> it >> a bit. >> Y. > I would think just smoke tested. > > David I would like to believe the code should pass smoke tests at all times. When it doesn't, it's a regression we should strive to fix asap. Y. From dfediuck at redhat.com Wed Nov 30 07:14:56 2011 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 30 Nov 2011 09:14:56 +0200 Subject: First community release In-Reply-To: <4ED55113.7040906@redhat.com> References: <4ED5253B.7060600@redhat.com> <9d1401ccaec9$12dec2b0$389c4810$@com> <4ED55113.7040906@redhat.com> Message-ID: <201111300914.57148.dfediuck@redhat.com> On Tuesday 29 November 2011 23:39:31 Yaniv Kaul wrote: > On 11/29/2011 09:02 PM, Itamar Heim wrote: > > > >> -----Original Message----- > >> From: arch-bounces at ovirt.org [mailto:arch-bounces at ovirt.org] On Behalf > > Of Andrew Cathrow > >> Sent: Tuesday, November 29, 2011 20:38 PM > >> To: arch at ovirt.org > >> Subject: Re: First community release > >> > >> > >> > >> ----- Original Message ----- > >>> From: "Jon Choate" > >>> To: arch at ovirt.org > >>> Sent: Tuesday, November 29, 2011 1:32:27 PM > >>> Subject: Re: First community release > >>> > >>> On 11/29/2011 12:58 PM, Barak Azulay wrote: > >>>> On 11/29/2011 05:59 PM, Livnat Peer wrote: > >>>>> On 11/07/2011 06:51 PM, Carl Trieloff wrote: > >>>>>> At the workshop in the release / roadmap working session we > >>>>>> agreed that > >>>>>> around November 16th we see how all the distro's are making out > >>>>>> in > >>>>>> creating a release for oVirt and set the date for the first > >>>>>> community > >>>>>> release. > >>>>>> > >>>>>> Please have your comments / feedback for that time period, at > >>>>>> which > >>>>>> point I will start the release discussions. > >>>>>> > >>>>>> regards > >>>>>> Carl. > >>>>>> _______________________________________________ > >>>>>> Board mailing list > >>>>>> Board at ovirt.org > >>>>>> http://lists.ovirt.org/mailman/listinfo/board > >>>>> Hi All, > >>>>> Not sure where we stand with the first release but I think we > >>>>> should > >>>>> take into account the jboss version we use before releasing. > >>>>> > >>>>> We currently deploy on jboss as 5, but we started working on > >>>>> deployment > >>>>> on jboss as 7. > >>>>> Releasing on AS 5 and then moving to AS 7 might require some > >>>>> upgrade > >>>>> path and it is a time consuming task to do that. > >>>>> > >>>>> Since we already started working on the deployment on jboss AS 7, > >>>>> I > >>>>> think it might worth holding back first 'formal' release until it > >>>>> is > >>>>> done. It will help us avoiding upgrade support and get all the > >>>>> benefits > >>>>> of using as7 (startup time, memory/cpu foot-print etc.) - get > >>>>> better > >>>>> reviews? > >>>>> > >>>>> Livnat > >>>> > >>>> I think we should aim for a fast first release, meaning ASAP on > >>>> jboss 5 > >>>> > >>>> The questions are: > >>>> > >>>> 1 - what is a release? a tag in all the repos to be picked up by > >>>> each > >>>> distro ? > >>>> 2 - do we have any criteria on what is good enough for a release ? > >>>> or > >>>> any point in time that everybody feels ready? > >>>> > >>> I like to approach the idea in reverse: > >>> > >>> 1 - What is preventing us from releasing today? > >>> 2- Are these really worth holding up the release? > >>> 3 - What can we do today so that the issues in #1 are gone tomorrow? > >>> > >>> I too am unsure what a release really means when people can get the > >>> code > >>> any time that they want and we could declare a release whenever and > >>> as > >>> often as we want. > >> A release should be easily installable by a non-developer. > >> So in the case of RPM packaged content for Fedora that means yum install > > ovirt. > >> There's a lot of people who want to kick the tires but all the extra > > command line hackery that they have to > >> do scares them off and they may not return. > >> > >> Once we can do > >> # yum install ovirt > >> # ovirt-setup > >> > >> then it's ready to go. > > I'd like to try and avoid the ovirt-setup step if possible, either via the > > rpm or a service doing them. > > +1 > > The whole simplified 'yum install ovirt' instead of 'yum install ovirt > ovirt-backend ovirt-fronted ...' made few complications wrt package > dependencies. > I think we should revise this. > > > Per Jon's question, there are two main config items: > > a. create/upgrade the db > > this can be done manually today, but need to remember the next release > > will need to be simple to upgrade to as well. > > otherwise a user doing yum update will get a non-working system until they > > will upgrade their db. > > > > b. configure the certificate > > this can be done by the rpm, but will be based on the hostname, which > > probably isn't always right. I think we can live with this if we let the > > user to easily change it afterwards. I agree w/ Itamar & Yaniv. The 'yum install ovirt' is distro-specific. It's important we have it, but we should leave enough room for other distro's to join in. In that sense, as Carl suggested we should focus on a *stable* release of- * Tagged source tarball allowing other distro's to fetch & build it. * Latest binary tarball. * A simplified RPM or make rpm script (Makefile based), along with DB and certificate creation scripts. This should broaden the availability of oVirt to other Linux segments, which find it hard to handle currently. As a hint take a look on the init-script and networking thread we had in users and vdsm-devel lists. -- /d "Who's General Failure and why's he reading my disk?" From oschreib at redhat.com Wed Nov 30 10:49:01 2011 From: oschreib at redhat.com (Ofer Schreiber) Date: Wed, 30 Nov 2011 05:49:01 -0500 (EST) Subject: First community release In-Reply-To: <4ED55165.2080202@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED52118.6050706@redhat.com> <4ED55165.2080202@redhat.com> Message-ID: <02a101ccaf4d$aca922d0$05fb6870$@redhat.com> Following this discussion, I've created http://www.ovirt.org/wiki/Releases/First_Release in order to track the schedule/gaps/requirements for the first oVirt release. Thoughts? Ofer. > -----Original Message----- > From: board-bounces at ovirt.org [mailto:board-bounces at ovirt.org] On Behalf Of Yaniv Kaul > Sent: 29 November 2011 23:41 > To: cctrieloff at redhat.com > Cc: arch at ovirt.org; board at ovirt.org > Subject: Re: First community release > > On 11/29/2011 08:14 PM, Carl Trieloff wrote: > > On 11/29/2011 12:58 PM, Barak Azulay wrote: > >> I think we should aim for a fast first release, meaning ASAP on jboss 5 > >> > >> The questions are: > >> > >> 1 - what is a release? a tag in all the repos to be picked up by each > >> distro ? > >> 2 - do we have any criteria on what is good enough for a release ? or > >> any point in time that everybody feels ready? > > I would say a release needs to be at least. > > > > 1. a tag in the repos > > 2. tars posted on the site for easy download, build and install, with > > instructions. > > 3. these tars have basically been tested. > > Not sure what the definition of 'tested' is, probably worth expanding it > a bit. > Y. > > > 4. optional binary builds for different distros linked from the release page > > 5. a set of any relevant release notes. > > > > Carl. > > _______________________________________________ > > Arch mailing list > > Arch at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/arch > > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board From dfediuck at redhat.com Wed Nov 30 11:41:21 2011 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 30 Nov 2011 13:41:21 +0200 Subject: First community release In-Reply-To: <02a101ccaf4d$aca922d0$05fb6870$@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED55165.2080202@redhat.com> <02a101ccaf4d$aca922d0$05fb6870$@redhat.com> Message-ID: <201111301341.21596.dfediuck@redhat.com> WRT- Signing: Who should sign tarballs? We should KISS (keep it standard & simple...) ie- 1. All tarballs should have md5 / other hash published in the downloads page and possibly a hash file with the tarball. 2. Each distro will sign its packages in its own means, such as signing key, certificate, etc. On Wednesday 30 November 2011 12:49:01 Ofer Schreiber wrote: > Following this discussion, I've created > http://www.ovirt.org/wiki/Releases/First_Release in order to track the > schedule/gaps/requirements for the first oVirt release. > > Thoughts? > > Ofer. > > > -----Original Message----- > > From: board-bounces at ovirt.org [mailto:board-bounces at ovirt.org] On Behalf > Of Yaniv Kaul > > Sent: 29 November 2011 23:41 > > To: cctrieloff at redhat.com > > Cc: arch at ovirt.org; board at ovirt.org > > Subject: Re: First community release > > > > On 11/29/2011 08:14 PM, Carl Trieloff wrote: > > > On 11/29/2011 12:58 PM, Barak Azulay wrote: > > >> I think we should aim for a fast first release, meaning ASAP on jboss > 5 > > >> > > >> The questions are: > > >> > > >> 1 - what is a release? a tag in all the repos to be picked up by each > > >> distro ? > > >> 2 - do we have any criteria on what is good enough for a release ? or > > >> any point in time that everybody feels ready? > > > I would say a release needs to be at least. > > > > > > 1. a tag in the repos > > > 2. tars posted on the site for easy download, build and install, with > > > instructions. > > > 3. these tars have basically been tested. > > > > Not sure what the definition of 'tested' is, probably worth expanding it > > a bit. > > Y. > > > > > 4. optional binary builds for different distros linked from the > release page > > > 5. a set of any relevant release notes. > > > > > > Carl. > > > _______________________________________________ > > > Arch mailing list > > > Arch at ovirt.org > > > http://lists.ovirt.org/mailman/listinfo/arch > > > > _______________________________________________ > > Board mailing list > > Board at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/board > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch > -- /d "Do not look into laser with remaining eye." --On a laser pointer user-manual From pmyers at redhat.com Wed Nov 30 13:54:50 2011 From: pmyers at redhat.com (Perry Myers) Date: Wed, 30 Nov 2011 08:54:50 -0500 Subject: First community release In-Reply-To: <201111301341.21596.dfediuck@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED55165.2080202@redhat.com> <02a101ccaf4d$aca922d0$05fb6870$@redhat.com> <201111301341.21596.dfediuck@redhat.com> Message-ID: <4ED635AA.4040406@redhat.com> On 11/30/2011 06:41 AM, Doron Fediuck wrote: > WRT- > Signing: Who should sign tarballs? > > We should KISS (keep it standard & simple...) ie- > > 1. All tarballs should have md5 / other hash published > in the downloads page and possibly a hash file with the tarball. > > 2. Each distro will sign its packages in its own means, > such as signing key, certificate, etc. +1 Though for the case of oVirt Node, since the ISO isn't in Fedora officially, all we will provide is the md5 of the ISO itself. Perry From apevec at gmail.com Wed Nov 30 14:20:05 2011 From: apevec at gmail.com (Alan Pevec) Date: Wed, 30 Nov 2011 15:20:05 +0100 Subject: First community release In-Reply-To: <201111301341.21596.dfediuck@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED55165.2080202@redhat.com> <02a101ccaf4d$aca922d0$05fb6870$@redhat.com> <201111301341.21596.dfediuck@redhat.com> Message-ID: On Wed, Nov 30, 2011 at 12:41 PM, Doron Fediuck wrote: > 1. All tarballs should have md5 / other hash published > in the downloads page and possibly a hash file with the tarball. SHA512 please e.g. http://collectd.org/files/SHA512SUM > 2. Each distro will sign its packages in its own means, > such as signing key, certificate, etc. It would be nice if upstream release manager could sign release tag in git. Alan From cctrieloff at redhat.com Wed Nov 30 14:20:21 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Wed, 30 Nov 2011 09:20:21 -0500 Subject: First community release In-Reply-To: <4ED55165.2080202@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED52118.6050706@redhat.com> <4ED55165.2080202@redhat.com> Message-ID: <4ED63BA5.5060008@redhat.com> On 11/29/2011 04:40 PM, Yaniv Kaul wrote: >> 3. these tars have basically been tested. > > Not sure what the definition of 'tested' is, probably worth expanding > it a bit. > Y. this can be fuzzy, but on other projects it is. Have a few people down load the tar, follow the instructions and get it installed and running + passing the tests that are checked in. Carl. From jchoate at redhat.com Wed Nov 30 14:28:46 2011 From: jchoate at redhat.com (Jon Choate) Date: Wed, 30 Nov 2011 09:28:46 -0500 Subject: First community release In-Reply-To: <4ED63BA5.5060008@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED52118.6050706@redhat.com> <4ED55165.2080202@redhat.com> <4ED63BA5.5060008@redhat.com> Message-ID: <4ED63D9E.7020908@redhat.com> On 11/30/2011 09:20 AM, Carl Trieloff wrote: > On 11/29/2011 04:40 PM, Yaniv Kaul wrote: >>> 3. these tars have basically been tested. >> Not sure what the definition of 'tested' is, probably worth expanding >> it a bit. >> Y. > this can be fuzzy, but on other projects it is. > > Have a few people down load the tar, follow the instructions and get it > installed and running + passing the tests that are checked in. > > Carl. > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch Have we defined the process for approving the release (like having a vote which gets at least 3 +1's and no -1's)? From aliguori at us.ibm.com Wed Nov 30 14:36:12 2011 From: aliguori at us.ibm.com (Anthony Liguori) Date: Wed, 30 Nov 2011 08:36:12 -0600 Subject: KVM Call Agenda for 12/6 (Tuesday) @ 10am US/Eastern Message-ID: <4ED63F5C.8000808@us.ibm.com> Hi, I'd like to propose that we discuss guest agent convergence in our next KVM call. I've CC'd folks from oVirt and libvirt to join the discussion. I think we should probably attempt to have some structure to the discussion. I would suggest: 1. A short introduction to each of the guest agents, what guests they support, and what verbs they support. 2. A short description of key requirements from each party (oVirt, libvirt, QEMU) for a guest agent 3. An open discussion about possible ways to collaborate/converge. Regards, Anthony Liguori From bazulay at redhat.com Wed Nov 30 14:56:44 2011 From: bazulay at redhat.com (Barak Azulay) Date: Wed, 30 Nov 2011 16:56:44 +0200 Subject: KVM Call Agenda for 12/6 (Tuesday) @ 10am US/Eastern In-Reply-To: <4ED63F5C.8000808@us.ibm.com> References: <4ED63F5C.8000808@us.ibm.com> Message-ID: <4ED6442C.2060608@redhat.com> On 11/30/2011 04:36 PM, Anthony Liguori wrote: > Hi, > > I'd like to propose that we discuss guest agent convergence in our next > KVM call. I've CC'd folks from oVirt and libvirt to join the discussion. > > I think we should probably attempt to have some structure to the > discussion. I would suggest: > > 1. A short introduction to each of the guest agents, what guests they > support, and what verbs they support. Please take a look at http://www.ovirt.org/w/images/2/20/Ovirt-guest-agent.pdf. > > 2. A short description of key requirements from each party (oVirt, > libvirt, QEMU) for a guest agent we can start with http://www.ovirt.org/wiki/Guest_agent_proposals as a basis for the discussion > > 3. An open discussion about possible ways to collaborate/converge. > > Regards, > > Anthony Liguori > > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From kwade at redhat.com Wed Nov 30 15:09:04 2011 From: kwade at redhat.com (Karsten 'quaid' Wade) Date: Wed, 30 Nov 2011 07:09:04 -0800 Subject: First community release In-Reply-To: <02a101ccaf4d$aca922d0$05fb6870$@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED52118.6050706@redhat.com> <4ED55165.2080202@redhat.com> <02a101ccaf4d$aca922d0$05fb6870$@redhat.com> Message-ID: <4ED64710.2010109@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11/30/2011 02:49 AM, Ofer Schreiber wrote: > Following this discussion, I've created > http://www.ovirt.org/wiki/Releases/First_Release in order to track > the schedule/gaps/requirements for the first oVirt release. BTW, I changed the page name to: http://www.ovirt.org/wiki/First_release ... as per the guidelines on http://ovirt.org/wiki/How_to_make_pages . The redirect is setup automatically. - - Karsten - -- name: Karsten 'quaid' Wade, Sr. Community Architect team: Red Hat Community Architecture & Leadership uri: http://communityleadershipteam.org http://TheOpenSourceWay.org gpg: AD0E0C41 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFO1kcQ2ZIOBq0ODEERAhaOAKCwwyhHv9+M7HM3HGve4uzh9jQ+YgCgt6sh r6y48+57Q2eBYZQDrI3YNw8= =Ghyh -----END PGP SIGNATURE----- From kwade at redhat.com Wed Nov 30 16:24:17 2011 From: kwade at redhat.com (Karsten 'quaid' Wade) Date: Wed, 30 Nov 2011 08:24:17 -0800 Subject: adjustment to mailing lists Message-ID: <4ED658B1.4040902@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 At the meeting today we discussed opening mailing lists to allow posting by anyone to ovirt.org mailing lists in order to be friendlier to other communities and the need for cross-posting. We decided to first try removing the notice that tells people their post is held for moderation. If moderators are fast in working their queue, people shouldn't notice the hold. To set this on your mailing lists: 1. Open the list, go to 'General Options'. 2. Scroll down to 'Send mail to poster when their posting is held for approval?', which is the 'respond_to_post_requests' option, and set it to 'No'. Alternately, you can substitute your listname here for $LISTNAME: http://lists.ovirt.org/mailman/admin/$LISTNAME/?VARHELP=general/respond_to_post_requests - - Karsten - -- name: Karsten 'quaid' Wade, Sr. Community Architect team: Red Hat Community Architecture & Leadership uri: http://communityleadershipteam.org http://TheOpenSourceWay.org gpg: AD0E0C41 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFO1liw2ZIOBq0ODEERAlJ2AKCZVW0PDc0jPI2zCQ6aQyjmM5bkvwCgqSxl LZ4YhU0K5Mt6oRID57seSFo= =7k68 -----END PGP SIGNATURE----- From iheim at redhat.com Wed Nov 30 16:25:47 2011 From: iheim at redhat.com (Itamar Heim) Date: Wed, 30 Nov 2011 18:25:47 +0200 Subject: adjustment to mailing lists In-Reply-To: <4ED658B1.4040902@redhat.com> References: <4ED658B1.4040902@redhat.com> Message-ID: <4ED6590B.8040200@redhat.com> On 11/30/2011 06:24 PM, Karsten 'quaid' Wade wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > At the meeting today we discussed opening mailing lists to allow > posting by anyone to ovirt.org mailing lists in order to be friendlier > to other communities and the need for cross-posting. > > We decided to first try removing the notice that tells people their > post is held for moderation. If moderators are fast in working their > queue, people shouldn't notice the hold. > > To set this on your mailing lists: engine-devel and engine-patches changed accordingly. > > 1. Open the list, go to 'General Options'. > 2. Scroll down to 'Send mail to poster when their posting is held for > approval?', which is the 'respond_to_post_requests' option, and set it > to 'No'. > > Alternately, you can substitute your listname here for $LISTNAME: > > http://lists.ovirt.org/mailman/admin/$LISTNAME/?VARHELP=general/respond_to_post_requests > > - - Karsten > - -- > name: Karsten 'quaid' Wade, Sr. Community Architect > team: Red Hat Community Architecture& Leadership > uri: http://communityleadershipteam.org > http://TheOpenSourceWay.org > gpg: AD0E0C41 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iD8DBQFO1liw2ZIOBq0ODEERAlJ2AKCZVW0PDc0jPI2zCQ6aQyjmM5bkvwCgqSxl > LZ4YhU0K5Mt6oRID57seSFo= > =7k68 > -----END PGP SIGNATURE----- > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch From cctrieloff at redhat.com Wed Nov 30 16:27:13 2011 From: cctrieloff at redhat.com (Carl Trieloff) Date: Wed, 30 Nov 2011 11:27:13 -0500 Subject: adjustment to mailing lists In-Reply-To: <4ED6590B.8040200@redhat.com> References: <4ED658B1.4040902@redhat.com> <4ED6590B.8040200@redhat.com> Message-ID: <4ED65961.5010401@redhat.com> On 11/30/2011 11:25 AM, Itamar Heim wrote: > On 11/30/2011 06:24 PM, Karsten 'quaid' Wade wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> At the meeting today we discussed opening mailing lists to allow >> posting by anyone to ovirt.org mailing lists in order to be friendlier >> to other communities and the need for cross-posting. >> >> We decided to first try removing the notice that tells people their >> post is held for moderation. If moderators are fast in working their >> queue, people shouldn't notice the hold. >> >> To set this on your mailing lists: > > engine-devel and engine-patches changed accordingly. board changed. Carl. From jamie at shareable.org Thu Nov 17 14:31:11 2011 From: jamie at shareable.org (Jamie Lokier) Date: Thu, 17 Nov 2011 14:31:11 +0000 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <4EC3C6C3.1020208@redhat.com> References: <4EC3BC67.6030005@codemonkey.ws> <4EC3BD12.9000508@redhat.com> <4EC3BDCC.3050102@codemonkey.ws> <4EC3C6C3.1020208@redhat.com> Message-ID: <20111117143111.GA3751@jl-vm1.vm.bytemark.co.uk> >>> On 11/16/2011 03:36 PM, Anthony Liguori wrote: >>>> We have another requirement. We need to embed the source for the guest >>>> agent in the QEMU release tarball. This is for GPL compliance since we >>>> want to include an ISO (eventually) that contains binaries. Paolo Bonzini wrote: > ovirt-guest-agent is licensed under GPLv3, so you do not need to; > the options in GPLv3 include this one: > > d) Convey the object code by offering access from a designated > place (gratis or for a charge), and offer equivalent access to the > Corresponding Source in the same way through the same place at no > further charge. You need not require recipients to copy the > Corresponding Source along with the object code. If the place to > copy the object code is a network server, the Corresponding Source > may be on a different server (operated by you or a third party) > that supports equivalent copying facilities, provided you maintain > clear directions next to the object code saying where to find the > Corresponding Source. Regardless of what server hosts the > Corresponding Source, you remain obligated to ensure that it is > available for as long as needed to satisfy these requirements. Hi, GPLv2 also has a clause similar to the above. In GPLv2 it's not enumerated, but says: If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. I'm not sure why "mere aggregation" (GPLv2) and "aggregate" (GPLv3) aren't sufficient to allow shipping the different binaries together in a single ISO regardless of where the source code lives or how it's licensed. -- Jamie From lcapitulino at redhat.com Fri Nov 18 00:47:56 2011 From: lcapitulino at redhat.com (Luiz Capitulino) Date: Thu, 17 Nov 2011 22:47:56 -0200 Subject: [Qemu-devel] converging around a single guest agent In-Reply-To: <201111171909.11021.bazulay@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <20111116202451.GI2726@us.ibm.com> <201111171909.11021.bazulay@redhat.com> Message-ID: <20111117224756.2f07919f@doriath> On Thu, 17 Nov 2011 19:09:10 +0200 Barak Azulay wrote: > On Wednesday 16 November 2011 22:24:51 Adam Litke wrote: > > I have been following this thread pretty closely and the one sentence > > summary of the current argument is: ovirt-guest-agent is already > > featureful and tested, so let's drop qemu-ga and have everyone adopt > > ovirt-guest-agent. > > > Not exactly what I meant, but the above is also a factor that needs to be > taken into account. Maybe not, but I share Adam's impression. > to be on the safe side I'll add: > > 1 - true we want the ovirt-guest-agent to be widly adopted. > 2 - we are willing to do major changes in it to fit all (let's discuss it in > details) We should be willing to make major changes in qemu-ga too. Actually, it seems to me that the best way to handle most of what has been discussed in this thread is to submit patches to qemu-ga and make it do what you need it to do. I don't see problems in having many different guest agents out there, as long as they serve different purposes. If there's feature overlap, then let's start adding what's missing to qemu-ga (of course we want to keep it kvm centric though). > 3 - one can not ignore the: > * maturity of the 2 components > * capabilities This is subjective, let's talk about concrete things. > when considering a convergence into a single agent I don't think we should be considering choosing one or another agent, at least not as a first step. That's why people are interpreting you the wrong way. IMHO, the first thing to do is to define what qemu's agent should provide, and then check what's missing in qemu-ga and possibly implement it. For what's worth, Adam's idea on API and making qemu-ga provide the basic infrastructure seems quite reasonable to me. > > > > > > Unfortunately, this track strays completely away from > > the stated goal of convergence. I have at least two examples of why the > > greater KVM community can never adopt ovirt-guest-agent as-is. To address > > this, I would like to counter with an example on how qemu-ga can enable > > the deployment of ovirt-guest-agent features and satisfy the needs of the > > whole community at the same time. > > > > 1) Scope: The ovirt-guest-agent contains functionality that is incredibly > > useful within the context of oVirt. Single Sign-on is very handy but KVM > > users outside the scope of oVirt will not want this extra complexity in > > their agent. For simplicity they will probably just write something small > > that does what they need (and we have failed to provide a ubiquitous KVM > > agent). > > > the ovirt-guest-agent functions well when the SSO if not installed, so no need > to install it if one does not wish to enjoy it > > > > > > 1) Deployment complexity: The more complex the guest agent is, the more > > often it will need to be updated (bug/security fixes, distro > > compatibility, new features). Rolling out guest agent updates does not > > scale well in large environments (especially when the guest and host > > administrators are not the same person). > > > This is what we have been doing for the last few years (with the entire ovirt > stack). so we are basing our approach on that experiance. > In the session I gave I tried to touch the way we handle deployment of the > packages (both linux & windows guests), but it was a too large subject to > contain in that session. > > deploying and upgrading the agent was one of the issues handled. > I can elaborate more on the isse if required. > > I'm not sure that the deployment/upgrade of guest agent should be done through > the monitor, I think it should be left for a higher management layer. > > > > > > For these reasons (and many others), I support having an agent with very > > basic primitives that can be orchestrated by the host to provide needed > > functionality. This agent would present a low-level, stable, extensible > > API that everyone can use. Today qemu-ga supports the following verbs: > > sync ping info shutdown file-open file-close file-read file-write > > file-seek file-flush fsfreeze-status fsfreeze-freeze fsfreeze-thaw. If we > > add a generic execute mechanism, then the agent can provide everything > > needed by oVirt to deploy SSO. > > > I'm not sure this approach will pass common-criteria review. > > > > > > Let's assume that we have already agreed on some sort of security policy > > for the write-file and exec primitives. Consensus is possible on this > > issue but I don't want to get bogged down with that here. > > > > With the above primitives, SSO could be deployed automatically to a guest > > with the following sequence of commands: > > > > file-open "/sso-package.bin" "w" > > file-write > > file-close > > file-open "/sso-package.bin" "x" > > file-exec > > file-close > > > > At this point, the package is installed. It can contain whatever existing > > logic exists in the ovirt-guest-agent today. To perform a user login, > > we'll assume that sso-package.bin contains an executable > > 'sso/do-user-sso': > > > > file-open "/sso/do-user-sso" "x" > > exec > > file-close > > > > At this point the user would be logged in as before. > > > > Obviously, this type of approach could be made easier by providing a well > > designed exec API that returns command exit codes and (optionally) command > > output. We could also formalize the install of additional components into > > some sort of plugin interface. These are all relatively easy problems to > > solve. > > > > If we go in this direction, we would have a simple, general-purpose agent > > with low-level primitives that everyone can use. We would also be able to > > easily extend the agent based on the needs of individual deployments (not > > the least of which is an oVirt environment). If certain plugins become > > popular enough, they can always be promoted to first-order API calls in > > future versions of the API. > > > > What are your thoughts on this approach? > > > When thinking on various guest OSs, I think the approach of a well defined API > is much more appealing. The reason is that the implementation is contained > within the specific OS implementation of the guest-agent, and not left to a > person coding this from the high level management system. > e.g. on the SSO example you gave all the coding of the seaquence of login > needs to be done on the management system and needs to first check what OS it > is and only than execute the relevant code (which is much harder for error > handling in that situation), while when using a well defined interface all you > need to do is call the "Login()" API. > > > I agree that from development point of view the flexible approach of "I can do > anything in the guest" is much more appiling, I don't think this is the case > for enterprise virtualization system that needs to manage thousands of VMs > with different OS types > > Thanks > Barak Azulay > > > > > > > From lpeer at redhat.com Sun Nov 20 12:34:37 2011 From: lpeer at redhat.com (Livnat Peer) Date: Sun, 20 Nov 2011 14:34:37 +0200 Subject: Call for list moderators. In-Reply-To: References: Message-ID: <4EC8F3DD.3010003@redhat.com> On 11/17/2011 05:43 PM, Omer Frenkel wrote: > Hi, > i don't mind volunteer to help with engine-devel at ovirt.org > (maybe even more later on if needed) > > Omer. +1 for Omer helping with pending moderator requests. Livnat > > ----- Original Message ----- >> From: "Steve Gordon" >> To: dlaor at redhat.com >> Cc: arch at ovirt.org, board at ovirt.org >> Sent: Wednesday, November 16, 2011 6:05:48 PM >> Subject: Call for list moderators. >> >> Hi all, >> >> The issue of list moderation, particularly moderation of posts from >> unsubscribed users (usually their first post) vs spam, was raised at >> today's meeting. For now we have agreed that we should continue with >> the status quo whereby posts from unsubscribed users require >> moderation, as always there is a 'but' though. We need more >> moderators! >> >> The task is not too onerous at this time but ideally we'd like a few >> moderators for each list to ensure that any posts awaiting >> moderation are always processed in a timely manner. If you'd like to >> nominate yourself as a volunteer please just respond to this >> message, include the @ovirt.org list(s) you'd be happy to moderate >> in your message. >> >> Thanks, >> >> Steve >> >> ----- Original Message ----- >>> From: "Dor Laor" >>> To: board at ovirt.org >>> Sent: Wednesday, November 16, 2011 8:46:27 AM >>> Subject: Fwd: Your message to Arch awaits moderator approval >>> >>> How about allowing non registered members to send unapproved >>> emails? >>> >>> -------- Original Message -------- >>> Subject: Your message to Arch awaits moderator approval >>> Date: Wed, 16 Nov 2011 08:45:21 -0500 >>> From: arch-bounces at ovirt.org >>> To: dlaor at redhat.com >>> >>> Your mail to 'Arch' with the subject >>> >>> Re: [Qemu-devel] converging around a single guest agent >>> >>> Is being held until the list moderator can review it for approval. >>> >>> The reason it is being held: >>> >>> Post by non-member to a members-only list >>> >>> Either the message will get posted to the list, or you will receive >>> notification of the moderator's decision. If you would like to >>> cancel >>> this posting, please visit the following URL: >>> >>> >>> http://lists.ovirt.org/mailman/confirm/arch/458a765d8cab62007fc55c038948e23e11608a56 >>> >>> _______________________________________________ >>> Board mailing list >>> Board at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/board >>> >> _______________________________________________ >> Board mailing list >> Board at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/board >> > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board From rjones at redhat.com Thu Nov 24 16:47:04 2011 From: rjones at redhat.com (Richard W.M. Jones) Date: Thu, 24 Nov 2011 16:47:04 +0000 Subject: [Qemu-devel] wiki summary In-Reply-To: <4ECE3B27.3020708@redhat.com> References: <201111151924.41357.bazulay@redhat.com> <4EC459F2.5010701@linux.vnet.ibm.com> <201111171834.48040.bazulay@redhat.com> <4EC5674B.30807@linux.vnet.ibm.com> <4ECE3B27.3020708@redhat.com> Message-ID: <20111124164704.GW25839@amd.home.annexia.org> On Thu, Nov 24, 2011 at 02:40:07PM +0200, Dor Laor wrote: > Using QMP is an advantage, I agree. > However it can be used by another option - move the QMP schema out > of qemu.git so all projects like libvirt, agents, vdsm, etc will be > able to consume it directly. > > This way, adding a new (agent) command will immediately propagate to > all of the stack instead of each component to implement it > differently (while it would still be possible). > That's what libguestfs scheme do today. I think Dor CC'd me on this because we use aggressive code generation in libguestfs to simplify creating new APIs. For an example see: http://fedorapeople.org/gitweb?p=rjones/public_git/libguestfs.git;a=commitdiff;h=cbd1c45d95c530c8d94103dcc2c521bf5501ef59 The patch is in three parts: (1) C code to implement the new API. Constructing the command line, parsing the output. (2) Metadata in the generator to describe the new API (parameters, return, documentation, etc.) ==> From this part *everything* is generated <== - C header file - bindings in 8+ programming languages - documentation - RPC used over internal virtio-serial connection (3) A test. Intelligently generating code like this has been an enormous win. Here are two slightly more complicated examples demonstrating the same thing: http://fedorapeople.org/gitweb?p=rjones/public_git/libguestfs.git;a=commitdiff;h=c4bd6bba8d88ecf1ebf4a9c2c80a407d9971aaf7 http://fedorapeople.org/gitweb?p=rjones/public_git/libguestfs.git;a=commitdiff;h=c11a92751e003b3d4bc3584b598afc9bd9d9e703 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora From lpeer at redhat.com Tue Nov 29 15:59:00 2011 From: lpeer at redhat.com (Livnat Peer) Date: Tue, 29 Nov 2011 17:59:00 +0200 Subject: First community release In-Reply-To: <4EB80CAB.3030908@redhat.com> References: <4EB80CAB.3030908@redhat.com> Message-ID: <4ED50144.60406@redhat.com> On 11/07/2011 06:51 PM, Carl Trieloff wrote: > > At the workshop in the release / roadmap working session we agreed that > around November 16th we see how all the distro's are making out in > creating a release for oVirt and set the date for the first community > release. > > Please have your comments / feedback for that time period, at which > point I will start the release discussions. > > regards > Carl. > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board Hi All, Not sure where we stand with the first release but I think we should take into account the jboss version we use before releasing. We currently deploy on jboss as 5, but we started working on deployment on jboss as 7. Releasing on AS 5 and then moving to AS 7 might require some upgrade path and it is a time consuming task to do that. Since we already started working on the deployment on jboss AS 7, I think it might worth holding back first 'formal' release until it is done. It will help us avoiding upgrade support and get all the benefits of using as7 (startup time, memory/cpu foot-print etc.) - get better reviews? Livnat From jimjag at redhat.com Tue Nov 29 18:30:54 2011 From: jimjag at redhat.com (Jim Jagielski) Date: Tue, 29 Nov 2011 13:30:54 -0500 Subject: First community release In-Reply-To: <4ED52118.6050706@redhat.com> References: <4EB80CAB.3030908@redhat.com> <4ED50144.60406@redhat.com> <4ED51D3F.7050203@redhat.com> <4ED52118.6050706@redhat.com> Message-ID: <7935BB2F-78A7-4DDD-8C45-A0D9D42CF3AC@redhat.com> They should also be signed as well... On Nov 29, 2011, at 1:14 PM, Carl Trieloff wrote: > On 11/29/2011 12:58 PM, Barak Azulay wrote: >> >> I think we should aim for a fast first release, meaning ASAP on jboss 5 >> >> The questions are: >> >> 1 - what is a release? a tag in all the repos to be picked up by each >> distro ? >> 2 - do we have any criteria on what is good enough for a release ? or >> any point in time that everybody feels ready? > > I would say a release needs to be at least. > > 1. a tag in the repos > 2. tars posted on the site for easy download, build and install, with > instructions. > 3. these tars have basically been tested. > 4. optional binary builds for different distros linked from the release page > 5. a set of any relevant release notes. > > Carl. > _______________________________________________ > Board mailing list > Board at ovirt.org > http://lists.ovirt.org/mailman/listinfo/board -- Jim Jagielski | jimjag at redhat.com | 443-324-8390 (cell) From djorm at redhat.com Wed Nov 30 00:03:44 2011 From: djorm at redhat.com (David Jorm) Date: Tue, 29 Nov 2011 19:03:44 -0500 (EST) Subject: First community release In-Reply-To: <4ED55165.2080202@redhat.com> Message-ID: > > 3. these tars have basically been tested. > > Not sure what the definition of 'tested' is, probably worth expanding > it > a bit. > Y. I would think just smoke tested. David From djorm at redhat.com Wed Nov 30 00:16:31 2011 From: djorm at redhat.com (David Jorm) Date: Tue, 29 Nov 2011 19:16:31 -0500 (EST) Subject: First community release In-Reply-To: <4ED50144.60406@redhat.com> Message-ID: > Hi All, > Not sure where we stand with the first release but I think we should > take into account the jboss version we use before releasing. > > We currently deploy on jboss as 5, but we started working on > deployment > on jboss as 7. > Releasing on AS 5 and then moving to AS 7 might require some upgrade > path and it is a time consuming task to do that. > > Since we already started working on the deployment on jboss AS 7, I > think it might worth holding back first 'formal' release until it is > done. It will help us avoiding upgrade support and get all the > benefits > of using as7 (startup time, memory/cpu foot-print etc.) - get better > reviews? Please note that JBoss AS 5 has not been updated with any async security patches since it was released in 2009. There are multiple known high impact security flaws that affect it. I would prefer we only ship against AS 7, as it incorporates fixes for these flaws. David From chrisw at redhat.com Wed Nov 30 16:29:47 2011 From: chrisw at redhat.com (Chris Wright) Date: Wed, 30 Nov 2011 08:29:47 -0800 Subject: KVM Call Agenda for 12/6 (Tuesday) @ 10am US/Eastern In-Reply-To: <4ED63F5C.8000808@us.ibm.com> References: <4ED63F5C.8000808@us.ibm.com> Message-ID: <20111130162947.GA26932@x200.localdomain> * Anthony Liguori (aliguori at us.ibm.com) wrote: > Hi, > > I'd like to propose that we discuss guest agent convergence in our next KVM > call. I've CC'd folks from oVirt and libvirt to join the discussion. > > I think we should probably attempt to have some structure to the discussion. > I would suggest: > > 1. A short introduction to each of the guest agents, what guests they > support, and what verbs they support. I think we did this once before w/ Matahari. Can we please capture these things in email before the call, so people actually have time to ponder the details. > 2. A short description of key requirements from each party (oVirt, libvirt, > QEMU) for a guest agent Same here...call this the abstract/intro of the above detailed list of verbs and guest support, and send it by Friday this week. I know there's plenty of details buried in the current thread and old discussions of Matahari. But that's just it...buried... > 3. An open discussion about possible ways to collaborate/converge. That should really help facilitate this item ;) thanks, -chris From aliguori at us.ibm.com Wed Nov 30 16:36:40 2011 From: aliguori at us.ibm.com (Anthony Liguori) Date: Wed, 30 Nov 2011 10:36:40 -0600 Subject: KVM Call Agenda for 12/6 (Tuesday) @ 10am US/Eastern In-Reply-To: <20111130162947.GA26932@x200.localdomain> References: <4ED63F5C.8000808@us.ibm.com> <20111130162947.GA26932@x200.localdomain> Message-ID: <4ED65B98.9080607@us.ibm.com> On 11/30/2011 10:29 AM, Chris Wright wrote: > * Anthony Liguori (aliguori at us.ibm.com) wrote: >> Hi, >> >> I'd like to propose that we discuss guest agent convergence in our next KVM >> call. I've CC'd folks from oVirt and libvirt to join the discussion. >> >> I think we should probably attempt to have some structure to the discussion. >> I would suggest: >> >> 1. A short introduction to each of the guest agents, what guests they >> support, and what verbs they support. > > I think we did this once before w/ Matahari. Can we please capture > these things in email before the call, so people actually have time > to ponder the details. > >> 2. A short description of key requirements from each party (oVirt, libvirt, >> QEMU) for a guest agent > > Same here...call this the abstract/intro of the above detailed list of > verbs and guest support, and send it by Friday this week. > > I know there's plenty of details buried in the current thread and old > discussions of Matahari. But that's just it...buried... > >> 3. An open discussion about possible ways to collaborate/converge. > > That should really help facilitate this item ;) Good suggestions! Regards, Anthony Liguori > > thanks, > -chris > From pmyers at redhat.com Wed Nov 30 17:04:17 2011 From: pmyers at redhat.com (Perry Myers) Date: Wed, 30 Nov 2011 12:04:17 -0500 Subject: adjustment to mailing lists In-Reply-To: <4ED65961.5010401@redhat.com> References: <4ED658B1.4040902@redhat.com> <4ED6590B.8040200@redhat.com> <4ED65961.5010401@redhat.com> Message-ID: <4ED66211.4050703@redhat.com> On 11/30/2011 11:27 AM, Carl Trieloff wrote: > On 11/30/2011 11:25 AM, Itamar Heim wrote: >> On 11/30/2011 06:24 PM, Karsten 'quaid' Wade wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> At the meeting today we discussed opening mailing lists to allow >>> posting by anyone to ovirt.org mailing lists in order to be friendlier >>> to other communities and the need for cross-posting. >>> >>> We decided to first try removing the notice that tells people their >>> post is held for moderation. If moderators are fast in working their >>> queue, people shouldn't notice the hold. >>> >>> To set this on your mailing lists: >> >> engine-devel and engine-patches changed accordingly. > > board changed. node-devel changed From mburns at redhat.com Wed Nov 30 21:25:53 2011 From: mburns at redhat.com (Mike Burns) Date: Wed, 30 Nov 2011 16:25:53 -0500 Subject: adjustment to mailing lists In-Reply-To: <4ED66211.4050703@redhat.com> References: <4ED658B1.4040902@redhat.com> <4ED6590B.8040200@redhat.com> <4ED65961.5010401@redhat.com> <4ED66211.4050703@redhat.com> Message-ID: <1322688353.3038.16.camel@mburns-laptop> On Wed, 2011-11-30 at 12:04 -0500, Perry Myers wrote: > On 11/30/2011 11:27 AM, Carl Trieloff wrote: > > On 11/30/2011 11:25 AM, Itamar Heim wrote: > >> On 11/30/2011 06:24 PM, Karsten 'quaid' Wade wrote: > >>> -----BEGIN PGP SIGNED MESSAGE----- > >>> Hash: SHA1 > >>> > >>> At the meeting today we discussed opening mailing lists to allow > >>> posting by anyone to ovirt.org mailing lists in order to be friendlier > >>> to other communities and the need for cross-posting. > >>> > >>> We decided to first try removing the notice that tells people their > >>> post is held for moderation. If moderators are fast in working their > >>> queue, people shouldn't notice the hold. > >>> > >>> To set this on your mailing lists: > >> > >> engine-devel and engine-patches changed accordingly. > > > > board changed. > > node-devel changed node-patches changed > _______________________________________________ > Arch mailing list > Arch at ovirt.org > http://lists.ovirt.org/mailman/listinfo/arch