Hey all, Continuing the discussion about Aeolus instance data injection to a VM (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html) we propose a new VM Payload feature. The following wiki page contains a description page of the feature. http://www.ovirt.org/wiki/Features/VMPayload Please read and review. There are several approaches there, and we wish to head your opinions and thoughts about them. Once we agree on an approach, we will start designing.

Thank you, Oved _______________________________________________ Engine-devel mailing list Engine-devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel

----- Original Message ----- > From: "Ayal Baron" <abaron(a)redhat.com&gt; > To: "Oved Ourfalli" <ovedo(a)redhat.com&gt; > Cc: engine-devel(a)ovirt.org > Sent: Thursday, January 19, 2012 4:05:08 PM > Subject: Re: [Engine-devel] VM Payload feature > > > > ----- Original Message ----- > > Hey all, > > > > Continuing the discussion about Aeolus instance data injection to > > a > > VM > > (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html) > > we propose a new VM Payload feature. > > > > The following wiki page contains a description page of the > > feature. > > http://www.ovirt.org/wiki/Features/VMPayload > > > > Please read and review. > > There are several approaches there, and we wish to head your > > opinions > > and thoughts about them. > > > > Once we agree on an approach, we will start designing. > > Permanent payload availability requires determining where the > payload > is stored. > Makes sense to me to store it together with the VM disks on the > storage domain, but that requires the small object store which will > not be available in the coming version (payloads can be large and > keeping them in the DB and passing over the net every time the VM > is > run doesn't make much sense). > I guess we can start with storing it in the database, with some size limitation, and move it to the storage domain later on.

> Wrt availability, I don't see a reason to exclude attaching both a > CD > and a payload via another CD at the same time (i.e. multiple > devices). > > > > > Thank you, > > Oved > > _______________________________________________ > > Engine-devel mailing list > > Engine-devel(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/engine-devel > > > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel >

On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote: > > > ----- Original Message ----- >> From: "Ayal Baron" <abaron(a)redhat.com&gt; >> To: "Oved Ourfalli" <ovedo(a)redhat.com&gt; >> Cc: engine-devel(a)ovirt.org >> Sent: Thursday, January 19, 2012 4:05:08 PM >> Subject: Re: [Engine-devel] VM Payload feature >> >> >> >> ----- Original Message ----- >>> Hey all, >>> >>> Continuing the discussion about Aeolus instance data injection to a >>> VM >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html) >>> we propose a new VM Payload feature. >>> >>> The following wiki page contains a description page of the feature. >>> http://www.ovirt.org/wiki/Features/VMPayload >>> >>> Please read and review. >>> There are several approaches there, and we wish to head your >>> opinions >>> and thoughts about them. >>> >>> Once we agree on an approach, we will start designing. >> >> Permanent payload availability requires determining where the payload >> is stored. >> Makes sense to me to store it together with the VM disks on the >> storage domain, but that requires the small object store which will >> not be available in the coming version (payloads can be large and >> keeping them in the DB and passing over the net every time the VM is >> run doesn't make much sense). >> > I guess we can start with storing it in the database, with some size limitation, and move it to the storage domain later on. > >> Wrt availability, I don't see a reason to exclude attaching both a CD >> and a payload via another CD at the same time (i.e. multiple >> devices). >> >>> >>> Thank you, >>> Oved >>> _______________________________________________ >>> Engine-devel mailing list >>> Engine-devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/engine-devel >>> >> _______________________________________________ >> Engine-devel mailing list >> Engine-devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/engine-devel >> My perspective is that of the end user, the instance retrieving the data. From a functional standpoint I would like to see similar performance to what EC2 provides. AWS EC2 user data is limited to 16K. This limit applies to the data in raw form, not base64 encoded form. see: http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-da... I am concerned about the 512k limit as mentioned in the notes of: http://www.ovirt.org/wiki/Features/VMPayload "if the content of the file is bigger the 512K it will pass an nfs share for vdsm to fetch the file/s" Please confirm: - Will it be possible to pass user data to larger than 512k? - If so what will the instance need to do in order to retrieve user-data bigger than 512k. - What will the MAX size supported for the user-data?

Thank you. Joe VLcek

----- Original Message ----- > From: "Shahar Havivi" <shaharh(a)redhat.com&gt; > To: "Joseph VLcek" <jvlcek(a)redhat.com&gt; > Cc: "Oved Ourfalli" <ovedo(a)redhat.com&gt;, engine-devel(a)ovirt.org, "Michal Fojtik" <mfojtik(a)redhat.com&gt;, "David > Lutterkort" <dlutter(a)redhat.com&gt; > Sent: Monday, January 23, 2012 10:07:30 AM > Subject: Re: [Engine-devel] VM Payload feature > > On 23.01.12 09:39, Joseph VLcek wrote: > > > > On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote: > > > > > > > > > > > ----- Original Message ----- > > >> From: "Ayal Baron" <abaron(a)redhat.com&gt; > > >> To: "Oved Ourfalli" <ovedo(a)redhat.com&gt; > > >> Cc: engine-devel(a)ovirt.org > > >> Sent: Thursday, January 19, 2012 4:05:08 PM > > >> Subject: Re: [Engine-devel] VM Payload feature > > >> > > >> > > >> > > >> ----- Original Message ----- > > >>> Hey all, > > >>> > > >>> Continuing the discussion about Aeolus instance data injection > > >>> to a > > >>> VM > > >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html) > > >>> we propose a new VM Payload feature. > > >>> > > >>> The following wiki page contains a description page of the > > >>> feature. > > >>> http://www.ovirt.org/wiki/Features/VMPayload > > >>> > > >>> Please read and review. > > >>> There are several approaches there, and we wish to head your > > >>> opinions > > >>> and thoughts about them. > > >>> > > >>> Once we agree on an approach, we will start designing. > > >> > > >> Permanent payload availability requires determining where the > > >> payload > > >> is stored. > > >> Makes sense to me to store it together with the VM disks on the > > >> storage domain, but that requires the small object store which > > >> will > > >> not be available in the coming version (payloads can be large > > >> and > > >> keeping them in the DB and passing over the net every time the > > >> VM is > > >> run doesn't make much sense). > > >> > > > I guess we can start with storing it in the database, with some > > > size limitation, and move it to the storage domain later on. > > > > > >> Wrt availability, I don't see a reason to exclude attaching both > > >> a CD > > >> and a payload via another CD at the same time (i.e. multiple > > >> devices). > > >> > > >>> > > >>> Thank you, > > >>> Oved > > >>> _______________________________________________ > > >>> Engine-devel mailing list > > >>> Engine-devel(a)ovirt.org > > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel > > >>> > > >> _______________________________________________ > > >> Engine-devel mailing list > > >> Engine-devel(a)ovirt.org > > >> http://lists.ovirt.org/mailman/listinfo/engine-devel > > >> > > > > > > > > My perspective is that of the end user, the instance retrieving the > > data. > > > > From a functional standpoint I would like to see similar > > performance to > > what EC2 provides. AWS EC2 user data is limited to 16K. This limit > > applies to the data in raw form, not base64 encoded form. > > see: > > http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-da... > > > > I am concerned about the 512k limit as mentioned in the notes > > of: http://www.ovirt.org/wiki/Features/VMPayload > > "if the content of the file is bigger the 512K it will pass an nfs > > share for vdsm to fetch the file/s" > > > > Please confirm: > > - Will it be possible to pass user data to larger than 512k? > > - If so what will the instance need to do in order to retrieve > > user-data bigger than 512k. > > - What will the MAX size supported for the user-data? > 512k is a suggestion, > we don't want to embed large files in the verb that ovirt calls vdsm, > instead > if it bigger then 512k/1M we will pass urls/nfs path of the files and > VDSM > will add them to the iso file. > there is not limitation of size... If we're talking about URLs keep in mind SELinux restrictions (eg. passing a URL to a HTTP hosted IS will be blocked, iirc)

> > > > Thank you. > > Joe VLcek > > > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel > _______________________________________________ Engine-devel mailing list Engine-devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel

On 01/22/2012 08:42 PM, Ayal Baron wrote:

>>>> The following wiki page contains a description page of the >>>> feature. >>>> http://www.ovirt.org/wiki/Features/VMPayload >>>>

Currently it seems that there are too many options for it - floppy, cd, nfs and maybe more. It would be nice to have a single option that works for all cases. How about creating something like s3 compatible storage access that the guest can access? If you need boot time access then I'll recommend cdrom or plain virtio-blk.

On Wed, 2012-01-25 at 12:02 +0200, Itamar Heim wrote: > On 01/25/2012 11:09 AM, Dor Laor wrote: >> On 01/22/2012 08:42 PM, Ayal Baron wrote: > ... > >>>>>> The following wiki page contains a description page of the >>>>>> feature. >>>>>> http://www.ovirt.org/wiki/Features/VMPayload >>>>>> > ... >> >> Currently it seems that there are too many options for it - floppy, cd, >> nfs and maybe more. It would be nice to have a single option that works >> for all cases. How about creating something like s3 compatible storage >> access that the guest can access? If you need boot time access then I'll >> recommend cdrom or plain virtio-blk. I agree with Dor that there seems to be a large number of options here. From the Aeolus and Deltacloud perspective, we only need something that makes that information available fairly late during boot (certainly until after file systems have been mounted, even after network start isn't a deal killer) The payload data that the VM sees should not change when the VM is rebooted or stopped/started. > I think there are different use cases here: > 1. floppy and iso cover the same use case, for similar needs (and behave > the same). this would cover windows sysprep approach and basic > attachment of files Just picking one or the other should be sufficient. > 2. http://192.169.192.168 - this would provide compatibility to cloud > approaches iiuc Except the address is 169.254.169.254 (link-local) ;) > 3. injecting into the file system - this covers various other needs, > like injecting ssh key, and is relevant not only during bootstrap, > should we want to allow editing a guest when it is down to troubleshoot > issues. You don't need that as a feature for troubleshooting; I've unmangled EBS root volumes in AWS before simply by mounting the EBS disk from another machine. The thing I don't like about file injection is that it's inherently fragile, since oVirt needs to understand the intricacies of disk layout, volume manager (as in lvm), and filesystem.

Even worse if it is exposed via the API so that you can provide target paths - now you've tightly coupled your API user to the OS inside the VM.

I would only entertain (3) if there is an absolutely compelling use case to do it.

We encounter a problem with persisting the content to engine database (we don't want to save the file the database). There are some solution for that: 1. we do want to persist files to the database (with size limitation). 2. engine can expect file path (nfs or http) and persist only the file url 3. we can use this feature in run-once only hence no persistence is needed. Thoughts, other ideas?

On 23.01.12 17:12, Shahar Havivi wrote: > On 23.01.12 10:11, Andrew Cathrow wrote: > > > > > > ----- Original Message ----- > > > From: "Shahar Havivi" <shaharh(a)redhat.com&gt; > > > To: "Joseph VLcek" <jvlcek(a)redhat.com&gt; > > > Cc: "Oved Ourfalli" <ovedo(a)redhat.com&gt;, engine-devel(a)ovirt.org, "Michal Fojtik" <mfojtik(a)redhat.com&gt;, "David > > > Lutterkort" <dlutter(a)redhat.com&gt; > > > Sent: Monday, January 23, 2012 10:07:30 AM > > > Subject: Re: [Engine-devel] VM Payload feature > > > > > > On 23.01.12 09:39, Joseph VLcek wrote: > > > > > > > > On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote: > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > >> From: "Ayal Baron" <abaron(a)redhat.com&gt; > > > > >> To: "Oved Ourfalli" <ovedo(a)redhat.com&gt; > > > > >> Cc: engine-devel(a)ovirt.org > > > > >> Sent: Thursday, January 19, 2012 4:05:08 PM > > > > >> Subject: Re: [Engine-devel] VM Payload feature > > > > >> > > > > >> > > > > >> > > > > >> ----- Original Message ----- > > > > >>> Hey all, > > > > >>> > > > > >>> Continuing the discussion about Aeolus instance data injection > > > > >>> to a > > > > >>> VM > > > > >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html) > > > > >>> we propose a new VM Payload feature. > > > > >>> > > > > >>> The following wiki page contains a description page of the > > > > >>> feature. > > > > >>> http://www.ovirt.org/wiki/Features/VMPayload > > > > >>> > > > > >>> Please read and review. > > > > >>> There are several approaches there, and we wish to head your > > > > >>> opinions > > > > >>> and thoughts about them. > > > > >>> > > > > >>> Once we agree on an approach, we will start designing. > > > > >> > > > > >> Permanent payload availability requires determining where the > > > > >> payload > > > > >> is stored. > > > > >> Makes sense to me to store it together with the VM disks on the > > > > >> storage domain, but that requires the small object store which > > > > >> will > > > > >> not be available in the coming version (payloads can be large > > > > >> and > > > > >> keeping them in the DB and passing over the net every time the > > > > >> VM is > > > > >> run doesn't make much sense). > > > > >> > > > > > I guess we can start with storing it in the database, with some > > > > > size limitation, and move it to the storage domain later on. > > > > > > > > > >> Wrt availability, I don't see a reason to exclude attaching both > > > > >> a CD > > > > >> and a payload via another CD at the same time (i.e. multiple > > > > >> devices). > > > > >> > > > > >>> > > > > >>> Thank you, > > > > >>> Oved > > > > >>> _______________________________________________ > > > > >>> Engine-devel mailing list > > > > >>> Engine-devel(a)ovirt.org > > > > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel > > > > >>> > > > > >> _______________________________________________ > > > > >> Engine-devel mailing list > > > > >> Engine-devel(a)ovirt.org > > > > >> http://lists.ovirt.org/mailman/listinfo/engine-devel > > > > >> > > > > > > > > > > > > > > > > My perspective is that of the end user, the instance retrieving the > > > > data. > > > > > > > > From a functional standpoint I would like to see similar > > > > performance to > > > > what EC2 provides. AWS EC2 user data is limited to 16K. This limit > > > > applies to the data in raw form, not base64 encoded form. > > > > see: > > > > http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-da... > > > > > > > > I am concerned about the 512k limit as mentioned in the notes > > > > of: http://www.ovirt.org/wiki/Features/VMPayload > > > > "if the content of the file is bigger the 512K it will pass an nfs > > > > share for vdsm to fetch the file/s" > > > > > > > > Please confirm: > > > > - Will it be possible to pass user data to larger than 512k? > > > > - If so what will the instance need to do in order to retrieve > > > > user-data bigger than 512k. > > > > - What will the MAX size supported for the user-data? > > > 512k is a suggestion, > > > we don't want to embed large files in the verb that ovirt calls vdsm, > > > instead > > > if it bigger then 512k/1M we will pass urls/nfs path of the files and > > > VDSM > > > will add them to the iso file. > > > there is not limitation of size... > > > > If we're talking about URLs keep in mind SELinux restrictions (eg. passing a URL to a HTTP hosted IS will be blocked, iirc) > right, > its proffered to be common share directory > > > > > > > > > > Thank you. > > > > Joe VLcek > > > > > > > _______________________________________________ > > > Engine-devel mailing list > > > Engine-devel(a)ovirt.org > > > http://lists.ovirt.org/mailman/listinfo/engine-devel > > > > > _______________________________________________ > > Engine-devel mailing list > > Engine-devel(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/engine-devel

On 02/27/2012 05:45 PM, Dan Kenigsberg wrote: > On Mon, Feb 27, 2012 at 04:19:02PM +0200, Shahar Havivi wrote: >> We encounter a problem with persisting the content to engine database (we don't >> want to save the file the database). >> >> There are some solution for that: >> 1. we do want to persist files to the database (with size limitation). >> 2. engine can expect file path (nfs or http) and persist only the file url >> 3. we can use this feature in run-once only hence no persistence is needed. i think size limitation should be ok.

looking forward to next phase of supporting the cloud based api for guests which david mentioned earlier in the thread, vm payload should allow passing things like guest keypair files. looking at the API[1] david mentioned, i didn't notice files other than keypair's, but i may have missed them. the rest of the API seems to be based on data that can be retrieved from the DB. David - any more insight as to when/how files are needed?

(btw, can you please remind where in the metadata do you pass extra information aelous needs?)

>> >> Thoughts, other ideas? > > Let's re-ask the question about the (Engine) API: Do we need the payload > to be passed on VM definition? Or is it enough to pass it on VM startup? my view is vm definition, not only startup.

On 02/27/2012 09:14 PM, David Lutterkort wrote: > On Mon, 2012-02-27 at 18:22 +0200, Itamar Heim wrote: >> On 02/27/2012 05:45 PM, Dan Kenigsberg wrote: >>> On Mon, Feb 27, 2012 at 04:19:02PM +0200, Shahar Havivi wrote: >>>> >>> Let's re-ask the question about the (Engine) API: Do we need the payload >>> to be passed on VM definition? Or is it enough to pass it on VM startup? >> >> my view is vm definition, not only startup. > > For a cloud, you generally want to launch multiple VM's off the same > image, and therefore need to be able to specify the user data for each > VM. Whether that happens when you define or launch the VM isn't all that > important, though it would be a little friendlier to users to do it at > VM start, since they then do not need to have the user data ready (or > know it) when they define a VM. well, true for cloud when image start is also creating the instance, while in ovirt, at least for now, you need to create the VM then launch it. i agree you need to be able to pass this at VM start, but considering VM are mostly stateful, i don't see why you wouldn't let user persist this like the VM is presistent.