[Engine-devel] VM Payload feature

[Dan Kenigsberg]

On Mon, Feb 27, 2012 at 04:19:02PM +0200, Shahar Havivi wrote:
> We encounter a problem with persisting the content to engine database (we don't
> want to save the file the database).
> 
> There are some solution for that:
> 1. we do want to persist files to the database (with size limitation).
> 2. engine can expect file path (nfs or http) and persist only the file url
> 3. we can use this feature in run-once only hence no persistence is needed.
> 
> Thoughts, other ideas?

Let's re-ask the question about the (Engine) API: Do we need the payload
to be passed on VM definition? Or is it enough to pass it on VM startup?

> 
> 
> On 23.01.12 17:12, Shahar Havivi wrote:
> > On 23.01.12 10:11, Andrew Cathrow wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Shahar Havivi" <shaharh at redhat.com>
> > > > To: "Joseph VLcek" <jvlcek at redhat.com>
> > > > Cc: "Oved Ourfalli" <ovedo at redhat.com>, engine-devel at ovirt.org, "Michal Fojtik" <mfojtik at redhat.com>, "David
> > > > Lutterkort" <dlutter at redhat.com>
> > > > Sent: Monday, January 23, 2012 10:07:30 AM
> > > > Subject: Re: [Engine-devel] VM Payload feature
> > > > 
> > > > On 23.01.12 09:39, Joseph VLcek wrote:
> > > > > 
> > > > > On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote:
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > ----- Original Message -----
> > > > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > > > >> To: "Oved Ourfalli" <ovedo at redhat.com>
> > > > > >> Cc: engine-devel at ovirt.org
> > > > > >> Sent: Thursday, January 19, 2012 4:05:08 PM
> > > > > >> Subject: Re: [Engine-devel] VM Payload feature
> > > > > >> 
> > > > > >> 
> > > > > >> 
> > > > > >> ----- Original Message -----
> > > > > >>> Hey all,
> > > > > >>> 
> > > > > >>> Continuing the discussion about Aeolus instance data injection
> > > > > >>> to a
> > > > > >>> VM
> > > > > >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> > > > > >>> we propose a new VM Payload feature.
> > > > > >>> 
> > > > > >>> The following wiki page contains a description page of the
> > > > > >>> feature.
> > > > > >>> http://www.ovirt.org/wiki/Features/VMPayload
> > > > > >>> 
> > > > > >>> Please read and review.
> > > > > >>> There are several approaches there, and we wish to head your
> > > > > >>> opinions
> > > > > >>> and thoughts about them.
> > > > > >>> 
> > > > > >>> Once we agree on an approach, we will start designing.
> > > > > >> 
> > > > > >> Permanent payload availability requires determining where the
> > > > > >> payload
> > > > > >> is stored.
> > > > > >> Makes sense to me to store it together with the VM disks on the
> > > > > >> storage domain, but that requires the small object store which
> > > > > >> will
> > > > > >> not be available in the coming version (payloads can be large
> > > > > >> and
> > > > > >> keeping them in the DB and passing over the net every time the
> > > > > >> VM is
> > > > > >> run doesn't make much sense).
> > > > > >> 
> > > > > > I guess we can start with storing it in the database, with some
> > > > > > size limitation, and move it to the storage domain later on.
> > > > > > 
> > > > > >> Wrt availability, I don't see a reason to exclude attaching both
> > > > > >> a CD
> > > > > >> and a payload via another CD at the same time (i.e. multiple
> > > > > >> devices).
> > > > > >> 
> > > > > >>> 
> > > > > >>> Thank you,
> > > > > >>> Oved
> > > > > >>> _______________________________________________
> > > > > >>> Engine-devel mailing list
> > > > > >>> Engine-devel at ovirt.org
> > > > > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > > >>> 
> > > > > >> _______________________________________________
> > > > > >> Engine-devel mailing list
> > > > > >> Engine-devel at ovirt.org
> > > > > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > > >> 
> > > > > 
> > > > > 
> > > > > 
> > > > > My perspective is that of the end user, the instance retrieving the
> > > > > data.
> > > > > 
> > > > > From a functional standpoint I would like to see similar
> > > > > performance to
> > > > > what EC2 provides. AWS EC2 user data is limited to 16K. This limit
> > > > > applies to the data in raw form, not base64 encoded form.
> > > > > see:
> > > > > http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html
> > > > > 
> > > > > I am concerned about the 512k limit as mentioned in the notes
> > > > > of: http://www.ovirt.org/wiki/Features/VMPayload
> > > > > "if the content of the file is bigger the 512K it will pass an nfs
> > > > > share for vdsm to fetch the file/s"
> > > > > 
> > > > > Please confirm:
> > > > > - Will it be possible to pass user data to larger than 512k?
> > > > > - If so what will the instance need to do in order to retrieve
> > > > >    user-data bigger than 512k.
> > > > > - What will the MAX size supported for the user-data?
> > > > 512k is a suggestion,
> > > > we don't want to embed large files in the verb that ovirt calls vdsm,
> > > > instead
> > > > if it bigger then 512k/1M we will pass urls/nfs path of the files and
> > > > VDSM
> > > > will add them to the iso file.
> > > > there is not limitation of size...
> > > 
> > > If we're talking about URLs keep in mind SELinux restrictions (eg. passing a URL to a HTTP hosted IS will be blocked, iirc)
> > right,
> > its proffered to be common share directory
> > > 
> > > > > 
> > > > > Thank you.
> > > > >     Joe VLcek
> > > > > 
> > > > _______________________________________________
> > > > Engine-devel mailing list
> > > > Engine-devel at ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > 
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel