[Engine-devel] VM Payload feature

[Shahar Havivi]

We encounter a problem with persisting the content to engine database (we don't
want to save the file the database).

There are some solution for that:
1. we do want to persist files to the database (with size limitation).
2. engine can expect file path (nfs or http) and persist only the file url
3. we can use this feature in run-once only hence no persistence is needed.

Thoughts, other ideas?


On 23.01.12 17:12, Shahar Havivi wrote:
> On 23.01.12 10:11, Andrew Cathrow wrote:
> > 
> > 
> > ----- Original Message -----
> > > From: "Shahar Havivi" <shaharh at redhat.com>
> > > To: "Joseph VLcek" <jvlcek at redhat.com>
> > > Cc: "Oved Ourfalli" <ovedo at redhat.com>, engine-devel at ovirt.org, "Michal Fojtik" <mfojtik at redhat.com>, "David
> > > Lutterkort" <dlutter at redhat.com>
> > > Sent: Monday, January 23, 2012 10:07:30 AM
> > > Subject: Re: [Engine-devel] VM Payload feature
> > > 
> > > On 23.01.12 09:39, Joseph VLcek wrote:
> > > > 
> > > > On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote:
> > > > 
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > > >> To: "Oved Ourfalli" <ovedo at redhat.com>
> > > > >> Cc: engine-devel at ovirt.org
> > > > >> Sent: Thursday, January 19, 2012 4:05:08 PM
> > > > >> Subject: Re: [Engine-devel] VM Payload feature
> > > > >> 
> > > > >> 
> > > > >> 
> > > > >> ----- Original Message -----
> > > > >>> Hey all,
> > > > >>> 
> > > > >>> Continuing the discussion about Aeolus instance data injection
> > > > >>> to a
> > > > >>> VM
> > > > >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> > > > >>> we propose a new VM Payload feature.
> > > > >>> 
> > > > >>> The following wiki page contains a description page of the
> > > > >>> feature.
> > > > >>> http://www.ovirt.org/wiki/Features/VMPayload
> > > > >>> 
> > > > >>> Please read and review.
> > > > >>> There are several approaches there, and we wish to head your
> > > > >>> opinions
> > > > >>> and thoughts about them.
> > > > >>> 
> > > > >>> Once we agree on an approach, we will start designing.
> > > > >> 
> > > > >> Permanent payload availability requires determining where the
> > > > >> payload
> > > > >> is stored.
> > > > >> Makes sense to me to store it together with the VM disks on the
> > > > >> storage domain, but that requires the small object store which
> > > > >> will
> > > > >> not be available in the coming version (payloads can be large
> > > > >> and
> > > > >> keeping them in the DB and passing over the net every time the
> > > > >> VM is
> > > > >> run doesn't make much sense).
> > > > >> 
> > > > > I guess we can start with storing it in the database, with some
> > > > > size limitation, and move it to the storage domain later on.
> > > > > 
> > > > >> Wrt availability, I don't see a reason to exclude attaching both
> > > > >> a CD
> > > > >> and a payload via another CD at the same time (i.e. multiple
> > > > >> devices).
> > > > >> 
> > > > >>> 
> > > > >>> Thank you,
> > > > >>> Oved
> > > > >>> _______________________________________________
> > > > >>> Engine-devel mailing list
> > > > >>> Engine-devel at ovirt.org
> > > > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > >>> 
> > > > >> _______________________________________________
> > > > >> Engine-devel mailing list
> > > > >> Engine-devel at ovirt.org
> > > > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > >> 
> > > > 
> > > > 
> > > > 
> > > > My perspective is that of the end user, the instance retrieving the
> > > > data.
> > > > 
> > > > From a functional standpoint I would like to see similar
> > > > performance to
> > > > what EC2 provides. AWS EC2 user data is limited to 16K. This limit
> > > > applies to the data in raw form, not base64 encoded form.
> > > > see:
> > > > http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html
> > > > 
> > > > I am concerned about the 512k limit as mentioned in the notes
> > > > of: http://www.ovirt.org/wiki/Features/VMPayload
> > > > "if the content of the file is bigger the 512K it will pass an nfs
> > > > share for vdsm to fetch the file/s"
> > > > 
> > > > Please confirm:
> > > > - Will it be possible to pass user data to larger than 512k?
> > > > - If so what will the instance need to do in order to retrieve
> > > >    user-data bigger than 512k.
> > > > - What will the MAX size supported for the user-data?
> > > 512k is a suggestion,
> > > we don't want to embed large files in the verb that ovirt calls vdsm,
> > > instead
> > > if it bigger then 512k/1M we will pass urls/nfs path of the files and
> > > VDSM
> > > will add them to the iso file.
> > > there is not limitation of size...
> > 
> > If we're talking about URLs keep in mind SELinux restrictions (eg. passing a URL to a HTTP hosted IS will be blocked, iirc)
> right,
> its proffered to be common share directory
> > 
> > > > 
> > > > Thank you.
> > > >     Joe VLcek
> > > > 
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel