Re: [Engine-devel] Storage Device Management in VDSM and oVirt

On Wed, Apr 18, 2012 at 09:06:36AM -0400, Ayal Baron wrote: > > > ----- Original Message ----- > > On Tue, Apr 17, 2012 at 03:38:25PM +0530, Shireesh Anjal wrote: > > > Hi all, > > > > > > As part of adding Gluster support in ovirt, we need to > > > introduce > > > some Storage Device management capabilities (on the host). > > > Since > > > these are quite generic and not specific to Gluster as such, we > > > think it might be useful to add it as a core vdsm and oVirt > > > feature. > > > At a high level, this involves following: > > > > > > - A "Storage Devices" sub-tab on "Host" entity, displaying > > > information about all the storage devices* > > > - Listing of different types of storage devices of a host > > > - Regular Disks and Partitions* > > > - LVM* > > > - Software RAID* > > > - Various actions related to device configuration > > > - Partition disks* > > > - Format and mount disks / partitions* > > > - Create, resize and delete LVM Volume Groups (VGs) > > > - Create, resize, delete, format and mount LVM Logical > > > Volumes > > > (LVs) > > > - Create, resize, delete, partition, format and mount > > > Software > > > RAID devices > > > - Edit properties of the devices > > > - UI can be modeled similar to the system-config-lvm tool > > > > > > The items marked with (*) in above list are urgently required > > > for > > > the Gluster feature, and will be developed first. > > > > > > Comments / inputs welcome. > > > > This seems like a big undertaking, and I would like to understand > > the > > complete use case of this. Is it intended to create the block > > storage > > devices on top of which a Gluster volume will be created? > > Yes, but not only. > It could also be used to create the file system on top of which you > create a local storage domain (just an example, there are many > others, more listed below). > > > > > I must tell that we had a bad experience with exposing low level > > commands over the Vdsm API: A Vdsm storage domain is a VG with > > some > > metadata on top. We used to have two API calls for creating a > > storage > > domain: one to create the VG and one to add the metadata and make > > it > > an > > SD. But it is pretty hard to handle all the error cases remotely. > > It > > proved more useful to have one atomic command for the whole > > sequence. > > > > I suspect that this would be the case here, too. I'm not sure if > > using > > Vdsm as an ssh-replacement for transporting lvm/md/fdisk commands > > is > > the > > best approach. > > I agree, we should either provide added value or figure out a way > where we don't need to simply add a verb every time the underlying > APIs added something. > > > > > It may be better to have a single verb for creating Gluster > > volume > > out > > of block storage devices. Something like: "take these disks, > > partition > > them, build a raid, cover with a vg, carve some PVs and make each > > of > > them a Gluster volume". > > > > Obviously, it is not simple to define a good language to describe > > a > > general architecture of a Gluster voluem. But it would have to be > > done > > somewhere - if not in Vdsm then in Engine; and I suspect it would > > be > > better done on the local host, not beyond a fragile network link. > > > > Please note that currently, Vdsm makes a lot of effort not to > > touch > > LVM > > metadata of existing VGs on regular "HSM" hosts. All such > > operations > > are > > done on the engine-selected "SPM" host. When implementing this, > > we > > must > > bear in mind these safeguards and think whether we want to break > > them. > > I'm not sure I see how this is relevant, we allow creating a VG on > any host today and that isn't going to change... We have one painful exception, that alone is no reason to add more. Note that currently, Engine uses the would-be-spm for vg creation. In the gluster use case, any host is expected to do this on async timing. It might be required, but it's not warm and fuzzy.

> > In general, we know that we already need to support using a LUN > even if it has partitions on it (with force or something). > > We know that we have requirements for more control over the VG that > we create e.g. support striping, control over max LV size (limited > by pv extent size today) etc. > > We also know that users would like a way not only to use a local > dir for a storage domain but also create the directory + fs? These three examples are storage domain flavors..

> > We know that in the gLuster use case we would like the ability to > setup samba over the gluster volume as well as iSCSI probably. Now I do not see the relevance. Configuring gluster and how it exposes its volume is something other than preparing block storage for gluster bricks.

> > So although I believe that when we create a gluster volume or an > ovirt storage domain then indeed we shouldn't need a lot of low > level commands, but it would appear to me that not allowing for > more control when needed is not going to work and that there are > enough use cases which do not involve a gluster volume nor a > storage domain to warrant this to be generic. I'm not against more control; I'm against uncontrollable API such as runThisLvmCommandAsRoot()