[Engine-devel] [vdsm] RFC: Writeup on VDSM-libstoragemgmt integration

Shu Ming shuming at linux.vnet.ibm.com
Mon Jun 18 15:56:18 UTC 2012


On 2012-5-30 17:38, Deepak C Shetty wrote:
> Hello All,
>
>     I have a draft write-up on the VDSM-libstoragemgmt integration.
> I wanted to run this thru' the mailing list(s) to help tune and 
> crystallize it, before putting it on the ovirt wiki.
> I have run this once thru Ayal and Tony, so have some of their 
> comments incorporated.
>
> I still have few doubts/questions, which I have posted below with 
> lines ending with '?'
>
> Comments / Suggestions are welcome & appreciated.
>
> thanx,
> deepak
>
> [Ccing engine-devel and libstoragemgmt lists as this stuff is relevant 
> to them too]
>
> -------------------------------------------------------------------------------------------------------------- 
>
>
> 1) Background:
>
> VDSM provides high level API for node virtualization management. It 
> acts in response to the requests sent by oVirt Engine, which uses VDSM 
> to do all node virtualization related tasks, including but not limited 
> to storage management.
>
> libstoragemgmt aims to provide vendor agnostic API for managing 
> external storage array. It should help system administrators utilizing 
> open source solutions have a way to programmatically manage their 
> storage hardware in a vendor neutral way. It also aims to facilitate 
> management automation, ease of use and take advantage of storage 
> vendor supported features which improve storage performance and space 
> utilization.
>
> Home Page: http://sourceforge.net/apps/trac/libstoragemgmt/
>
> libstoragemgmt (LSM) today supports C and python plugins for talking 
> to external storage array using SMI-S as well as native interfaces 
> (eg: netapp plugin )
> Plan is to grow the SMI-S interface as needed over time and add more 
> vendor specific plugins for exploiting features not possible via SMI-S 
> or have better alternatives than using SMI-S.
> For eg: Many of the copy offload features require to use vendor 
> specific commands, which justifies the need for a vendor specific plugin.
>
>
> 2) Goals:
>
>     2a) Ability to plugin external storage array into oVirt/VDSM 
> virtualization stack, in a vendor neutral way.
>
>     2b) Ability to list features/capabilities and other statistical 
> info of the array
>
>     2c) Ability to utilize the storage array offload capabilities from 
> oVirt/VDSM.
>
>
> 3) Details:
>
> LSM will sit as a new repository engine in VDSM.
> VDSM Repository Engine WIP @ http://gerrit.ovirt.org/#change,192
>
> Current plan is to have LSM co-exist with VDSM on the virtualization 
> nodes.

Does that mean LSM will be a different daemon process than VDSM?  Also, 
how about the vendor's plugin, another process in the nodes?

>
> *Note : 'storage' used below is generic. It can be a file/nfs-export 
> for NAS targets and LUN/logical-drive for SAN targets.
>
> VDSM can use LSM and do the following...
>     - Provision storage
>     - Consume storage
>
> 3.1) Provisioning Storage using LSM
>
> Typically this will be done by a Storage administrator.
>
> oVirt/VDSM should provide storage admin the
>     - ability to list the different storage arrays along with their 
> types (NAS/SAN), capabilities, free/used space.
>     - ability to provision storage using any of the array capabilities 
> (eg: thin provisioned lun or new NFS export )
>     - ability to manage the provisioned storage (eg: resize/delete 
> storage)
>
> Once the storage is provisioned by the storage admin, VDSM will have 
> to refresh the host(s) for them to be able to see the newly 
> provisioned storage.
>
> 3.1.1) Potential flows:
>
> Mgmt -> vdsm -> lsm: create LUN + LUN Mapping / Zoning / whatever is 
> needed to make LUN available to list of hosts passed by mgmt
> Mgmt -> vdsm: getDeviceList (refreshes host and gets list of devices)
>  Repeat above for all relevant hosts (depending on list passed 
> earlier, mostly relevant when extending an existing VG)
> Mgmt -> use LUN in normal flows.
>
>
> 3.1.2) How oVirt Engine will know which LSM to use ?
>
> Normally the way this works today is that user can choose the host to 
> use (default today is SPM), however there are a few flows where mgmt 
> will know which host to use:
> 1. extend storage domain (add LUN to existing VG) - Use SPM and make 
> sure *all* hosts that need access to this SD can see the new LUN
> 2. attach new LUN to a VM which is pinned to a specific host - use 
> this host
> 3. attach new LUN to a VM which is not pinned - use a host from the 
> cluster the VM belongs to and make sure all nodes in cluster can see 
> the new LUN

So this model depend on the work of removing storage pool?

>
> Flows for which there is no clear candidate (Maybe we can use the SPM 
> host itself which is the default ?)
> 1. create a new disk without attaching it to any VM

So the new floating disk should be exported to all nodes and all VMs?

> 2. create a LUN for a new storage domain
>
>
> 3.2) Consuming storage using LSM
>
> Typically this will be done by a virtualization administrator
>
> oVirt/VDSM should allow virtualization admin to
>     - Create a new storage domain using the storage on the array.
>     - Be able to specify whether VDSM should use the storage offload 
> capability (default) or override it to use its own internal logic.
>
> 4) VDSM potential changes:
>
> 4.1) How to represent a VM disk, 1 LUN = 1 VMdisk or 1 LV = 1 VMdisk ? 
> which bring another question...1 array == 1 storage domain OR 1 
> LUN/nfs-export on the array == 1 storage domain ?
>
> Pros & Cons of each...
>
> 1 array == 1 storage domain
>     - Each new vmdisk (aka volume) will be a new lun/file on the array.
>     - Easier to exploit offload capabilities, as they are available at 
> the LUN/File granularity
>     - Will there be any issues where there will be too many LUNs/Files 
> ... any maxluns limit on linux hosts that we might hit ?
>         -- VDSM has been tested with 1K LUNs and it worked fine - ayal
>     - Storage array limitations on the number of LUNs can be a 
> downside here.
>     - Would it be ok to share the array for hosting another storage 
> domain if need be ?
>         -- Provided the existing domain is not utilising all of the 
> free space
>         -- We can create new LUNs and hand it over to anyone needed ?
>     -- Changes needed in VDSM to work with raw LUNs, today it only has 
> support for consuming LUNs via VG/LV.
>
> 1 LUN/nfs-export on the array == 1 storage domain
>     - How to represent a new vmdisk (aka vdsm volume) if its a LUN 
> provisioned using SAN target ?
>         -- Will it be VG/LV as is done today for block domains ?
>         -- If yes, then it will be difficult to exploit offload 
> capabilities, as they are at LUN level, not at LV level.
>     - Each new vmdisk will be a new file on the nfs-export, assuming 
> offload capability is available at the file level, so this should work 
> for NAS targets ?
>     - Can use the storage array for hosting multiple storage domains.
>         -- Provision one more LUN and use it for another storage 
> domain if need be.
>     - VDSM already supports this today, as part of block storage 
> domains for LUNs case.
>
> Note that we will allow user to do either one of the two options 
> above, depending on need.
>
> 4.2) Storage domain metadata will also include the 
> features/capabilities of the storage array as reported by LSM.
>         - Capabilities (taken via LSM) will be stored in the domain 
> metadata during storage domain create flow.
>         - Need changes in oVirt engine as well ( see 'oVirt Engine 
> potential changes' section below )
>
> 4.3) VDSM to poll LSM for array capabilities on a regular basis ?
> Per ayal:
>     - If we have a 'storage array' entity in oVirt Engine (see 'oVirt 
> Engine potential changes' section below ) then we can have a 'refresh 
> capabilities' button/verb.
>     - We can periodically query the storage array.
>     - Query LSM before running operations (sounds redundant to me, but 
> if it's cheap enough it could be simplest).
>
>     Probably need a combination of 1+2 (query at very low frequency - 
> 1/hour or 1/day + refresh button)
>
>
> 5) oVirt Engine potential changes - as described by ayal :
>
>     - We will either need a new 'storage array' entity in engine to 
> keep credentials, or, in case of storage array as storage domain, just 
> keep this info as part of the domain at engine level.
>     - Have a 'storage array' entity in oVirt Engine to support  
> 'refresh capabilities' as a button/verb.
>     - When user during storage provisioning, selects a LUN exported 
> from a storage array (via LSM), the oVirt Engine would know from then 
> onwards that this LUN is being served via LSM.
>       It would then  be able to query the  capabilities of the LUN and 
> show it to the virt admin during storage consumption flow.
>
> 6) Potential flows:
>     - Create snapshot flow
>         -- VDSM will check the snapshot offload capability in the 
> domain metadata
>         -- If available, and override is not configured, it will use 
> LSM to offload LUN/File snapshot

If a LSM try to snapshot a running volume, does that mean all the IO 
activity to the volume will be blocked when the snapshot is undergoing?

>         -- If override is configured or capability is not available, 
> it will use its internal logic to create
>            snapshot (qcow2).
>
>     - Copy/Clone vmdisk flow
>         -- VDSM will check the copy offload capability in the domain 
> metadata
>         -- If available, and override is not configured, it will use 
> LSM to offload LUN/File copy
>         -- If override is configured or capability is not available, 
> it will use its internal logic to create
>            snapshot (eg: dd cmd in case of LUN).
>
> 7) LSM potential changes:
>
>     - list features/capabilities of the array. Eg: copy offload, thin 
> prov. etc.
>     - list containers (aka pools) (present in LSM today)
>     - Ability to list different types of arrays being managed, their 
> capabilities and used/free space
>     - Ability to create/list/delete/resize volumes ( LUN or exports, 
> available in LSM as of today)
>     - Get monitoring info with object (LUN/snapshot/volume) as 
> optional parameter for specific info. eg: container/pool free/used 
> space, raid type etc.
>
> Need to make sure above info is listed in a coherent way across arrays 
> (number of LUNs, raid type used? free/total per container/pool, per 
> LUN?. Also need I/O statistics wherever possible.
>
>
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel


-- 
Shu Ming<shuming at linux.vnet.ibm.com>
IBM China Systems and Technology Laboratory





More information about the Engine-devel mailing list