I've been throwing a lot of bits out about the new storage API and I think it's time to talk a bit. I will purposefully try and keep implementation details away and concentrate about how the API looks and how you use it. First major change is in terminology, there is no long a storage domain but a storage repository. This change is done because so many things are already called domain in the system and this will make things less confusing for new-commers with a libvirt background. One other changes is that repositories no longer have a UUID. The UUID was only used in the pool members manifest and is no longer needed. connectStorageRepository(repoId, repoFormat, connectionParameters={}):

repoId - is a transient name that will be used to refer to the connected domain, it is not persisted and doesn't have to be the same across the cluster. repoFormat - Similar to what used to be type (eg. localfs-1.0, nfs-3.4, clvm-1.2). connectionParameters - This is format specific and will used to tell VDSM how to connect to the repo. disconnectStorageRepository(self, repoId):

In the new API there are only images, some images are mutable and some are not. mutable images are also called VirtualDisks immutable images are also called Snapshots

There are no explicit templates, you can create as many images as you want from any snapshot. There are 4 major image operations: createVirtualDisk(targetRepoId, size, baseSnapshotId=None, userData={}, options={}):

targetRepoId - ID of a connected repo where the disk will be created size - The size of the image you wish to create baseSnapshotId - the ID of the snapshot you want the base the new virtual disk on userData - optional data that will be attached to the new VD, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new VD createSnapshot(targetRepoId, baseVirtualDiskId, userData={}, options={}): targetRepoId - The ID of a connected repo where the new sanpshot will be created and the original image exists as well. size - The size of the image you wish to create

baseVirtualDisk - the ID of a mutable image (Virtual Disk) you want to snapshot

userData - optional data that will be attached to the new Snapshot, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new Snapshot copyImage(targetRepoId, imageId, baseImageId=None, userData={}, options={}) targetRepoId - The ID of a connected repo where the new image will be created imageId - The image you wish to copy

baseImageId - if specified, the new image will contain only the diff between image and Id. If None the new image will contain all the bits of image Id. This can be used to copy partial parts of images for export. userData - optional data that will be attached to the new image, could be anything that the user desires. options - options to modify VDSMs default behavior return the Id of the new image. In case of copying an immutable image the ID will be identical to the original image as they contain the same data. However the user should not assume that and always use the value returned from the method. removeImage(repositoryId, imageId, options={}): repositoryId - The ID of a connected repo where the image to delete resides imageId - The id of the image you wish to delete. ---- getImageStatus(repositoryId, imageId) repositoryId - The ID of a connected repo where the image to check resides imageId - The id of the image you wish to check.

All operations return once the operations has been committed to disk NOT when the operation actually completes. This is done so that: - operation come to a stable state as quickly as possible. - In case where there is an SDM, only small portion of the operation actually needs to be performed on the SDM host. - No matter how many times the operation fails and on how many hosts, you can always resume the operation and choose when to do it. - You can stop an operation at any time and remove the resulting object making a distinction between "stop because the host is overloaded" to "I don't want that image" This means that after calling any operation that creates a new image the user must then call getImageStatus() to check what is the status of the image. The status of the image can be either optimized, degraded, or broken. "Optimized" means that the image is available and you can run VMs of it. "Degraded" means that the image is available and will run VMs but it might be a better way VDSM can represent the underlying data. "Broken" means that the image can't be used at the moment, probably because not all the data has been set up on the volume. Apart from that VDSM will also return the last persisted status information which will conatin hostID - the last host to try and optimize of fix the image stage - X/Y (eg. 1/10) the last persisted stage of the fix.

percent_complete - -1 or 0-100, the last persisted completion percentage of the aforementioned stage. -1 means that no progress is available for that operation.

last_error - This will only be filled if the operation failed because of something other then IO or a VDSM crash for obvious reasons. It will usually be set if the task was manually stopped The user can either be satisfied with that information or as the host specified in host ID if it is still working on that image by checking it's running tasks. checkStorageRepository(self, repositoryId, options={}): A method to go over a storage repository and scan for any existing problems. This includes degraded\broken images and deleted images that have no yet been physically deleted\merged. It returns a list of Fix objects. Fix objects come in 4 types: clean - cleans data, run them to get more space. optimize - run them to optimize a degraded image

merge - Merges two images together. Doing this sometimes makes more images ready optimizing or cleaning. The reason it is different from optimize is that unmerged images are considered optimized. mend - mends a broken image

The user can read these types and prioritize fixes. Fixes also contain opaque FIX data and they should be sent as received to fixStorageRepository(self, repositoryId, fix, options={}): That will start a fix operation.

All major operations automatically start the appropriate "Fix" to bring the created object to an optimize\degraded state (the one that is quicker) unless one of the options is AutoFix=False. This is only useful for repos that might not be able to create volumes on all hosts (SDM) but would like to have the actual IO distributed in the cluster. Other common options is the strategy option: It has currently 2 possible values space and performance - In case VDSM has 2 ways of completing the same operation it will tell it to value one over the other. For example, whether to copy all the data or just create a qcow based of a snapshot. The default is space.

You might have also noticed that it is never explicitly specified where to look for existing images. This is done purposefully, VDSM will always look in all connected repositories for existing objects. For very large setups this might be problematic. To mitigate the problem you have these options: participatingRepositories=[repoId, ...] which tell VDSM to narrow the search to just these repositories and imageHints={imgId: repoId} which will force VDSM to look for those image ID just in those repositories and fail if it doesn't find them there.

I've been throwing a lot of bits out about the new storage API and I think it's time to talk a bit. I will purposefully try and keep implementation details away and concentrate about how the API looks and how you use it. First major change is in terminology, there is no long a storage domain but a storage repository. This change is done because so many things are already called domain in the system and this will make things less confusing for new-commers with a libvirt background. One other changes is that repositories no longer have a UUID. The UUID was only used in the pool members manifest and is no longer needed. connectStorageRepository(repoId, repoFormat, connectionParameters={}): repoId - is a transient name that will be used to refer to the connected domain, it is not persisted and doesn't have to be the same across the cluster. repoFormat - Similar to what used to be type (eg. localfs-1.0, nfs-3.4, clvm-1.2). connectionParameters - This is format specific and will used to tell VDSM how to connect to the repo.

disconnectStorageRepository(self, repoId) In the new API there are only images, some images are mutable and some are not. mutable images are also called VirtualDisks immutable images are also called Snapshots There are no explicit templates, you can create as many images as you want from any snapshot. There are 4 major image operations: createVirtualDisk(targetRepoId, size, baseSnapshotId=None, userData={}, options={}): targetRepoId - ID of a connected repo where the disk will be created size - The size of the image you wish to create baseSnapshotId - the ID of the snapshot you want the base the new virtual disk on userData - optional data that will be attached to the new VD, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new VD

createSnapshot(targetRepoId, baseVirtualDiskId, userData={}, options={}): targetRepoId - The ID of a connected repo where the new sanpshot will be created and the original image exists as well. size - The size of the image you wish to create baseVirtualDisk - the ID of a mutable image (Virtual Disk) you want to snapshot userData - optional data that will be attached to the new Snapshot, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new Snapshot copyImage(targetRepoId, imageId, baseImageId=None, userData={}, options={}) targetRepoId - The ID of a connected repo where the new image will be created imageId - The image you wish to copy baseImageId - if specified, the new image will contain only the diff between image and Id. If None the new image will contain all the bits of image Id. This can be used to copy partial parts of images for export. userData - optional data that will be attached to the new image, could be anything that the user desires. options - options to modify VDSMs default behavior

return the Id of the new image. In case of copying an immutable image the ID will be identical to the original image as they contain the same data. However the user should not assume that and always use the value returned from the method. removeImage(repositoryId, imageId, options={}): repositoryId - The ID of a connected repo where the image to delete resides imageId - The id of the image you wish to delete. ---- getImageStatus(repositoryId, imageId) repositoryId - The ID of a connected repo where the image to check resides imageId - The id of the image you wish to check. All operations return once the operations has been committed to disk NOT when the operation actually completes. This is done so that: - operation come to a stable state as quickly as possible. - In case where there is an SDM, only small portion of the operation actually needs to be performed on the SDM host. - No matter how many times the operation fails and on how many hosts, you can always resume the operation and choose when to do it. - You can stop an operation at any time and remove the resulting object making a distinction between "stop because the host is overloaded" to "I don't want that image" This means that after calling any operation that creates a new image the user must then call getImageStatus() to check what is the status of the image. The status of the image can be either optimized, degraded, or broken. "Optimized" means that the image is available and you can run VMs of it. "Degraded" means that the image is available and will run VMs but it might be a better way VDSM can represent the underlying data.

"Broken" means that the image can't be used at the moment, probably because not all the data has been set up on the volume. Apart from that VDSM will also return the last persisted status information which will conatin hostID - the last host to try and optimize of fix the image

stage - X/Y (eg. 1/10) the last persisted stage of the fix. percent_complete - -1 or 0-100, the last persisted completion percentage of the aforementioned stage. -1 means that no progress is available for that operation. last_error - This will only be filled if the operation failed because of something other then IO or a VDSM crash for obvious reasons. It will usually be set if the task was manually stopped The user can either be satisfied with that information or as the host specified in host ID if it is still working on that image by checking it's running tasks.

checkStorageRepository(self, repositoryId, options={}): A method to go over a storage repository and scan for any existing problems. This includes degraded\broken images and deleted images that have no yet been physically deleted\merged. It returns a list of Fix objects. Fix objects come in 4 types: clean - cleans data, run them to get more space. optimize - run them to optimize a degraded image merge - Merges two images together. Doing this sometimes makes more images ready optimizing or cleaning. The reason it is different from optimize is that unmerged images are considered optimized. mend - mends a broken image The user can read these types and prioritize fixes. Fixes also contain opaque FIX data and they should be sent as received to fixStorageRepository(self, repositoryId, fix, options={}): That will start a fix operation. All major operations automatically start the appropriate "Fix" to bring the created object to an optimize\degraded state (the one that is quicker) unless one of the options is AutoFix=False. This is only useful for repos that might not be able to create volumes on all hosts (SDM) but would like to have the actual IO distributed in the cluster. Other common options is the strategy option: It has currently 2 possible values space and performance - In case VDSM has 2 ways of completing the same operation it will tell it to value one over the other. For example, whether to copy all the data or just create a qcow based of a snapshot. The default is space. You might have also noticed that it is never explicitly specified where to look for existing images. This is done purposefully, VDSM will always look in all connected repositories for existing objects. For very large setups this might be problematic. To mitigate the problem you have these options: participatingRepositories=[repoId, ...] which tell VDSM to narrow the search to just these repositories and imageHints={imgId: repoId} which will force VDSM to look for those image ID just in those repositories and fail if it doesn't find them there. _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

----- Original Message ----- > From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "engine-devel" <engine-devel(a)ovirt.org&gt; > Sent: Thursday, December 6, 2012 11:02:02 AM > Subject: Re: [vdsm] RFC: New Storage API > > Saggi, > > Thanks for sharing your thought and I get some comments below. > > > Saggi Mizrahi: >> I've been throwing a lot of bits out about the new storage API and >> I think it's time to talk a bit. >> I will purposefully try and keep implementation details away and >> concentrate about how the API looks and how you use it. >> >> First major change is in terminology, there is no long a storage >> domain but a storage repository. >> This change is done because so many things are already called >> domain in the system and this will make things less confusing for >> new-commers with a libvirt background. >> >> One other changes is that repositories no longer have a UUID. >> The UUID was only used in the pool members manifest and is no >> longer needed. >> >> >> connectStorageRepository(repoId, repoFormat, >> connectionParameters={}): >> repoId - is a transient name that will be used to refer to the >> connected domain, it is not persisted and doesn't have to be the >> same across the cluster. >> repoFormat - Similar to what used to be type (eg. localfs-1.0, >> nfs-3.4, clvm-1.2). >> connectionParameters - This is format specific and will used to >> tell VDSM how to connect to the repo. > > Where does repoID come from? I think repoID doesn't exist before > connectStorageRepository() return. Isn't repoID a return value of > connectStorageRepository()? No, repoIDs are no longer part of the domain, they are just a transient handle. The user can put whatever it wants there as long as it isn't already taken by another currently connected domain.

>> disconnectStorageRepository(self, repoId) >> >> >> In the new API there are only images, some images are mutable and >> some are not. >> mutable images are also called VirtualDisks >> immutable images are also called Snapshots >> >> There are no explicit templates, you can create as many images as >> you want from any snapshot. >> >> There are 4 major image operations: >> >> >> createVirtualDisk(targetRepoId, size, baseSnapshotId=None, >> userData={}, options={}): >> >> targetRepoId - ID of a connected repo where the disk will be >> created >> size - The size of the image you wish to create >> baseSnapshotId - the ID of the snapshot you want the base the new >> virtual disk on >> userData - optional data that will be attached to the new VD, could >> be anything that the user desires. >> options - options to modify VDSMs default behavior

>> >> returns the id of the new VD > I think we will also need a function to check if a a VirtualDisk is > based on a specific snapshot. > Like: isSnapshotOf(virtualDiskId, baseSnapshotID): No, the design is that volume dependencies are an implementation detail. There is no reason for you to know that an image is physically a snapshot of another. Logical snapshots, template information, and any other information can be set by the user by using the userData field available for every image. >> createSnapshot(targetRepoId, baseVirtualDiskId, >> userData={}, options={}): >> targetRepoId - The ID of a connected repo where the new sanpshot >> will be created and the original image exists as well. >> size - The size of the image you wish to create >> baseVirtualDisk - the ID of a mutable image (Virtual Disk) you want >> to snapshot >> userData - optional data that will be attached to the new Snapshot, >> could be anything that the user desires. >> options - options to modify VDSMs default behavior >> >> returns the id of the new Snapshot >> >> copyImage(targetRepoId, imageId, baseImageId=None, userData={}, >> options={}) >> targetRepoId - The ID of a connected repo where the new image will >> be created >> imageId - The image you wish to copy >> baseImageId - if specified, the new image will contain only the >> diff between image and Id. >> If None the new image will contain all the bits of >> image Id. This can be used to copy partial parts of >> images for export. >> userData - optional data that will be attached to the new image, >> could be anything that the user desires. >> options - options to modify VDSMs default behavior > Does this function mean that we can copy the image from one > repository > to another repository? Does it cover the semantics of storage > migration, > storage backup, storage incremental backup? Yes, the main purpose is copying to another repo. and you can even do incremental backups. Also the following flow 1. Run a VM using imageA 2. write to disk 3. Stop VM 4. copy imageA to repoB 5. Run a VM using imageA again 6. Write to disk 7. Stop VM 8. Copy imageA again basing it of imageA_copy1 on repoB creating a diff on repo diff without snapshotting the original image. >> return the Id of the new image. In case of copying an immutable >> image the ID will be identical to the original image as they >> contain the same data. However the user should not assume that and >> always use the value returned from the method. >> >> removeImage(repositoryId, imageId, options={}): >> repositoryId - The ID of a connected repo where the image to delete >> resides >> imageId - The id of the image you wish to delete. >> >> >> ---- >> getImageStatus(repositoryId, imageId) >> repositoryId - The ID of a connected repo where the image to check >> resides >> imageId - The id of the image you wish to check. >> >> All operations return once the operations has been committed to >> disk NOT when the operation actually completes. >> This is done so that: >> - operation come to a stable state as quickly as possible. >> - In case where there is an SDM, only small portion of the >> operation actually needs to be performed on the SDM host. >> - No matter how many times the operation fails and on how many >> hosts, you can always resume the operation and choose when to do >> it. >> - You can stop an operation at any time and remove the resulting >> object making a distinction between "stop because the host is >> overloaded" to "I don't want that image" >> >> This means that after calling any operation that creates a new >> image the user must then call getImageStatus() to check what is >> the status of the image. >> The status of the image can be either optimized, degraded, or >> broken. >> "Optimized" means that the image is available and you can run VMs >> of it. >> "Degraded" means that the image is available and will run VMs but >> it might be a better way VDSM can represent the underlying data.

> What does the "represent" mean here? Anything, but mostly image formate RAW\QCOW2 when performance strategy has been selected. >> "Broken" means that the image can't be used at the moment, probably >> because not all the data has been set up on the volume. >> >> Apart from that VDSM will also return the last persisted status >> information which will conatin >> hostID - the last host to try and optimize of fix the image > Any host can optimize the image? No need to be SDM? On anything but lvm based block domains there will not even be an SDM. On SDM based domains we will try as hard as we can to have as many operations executable on any host.

>> stage - X/Y (eg. 1/10) the last persisted stage of the fix. >> percent_complete - -1 or 0-100, the last persisted completion >> percentage of the aforementioned stage. -1 means that no progress >> is available for that operation. >> last_error - This will only be filled if the operation failed >> because of something other then IO or a VDSM crash for obvious >> reasons. >> It will usually be set if the task was manually >> stopped >> >> The user can either be satisfied with that information or as the >> host specified in host ID if it is still working on that image by >> checking it's running tasks. > So we need a function to know what tasks are running on the image getImageStatus() >> checkStorageRepository(self, repositoryId, options={}): >> A method to go over a storage repository and scan for any existing >> problems. This includes degraded\broken images and deleted images >> that have no yet been physically deleted\merged. >> It returns a list of Fix objects. >> Fix objects come in 4 types: >> clean - cleans data, run them to get more space. >> optimize - run them to optimize a degraded image >> merge - Merges two images together. Doing this sometimes >> makes more images ready optimizing or cleaning. >> The reason it is different from optimize is that >> unmerged images are considered optimized. >> mend - mends a broken image >> >> The user can read these types and prioritize fixes. Fixes also >> contain opaque FIX data and they should be sent as received to >> fixStorageRepository(self, repositoryId, fix, options={}): >> >> That will start a fix operation.

>> >> >> All major operations automatically start the appropriate "Fix" to >> bring the created object to an optimize\degraded state (the one >> that is quicker) unless one of the options is >> AutoFix=False. This is only useful for repos that might not be able >> to create volumes on all hosts (SDM) but would like to have the >> actual IO distributed in the cluster. >> >> Other common options is the strategy option: >> It has currently 2 possible values >> space and performance - In case VDSM has 2 ways of completing the >> same operation it will tell it to value one over the other. For >> example, whether to copy all the data or just create a qcow based >> of a snapshot. >> The default is space. >> >> You might have also noticed that it is never explicitly specified >> where to look for existing images. This is done purposefully, VDSM >> will always look in all connected repositories for existing >> objects. >> For very large setups this might be problematic. To mitigate the >> problem you have these options: >> participatingRepositories=[repoId, ...] which tell VDSM to narrow >> the search to just these repositories >> and >> imageHints={imgId: repoId} which will force VDSM to look for those >> image ID just in those repositories and fail if it doesn't find >> them there. >> _______________________________________________ >> vdsm-devel mailing list >> vdsm-devel(a)lists.fedorahosted.org >> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > > -- > --- > 舒明 Shu Ming > Open Virtualization Engineerning; CSTL, IBM Corp. > Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming(a)cn.ibm.com or > shuming(a)linux.vnet.ibm.com > Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian > District, Beijing 100193, PRC > > > _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; To: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; Cc: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;, "engine-devel" <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; Sent: Friday, December 7, 2012 1:37:20 AM Subject: Re: [vdsm] RFC: New Storage API 于 2012-12-7 13:23, Deepak C Shetty: > On 12/06/2012 10:22 PM, Saggi Mizrahi wrote: >> >> ----- Original Message ----- >>> From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; >>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; >>> Cc: "VDSM Project Development" >>> <vdsm-devel(a)lists.fedorahosted.org&gt;, >>> "engine-devel" <engine-devel(a)ovirt.org&gt; >>> Sent: Thursday, December 6, 2012 11:02:02 AM >>> Subject: Re: [vdsm] RFC: New Storage API >>> >>> Saggi, >>> >>> Thanks for sharing your thought and I get some comments below. >>> >>> >>> Saggi Mizrahi: >>>> I've been throwing a lot of bits out about the new storage API >>>> and >>>> I think it's time to talk a bit. >>>> I will purposefully try and keep implementation details away and >>>> concentrate about how the API looks and how you use it. >>>> >>>> First major change is in terminology, there is no long a storage >>>> domain but a storage repository. >>>> This change is done because so many things are already called >>>> domain in the system and this will make things less confusing >>>> for >>>> new-commers with a libvirt background. >>>> >>>> One other changes is that repositories no longer have a UUID. >>>> The UUID was only used in the pool members manifest and is no >>>> longer needed. >>>> >>>> >>>> connectStorageRepository(repoId, repoFormat, >>>> connectionParameters={}): >>>> repoId - is a transient name that will be used to refer to the >>>> connected domain, it is not persisted and doesn't have to be the >>>> same across the cluster. >>>> repoFormat - Similar to what used to be type (eg. localfs-1.0, >>>> nfs-3.4, clvm-1.2). >>>> connectionParameters - This is format specific and will used to >>>> tell VDSM how to connect to the repo. >>> >>> Where does repoID come from? I think repoID doesn't exist before >>> connectStorageRepository() return. Isn't repoID a return value >>> of >>> connectStorageRepository()? >> No, repoIDs are no longer part of the domain, they are just a >> transient handle. >> The user can put whatever it wants there as long as it isn't >> already >> taken by another currently connected domain. > > So what happens when user mistakenly gives a repoID that is in use > before.. there should be something in the return value that > specifies > the error and/or reason for error so that user can try with a > new/diff > repoID ? I think let the user to give the repoID is meaningless and error-prune.

Developer must maintain a a unique ID list for every storage repository connected.

> >>>> disconnectStorageRepository(self, repoId) >>>> >>>> >>>> In the new API there are only images, some images are mutable >>>> and >>>> some are not. >>>> mutable images are also called VirtualDisks >>>> immutable images are also called Snapshots >>>> >>>> There are no explicit templates, you can create as many images >>>> as >>>> you want from any snapshot. >>>> >>>> There are 4 major image operations: >>>> >>>> >>>> createVirtualDisk(targetRepoId, size, baseSnapshotId=None, >>>> userData={}, options={}): >>>> >>>> targetRepoId - ID of a connected repo where the disk will be >>>> created >>>> size - The size of the image you wish to create >>>> baseSnapshotId - the ID of the snapshot you want the base the >>>> new >>>> virtual disk on >>>> userData - optional data that will be attached to the new VD, >>>> could >>>> be anything that the user desires. >>>> options - options to modify VDSMs default behavior > > IIUC, i can use options to do storage offloads ? For eg. I can > create > a LUN that represents this VD on my storage array based on the > 'options' parameter ? Is this the intended way to use 'options' ? > >>>> >>>> returns the id of the new VD >>> I think we will also need a function to check if a a VirtualDisk >>> is >>> based on a specific snapshot. >>> Like: isSnapshotOf(virtualDiskId, baseSnapshotID): >> No, the design is that volume dependencies are an implementation >> detail. >> There is no reason for you to know that an image is physically a >> snapshot of another. >> Logical snapshots, template information, and any other information >> can be set by the user by using the userData field available for >> every image. >>>> createSnapshot(targetRepoId, baseVirtualDiskId, >>>> userData={}, options={}): >>>> targetRepoId - The ID of a connected repo where the new sanpshot >>>> will be created and the original image exists as well. >>>> size - The size of the image you wish to create >>>> baseVirtualDisk - the ID of a mutable image (Virtual Disk) you >>>> want >>>> to snapshot >>>> userData - optional data that will be attached to the new >>>> Snapshot, >>>> could be anything that the user desires. >>>> options - options to modify VDSMs default behavior >>>> >>>> returns the id of the new Snapshot >>>> >>>> copyImage(targetRepoId, imageId, baseImageId=None, userData={}, >>>> options={}) >>>> targetRepoId - The ID of a connected repo where the new image >>>> will >>>> be created >>>> imageId - The image you wish to copy >>>> baseImageId - if specified, the new image will contain only the >>>> diff between image and Id. >>>> If None the new image will contain all the bits >>>> of >>>> image Id. This can be used to copy partial parts >>>> of >>>> images for export. >>>> userData - optional data that will be attached to the new image, >>>> could be anything that the user desires. >>>> options - options to modify VDSMs default behavior >>> Does this function mean that we can copy the image from one >>> repository >>> to another repository? Does it cover the semantics of storage >>> migration, >>> storage backup, storage incremental backup? >> Yes, the main purpose is copying to another repo. and you can even >> do >> incremental backups. >> Also the following flow >> 1. Run a VM using imageA >> 2. write to disk >> 3. Stop VM >> 4. copy imageA to repoB >> 5. Run a VM using imageA again >> 6. Write to disk >> 7. Stop VM >> 8. Copy imageA again basing it of imageA_copy1 on repoB creating a >> diff on repo diff without snapshotting the original image. >>>> return the Id of the new image. In case of copying an immutable >>>> image the ID will be identical to the original image as they >>>> contain the same data. However the user should not assume that >>>> and >>>> always use the value returned from the method. Do you mean the ID returned by copyImage() will be the same as the origin ID? It is okay for copying to a different repository than the original repository because they have different namespace. However, if we do the copy inside in the same repository, we should get a new different ID.

>>>> >>>> removeImage(repositoryId, imageId, options={}): >>>> repositoryId - The ID of a connected repo where the image to >>>> delete >>>> resides >>>> imageId - The id of the image you wish to delete. >>>> >>>> >>>> ---- >>>> getImageStatus(repositoryId, imageId) >>>> repositoryId - The ID of a connected repo where the image to >>>> check >>>> resides >>>> imageId - The id of the image you wish to check. >>>> >>>> All operations return once the operations has been committed to >>>> disk NOT when the operation actually completes. >>>> This is done so that: >>>> - operation come to a stable state as quickly as possible. >>>> - In case where there is an SDM, only small portion of the >>>> operation actually needs to be performed on the SDM host. >>>> - No matter how many times the operation fails and on how many >>>> hosts, you can always resume the operation and choose when to do >>>> it. >>>> - You can stop an operation at any time and remove the resulting >>>> object making a distinction between "stop because the host is >>>> overloaded" to "I don't want that image" >>>> >>>> This means that after calling any operation that creates a new >>>> image the user must then call getImageStatus() to check what is >>>> the status of the image. >>>> The status of the image can be either optimized, degraded, or >>>> broken. >>>> "Optimized" means that the image is available and you can run >>>> VMs >>>> of it. >>>> "Degraded" means that the image is available and will run VMs >>>> but >>>> it might be a better way VDSM can represent the underlying data. > > Calling qcow2 based snapshot degraded (meaning its degraded in > perf, > as its space optimzed ) and calling raw images as optimised ( > meaning > its optimised for perf as its space in-efficient) is confusing. > Degraded sounds like a bad thing when seen by the end-user :) I > think > there is scope for having some better and less confusing > terminology > here ? > >>> What does the "represent" mean here? >> Anything, but mostly image formate RAW\QCOW2 when performance >> strategy has been selected. >>>> "Broken" means that the image can't be used at the moment, >>>> probably >>>> because not all the data has been set up on the volume. >>>> >>>> Apart from that VDSM will also return the last persisted status >>>> information which will conatin >>>> hostID - the last host to try and optimize of fix the image >>> Any host can optimize the image? No need to be SDM? >> On anything but lvm based block domains there will not even be an >> SDM. >> On SDM based domains we will try as hard as we can to have as many >> operations executable on any host. > > 1) Can you provide more info on why there is a exception for 'lvm > based block domain'. Its not coming out clearly. I guess Saggi means we can leverage CLVM to build a cluster based LVM group and no need to have SDM. If I understand correctly, in this proposal, we manage different types of the repositories at the same type, say, one localFS repository, one iSCSI repository, one nfs repository at the same time, also the images can be copied to each other back and forth. When you copy a image from repository without SDM to a repository with SDM, we should use the later SDM to execute the copy operation. > 2) Based on the terminology change, domain is now replaced by > repository, SDM should now be more aptly called SRM (storage repo > manager) so that we are consistent in the usage of terminology > 3) Can you provide some example flow / scenario to understnad how > with > and without SDM domains work ? Especially how the disk based lock > is > taken if no SDM ? > >>>> stage - X/Y (eg. 1/10) the last persisted stage of the fix. >>>> percent_complete - -1 or 0-100, the last persisted completion >>>> percentage of the aforementioned stage. -1 means that no >>>> progress >>>> is available for that operation. >>>> last_error - This will only be filled if the operation failed >>>> because of something other then IO or a VDSM crash for obvious >>>> reasons. >>>> It will usually be set if the task was manually >>>> stopped >>>> >>>> The user can either be satisfied with that information or as the >>>> host specified in host ID if it is still working on that image >>>> by >>>> checking it's running tasks. >>> So we need a function to know what tasks are running on the image >> getImageStatus() >>>> checkStorageRepository(self, repositoryId, options={}): >>>> A method to go over a storage repository and scan for any >>>> existing >>>> problems. This includes degraded\broken images and deleted >>>> images >>>> that have no yet been physically deleted\merged. >>>> It returns a list of Fix objects. >>>> Fix objects come in 4 types: >>>> clean - cleans data, run them to get more space. >>>> optimize - run them to optimize a degraded image >>>> merge - Merges two images together. Doing this sometimes >>>> makes more images ready optimizing or cleaning. >>>> The reason it is different from optimize is that >>>> unmerged images are considered optimized. >>>> mend - mends a broken image What is a broken image? Does broken mean corruption? If it is corruption, we may need some recoverable checksum for each image. >>>> >>>> The user can read these types and prioritize fixes. Fixes also >>>> contain opaque FIX data and they should be sent as received to >>>> fixStorageRepository(self, repositoryId, fix, options={}): >>>> >>>> That will start a fix operation. > > It would be good if you can provide some example or flow of "fix" > operation. > When and Why would somebody want to do it ? > > Does 'Fix' here mean that i move from raw to qcow2 format or > vice-versa, or there is more to it ? > >>>> >>>> >>>> All major operations automatically start the appropriate "Fix" >>>> to >>>> bring the created object to an optimize\degraded state (the one >>>> that is quicker) unless one of the options is >>>> AutoFix=False. This is only useful for repos that might not be >>>> able >>>> to create volumes on all hosts (SDM) but would like to have the >>>> actual IO distributed in the cluster. >>>> >>>> Other common options is the strategy option: >>>> It has currently 2 possible values >>>> space and performance - In case VDSM has 2 ways of completing >>>> the >>>> same operation it will tell it to value one over the other. For >>>> example, whether to copy all the data or just create a qcow >>>> based >>>> of a snapshot. >>>> The default is space. >>>> >>>> You might have also noticed that it is never explicitly >>>> specified >>>> where to look for existing images. This is done purposefully, >>>> VDSM >>>> will always look in all connected repositories for existing >>>> objects. >>>> For very large setups this might be problematic. To mitigate the >>>> problem you have these options: >>>> participatingRepositories=[repoId, ...] which tell VDSM to >>>> narrow >>>> the search to just these repositories >>>> and >>>> imageHints={imgId: repoId} which will force VDSM to look for >>>> those >>>> image ID just in those repositories and fail if it doesn't find >>>> them there. >>>> _______________________________________________ >>>> vdsm-devel mailing list >>>> vdsm-devel(a)lists.fedorahosted.org >>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel >>> >>> -- >>> --- >>> 舒明 Shu Ming >>> Open Virtualization Engineerning; CSTL, IBM Corp. >>> Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming(a)cn.ibm.com >>> or >>> shuming(a)linux.vnet.ibm.com >>> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian >>> District, Beijing 100193, PRC >>> >>> >>> >> _______________________________________________ >> vdsm-devel mailing list >> vdsm-devel(a)lists.fedorahosted.org >> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > -- --- 舒明 Shu Ming Open Virtualization Engineerning; CSTL, IBM Corp. Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming(a)cn.ibm.com or shuming(a)linux.vnet.ibm.com Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC

> 1) Can you provide more info on why there is a exception for 'lvm > based > block domain'. Its not coming out clearly. File based domains are responsible for syncing up object manipulation (creation\deletion) The backend is responsible for making sure it all works either by having a single writer (NFS) or having it's own locking mechanism (gluster). In our LVM based domains VDSM is responsible for basic object manipulation. The current design uses an approach where there is a single host responsible for object creation\deleteion it is the SRM\SDM\SPM\S?M. If we ever find a way to make it fully clustered without a big hit in performance the S?M requirement will be removed form that type of domain.

----- Original Message ----- > From: "Adam Litke" <agl(a)us.ibm.com&gt; > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > Cc: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt;, "engine-devel" <engine-devel(a)ovirt.org&gt;, "VDSM Project > Development" <vdsm-devel(a)lists.fedorahosted.org&gt; > Sent: Monday, December 10, 2012 1:49:31 PM > Subject: Re: [vdsm] RFC: New Storage API > > On Fri, Dec 07, 2012 at 02:53:41PM -0500, Saggi Mizrahi wrote: > > <snip> > > > > 1) Can you provide more info on why there is a exception for 'lvm > > > based > > > block domain'. Its not coming out clearly. > > File based domains are responsible for syncing up object > > manipulation (creation\deletion) > > The backend is responsible for making sure it all works either by > > having a single writer (NFS) or having it's own locking mechanism > > (gluster). > > In our LVM based domains VDSM is responsible for basic object > > manipulation. > > The current design uses an approach where there is a single host > > responsible for object creation\deleteion it is the > > SRM\SDM\SPM\S?M. > > If we ever find a way to make it fully clustered without a big hit > > in performance the S?M requirement will be removed form that type > > of domain. > > I would like to see us maintain a LOCALFS domain as well. For this, > we would > also need SRM, correct? No, why?

From: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; Cc: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;, "engine-devel" <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; Sent: Sunday, December 16, 2012 11:40:01 PM Subject: Re: [vdsm] RFC: New Storage API On 12/08/2012 01:23 AM, Saggi Mizrahi wrote: > > ----- Original Message ----- >> From: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; >> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; >> Cc: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;, "engine-devel" >> <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" >> <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" >> <deepakcs(a)linux.vnet.ibm.com&gt; >> Sent: Friday, December 7, 2012 12:23:15 AM >> Subject: Re: [vdsm] RFC: New Storage API >> >> On 12/06/2012 10:22 PM, Saggi Mizrahi wrote: >>> ----- Original Message ----- >>>> From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; >>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; >>>> Cc: "VDSM Project Development" >>>> <vdsm-devel(a)lists.fedorahosted.org&gt;, "engine-devel" >>>> <engine-devel(a)ovirt.org&gt; >>>> Sent: Thursday, December 6, 2012 11:02:02 AM >>>> Subject: Re: [vdsm] RFC: New Storage API >>>> >>>> Saggi, >>>> >>>> Thanks for sharing your thought and I get some comments below. >>>> >>>> >>>> Saggi Mizrahi: >>>>> I've been throwing a lot of bits out about the new storage API >>>>> and >>>>> I think it's time to talk a bit. >>>>> I will purposefully try and keep implementation details away >>>>> and >>>>> concentrate about how the API looks and how you use it. >>>>> >>>>> First major change is in terminology, there is no long a >>>>> storage >>>>> domain but a storage repository. >>>>> This change is done because so many things are already called >>>>> domain in the system and this will make things less confusing >>>>> for >>>>> new-commers with a libvirt background. >>>>> >>>>> One other changes is that repositories no longer have a UUID. >>>>> The UUID was only used in the pool members manifest and is no >>>>> longer needed. >>>>> >>>>> >>>>> connectStorageRepository(repoId, repoFormat, >>>>> connectionParameters={}): >>>>> repoId - is a transient name that will be used to refer to the >>>>> connected domain, it is not persisted and doesn't have to be >>>>> the >>>>> same across the cluster. >>>>> repoFormat - Similar to what used to be type (eg. localfs-1.0, >>>>> nfs-3.4, clvm-1.2). >>>>> connectionParameters - This is format specific and will used to >>>>> tell VDSM how to connect to the repo. >>>> Where does repoID come from? I think repoID doesn't exist before >>>> connectStorageRepository() return. Isn't repoID a return value >>>> of >>>> connectStorageRepository()? >>> No, repoIDs are no longer part of the domain, they are just a >>> transient handle. >>> The user can put whatever it wants there as long as it isn't >>> already taken by another currently connected domain. >> So what happens when user mistakenly gives a repoID that is in use >> before.. there should be something in the return value that >> specifies >> the error and/or reason for error so that user can try with a >> new/diff >> repoID ? > Asi I said, connect fails if the repoId is in use ATM. >>>>> disconnectStorageRepository(self, repoId) >>>>> >>>>> >>>>> In the new API there are only images, some images are mutable >>>>> and >>>>> some are not. >>>>> mutable images are also called VirtualDisks >>>>> immutable images are also called Snapshots >>>>> >>>>> There are no explicit templates, you can create as many images >>>>> as >>>>> you want from any snapshot. >>>>> >>>>> There are 4 major image operations: >>>>> >>>>> >>>>> createVirtualDisk(targetRepoId, size, baseSnapshotId=None, >>>>> userData={}, options={}): >>>>> >>>>> targetRepoId - ID of a connected repo where the disk will be >>>>> created >>>>> size - The size of the image you wish to create >>>>> baseSnapshotId - the ID of the snapshot you want the base the >>>>> new >>>>> virtual disk on >>>>> userData - optional data that will be attached to the new VD, >>>>> could >>>>> be anything that the user desires. >>>>> options - options to modify VDSMs default behavior >> IIUC, i can use options to do storage offloads ? For eg. I can >> create >> a >> LUN that represents this VD on my storage array based on the >> 'options' >> parameter ? Is this the intended way to use 'options' ? > No, this has nothing to do with offloads. > If by "offloads" you mean having other VDSM hosts to the heavy > lifting then this is what the option autoFix=False and the fix > mechanism is for. > If you are talking about advanced scsi features (ie. write same) > they will be used automatically whenever possible. > In any case, how we manage LUNs (if they are even used) is an > implementation detail. I am a bit more interested in how storage array offloads ( by that I mean, offload VD creation, snapshot, clone etc to the storage array when available/possible) can be done from VDSM ? In the past there were talks of using libSM to do that. How does that strategy play in this new Storage API scenario ? I agree its implmn detail, but how & where does that implm sit and how it would be triggered is not very clear to me. Looking at createVD args, it sounded like 'options' seems to be a trigger point for deciding whether to use storage offloads or not, but you spoke otherwise :) Can you provide your vision on how VDSM can understand the storage array capabilities & exploit storgae array offloads in this New Storage API context ? -- Thanks deepak

From: "Ayal Baron" <abaron(a)redhat.com&gt; To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; Cc: "engine-devel" <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; Sent: Monday, January 14, 2013 4:56:05 PM Subject: Re: [vdsm] RFC: New Storage API ----- Original Message ----- > > > ----- Original Message ----- > > From: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; > > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > > Cc: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;, "engine-devel" > > <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" > > <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" > > <deepakcs(a)linux.vnet.ibm.com&gt; > > Sent: Sunday, December 16, 2012 11:40:01 PM > > Subject: Re: [vdsm] RFC: New Storage API > > > > On 12/08/2012 01:23 AM, Saggi Mizrahi wrote: > > > > > > ----- Original Message ----- > > >> From: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; > > >> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > > >> Cc: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;, "engine-devel" > > >> <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" > > >> <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" > > >> <deepakcs(a)linux.vnet.ibm.com&gt; > > >> Sent: Friday, December 7, 2012 12:23:15 AM > > >> Subject: Re: [vdsm] RFC: New Storage API > > >> > > >> On 12/06/2012 10:22 PM, Saggi Mizrahi wrote: > > >>> ----- Original Message ----- > > >>>> From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; > > >>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > > >>>> Cc: "VDSM Project Development" > > >>>> <vdsm-devel(a)lists.fedorahosted.org&gt;, "engine-devel" > > >>>> <engine-devel(a)ovirt.org&gt; > > >>>> Sent: Thursday, December 6, 2012 11:02:02 AM > > >>>> Subject: Re: [vdsm] RFC: New Storage API > > >>>> > > >>>> Saggi, > > >>>> > > >>>> Thanks for sharing your thought and I get some comments > > >>>> below. > > >>>> > > >>>> > > >>>> Saggi Mizrahi: > > >>>>> I've been throwing a lot of bits out about the new storage > > >>>>> API > > >>>>> and > > >>>>> I think it's time to talk a bit. > > >>>>> I will purposefully try and keep implementation details > > >>>>> away > > >>>>> and > > >>>>> concentrate about how the API looks and how you use it. > > >>>>> > > >>>>> First major change is in terminology, there is no long a > > >>>>> storage > > >>>>> domain but a storage repository. > > >>>>> This change is done because so many things are already > > >>>>> called > > >>>>> domain in the system and this will make things less > > >>>>> confusing > > >>>>> for > > >>>>> new-commers with a libvirt background. > > >>>>> > > >>>>> One other changes is that repositories no longer have a > > >>>>> UUID. > > >>>>> The UUID was only used in the pool members manifest and is > > >>>>> no > > >>>>> longer needed. > > >>>>> > > >>>>> > > >>>>> connectStorageRepository(repoId, repoFormat, > > >>>>> connectionParameters={}): > > >>>>> repoId - is a transient name that will be used to refer to > > >>>>> the > > >>>>> connected domain, it is not persisted and doesn't have to > > >>>>> be > > >>>>> the > > >>>>> same across the cluster. > > >>>>> repoFormat - Similar to what used to be type (eg. > > >>>>> localfs-1.0, > > >>>>> nfs-3.4, clvm-1.2). > > >>>>> connectionParameters - This is format specific and will > > >>>>> used > > >>>>> to > > >>>>> tell VDSM how to connect to the repo. > > >>>> Where does repoID come from? I think repoID doesn't exist > > >>>> before > > >>>> connectStorageRepository() return. Isn't repoID a return > > >>>> value > > >>>> of > > >>>> connectStorageRepository()? > > >>> No, repoIDs are no longer part of the domain, they are just a > > >>> transient handle. > > >>> The user can put whatever it wants there as long as it isn't > > >>> already taken by another currently connected domain. > > >> So what happens when user mistakenly gives a repoID that is in > > >> use > > >> before.. there should be something in the return value that > > >> specifies > > >> the error and/or reason for error so that user can try with a > > >> new/diff > > >> repoID ? > > > Asi I said, connect fails if the repoId is in use ATM. > > >>>>> disconnectStorageRepository(self, repoId) > > >>>>> > > >>>>> > > >>>>> In the new API there are only images, some images are > > >>>>> mutable > > >>>>> and > > >>>>> some are not. > > >>>>> mutable images are also called VirtualDisks > > >>>>> immutable images are also called Snapshots > > >>>>> > > >>>>> There are no explicit templates, you can create as many > > >>>>> images > > >>>>> as > > >>>>> you want from any snapshot. > > >>>>> > > >>>>> There are 4 major image operations: > > >>>>> > > >>>>> > > >>>>> createVirtualDisk(targetRepoId, size, baseSnapshotId=None, > > >>>>> userData={}, options={}): > > >>>>> > > >>>>> targetRepoId - ID of a connected repo where the disk will > > >>>>> be > > >>>>> created > > >>>>> size - The size of the image you wish to create > > >>>>> baseSnapshotId - the ID of the snapshot you want the base > > >>>>> the > > >>>>> new > > >>>>> virtual disk on > > >>>>> userData - optional data that will be attached to the new > > >>>>> VD, > > >>>>> could > > >>>>> be anything that the user desires. > > >>>>> options - options to modify VDSMs default behavior > > >> IIUC, i can use options to do storage offloads ? For eg. I can > > >> create > > >> a > > >> LUN that represents this VD on my storage array based on the > > >> 'options' > > >> parameter ? Is this the intended way to use 'options' ? > > > No, this has nothing to do with offloads. > > > If by "offloads" you mean having other VDSM hosts to the heavy > > > lifting then this is what the option autoFix=False and the fix > > > mechanism is for. > > > If you are talking about advanced scsi features (ie. write > > > same) > > > they will be used automatically whenever possible. > > > In any case, how we manage LUNs (if they are even used) is an > > > implementation detail. > > > > I am a bit more interested in how storage array offloads ( by > > that > > I > > mean, offload VD creation, snapshot, clone etc to the storage > > array > > when > > available/possible) can be done from VDSM ? > > In the past there were talks of using libSM to do that. How does > > that > > strategy play in this new Storage API scenario ? I agree its > > implmn > > detail, but how & where does that implm sit and how it would be > > triggered is not very clear to me. Looking at createVD args, it > > sounded > > like 'options' seems to be a trigger point for deciding whether > > to > > use > > storage offloads or not, but you spoke otherwise :) Can you > > provide > > your > > vision on how VDSM can understand the storage array capabilities > > & > > exploit storgae array offloads in this New Storage API context ? > > -- > > Thanks deepak > Some will be used automatically whenever possible (storage > offloading). > Features that favor a specific strategy will be activated when the > proper strategy (space, performance) option is selected. > In cases when only the user can know whether to use a feature or > not > we will have options to turn that on. > In any case every domain exports a capability list through > GetRepositoryCapabilities() that returns a list off repository > capabilities. > Some capabilities are VDSM specific like CLUSTERED or REQUIRES_SRM. > Some are storage capabilities like NATIVE_SNAPSHOTS, > NATIVE_THIN_PROVISIONING, SPARSE_VOLUMES, etc... > > We are also considering an override mechanism where you can disable > features in storage that supports it by setting it in the domain > options. This will be done with NO_XXXXX (eg. NO_NATIVE_SNAPSHOTS). > This will make the domain not use or expose the capability through > the API. I assume it will only be used for testing or in cases > where > the storage array is known to have problems with a certain feature. > Not everything can be disables as an example there is no real way > to > disable NATIVE_THING_PROVISIONING or SPARSE_VOLUMES. Saggi, there are several different discussions going on here which I think require some clearing up (and perhaps splitting). what I think is missing here: 1. distinction of what we believe should be at repo level and what at disk level e.g. pre/postZero at repo level, native snapshots as described above would also be repo level (not defined per disk) etc.

2. how/where storage offload would work? is there a single implementation for repos which detects automatically storage capabilities or repo class for each storage type

3. and biggest topic is probably - a mapping of the image operations and details about the flows (did you send something about that?) - i.e. create vdisk flow, copy, etc.

> > > > > _______________________________________________ > vdsm-devel mailing list > vdsm-devel(a)lists.fedorahosted.org > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel >

From: "Ayal Baron" <abaron(a)redhat.com&gt; To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; Cc: "engine-devel" <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; Sent: Monday, January 14, 2013 6:23:32 PM Subject: Re: [vdsm] RFC: New Storage API ----- Original Message ----- > > > ----- Original Message ----- > > From: "Ayal Baron" <abaron(a)redhat.com&gt; > > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > > Cc: "engine-devel" <engine-devel(a)ovirt.org&gt;, "VDSM Project > > Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak > > C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; > > Sent: Monday, January 14, 2013 4:56:05 PM > > Subject: Re: [vdsm] RFC: New Storage API > > > > > > > > ----- Original Message ----- > > > > > > > > > ----- Original Message ----- > > > > From: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; > > > > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > > > > Cc: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;, "engine-devel" > > > > <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" > > > > <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" > > > > <deepakcs(a)linux.vnet.ibm.com&gt; > > > > Sent: Sunday, December 16, 2012 11:40:01 PM > > > > Subject: Re: [vdsm] RFC: New Storage API > > > > > > > > On 12/08/2012 01:23 AM, Saggi Mizrahi wrote: > > > > > > > > > > ----- Original Message ----- > > > > >> From: "Deepak C Shetty" <deepakcs(a)linux.vnet.ibm.com&gt; > > > > >> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > > > > >> Cc: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;, > > > > >> "engine-devel" > > > > >> <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" > > > > >> <vdsm-devel(a)lists.fedorahosted.org&gt;, "Deepak C Shetty" > > > > >> <deepakcs(a)linux.vnet.ibm.com&gt; > > > > >> Sent: Friday, December 7, 2012 12:23:15 AM > > > > >> Subject: Re: [vdsm] RFC: New Storage API > > > > >> > > > > >> On 12/06/2012 10:22 PM, Saggi Mizrahi wrote: > > > > >>> ----- Original Message ----- > > > > >>>> From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; > > > > >>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > > > > >>>> Cc: "VDSM Project Development" > > > > >>>> <vdsm-devel(a)lists.fedorahosted.org&gt;, "engine-devel" > > > > >>>> <engine-devel(a)ovirt.org&gt; > > > > >>>> Sent: Thursday, December 6, 2012 11:02:02 AM > > > > >>>> Subject: Re: [vdsm] RFC: New Storage API > > > > >>>> > > > > >>>> Saggi, > > > > >>>> > > > > >>>> Thanks for sharing your thought and I get some comments > > > > >>>> below. > > > > >>>> > > > > >>>> > > > > >>>> Saggi Mizrahi: > > > > >>>>> I've been throwing a lot of bits out about the new > > > > >>>>> storage > > > > >>>>> API > > > > >>>>> and > > > > >>>>> I think it's time to talk a bit. > > > > >>>>> I will purposefully try and keep implementation details > > > > >>>>> away > > > > >>>>> and > > > > >>>>> concentrate about how the API looks and how you use it. > > > > >>>>> > > > > >>>>> First major change is in terminology, there is no long > > > > >>>>> a > > > > >>>>> storage > > > > >>>>> domain but a storage repository. > > > > >>>>> This change is done because so many things are already > > > > >>>>> called > > > > >>>>> domain in the system and this will make things less > > > > >>>>> confusing > > > > >>>>> for > > > > >>>>> new-commers with a libvirt background. > > > > >>>>> > > > > >>>>> One other changes is that repositories no longer have a > > > > >>>>> UUID. > > > > >>>>> The UUID was only used in the pool members manifest and > > > > >>>>> is > > > > >>>>> no > > > > >>>>> longer needed. > > > > >>>>> > > > > >>>>> > > > > >>>>> connectStorageRepository(repoId, repoFormat, > > > > >>>>> connectionParameters={}): > > > > >>>>> repoId - is a transient name that will be used to refer > > > > >>>>> to > > > > >>>>> the > > > > >>>>> connected domain, it is not persisted and doesn't have > > > > >>>>> to > > > > >>>>> be > > > > >>>>> the > > > > >>>>> same across the cluster. > > > > >>>>> repoFormat - Similar to what used to be type (eg. > > > > >>>>> localfs-1.0, > > > > >>>>> nfs-3.4, clvm-1.2). > > > > >>>>> connectionParameters - This is format specific and will > > > > >>>>> used > > > > >>>>> to > > > > >>>>> tell VDSM how to connect to the repo. > > > > >>>> Where does repoID come from? I think repoID doesn't > > > > >>>> exist > > > > >>>> before > > > > >>>> connectStorageRepository() return. Isn't repoID a > > > > >>>> return > > > > >>>> value > > > > >>>> of > > > > >>>> connectStorageRepository()? > > > > >>> No, repoIDs are no longer part of the domain, they are > > > > >>> just > > > > >>> a > > > > >>> transient handle. > > > > >>> The user can put whatever it wants there as long as it > > > > >>> isn't > > > > >>> already taken by another currently connected domain. > > > > >> So what happens when user mistakenly gives a repoID that > > > > >> is > > > > >> in > > > > >> use > > > > >> before.. there should be something in the return value > > > > >> that > > > > >> specifies > > > > >> the error and/or reason for error so that user can try > > > > >> with > > > > >> a > > > > >> new/diff > > > > >> repoID ? > > > > > Asi I said, connect fails if the repoId is in use ATM. > > > > >>>>> disconnectStorageRepository(self, repoId) > > > > >>>>> > > > > >>>>> > > > > >>>>> In the new API there are only images, some images are > > > > >>>>> mutable > > > > >>>>> and > > > > >>>>> some are not. > > > > >>>>> mutable images are also called VirtualDisks > > > > >>>>> immutable images are also called Snapshots > > > > >>>>> > > > > >>>>> There are no explicit templates, you can create as many > > > > >>>>> images > > > > >>>>> as > > > > >>>>> you want from any snapshot. > > > > >>>>> > > > > >>>>> There are 4 major image operations: > > > > >>>>> > > > > >>>>> > > > > >>>>> createVirtualDisk(targetRepoId, size, > > > > >>>>> baseSnapshotId=None, > > > > >>>>> userData={}, options={}): > > > > >>>>> > > > > >>>>> targetRepoId - ID of a connected repo where the disk > > > > >>>>> will > > > > >>>>> be > > > > >>>>> created > > > > >>>>> size - The size of the image you wish to create > > > > >>>>> baseSnapshotId - the ID of the snapshot you want the > > > > >>>>> base > > > > >>>>> the > > > > >>>>> new > > > > >>>>> virtual disk on > > > > >>>>> userData - optional data that will be attached to the > > > > >>>>> new > > > > >>>>> VD, > > > > >>>>> could > > > > >>>>> be anything that the user desires. > > > > >>>>> options - options to modify VDSMs default behavior > > > > >> IIUC, i can use options to do storage offloads ? For eg. I > > > > >> can > > > > >> create > > > > >> a > > > > >> LUN that represents this VD on my storage array based on > > > > >> the > > > > >> 'options' > > > > >> parameter ? Is this the intended way to use 'options' ? > > > > > No, this has nothing to do with offloads. > > > > > If by "offloads" you mean having other VDSM hosts to the > > > > > heavy > > > > > lifting then this is what the option autoFix=False and the > > > > > fix > > > > > mechanism is for. > > > > > If you are talking about advanced scsi features (ie. write > > > > > same) > > > > > they will be used automatically whenever possible. > > > > > In any case, how we manage LUNs (if they are even used) is > > > > > an > > > > > implementation detail. > > > > > > > > I am a bit more interested in how storage array offloads ( by > > > > that > > > > I > > > > mean, offload VD creation, snapshot, clone etc to the storage > > > > array > > > > when > > > > available/possible) can be done from VDSM ? > > > > In the past there were talks of using libSM to do that. How > > > > does > > > > that > > > > strategy play in this new Storage API scenario ? I agree its > > > > implmn > > > > detail, but how & where does that implm sit and how it would > > > > be > > > > triggered is not very clear to me. Looking at createVD args, > > > > it > > > > sounded > > > > like 'options' seems to be a trigger point for deciding > > > > whether > > > > to > > > > use > > > > storage offloads or not, but you spoke otherwise :) Can you > > > > provide > > > > your > > > > vision on how VDSM can understand the storage array > > > > capabilities > > > > & > > > > exploit storgae array offloads in this New Storage API > > > > context > > > > ? > > > > -- > > > > Thanks deepak > > > Some will be used automatically whenever possible (storage > > > offloading). > > > Features that favor a specific strategy will be activated when > > > the > > > proper strategy (space, performance) option is selected. > > > In cases when only the user can know whether to use a feature > > > or > > > not > > > we will have options to turn that on. > > > In any case every domain exports a capability list through > > > GetRepositoryCapabilities() that returns a list off repository > > > capabilities. > > > Some capabilities are VDSM specific like CLUSTERED or > > > REQUIRES_SRM. > > > Some are storage capabilities like NATIVE_SNAPSHOTS, > > > NATIVE_THIN_PROVISIONING, SPARSE_VOLUMES, etc... > > > > > > We are also considering an override mechanism where you can > > > disable > > > features in storage that supports it by setting it in the > > > domain > > > options. This will be done with NO_XXXXX (eg. > > > NO_NATIVE_SNAPSHOTS). > > > This will make the domain not use or expose the capability > > > through > > > the API. I assume it will only be used for testing or in cases > > > where > > > the storage array is known to have problems with a certain > > > feature. > > > Not everything can be disables as an example there is no real > > > way > > > to > > > disable NATIVE_THING_PROVISIONING or SPARSE_VOLUMES. > > > > Saggi, there are several different discussions going on here > > which > > I > > think require some clearing up (and perhaps splitting). > > what I think is missing here: > > 1. distinction of what we believe should be at repo level and > > what > > at > > disk level > > e.g. pre/postZero at repo level, native snapshots as > > described > > above would also be repo level (not defined per disk) etc. > No option is inherently repo or disk based (even the ones you > mantioned). > It all depends on the VDSM version you are interacting with. I am > open to > the suggestion on having a way to query whether certain options are > supported by what operation and in what level dynamically. > > As for post-zeroing, currently it is kind of insulting to our users > that > we push it as a solution to prevent new VMs from reading old VMs > data. It > is just not what it does. It will be replaced with pre-zeroing. > > The only reason I see for post zeroing is the "wipe HDD before > selling it off" > use case but I don't see how it is valid in virtualized > environments > especially > with enterprise storage. I have 2 problems with this: 1. you're assuming 'enterprise' storage and in cloud this is many times not the case at all. On local storage it's pretty safe to assume that what you see is what you get (on block devices at least). 2. we have requests to in fact add additional wiping algorithms to really clean things up exactly for the 'wipe HDD before selling it off' scenario (only valid of course if we actually write on the relevant sectors which is something we cannot guarantee, but with the right disclaimers, this should be fine). Rethinking this, at most post-zero is wasteful, but storage array should guarantee that next time data is read from disk then zeros are served (if it doesn't then it's a security flaw on the storage side which we shouldn't care about). > > > 2. how/where storage offload would work? is there a single > > implementation for repos which detects automatically storage > > capabilities or repo class for each storage type > They will be used automatically if available and supported by VDSM > and the > repo format. It doesn't matter to this section of the API whether > the > user > will have to manually create repositories in specific formats or if > it will > be automatic. As it seems, it will probably be automatically. > > 3. and biggest topic is probably - a mapping of the image > > operations > > and details about the flows (did you send something about that?) > > - > > i.e. create vdisk flow, copy, etc. > The flows you suggest here are basic flows supported with a single > API call. > > Because of the nature of image states users need to have a > mechanism > to > track said image states and make sure fixes are running (if > desired) > > Everyone is welcome to give it a shot and see how they can > implement > high > level flows using this API in their own systems. I don't mind > helping > people > figure it out, just ask. Actually I meant the new images API flows here (i.e. the internals, not how to use it).

> > > > > > > > > > > > > > > > > _______________________________________________ > > > vdsm-devel mailing list > > > vdsm-devel(a)lists.fedorahosted.org > > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > > > > > >

----- Original Message ----- > From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "engine-devel" <engine-devel(a)ovirt.org&gt; > Sent: Thursday, December 6, 2012 11:02:02 AM > Subject: Re: [vdsm] RFC: New Storage API > > Saggi, > > Thanks for sharing your thought and I get some comments below. > > > Saggi Mizrahi: > > I've been throwing a lot of bits out about the new storage API and > > I think it's time to talk a bit. > > I will purposefully try and keep implementation details away and > > concentrate about how the API looks and how you use it. > > > > First major change is in terminology, there is no long a storage > > domain but a storage repository. > > This change is done because so many things are already called > > domain in the system and this will make things less confusing for > > new-commers with a libvirt background. > > > > One other changes is that repositories no longer have a UUID. > > The UUID was only used in the pool members manifest and is no > > longer needed. > > > > > > connectStorageRepository(repoId, repoFormat, > > connectionParameters={}): > > repoId - is a transient name that will be used to refer to the > > connected domain, it is not persisted and doesn't have to be the > > same across the cluster. > > repoFormat - Similar to what used to be type (eg. localfs-1.0, > > nfs-3.4, clvm-1.2). > > connectionParameters - This is format specific and will used to > > tell VDSM how to connect to the repo. > > > Where does repoID come from? I think repoID doesn't exist before > connectStorageRepository() return. Isn't repoID a return value of > connectStorageRepository()? No, repoIDs are no longer part of the domain, they are just a transient handle. The user can put whatever it wants there as long as it isn't already taken by another currently connected domain. > > > > > disconnectStorageRepository(self, repoId) > > > > > > In the new API there are only images, some images are mutable and > > some are not. > > mutable images are also called VirtualDisks > > immutable images are also called Snapshots > > > > There are no explicit templates, you can create as many images as > > you want from any snapshot. > > > > There are 4 major image operations: > > > > > > createVirtualDisk(targetRepoId, size, baseSnapshotId=None, > > userData={}, options={}): > > > > targetRepoId - ID of a connected repo where the disk will be > > created > > size - The size of the image you wish to create > > baseSnapshotId - the ID of the snapshot you want the base the new > > virtual disk on > > userData - optional data that will be attached to the new VD, could > > be anything that the user desires. > > options - options to modify VDSMs default behavior > > > > returns the id of the new VD > > I think we will also need a function to check if a a VirtualDisk is > based on a specific snapshot. > Like: isSnapshotOf(virtualDiskId, baseSnapshotID): No, the design is that volume dependencies are an implementation detail. There is no reason for you to know that an image is physically a snapshot of another. Logical snapshots, template information, and any other information can be set by the user by using the userData field available for every image.

> > createSnapshot(targetRepoId, baseVirtualDiskId, > > userData={}, options={}): > > targetRepoId - The ID of a connected repo where the new sanpshot > > will be created and the original image exists as well. > > size - The size of the image you wish to create > > baseVirtualDisk - the ID of a mutable image (Virtual Disk) you want > > to snapshot > > userData - optional data that will be attached to the new Snapshot, > > could be anything that the user desires. > > options - options to modify VDSMs default behavior > > > > returns the id of the new Snapshot > > > > copyImage(targetRepoId, imageId, baseImageId=None, userData={}, > > options={}) > > targetRepoId - The ID of a connected repo where the new image will > > be created > > imageId - The image you wish to copy > > baseImageId - if specified, the new image will contain only the > > diff between image and Id. > > If None the new image will contain all the bits of > > image Id. This can be used to copy partial parts of > > images for export. > > userData - optional data that will be attached to the new image, > > could be anything that the user desires. > > options - options to modify VDSMs default behavior > > Does this function mean that we can copy the image from one > repository > to another repository? Does it cover the semantics of storage > migration, > storage backup, storage incremental backup? Yes, the main purpose is copying to another repo. and you can even do incremental backups. Also the following flow 1. Run a VM using imageA 2. write to disk 3. Stop VM 4. copy imageA to repoB 5. Run a VM using imageA again 6. Write to disk 7. Stop VM 8. Copy imageA again basing it of imageA_copy1 on repoB creating a diff on repo diff without snapshotting the original image. > > > > > return the Id of the new image. In case of copying an immutable > > image the ID will be identical to the original image as they > > contain the same data. However the user should not assume that and > > always use the value returned from the method. > > > > removeImage(repositoryId, imageId, options={}): > > repositoryId - The ID of a connected repo where the image to delete > > resides > > imageId - The id of the image you wish to delete. > > > > > > ---- > > getImageStatus(repositoryId, imageId) > > repositoryId - The ID of a connected repo where the image to check > > resides > > imageId - The id of the image you wish to check. > > > > All operations return once the operations has been committed to > > disk NOT when the operation actually completes. > > This is done so that: > > - operation come to a stable state as quickly as possible. > > - In case where there is an SDM, only small portion of the > > operation actually needs to be performed on the SDM host. > > - No matter how many times the operation fails and on how many > > hosts, you can always resume the operation and choose when to do > > it. > > - You can stop an operation at any time and remove the resulting > > object making a distinction between "stop because the host is > > overloaded" to "I don't want that image" > > > > This means that after calling any operation that creates a new > > image the user must then call getImageStatus() to check what is > > the status of the image. > > The status of the image can be either optimized, degraded, or > > broken. > > "Optimized" means that the image is available and you can run VMs > > of it. > > "Degraded" means that the image is available and will run VMs but > > it might be a better way VDSM can represent the underlying data. > > What does the "represent" mean here? Anything, but mostly image formate RAW\QCOW2 when performance strategy has been selected. > > "Broken" means that the image can't be used at the moment, probably > > because not all the data has been set up on the volume. > > > > Apart from that VDSM will also return the last persisted status > > information which will conatin > > hostID - the last host to try and optimize of fix the image > Any host can optimize the image? No need to be SDM? On anything but lvm based block domains there will not even be an SDM. On SDM based domains we will try as hard as we can to have as many operations executable on any host. > > > stage - X/Y (eg. 1/10) the last persisted stage of the fix. > > percent_complete - -1 or 0-100, the last persisted completion > > percentage of the aforementioned stage. -1 means that no progress > > is available for that operation. > > last_error - This will only be filled if the operation failed > > because of something other then IO or a VDSM crash for obvious > > reasons. > > It will usually be set if the task was manually > > stopped > > > > The user can either be satisfied with that information or as the > > host specified in host ID if it is still working on that image by > > checking it's running tasks. > > So we need a function to know what tasks are running on the image getImageStatus() > > > > checkStorageRepository(self, repositoryId, options={}): > > A method to go over a storage repository and scan for any existing > > problems. This includes degraded\broken images and deleted images > > that have no yet been physically deleted\merged. > > It returns a list of Fix objects. > > Fix objects come in 4 types: > > clean - cleans data, run them to get more space. > > optimize - run them to optimize a degraded image > > merge - Merges two images together. Doing this sometimes > > makes more images ready optimizing or cleaning. > > The reason it is different from optimize is that > > unmerged images are considered optimized. > > mend - mends a broken image > > > > The user can read these types and prioritize fixes. Fixes also > > contain opaque FIX data and they should be sent as received to > > fixStorageRepository(self, repositoryId, fix, options={}): > > > > That will start a fix operation. > > > > > > All major operations automatically start the appropriate "Fix" to > > bring the created object to an optimize\degraded state (the one > > that is quicker) unless one of the options is > > AutoFix=False. This is only useful for repos that might not be able > > to create volumes on all hosts (SDM) but would like to have the > > actual IO distributed in the cluster. > > > > Other common options is the strategy option: > > It has currently 2 possible values > > space and performance - In case VDSM has 2 ways of completing the > > same operation it will tell it to value one over the other. For > > example, whether to copy all the data or just create a qcow based > > of a snapshot. > > The default is space. > > > > You might have also noticed that it is never explicitly specified > > where to look for existing images. This is done purposefully, VDSM > > will always look in all connected repositories for existing > > objects. > > For very large setups this might be problematic. To mitigate the > > problem you have these options: > > participatingRepositories=[repoId, ...] which tell VDSM to narrow > > the search to just these repositories > > and > > imageHints={imgId: repoId} which will force VDSM to look for those > > image ID just in those repositories and fail if it doesn't find > > them there. > > _______________________________________________ > > vdsm-devel mailing list > > vdsm-devel(a)lists.fedorahosted.org > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > > > -- > --- > 舒明 Shu Ming > Open Virtualization Engineerning; CSTL, IBM Corp. > Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming(a)cn.ibm.com or > shuming(a)linux.vnet.ibm.com > Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian > District, Beijing 100193, PRC > > > _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

> Statements like this make me start to worry about your userData > concept. It's a > sign of a bad API if the user needs to invent a custom metadata > scheme for > itself. This reminds me of the abomination that is the 'custom' > property in the > vm definition today. In one sentence: If VDSM doesn't care about it, VDSM doesn't manage it. userData being a "void*" is quite common and I don't understand why you would thing it's a sign of a bad API. Further more, giving the user choice about how to represent it's own metadata and what fields it want to keep seems reasonable to me. Especially given the fact that VDSM never reads it. The reason we are pulling away from the current system of VDSM understanding the extra data is that it makes that data tied to VDSMs on disk format. VDSM on disk format has to be very stable because of clusters with multiple VDSM versions. Further more, since this is actually manager data it has to be tied to the manager backward compatibility lifetime as well. Having it be opaque to VDSM ties it to only one, simpler, support lifetime instead of two. I guess you are implying that it will make it problematic for multiple users to read userData left by another user because the formats might not be compatible. The solution is that all parties interested in using VDSM storage agree on format, and common fields, and supportability, and all the other things that choosing a supporting *something* entails. This is, however, out of the scope of VDSM. When the time comes I think how the userData blob is actually parsed and what fields it keeps should be discussed on ovirt-devel or engine-devel. The crux of the issue is that VDSM manages only what it cares about and the user can't modify directly. This is done because everything we expose we commit to. If you want any information persisted like: - Human readable name (in whatever encoding) - Is this a template or a snapshot - What user owns this image You can just put it in the userData. VDSM is not going to impose what encoding you use. It's not going to decide if you represent your users as IDs or names or ldap queries or Public Keys. It's not going to decide if you have explicit templates or not. It's not going to decide if you care what is the logical image chain. It's not going to decide anything that is out of it's scope. No format is future proof, no selection of fields will be good for any situation. I'd much rather it be someone else's problem when any of them need to be changed. They have currently been VDSMs problem and it has been hell to maintain.

----- Original Message ----- > From: "Adam Litke" <agl(a)us.ibm.com&gt; > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > Cc: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;, "engine-devel" <engine-devel(a)ovirt.org&gt;, "VDSM Project Development" > <vdsm-devel(a)lists.fedorahosted.org&gt; > Sent: Monday, December 10, 2012 1:39:51 PM > Subject: Re: [vdsm] RFC: New Storage API > > On Thu, Dec 06, 2012 at 11:52:01AM -0500, Saggi Mizrahi wrote: >> >> ----- Original Message ----- >>> From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt; >>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; >>> Cc: "VDSM Project Development" >>> <vdsm-devel(a)lists.fedorahosted.org&gt;, "engine-devel" >>> <engine-devel(a)ovirt.org&gt; >>> Sent: Thursday, December 6, 2012 11:02:02 AM >>> Subject: Re: [vdsm] RFC: New Storage API >>> >>> Saggi, >>> >>> Thanks for sharing your thought and I get some comments below. >>> >>> >>> Saggi Mizrahi: >>>> I've been throwing a lot of bits out about the new storage API >>>> and >>>> I think it's time to talk a bit. >>>> I will purposefully try and keep implementation details away >>>> and >>>> concentrate about how the API looks and how you use it. >>>> >>>> First major change is in terminology, there is no long a >>>> storage >>>> domain but a storage repository. >>>> This change is done because so many things are already called >>>> domain in the system and this will make things less confusing >>>> for >>>> new-commers with a libvirt background. >>>> >>>> One other changes is that repositories no longer have a UUID. >>>> The UUID was only used in the pool members manifest and is no >>>> longer needed. >>>> >>>> >>>> connectStorageRepository(repoId, repoFormat, >>>> connectionParameters={}): >>>> repoId - is a transient name that will be used to refer to the >>>> connected domain, it is not persisted and doesn't have to be >>>> the >>>> same across the cluster. >>>> repoFormat - Similar to what used to be type (eg. localfs-1.0, >>>> nfs-3.4, clvm-1.2). >>>> connectionParameters - This is format specific and will used to >>>> tell VDSM how to connect to the repo. >>> >>> Where does repoID come from? I think repoID doesn't exist before >>> connectStorageRepository() return. Isn't repoID a return value >>> of >>> connectStorageRepository()? >> No, repoIDs are no longer part of the domain, they are just a >> transient handle. >> The user can put whatever it wants there as long as it isn't >> already taken by another currently connected domain. >>>> disconnectStorageRepository(self, repoId) >>>> >>>> >>>> In the new API there are only images, some images are mutable >>>> and >>>> some are not. >>>> mutable images are also called VirtualDisks >>>> immutable images are also called Snapshots >>>> >>>> There are no explicit templates, you can create as many images >>>> as >>>> you want from any snapshot. >>>> >>>> There are 4 major image operations: >>>> >>>> >>>> createVirtualDisk(targetRepoId, size, baseSnapshotId=None, >>>> userData={}, options={}): >>>> >>>> targetRepoId - ID of a connected repo where the disk will be >>>> created >>>> size - The size of the image you wish to create >>>> baseSnapshotId - the ID of the snapshot you want the base the >>>> new >>>> virtual disk on >>>> userData - optional data that will be attached to the new VD, >>>> could >>>> be anything that the user desires. >>>> options - options to modify VDSMs default behavior >>>> >>>> returns the id of the new VD >>> I think we will also need a function to check if a a VirtualDisk >>> is >>> based on a specific snapshot. >>> Like: isSnapshotOf(virtualDiskId, baseSnapshotID): >> No, the design is that volume dependencies are an implementation >> detail. >> There is no reason for you to know that an image is physically a >> snapshot of another. >> Logical snapshots, template information, and any other information >> can be set by the user by using the userData field available for >> every image. > Statements like this make me start to worry about your userData > concept. It's a > sign of a bad API if the user needs to invent a custom metadata > scheme for > itself. This reminds me of the abomination that is the 'custom' > property in the > vm definition today. In one sentence: If VDSM doesn't care about it, VDSM doesn't manage it. userData being a "void*" is quite common and I don't understand why you would thing it's a sign of a bad API. Further more, giving the user choice about how to represent it's own metadata and what fields it want to keep seems reasonable to me. Especially given the fact that VDSM never reads it. The reason we are pulling away from the current system of VDSM understanding the extra data is that it makes that data tied to VDSMs on disk format. VDSM on disk format has to be very stable because of clusters with multiple VDSM versions. Further more, since this is actually manager data it has to be tied to the manager backward compatibility lifetime as well. Having it be opaque to VDSM ties it to only one, simpler, support lifetime instead of two.

I guess you are implying that it will make it problematic for multiple users to read userData left by another user because the formats might not be compatible. The solution is that all parties interested in using VDSM storage agree on format, and common fields, and supportability, and all the other things that choosing a supporting *something* entails. This is, however, out of the scope of VDSM. When the time comes I think how the userData blob is actually parsed and what fields it keeps should be discussed on ovirt-devel or engine-devel. The crux of the issue is that VDSM manages only what it cares about and the user can't modify directly. This is done because everything we expose we commit to. If you want any information persisted like: - Human readable name (in whatever encoding) - Is this a template or a snapshot - What user owns this image You can just put it in the userData. VDSM is not going to impose what encoding you use. It's not going to decide if you represent your users as IDs or names or ldap queries or Public Keys. It's not going to decide if you have explicit templates or not. It's not going to decide if you care what is the logical image chain. It's not going to decide anything that is out of it's scope. No format is future proof, no selection of fields will be good for any situation. I'd much rather it be someone else's problem when any of them need to be changed. They have currently been VDSMs problem and it has been hell to maintain. >>>> createSnapshot(targetRepoId, baseVirtualDiskId, >>>> userData={}, options={}): >>>> targetRepoId - The ID of a connected repo where the new >>>> sanpshot >>>> will be created and the original image exists as well. >>>> size - The size of the image you wish to create >>>> baseVirtualDisk - the ID of a mutable image (Virtual Disk) you >>>> want >>>> to snapshot >>>> userData - optional data that will be attached to the new >>>> Snapshot, >>>> could be anything that the user desires. >>>> options - options to modify VDSMs default behavior >>>> >>>> returns the id of the new Snapshot >>>> >>>> copyImage(targetRepoId, imageId, baseImageId=None, userData={}, >>>> options={}) >>>> targetRepoId - The ID of a connected repo where the new image >>>> will >>>> be created >>>> imageId - The image you wish to copy >>>> baseImageId - if specified, the new image will contain only the >>>> diff between image and Id. >>>> If None the new image will contain all the bits >>>> of >>>> image Id. This can be used to copy partial parts >>>> of >>>> images for export. >>>> userData - optional data that will be attached to the new >>>> image, >>>> could be anything that the user desires. >>>> options - options to modify VDSMs default behavior >>> Does this function mean that we can copy the image from one >>> repository >>> to another repository? Does it cover the semantics of storage >>> migration, >>> storage backup, storage incremental backup? >> Yes, the main purpose is copying to another repo. and you can even >> do incremental backups. >> Also the following flow >> 1. Run a VM using imageA >> 2. write to disk >> 3. Stop VM >> 4. copy imageA to repoB >> 5. Run a VM using imageA again >> 6. Write to disk >> 7. Stop VM >> 8. Copy imageA again basing it of imageA_copy1 on repoB creating a >> diff on repo diff without snapshotting the original image. >> >>>> return the Id of the new image. In case of copying an immutable >>>> image the ID will be identical to the original image as they >>>> contain the same data. However the user should not assume that >>>> and >>>> always use the value returned from the method. >>>> >>>> removeImage(repositoryId, imageId, options={}): >>>> repositoryId - The ID of a connected repo where the image to >>>> delete >>>> resides >>>> imageId - The id of the image you wish to delete. >>>> >>>> >>>> ---- >>>> getImageStatus(repositoryId, imageId) >>>> repositoryId - The ID of a connected repo where the image to >>>> check >>>> resides >>>> imageId - The id of the image you wish to check. >>>> >>>> All operations return once the operations has been committed to >>>> disk NOT when the operation actually completes. >>>> This is done so that: >>>> - operation come to a stable state as quickly as possible. >>>> - In case where there is an SDM, only small portion of the >>>> operation actually needs to be performed on the SDM host. >>>> - No matter how many times the operation fails and on how many >>>> hosts, you can always resume the operation and choose when to >>>> do >>>> it. >>>> - You can stop an operation at any time and remove the >>>> resulting >>>> object making a distinction between "stop because the host is >>>> overloaded" to "I don't want that image" >>>> >>>> This means that after calling any operation that creates a new >>>> image the user must then call getImageStatus() to check what is >>>> the status of the image. >>>> The status of the image can be either optimized, degraded, or >>>> broken. >>>> "Optimized" means that the image is available and you can run >>>> VMs >>>> of it. >>>> "Degraded" means that the image is available and will run VMs >>>> but >>>> it might be a better way VDSM can represent the underlying >>>> data. >>> What does the "represent" mean here? >> Anything, but mostly image formate RAW\QCOW2 when performance >> strategy has been selected. >>>> "Broken" means that the image can't be used at the moment, >>>> probably >>>> because not all the data has been set up on the volume. >>>> >>>> Apart from that VDSM will also return the last persisted status >>>> information which will conatin >>>> hostID - the last host to try and optimize of fix the image >>> Any host can optimize the image? No need to be SDM? >> On anything but lvm based block domains there will not even be an >> SDM. >> On SDM based domains we will try as hard as we can to have as many >> operations executable on any host. >>>> stage - X/Y (eg. 1/10) the last persisted stage of the fix. >>>> percent_complete - -1 or 0-100, the last persisted completion >>>> percentage of the aforementioned stage. -1 means that no >>>> progress >>>> is available for that operation. >>>> last_error - This will only be filled if the operation failed >>>> because of something other then IO or a VDSM crash for obvious >>>> reasons. >>>> It will usually be set if the task was manually >>>> stopped >>>> >>>> The user can either be satisfied with that information or as >>>> the >>>> host specified in host ID if it is still working on that image >>>> by >>>> checking it's running tasks. >>> So we need a function to know what tasks are running on the image >> getImageStatus() >>>> checkStorageRepository(self, repositoryId, options={}): >>>> A method to go over a storage repository and scan for any >>>> existing >>>> problems. This includes degraded\broken images and deleted >>>> images >>>> that have no yet been physically deleted\merged. >>>> It returns a list of Fix objects. >>>> Fix objects come in 4 types: >>>> clean - cleans data, run them to get more space. >>>> optimize - run them to optimize a degraded image >>>> merge - Merges two images together. Doing this sometimes >>>> makes more images ready optimizing or cleaning. >>>> The reason it is different from optimize is that >>>> unmerged images are considered optimized. >>>> mend - mends a broken image >>>> >>>> The user can read these types and prioritize fixes. Fixes also >>>> contain opaque FIX data and they should be sent as received to >>>> fixStorageRepository(self, repositoryId, fix, options={}): >>>> >>>> That will start a fix operation. >>>> >>>> >>>> All major operations automatically start the appropriate "Fix" >>>> to >>>> bring the created object to an optimize\degraded state (the one >>>> that is quicker) unless one of the options is >>>> AutoFix=False. This is only useful for repos that might not be >>>> able >>>> to create volumes on all hosts (SDM) but would like to have the >>>> actual IO distributed in the cluster. >>>> >>>> Other common options is the strategy option: >>>> It has currently 2 possible values >>>> space and performance - In case VDSM has 2 ways of completing >>>> the >>>> same operation it will tell it to value one over the other. For >>>> example, whether to copy all the data or just create a qcow >>>> based >>>> of a snapshot. >>>> The default is space. >>>> >>>> You might have also noticed that it is never explicitly >>>> specified >>>> where to look for existing images. This is done purposefully, >>>> VDSM >>>> will always look in all connected repositories for existing >>>> objects. >>>> For very large setups this might be problematic. To mitigate >>>> the >>>> problem you have these options: >>>> participatingRepositories=[repoId, ...] which tell VDSM to >>>> narrow >>>> the search to just these repositories >>>> and >>>> imageHints={imgId: repoId} which will force VDSM to look for >>>> those >>>> image ID just in those repositories and fail if it doesn't find >>>> them there. >>>> _______________________________________________ >>>> vdsm-devel mailing list >>>> vdsm-devel(a)lists.fedorahosted.org >>>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel >>> >>> -- >>> --- >>> 舒明 Shu Ming >>> Open Virtualization Engineerning; CSTL, IBM Corp. >>> Tel: 86-10-82451626 Tieline: 9051626 E-mail: shuming(a)cn.ibm.com >>> or >>> shuming(a)linux.vnet.ibm.com >>> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian >>> District, Beijing 100193, PRC >>> >>> >>> >> _______________________________________________ >> vdsm-devel mailing list >> vdsm-devel(a)lists.fedorahosted.org >> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > -- > Adam Litke <agl(a)us.ibm.com&gt; > IBM Linux Technology Center > >

I've been throwing a lot of bits out about the new storage API and I think it's time to talk a bit. I will purposefully try and keep implementation details away and concentrate about how the API looks and how you use it. First major change is in terminology, there is no long a storage domain but a storage repository. This change is done because so many things are already called domain in the system and this will make things less confusing for new-commers with a libvirt background. One other changes is that repositories no longer have a UUID. The UUID was only used in the pool members manifest and is no longer needed. connectStorageRepository(repoId, repoFormat, connectionParameters={}): repoId - is a transient name that will be used to refer to the connected domain, it is not persisted and doesn't have to be the same across the cluster. repoFormat - Similar to what used to be type (eg. localfs-1.0, nfs-3.4, clvm-1.2). connectionParameters - This is format specific and will used to tell VDSM how to connect to the repo. disconnectStorageRepository(self, repoId): In the new API there are only images, some images are mutable and some are not. mutable images are also called VirtualDisks immutable images are also called Snapshots There are no explicit templates, you can create as many images as you want from any snapshot. There are 4 major image operations: createVirtualDisk(targetRepoId, size, baseSnapshotId=None, userData={}, options={}): targetRepoId - ID of a connected repo where the disk will be created size - The size of the image you wish to create baseSnapshotId - the ID of the snapshot you want the base the new virtual disk on userData - optional data that will be attached to the new VD, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new VD createSnapshot(targetRepoId, baseVirtualDiskId, userData={}, options={}): targetRepoId - The ID of a connected repo where the new sanpshot will be created and the original image exists as well. size - The size of the image you wish to create baseVirtualDisk - the ID of a mutable image (Virtual Disk) you want to snapshot userData - optional data that will be attached to the new Snapshot, could be anything that the user desires. options - options to modify VDSMs default behavior returns the id of the new Snapshot copyImage(targetRepoId, imageId, baseImageId=None, userData={}, options={}) targetRepoId - The ID of a connected repo where the new image will be created imageId - The image you wish to copy baseImageId - if specified, the new image will contain only the diff between image and Id. If None the new image will contain all the bits of image Id. This can be used to copy partial parts of images for export. userData - optional data that will be attached to the new image, could be anything that the user desires. options - options to modify VDSMs default behavior return the Id of the new image. In case of copying an immutable image the ID will be identical to the original image as they contain the same data. However the user should not assume that and always use the value returned from the method. removeImage(repositoryId, imageId, options={}): repositoryId - The ID of a connected repo where the image to delete resides imageId - The id of the image you wish to delete. ---- getImageStatus(repositoryId, imageId) repositoryId - The ID of a connected repo where the image to check resides imageId - The id of the image you wish to check. All operations return once the operations has been committed to disk NOT when the operation actually completes. This is done so that: - operation come to a stable state as quickly as possible. - In case where there is an SDM, only small portion of the operation actually needs to be performed on the SDM host. - No matter how many times the operation fails and on how many hosts, you can always resume the operation and choose when to do it. - You can stop an operation at any time and remove the resulting object making a distinction between "stop because the host is overloaded" to "I don't want that image" This means that after calling any operation that creates a new image the user must then call getImageStatus() to check what is the status of the image. The status of the image can be either optimized, degraded, or broken. "Optimized" means that the image is available and you can run VMs of it. "Degraded" means that the image is available and will run VMs but it might be a better way VDSM can represent the underlying data. "Broken" means that the image can't be used at the moment, probably because not all the data has been set up on the volume. Apart from that VDSM will also return the last persisted status information which will conatin hostID - the last host to try and optimize of fix the image stage - X/Y (eg. 1/10) the last persisted stage of the fix. percent_complete - -1 or 0-100, the last persisted completion percentage of the aforementioned stage. -1 means that no progress is available for that operation. last_error - This will only be filled if the operation failed because of something other then IO or a VDSM crash for obvious reasons. It will usually be set if the task was manually stopped The user can either be satisfied with that information or as the host specified in host ID if it is still working on that image by checking it's running tasks. checkStorageRepository(self, repositoryId, options={}): A method to go over a storage repository and scan for any existing problems. This includes degraded\broken images and deleted images that have no yet been physically deleted\merged. It returns a list of Fix objects. Fix objects come in 4 types: clean - cleans data, run them to get more space. optimize - run them to optimize a degraded image merge - Merges two images together. Doing this sometimes makes more images ready optimizing or cleaning. The reason it is different from optimize is that unmerged images are considered optimized. mend - mends a broken image The user can read these types and prioritize fixes. Fixes also contain opaque FIX data and they should be sent as received to fixStorageRepository(self, repositoryId, fix, options={}): That will start a fix operation. All major operations automatically start the appropriate "Fix" to bring the created object to an optimize\degraded state (the one that is quicker) unless one of the options is AutoFix=False. This is only useful for repos that might not be able to create volumes on all hosts (SDM) but would like to have the actual IO distributed in the cluster. Other common options is the strategy option: It has currently 2 possible values space and performance - In case VDSM has 2 ways of completing the same operation it will tell it to value one over the other. For example, whether to copy all the data or just create a qcow based of a snapshot. The default is space. You might have also noticed that it is never explicitly specified where to look for existing images. This is done purposefully, VDSM will always look in all connected repositories for existing objects. For very large setups this might be problematic. To mitigate the problem you have these options: participatingRepositories=[repoId, ...] which tell VDSM to narrow the search to just these repositories and imageHints={imgId: repoId} which will force VDSM to look for those image ID just in those repositories and fail if it doesn't find them there. _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

----- Original Message ----- > From: "Itamar Heim" <iheim(a)redhat.com&gt; > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt; > Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, > "engine-devel" <engine-devel(a)ovirt.org&gt; > Sent: Monday, January 14, 2013 6:18:13 AM > Subject: Re: [vdsm] RFC: New Storage API > > On 12/04/2012 11:52 PM, Saggi Mizrahi wrote: > > I've been throwing a lot of bits out about the new storage API > > and > > I think it's time to talk a bit. > > I will purposefully try and keep implementation details away and > > concentrate about how the API looks and how you use it. > > > > First major change is in terminology, there is no long a storage > > domain but a storage repository. > > This change is done because so many things are already called > > domain in the system and this will make things less confusing for > > new-commers with a libvirt background. > > > > One other changes is that repositories no longer have a UUID. > > The UUID was only used in the pool members manifest and is no > > longer needed. > > > > > > connectStorageRepository(repoId, repoFormat, > > connectionParameters={}): > > repoId - is a transient name that will be used to refer to the > > connected domain, it is not persisted and doesn't have to be the > > same across the cluster. > > repoFormat - Similar to what used to be type (eg. localfs-1.0, > > nfs-3.4, clvm-1.2). > > connectionParameters - This is format specific and will used to > > tell VDSM how to connect to the repo. > > > > disconnectStorageRepository(self, repoId): > > > > > > In the new API there are only images, some images are mutable and > > some are not. > > mutable images are also called VirtualDisks > > immutable images are also called Snapshots > > > > There are no explicit templates, you can create as many images as > > you want from any snapshot. > > > > There are 4 major image operations: > > > > > > createVirtualDisk(targetRepoId, size, baseSnapshotId=None, > > userData={}, options={}): > > > > targetRepoId - ID of a connected repo where the disk will be > > created > > size - The size of the image you wish to create > > baseSnapshotId - the ID of the snapshot you want the base the new > > virtual disk on > > userData - optional data that will be attached to the new VD, > > could > > be anything that the user desires. > > options - options to modify VDSMs default behavior > > > > returns the id of the new VD > > > > createSnapshot(targetRepoId, baseVirtualDiskId, > > userData={}, options={}): > > targetRepoId - The ID of a connected repo where the new sanpshot > > will be created and the original image exists as well. > > size - The size of the image you wish to create > > baseVirtualDisk - the ID of a mutable image (Virtual Disk) you > > want > > to snapshot > > userData - optional data that will be attached to the new > > Snapshot, > > could be anything that the user desires. > > options - options to modify VDSMs default behavior > > > > returns the id of the new Snapshot > > > > copyImage(targetRepoId, imageId, baseImageId=None, userData={}, > > options={}) > > targetRepoId - The ID of a connected repo where the new image > > will > > be created > > imageId - The image you wish to copy > > baseImageId - if specified, the new image will contain only the > > diff between image and Id. > > If None the new image will contain all the bits of > > image Id. This can be used to copy partial parts > > of > > images for export. > > userData - optional data that will be attached to the new image, > > could be anything that the user desires. > > options - options to modify VDSMs default behavior > > > > return the Id of the new image. In case of copying an immutable > > image the ID will be identical to the original image as they > > contain the same data. However the user should not assume that > > and > > always use the value returned from the method. > > > > removeImage(repositoryId, imageId, options={}): > > repositoryId - The ID of a connected repo where the image to > > delete > > resides > > imageId - The id of the image you wish to delete. > > > > > > ---- > > getImageStatus(repositoryId, imageId) > > repositoryId - The ID of a connected repo where the image to > > check > > resides > > imageId - The id of the image you wish to check. > > > > All operations return once the operations has been committed to > > disk NOT when the operation actually completes. > > This is done so that: > > - operation come to a stable state as quickly as possible. > > - In case where there is an SDM, only small portion of the > > operation actually needs to be performed on the SDM host. > > - No matter how many times the operation fails and on how many > > hosts, you can always resume the operation and choose when to do > > it. > > - You can stop an operation at any time and remove the resulting > > object making a distinction between "stop because the host is > > overloaded" to "I don't want that image" > > > > This means that after calling any operation that creates a new > > image the user must then call getImageStatus() to check what is > > the status of the image. > > The status of the image can be either optimized, degraded, or > > broken. > > "Optimized" means that the image is available and you can run VMs > > of it. > > "Degraded" means that the image is available and will run VMs but > > it might be a better way VDSM can represent the underlying data. > > "Broken" means that the image can't be used at the moment, > > probably > > because not all the data has been set up on the volume. > > > > Apart from that VDSM will also return the last persisted status > > information which will conatin > > hostID - the last host to try and optimize of fix the image > > stage - X/Y (eg. 1/10) the last persisted stage of the fix. > > percent_complete - -1 or 0-100, the last persisted completion > > percentage of the aforementioned stage. -1 means that no progress > > is available for that operation. > > last_error - This will only be filled if the operation failed > > because of something other then IO or a VDSM crash for obvious > > reasons. > > It will usually be set if the task was manually > > stopped > > > > The user can either be satisfied with that information or as the > > host specified in host ID if it is still working on that image by > > checking it's running tasks. > > > > checkStorageRepository(self, repositoryId, options={}): > > A method to go over a storage repository and scan for any > > existing > > problems. This includes degraded\broken images and deleted images > > that have no yet been physically deleted\merged. > > It returns a list of Fix objects. > > Fix objects come in 4 types: > > clean - cleans data, run them to get more space. > > optimize - run them to optimize a degraded image > > merge - Merges two images together. Doing this sometimes > > makes more images ready optimizing or cleaning. > > The reason it is different from optimize is that > > unmerged images are considered optimized. > > mend - mends a broken image > > > > The user can read these types and prioritize fixes. Fixes also > > contain opaque FIX data and they should be sent as received to > > fixStorageRepository(self, repositoryId, fix, options={}): > > > > That will start a fix operation. > > > > > > All major operations automatically start the appropriate "Fix" to > > bring the created object to an optimize\degraded state (the one > > that is quicker) unless one of the options is > > AutoFix=False. This is only useful for repos that might not be > > able > > to create volumes on all hosts (SDM) but would like to have the > > actual IO distributed in the cluster. > > > > Other common options is the strategy option: > > It has currently 2 possible values > > space and performance - In case VDSM has 2 ways of completing the > > same operation it will tell it to value one over the other. For > > example, whether to copy all the data or just create a qcow based > > of a snapshot. > > The default is space. > > > > You might have also noticed that it is never explicitly specified > > where to look for existing images. This is done purposefully, > > VDSM > > will always look in all connected repositories for existing > > objects. > > For very large setups this might be problematic. To mitigate the > > problem you have these options: > > participatingRepositories=[repoId, ...] which tell VDSM to narrow > > the search to just these repositories > > and > > imageHints={imgId: repoId} which will force VDSM to look for > > those > > image ID just in those repositories and fail if it doesn't find > > them there. > > _______________________________________________ > > vdsm-devel mailing list > > vdsm-devel(a)lists.fedorahosted.org > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > > > > you are using VirtualDisk and Snapshot for mutable and immutable. > the terminology of "image" for immutable and "volume" for mutable > seems > to be the one used by ec2/openstack/etc. - any thoughts on using > similar > terminology? > (though i think they also differ in forcing all images to be in an > image > repo, and all volumes in a volume repo, while we allow to mix them > in > same repo). > We have volumes internally though, I wanted to call them slabs but Ayal and Edu feel strongly about calling them volumes and the current naming system in general.

_______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

2013-1-15 5:34, Ayal Baron: >image and volume are overused everywhere and it would be extremely confusing to have multiple meanings to the same terms in the same system (we have image today which means virtual disk and volume which means a part of a virtual disk). >Personally I don't like the distinction between image and volume done in ec2/openstack/etc seeing as they're treated as different types of entities there while the only real difference is mutability (images are read-only, volumes are read-write). >To move to the industry terminology we would need to first change all references we have today to image and volume in the system (I would say also in ovirt-engine side) to align with the new meaning. >Despite my personal dislike of the terms, I definitely see the value in converging on the same terminology as the rest of the industry but to do so would be an arduous task which is out of scope of this discussion imo (patches welcome though ;) Another distinction between Openstack and oVirt is how the Nova/ovirt-engine look upon storage systems. In Openstack, a stand alone storage service(Cinder) exports the raw storage block device to Nova. On the other hand, in oVirt, storage system is highly bounded with the cluster scheduling system which integrates storage sub-system, VM dispatching sub-system, ISO image sub systems. This combination make all of the sub-system integrated in a whole which is easy to deploy, but it make the sub-system more opaque and not harder to reuse and maintain. This new storage API proposal give us an opportunity to distinct these sub-systems as new components which export better, loose-coupling APIs to VDSM.