3:30 p.m.
--_000_CC3DB8AB143Crickyhnetappcom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Hey all,
As I'm making progress with the domain rescan functionality, I've realized =
that I'm unsure what to do with any disks that are detected on the domain. =
Should I add them back into the database to be listed as floating disks, or=
should I just return a list of disk images to be attached to whatever the =
caller of the query needs?
- Ricky
--_000_CC3DB8AB143Crickyhnetappcom_
Content-Type: text/html; charset="us-ascii"
Content-ID: <AFED18B724BEEE4683245253976CBF7B(a)tahoe.netapp.com Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Dus-ascii"=
</head <body
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-fami=
ly: Calibri, sans-serif; " <div>Hey all,</div <div><br
</div <div>As I'm making
progress with the domain rescan functionality, I've real=
ized that I'm unsure what to do with any disks that are detected on the dom=
ain. Should I add them back into the database to be listed as floating disk=
s, or should I just return a list
of disk images to be attached to whatever the caller of the query needs?</=
div <div><br
</div <div>- Ricky</div </body
</html
--_000_CC3DB8AB143Crickyhnetappcom_--
3:44 p.m.
On 07/31/2012 11:30 PM, Hopper, Ricky wrote:
i'm not sure they should be added automatically.
I think a dialog[1] showing orphan disks/images on the storage domain
for user to choose which to import as 'floating' disks would be better
than auto importing them.
there is also the reverse of flagging existing disks as 'missing' in
storage?
[1] or a subtab on the storage domain.
4:21 p.m.
----- Original Message -----
why? this same functionality would be used to import an existing domain.
If these disks are referenced in OVFs we are not familiar with on this domain then we
should import the *VMs*.
If they are referenced by other VMs that are already in the system (but disks have been
unreachable until now) then the disks should just be added to the db in attached mode.
If neither, then the disks should be added as floating disks.
For the import functionality, once you subsequently import another domain with OVFs which
do reference these disks then if user hasn't appropriated them for other VMs then they
would move to attached state, otherwise need to add those VMs with errors.
Note that there are 2 reasons for unknown valid disks to be on the domain:
1. delete was initiated in engine but not performed on storage (then floating is fine as
only way to automatically delete them is if user chooses to do so)
2. disks were created there outside of the system - should just detect and import and use
logic above.
there is also the reverse of flagging existing disks as 'missing' in
storage?
If disks were floating then they should just be removed, otherwise should be moved to
illegal state (we have this state for disks today).
[1] or a subtab on the storage domain.
Another sub-tab for disks?
It's possible but what would you do when importing an existing domain into the system?
require user to manually select which disks to import? or would you have different flows
for import of domain and rescan of contents? (I'd rather keep it simple with less tabs
and manual operations for user to perform, seems more intuitive to me).
11:45 p.m.
On 08/01/2012 12:21 AM, Ayal Baron wrote:
I'm not sure orphaed images and importing an entire domain should be the
same flow from UX part.
and for a netapp native clone, you'd want to only import the newly
cloned ("orphaned) image from the storage via an api call, not the
entire domain/images
2:48 a.m.
----- Original Message -----
For proper bookkeeping I think we'd want these disks to be reflected in the GUI. I do
not however think we need to add additional tabs for this.
We could have yet another state ('orphaned' or sth.) for disks to be able to
easily differentiate in the GUI etc.
I would imagine we'd regularly scan domains to account for storage usage.
If we do add such a state then we'd need an API to 'enable' it (so that it
could be done automatically).
4:24 p.m.
----- Original Message -----
Perhaps we should start a feature page to discuss and better scope it.
There is a feature page that we could expand, it doesn't discuss the notion of
importing those disks which is certainly something we need to address.
http://wiki.ovirt.org/wiki/Features/Orphaned_Images
4:59 a.m.
----- Original Message -----
The original idea was to scan the storage domains and compare the images lists to the
database, thus getting a list of images no longer relevant and scrubbing the storage. This
will actually be addressed properly in the future (Ayal can elaborate on that) but for now
this is needed at least for that use case.
As I understand, the conversation here is about trying to take an already populated SD
(from another setup I suppose), scanning it and putting it into RHEV?
--
Regards,
Dan Yasny
Red Hat Israel
+972 9769 2280
8:34 a.m.
On 8/1/12 5:59 AM, "Dan Yasny" <dyasny(a)redhat.com> wrote:
As I understood it, the purpose of this functionality wasn't to find
images which should be removed from storage, but to find images on the
domain that oVirt was unaware of and importing them for use (for instance,
if a disk was created outside of oVirt on the domain). If one of the use
cases for this feature is also the orphaned images mentioned on the
feature page, that may expand the functionality into a separate domain
scrub and storage import, both of which would call the rescan (meaning the
rescan would not actually add to the database, but instead return a list
of "orphaned" disk images).
Another solution would be to import all disk images into the database
either way, and let the user delete any orphaned images from the GUI.
8:42 a.m.
----- Original Message -----
I think are nice to have, but the problem with the scanning is that if we're not
scanning a master domain or an export domain, all we will see is a bunch of images with no
context or even hints as to where they belong. The data that makes it all usable is in the
engine database and in the ovf files on the master domain.
This is why I stopped at the orphaned images part of the feature - because there it's
feasible, I would rely on the engine database for image ID comparisons.
If we present a user with a list of nameless disks, I doubt it will be of any use.
--
Regards,
Dan Yasny
Red Hat Israel
+972 9769 2280
8:56 a.m.
On 8/1/12 9:42 AM, "Dan Yasny" <dyasny(a)redhat.com> wrote:
The way this would work is by comparing a list of disk images from vdsm
and from oVirt's database, finding the ones vdsm returns that oVirt
doesn't have, and then either adding or returning those images. So oVirt's
db will be used in the comparison.
As far as presenting the user with nameless disks, that's a point I hadn't
considered; we could generate some sort of placeholder metadata upon
addition to show the user that these are new/orphaned disks that were
found on the storage domain. Is it safe to assume that the disks
discovered by this feature won't be attached to anything?
9:05 a.m.
----- Original Message -----
This will work only when scanning storage domains already attached and in use by the
current oVirt setup. What I am talking about is what will happen if a LUN that used to be
a SD in another oVirt setup is discovered and scanned, with no engine db to compare with.
If we don't consider such a use case, life is definitely quite easy, and we're
basically within the scope of the orphaned images feature
The oVirt paradigm says "if it isn't in the engine db, it's not ours",
so any LV or image we discover that is missing from the DB or the snapshot chain of the
image in the DB, is nameless, and orphaned.
Such an image on a current SD, belonging to a working oVirt setup is definitely an
orphaned image. Attaching these to VMs is usually also useless, because they are more
often than not discarded snapshots that didn't get discarded cleanly for some reason.
Now, if we want to make this usable, we might want to actually check the qcow2 metadata of
the image to see whether it's a mid-chain snapshot (and if so it's probably just a
candidate for cleanup), or a standalone qcow2 or raw image, and then we can move on with
the virt-* tools, to find out the image size and the filesystems it contains. This will at
least provide the user with some usable information about the detected image.
If we're talking about scanning an SD that doesn't presently belong to the current
oVirt setup, then this is even more relevant, because all of the images will have no
VM-related context.
--
Regards,
Dan Yasny
Red Hat Israel
+972 9769 2280
12:36 p.m.
On 8/1/12 10:05 AM, "Dan Yasny" <dyasny(a)redhat.com> wrote:
This use case should definitely be considered, maybe have a separate case
where the rescan would return all "compatible" disks (i.e. disks that
aren't just partial snapshots and the like) if the domain has not yet been
mounted. Essentially, it would run the same comparison, but compare
against an empty list rather than a list of disks. There's no way it's as
simple as that (I'm unsure of the methods oVirt uses to mount a domain),
but it's a good starting point.
We're currently working on having disks created outside of the oVirt
environment, so not all orphaned disks on the existing storage domain will
be artifacts of supposedly-deleted data. For our use case, disk images
created by us will be able to be imported into oVirt and attached to a VM
created through the engine. Because of this, saying "if it isn't in the
engine db, it's not ours" wouldn't necessarily be true.
When you talk about checking the metadata, does either oVirt or vdsm have
a simple way to do this? A query of some sort would be ideal for this, as
it could be run for each image as a qualifier for import.
Also, as far as writing the functionality itself, I'm gathering that it
should be structured as a query to return these orphaned images, which can
then be acted upon/added to the database through a separate command after
checking the validity of each image?
1:35 p.m.
----- Original Message -----
There is no complex method there. For file storage it's just a mount command, and for
block it's LVM (plus iscsi session establishment, if needed)
Do you mean like rhevm-image-upload, or something different?
qemu-img info and libguestfs commands should do.
Besides, our images do come with some metadata (in the LVM tags or a .meta file)
Yes, a simple way to say "import this one to DB, attach to VM X or make
floating", "delete that one", "skip"
--
Regards,
Dan Yasny
Red Hat Israel
+972 9769 2280
9:37 a.m.
In the interest of good discussion, we've put up a feature page for this
feature (http://wiki.ovirt.org/wiki/Features/Domain_Scan), which links to
a talk page where modifications can be proposed to how I've laid out the
feature. So far, it covers how the query works and which commands will
come about to implement it. I'd appreciate it if anyone concerned could
check this out and make any changes as they see fit so we can get going
with the coding.
- Ricky
On 8/1/12 2:35 PM, "Dan Yasny" <dyasny(a)redhat.com> wrote:
8:44 p.m.
No comments below. I am just curious how you can have disk created
outside of the oVirt environment if the storage domain is mounted on
oVirt engine. Does your application try to communicate with VDSM though
SPM host to do that? Are the two applications(oVirt engine and your
application) coexisting and manipulating the storage domain at the same
time or one application is quiet when the other one is functioning?
IMHO, modifying the storage domain meta data from two application are
dangerous and I suppose your application should know the reason and what
these disks are for. So telling oVirt to import the disks
automatically should be reasonable because you know what these disks are
for and it is unnecessary for the oVirt administrator to select which
one to import.
On 2012-8-2 1:36, Hopper, Ricky wrote:
--
Shu Ming <shuming(a)linux.vnet.ibm.com>
IBM China Systems and Technology Laboratory
4628
days inactive
4630
days old
17 comments
7 participants
participants (7)
-
Andrew Cathrow -
Ayal Baron -
Dan Yasny -
Hopper, Ricky -
Itamar Heim -
Omer Frenkel -
Shu Ming