----- Original Message ----- > From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; > To: devel(a)ovirt.org > Sent: Sunday, May 4, 2014 8:08:33 PM > Subject: [ovirt-devel] vdsm disabling logical volumes > > Greetings vdsm developers! > > While working on adding ISCSI support to the hosted engine tools, I ran > into a problem with vdms. It seems that when stopped vdsm deactivates > ALL logical volumes in it's volume group and when it starts it > reactivates only specific logical volumes. This is a problem for hosted > engine tools as they create logical volumes in the same volume group and > when vdsm deactivates the LVs the hosted engine tools don't have a way > to reactivate it, because the services drop the root permissions and are > running as vdsm and apparently only root can activate LVs. Can you describe what volumes are you creating, and why?

> So far the > only suitable solution seems to be to change vdsm to only > deactivate/activate it's own LVs. This sounds reasonable. You can add a list of hosted engine lv names and skip these volumes when deactivating vdsm volumes.

Another solution is to tag hosted engine lvs, and have vdsm ignore lvs that contains this tag.

Nir

From: "Nir Soffer" <nsoffer(a)redhat.com&gt; To: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; Cc: devel(a)ovirt.org Sent: Monday, May 5, 2014 1:01:12 AM Subject: Re: [ovirt-devel] vdsm disabling logical volumes ----- Original Message ----- > From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; > To: "Nir Soffer" <nsoffer(a)redhat.com&gt; > Cc: devel(a)ovirt.org > Sent: Sunday, May 4, 2014 9:23:49 PM > Subject: Re: [ovirt-devel] vdsm disabling logical volumes > > On 05/04/2014 07:57 PM, Nir Soffer wrote: > > ----- Original Message ----- > >> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; > >> To: devel(a)ovirt.org > >> Sent: Sunday, May 4, 2014 8:08:33 PM > >> Subject: [ovirt-devel] vdsm disabling logical volumes > >> > >> Greetings vdsm developers! > >> > >> While working on adding ISCSI support to the hosted engine tools, I ran > >> into a problem with vdms. It seems that when stopped vdsm deactivates > >> ALL logical volumes in it's volume group and when it starts it > >> reactivates only specific logical volumes. This is a problem for hosted > >> engine tools as they create logical volumes in the same volume group and > >> when vdsm deactivates the LVs the hosted engine tools don't have a way > >> to reactivate it, because the services drop the root permissions and are > >> running as vdsm and apparently only root can activate LVs. > > > > Can you describe what volumes are you creating, and why? > > We create hosted-engine.lockspace (for sanlock) and > hosted-engine.metadata (keeps data about hosted engine hosts) Do you create these lvs in every vdsm vg? Is this part of the domain structure used by hosted engine, or it has nothing to do with the storage domain? > > > > >> So far the > >> only suitable solution seems to be to change vdsm to only > >> deactivate/activate it's own LVs. > > > > This sounds reasonable. You can add a list of hosted engine lv names > > and skip these volumes when deactivating vdsm volumes. > > - this sounds a bit suboptimal, vdsm already has list of it's LVs, so it > can just disable only LVs known to it, otherwise we would have to change > the list everytime we add some LV to the group vdsm has a list of special lvs, that needs special treatment. Otherwise, it consider any other lv as owned by vdsm, and will deactivate them when they are not used. I agree that this will create a dependency, but this can also be solved. For example, vdsm can load the list from a file installed by hosted engine, like the typical conf.d directories. > > > > > Another solution is to tag hosted engine lvs, and have vdsm ignore > > lvs that contains this tag. > > - this sounds good, because if we teach vdsm to ignore LVs with some tag > we can add new LVs in future without changing vdsm. This however applies > also to the solution where vdsm only disables it's own LVs, vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like this using some historic tags we keep (e.g. RHAT_*), but I'd rather add new tag with clear semantic than use some random historic value we have. > so it > depends on vdsm devels which solution they find better. I think the > solution without tags is better, because is simpler and others (like > hosted-engine) can just createlv and don't bother with tags.. I think that a generic tag like OVIRT_IGNORE is an easy solution for everyone. Federico, what do you think?

Nir _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; To: "Nir Soffer" <nsoffer(a)redhat.com&gt; Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg Padgett" <gpadgett(a)redhat.com&gt; Sent: Monday, May 5, 2014 3:16:37 PM Subject: Re: [ovirt-devel] vdsm disabling logical volumes On 05/05/2014 12:01 AM, Nir Soffer wrote: > ----- Original Message ----- >> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >> Cc: devel(a)ovirt.org >> Sent: Sunday, May 4, 2014 9:23:49 PM >> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >> >> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>> ----- Original Message ----- >>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>> To: devel(a)ovirt.org >>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>> >>>> Greetings vdsm developers! >>>> >>>> While working on adding ISCSI support to the hosted engine tools, I ran >>>> into a problem with vdms. It seems that when stopped vdsm deactivates >>>> ALL logical volumes in it's volume group and when it starts it >>>> reactivates only specific logical volumes. This is a problem for hosted >>>> engine tools as they create logical volumes in the same volume group and >>>> when vdsm deactivates the LVs the hosted engine tools don't have a way >>>> to reactivate it, because the services drop the root permissions and are >>>> running as vdsm and apparently only root can activate LVs. >>> >>> Can you describe what volumes are you creating, and why? >> >> We create hosted-engine.lockspace (for sanlock) and >> hosted-engine.metadata (keeps data about hosted engine hosts) > > Do you create these lvs in every vdsm vg? - only in the first vg created by vdsm while deploying hosted-engine > Is this part of the domain structure > used by hosted engine, or it has nothing to do with the storage domain? - sorry, I don't understand this question. How can I tell if it has something to do with the storage domain? It's for storing data about hosts set up to run the hosted-engine and data about state of engine and the state of VM running the engine.

> >> >>> >>>> So far the >>>> only suitable solution seems to be to change vdsm to only >>>> deactivate/activate it's own LVs. >>> >>> This sounds reasonable. You can add a list of hosted engine lv names >>> and skip these volumes when deactivating vdsm volumes. >> >> - this sounds a bit suboptimal, vdsm already has list of it's LVs, so it >> can just disable only LVs known to it, otherwise we would have to change >> the list everytime we add some LV to the group > > vdsm has a list of special lvs, that needs special treatment. Otherwise, it > consider any other lv as owned by vdsm, and will deactivate them when they > are > not used. > > I agree that this will create a dependency, but this can also be solved. > For example, vdsm can load the list from a file installed by hosted engine, > like the typical conf.d directories. > - ok, this is something I actually don't have strong opinion about, for me adding a file with a list of LVs or tagging the logical volumes is almost the same, I just need a way to tell vdsm which LVs to ignore.. >> >>> >>> Another solution is to tag hosted engine lvs, and have vdsm ignore >>> lvs that contains this tag. >> >> - this sounds good, because if we teach vdsm to ignore LVs with some tag >> we can add new LVs in future without changing vdsm. This however applies >> also to the solution where vdsm only disables it's own LVs, > > vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like > this > using some historic tags we keep (e.g. RHAT_*), but I'd rather add new tag > with > clear semantic than use some random historic value we have. > >> so it >> depends on vdsm devels which solution they find better. I think the >> solution without tags is better, because is simpler and others (like >> hosted-engine) can just createlv and don't bother with tags.. > > I think that a generic tag like OVIRT_IGNORE is an easy solution for > everyone. > > Federico, what do you think? > > Nir >

From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; To: "Nir Soffer" <nsoffer(a)redhat.com&gt; Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg Padgett" <gpadgett(a)redhat.com&gt; Sent: Monday, May 5, 2014 3:44:21 PM Subject: Re: [ovirt-devel] vdsm disabling logical volumes On 05/05/2014 02:37 PM, Nir Soffer wrote: > ----- Original Message ----- >> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >> Mureinik" <amureini(a)redhat.com&gt;, "Greg >> Padgett" <gpadgett(a)redhat.com&gt; >> Sent: Monday, May 5, 2014 3:16:37 PM >> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >> >> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>> ----- Original Message ----- >>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>> Cc: devel(a)ovirt.org >>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>> >>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>> ----- Original Message ----- >>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>> To: devel(a)ovirt.org >>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>> >>>>>> Greetings vdsm developers! >>>>>> >>>>>> While working on adding ISCSI support to the hosted engine tools, I >>>>>> ran >>>>>> into a problem with vdms. It seems that when stopped vdsm deactivates >>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>> reactivates only specific logical volumes. This is a problem for >>>>>> hosted >>>>>> engine tools as they create logical volumes in the same volume group >>>>>> and >>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a way >>>>>> to reactivate it, because the services drop the root permissions and >>>>>> are >>>>>> running as vdsm and apparently only root can activate LVs. >>>>> >>>>> Can you describe what volumes are you creating, and why? >>>> >>>> We create hosted-engine.lockspace (for sanlock) and >>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>> >>> Do you create these lvs in every vdsm vg? >> >> - only in the first vg created by vdsm while deploying hosted-engine

>> >>> Is this part of the domain structure >>> used by hosted engine, or it has nothing to do with the storage domain? >> >> - sorry, I don't understand this question. How can I tell if it has >> something to do with the storage domain? It's for storing data about >> hosts set up to run the hosted-engine and data about state of engine and >> the state of VM running the engine. > > Can you tell us exactly what lvs you are creating, and on which vg? > > And how are you creating those lvs - I guess through vdsm? > - no hosted-engine tools do that by calling: lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", "-n", lv_name, vg_uuid]) ..

> The output of lvs command on a host with hosted engine installed will > help us to understand what you are doing, and then we can think more > clearly > what would be the best way to support this in vdsm. The output of lvs: http://fpaste.org/99196/93619139/ HE created these two LVs: ha_agent-hosted-engine.lockspace ha_agent-hosted-engine.metadata

--Jirka > >> >>> >>>> >>>>> >>>>>> So far the >>>>>> only suitable solution seems to be to change vdsm to only >>>>>> deactivate/activate it's own LVs. >>>>> >>>>> This sounds reasonable. You can add a list of hosted engine lv names >>>>> and skip these volumes when deactivating vdsm volumes. >>>> >>>> - this sounds a bit suboptimal, vdsm already has list of it's LVs, so it >>>> can just disable only LVs known to it, otherwise we would have to change >>>> the list everytime we add some LV to the group >>> >>> vdsm has a list of special lvs, that needs special treatment. Otherwise, >>> it >>> consider any other lv as owned by vdsm, and will deactivate them when >>> they >>> are >>> not used. >>> >>> I agree that this will create a dependency, but this can also be solved. >>> For example, vdsm can load the list from a file installed by hosted >>> engine, >>> like the typical conf.d directories. >>> >> >> - ok, this is something I actually don't have strong opinion about, for >> me adding a file with a list of LVs or tagging the logical volumes is >> almost the same, I just need a way to tell vdsm which LVs to ignore.. >> >>>> >>>>> >>>>> Another solution is to tag hosted engine lvs, and have vdsm ignore >>>>> lvs that contains this tag. >>>> >>>> - this sounds good, because if we teach vdsm to ignore LVs with some tag >>>> we can add new LVs in future without changing vdsm. This however applies >>>> also to the solution where vdsm only disables it's own LVs, >>> >>> vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like >>> this >>> using some historic tags we keep (e.g. RHAT_*), but I'd rather add new >>> tag >>> with >>> clear semantic than use some random historic value we have. >>> >>>> so it >>>> depends on vdsm devels which solution they find better. I think the >>>> solution without tags is better, because is simpler and others (like >>>> hosted-engine) can just createlv and don't bother with tags.. >>> >>> I think that a generic tag like OVIRT_IGNORE is an easy solution for >>> everyone. >>> >>> Federico, what do you think? >>> >>> Nir >>> >> >>

On 05/05/2014 03:19 PM, Nir Soffer wrote: > ----- Original Message ----- >> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg >> Padgett" <gpadgett(a)redhat.com&gt; >> Sent: Monday, May 5, 2014 3:44:21 PM >> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >> >> On 05/05/2014 02:37 PM, Nir Soffer wrote: >>> ----- Original Message ----- >>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>> Padgett" <gpadgett(a)redhat.com&gt; >>>> Sent: Monday, May 5, 2014 3:16:37 PM >>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>> >>>> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>>>> ----- Original Message ----- >>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>> Cc: devel(a)ovirt.org >>>>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>> >>>>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>>>> ----- Original Message ----- >>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>> To: devel(a)ovirt.org >>>>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>>>> >>>>>>>> Greetings vdsm developers! >>>>>>>> >>>>>>>> While working on adding ISCSI support to the hosted engine tools, I >>>>>>>> ran >>>>>>>> into a problem with vdms. It seems that when stopped vdsm deactivates >>>>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>>>> reactivates only specific logical volumes. This is a problem for >>>>>>>> hosted >>>>>>>> engine tools as they create logical volumes in the same volume group >>>>>>>> and >>>>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a way >>>>>>>> to reactivate it, because the services drop the root permissions and >>>>>>>> are >>>>>>>> running as vdsm and apparently only root can activate LVs. >>>>>>> >>>>>>> Can you describe what volumes are you creating, and why? >>>>>> >>>>>> We create hosted-engine.lockspace (for sanlock) and >>>>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>>>> >>>>> Do you create these lvs in every vdsm vg? >>>> >>>> - only in the first vg created by vdsm while deploying hosted-engine > > It seems that the hosted engine has single point of failure - the random > vg that contains hosted engine data. - yes, seems like it, but that's for another discussion > >>>> >>>>> Is this part of the domain structure >>>>> used by hosted engine, or it has nothing to do with the storage domain? >>>> >>>> - sorry, I don't understand this question. How can I tell if it has >>>> something to do with the storage domain? It's for storing data about >>>> hosts set up to run the hosted-engine and data about state of engine and >>>> the state of VM running the engine. >>> >>> Can you tell us exactly what lvs you are creating, and on which vg? >>> >>> And how are you creating those lvs - I guess through vdsm? >>> >> >> - no hosted-engine tools do that by calling: >> >> lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, >> stderr=subprocess.PIPE, >> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", >> "-n", lv_name, vg_uuid]) >> .. > > How do you ensure that another host is not modifying the same vg in the > same time? > > If you are not ensuring this, you will corrupt this vg sooner or later. > > When a storage domain is detached from a host, for example when the host > is in maintenance mode, lvs on the shared storage may be deleted, invalidating > the devices mapper maps for these devices. If you write to an lv with wrong > maps, you may be writing to an extent belonging to another lv, corrupting that > lv data, or even worse corrupting the engine vg data. > - when the vdsm wants to move some host to maintenance it contacts the ha-agent which stops writing data to the storage, but it might be a problem if the VG is changed while in maintenance, I don't think we handle such situation > How do you ensure that the lvs are not deleted while you are using them? - obviously we're not handling this, otherwise the vdsm wouldn't be able to disable those LVs - I'm no expert on lvm, so I could use some advice on how to do it, so how does the vdsm ensures that it's VG is not modified? > >> >>> The output of lvs command on a host with hosted engine installed will >>> help us to understand what you are doing, and then we can think more >>> clearly >>> what would be the best way to support this in vdsm. >> >> The output of lvs: http://fpaste.org/99196/93619139/ >> >> HE created these two LVs: >> ha_agent-hosted-engine.lockspace >> ha_agent-hosted-engine.metadata > > Why do you create these lvs on a vg owned by vdsm? > - I actually don't know this decision has been taken before I started working on the code > If you want total control of these lvs, I suggest that you create your own > vg and put what ever lvs you like there. > Yes, that would solve the problem with vdsm, I hope Sandro or Martin could explain why we use the vdsm's VG instead of creating our own...

--Jirka >> >> --Jirka >>> >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> So far the >>>>>>>> only suitable solution seems to be to change vdsm to only >>>>>>>> deactivate/activate it's own LVs. >>>>>>> >>>>>>> This sounds reasonable. You can add a list of hosted engine lv names >>>>>>> and skip these volumes when deactivating vdsm volumes. >>>>>> >>>>>> - this sounds a bit suboptimal, vdsm already has list of it's LVs, so it >>>>>> can just disable only LVs known to it, otherwise we would have to change >>>>>> the list everytime we add some LV to the group >>>>> >>>>> vdsm has a list of special lvs, that needs special treatment. Otherwise, >>>>> it >>>>> consider any other lv as owned by vdsm, and will deactivate them when >>>>> they >>>>> are >>>>> not used. >>>>> >>>>> I agree that this will create a dependency, but this can also be solved. >>>>> For example, vdsm can load the list from a file installed by hosted >>>>> engine, >>>>> like the typical conf.d directories. >>>>> >>>> >>>> - ok, this is something I actually don't have strong opinion about, for >>>> me adding a file with a list of LVs or tagging the logical volumes is >>>> almost the same, I just need a way to tell vdsm which LVs to ignore.. >>>> >>>>>> >>>>>>> >>>>>>> Another solution is to tag hosted engine lvs, and have vdsm ignore >>>>>>> lvs that contains this tag. >>>>>> >>>>>> - this sounds good, because if we teach vdsm to ignore LVs with some tag >>>>>> we can add new LVs in future without changing vdsm. This however applies >>>>>> also to the solution where vdsm only disables it's own LVs, >>>>> >>>>> vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like >>>>> this >>>>> using some historic tags we keep (e.g. RHAT_*), but I'd rather add new >>>>> tag >>>>> with >>>>> clear semantic than use some random historic value we have. >>>>> >>>>>> so it >>>>>> depends on vdsm devels which solution they find better. I think the >>>>>> solution without tags is better, because is simpler and others (like >>>>>> hosted-engine) can just createlv and don't bother with tags.. >>>>> >>>>> I think that a generic tag like OVIRT_IGNORE is an easy solution for >>>>> everyone. >>>>> >>>>> Federico, what do you think? >>>>> >>>>> Nir >>>>> >>>> >>>> >> >>

From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; To: "Nir Soffer" <nsoffer(a)redhat.com&gt; Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg Padgett" <gpadgett(a)redhat.com&gt;, "Doron Fediuck" <dfediuck(a)redhat.com&gt; Sent: Wednesday, May 7, 2014 10:21:28 AM Subject: Re: [ovirt-devel] vdsm disabling logical volumes On 05/05/2014 03:19 PM, Nir Soffer wrote: > ----- Original Message ----- >> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >> Mureinik" <amureini(a)redhat.com&gt;, "Greg >> Padgett" <gpadgett(a)redhat.com&gt; >> Sent: Monday, May 5, 2014 3:44:21 PM >> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >> >> On 05/05/2014 02:37 PM, Nir Soffer wrote: >>> ----- Original Message ----- >>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>> Padgett" <gpadgett(a)redhat.com&gt; >>>> Sent: Monday, May 5, 2014 3:16:37 PM >>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>> >>>> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>>>> ----- Original Message ----- >>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>> Cc: devel(a)ovirt.org >>>>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>> >>>>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>>>> ----- Original Message ----- >>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>> To: devel(a)ovirt.org >>>>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>>>> >>>>>>>> Greetings vdsm developers! >>>>>>>> >>>>>>>> While working on adding ISCSI support to the hosted engine tools, I >>>>>>>> ran >>>>>>>> into a problem with vdms. It seems that when stopped vdsm >>>>>>>> deactivates >>>>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>>>> reactivates only specific logical volumes. This is a problem for >>>>>>>> hosted >>>>>>>> engine tools as they create logical volumes in the same volume group >>>>>>>> and >>>>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a >>>>>>>> way >>>>>>>> to reactivate it, because the services drop the root permissions and >>>>>>>> are >>>>>>>> running as vdsm and apparently only root can activate LVs. >>>>>>> >>>>>>> Can you describe what volumes are you creating, and why? >>>>>> >>>>>> We create hosted-engine.lockspace (for sanlock) and >>>>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>>>> >>>>> Do you create these lvs in every vdsm vg? >>>> >>>> - only in the first vg created by vdsm while deploying hosted-engine > > It seems that the hosted engine has single point of failure - the random > vg that contains hosted engine data. > >>>> >>>>> Is this part of the domain structure >>>>> used by hosted engine, or it has nothing to do with the storage domain? >>>> >>>> - sorry, I don't understand this question. How can I tell if it has >>>> something to do with the storage domain? It's for storing data about >>>> hosts set up to run the hosted-engine and data about state of engine and >>>> the state of VM running the engine. >>> >>> Can you tell us exactly what lvs you are creating, and on which vg? >>> >>> And how are you creating those lvs - I guess through vdsm? >>> >> >> - no hosted-engine tools do that by calling: >> >> lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, >> stderr=subprocess.PIPE, >> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", >> "-n", lv_name, vg_uuid]) >> .. > > How do you ensure that another host is not modifying the same vg in the > same time? > > If you are not ensuring this, you will corrupt this vg sooner or later. > > When a storage domain is detached from a host, for example when the host > is in maintenance mode, lvs on the shared storage may be deleted, > invalidating > the devices mapper maps for these devices. If you write to an lv with wrong > maps, you may be writing to an extent belonging to another lv, corrupting > that > lv data, or even worse corrupting the engine vg data. > > How do you ensure that the lvs are not deleted while you are using them? > >> >>> The output of lvs command on a host with hosted engine installed will >>> help us to understand what you are doing, and then we can think more >>> clearly >>> what would be the best way to support this in vdsm. >> >> The output of lvs: http://fpaste.org/99196/93619139/ >> >> HE created these two LVs: >> ha_agent-hosted-engine.lockspace >> ha_agent-hosted-engine.metadata > > Why do you create these lvs on a vg owned by vdsm? > > If you want total control of these lvs, I suggest that you create your own > vg and put what ever lvs you like there. > I would rather not go this way (at least not for 3.5) as it's too much code changes in hosted-engine. On the other hand the logic in vdsm seems wrong because it's not complementary (disabling all LVs and then enabling just some of them) and should be fixed anyway. This problem is blocking one of our 3.5 features so I've created rhbz#1094657 to track it.

--Jirka >> >> --Jirka >>> >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> So far the >>>>>>>> only suitable solution seems to be to change vdsm to only >>>>>>>> deactivate/activate it's own LVs. >>>>>>> >>>>>>> This sounds reasonable. You can add a list of hosted engine lv names >>>>>>> and skip these volumes when deactivating vdsm volumes. >>>>>> >>>>>> - this sounds a bit suboptimal, vdsm already has list of it's LVs, so >>>>>> it >>>>>> can just disable only LVs known to it, otherwise we would have to >>>>>> change >>>>>> the list everytime we add some LV to the group >>>>> >>>>> vdsm has a list of special lvs, that needs special treatment. >>>>> Otherwise, >>>>> it >>>>> consider any other lv as owned by vdsm, and will deactivate them when >>>>> they >>>>> are >>>>> not used. >>>>> >>>>> I agree that this will create a dependency, but this can also be >>>>> solved. >>>>> For example, vdsm can load the list from a file installed by hosted >>>>> engine, >>>>> like the typical conf.d directories. >>>>> >>>> >>>> - ok, this is something I actually don't have strong opinion about, for >>>> me adding a file with a list of LVs or tagging the logical volumes is >>>> almost the same, I just need a way to tell vdsm which LVs to ignore.. >>>> >>>>>> >>>>>>> >>>>>>> Another solution is to tag hosted engine lvs, and have vdsm ignore >>>>>>> lvs that contains this tag. >>>>>> >>>>>> - this sounds good, because if we teach vdsm to ignore LVs with some >>>>>> tag >>>>>> we can add new LVs in future without changing vdsm. This however >>>>>> applies >>>>>> also to the solution where vdsm only disables it's own LVs, >>>>> >>>>> vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like >>>>> this >>>>> using some historic tags we keep (e.g. RHAT_*), but I'd rather add new >>>>> tag >>>>> with >>>>> clear semantic than use some random historic value we have. >>>>> >>>>>> so it >>>>>> depends on vdsm devels which solution they find better. I think the >>>>>> solution without tags is better, because is simpler and others (like >>>>>> hosted-engine) can just createlv and don't bother with tags.. >>>>> >>>>> I think that a generic tag like OVIRT_IGNORE is an easy solution for >>>>> everyone. >>>>> >>>>> Federico, what do you think? >>>>> >>>>> Nir >>>>> >>>> >>>> >> >>

On 05/07/2014 09:28 AM, Nir Soffer wrote: >----- Original Message ----- >>From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg >>Padgett" <gpadgett(a)redhat.com&gt;, "Doron Fediuck" <dfediuck(a)redhat.com&gt; >>Sent: Wednesday, May 7, 2014 10:21:28 AM >>Subject: Re: [ovirt-devel] vdsm disabling logical volumes >> >>On 05/05/2014 03:19 PM, Nir Soffer wrote: >>>----- Original Message ----- >>>>From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>>Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>Padgett" <gpadgett(a)redhat.com&gt; >>>>Sent: Monday, May 5, 2014 3:44:21 PM >>>>Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>> >>>>On 05/05/2014 02:37 PM, Nir Soffer wrote: >>>>>----- Original Message ----- >>>>>>From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>>>>Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>>Padgett" <gpadgett(a)redhat.com&gt; >>>>>>Sent: Monday, May 5, 2014 3:16:37 PM >>>>>>Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>> >>>>>>On 05/05/2014 12:01 AM, Nir Soffer wrote: >>>>>>>----- Original Message ----- >>>>>>>>From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>>Cc: devel(a)ovirt.org >>>>>>>>Sent: Sunday, May 4, 2014 9:23:49 PM >>>>>>>>Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>> >>>>>>>>On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>>>>>>----- Original Message ----- >>>>>>>>>>From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>>To: devel(a)ovirt.org >>>>>>>>>>Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>>>>>>Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>> >>>>>>>>>>Greetings vdsm developers! >>>>>>>>>> >>>>>>>>>>While working on adding ISCSI support to the hosted engine tools, I >>>>>>>>>>ran >>>>>>>>>>into a problem with vdms. It seems that when stopped vdsm >>>>>>>>>>deactivates >>>>>>>>>>ALL logical volumes in it's volume group and when it starts it >>>>>>>>>>reactivates only specific logical volumes. This is a problem for >>>>>>>>>>hosted >>>>>>>>>>engine tools as they create logical volumes in the same volume group >>>>>>>>>>and >>>>>>>>>>when vdsm deactivates the LVs the hosted engine tools don't have a >>>>>>>>>>way >>>>>>>>>>to reactivate it, because the services drop the root permissions and >>>>>>>>>>are >>>>>>>>>>running as vdsm and apparently only root can activate LVs. >>>>>>>>> >>>>>>>>>Can you describe what volumes are you creating, and why? >>>>>>>> >>>>>>>>We create hosted-engine.lockspace (for sanlock) and >>>>>>>>hosted-engine.metadata (keeps data about hosted engine hosts) >>>>>>> >>>>>>>Do you create these lvs in every vdsm vg? >>>>>> >>>>>>- only in the first vg created by vdsm while deploying hosted-engine >>> >>>It seems that the hosted engine has single point of failure - the random >>>vg that contains hosted engine data. >>> >>>>>> >>>>>>>Is this part of the domain structure >>>>>>>used by hosted engine, or it has nothing to do with the storage domain? >>>>>> >>>>>>- sorry, I don't understand this question. How can I tell if it has >>>>>>something to do with the storage domain? It's for storing data about >>>>>>hosts set up to run the hosted-engine and data about state of engine and >>>>>>the state of VM running the engine. >>>>> >>>>>Can you tell us exactly what lvs you are creating, and on which vg? >>>>> >>>>>And how are you creating those lvs - I guess through vdsm? >>>>> >>>> >>>>- no hosted-engine tools do that by calling: >>>> >>>>lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, >>>> stderr=subprocess.PIPE, >>>> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", >>>> "-n", lv_name, vg_uuid]) >>>>.. >>> >>>How do you ensure that another host is not modifying the same vg in the >>>same time? >>> >>>If you are not ensuring this, you will corrupt this vg sooner or later. >>> >>>When a storage domain is detached from a host, for example when the host >>>is in maintenance mode, lvs on the shared storage may be deleted, >>>invalidating >>>the devices mapper maps for these devices. If you write to an lv with wrong >>>maps, you may be writing to an extent belonging to another lv, corrupting >>>that >>>lv data, or even worse corrupting the engine vg data. >>> >>>How do you ensure that the lvs are not deleted while you are using them? >>> >>>> >>>>>The output of lvs command on a host with hosted engine installed will >>>>>help us to understand what you are doing, and then we can think more >>>>>clearly >>>>>what would be the best way to support this in vdsm. >>>> >>>>The output of lvs: http://fpaste.org/99196/93619139/ >>>> >>>>HE created these two LVs: >>>>ha_agent-hosted-engine.lockspace >>>>ha_agent-hosted-engine.metadata >>> >>>Why do you create these lvs on a vg owned by vdsm? >>> >>>If you want total control of these lvs, I suggest that you create your own >>>vg and put what ever lvs you like there. >>> >> >>I would rather not go this way (at least not for 3.5) as it's too much >>code changes in hosted-engine. On the other hand the logic in vdsm seems >>wrong because it's not complementary (disabling all LVs and then >>enabling just some of them) and should be fixed anyway. This problem is >>blocking one of our 3.5 features so I've created rhbz#1094657 to track it. > >Can you elaborate on this? How should vdsm behave better, and why? Sure. So far I didn't hear any reason why it behaves like this and it seems not logical to disable *all* and then enable just *some*. How: Disabling and enabling operations should be complementary. Why: To be less surprising.

On 05/07/2014 11:37 AM, Dan Kenigsberg wrote: > On Wed, May 07, 2014 at 09:56:03AM +0200, Jiri Moskovcak wrote: >> On 05/07/2014 09:28 AM, Nir Soffer wrote: >>> ----- Original Message ----- >>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, >>>> "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>> Padgett" <gpadgett(a)redhat.com&gt;, "Doron Fediuck" <dfediuck(a)redhat.com&gt; >>>> Sent: Wednesday, May 7, 2014 10:21:28 AM >>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>> >>>> On 05/05/2014 03:19 PM, Nir Soffer wrote: >>>>> ----- Original Message ----- >>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, >>>>>> "Allon >>>>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>> Padgett" <gpadgett(a)redhat.com&gt; >>>>>> Sent: Monday, May 5, 2014 3:44:21 PM >>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>> >>>>>> On 05/05/2014 02:37 PM, Nir Soffer wrote: >>>>>>> ----- Original Message ----- >>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" >>>>>>>> <fsimonce(a)redhat.com&gt;, "Allon >>>>>>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>>>> Padgett" <gpadgett(a)redhat.com&gt; >>>>>>>> Sent: Monday, May 5, 2014 3:16:37 PM >>>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>> >>>>>>>> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>>>>>>>> ----- Original Message ----- >>>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>>>> Cc: devel(a)ovirt.org >>>>>>>>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>>>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>> >>>>>>>>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>>>>>>>> ----- Original Message ----- >>>>>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>>>> To: devel(a)ovirt.org >>>>>>>>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>>>>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>>>> >>>>>>>>>>>> Greetings vdsm developers! >>>>>>>>>>>> >>>>>>>>>>>> While working on adding ISCSI support to the hosted engine >>>>>>>>>>>> tools, I >>>>>>>>>>>> ran >>>>>>>>>>>> into a problem with vdms. It seems that when stopped vdsm >>>>>>>>>>>> deactivates >>>>>>>>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>>>>>>>> reactivates only specific logical volumes. This is a >>>>>>>>>>>> problem for >>>>>>>>>>>> hosted >>>>>>>>>>>> engine tools as they create logical volumes in the same >>>>>>>>>>>> volume group >>>>>>>>>>>> and >>>>>>>>>>>> when vdsm deactivates the LVs the hosted engine tools don't >>>>>>>>>>>> have a >>>>>>>>>>>> way >>>>>>>>>>>> to reactivate it, because the services drop the root >>>>>>>>>>>> permissions and >>>>>>>>>>>> are >>>>>>>>>>>> running as vdsm and apparently only root can activate LVs. >>>>>>>>>>> >>>>>>>>>>> Can you describe what volumes are you creating, and why? >>>>>>>>>> >>>>>>>>>> We create hosted-engine.lockspace (for sanlock) and >>>>>>>>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>>>>>>>> >>>>>>>>> Do you create these lvs in every vdsm vg? >>>>>>>> >>>>>>>> - only in the first vg created by vdsm while deploying >>>>>>>> hosted-engine >>>>> >>>>> It seems that the hosted engine has single point of failure - the >>>>> random >>>>> vg that contains hosted engine data. >>>>> >>>>>>>> >>>>>>>>> Is this part of the domain structure >>>>>>>>> used by hosted engine, or it has nothing to do with the >>>>>>>>> storage domain? >>>>>>>> >>>>>>>> - sorry, I don't understand this question. How can I tell if it >>>>>>>> has >>>>>>>> something to do with the storage domain? It's for storing data >>>>>>>> about >>>>>>>> hosts set up to run the hosted-engine and data about state of >>>>>>>> engine and >>>>>>>> the state of VM running the engine. >>>>>>> >>>>>>> Can you tell us exactly what lvs you are creating, and on which vg? >>>>>>> >>>>>>> And how are you creating those lvs - I guess through vdsm? >>>>>>> >>>>>> >>>>>> - no hosted-engine tools do that by calling: >>>>>> >>>>>> lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, >>>>>> stderr=subprocess.PIPE, >>>>>> args=["lvm", "lvcreate", "-L", >>>>>> str(size_bytes)+"B", >>>>>> "-n", lv_name, vg_uuid]) >>>>>> .. >>>>> >>>>> How do you ensure that another host is not modifying the same vg >>>>> in the >>>>> same time? >>>>> >>>>> If you are not ensuring this, you will corrupt this vg sooner or >>>>> later. >>>>> >>>>> When a storage domain is detached from a host, for example when >>>>> the host >>>>> is in maintenance mode, lvs on the shared storage may be deleted, >>>>> invalidating >>>>> the devices mapper maps for these devices. If you write to an lv >>>>> with wrong >>>>> maps, you may be writing to an extent belonging to another lv, >>>>> corrupting >>>>> that >>>>> lv data, or even worse corrupting the engine vg data. >>>>> >>>>> How do you ensure that the lvs are not deleted while you are using >>>>> them? >>>>> >>>>>> >>>>>>> The output of lvs command on a host with hosted engine installed >>>>>>> will >>>>>>> help us to understand what you are doing, and then we can think >>>>>>> more >>>>>>> clearly >>>>>>> what would be the best way to support this in vdsm. >>>>>> >>>>>> The output of lvs: http://fpaste.org/99196/93619139/ >>>>>> >>>>>> HE created these two LVs: >>>>>> ha_agent-hosted-engine.lockspace >>>>>> ha_agent-hosted-engine.metadata >>>>> >>>>> Why do you create these lvs on a vg owned by vdsm? >>>>> >>>>> If you want total control of these lvs, I suggest that you create >>>>> your own >>>>> vg and put what ever lvs you like there. >>>>> >>>> >>>> I would rather not go this way (at least not for 3.5) as it's too much >>>> code changes in hosted-engine. On the other hand the logic in vdsm >>>> seems >>>> wrong because it's not complementary (disabling all LVs and then >>>> enabling just some of them) and should be fixed anyway. This >>>> problem is >>>> blocking one of our 3.5 features so I've created rhbz#1094657 to >>>> track it. >>> >>> Can you elaborate on this? How should vdsm behave better, and why? >> >> Sure. So far I didn't hear any reason why it behaves like this and >> it seems not logical to disable *all* and then enable just *some*. >> >> How: Disabling and enabling operations should be complementary. >> Why: To be less surprising. > > There is an asymmetry between activation and deactivation of an LV. A > mistakenly-active LV can cause data corruption. Making sure that this > does not happen is more important than a new feature. - just out of a curiosity, how can mistakenly-active LV cause data corruption? something like a stalled LV which refers to a volume which doesn't exists anymore? > > We do not want to deactivate and then re-activating the same set of LVs. > That would be illogical. We intentionally deactivate LVs that are no > longer used on the specific host - that's important if a qemu died while > Vdsm was down, leaving a stale LV behind. > > Design-wise, Vdsm would very much like to keep its ownership of > Vdsm-created storage domain. Let us discuss how your feature can be > implemented without this breach of ownership. > Ok, I agree that this should have been discussed with the storage team at the design phase, so let's start from the beginning and try to come up with a better solution. My problem is that I need a storage for the hosted-engine data which is accessible from all hosts. It seems logical to use the same physical storage as we use for "the storage". For NFS it's just a file in /rhev/data-center/mnt/<IP>\mountpoint/<UUID>/ha_agent/. So where/how do you suggest to store such data in case of using lvm (iscsi in this case). Can we use vdsm to set it up or do we have to duplicate the lvm code and handle it our self?

On Wed, May 07, 2014 at 02:16:34PM +0200, Jiri Moskovcak wrote: > On 05/07/2014 11:37 AM, Dan Kenigsberg wrote: >> On Wed, May 07, 2014 at 09:56:03AM +0200, Jiri Moskovcak wrote: >>> On 05/07/2014 09:28 AM, Nir Soffer wrote: >>>> ----- Original Message ----- >>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>> Padgett" <gpadgett(a)redhat.com&gt;, "Doron Fediuck" <dfediuck(a)redhat.com&gt; >>>>> Sent: Wednesday, May 7, 2014 10:21:28 AM >>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>> >>>>> On 05/05/2014 03:19 PM, Nir Soffer wrote: >>>>>> ----- Original Message ----- >>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>>>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>>> Padgett" <gpadgett(a)redhat.com&gt; >>>>>>> Sent: Monday, May 5, 2014 3:44:21 PM >>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>> >>>>>>> On 05/05/2014 02:37 PM, Nir Soffer wrote: >>>>>>>> ----- Original Message ----- >>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>>>>>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>>>>> Padgett" <gpadgett(a)redhat.com&gt; >>>>>>>>> Sent: Monday, May 5, 2014 3:16:37 PM >>>>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>> >>>>>>>>> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>>>>>>>>> ----- Original Message ----- >>>>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>>>>> Cc: devel(a)ovirt.org >>>>>>>>>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>>>>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>>> >>>>>>>>>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>>>>>>>>> ----- Original Message ----- >>>>>>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>>>>> To: devel(a)ovirt.org >>>>>>>>>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>>>>>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>>>>> >>>>>>>>>>>>> Greetings vdsm developers! >>>>>>>>>>>>> >>>>>>>>>>>>> While working on adding ISCSI support to the hosted engine tools, I >>>>>>>>>>>>> ran >>>>>>>>>>>>> into a problem with vdms. It seems that when stopped vdsm >>>>>>>>>>>>> deactivates >>>>>>>>>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>>>>>>>>> reactivates only specific logical volumes. This is a problem for >>>>>>>>>>>>> hosted >>>>>>>>>>>>> engine tools as they create logical volumes in the same volume group >>>>>>>>>>>>> and >>>>>>>>>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a >>>>>>>>>>>>> way >>>>>>>>>>>>> to reactivate it, because the services drop the root permissions and >>>>>>>>>>>>> are >>>>>>>>>>>>> running as vdsm and apparently only root can activate LVs. >>>>>>>>>>>> >>>>>>>>>>>> Can you describe what volumes are you creating, and why? >>>>>>>>>>> >>>>>>>>>>> We create hosted-engine.lockspace (for sanlock) and >>>>>>>>>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>>>>>>>>> >>>>>>>>>> Do you create these lvs in every vdsm vg? >>>>>>>>> >>>>>>>>> - only in the first vg created by vdsm while deploying hosted-engine >>>>>> >>>>>> It seems that the hosted engine has single point of failure - the random >>>>>> vg that contains hosted engine data. >>>>>> >>>>>>>>> >>>>>>>>>> Is this part of the domain structure >>>>>>>>>> used by hosted engine, or it has nothing to do with the storage domain? >>>>>>>>> >>>>>>>>> - sorry, I don't understand this question. How can I tell if it has >>>>>>>>> something to do with the storage domain? It's for storing data about >>>>>>>>> hosts set up to run the hosted-engine and data about state of engine and >>>>>>>>> the state of VM running the engine. >>>>>>>> >>>>>>>> Can you tell us exactly what lvs you are creating, and on which vg? >>>>>>>> >>>>>>>> And how are you creating those lvs - I guess through vdsm? >>>>>>>> >>>>>>> >>>>>>> - no hosted-engine tools do that by calling: >>>>>>> >>>>>>> lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, >>>>>>> stderr=subprocess.PIPE, >>>>>>> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", >>>>>>> "-n", lv_name, vg_uuid]) >>>>>>> .. >>>>>> >>>>>> How do you ensure that another host is not modifying the same vg in the >>>>>> same time? >>>>>> >>>>>> If you are not ensuring this, you will corrupt this vg sooner or later. >>>>>> >>>>>> When a storage domain is detached from a host, for example when the host >>>>>> is in maintenance mode, lvs on the shared storage may be deleted, >>>>>> invalidating >>>>>> the devices mapper maps for these devices. If you write to an lv with wrong >>>>>> maps, you may be writing to an extent belonging to another lv, corrupting >>>>>> that >>>>>> lv data, or even worse corrupting the engine vg data. >>>>>> >>>>>> How do you ensure that the lvs are not deleted while you are using them? >>>>>> >>>>>>> >>>>>>>> The output of lvs command on a host with hosted engine installed will >>>>>>>> help us to understand what you are doing, and then we can think more >>>>>>>> clearly >>>>>>>> what would be the best way to support this in vdsm. >>>>>>> >>>>>>> The output of lvs: http://fpaste.org/99196/93619139/ >>>>>>> >>>>>>> HE created these two LVs: >>>>>>> ha_agent-hosted-engine.lockspace >>>>>>> ha_agent-hosted-engine.metadata >>>>>> >>>>>> Why do you create these lvs on a vg owned by vdsm? >>>>>> >>>>>> If you want total control of these lvs, I suggest that you create your own >>>>>> vg and put what ever lvs you like there. >>>>>> >>>>> >>>>> I would rather not go this way (at least not for 3.5) as it's too much >>>>> code changes in hosted-engine. On the other hand the logic in vdsm seems >>>>> wrong because it's not complementary (disabling all LVs and then >>>>> enabling just some of them) and should be fixed anyway. This problem is >>>>> blocking one of our 3.5 features so I've created rhbz#1094657 to track it. >>>> >>>> Can you elaborate on this? How should vdsm behave better, and why? >>> >>> Sure. So far I didn't hear any reason why it behaves like this and >>> it seems not logical to disable *all* and then enable just *some*. >>> >>> How: Disabling and enabling operations should be complementary. >>> Why: To be less surprising. >> >> There is an asymmetry between activation and deactivation of an LV. A >> mistakenly-active LV can cause data corruption. Making sure that this >> does not happen is more important than a new feature. > > - just out of a curiosity, how can mistakenly-active LV cause data > corruption? something like a stalled LV which refers to a volume > which doesn't exists anymore? > >> >> We do not want to deactivate and then re-activating the same set of LVs. >> That would be illogical. We intentionally deactivate LVs that are no >> longer used on the specific host - that's important if a qemu died while >> Vdsm was down, leaving a stale LV behind. >> >> Design-wise, Vdsm would very much like to keep its ownership of >> Vdsm-created storage domain. Let us discuss how your feature can be >> implemented without this breach of ownership. >> > > Ok, I agree that this should have been discussed with the storage > team at the design phase, so let's start from the beginning and try > to come up with a better solution. > My problem is that I need a storage for the hosted-engine data > which is accessible from all hosts. It seems logical to use the same > physical storage as we use for "the storage". For NFS it's just a > file in > /rhev/data-center/mnt/<IP>\mountpoint/<UUID>/ha_agent/. So where/how > do you suggest to store such data in case of using lvm (iscsi in > this case). Can we use vdsm to set it up or do we have to duplicate > the lvm code and handle it our self? I think that for this to happen, we need to define a Vdsm verb that creates a volume on a storage domain that is unrelated to any pool. Such a verb is in planning; Federico, can its implementation be hasten in favor of hosted engine? On its own, this would not solve the problem of Vdsm deactivating all unused LVs. Jiri, could you describe why you keep your LV active, but not open?

Dan.

On Mon, May 12, 2014 at 01:26:33PM +0200, Jiri Moskovcak wrote: > On 05/12/2014 11:22 AM, Dan Kenigsberg wrote: >> On Wed, May 07, 2014 at 02:16:34PM +0200, Jiri Moskovcak wrote: >>> On 05/07/2014 11:37 AM, Dan Kenigsberg wrote: >>>> On Wed, May 07, 2014 at 09:56:03AM +0200, Jiri Moskovcak wrote: >>>>> On 05/07/2014 09:28 AM, Nir Soffer wrote: >>>>>> ----- Original Message ----- >>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>>> Padgett" <gpadgett(a)redhat.com&gt;, "Doron Fediuck" <dfediuck(a)redhat.com&gt; >>>>>>> Sent: Wednesday, May 7, 2014 10:21:28 AM >>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>> >>>>>>> On 05/05/2014 03:19 PM, Nir Soffer wrote: >>>>>>>> ----- Original Message ----- >>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>>>>>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>>>>> Padgett" <gpadgett(a)redhat.com&gt; >>>>>>>>> Sent: Monday, May 5, 2014 3:44:21 PM >>>>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>> >>>>>>>>> On 05/05/2014 02:37 PM, Nir Soffer wrote: >>>>>>>>>> ----- Original Message ----- >>>>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>>>>> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >>>>>>>>>>> Mureinik" <amureini(a)redhat.com&gt;, "Greg >>>>>>>>>>> Padgett" <gpadgett(a)redhat.com&gt; >>>>>>>>>>> Sent: Monday, May 5, 2014 3:16:37 PM >>>>>>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>>> >>>>>>>>>>> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>>>>>>>>>>> ----- Original Message ----- >>>>>>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>>>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>>>>>>>>>>> Cc: devel(a)ovirt.org >>>>>>>>>>>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>>>>>>>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>>>>> >>>>>>>>>>>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>>>>>>>>>>> ----- Original Message ----- >>>>>>>>>>>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>>>>>>>>>>> To: devel(a)ovirt.org >>>>>>>>>>>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>>>>>>>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Greetings vdsm developers! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> While working on adding ISCSI support to the hosted engine tools, I >>>>>>>>>>>>>>> ran >>>>>>>>>>>>>>> into a problem with vdms. It seems that when stopped vdsm >>>>>>>>>>>>>>> deactivates >>>>>>>>>>>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>>>>>>>>>>> reactivates only specific logical volumes. This is a problem for >>>>>>>>>>>>>>> hosted >>>>>>>>>>>>>>> engine tools as they create logical volumes in the same volume group >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a >>>>>>>>>>>>>>> way >>>>>>>>>>>>>>> to reactivate it, because the services drop the root permissions and >>>>>>>>>>>>>>> are >>>>>>>>>>>>>>> running as vdsm and apparently only root can activate LVs. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you describe what volumes are you creating, and why? >>>>>>>>>>>>> >>>>>>>>>>>>> We create hosted-engine.lockspace (for sanlock) and >>>>>>>>>>>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>>>>>>>>>>> >>>>>>>>>>>> Do you create these lvs in every vdsm vg? >>>>>>>>>>> >>>>>>>>>>> - only in the first vg created by vdsm while deploying hosted-engine >>>>>>>> >>>>>>>> It seems that the hosted engine has single point of failure - the random >>>>>>>> vg that contains hosted engine data. >>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Is this part of the domain structure >>>>>>>>>>>> used by hosted engine, or it has nothing to do with the storage domain? >>>>>>>>>>> >>>>>>>>>>> - sorry, I don't understand this question. How can I tell if it has >>>>>>>>>>> something to do with the storage domain? It's for storing data about >>>>>>>>>>> hosts set up to run the hosted-engine and data about state of engine and >>>>>>>>>>> the state of VM running the engine. >>>>>>>>>> >>>>>>>>>> Can you tell us exactly what lvs you are creating, and on which vg? >>>>>>>>>> >>>>>>>>>> And how are you creating those lvs - I guess through vdsm? >>>>>>>>>> >>>>>>>>> >>>>>>>>> - no hosted-engine tools do that by calling: >>>>>>>>> >>>>>>>>> lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, >>>>>>>>> stderr=subprocess.PIPE, >>>>>>>>> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", >>>>>>>>> "-n", lv_name, vg_uuid]) >>>>>>>>> .. >>>>>>>> >>>>>>>> How do you ensure that another host is not modifying the same vg in the >>>>>>>> same time? >>>>>>>> >>>>>>>> If you are not ensuring this, you will corrupt this vg sooner or later. >>>>>>>> >>>>>>>> When a storage domain is detached from a host, for example when the host >>>>>>>> is in maintenance mode, lvs on the shared storage may be deleted, >>>>>>>> invalidating >>>>>>>> the devices mapper maps for these devices. If you write to an lv with wrong >>>>>>>> maps, you may be writing to an extent belonging to another lv, corrupting >>>>>>>> that >>>>>>>> lv data, or even worse corrupting the engine vg data. >>>>>>>> >>>>>>>> How do you ensure that the lvs are not deleted while you are using them? >>>>>>>> >>>>>>>>> >>>>>>>>>> The output of lvs command on a host with hosted engine installed will >>>>>>>>>> help us to understand what you are doing, and then we can think more >>>>>>>>>> clearly >>>>>>>>>> what would be the best way to support this in vdsm. >>>>>>>>> >>>>>>>>> The output of lvs: http://fpaste.org/99196/93619139/ >>>>>>>>> >>>>>>>>> HE created these two LVs: >>>>>>>>> ha_agent-hosted-engine.lockspace >>>>>>>>> ha_agent-hosted-engine.metadata >>>>>>>> >>>>>>>> Why do you create these lvs on a vg owned by vdsm? >>>>>>>> >>>>>>>> If you want total control of these lvs, I suggest that you create your own >>>>>>>> vg and put what ever lvs you like there. >>>>>>>> >>>>>>> >>>>>>> I would rather not go this way (at least not for 3.5) as it's too much >>>>>>> code changes in hosted-engine. On the other hand the logic in vdsm seems >>>>>>> wrong because it's not complementary (disabling all LVs and then >>>>>>> enabling just some of them) and should be fixed anyway. This problem is >>>>>>> blocking one of our 3.5 features so I've created rhbz#1094657 to track it. >>>>>> >>>>>> Can you elaborate on this? How should vdsm behave better, and why? >>>>> >>>>> Sure. So far I didn't hear any reason why it behaves like this and >>>>> it seems not logical to disable *all* and then enable just *some*. >>>>> >>>>> How: Disabling and enabling operations should be complementary. >>>>> Why: To be less surprising. >>>> >>>> There is an asymmetry between activation and deactivation of an LV. A >>>> mistakenly-active LV can cause data corruption. Making sure that this >>>> does not happen is more important than a new feature. >>> >>> - just out of a curiosity, how can mistakenly-active LV cause data >>> corruption? something like a stalled LV which refers to a volume >>> which doesn't exists anymore? >>> >>>> >>>> We do not want to deactivate and then re-activating the same set of LVs. >>>> That would be illogical. We intentionally deactivate LVs that are no >>>> longer used on the specific host - that's important if a qemu died while >>>> Vdsm was down, leaving a stale LV behind. >>>> >>>> Design-wise, Vdsm would very much like to keep its ownership of >>>> Vdsm-created storage domain. Let us discuss how your feature can be >>>> implemented without this breach of ownership. >>>> >>> >>> Ok, I agree that this should have been discussed with the storage >>> team at the design phase, so let's start from the beginning and try >>> to come up with a better solution. >>> My problem is that I need a storage for the hosted-engine data >>> which is accessible from all hosts. It seems logical to use the same >>> physical storage as we use for "the storage". For NFS it's just a >>> file in >>> /rhev/data-center/mnt/<IP>\mountpoint/<UUID>/ha_agent/. So where/how >>> do you suggest to store such data in case of using lvm (iscsi in >>> this case). Can we use vdsm to set it up or do we have to duplicate >>> the lvm code and handle it our self? >> >> I think that for this to happen, we need to define a Vdsm verb that >> creates a volume on a storage domain that is unrelated to any pool. Such >> a verb is in planning; Federico, can its implementation be hasten in >> favor of hosted engine? >> >> On its own, this would not solve the problem of Vdsm deactivating all >> unused LVs. >> >> Jiri, could you describe why you keep your LV active, but not open? >> > > - the setup flow goes approximately like this: > > 1. create the LVs for the hosted-engine > 2. install the engine into the VM > 3. add the host to the engine > - this causes re-deploy of vdsm and deactivating the LVs > 4. start the ha-agent and ha-broker services which uses the LVs > > - I guess we could move the creation of the LVs after the vdsm is > re-deployed, just before the HE services are started, but it won't > fix the problem if the vdsm is restarted It's not going to solve our design problem, but can the users of that LV activate it just before they open it? This way there's only a small window where vdsm can crash, restart, and deactivate the LV.

Dan.

On Mon, May 12, 2014 at 02:04:05PM +0200, Jiri Moskovcak wrote: >>>> Jiri, could you describe why you keep your LV active, but not open? >>>> >>> >>> - the setup flow goes approximately like this: >>> >>> 1. create the LVs for the hosted-engine >>> 2. install the engine into the VM >>> 3. add the host to the engine >>> - this causes re-deploy of vdsm and deactivating the LVs >>> 4. start the ha-agent and ha-broker services which uses the LVs >>> >>> - I guess we could move the creation of the LVs after the vdsm is >>> re-deployed, just before the HE services are started, but it won't >>> fix the problem if the vdsm is restarted >> >> It's not going to solve our design problem, but can the users of that LV >> activate it just before they open it? This way there's only a small >> window where vdsm can crash, restart, and deactivate the LV. >> > > - no, it runs as vdsm user and as it seems only root can activate LVs and > during the startup when it still has the root privs, it doesn't know > anything about the storage, the storage is connected on a client request > when it already doesn't have root privs Vdsm uses "sudo" for these purposes.

> - we could use vdsm to activate the LV if it provides API which allows that There's the undocumented prepareImage(), but please do not use it before an ack by the storage team.

Dan.

On 05/13/2014 03:16 AM, Jiri Moskovcak wrote: > On 05/12/2014 05:27 PM, Dan Kenigsberg wrote: >> On Mon, May 12, 2014 at 02:04:05PM +0200, Jiri Moskovcak wrote: >> >>>>>> Jiri, could you describe why you keep your LV active, but not open? >>>>>> >>>>> >>>>> - the setup flow goes approximately like this: >>>>> >>>>> 1. create the LVs for the hosted-engine >>>>> 2. install the engine into the VM >>>>> 3. add the host to the engine >>>>> - this causes re-deploy of vdsm and deactivating the LVs >>>>> 4. start the ha-agent and ha-broker services which uses the LVs >>>>> >>>>> - I guess we could move the creation of the LVs after the vdsm is >>>>> re-deployed, just before the HE services are started, but it won't >>>>> fix the problem if the vdsm is restarted >>>> >>>> It's not going to solve our design problem, but can the users of >>>> that LV >>>> activate it just before they open it? This way there's only a small >>>> window where vdsm can crash, restart, and deactivate the LV. >>>> >>> >>> - no, it runs as vdsm user and as it seems only root can activate LVs >>> and >>> during the startup when it still has the root privs, it doesn't know >>> anything about the storage, the storage is connected on a client >>> request >>> when it already doesn't have root privs >> >> Vdsm uses "sudo" for these purposes. > > - that would solve the problem with privs, but I would really like to > avoid the situation where the vdsm disables the LVs as there is no > reason for disabling it as there can be situation where vdsm is down and > the VM with the engine is still running ok > >> >>> - we could use vdsm to activate the LV if it provides API which >>> allows that >> >> There's the undocumented prepareImage(), but please do not use it before >> an ack by the storage team. >> > > - sure, I'm working on it with Nir now, I won't do anything without the > storage team approval ;) > >> Dan. >> > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel I'm missing something here - why doesn't VDSM knows about this LV? its a VM started by VDSM?

On 05/14/2014 03:02 AM, Jiri Moskovcak wrote: > On 05/14/2014 02:57 AM, Itamar Heim wrote: >> On 05/13/2014 03:16 AM, Jiri Moskovcak wrote: >>> On 05/12/2014 05:27 PM, Dan Kenigsberg wrote: >>>> On Mon, May 12, 2014 at 02:04:05PM +0200, Jiri Moskovcak wrote: >>>> >>>>>>>> Jiri, could you describe why you keep your LV active, but not >>>>>>>> open? >>>>>>>> >>>>>>> >>>>>>> - the setup flow goes approximately like this: >>>>>>> >>>>>>> 1. create the LVs for the hosted-engine >>>>>>> 2. install the engine into the VM >>>>>>> 3. add the host to the engine >>>>>>> - this causes re-deploy of vdsm and deactivating the LVs >>>>>>> 4. start the ha-agent and ha-broker services which uses the LVs >>>>>>> >>>>>>> - I guess we could move the creation of the LVs after the vdsm is >>>>>>> re-deployed, just before the HE services are started, but it won't >>>>>>> fix the problem if the vdsm is restarted >>>>>> >>>>>> It's not going to solve our design problem, but can the users of >>>>>> that LV >>>>>> activate it just before they open it? This way there's only a small >>>>>> window where vdsm can crash, restart, and deactivate the LV. >>>>>> >>>>> >>>>> - no, it runs as vdsm user and as it seems only root can activate LVs >>>>> and >>>>> during the startup when it still has the root privs, it doesn't know >>>>> anything about the storage, the storage is connected on a client >>>>> request >>>>> when it already doesn't have root privs >>>> >>>> Vdsm uses "sudo" for these purposes. >>> >>> - that would solve the problem with privs, but I would really like to >>> avoid the situation where the vdsm disables the LVs as there is no >>> reason for disabling it as there can be situation where vdsm is down >>> and >>> the VM with the engine is still running ok >>> >>>> >>>>> - we could use vdsm to activate the LV if it provides API which >>>>> allows that >>>> >>>> There's the undocumented prepareImage(), but please do not use it >>>> before >>>> an ack by the storage team. >>>> >>> >>> - sure, I'm working on it with Nir now, I won't do anything without the >>> storage team approval ;) >>> >>>> Dan. >>>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >> >> I'm missing something here - why doesn't VDSM knows about this LV? its a >> VM started by VDSM? > > - these LVs are created by hosted-engine-setup and are not connected > with the VM for the metadata/config/etc.?