Re: [ovirt-devel] vdsm disabling logical volumes

From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; To: "Nir Soffer" <nsoffer(a)redhat.com&gt; Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon Mureinik" <amureini(a)redhat.com&gt;, "Greg Padgett" <gpadgett(a)redhat.com&gt; Sent: Monday, May 5, 2014 3:44:21 PM Subject: Re: [ovirt-devel] vdsm disabling logical volumes On 05/05/2014 02:37 PM, Nir Soffer wrote: > ----- Original Message ----- >> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >> Cc: devel(a)ovirt.org, "Federico Simoncelli" <fsimonce(a)redhat.com&gt;, "Allon >> Mureinik" <amureini(a)redhat.com&gt;, "Greg >> Padgett" <gpadgett(a)redhat.com&gt; >> Sent: Monday, May 5, 2014 3:16:37 PM >> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >> >> On 05/05/2014 12:01 AM, Nir Soffer wrote: >>> ----- Original Message ----- >>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>> To: "Nir Soffer" <nsoffer(a)redhat.com&gt; >>>> Cc: devel(a)ovirt.org >>>> Sent: Sunday, May 4, 2014 9:23:49 PM >>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes >>>> >>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: >>>>> ----- Original Message ----- >>>>>> From: "Jiri Moskovcak" <jmoskovc(a)redhat.com&gt; >>>>>> To: devel(a)ovirt.org >>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM >>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes >>>>>> >>>>>> Greetings vdsm developers! >>>>>> >>>>>> While working on adding ISCSI support to the hosted engine tools, I >>>>>> ran >>>>>> into a problem with vdms. It seems that when stopped vdsm deactivates >>>>>> ALL logical volumes in it's volume group and when it starts it >>>>>> reactivates only specific logical volumes. This is a problem for >>>>>> hosted >>>>>> engine tools as they create logical volumes in the same volume group >>>>>> and >>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a way >>>>>> to reactivate it, because the services drop the root permissions and >>>>>> are >>>>>> running as vdsm and apparently only root can activate LVs. >>>>> >>>>> Can you describe what volumes are you creating, and why? >>>> >>>> We create hosted-engine.lockspace (for sanlock) and >>>> hosted-engine.metadata (keeps data about hosted engine hosts) >>> >>> Do you create these lvs in every vdsm vg? >> >> - only in the first vg created by vdsm while deploying hosted-engine

>> >>> Is this part of the domain structure >>> used by hosted engine, or it has nothing to do with the storage domain? >> >> - sorry, I don't understand this question. How can I tell if it has >> something to do with the storage domain? It's for storing data about >> hosts set up to run the hosted-engine and data about state of engine and >> the state of VM running the engine. > > Can you tell us exactly what lvs you are creating, and on which vg? > > And how are you creating those lvs - I guess through vdsm? > - no hosted-engine tools do that by calling: lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", "-n", lv_name, vg_uuid]) ..

> The output of lvs command on a host with hosted engine installed will > help us to understand what you are doing, and then we can think more > clearly > what would be the best way to support this in vdsm. The output of lvs: http://fpaste.org/99196/93619139/ HE created these two LVs: ha_agent-hosted-engine.lockspace ha_agent-hosted-engine.metadata

--Jirka > >> >>> >>>> >>>>> >>>>>> So far the >>>>>> only suitable solution seems to be to change vdsm to only >>>>>> deactivate/activate it's own LVs. >>>>> >>>>> This sounds reasonable. You can add a list of hosted engine lv names >>>>> and skip these volumes when deactivating vdsm volumes. >>>> >>>> - this sounds a bit suboptimal, vdsm already has list of it's LVs, so it >>>> can just disable only LVs known to it, otherwise we would have to change >>>> the list everytime we add some LV to the group >>> >>> vdsm has a list of special lvs, that needs special treatment. Otherwise, >>> it >>> consider any other lv as owned by vdsm, and will deactivate them when >>> they >>> are >>> not used. >>> >>> I agree that this will create a dependency, but this can also be solved. >>> For example, vdsm can load the list from a file installed by hosted >>> engine, >>> like the typical conf.d directories. >>> >> >> - ok, this is something I actually don't have strong opinion about, for >> me adding a file with a list of LVs or tagging the logical volumes is >> almost the same, I just need a way to tell vdsm which LVs to ignore.. >> >>>> >>>>> >>>>> Another solution is to tag hosted engine lvs, and have vdsm ignore >>>>> lvs that contains this tag. >>>> >>>> - this sounds good, because if we teach vdsm to ignore LVs with some tag >>>> we can add new LVs in future without changing vdsm. This however applies >>>> also to the solution where vdsm only disables it's own LVs, >>> >>> vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like >>> this >>> using some historic tags we keep (e.g. RHAT_*), but I'd rather add new >>> tag >>> with >>> clear semantic than use some random historic value we have. >>> >>>> so it >>>> depends on vdsm devels which solution they find better. I think the >>>> solution without tags is better, because is simpler and others (like >>>> hosted-engine) can just createlv and don't bother with tags.. >>> >>> I think that a generic tag like OVIRT_IGNORE is an easy solution for >>> everyone. >>> >>> Federico, what do you think? >>> >>> Nir >>> >> >>