[ovirt-users] thin_check run on VM disk by host on startup ?!

[Nir Soffer]

On Wed, Aug 31, 2016 at 2:30 PM, Rik Theys <Rik.Theys at esat.kuleuven.be> wrote:
> Hi,
>
> On 08/31/2016 11:51 AM, Nir Soffer wrote:
>> On Wed, Aug 31, 2016 at 11:07 AM, Rik Theys <Rik.Theys at esat.kuleuven.be> wrote:
>>> On 08/31/2016 09:43 AM, Rik Theys wrote:
>>>> On 08/30/2016 04:47 PM, Nir Soffer wrote:
>>>>> On Tue, Aug 30, 2016 at 3:51 PM, Rik Theys <Rik.Theys at esat.kuleuven.be> wrote:
>>>>>> While rebooting one of the hosts in an oVirt cluster, I noticed that
>>>>>> thin_check is run on the thin pool devices of one of the VM's on which
>>>>>> the disk is assigned to.
>>>>>>
>>>>>> That seems strange to me. I would expect the host to stay clear of any
>>>>>> VM disks.
>>>>>
>>>>> We expect the same thing, but unfortunately systemd and lvm try to
>>>>> auto activate stuff. This may be good idea for desktop system, but
>>>>> probably bad idea for a server and in particular a hypervisor.
>>>>>
>>>>> We don't have a solution yet, but you can try these:
>>>>>
>>>>> 1. disable lvmetad service
>>>>>
>>>>>     systemctl stop lvm2-lvmetad.service lvm2-lvmetad.socket
>>>>>     systemctl mask lvm2-lvmetad.service lvm2-lvmetad.socket
>>>>>
>>>>> Edit /etc/lvm/lvm.conf:
>>>>>
>>>>>     use_lvmetad = 0
>>>>>
>>>>> 2. disable lvm auto activation
>>>>>
>>>>> Edit /etc/lvm/lvm.conf:
>>>>>
>>>>>     auto_activation_volume_list = []
>>>>>
>>>>> 3. both 1 and 2
>>>>>
>>>>
>>>> I've now applied both of the above and regenerated the initramfs and
>>>> rebooted and the host no longer lists the LV's of the VM. Since I
>>>> rebooted the host before without this issue, I'm not sure a single
>>>> reboot is enough to conclude it has fully fixed the issue.
>>>>
>>>> You mention that there's no solution yet. Does that mean the above
>>>> settings are not 100% certain to avoid this behaviour?
>>>>
>>>> I was thinking of setting a global_filter in /etc/lvm/lvm.conf to only
>>>> include the PV's for the hypervisor disks (on which the OS is installed)
>>>> so the system lvm commands only touches those. Since vdsm is using its
>>>> own lvm.conf this should be OK for vdsm?
>>>
>>> This does not seem to work. The host can not be activated as it can't
>>> find his volume group(s). To be able to use the global_filter in
>>> /etc/lvm/lvm.conf, I believe it should be overridden in vdsm's lvm.conf
>>> to revert back to the default.
>>>
>>> I've moved my filter from global_filter to filter and that seems to
>>> work. When lvmetad is disabled I believe this should have the same
>>> effect as global_filter? The comments in /etc/lvm/lvm.conf indicate also
>>> udev might ignore the filter setting?
>>
>> Right, global_filter exist so you can override filter used from the command
>> line.
>>
>> For example, hiding certain devices from vdsm. This is why we are using
>> filter in vdsm, leaving global_filter for the administrator.
>>
>> Can you explain why do you need global_filter or filter for the
>> hypervisor disks?
>
> Based on the comment in /etc/lvm/lvm.conf regarding global_filter I
> concluded that not only lvmetad but also udev might perform action on
> the devices and I wanted to prevent that.
>
> I've now set the following settings in /etc/lvm/lvm.conf:
>
> use_lvmetad = 0
> auto_activation_volume_list = []
> filter = ["a|/dev/sda5|", "r|.*|" ]

Better use /dev/disk/by-uuid/ to select the specific device, without
depending on device order.

>
> On other systems I have kept the default filter.
>
>> Do you have any issue with the current settings, disabling auto activation and
>> lvmetad?
>
> Keeping those two disabled also seems to work. The ovirt LV's do show up
> in 'lvs' output but are not activated.

Good

> I wanted to be absolutely sure the VM LV's were not touched, I added the
> filter on some of our hosts.

The only problem with this filter is it may break if you change the host in some
way, like boot from another disk.

It would be nice if you file a bug for this, and mention the configuration that
fixes this issue, we certainly need to improve the way we configure lvm.

Nir