[ovirt-devel] ovirt nfs mount caused sanlock failed to access data storage

Tue Sep 12 11:57:44 UTC 2017

I know that,

after nfs mounted in client, it used nfs server's permission, although i don't know why it's this, but i execute as follows in nfs server:
# useradd sanlock -u 114
# usermod -a -G kvm sanlock

then it's works well!

thanks anyhow!

At 2017-09-12 18:47:55, "pengyixiang" <yxpengi386 at 163.com> wrote:

hello Nir
    I just do a test, as follows, i add linx to group kvm

# cat /etc/group | grep "kvm"
kvm:x:112:qemu,vdsm,linx,sanlock

and i see the permission of leases of nfs storage:

$ ls -l /rhev/data-center/mnt/192.168.11.55\:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/dom_md/leases
-rw-rw---- 1 vdsm kvm 2097152 9月  11 19:21 /rhev/data-center/mnt/192.168.11.55:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/dom_md/leases

sanlock cannot read it
425138     resource = sanlock.read_resource(lease.path, lease.offset)
425139 SanlockException: (13, 'Sanlock resource read failure', 'Permission denied')

and i cannot change to sanlock like that:
# su sanlock
root at Node:~#

so i add a user linx to group kvm, and access to a o660 file with vdsm:kvm as follows:

$ ls -l
总用量 16
-rw-rw---- 1 vdsm kvm     6 9月  11 20:06 1.txt
drwxr-xr-x 9 linx linx 4096 9月   1 15:58 linx-virtualization
drw-rw---- 3 linx linx 4096 9月  11 20:13 test2
drw-rw---- 2 linx linx 4096 9月  11 20:19 test3

then we can view the file in user "linx":
$ cat ~/1.txt
pencc

then i create 1.txt as same as ~/1.txt in nfs mount point like that with permission o666 vdsm:kvm :

/rhev/data-center/mnt/192.168.11.55\:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/1.txt

then i read it, but failed too

Permission denied

if i change 1.txt's permission in mount point to o666

then i can read it succeed.

i think, maybe nfs's mount caused group permission failed, group permission as the second '6' of o666

i cannot understand it, this my nfs-server's exports configurations:
/home/dataStorage *(rw,sync,no_subtree_check)

have any  ideas? QAQ

在 2017-09-12 18:06:08，"Nir Soffer" <nsoffer at redhat.com> 写道：

On Tue, 12 Sep 2017, 4:52 pengyixiang <yxpengi386 at 163.com> wrote:

hello,everyone
    sanlock failed due to cannot read nfs storage's data , i tried to chmod 777 /rhev/data-center/mnt/192.168.11.55\:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/dom_md/*(add others permission), then it's ok

    sanlock's log:
425120 Traceback (most recent call last):
425121   File "/usr/lib/python2.7/dist-packages/vdsm/storage/task.py", line 878, in _run
425122     return fn(*args, **kargs)
425123   File "/usr/lib/python2.7/dist-packages/vdsm/logUtils.py", line 52, in wrapper
425124     res = f(*args, **kwargs)
425125   File "/usr/share/vdsm/storage/hsm.py", line 619, in getSpmStatus
425126     status = self._getSpmStatusInfo(pool)
425127   File "/usr/share/vdsm/storage/hsm.py", line 613, in _getSpmStatusInfo
425128     (pool.spmRole,) + pool.getSpmStatus()))
425129   File "/usr/share/vdsm/storage/sp.py", line 141, in getSpmStatus
425130     return self._backend.getSpmStatus()
425131   File "/usr/share/vdsm/storage/spbackends.py", line 433, in getSpmStatus
425132     lVer, spmId = self.masterDomain.inquireClusterLock()
425133   File "/usr/share/vdsm/storage/sd.py", line 817, in inquireClusterLock
425134     return self._manifest.inquireDomainLock()
425135   File "/usr/share/vdsm/storage/sd.py", line 522, in inquireDomainLock
425136     return self._domainLock.inquire(self.getDomainLease())
425137   File "/usr/lib/python2.7/dist-packages/vdsm/storage/clusterlock.py", line 372, in i       nquire
425138     resource = sanlock.read_resource(lease.path, lease.offset)
425139 SanlockException: (13, 'Sanlock resource read failure', 'Permission denied')

i test it, and in node,I add user "linx" to group "kvm"

$ cat /etc/group | grep "kvm"
kvm:x:112:qemu,vdsm,linx,sanlock

then i create a file in $HOME:
$ ls -l
总用量 16
-rw-rw---- 1 vdsm kvm     6 9月  11 20:06 1.txt
drwxr-xr-x 9 linx linx 4096 9月   1 15:58 linx-virtualization
drw-rw---- 3 linx linx 4096 9月  11 20:13 test2
drw-rw---- 2 linx linx 4096 9月  11 20:19 test3

then we can view the file in user "linx":
$ cat 1.txt
pencc

leases if vdsm:kvm too:
$ ls -l /rhev/data-center/mnt/192.168.11.55\:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/dom_md/leases
-rw-rw---- 1 vdsm kvm 2097152 9月  11 19:21 /rhev/data-center/mnt/192.168.11.55:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/dom_md/leases

but we cannot read the file in user "linx":
$ cat /rhev/data-center/mnt/192.168.11.55\:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/dom_md/leases
cat: '/rhev/data-center/mnt/192.168.11.55:_home_dataStorage/1845be22-1ac4-4e42-bbcb-7ba9ccd6e569/dom_md/leases': 权限不够

why is this? follows the nfs server configure
# cat /etc/exports

/home/dataStorage 192.168.11.*(rw,sync)
/home/dataStorage2 192.168.11.*(rw,sync,no_root_squash,no_subtree_check)
/home/isoStorage 192.168.11.*(rw,sync,no_root_squash,no_subtree_check)

Is my nfs-server configurations miss some arguments? have any idea?

Try to setup a supported system (e.g centos, fedora) and compare the setup.

Can explain why user linx need access to vdsm/sanlock/qemu data?

Nir

_______________________________________________
Devel mailing list
Devel at ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170912/0c270062/attachment.html>