Still having NFS issues. (Permissions)
So I have doubled checked everything I can think of and am still having issues. I have an NFS Storage Domain attached and I can now create, modify, and remove VM's with no issues. The issue I am having is when it is time to power on the VM. The issue is that when oVirt tries to access the image, access is being denied. I have double checked the documentation against my setup and it all looks right. I have the VDSM user and KVM group created with id's of 36. New files are being created with permissions of 755, just as the documentation describes. What I have found out though, is that in order for oVirt to have access to the files, the "other" permission has to be a 7. So effectiviely I need to have 757, rwx, on the files. Is there a user that I am missing from the documentation, or a user on oVirt that should be in a group, that isn't? Someone previously mentioned the sanlock user, but I do not see anything special that is required for it. This is my last hurdle before really being able to dig into oVirt.
To add, the 757 permission does not need to be on the .lease or the .meta files.
On Mon, 2019-12-09 at 18:43 +0000, Robert Webb wrote:
To add, the 757 permission does not need to be on the .lease or the .meta files.
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KZF6RCSRW2QV3P...
Good morning, Check SELinux just in case. Here's my config: NFS server: /etc/exports: /data/ovirt *(rw,sync,no_subtree_check,all_squash,anonuid=36,anongid=36) Folder: [root@kst001 ~]# ls -ld /data/ovirt drwxr-xr-x 3 vdsm kvm 76 Jun 1 2017 /data/ovirt Subfolders: [root@kst001 ~]# ls -l /data/ovirt/* -rwxr-xr-x 1 vdsm kvm 0 Dec 10 06:38 /data/ovirt/__DIRECT_IO_TEST__ /data/ovirt/a597d0aa-bf22-47a3-a8a3-e5cecf3e20e0: total 4 drwxr-xr-x 2 vdsm kvm 117 Jun 1 2017 dom_md drwxr-xr-x 56 vdsm kvm 4096 Dec 2 14:51 images drwxr-xr-x 4 vdsm kvm 42 Jun 1 2017 master [root@kst001 ~]# The user: [root@kst001 ~]# id vdsm uid=36(vdsm) gid=36(kvm) groups=36(kvm) [root@kst001 ~]# And output from 'mount' on a host: kst001:/data/ovirt on /rhev/data-center/mnt/kst001:_data_ovirt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,nolock, nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,mountaddr=<nfs- server- ip>,mountvers=3,mountport=20048,mountproto=udp,local_lock=all,addr=<nfs -server-ip>) HTH /tony
On Tue, Dec 10, 2019 at 7:52 AM Tony Brian Albers <tba@kb.dk> wrote:
On Mon, 2019-12-09 at 18:43 +0000, Robert Webb wrote:
To add, the 757 permission does not need to be on the .lease or the .meta files.
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KZF6RCSRW2QV3P...
Good morning,
Check SELinux just in case.
Indeed, please do.
Here's my config:
NFS server: /etc/exports: /data/ovirt *(rw,sync,no_subtree_check,all_squash,anonuid=36,anongid=36)
Folder: [root@kst001 ~]# ls -ld /data/ovirt drwxr-xr-x 3 vdsm kvm 76 Jun 1 2017 /data/ovirt
You should not need the '7' for 'other'. Does it work for you with 750? See also: https://ovirt.org/develop/troubleshooting-nfs-storage-issues.html Generally speaking: Files there are created by vdsm (vdsmd), but are used (when running VMs) by qemu. So both of them need access. Good luck,
Subfolders: [root@kst001 ~]# ls -l /data/ovirt/* -rwxr-xr-x 1 vdsm kvm 0 Dec 10 06:38 /data/ovirt/__DIRECT_IO_TEST__
/data/ovirt/a597d0aa-bf22-47a3-a8a3-e5cecf3e20e0: total 4 drwxr-xr-x 2 vdsm kvm 117 Jun 1 2017 dom_md drwxr-xr-x 56 vdsm kvm 4096 Dec 2 14:51 images drwxr-xr-x 4 vdsm kvm 42 Jun 1 2017 master [root@kst001 ~]#
The user: [root@kst001 ~]# id vdsm uid=36(vdsm) gid=36(kvm) groups=36(kvm) [root@kst001 ~]#
And output from 'mount' on a host: kst001:/data/ovirt on /rhev/data-center/mnt/kst001:_data_ovirt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,nolock, nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,mountaddr=<nfs- server- ip>,mountvers=3,mountport=20048,mountproto=udp,local_lock=all,addr=<nfs -server-ip>)
HTH
/tony _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/T6S32XNRB6S67P...
-- Didi
Good morning,
Check SELinux just in case.
Indeed, please do.
For testing, I have set SELinux to permissive on the oVirt host. The NFS server is Debian based and does not use SELinux.
Here's my config:
NFS server: /etc/exports: /data/ovirt *(rw,sync,no_subtree_check,all_squash,anonuid=36,anongid=36)
Folder: [root@kst001 ~]# ls -ld /data/ovirt drwxr-xr-x 3 vdsm kvm 76 Jun 1 2017 /data/ovirt
You should not need the '7' for 'other'. Does it work for you with 750?
See also:
https://ovirt.org/develop/troubleshooting-nfs-storage-issues.html
Generally speaking:
Files there are created by vdsm (vdsmd), but are used (when running VMs) by qemu. So both of them need access.
So the link to the NFS storage troubleshooting page is where I found that the perms needed to be 755. So in my OpenMediaVault setup under shared folders, I have set the owner as vdsm:36 and group as kvm:36. I have set owner as rwx, group as rx, and other as rx. However, when oVirt writes new files, the image file and the .lease file gets perm of 660, and the .meta file gets perms of 644. Like this: drwxr-xr-x+ 2 vdsm kvm 4096 Dec 10 09:03 . drwxr-xr-x+ 3 vdsm kvm 4096 Dec 10 09:02 .. -rw-rw---- 1 vdsm kvm 53687091200 Dec 10 09:02 5a514067-82fb-42f9-b436-f8f93883fe27 -rw-rw---- 1 vdsm kvm 1048576 Dec 10 09:03 5a514067-82fb-42f9-b436-f8f93883fe27.lease -rw-r--r-- 1 vdsm kvm 298 Dec 10 09:03 5a514067-82fb-42f9-b436-f8f93883fe27.meta So, with all that said, I cleaned everything up and my directory permissions look like what Tony posted for his. I have added in his export options to my setup and rebooted my host. I created a new VM from scratch and the files under images now look like this: drwxr-xr-x+ 2 vdsm kvm 4096 Dec 10 09:03 . drwxr-xr-x+ 3 vdsm kvm 4096 Dec 10 09:02 .. -rw-rw---- 1 vdsm kvm 53687091200 Dec 10 09:02 5a514067-82fb-42f9-b436-f8f93883fe27 -rw-rw---- 1 vdsm kvm 1048576 Dec 10 09:03 5a514067-82fb-42f9-b436-f8f93883fe27.lease -rw-r--r-- 1 vdsm kvm 298 Dec 10 09:03 5a514067-82fb-42f9-b436-f8f93883fe27.meta Still not the 755 as expected, but I am guessing with the addition of the "anonuid=36,anongid=36" to the exports, everything is now working as expected. The VM will boot and run as expected. There was nothing in the any of the documentation which alluded to possibly needed the additional options in the NFS export options. Since I now know what to add to make it work, whether it is right or just a workaround, I can now move forward with more testing. For documentation purposes, here is what my mount looks like on the oVirt host: nfs_server:/export/Datastore2 on /rhev/data-center/mnt/nfs_server:_export_Datastore2 type nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,nolock,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,mountaddr=nfs_server,mountvers=3,mountport=36103,mountproto=udp,local_lock=all,addr=nfs_server) Here is what my vdsm user looks like: id vdsm uid=36(vdsm) gid=36(kvm) groups=36(kvm),179(sanlock),107(qemu) Thanks for all the help. ________________________________________ From: Yedidyah Bar David <didi@redhat.com> Sent: Tuesday, December 10, 2019 2:36 AM To: Tony Brian Albers Cc: users@ovirt.org; Robert Webb Subject: Re: [ovirt-users] Re: Still having NFS issues. (Permissions) On Tue, Dec 10, 2019 at 7:52 AM Tony Brian Albers <tba@kb.dk> wrote:
On Mon, 2019-12-09 at 18:43 +0000, Robert Webb wrote:
To add, the 757 permission does not need to be on the .lease or the .meta files.
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KZF6RCSRW2QV3P...
Good morning,
Check SELinux just in case.
Indeed, please do.
Here's my config:
NFS server: /etc/exports: /data/ovirt *(rw,sync,no_subtree_check,all_squash,anonuid=36,anongid=36)
Folder: [root@kst001 ~]# ls -ld /data/ovirt drwxr-xr-x 3 vdsm kvm 76 Jun 1 2017 /data/ovirt
You should not need the '7' for 'other'. Does it work for you with 750? See also: https://ovirt.org/develop/troubleshooting-nfs-storage-issues.html Generally speaking: Files there are created by vdsm (vdsmd), but are used (when running VMs) by qemu. So both of them need access. Good luck,
Subfolders: [root@kst001 ~]# ls -l /data/ovirt/* -rwxr-xr-x 1 vdsm kvm 0 Dec 10 06:38 /data/ovirt/__DIRECT_IO_TEST__
/data/ovirt/a597d0aa-bf22-47a3-a8a3-e5cecf3e20e0: total 4 drwxr-xr-x 2 vdsm kvm 117 Jun 1 2017 dom_md drwxr-xr-x 56 vdsm kvm 4096 Dec 2 14:51 images drwxr-xr-x 4 vdsm kvm 42 Jun 1 2017 master [root@kst001 ~]#
The user: [root@kst001 ~]# id vdsm uid=36(vdsm) gid=36(kvm) groups=36(kvm) [root@kst001 ~]#
And output from 'mount' on a host: kst001:/data/ovirt on /rhev/data-center/mnt/kst001:_data_ovirt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,nolock, nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,mountaddr=<nfs- server- ip>,mountvers=3,mountport=20048,mountproto=udp,local_lock=all,addr=<nfs -server-ip>)
HTH
/tony _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/T6S32XNRB6S67P...
-- Didi
participants (3)
-
Robert Webb -
Tony Brian Albers -
Yedidyah Bar David