Hi, When I'm trying to add a storage domain via the webadmin I'm getting the error message that's on the subject. in the vdsm.log I see this error: 2020-07-29 04:53:48,045-0400 ERROR (check/loop) [storage.Monitor] Error checking path /rhev/data-center/mnt/vserver-spider.eng.lab.tlv.redhat.com:_pub_delfassy_nfs__42/17093147-8e4b-4161-9ae3-4d4606b140df/dom_md/metadata (monitor:525) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/monitor.py", line 523, in _pathChecked delay = result.delay() File "/usr/lib/python3.6/site-packages/vdsm/storage/check.py", line 391, in delay raise exception.MiscFileReadException(self.path, self.rc, self.err) vdsm.storage.exception.MiscFileReadException: Internal file read failure: ('/rhev/data-center/mnt/vserver-spider.eng.lab.tlv.redhat.com:_pub_delfassy_nfs__42/17093147-8e4b-4161-9ae3-4d4606b140df/dom_md/metadata', 1, bytearray(b"/usr/bin/dd: failed to open \'/rhev/data-center/mnt/vserver-spider.eng.lab.tlv.redhat.com:_pub_delfassy_nfs__42/17093147-8e4b-4161-9ae3-4d4606b140df/dom_md/metadata\': No such file or directory\n"))

when looking at the directory I do see that metadata was created under this directory, with the following content: [delfassy@delfassy mnt]$ cd new_nfs/cddff7bb-bb0a-4fc8-8ba4-023be91d822d/dom_md/ [delfassy@delfassy dom_md]$ less metadata CLASS=Data DESCRIPTION=spider IOOPTIMEOUTSEC=10 LEASERETRIES=3 LEASETIMESEC=60 LOCKPOLICY= LOCKRENEWALINTERVALSEC=5 POOL_UUID= REMOTE_PATH=vserver-spider.eng.lab.tlv.redhat.com:/pub/delfassy/new_nfs ROLE=Regular SDUUID=cddff7bb-bb0a-4fc8-8ba4-023be91d822d TYPE=NFS VERSION=4 _SHA_CKSUM=f4953016c94f205127b00a5b10f0e7d5d9cf24f7

The permissions on the directory are: drwxrwxrwx. 3 vdsm kvm 4096 Jul 29 11:49 new_mnt

Are there any other configurations I'm missing? Any idea how this issue can be resolved? Thanks, Dana _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/L6MUN6YP74W...

Thanks Nir, I just checked again, it still happens This is the original error: 2020-07-30 03:53:26,071-0400 ERROR (jsonrpc/4) [storage.TaskManager.Task] (Task='2904a936-d6ca-464e-9c86-973c9fbb9b78') Unexpected error (task:880) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/clusterlock.py", line 250, in initSANLock lockspace_name, idsPath, align=alignment, sector=block_size) sanlock.SanlockException: (-202, 'Sanlock lockspace write failure', 'IO timeout')

During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 887, in _run return fn(*args, **kargs) File "<decorator-gen-121>", line 2, in createStorageDomain File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2644, in createStorageDomain max_hosts=max_hosts) File "/usr/lib/python3.6/site-packages/vdsm/storage/nfsSD.py", line 120, in create fsd.initSPMlease() File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 999, in initSPMlease return self._manifest.initDomainLock() File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 627, in initDomainLock self._domainLock.initLock(self.getDomainLease()) File "/usr/lib/python3.6/site-packages/vdsm/storage/clusterlock.py", line 308, in initLock block_size=self._block_size) File "/usr/lib/python3.6/site-packages/vdsm/storage/clusterlock.py", line 260, in initSANLock raise se.ClusterLockInitError() vdsm.storage.exception.ClusterLockInitError: Could not initialize cluster lock: () 2020-07-30 03:53:26,071-0400 INFO (jsonrpc/4) [storage.TaskManager.Task] (Task='2904a936-d6ca-464e-9c86-973c9fbb9b78') aborting: Task is aborted: 'value=Could not initialize cluster lock: () abortedcode=701' (task:1190) 2020-07-30 03:53:26,071-0400 ERROR (jsonrpc/4) [storage.Dispatcher] FINISH createStorageDomain error=Could not initialize cluster lock: () (dispatcher:83) [delfassy@delfassy mnt]$ ls -lhZ drwxrwxrwx. 3 vdsm kvm system_u:object_r:nfs_t:s0 4.0K Jul 30 10:53 new_mnt On Wed, Jul 29, 2020 at 12:30 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > > > On Wed, Jul 29, 2020, 12:09 Dana Elfassy <delfassy(a)redhat.com&gt; wrote: > >> Hi, >> When I'm trying to add a storage domain via the webadmin I'm getting the >> error message that's on the subject. >> in the vdsm.log I see this error: >> 2020-07-29 04:53:48,045-0400 ERROR (check/loop) [storage.Monitor] Error >> checking path /rhev/data-center/mnt/vserver-spider.eng.lab.tlv.redhat.com:_pub_delfassy_nfs__42/17093147-8e4b-4161-9ae3-4d4606b140df/dom_md/metadata >> (monitor:525) >> Traceback (most recent call last): >> File "/usr/lib/python3.6/site-packages/vdsm/storage/monitor.py", line >> 523, in _pathChecked >> delay = result.delay() >> File "/usr/lib/python3.6/site-packages/vdsm/storage/check.py", line >> 391, in delay >> raise exception.MiscFileReadException(self.path, self.rc, self.err) >> vdsm.storage.exception.MiscFileReadException: Internal file read >> failure: ('/rhev/data-center/mnt/vserver-spider.eng.lab.tlv.redhat.com:_pub_delfassy_nfs__42/17093147-8e4b-4161-9ae3-4d4606b140df/dom_md/metadata', >> 1, bytearray(b"/usr/bin/dd: failed to open >> \'/rhev/data-center/mnt/vserver-spider.eng.lab.tlv.redhat.com:_pub_delfassy_nfs__42/17093147-8e4b-4161-9ae3-4d4606b140df/dom_md/metadata\': >> No such file or directory\n")) >> > > This means the metadata file was not accessible when the monitor tried to > read. > > Does it happen now? > > Anyway this is not the unexpected error that failed the operation. > > Can you share vdsm log showing the original error? > > >> when looking at the directory I do see that metadata was created under >> this directory, with the following content: >> [delfassy@delfassy mnt]$ cd >> new_nfs/cddff7bb-bb0a-4fc8-8ba4-023be91d822d/dom_md/ >> [delfassy@delfassy dom_md]$ less metadata >> CLASS=Data >> DESCRIPTION=spider >> IOOPTIMEOUTSEC=10 >> LEASERETRIES=3 >> LEASETIMESEC=60 >> LOCKPOLICY= >> LOCKRENEWALINTERVALSEC=5 >> POOL_UUID= >> REMOTE_PATH=vserver-spider.eng.lab.tlv.redhat.com:/pub/delfassy/new_nfs >> ROLE=Regular >> SDUUID=cddff7bb-bb0a-4fc8-8ba4-023be91d822d >> TYPE=NFS >> VERSION=4 >> _SHA_CKSUM=f4953016c94f205127b00a5b10f0e7d5d9cf24f7 >> > > Looks correct > > >> The permissions on the directory are: >> drwxrwxrwx. 3 vdsm kvm 4096 Jul 29 11:49 new_mnt >> > > You don't need 0777, 0750 should work. > > ls -lhZ? > > >> Are there any other configurations I'm missing? Any idea how this issue >> can be resolved? >> Thanks, >> Dana >> _______________________________________________ >> Devel mailing list -- devel(a)ovirt.org >> To unsubscribe send an email to devel-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/L6MUN6YP74W... >> >