[ovirt-users] ovirt 4.1.2 sanlock issue with nfs data domain

[Moritz Baumann]

> Is there a way to "reinitialize" the lockspace so one node can become 
> SPM again and we can run VMS.

errors in /var/log/sanlock.log look like this:


2017-06-19 10:57:00+0200 1617673 [126217]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
2017-06-19 10:57:00+0200 1617673 [126217]: s51 open_disk 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids 
error -13
2017-06-19 10:57:01+0200 1617674 [880]: s51 add_lockspace fail result -19
2017-06-19 10:57:02+0200 1617674 [881]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:02+0200 1617674 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:10+0200 1617683 [881]: s52 lockspace 
c17d9d7f-e578-4626-a5d9-94ea555d7115:2:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids:0
2017-06-19 10:57:10+0200 1617683 [126235]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
2017-06-19 10:57:10+0200 1617683 [126235]: s52 open_disk 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids 
error -13
2017-06-19 10:57:11+0200 1617684 [881]: s52 add_lockspace fail result -19
2017-06-19 10:57:13+0200 1617685 [880]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:13+0200 1617685 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:15+0200 1617688 [881]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:15+0200 1617688 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:20+0200 1617693 [881]: s53 lockspace 
c17d9d7f-e578-4626-a5d9-94ea555d7115:2:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids:0
2017-06-19 10:57:20+0200 1617693 [126255]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
2017-06-19 10:57:20+0200 1617693 [126255]: s53 open_disk 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids 
error -13
2017-06-19 10:57:21+0200 1617694 [881]: s53 add_lockspace fail result -19
2017-06-19 10:57:26+0200 1617699 [880]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:26+0200 1617699 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:29+0200 1617702 [881]: open error -13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:29+0200 1617702 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:30+0200 1617703 [880]: s54 lockspace 
c17d9d7f-e578-4626-a5d9-94ea555d7115:2:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids:0



ovirt-node01[0]:/var/log# ls -ld 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids

-rw-rw----. 1 vdsm kvm 1048576 28. Mai 23:13 
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids

The nfs share is writeable:

ovirt-node01[0]:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md# 
touch blabla
ovirt-node01[0]:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md# 
ls -l
total 3320
-rw-r--r--. 1 root root        0 19. Jun 11:00 blabla
-rw-rw----. 1 vdsm kvm   1048576 28. Mai 23:13 ids
-rw-rw----. 1 vdsm kvm  16777216 19. Jun 10:56 inbox
-rw-rw----. 1 vdsm kvm   2097152 22. Mai 15:48 leases
-rw-r--r--. 1 vdsm kvm       361  1. Mär 18:21 metadata
-rw-rw----. 1 vdsm kvm  16777216 22. Mai 15:48 outbox
-rw-rw----. 1 vdsm kvm   1305088  1. Mär 18:21 xleases