Is there a way to "reinitialize" the lockspace so one node
can become
SPM again and we can run VMS.
errors in /var/log/sanlock.log look like this:
2017-06-19 10:57:00+0200 1617673 [126217]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
2017-06-19 10:57:00+0200 1617673 [126217]: s51 open_disk
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
error -13
2017-06-19 10:57:01+0200 1617674 [880]: s51 add_lockspace fail result -19
2017-06-19 10:57:02+0200 1617674 [881]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:02+0200 1617674 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:10+0200 1617683 [881]: s52 lockspace
c17d9d7f-e578-4626-a5d9-94ea555d7115:2:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids:0
2017-06-19 10:57:10+0200 1617683 [126235]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
2017-06-19 10:57:10+0200 1617683 [126235]: s52 open_disk
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
error -13
2017-06-19 10:57:11+0200 1617684 [881]: s52 add_lockspace fail result -19
2017-06-19 10:57:13+0200 1617685 [880]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:13+0200 1617685 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:15+0200 1617688 [881]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:15+0200 1617688 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:20+0200 1617693 [881]: s53 lockspace
c17d9d7f-e578-4626-a5d9-94ea555d7115:2:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids:0
2017-06-19 10:57:20+0200 1617693 [126255]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
2017-06-19 10:57:20+0200 1617693 [126255]: s53 open_disk
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
error -13
2017-06-19 10:57:21+0200 1617694 [881]: s53 add_lockspace fail result -19
2017-06-19 10:57:26+0200 1617699 [880]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:26+0200 1617699 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:29+0200 1617702 [881]: open error -13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/leases
2017-06-19 10:57:29+0200 1617702 [828]: ci 2 fd 11 pid -1 recv errno 104
2017-06-19 10:57:30+0200 1617703 [880]: s54 lockspace
c17d9d7f-e578-4626-a5d9-94ea555d7115:2:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids:0
ovirt-node01[0]:/var/log# ls -ld
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
-rw-rw----. 1 vdsm kvm 1048576 28. Mai 23:13
/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md/ids
The nfs share is writeable:
ovirt-node01[0]:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md#
touch blabla
ovirt-node01[0]:/rhev/data-center/mnt/scratch-inf.inf.ethz.ch:_export_scratch_ovirt_data/c17d9d7f-e578-4626-a5d9-94ea555d7115/dom_md#
ls -l
total 3320
-rw-r--r--. 1 root root 0 19. Jun 11:00 blabla
-rw-rw----. 1 vdsm kvm 1048576 28. Mai 23:13 ids
-rw-rw----. 1 vdsm kvm 16777216 19. Jun 10:56 inbox
-rw-rw----. 1 vdsm kvm 2097152 22. Mai 15:48 leases
-rw-r--r--. 1 vdsm kvm 361 1. Mär 18:21 metadata
-rw-rw----. 1 vdsm kvm 16777216 22. Mai 15:48 outbox
-rw-rw----. 1 vdsm kvm 1305088 1. Mär 18:21 xleases