[ovirt-users] sanlock ids file broken after server crash

[Johan Bernhardsson]

(First reply did not get to the list)

>From sanlock.log:

2017-07-30 10:49:31+0200 1766275 [1171]: s310751 lockspace 0924ff77-
ef51-435b-b90d-50bfbf2e8de7:1:/rhev/data-
center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d-
50bfbf2e8de7/dom_md/ids:0
2017-07-30 10:49:31+0200 1766275 [10496]: verify_leader 1 wrong space
name 0924ff77-ef51-435b-b90d-50bfbf2e<D5>ke7 0924ff77-ef51-435b-b90d-
50bfbf2e8de7 /rhev/data-center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-
ef51-435b-b90d-50bfbf2e8de7/dom_md/ids
2017-07-30 10:49:31+0200 1766275 [10496]: leader1 delta_acquire_begin
error -226 lockspace 0924ff77-ef51-435b-b90d-50bfbf2e8de7 host_id 1
2017-07-30 10:49:31+0200 1766275 [10496]: leader2 path /rhev/data-
center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d-
50bfbf2e8de7/dom_md/ids offset 0
2017-07-30 10:49:31+0200 1766275 [10496]: leader3 m 12212010 v 30003 ss
512 nh 0 mh 4076 oi 1 og 2031079063 lv 0
2017-07-30 10:49:31+0200 1766275 [10496]: leader4 sn 0924ff77-ef51-
435b-b90d-50bfbf2e<D5>ke7 rn <93><F6>7^\afa5-3a91-415b-a04c-
221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7
2017-07-30 10:49:32+0200 1766276 [1171]: s310751 add_lockspace fail
result -226


vdsm logs doesnt have any errors and engine.log does not have any
errors. 

And if i check the ids file manually. I can see that everything in it
is correct except for the first host in the cluster where the space
name and host id is broken. 


/Johan

On Sun, 2017-07-30 at 11:18 +0300, Maor Lipchuk wrote:
> Hi Johan,
> 
> Can you please share the vdsm and engine logs.
> 
> Also, it won't harm to also get the sanlock logs just in case sanlock
> was configured to save all debugging in a log file (see
> http://people.redhat.com/teigland/sanlock-messages.txt)).
> Try to share the sanlock ouput by running  'sanlock client status',
> 'sanlock client log_dump'.
> 
> Regards,
> Maor
> 
> On Thu, Jul 27, 2017 at 6:18 PM, Johan Bernhardsson <johan at kafit.se>
> wrote:
> > 
> > Hello,
> > 
> > The ids file for sanlock is broken on one setup. The first host id
> > in
> > the file is wrong.
> > 
> > From the logfile i have:
> > 
> > verify_leader 1 wrong space name 0924ff77-ef51-435b-b90d-
> > 50bfbf2e�ke7
> > 0924ff77-ef51-435b-b90d-50bfbf2e8de7 /rhev/data-
> > center/mnt/glusterSD/
> > 
> > 
> > 
> > Note the broken char in the space name.
> > 
> > This also apears. And it seams as the hostid too is broken in the
> > ids
> > file:
> > 
> > leader4 sn 0924ff77-ef51-435b-b90d-50bfbf2e�ke7 rn ��7 afa5-3a91-
> > 415b-
> > a04c-221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7
> > 
> > Note the broken chars there as well.
> > 
> > If i check the ids file with less or strings the first row where my
> > vbgkvm01 host are. That has broken chars.
> > 
> > Can this be repaired in some way without taking down all the
> > virtual
> > machines on that storage?
> > 
> > 
> > /Johan
> > _______________________________________________
> > Users mailing list
> > Users at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users