[ovirt-users] sanlock ids file broken after server crash

[Maor Lipchuk]

Hi David,

I'm not sure how it got to that character in the first place.
Nir, Is there a safe way to fix that while there are running VMs?

Regards,
Maor

On Sun, Jul 30, 2017 at 11:58 AM, Johan Bernhardsson <johan at kafit.se> wrote:
> (First reply did not get to the list)
>
> From sanlock.log:
>
> 2017-07-30 10:49:31+0200 1766275 [1171]: s310751 lockspace 0924ff77-
> ef51-435b-b90d-50bfbf2e8de7:1:/rhev/data-
> center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d-
> 50bfbf2e8de7/dom_md/ids:0
> 2017-07-30 10:49:31+0200 1766275 [10496]: verify_leader 1 wrong space
> name 0924ff77-ef51-435b-b90d-50bfbf2e<D5>ke7 0924ff77-ef51-435b-b90d-
> 50bfbf2e8de7 /rhev/data-center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-
> ef51-435b-b90d-50bfbf2e8de7/dom_md/ids
> 2017-07-30 10:49:31+0200 1766275 [10496]: leader1 delta_acquire_begin
> error -226 lockspace 0924ff77-ef51-435b-b90d-50bfbf2e8de7 host_id 1
> 2017-07-30 10:49:31+0200 1766275 [10496]: leader2 path /rhev/data-
> center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d-
> 50bfbf2e8de7/dom_md/ids offset 0
> 2017-07-30 10:49:31+0200 1766275 [10496]: leader3 m 12212010 v 30003 ss
> 512 nh 0 mh 4076 oi 1 og 2031079063 lv 0
> 2017-07-30 10:49:31+0200 1766275 [10496]: leader4 sn 0924ff77-ef51-
> 435b-b90d-50bfbf2e<D5>ke7 rn <93><F6>7^\afa5-3a91-415b-a04c-
> 221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7
> 2017-07-30 10:49:32+0200 1766276 [1171]: s310751 add_lockspace fail
> result -226
>
>
> vdsm logs doesnt have any errors and engine.log does not have any
> errors.
>
> And if i check the ids file manually. I can see that everything in it
> is correct except for the first host in the cluster where the space
> name and host id is broken.
>
>
> /Johan
>
> On Sun, 2017-07-30 at 11:18 +0300, Maor Lipchuk wrote:
>> Hi Johan,
>>
>> Can you please share the vdsm and engine logs.
>>
>> Also, it won't harm to also get the sanlock logs just in case sanlock
>> was configured to save all debugging in a log file (see
>> http://people.redhat.com/teigland/sanlock-messages.txt)).
>> Try to share the sanlock ouput by running  'sanlock client status',
>> 'sanlock client log_dump'.
>>
>> Regards,
>> Maor
>>
>> On Thu, Jul 27, 2017 at 6:18 PM, Johan Bernhardsson <johan at kafit.se>
>> wrote:
>> >
>> > Hello,
>> >
>> > The ids file for sanlock is broken on one setup. The first host id
>> > in
>> > the file is wrong.
>> >
>> > From the logfile i have:
>> >
>> > verify_leader 1 wrong space name 0924ff77-ef51-435b-b90d-
>> > 50bfbf2e�ke7
>> > 0924ff77-ef51-435b-b90d-50bfbf2e8de7 /rhev/data-
>> > center/mnt/glusterSD/
>> >
>> >
>> >
>> > Note the broken char in the space name.
>> >
>> > This also apears. And it seams as the hostid too is broken in the
>> > ids
>> > file:
>> >
>> > leader4 sn 0924ff77-ef51-435b-b90d-50bfbf2e�ke7 rn ��7 afa5-3a91-
>> > 415b-
>> > a04c-221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7
>> >
>> > Note the broken chars there as well.
>> >
>> > If i check the ids file with less or strings the first row where my
>> > vbgkvm01 host are. That has broken chars.
>> >
>> > Can this be repaired in some way without taking down all the
>> > virtual
>> > machines on that storage?
>> >
>> >
>> > /Johan
>> > _______________________________________________
>> > Users mailing list
>> > Users at ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/users