From sanlock.log:

Hi Johan, Can you please share the vdsm and engine logs. Also, it won't harm to also get the sanlock logs just in case sanlock was configured to save all debugging in a log file (see http://people.redhat.com/teigland/sanlock-messages.txt)). Try to share the sanlock ouput by running  'sanlock client status', 'sanlock client log_dump'. Regards, Maor On Thu, Jul 27, 2017 at 6:18 PM, Johan Bernhardsson <johan(a)kafit.se&gt; wrote: > > Hello, > > The ids file for sanlock is broken on one setup. The first host id > in > the file is wrong. > > From the logfile i have: > > verify_leader 1 wrong space name 0924ff77-ef51-435b-b90d- > 50bfbf2e�ke7 > 0924ff77-ef51-435b-b90d-50bfbf2e8de7 /rhev/data- > center/mnt/glusterSD/ > > > > Note the broken char in the space name. > > This also apears. And it seams as the hostid too is broken in the > ids > file: > > leader4 sn 0924ff77-ef51-435b-b90d-50bfbf2e�ke7 rn ��7 afa5-3a91- > 415b- > a04c-221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7 > > Note the broken chars there as well. > > If i check the ids file with less or strings the first row where my > vbgkvm01 host are. That has broken chars. > > Can this be repaired in some way without taking down all the > virtual > machines on that storage? > > > /Johan > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users

Hi David,

I'm not sure how it got to that character in the first place. Nir, Is there a safe way to fix that while there are running VMs? Regards, Maor On Sun, Jul 30, 2017 at 11:58 AM, Johan Bernhardsson <johan(a)kafit.se&gt; wrote: > (First reply did not get to the list) > > From sanlock.log: > > 2017-07-30 10:49:31+0200 1766275 [1171]: s310751 lockspace 0924ff77- > ef51-435b-b90d-50bfbf2e8de7:1:/rhev/data- > center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d- > 50bfbf2e8de7/dom_md/ids:0 > 2017-07-30 10:49:31+0200 1766275 [10496]: verify_leader 1 wrong space > name 0924ff77-ef51-435b-b90d-50bfbf2e<D5>ke7 0924ff77-ef51-435b-b90d- > 50bfbf2e8de7 /rhev/data-center/mnt/glusterSD/vbgsan02:_fs02/0924ff77- > ef51-435b-b90d-50bfbf2e8de7/dom_md/ids > 2017-07-30 10:49:31+0200 1766275 [10496]: leader1 delta_acquire_begin > error -226 lockspace 0924ff77-ef51-435b-b90d-50bfbf2e8de7 host_id 1 > 2017-07-30 10:49:31+0200 1766275 [10496]: leader2 path /rhev/data- > center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d- > 50bfbf2e8de7/dom_md/ids offset 0 > 2017-07-30 10:49:31+0200 1766275 [10496]: leader3 m 12212010 v 30003 ss > 512 nh 0 mh 4076 oi 1 og 2031079063 lv 0 > 2017-07-30 10:49:31+0200 1766275 [10496]: leader4 sn 0924ff77-ef51- > 435b-b90d-50bfbf2e<D5>ke7 rn <93><F6>7^\afa5-3a91-415b-a04c- > 221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7 > 2017-07-30 10:49:32+0200 1766276 [1171]: s310751 add_lockspace fail > result -226 > > > vdsm logs doesnt have any errors and engine.log does not have any > errors. > > And if i check the ids file manually. I can see that everything in it > is correct except for the first host in the cluster where the space > name and host id is broken. > > > /Johan > > On Sun, 2017-07-30 at 11:18 +0300, Maor Lipchuk wrote: >> Hi Johan, >> >> Can you please share the vdsm and engine logs. >> >> Also, it won't harm to also get the sanlock logs just in case sanlock >> was configured to save all debugging in a log file (see >> http://people.redhat.com/teigland/sanlock-messages.txt)). >> Try to share the sanlock ouput by running 'sanlock client status', >> 'sanlock client log_dump'. >> >> Regards, >> Maor >> >> On Thu, Jul 27, 2017 at 6:18 PM, Johan Bernhardsson <johan(a)kafit.se&gt; >> wrote: >> > >> > Hello, >> > >> > The ids file for sanlock is broken on one setup. The first host id >> > in >> > the file is wrong. >> > >> > From the logfile i have: >> > >> > verify_leader 1 wrong space name 0924ff77-ef51-435b-b90d- >> > 50bfbf2e�ke7 >> > 0924ff77-ef51-435b-b90d-50bfbf2e8de7 /rhev/data- >> > center/mnt/glusterSD/ >> > >> > >> > >> > Note the broken char in the space name. >> > >> > This also apears. And it seams as the hostid too is broken in the >> > ids >> > file: >> > >> > leader4 sn 0924ff77-ef51-435b-b90d-50bfbf2e�ke7 rn ��7 afa5-3a91- >> > 415b- >> > a04c-221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7 >> > >> > Note the broken chars there as well. >> > >> > If i check the ids file with less or strings the first row where my >> > vbgkvm01 host are. That has broken chars. >> > >> > Can this be repaired in some way without taking down all the >> > virtual >> > machines on that storage? >> > >> > >> > /Johan >> > _______________________________________________ >> > Users mailing list >> > Users(a)ovirt.org >> > http://lists.ovirt.org/mailman/listinfo/users

Hi David, I'm not sure how it got to that character in the first place. Nir, Is there a safe way to fix that while there are running VMs? Regards, Maor On Sun, Jul 30, 2017 at 11:58 AM, Johan Bernhardsson <johan(a)kafit.se&gt; wrote: > > (First reply did not get to the list) > > From sanlock.log: > > 2017-07-30 10:49:31+0200 1766275 [1171]: s310751 lockspace > 0924ff77- > ef51-435b-b90d-50bfbf2e8de7:1:/rhev/data- > center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d- > 50bfbf2e8de7/dom_md/ids:0 > 2017-07-30 10:49:31+0200 1766275 [10496]: verify_leader 1 wrong > space > name 0924ff77-ef51-435b-b90d-50bfbf2e<D5>ke7 0924ff77-ef51-435b- > b90d- > 50bfbf2e8de7 /rhev/data- > center/mnt/glusterSD/vbgsan02:_fs02/0924ff77- > ef51-435b-b90d-50bfbf2e8de7/dom_md/ids > 2017-07-30 10:49:31+0200 1766275 [10496]: leader1 > delta_acquire_begin > error -226 lockspace 0924ff77-ef51-435b-b90d-50bfbf2e8de7 host_id 1 > 2017-07-30 10:49:31+0200 1766275 [10496]: leader2 path /rhev/data- > center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-435b-b90d- > 50bfbf2e8de7/dom_md/ids offset 0 > 2017-07-30 10:49:31+0200 1766275 [10496]: leader3 m 12212010 v > 30003 ss > 512 nh 0 mh 4076 oi 1 og 2031079063 lv 0 > 2017-07-30 10:49:31+0200 1766275 [10496]: leader4 sn 0924ff77-ef51- > 435b-b90d-50bfbf2e<D5>ke7 rn <93><F6>7^\afa5-3a91-415b-a04c- > 221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7 > 2017-07-30 10:49:32+0200 1766276 [1171]: s310751 add_lockspace fail > result -226 > > > vdsm logs doesnt have any errors and engine.log does not have any > errors. > > And if i check the ids file manually. I can see that everything in > it > is correct except for the first host in the cluster where the space > name and host id is broken. > > > /Johan > > On Sun, 2017-07-30 at 11:18 +0300, Maor Lipchuk wrote: > > > > Hi Johan, > > > > Can you please share the vdsm and engine logs. > > > > Also, it won't harm to also get the sanlock logs just in case > > sanlock > > was configured to save all debugging in a log file (see > > http://people.redhat.com/teigland/sanlock-messages.txt)). > > Try to share the sanlock ouput by running  'sanlock client > > status', > > 'sanlock client log_dump'. > > > > Regards, > > Maor > > > > On Thu, Jul 27, 2017 at 6:18 PM, Johan Bernhardsson <johan@kafit. > > se> > > wrote: > > > > > > > > > Hello, > > > > > > The ids file for sanlock is broken on one setup. The first host > > > id > > > in > > > the file is wrong. > > > > > > From the logfile i have: > > > > > > verify_leader 1 wrong space name 0924ff77-ef51-435b-b90d- > > > 50bfbf2e�ke7 > > > 0924ff77-ef51-435b-b90d-50bfbf2e8de7 /rhev/data- > > > center/mnt/glusterSD/ > > > > > > > > > > > > Note the broken char in the space name. > > > > > > This also apears. And it seams as the hostid too is broken in > > > the > > > ids > > > file: > > > > > > leader4 sn 0924ff77-ef51-435b-b90d-50bfbf2e�ke7 rn ��7 afa5- > > > 3a91- > > > 415b- > > > a04c-221d3e060163.vbgkvm01.a ts 4351980 cs eefa4dd7 > > > > > > Note the broken chars there as well. > > > > > > If i check the ids file with less or strings the first row > > > where my > > > vbgkvm01 host are. That has broken chars. > > > > > > Can this be repaired in some way without taking down all the > > > virtual > > > machines on that storage? > > > > > > > > > /Johan > > > _______________________________________________ > > > Users mailing list > > > Users(a)ovirt.org > > > http://lists.ovirt.org/mailman/listinfo/users