[ovirt-users] Sanlock add Lockspace Errors

[Nir Soffer]

 On Mon, May 30, 2016 at 11:06 AM, InterNetX - Juergen Gotteswinter
<jg at internetx.com> wrote:
> Hi,
>
> since some time we get Error Messages from Sanlock, and so far i was not
> able to figure out what exactly they try to tell and more important if
> its something which can be ignored or needs to be fixed (and how).

Sanlock errors messages are somewhat cryptic, hopefully David can explain them.

>
> Here are the Versions we are using currently:
>
> Engine
>
> ovirt-engine-3.5.6.2-1.el6.noarch
>
> Nodes
>
> vdsm-4.16.34-0.el7.centos.x86_64
> sanlock-3.2.4-1.el7.x86_64
> libvirt-lock-sanlock-1.2.17-13.el7_2.3.x86_64
> libvirt-daemon-1.2.17-13.el7_2.3.x86_64
> libvirt-lock-sanlock-1.2.17-13.el7_2.3.x86_64
> libvirt-1.2.17-13.el7_2.3.x86_64
>
> -- snip --
> May 30 09:55:27 vm2 sanlock[1094]: 2016-05-30 09:55:27+0200 294109
> [60137]: verify_leader 2 wrong space name
> 4643f652-8014-4951-8a1a-02af41e67d08
> f757b127-a951-4fa9-bf90-81180c0702e6
> /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids
> May 30 09:55:27 vm2 sanlock[1094]: 2016-05-30 09:55:27+0200 294109
> [60137]: leader1 delta_acquire_begin error -226 lockspace
> f757b127-a951-4fa9-bf90-81180c0702e6 host_id 2
> May 30 09:55:27 vm2 sanlock[1094]: 2016-05-30 09:55:27+0200 294109
> [60137]: leader2 path /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids offset 0
> May 30 09:55:27 vm2 sanlock[1094]: 2016-05-30 09:55:27+0200 294109
> [60137]: leader3 m 12212010 v 30003 ss 512 nh 0 mh 1 oi 2 og 8 lv 0
> May 30 09:55:27 vm2 sanlock[1094]: 2016-05-30 09:55:27+0200 294109
> [60137]: leader4 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 1eed8aa9-8fb5-4d27-8d1c-03ebce2c36d4.vm2.intern ts 3786679 cs 1474f033
> May 30 09:55:28 vm2 sanlock[1094]: 2016-05-30 09:55:28+0200 294110
> [1099]: s9703 add_lockspace fail result -226
> May 30 09:55:58 vm2 sanlock[1094]: 2016-05-30 09:55:58+0200 294140
> [60331]: verify_leader 2 wrong space name
> 4643f652-8014-4951-8a1a-02af41e67d08
> f757b127-a951-4fa9-bf90-81180c0702e6
> /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids
> May 30 09:55:58 vm2 sanlock[1094]: 2016-05-30 09:55:58+0200 294140
> [60331]: leader1 delta_acquire_begin error -226 lockspace
> f757b127-a951-4fa9-bf90-81180c0702e6 host_id 2
> May 30 09:55:58 vm2 sanlock[1094]: 2016-05-30 09:55:58+0200 294140
> [60331]: leader2 path /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids offset 0
> May 30 09:55:58 vm2 sanlock[1094]: 2016-05-30 09:55:58+0200 294140
> [60331]: leader3 m 12212010 v 30003 ss 512 nh 0 mh 1 oi 2 og 8 lv 0
> May 30 09:55:58 vm2 sanlock[1094]: 2016-05-30 09:55:58+0200 294140
> [60331]: leader4 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 1eed8aa9-8fb5-4d27-8d1c-03ebce2c36d4.vm2.intern ts 3786679 cs 1474f033
> May 30 09:55:59 vm2 sanlock[1094]: 2016-05-30 09:55:59+0200 294141
> [1098]: s9704 add_lockspace fail result -226
> May 30 09:56:05 vm2 sanlock[1094]: 2016-05-30 09:56:05+0200 294148
> [1094]: s1527 check_other_lease invalid for host 0 0 ts 7566376 name  in
> 4643f652-8014-4951-8a1a-02af41e67d08
> May 30 09:56:05 vm2 sanlock[1094]: 2016-05-30 09:56:05+0200 294148
> [1094]: s1527 check_other_lease leader 12212010 owner 1 11 ts 7566376 sn
> f757b127-a951-4fa9-bf90-81180c0702e6 rn
> f888524b-27aa-4724-8bae-051f9e950a21.vm1.intern
> May 30 09:56:28 vm2 sanlock[1094]: 2016-05-30 09:56:28+0200 294170
> [60496]: verify_leader 2 wrong space name
> 4643f652-8014-4951-8a1a-02af41e67d08
> f757b127-a951-4fa9-bf90-81180c0702e6
> /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids
> May 30 09:56:28 vm2 sanlock[1094]: 2016-05-30 09:56:28+0200 294170
> [60496]: leader1 delta_acquire_begin error -226 lockspace
> f757b127-a951-4fa9-bf90-81180c0702e6 host_id 2
> May 30 09:56:28 vm2 sanlock[1094]: 2016-05-30 09:56:28+0200 294170
> [60496]: leader2 path /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids offset 0
> May 30 09:56:28 vm2 sanlock[1094]: 2016-05-30 09:56:28+0200 294170
> [60496]: leader3 m 12212010 v 30003 ss 512 nh 0 mh 1 oi 2 og 8 lv 0
> May 30 09:56:28 vm2 sanlock[1094]: 2016-05-30 09:56:28+0200 294170
> [60496]: leader4 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 1eed8aa9-8fb5-4d27-8d1c-03ebce2c36d4.vm2.intern ts 3786679 cs 1474f033
> May 30 09:56:29 vm2 sanlock[1094]: 2016-05-30 09:56:29+0200 294171
> [6415]: s9705 add_lockspace fail result -226
> May 30 09:56:58 vm2 sanlock[1094]: 2016-05-30 09:56:58+0200 294200
> [60645]: verify_leader 2 wrong space name
> 4643f652-8014-4951-8a1a-02af41e67d08
> f757b127-a951-4fa9-bf90-81180c0702e6
> /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids
> May 30 09:56:58 vm2 sanlock[1094]: 2016-05-30 09:56:58+0200 294200
> [60645]: leader1 delta_acquire_begin error -226 lockspace
> f757b127-a951-4fa9-bf90-81180c0702e6 host_id 2
> May 30 09:56:58 vm2 sanlock[1094]: 2016-05-30 09:56:58+0200 294200
> [60645]: leader2 path /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids offset 0
> May 30 09:56:58 vm2 sanlock[1094]: 2016-05-30 09:56:58+0200 294200
> [60645]: leader3 m 12212010 v 30003 ss 512 nh 0 mh 1 oi 2 og 8 lv 0
> May 30 09:56:58 vm2 sanlock[1094]: 2016-05-30 09:56:58+0200 294200
> [60645]: leader4 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 1eed8aa9-8fb5-4d27-8d1c-03ebce2c36d4.vm2.intern ts 3786679 cs 1474f033
> May 30 09:56:59 vm2 sanlock[1094]: 2016-05-30 09:56:59+0200 294201
> [6373]: s9706 add_lockspace fail result -226
> May 30 09:57:28 vm2 sanlock[1094]: 2016-05-30 09:57:28+0200 294230
> [60806]: verify_leader 2 wrong space name
> 4643f652-8014-4951-8a1a-02af41e67d08
> f757b127-a951-4fa9-bf90-81180c0702e6
> /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids
> May 30 09:57:28 vm2 sanlock[1094]: 2016-05-30 09:57:28+0200 294230
> [60806]: leader1 delta_acquire_begin error -226 lockspace
> f757b127-a951-4fa9-bf90-81180c0702e6 host_id 2
> May 30 09:57:28 vm2 sanlock[1094]: 2016-05-30 09:57:28+0200 294230
> [60806]: leader2 path /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids offset 0
> May 30 09:57:28 vm2 sanlock[1094]: 2016-05-30 09:57:28+0200 294230
> [60806]: leader3 m 12212010 v 30003 ss 512 nh 0 mh 1 oi 2 og 8 lv 0
> May 30 09:57:28 vm2 sanlock[1094]: 2016-05-30 09:57:28+0200 294230
> [60806]: leader4 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 1eed8aa9-8fb5-4d27-8d1c-03ebce2c36d4.vm2.intern ts 3786679 cs 1474f033
> May 30 09:57:29 vm2 sanlock[1094]: 2016-05-30 09:57:29+0200 294231
> [6399]: s9707 add_lockspace fail result -226
> May 30 09:57:58 vm2 sanlock[1094]: 2016-05-30 09:57:58+0200 294260
> [60946]: verify_leader 2 wrong space name
> 4643f652-8014-4951-8a1a-02af41e67d08
> f757b127-a951-4fa9-bf90-81180c0702e6
> /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids
> May 30 09:57:58 vm2 sanlock[1094]: 2016-05-30 09:57:58+0200 294260
> [60946]: leader1 delta_acquire_begin error -226 lockspace
> f757b127-a951-4fa9-bf90-81180c0702e6 host_id 2
> May 30 09:57:58 vm2 sanlock[1094]: 2016-05-30 09:57:58+0200 294260
> [60946]: leader2 path /dev/f757b127-a951-4fa9-bf90-81180c0702e6/ids offset 0
> May 30 09:57:58 vm2 sanlock[1094]: 2016-05-30 09:57:58+0200 294260
> [60946]: leader3 m 12212010 v 30003 ss 512 nh 0 mh 1 oi 2 og 8 lv 0
> May 30 09:57:58 vm2 sanlock[1094]: 2016-05-30 09:57:58+0200 294260
> [60946]: leader4 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 1eed8aa9-8fb5-4d27-8d1c-03ebce2c36d4.vm2.intern ts 3786679 cs 1474f033
> May 30 09:57:59 vm2 sanlock[1094]: 2016-05-30 09:57:59+0200 294261
> [6352]: s9708 add_lockspace fail result -226
> -- snip --
>
> sanlock log_dump also shows errors
>
> -- snip --
>
> 2016-05-30 09:53:23+0200 7526415 [1017]: s567 check_other_lease invalid
> for host 0 0 ts 7566376 name  in 4643f652-8014-4951-8a1a-02af41e67d08
> 2016-05-30 09:53:23+0200 7526415 [1017]: s567 check_other_lease leader
> 12212010 owner 1 11 ts 7566376 sn f757b127-a951-4fa9-bf90-81180c0702e6
> rn f888524b-27aa-4724-8bae-051f9e950a21.vm1.intern
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease invalid
> for host 0 0 ts 3786679 name  in f757b127-a951-4fa9-bf90-81180c0702e6
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease leader
> 12212010 owner 2 8 ts 3786679 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 1eed8aa9-8fb5-4d27-8d1c-03ebce2c36d4.vm2.intern
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease invalid
> for host 0 0 ts 6622415 name  in f757b127-a951-4fa9-bf90-81180c0702e6
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease leader
> 12212010 owner 3 14 ts 6622415 sn 4643f652-8014-4951-8a1a-02af41e67d08
> rn 51c8f8e2-f9d8-462c-866c-e4052213ea81.vm3.intern
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease invalid
> for host 0 0 ts 6697413 name  in f757b127-a951-4fa9-bf90-81180c0702e6
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease leader
> 12212010 owner 4 4 ts 6697413 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 8d4f32dc-f595-4254-bfcb-d96e5057e110.vm4.intern
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease invalid
> for host 0 0 ts 7563413 name  in f757b127-a951-4fa9-bf90-81180c0702e6
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease leader
> 12212010 owner 5 8 ts 7563413 sn 4643f652-8014-4951-8a1a-02af41e67d08 rn
> 9fe564e6-cf40-4403-9fc0-eb118e1ee2cf.vm5.intern
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease invalid
> for host 0 0 ts 6129706 name  in f757b127-a951-4fa9-bf90-81180c0702e6
> 2016-05-30 09:53:33+0200 7526425 [1017]: s568 check_other_lease leader
> 12212010 owner 6 169 ts 6129706 sn 4643f652-8014-4951-8a1a-02af41e67d08
> rn b99ff588-f1ad-43f2-bd6c-6869f54d424d.vm1-tiny.i
> 2016-05-30 09:53:47+0200 7526439 [14576]: cmd_read_resource 23,58
> /dev/c2212f15-35b7-4fa0-b13c-b50befda0af9/leases:1048576
> -- snip --
>
> any hint whould be highly appreciated
>
> Juergen
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users