On problematic node: [root@virtual5 ~]# ls -la /var/lib/glusterd/peers/ total 20 drwxr-xr-x. 2 root root 4096 May 21 16:33 . drwxr-xr-x. 9 root root 4096 May 21 16:33 .. -rw-------. 1 root root 73 May 21 16:33 85c2a08c-a955-47cc-a924-cf66c6814654 -rw-------. 1 root root 73 May 21 16:33 c22e41b8-2818-4a96-a6df-a237517836d6 -rw-------. 1 root root 73 May 21 16:33 d95558a0-a306-4812-aec2-a361a9ddde3e [root@virtual5 ~]# cat /var/lib/glusterd/peers/85c2a08c-a955-47cc-a924-cf66c6814654 uuid=85c2a08c-a955-47cc-a924-cf66c6814654 state=3 hostname1=10.125.1.194 [root@virtual5 ~]# cat /var/lib/glusterd/peers/c22e41b8-2818-4a96-a6df-a237517836d6 uuid=c22e41b8-2818-4a96-a6df-a237517836d6 state=3 hostname1=10.125.1.196 [root@virtual5 ~]# cat /var/lib/glusterd/peers/d95558a0-a306-4812-aec2-a361a9ddde3e uuid=85c2a08c-a955-47cc-a924-cf66c6814654 state=3 hostname1=10.125.1.194 on other 2 nodes [root@virtual4 ~]# ls -la /var/lib/glusterd/peers/ total 16 drwxr-xr-x. 2 root root 4096 May 21 16:34 . drwxr-xr-x. 9 root root 4096 May 21 16:34 .. -rw-------. 1 root root 73 May 21 16:34 bd2e35c6-bb9a-4ec0-a6e4-23baa123dd84 -rw-------. 1 root root 73 May 21 11:09 c22e41b8-2818-4a96-a6df-a237517836d6 [root@virtual4 ~]# cat /var/lib/glusterd/peers/bd2e35c6-bb9a-4ec0-a6e4-23baa123dd84 uuid=bd2e35c6-bb9a-4ec0-a6e4-23baa123dd84 state=3 hostname1=10.125.1.195 [root@virtual4 ~]# cat /var/lib/glusterd/peers/c22e41b8-2818-4a96-a6df-a237517836d6 uuid=c22e41b8-2818-4a96-a6df-a237517836d6 state=3 hostname1=10.125.1.196 [root@virtual6 ~]# ls -la /var/lib/glusterd/peers/ total 16 drwxr-xr-x. 2 root root 4096 May 21 16:34 . drwxr-xr-x. 9 root root 4096 May 21 16:34 .. -rw-------. 1 root root 73 May 21 11:10 85c2a08c-a955-47cc-a924-cf66c6814654 -rw-------. 1 root root 73 May 21 16:34 bd2e35c6-bb9a-4ec0-a6e4-23baa123dd84 [root@virtual6 ~]# cat /var/lib/glusterd/peers/85c2a08c-a955-47cc-a924-cf66c6814654 uuid=85c2a08c-a955-47cc-a924-cf66c6814654 state=3 hostname1=10.125.1.194 [root@virtual6 ~]# cat /var/lib/glusterd/peers/bd2e35c6-bb9a-4ec0-a6e4-23baa123dd84 uuid=bd2e35c6-bb9a-4ec0-a6e4-23baa123dd84 state=3 hostname1=10.125.1.195 [root@virtual6 ~]# On Fri, May 23, 2014 at 2:05 PM, Vijay Bellur <vbellur@redhat.com> wrote:
On 05/21/2014 07:22 PM, Kanagaraj wrote:
Ok.
I am not sure deleting the file or re-peer probe would be the right way to go.
Gluster-users can help you here.
On 05/21/2014 07:08 PM, Gabi C wrote:
Hello!
I haven't change the IP, nor reinstall nodes. All nodes are updated via yum. All I can think of was that after having some issue with gluster,from WebGUI I deleted VM, deactivate and detach storage domains ( I have 2) , than, _manually_, from one of the nodes , remove
bricks, then detach peers, probe them, add bricks again, bring the volume up, and readd storage domains from the webGUI.
On Wed, May 21, 2014 at 4:26 PM, Kanagaraj <kmayilsa@redhat.com <mailto:kmayilsa@redhat.com>> wrote:
What are the steps which led this situation?
Did you re-install one of the nodes after forming the cluster or reboot which could have changed the ip?
On 05/21/2014 03:43 PM, Gabi C wrote:
On afected node:
gluster peer status
gluster peer status Number of Peers: 3
Hostname: 10.125.1.194 Uuid: 85c2a08c-a955-47cc-a924-cf66c6814654 State: Peer in Cluster (Connected)
Hostname: 10.125.1.196 Uuid: c22e41b8-2818-4a96-a6df-a237517836d6 State: Peer in Cluster (Connected)
Hostname: 10.125.1.194 Uuid: 85c2a08c-a955-47cc-a924-cf66c6814654 State: Peer in Cluster (Connected)
ls -la /var/lib/gluster
ls -la /var/lib/glusterd/peers/ total 20 drwxr-xr-x. 2 root root 4096 May 21 11:10 . drwxr-xr-x. 9 root root 4096 May 21 11:09 .. -rw-------. 1 root root 73 May 21 11:10 85c2a08c-a955-47cc-a924-cf66c6814654 -rw-------. 1 root root 73 May 21 10:52 c22e41b8-2818-4a96-a6df-a237517836d6 -rw-------. 1 root root 73 May 21 11:10 d95558a0-a306-4812-aec2-a361a9ddde3e
Can you please check the output of cat /var/lib/glusterd/peers/ d95558a0-a306-4812-aec2-a361a9ddde3e ?
If it does contain information about the duplicated peer and none of the other 2 nodes do have this file in /var/lib/glusterd/peers/, the file can be moved out of /var/lib/glusterd or deleted.
Regards, Vijay