On Fri, Jan 25, 2019 at 7:36 PM Jorick Astrego <jorick@netbulae.eu> wrote:

The mount failure is probably related to glusterfs. There are glusterfs logs on the host that 

can give more info on this error. 

Oh duh, sorry forgot to check :-&

"failed to fetch volume file"

[2019-01-25 14:02:45.067440] I [glusterfsd-mgmt.c:2424:mgmt_rpc_notify] 0-glusterfsd-mgmt: disconnected from remote-host: *.*.*.*

Are you able to access this host, and the gluster ports open?

Anything in the glusterd.log of the gluster server?

Adding Sanju to help

From the host:

Telnet *.*.*.14 24007
Trying *.*.*.14...
Connected to *.*.*.14.
Escape character is '^]'.

glusterd.log on *.*.*.14:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination        
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     icmp --  anywhere             anywhere           
ACCEPT     all  --  anywhere             anywhere           
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:54321
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:54322
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:sunrpc
ACCEPT     udp  --  anywhere             anywhere             udp dpt:sunrpc
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh
ACCEPT     udp  --  anywhere             anywhere             udp dpt:snmp
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:websm
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:24007
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:webcache
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:38465
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:38466
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:38467
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:nfs
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:38469
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:5666
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:39543
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:55863
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:38468
ACCEPT     udp  --  anywhere             anywhere             udp dpt:963
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:965
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ctdb
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:netbios-ssn
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:microsoft-ds
ACCEPT     tcp  --  anywhere             anywhere             tcp dpts:24009:24108
ACCEPT     tcp  --  anywhere             anywhere             tcp dpts:49152:49251
ACCEPT     tcp  --  anywhere             anywhere             tcp dpts:49217:49316
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:zabbix-agent /* Zabbix agent */
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination        
REJECT     all  --  anywhere             anywhere             PHYSDEV match ! --physdev-is-bridged reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination        

And in the glusterd.log has nothing else but:

[2019-01-25 11:02:32.617694] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:02:32.623892] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:03:31.847006] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:04:33.023685] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:04:33.030549] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:04:33.037024] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:04:33.043442] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:04:33.049494] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:04:33.056049] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:04:33.062081] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:05:47.710098] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:06:33.481200] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:06:33.488137] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:06:33.494492] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:06:33.501053] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:06:33.507302] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:06:33.513299] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:06:33.519269] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:08:33.940033] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:08:33.947011] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:08:33.953445] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:08:33.959618] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:08:33.965659] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:08:33.971609] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:08:33.977763] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:10:58.656508] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:11:05.746810] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:11:08.153573] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:11:11.108019] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:11:13.456055] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:11:13.584535] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:11:16.046244] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:11:16.172035] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:11:18.370339] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:11:18.504515] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:11:20.710031] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:11:05.886182] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:11:08.284806] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:11:11.244245] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:11:20.835972] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:13:33.238060] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:15:36.929536] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:15:36.937647] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:15:36.943936] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:15:36.950206] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:15:36.956460] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:15:36.962459] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:15:36.968594] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:21:14.123760] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:22:38.575269] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:22:38.582011] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:22:38.588583] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3