Hi, I will probably be able to look into hanged supervdsmd issue only next week. I'll reply as soon as I have some insight. Regards, Marcin On 1/29/19 12:19 PM, Jorick Astrego wrote:
Hi,
I know people are busy but do we have any progress on this, I removed the OVS cluster and recreated it with bridge networking.
But I'm still stuck on the "failed to fetch volume file" and am unable to activate the hosts to continue testing...
Anything else I can try to debug? The storage network is ip based without dns.
Regards, Jorick
On 1/25/19 3:53 PM, Jorick Astrego wrote:
On 1/25/19 3:26 PM, Sahina Bose wrote:
On Fri, Jan 25, 2019 at 7:36 PM Jorick Astrego <jorick@netbulae.eu <mailto:jorick@netbulae.eu>> wrote:
The mount failure is probably related to glusterfs. There are glusterfs logs on the host that can give more info on this error.
Oh duh, sorry forgot to check :-&
"failed to fetch volume file"
[2019-01-25 14:02:45.067440] I [glusterfsd-mgmt.c:2424:mgmt_rpc_notify] 0-glusterfsd-mgmt: disconnected from remote-host: *.*.*.*
Are you able to access this host, and the gluster ports open? Anything in the glusterd.log of the gluster server?
Adding Sanju to help
From the host:
Telnet *.*.*.14 24007 Trying *.*.*.14... Connected to *.*.*.14. Escape character is '^]'.
glusterd.log on *.*.*.14:
Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT icmp -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT tcp -- anywhere anywhere tcp dpt:54321 ACCEPT tcp -- anywhere anywhere tcp dpt:54322 ACCEPT tcp -- anywhere anywhere tcp dpt:sunrpc ACCEPT udp -- anywhere anywhere udp dpt:sunrpc ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ACCEPT udp -- anywhere anywhere udp dpt:snmp ACCEPT tcp -- anywhere anywhere tcp dpt:websm ACCEPT tcp -- anywhere anywhere tcp dpt:24007 ACCEPT tcp -- anywhere anywhere tcp dpt:webcache ACCEPT tcp -- anywhere anywhere tcp dpt:38465 ACCEPT tcp -- anywhere anywhere tcp dpt:38466 ACCEPT tcp -- anywhere anywhere tcp dpt:38467 ACCEPT tcp -- anywhere anywhere tcp dpt:nfs ACCEPT tcp -- anywhere anywhere tcp dpt:38469 ACCEPT tcp -- anywhere anywhere tcp dpt:5666 ACCEPT tcp -- anywhere anywhere tcp dpt:39543 ACCEPT tcp -- anywhere anywhere tcp dpt:55863 ACCEPT tcp -- anywhere anywhere tcp dpt:38468 ACCEPT udp -- anywhere anywhere udp dpt:963 ACCEPT tcp -- anywhere anywhere tcp dpt:965 ACCEPT tcp -- anywhere anywhere tcp dpt:ctdb ACCEPT tcp -- anywhere anywhere tcp dpt:netbios-ssn ACCEPT tcp -- anywhere anywhere tcp dpt:microsoft-ds ACCEPT tcp -- anywhere anywhere tcp dpts:24009:24108 ACCEPT tcp -- anywhere anywhere tcp dpts:49152:49251 ACCEPT tcp -- anywhere anywhere tcp dpts:49217:49316 ACCEPT tcp -- anywhere anywhere tcp dpt:zabbix-agent /* Zabbix agent */ REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT) target prot opt source destination REJECT all -- anywhere anywhere PHYSDEV match ! --physdev-is-bridged reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT) target prot opt source destination
And in the glusterd.log has nothing else but:
[2019-01-25 11:02:32.617694] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6 [2019-01-25 11:02:32.623892] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9 [2019-01-25 11:03:31.847006] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req [2019-01-25 11:04:33.023685] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2 [2019-01-25 11:04:33.030549] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8 [2019-01-25 11:04:33.037024] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3 [2019-01-25 11:04:33.043442] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4 [2019-01-25 11:04:33.049494] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5 [2019-01-25 11:04:33.056049] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6 [2019-01-25 11:04:33.062081] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9 [2019-01-25 11:05:47.710098] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req [2019-01-25 11:06:33.481200] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2 [2019-01-25 11:06:33.488137] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8 [2019-01-25 11:06:33.494492] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3 [2019-01-25 11:06:33.501053] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4 [2019-01-25 11:06:33.507302] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5 [2019-01-25 11:06:33.513299] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6 [2019-01-25 11:06:33.519269] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9 [2019-01-25 11:08:33.940033] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2 [2019-01-25 11:08:33.947011] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8 [2019-01-25 11:08:33.953445] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3 [2019-01-25 11:08:33.959618] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4 [2019-01-25 11:08:33.965659] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5 [2019-01-25 11:08:33.971609] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6 [2019-01-25 11:08:33.977763] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9 [2019-01-25 11:10:58.656508] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req [2019-01-25 11:11:05.746810] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4 [2019-01-25 11:11:08.153573] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5 [2019-01-25 11:11:11.108019] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6 [2019-01-25 11:11:13.456055] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2 [2019-01-25 11:11:13.584535] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2 [2019-01-25 11:11:16.046244] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3 [2019-01-25 11:11:16.172035] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3 [2019-01-25 11:11:18.370339] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8 [2019-01-25 11:11:18.504515] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8 [2019-01-25 11:11:20.710031] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9 [2019-01-25 11:11:05.886182] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4 [2019-01-25 11:11:08.284806] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5 [2019-01-25 11:11:11.244245] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6 [2019-01-25 11:11:20.835972] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9 [2019-01-25 11:13:33.238060] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req [2019-01-25 11:15:36.929536] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2 [2019-01-25 11:15:36.937647] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8 [2019-01-25 11:15:36.943936] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3 [2019-01-25 11:15:36.950206] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4 [2019-01-25 11:15:36.956460] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5 [2019-01-25 11:15:36.962459] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6 [2019-01-25 11:15:36.968594] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9 [2019-01-25 11:21:14.123760] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req [2019-01-25 11:22:38.575269] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2 [2019-01-25 11:22:38.582011] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8 [2019-01-25 11:22:38.588583] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
Met vriendelijke groet, With kind regards,
Jorick Astrego * Netbulae Virtualization Experts * ------------------------------------------------------------------------ Tel: 053 20 30 270 info@netbulae.eu Staalsteden 4-3A KvK 08198180 Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW NL821234584B01
------------------------------------------------------------------------