Hi,
I will probably be able to look into hanged supervdsmd issue only next week.Regards, Marcin
Hi,
I know people are busy but do we have any progress on this, I removed the OVS cluster and recreated it with bridge networking.
But I'm still stuck on the "failed to fetch volume file" and am unable to activate the hosts to continue testing...
Anything else I can try to debug? The storage network is ip based without dns.
Regards, Jorick
On 1/25/19 3:53 PM, Jorick Astrego wrote:
On 1/25/19 3:26 PM, Sahina Bose wrote:
On Fri, Jan 25, 2019 at 7:36 PM Jorick Astrego <jorick@netbulae.eu> wrote:
Oh duh, sorry forgot to check :-&
The mount failure is probably related to glusterfs. There are glusterfs logs on the host thatcan give more info on this error.
"failed to fetch volume file"
[2019-01-25 14:02:45.067440] I [glusterfsd-mgmt.c:2424:mgmt_rpc_notify] 0-glusterfsd-mgmt: disconnected from remote-host: *.*.*.*
Are you able to access this host, and the gluster ports open?Anything in the glusterd.log of the gluster server?
Adding Sanju to help
From the host:
Telnet *.*.*.14 24007
Trying *.*.*.14...
Connected to *.*.*.14.
Escape character is '^]'.
glusterd.log on *.*.*.14:
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere tcp dpt:54321
ACCEPT tcp -- anywhere anywhere tcp dpt:54322
ACCEPT tcp -- anywhere anywhere tcp dpt:sunrpc
ACCEPT udp -- anywhere anywhere udp dpt:sunrpc
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh
ACCEPT udp -- anywhere anywhere udp dpt:snmp
ACCEPT tcp -- anywhere anywhere tcp dpt:websm
ACCEPT tcp -- anywhere anywhere tcp dpt:24007
ACCEPT tcp -- anywhere anywhere tcp dpt:webcache
ACCEPT tcp -- anywhere anywhere tcp dpt:38465
ACCEPT tcp -- anywhere anywhere tcp dpt:38466
ACCEPT tcp -- anywhere anywhere tcp dpt:38467
ACCEPT tcp -- anywhere anywhere tcp dpt:nfs
ACCEPT tcp -- anywhere anywhere tcp dpt:38469
ACCEPT tcp -- anywhere anywhere tcp dpt:5666
ACCEPT tcp -- anywhere anywhere tcp dpt:39543
ACCEPT tcp -- anywhere anywhere tcp dpt:55863
ACCEPT tcp -- anywhere anywhere tcp dpt:38468
ACCEPT udp -- anywhere anywhere udp dpt:963
ACCEPT tcp -- anywhere anywhere tcp dpt:965
ACCEPT tcp -- anywhere anywhere tcp dpt:ctdb
ACCEPT tcp -- anywhere anywhere tcp dpt:netbios-ssn
ACCEPT tcp -- anywhere anywhere tcp dpt:microsoft-ds
ACCEPT tcp -- anywhere anywhere tcp dpts:24009:24108
ACCEPT tcp -- anywhere anywhere tcp dpts:49152:49251
ACCEPT tcp -- anywhere anywhere tcp dpts:49217:49316
ACCEPT tcp -- anywhere anywhere tcp dpt:zabbix-agent /* Zabbix agent */
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- anywhere anywhere PHYSDEV match ! --physdev-is-bridged reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
And in the glusterd.log has nothing else but:
[2019-01-25 11:02:32.617694] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:02:32.623892] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:03:31.847006] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:04:33.023685] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:04:33.030549] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:04:33.037024] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:04:33.043442] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:04:33.049494] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:04:33.056049] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:04:33.062081] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:05:47.710098] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:06:33.481200] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:06:33.488137] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:06:33.494492] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:06:33.501053] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:06:33.507302] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:06:33.513299] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:06:33.519269] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:08:33.940033] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:08:33.947011] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:08:33.953445] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:08:33.959618] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:08:33.965659] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:08:33.971609] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:08:33.977763] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:10:58.656508] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:11:05.746810] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:11:08.153573] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:11:11.108019] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:11:13.456055] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:11:13.584535] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:11:16.046244] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:11:16.172035] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:11:18.370339] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:11:18.504515] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:11:20.710031] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:11:05.886182] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:11:08.284806] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:11:11.244245] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:11:20.835972] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:13:33.238060] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:15:36.929536] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:15:36.937647] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:15:36.943936] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
[2019-01-25 11:15:36.950206] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd4
[2019-01-25 11:15:36.956460] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd5
[2019-01-25 11:15:36.962459] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd6
[2019-01-25 11:15:36.968594] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd9
[2019-01-25 11:21:14.123760] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2019-01-25 11:22:38.575269] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume hdd2
[2019-01-25 11:22:38.582011] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume sdd8
[2019-01-25 11:22:38.588583] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume ssd3
Met vriendelijke groet, With kind regards,
Jorick Astrego
Netbulae Virtualization Experts
Tel: 053 20 30 270 info@netbulae.eu Staalsteden 4-3A KvK 08198180 Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW NL821234584B01