[ovirt-users] Replica2 stripe2 hang on write to VM disk

Sahina Bose sabose at redhat.com
Wed May 27 01:54:10 EDT 2015


[+ gluster list]

On 05/25/2015 09:32 PM, paf1 at email.cz wrote:
> Hello,
> can anybody help me with hanging  replica2 stripe2 datastore on 4 
> nodes cluster ??
>
> oVirt -  ovirt-engine-lib-3.5.2.1-1.el7.centos.noarch
> gluster - glusterfs-server-3.7.0-2.el7.x86_64
> VM - Centos 7.1
>
> If I use any bigger write to VM disk ( eg 2-5GB ) storage hosted 
> virtual disk will hang = I/O error
> created by :
> gluster volume create 12KVM12SC4 replica 2 stripe 2 
> 16.0.0.161:/STORAGES/SlowClass/p4/GFS1 
> 16.0.0.162:/STORAGES/SlowClass/p4/GFS1 
> 16.0.0.163:/STORAGES/SlowClass/p4/GFS1 
> 16.0.0.164:/STORAGES/SlowClass/p4/GFS1
>
> rhev-data-center-mnt-glusterSD-localhost:_12KVM12SC4.log
> ----------------------------------------------------------------------------------- 
>
> [2015-05-25 14:47:24.205609] I [rpc-clnt.c:1807:rpc_clnt_reconfig] 
> 0-12KVM12SC4-client-3: changing port to 49158 (from 0)
> [2015-05-25 14:47:24.210824] I 
> [client-handshake.c:1405:select_server_supported_programs] 
> 0-12KVM12SC4-client-3: Using Program GlusterFS 3.3, Num (1298437), 
> Version (330)
> [2015-05-25 14:47:24.211204] I 
> [client-handshake.c:1193:client_setvolume_cbk] 0-12KVM12SC4-client-3: 
> Connected to 12KVM12SC4-client-3, attached to remote volume 
> '/STORAGES/SlowClass/p4/GFS1'.
> [2015-05-25 14:47:24.211225] I 
> [client-handshake.c:1203:client_setvolume_cbk] 0-12KVM12SC4-client-3: 
> Server and Client lk-version numbers are not same, reopening the fds
> [2015-05-25 14:47:24.211275] I [MSGID: 108005] 
> [afr-common.c:3880:afr_notify] 0-12KVM12SC4-replicate-1: Subvolume 
> '12KVM12SC4-client-3' came back up; going online.
> [2015-05-25 14:47:24.216465] I [fuse-bridge.c:5077:fuse_graph_setup] 
> 0-fuse: switched to graph 0
> [2015-05-25 14:47:24.216556] I 
> [client-handshake.c:187:client_set_lk_version_cbk] 
> 0-12KVM12SC4-client-3: Server lk version = 1
> [2015-05-25 14:47:24.216643] I [fuse-bridge.c:4007:fuse_init] 
> 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 
> kernel 7.22
> [2015-05-25 14:47:24.217998] I 
> [afr-common.c:1673:afr_local_discovery_cbk] 0-12KVM12SC4-replicate-0: 
> selecting local read_child 12KVM12SC4-client-0
> [2015-05-25 14:47:29.737732] W [fuse-bridge.c:1080:fuse_setattr_cbk] 
> 0-glusterfs-fuse: 40: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only 
> file system)
> [2015-05-25 14:49:18.266212] E 
> [client-handshake.c:1488:client_query_portmap_cbk] 
> 0-12KVM12SC4-client-2: failed to get the port number for remote 
> subvolume. Please run 'gluster volume status' on server to see if 
> brick process is running.
> [2015-05-25 14:49:18.266274] I [client.c:2086:client_rpc_notify] 
> 0-12KVM12SC4-client-2: disconnected from 12KVM12SC4-client-2. Client 
> process will keep trying to connect to glusterd until brick's port is 
> available
> [2015-05-25 14:49:19.346555] I [rpc-clnt.c:1807:rpc_clnt_reconfig] 
> 0-12KVM12SC4-client-2: changing port to 49158 (from 0)
> [2015-05-25 14:49:19.351812] I 
> [client-handshake.c:1405:select_server_supported_programs] 
> 0-12KVM12SC4-client-2: Using Program GlusterFS 3.3, Num (1298437), 
> Version (330)
> [2015-05-25 14:49:19.352169] I 
> [client-handshake.c:1193:client_setvolume_cbk] 0-12KVM12SC4-client-2: 
> Connected to 12KVM12SC4-client-2, attached to remote volume 
> '/STORAGES/SlowClass/p4/GFS1'.
> [2015-05-25 14:49:19.352191] I 
> [client-handshake.c:1203:client_setvolume_cbk] 0-12KVM12SC4-client-2: 
> Server and Client lk-version numbers are not same, reopening the fds
> [2015-05-25 14:49:19.352242] I [MSGID: 108002] 
> [afr-common.c:3959:afr_notify] 0-12KVM12SC4-replicate-1: Client-quorum 
> is met
> [2015-05-25 14:49:19.352353] I 
> [client-handshake.c:187:client_set_lk_version_cbk] 
> 0-12KVM12SC4-client-2: Server lk version = 1
> [2015-05-25 14:49:27.843616] W [fuse-bridge.c:1263:fuse_err_cbk] 
> 0-glusterfs-fuse: 151: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No 
> data available)
> [2015-05-25 14:49:58.356900] W [fuse-bridge.c:1263:fuse_err_cbk] 
> 0-glusterfs-fuse: 327: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No 
> data available)
>
>
> # gluster volume status
> Status of volume: 12KVM12SC4
> Gluster process                             TCP Port  RDMA Port 
> Online  Pid
> ------------------------------------------------------------------------------ 
>
> Brick 16.0.0.161:/STORAGES/SlowClass/p4/GFS
> 1                                           49173 0 Y       17678
> Brick 16.0.0.162:/STORAGES/SlowClass/p4/GFS
> 1                                           49158 0 Y       19184
> Brick 16.0.0.163:/STORAGES/SlowClass/p4/GFS
> 1                                           49158 0 Y       9784
> Brick 16.0.0.164:/STORAGES/SlowClass/p4/GFS
> 1                                           49158 0 Y       9327
> NFS Server on localhost                     2049 0 Y       17697
> Self-heal Daemon on localhost               N/A N/A        Y 17708
> NFS Server on 16.0.0.162                    2049 0 Y       19205
> Self-heal Daemon on 16.0.0.162              N/A N/A        Y 19215
> NFS Server on 16.0.0.163                    2049 0 Y       9806
> Self-heal Daemon on 16.0.0.163              N/A N/A        Y 9813
> NFS Server on 16.0.0.164                    2049 0 Y       9347
> Self-heal Daemon on 16.0.0.164              N/A N/A        Y 9359
>
> Task Status of Volume 12KVM12SC4
> ------------------------------------------------------------------------------ 
>
> There are no active volume tasks
>
>
>
> any idea ??
> regs.
> Pavel
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users



More information about the Users mailing list