[ovirt-users] Replica2 stripe2 hang on write to VM disk
Sahina Bose
sabose at redhat.com
Wed May 27 05:54:10 UTC 2015
[+ gluster list]
On 05/25/2015 09:32 PM, paf1 at email.cz wrote:
> Hello,
> can anybody help me with hanging replica2 stripe2 datastore on 4
> nodes cluster ??
>
> oVirt - ovirt-engine-lib-3.5.2.1-1.el7.centos.noarch
> gluster - glusterfs-server-3.7.0-2.el7.x86_64
> VM - Centos 7.1
>
> If I use any bigger write to VM disk ( eg 2-5GB ) storage hosted
> virtual disk will hang = I/O error
> created by :
> gluster volume create 12KVM12SC4 replica 2 stripe 2
> 16.0.0.161:/STORAGES/SlowClass/p4/GFS1
> 16.0.0.162:/STORAGES/SlowClass/p4/GFS1
> 16.0.0.163:/STORAGES/SlowClass/p4/GFS1
> 16.0.0.164:/STORAGES/SlowClass/p4/GFS1
>
> rhev-data-center-mnt-glusterSD-localhost:_12KVM12SC4.log
> -----------------------------------------------------------------------------------
>
> [2015-05-25 14:47:24.205609] I [rpc-clnt.c:1807:rpc_clnt_reconfig]
> 0-12KVM12SC4-client-3: changing port to 49158 (from 0)
> [2015-05-25 14:47:24.210824] I
> [client-handshake.c:1405:select_server_supported_programs]
> 0-12KVM12SC4-client-3: Using Program GlusterFS 3.3, Num (1298437),
> Version (330)
> [2015-05-25 14:47:24.211204] I
> [client-handshake.c:1193:client_setvolume_cbk] 0-12KVM12SC4-client-3:
> Connected to 12KVM12SC4-client-3, attached to remote volume
> '/STORAGES/SlowClass/p4/GFS1'.
> [2015-05-25 14:47:24.211225] I
> [client-handshake.c:1203:client_setvolume_cbk] 0-12KVM12SC4-client-3:
> Server and Client lk-version numbers are not same, reopening the fds
> [2015-05-25 14:47:24.211275] I [MSGID: 108005]
> [afr-common.c:3880:afr_notify] 0-12KVM12SC4-replicate-1: Subvolume
> '12KVM12SC4-client-3' came back up; going online.
> [2015-05-25 14:47:24.216465] I [fuse-bridge.c:5077:fuse_graph_setup]
> 0-fuse: switched to graph 0
> [2015-05-25 14:47:24.216556] I
> [client-handshake.c:187:client_set_lk_version_cbk]
> 0-12KVM12SC4-client-3: Server lk version = 1
> [2015-05-25 14:47:24.216643] I [fuse-bridge.c:4007:fuse_init]
> 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22
> kernel 7.22
> [2015-05-25 14:47:24.217998] I
> [afr-common.c:1673:afr_local_discovery_cbk] 0-12KVM12SC4-replicate-0:
> selecting local read_child 12KVM12SC4-client-0
> [2015-05-25 14:47:29.737732] W [fuse-bridge.c:1080:fuse_setattr_cbk]
> 0-glusterfs-fuse: 40: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only
> file system)
> [2015-05-25 14:49:18.266212] E
> [client-handshake.c:1488:client_query_portmap_cbk]
> 0-12KVM12SC4-client-2: failed to get the port number for remote
> subvolume. Please run 'gluster volume status' on server to see if
> brick process is running.
> [2015-05-25 14:49:18.266274] I [client.c:2086:client_rpc_notify]
> 0-12KVM12SC4-client-2: disconnected from 12KVM12SC4-client-2. Client
> process will keep trying to connect to glusterd until brick's port is
> available
> [2015-05-25 14:49:19.346555] I [rpc-clnt.c:1807:rpc_clnt_reconfig]
> 0-12KVM12SC4-client-2: changing port to 49158 (from 0)
> [2015-05-25 14:49:19.351812] I
> [client-handshake.c:1405:select_server_supported_programs]
> 0-12KVM12SC4-client-2: Using Program GlusterFS 3.3, Num (1298437),
> Version (330)
> [2015-05-25 14:49:19.352169] I
> [client-handshake.c:1193:client_setvolume_cbk] 0-12KVM12SC4-client-2:
> Connected to 12KVM12SC4-client-2, attached to remote volume
> '/STORAGES/SlowClass/p4/GFS1'.
> [2015-05-25 14:49:19.352191] I
> [client-handshake.c:1203:client_setvolume_cbk] 0-12KVM12SC4-client-2:
> Server and Client lk-version numbers are not same, reopening the fds
> [2015-05-25 14:49:19.352242] I [MSGID: 108002]
> [afr-common.c:3959:afr_notify] 0-12KVM12SC4-replicate-1: Client-quorum
> is met
> [2015-05-25 14:49:19.352353] I
> [client-handshake.c:187:client_set_lk_version_cbk]
> 0-12KVM12SC4-client-2: Server lk version = 1
> [2015-05-25 14:49:27.843616] W [fuse-bridge.c:1263:fuse_err_cbk]
> 0-glusterfs-fuse: 151: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No
> data available)
> [2015-05-25 14:49:58.356900] W [fuse-bridge.c:1263:fuse_err_cbk]
> 0-glusterfs-fuse: 327: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No
> data available)
>
>
> # gluster volume status
> Status of volume: 12KVM12SC4
> Gluster process TCP Port RDMA Port
> Online Pid
> ------------------------------------------------------------------------------
>
> Brick 16.0.0.161:/STORAGES/SlowClass/p4/GFS
> 1 49173 0 Y 17678
> Brick 16.0.0.162:/STORAGES/SlowClass/p4/GFS
> 1 49158 0 Y 19184
> Brick 16.0.0.163:/STORAGES/SlowClass/p4/GFS
> 1 49158 0 Y 9784
> Brick 16.0.0.164:/STORAGES/SlowClass/p4/GFS
> 1 49158 0 Y 9327
> NFS Server on localhost 2049 0 Y 17697
> Self-heal Daemon on localhost N/A N/A Y 17708
> NFS Server on 16.0.0.162 2049 0 Y 19205
> Self-heal Daemon on 16.0.0.162 N/A N/A Y 19215
> NFS Server on 16.0.0.163 2049 0 Y 9806
> Self-heal Daemon on 16.0.0.163 N/A N/A Y 9813
> NFS Server on 16.0.0.164 2049 0 Y 9347
> Self-heal Daemon on 16.0.0.164 N/A N/A Y 9359
>
> Task Status of Volume 12KVM12SC4
> ------------------------------------------------------------------------------
>
> There are no active volume tasks
>
>
>
> any idea ??
> regs.
> Pavel
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
More information about the Users
mailing list