6:41 p.m.
Hello,
This is weird. Today I was able to do a live migration. Has happened to me before ,
sometimes I can do a disk migration some other times don't, without changing anything.
But with the VM down it fails.
Version 4.3.10.4-1.el7
# gluster --version
glusterfs 6.10
LIVE MIGRATION gluster brick log:
[2020-11-25 09:40:43.097977] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:40:43.097988] D [MSGID: 0] [posix-metadata.c:131:posix_fetch_mdata_xattr]
0-data-posix: No such attribute:trusted.glusterfs.mdata for file null gfid:
fafaa24f-7174-4fb6-b9ac-6d10974598ed
[2020-11-25 09:40:43.097995] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[2020-11-25 09:40:43.098011] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
[2020-11-25 09:40:43.098014] D [socket.c:720:__socket_rwv] 0-tcp.data-server: would have
passed zero length to read/write
[2020-11-25 09:40:43.098003] D [MSGID: 0] [posix-metadata.c:131:posix_fetch_mdata_xattr]
0-data-posix: No such attribute:trusted.glusterfs.mdata for file null gfid:
fafaa24f-7174-4fb6-b9ac-6d10974598ed
[2020-11-25 09:40:43.098010] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: READ scheduled as slow priority fop
[2020-11-25 09:40:43.098045] D [MSGID: 0] [posix-metadata.c:131:posix_fetch_mdata_xattr]
0-data-posix: No such attribute:trusted.glusterfs.mdata for file null gfid:
fafaa24f-7174-4fb6-b9ac-6d10974598ed
[2020-11-25 09:40:43.098047] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:40:43.098046] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:40:53.843509] D [socket.c:720:__socket_rwv] 0-tcp.data-server: would have
passed zero length to read/write
[2020-11-25 09:40:53.843517] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 4
[2020-11-25 09:40:53.843520] D [socket.c:720:__socket_rwv] 0-tcp.data-server: would have
passed zero length to read/write
[2020-11-25 09:40:53.843530] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[2020-11-25 09:40:53.843530] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
[2020-11-25 09:40:53.843545] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
The message "D [MSGID: 0] [posix-metadata.c:131:posix_fetch_mdata_xattr]
0-data-posix: No such attribute:trusted.glusterfs.mdata for file null gfid:
d6a457b3-d600-477c-a123-2f939b96f6fc" repeated 3 times between [2020-11-25
09:40:53.843456] and [2020-11-25 09:40:53.843496]
[2020-11-25 09:40:53.843529] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: READ scheduled as slow priority fop
[2020-11-25 09:40:53.843578] D [MSGID: 0] [posix-metadata.c:131:posix_fetch_mdata_xattr]
0-data-posix: No such attribute:trusted.glusterfs.mdata for file null gfid:
d6a457b3-d600-477c-a123-2f939b96f6fc
[2020-11-25 09:40:53.843586] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:41:05.902502] D [socket.c:720:__socket_rwv] 0-tcp.data-server: would have
passed zero length to read/write
[2020-11-25 09:41:05.902521] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:05.902527] D [socket.c:720:__socket_rwv] 0-tcp.data-server: would have
passed zero length to read/write
[2020-11-25 09:41:05.902545] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:05.902590] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:05.902600] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
The message "D [MSGID: 0] [posix-metadata.c:131:posix_fetch_mdata_xattr]
0-data-posix: No such attribute:trusted.glusterfs.mdata for file null gfid:
9fca07e6-5d59-4c99-8129-bca123f0d876" repeated 2 times between [2020-11-25
09:41:05.902476] and [2020-11-25 09:41:05.902518]
[2020-11-25 09:41:05.902600] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: READ scheduled as slow priority fop
[2020-11-25 09:41:05.902638] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:4dc35afd-d6a1-4aaa-9d25-6254f6a3df6d-GRAPH_ID:0-PID:4191-HOST:node3.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[2020-11-25 09:41:05.902639] D [MSGID: 0] [posix-metadata.c:131:posix_fetch_mdata_xattr]
0-data-posix: No such attribute:trusted.glusterfs.mdata for file null gfid:
9fca07e6-5d59-4c99-8129-bca123f0d876
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:41:21.536567] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 7
[2020-11-25 09:41:21.536598] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[2020-11-25 09:41:21.536646] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 6
[2020-11-25 09:41:21.536708] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 5
[2020-11-25 09:41:21.536763] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:21.537487] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:21.537516] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
The message "D [MSGID: 0] [io-threads.c:376:iot_schedule] 0-data-io-threads: FSYNC
scheduled as slow priority fop" repeated 8 times between [2020-11-25 09:41:21.533386]
and [2020-11-25 09:41:21.533940]
[2020-11-25 09:41:21.537516] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: WRITE scheduled as slow priority fop
[2020-11-25 09:41:21.537637] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x555ab) [0x7f2324ef15ab]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:41:33.722393] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x555ab) [0x7f2324ef15ab]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:33.722927] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:33.722956] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[2020-11-25 09:41:33.722961] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
The message "D [MSGID: 0] [io-threads.c:376:iot_schedule] 0-data-io-threads: WRITE
scheduled as slow priority fop" repeated 13 times between [2020-11-25
09:41:33.216986] and [2020-11-25 09:41:33.722236]
[2020-11-25 09:41:33.722959] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: FSYNC scheduled as slow priority fop
[2020-11-25 09:41:33.723067] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 4
[2020-11-25 09:41:33.723166] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[2020-11-25 09:41:33.723445] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:33.723984] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:41:34.968525] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:34.968544] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: FLUSH scheduled as normal priority fop
[2020-11-25 09:41:34.968642] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x4fbad) [0x7f2324eebbad]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:35.742535] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:35.742574] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: FSTAT scheduled as fast priority fop
[2020-11-25 09:41:35.742668] D [MSGID: 101016] [glusterfs3.h:781:dict_to_xdr] 0-dict: key
'trusted.glusterfs.shard.file-size' would not be sent on wire in the future
[Invalid argument]
[2020-11-25 09:41:35.742729] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x6311e) [0x7f2324eff11e]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:36.608888] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:36.608925] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: WRITE scheduled as slow priority fop
[2020-11-25 09:41:36.609122] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x555ab) [0x7f2324ef15ab]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:41:40.208226] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x58c3b) [0x7f2324ef4c3b]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:40.208932] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:40.209125] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x58c3b) [0x7f2324ef4c3b]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:40.209653] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:40.209687] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
The message "D [MSGID: 101016] [glusterfs3.h:781:dict_to_xdr] 0-dict: key
'trusted.glusterfs.mdata' would not be sent on wire in the future [Invalid
argument]" repeated 2 times between [2020-11-25 09:41:39.550205] and [2020-11-25
09:41:40.209075]
[2020-11-25 09:41:40.209686] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: STATFS scheduled as fast priority fop
[2020-11-25 09:41:40.209820] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x5c0f3) [0x7f2324ef80f3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:40.211481] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:40.211628] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x5c0f3) [0x7f2324ef80f3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:7606cd9d-c765-49cc-95a4-cabc3bcffdce-GRAPH_ID:0-PID:12047-HOST:node5.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:41:44.702715] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:44.702738] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: OPEN scheduled as fast priority fop
[2020-11-25 09:41:44.702869] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x53319) [0x7f2324eef319]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:44.703204] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:44.703331] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:41:44.704059] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:41:44.704079] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
[2020-11-25 09:41:44.703220] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: READ scheduled as slow priority fop
[2020-11-25 09:41:44.704078] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: FLUSH scheduled as normal priority fop
[2020-11-25 09:41:44.704171] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x4fbad) [0x7f2324eebbad]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:42:14.711157] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:42:14.711316] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x53319) [0x7f2324eef319]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:42:14.711859] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:42:14.711875] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
The message "D [MSGID: 0] [io-threads.c:376:iot_schedule] 0-data-io-threads: OPEN
scheduled as fast priority fop" repeated 2 times between [2020-11-25 09:42:14.214506]
and [2020-11-25 09:42:14.711180]
[2020-11-25 09:42:14.711874] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: READ scheduled as slow priority fop
[2020-11-25 09:42:14.712000] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x61cd3) [0x7f2324efdcd3]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:42:14.712651] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:42:14.712670] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: FLUSH scheduled as normal priority fop
[2020-11-25 09:42:14.712769] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x4fbad) [0x7f2324eebbad]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:3f9791be-32b5-42fb-8292-c180eab1ddd6-GRAPH_ID:0-PID:1265-HOST:node2.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[root@gfs3 bricks]# tail home-brick1.log
[2020-11-25 09:42:56.482283] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:42:56.482323] D [logging.c:2006:_gf_msg_internal] 0-logging-infra: Buffer
overflow of a buffer whose size limit is 5. About to flush least recently used log message
to disk
The message "D [MSGID: 0] [io-threads.c:376:iot_schedule] 0-data-io-threads: FSYNC
scheduled as slow priority fop" repeated 2 times between [2020-11-25 09:42:56.463753]
and [2020-11-25 09:42:56.464000]
[2020-11-25 09:42:56.482320] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: WRITE scheduled as slow priority fop
[2020-11-25 09:42:56.482497] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x555ab) [0x7f2324ef15ab]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
[2020-11-25 09:42:56.483120] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:42:56.483138] D [client_t.c:324:gf_client_ref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x34a85) [0x7f2324ed0a85]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x1171d) [0x7f2324ead71d]
-->/lib64/libglusterfs.so.0(gf_client_ref+0x6e) [0x7f233a64438e] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 3
[2020-11-25 09:42:56.483140] D [MSGID: 0] [io-threads.c:376:iot_schedule]
0-data-io-threads: FSYNC scheduled as slow priority fop
[2020-11-25 09:42:56.483274] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 2
[2020-11-25 09:42:56.483683] D [client_t.c:433:gf_client_unref]
(-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0x550eb) [0x7f2324ef10eb]
-->/usr/lib64/glusterfs/6.10/xlator/protocol/server.so(+0xaecb) [0x7f2324ea6ecb]
-->/lib64/libglusterfs.so.0(gf_client_unref+0x7b) [0x7f233a6444db] ) 0-client_t:
CTX_ID:831d9da7-0563-40d2-bfc1-964284a8b556-GRAPH_ID:0-PID:4201-HOST:node4.acloud.pt-PC_NAME:data-client-0-RECON_NO:-0:
ref-count 1
VM down brick log:
You can find the log's here: https://drive.acloud.pt/s/F5xpXU3sSl3T09Z
De: "Strahil Nikolov" <hunter86_bg(a)yahoo.com>
Para: suporte(a)logicworks.pt
Cc: users(a)ovirt.org
Enviadas: Segunda-feira, 23 De Novembro de 2020 5:45:37
Assunto: Re: [ovirt-users] Re: Unable to move or copy disks
No, but keep an eye on you "/var/log" as debug is providing a lot of info.
Usually when you got a failure to move the disk, you can disable and check the logs.
Best Regards,
Strahil Nikolov
В неделя, 22 ноември 2020 г., 21:12:26 Гринуич+2, <suporte(a)logicworks.pt> написа:
Do I need to restart gluster after enable debug level?
gluster volume set data2 diagnostics.brick-log-level DEBUG
________________________________
De: "Strahil Nikolov" <hunter86_bg(a)yahoo.com>
Para: suporte(a)logicworks.pt
Cc: users(a)ovirt.org
Enviadas: Sábado, 21 De Novembro de 2020 19:42:44
Assunto: Re: [ovirt-users] Re: Unable to move or copy disks
You still haven't provided debug logs from the Gluster Bricks.
There will be always a chance that a bug hits you ... no matter OS and tech. What matters
- is how you debug and overcome that bug.
Check the gluster brick debug logs and you can test if the issue happens with an older
version.
Also, consider providing oVirt version, Gluster version and some details about your setup
- otherwise helping you is almost impossible.
Best Regards,
Strahil Nikolov
В събота, 21 ноември 2020 г., 18:16:13 Гринуич+2, suporte(a)logicworks.pt
<suporte(a)logicworks.pt> написа:
With older gluster verison this does not happens.
Always get this error: VDSM NODE3 command HSMGetAllTasksStatusesVDS failed: low level
Image copy failed: ()
and in the vdsm.log:
ERROR (tasks/7) [storage.Image] Copy image error:
image=6939cc5d-dbca-488d-ab7a-c8b8d39c3656, src
domain=70e33d55-f1be-4826-b85a-9650c76c8db8, dst
domain=0b80eac1-8bbb-4634-9098-4155602c7b38 (image:485) ERROR (tasks/7)
[storage.TaskManager.Task] (Task='fbf02f6b-c107-4c1e-a9c7-b13261bf99e0')
Unexpected error (task:875)
For smaller installations, without the need of storage HA, maybe it's better to use
NFS? Is more stable?
Regards
José
________________________________
De: "Strahil Nikolov" <hunter86_bg(a)yahoo.com>
Para: suporte(a)logicworks.pt
Cc: users(a)ovirt.org
Enviadas: Sexta-feira, 20 De Novembro de 2020 21:55:28
Assunto: Re: [ovirt-users] Unable to move or copy disks
I can recommend you to:
- enable debug level of gluster's bricks
- try to reproduce the issue
I had similar issue with gluster v6.6 and above.
Best Regards,
Strahil Nikolov
В петък, 20 ноември 2020 г., 23:35:14 Гринуич+2, <suporte(a)logicworks.pt> написа:
I tried to move the VM disk with the VM up
I also tried to move or copy the disk with the VM down, and get the same error.
Strange is, I have a gluster storage domain, with an older gluster version, that worked. I
was able to move the disk to this older gluster and also I was able to copy the disk from
to this older gluster storage
What should I do with this VM? Should I shut it down and try delete the snapshot?
Regards
José
________________________________
De: "Strahil Nikolov" <hunter86_bg(a)yahoo.com>
Para: users(a)ovirt.org, suporte(a)logicworks.pt
Enviadas: Sexta-feira, 20 De Novembro de 2020 21:05:19
Assunto: Re: [ovirt-users] Unable to move or copy disks
Can you try a live migration ?
I got a similar case and the live migration somehow triggered a fix.
Best Regards,
Strahil Nikolov
В петък, 20 ноември 2020 г., 21:04:13 Гринуич+2, <suporte(a)logicworks.pt> написа:
Hi,
I was trying to move a disk between glusters domain storage without success.
# gluster --version glusterfs 6.10
Now have a VM with this message:
The VM has snapshot(s) with disk(s) in illegal status. Please don't shutdown the VM
before successfully retrying the snapshot delete
oVirt verison is
Version 4.3.10.4-1.el7
I cannot delete the snapshot. What should I do?
hanks
--
________________________________
Jose Ferradeira
http://www.logicworks.pt
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/EAYBO6KEKQS...
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/VIVNXE4C3MZ...