Here below all the logs around time at the different layers.
Let me know if you need another log file not yet considered.
[2020-04-07 13:27:28.721262] E [MSGID: 133010] [shard.c:2327:shard_common_lookup_shards_cbk] 0-vmstore-shard: Lookup on shard 523 failed. Base file gfid = d22530cf-2e50-4059-8924-0aafe38497b1 [No such file or directory]
[2020-04-07 13:27:28.721432] W [fuse-bridge.c:2918:fuse_writev_cbk] 0-glusterfs-fuse: 4435189: WRITE => -1 gfid=d22530cf-2e50-4059-8924-0aafe38497b1 fd=0x7f3c4c07ab38 (No such file or directory)
[2020-04-07 13:27:28.719391] W [MSGID: 113020] [posix-helpers.c:1051:posix_gfid_set] 0-vmstore-posix: setting GFID on /gluster_bricks/vmstore
/vmstore/.shard/d22530cf-2e50-4059-8924-0aafe38497b1.523 failed [File exists]
[2020-04-07 13:27:28.719978] E [MSGID: 113020] [posix-entry-ops.c:517:posix_mknod] 0-vmstore-posix: setting gfid on /gluster_bricks/vmstore/v
mstore/.shard/d22530cf-2e50-4059-8924-0aafe38497b1.523 failed [File exists]
Here below all the files checked.
- qemu.log of the vm
2020-04-07T12:30:29.954084Z qemu-kvm: -drive file=/rhev/data-center/mnt/glusterSD/ovirtst.mydomain.storage:_vmstore/81b97244-4b69-4d49-84c4-c822387adc6a/images/04ec5a8d-4ee6-4661-a832-0c094968aa5b/716fdc98-2d0e-44c3-b5fe-c0187cdad751,format=raw,if=none,id=drive-ua-04ec5a8d-4ee6-4661-a832-0c094968aa5b,serial=04ec5a8d-4ee6-4661-a832-0c094968aa5b,werror=stop,rerror=stop,cache=none,aio=threads: 'serial' is deprecated, please use the corresponding option of '-device' instead
Spice-Message: 14:30:29.963: setting TLS option 'CipherString' to 'kECDHE+FIPS:kDHE+FIPS:kRSA+FIPS:!eNULL:!aNULL' from /etc/pki/tls/spice.cnf configuration file
2020-04-07T12:30:29.976216Z qemu-kvm: warning: CPU(s) not present in any NUMA nodes: CPU 8 [socket-id: 2, core-id: 0, thread-id: 0], CPU 9 [socket-id: 2, core-id: 1, thread-id: 0], CPU 10 [socket-id: 2, core-id: 2, thread-id: 0], CPU 11 [socket-id: 2, core-id: 3, thread-id: 0], CPU 12 [socket-id: 3, core-id: 0, thread-id: 0], CPU 13 [socket-id: 3, core-id: 1, thread-id: 0], CPU 14 [socket-id: 3, core-id: 2, thread-id: 0], CPU 15 [socket-id: 3, core-id: 3, thread-id: 0], CPU 16 [socket-id: 4, core-id: 0, thread-id: 0], CPU 17 [socket-id: 4, core-id: 1, thread-id: 0], CPU 18 [socket-id: 4, core-id: 2, thread-id: 0], CPU 19 [socket-id: 4, core-id: 3, thread-id: 0], CPU 20 [socket-id: 5, core-id: 0, thread-id: 0], CPU 21 [socket-id: 5, core-id: 1, thread-id: 0], CPU 22 [socket-id: 5, core-id: 2, thread-id: 0], CPU 23 [socket-id: 5, core-id: 3, thread-id: 0], CPU 24 [socket-id: 6, core-id: 0, thread-id: 0], CPU 25 [socket-id: 6, core-id: 1, thread-id: 0], CPU 26 [socket-id: 6, core-id: 2, thread-id: 0], CPU 27 [socket-id: 6, core-id: 3, thread-id: 0], CPU 28 [socket-id: 7, core-id: 0, thread-id: 0], CPU 29 [socket-id: 7, core-id: 1, thread-id: 0], CPU 30 [socket-id: 7, core-id: 2, thread-id: 0], CPU 31 [socket-id: 7, core-id: 3, thread-id: 0], CPU 32 [socket-id: 8, core-id: 0, thread-id: 0], CPU 33 [socket-id: 8, core-id: 1, thread-id: 0], CPU 34 [socket-id: 8, core-id: 2, thread-id: 0], CPU 35 [socket-id: 8, core-id: 3, thread-id: 0], CPU 36 [socket-id: 9, core-id: 0, thread-id: 0], CPU 37 [socket-id: 9, core-id: 1, thread-id: 0], CPU 38 [socket-id: 9, core-id: 2, thread-id: 0], CPU 39 [socket-id: 9, core-id: 3, thread-id: 0], CPU 40 [socket-id: 10, core-id: 0, thread-id: 0], CPU 41 [socket-id: 10, core-id: 1, thread-id: 0], CPU 42 [socket-id: 10, core-id: 2, thread-id: 0], CPU 43 [socket-id: 10, core-id: 3, thread-id: 0], CPU 44 [socket-id: 11, core-id: 0, thread-id: 0], CPU 45 [socket-id: 11, core-id: 1, thread-id: 0], CPU 46 [socket-id: 11, core-id: 2, thread-id: 0], CPU 47 [socket-id: 11, core-id: 3, thread-id: 0], CPU 48 [socket-id: 12, core-id: 0, thread-id: 0], CPU 49 [socket-id: 12, core-id: 1, thread-id: 0], CPU 50 [socket-id: 12, core-id: 2, thread-id: 0], CPU 51 [socket-id: 12, core-id: 3, thread-id: 0], CPU 52 [socket-id: 13, core-id: 0, thread-id: 0], CPU 53 [socket-id: 13, core-id: 1, thread-id: 0], CPU 54 [socket-id: 13, core-id: 2, thread-id: 0], CPU 55 [socket-id: 13, core-id: 3, thread-id: 0], CPU 56 [socket-id: 14, core-id: 0, thread-id: 0], CPU 57 [socket-id: 14, core-id: 1, thread-id: 0], CPU 58 [socket-id: 14, core-id: 2, thread-id: 0], CPU 59 [socket-id: 14, core-id: 3, thread-id: 0], CPU 60 [socket-id: 15, core-id: 0, thread-id: 0], CPU 61 [socket-id: 15, core-id: 1, thread-id: 0], CPU 62 [socket-id: 15, core-id: 2, thread-id: 0], CPU 63 [socket-id: 15, core-id: 3, thread-id: 0]
2020-04-07T12:30:29.976247Z qemu-kvm: warning: All CPU(s) up to maxcpus should be described in NUMA config, ability to start up with partial NUMA mappings is obsoleted and will be removed in future
main_channel_link: add main channel client
main_channel_client_handle_pong: net test: latency 52.540000 ms, bitrate 6113597 bps (5.830380 Mbps) LOW BANDWIDTH
inputs_connect: inputs channel client create
red_qxl_set_cursor_peer:
inputs_channel_detach_tablet:
red_channel_client_disconnect: rcc=0x5626c717a9c0 (channel=0x5626c616a880 type=3 id=0)
red_channel_client_disconnect: rcc=0x5626c8351dd0 (channel=0x5626c7bd38d0 type=4 id=0)
red_channel_client_disconnect: rcc=0x5626c834c5b0 (channel=0x5626c616a230 type=2 id=0)
red_channel_client_disconnect: rcc=0x5626c719e9f0 (channel=0x5626c616a160 type=1 id=0)
main_channel_client_on_disconnect: rcc=0x5626c719e9f0
red_client_destroy: destroy client 0x5626c6195570 with #channels=4
red_qxl_disconnect_cursor_peer:
red_qxl_disconnect_display_peer:
inputs_channel_detach_tablet:
inputs_channel_detach_tablet:
main_channel_link: add main channel client
main_channel_client_handle_pong: net test: latency 126.308000 ms, bitrate 9060022 bps (8.640310 Mbps) LOW BANDWIDTH
red_qxl_set_cursor_peer:
inputs_connect: inputs channel client create
red_channel_client_disconnect: rcc=0x5626c717a9f0 (channel=0x5626c616a160 type=1 id=0)
main_channel_client_on_disconnect: rcc=0x5626c717a9f0
red_client_destroy: destroy client 0x5626c7ab6870 with #channels=4
red_channel_client_disconnect: rcc=0x5626c71529c0 (channel=0x5626c616a880 type=3 id=0)
red_qxl_disconnect_cursor_peer:
red_channel_client_disconnect: rcc=0x5626c7e59dd0 (channel=0x5626c7bd38d0 type=4 id=0)
red_qxl_disconnect_display_peer:
red_channel_client_disconnect: rcc=0x5626c7e5e5b0 (channel=0x5626c616a230 type=2 id=0)
[root@ovirt qemu]#
- /var/log/glusterfs/cli.log
[2020-04-07 13:25:39.305818] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:39.306091] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:39.306531] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:39.306758] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:39.307081] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:25:54.365843] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:25:54.448475] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:25:54.448600] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:25:54.449348] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:25:54.485288] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:25:54.566723] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:25:54.566792] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:25:54.567275] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:25:54.567305] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:25:54.603618] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:25:54.685718] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:25:54.685821] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:25:54.686537] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:54.686994] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:54.687289] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:54.687774] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:54.688061] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:25:54.688368] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:09.748782] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:09.831541] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:09.831610] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:09.832596] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:09.871269] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:09.955626] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:09.955726] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:09.956239] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:26:09.956284] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:09.993929] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:10.073634] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:10.073680] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:10.074454] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:10.074932] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:10.075241] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:10.075677] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:10.075920] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:10.076233] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:22.933923] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:23.016382] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:23.016486] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:23.020067] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:25.149347] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:25.229667] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:25.229716] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:25.230491] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:25.266853] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:25.348774] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:25.348815] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:25.349253] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:26:25.349284] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:25.385951] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:25.468521] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:25.468571] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:25.469286] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:25.469703] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:25.469965] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:25.470381] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:25.470595] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:25.470877] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:40.546233] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:40.629567] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:40.629667] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:40.630529] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:40.666769] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:40.746861] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:40.746934] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:40.747436] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:26:40.747474] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:40.789486] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:40.881750] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:40.881871] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:40.882615] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:40.883142] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:40.883532] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:40.884020] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:40.884283] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:40.884615] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:55.966643] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:56.049639] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:56.049735] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:56.050543] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:56.096892] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:56.182275] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:56.182335] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:56.182883] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:26:56.182918] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:26:56.221093] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:26:56.303302] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:26:56.303337] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:26:56.303996] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:56.304432] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:56.304706] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:56.305135] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:56.305359] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:26:56.305645] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:11.378699] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:11.457630] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:11.457697] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:11.458355] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:11.493643] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:11.571358] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:11.571360] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:11.572137] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:27:11.572167] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:11.608432] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:11.689353] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:11.689437] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:11.690117] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:11.690569] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:11.690910] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:11.691343] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:11.691577] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:11.691853] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:23.058796] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:23.141280] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:23.141326] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:23.144418] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:26.767415] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:26.846945] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:26.847030] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:26.847818] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:26.883554] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:26.962804] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:26.962905] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:26.963380] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:27:26.963415] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:27.000197] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:27.077981] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:27.078029] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:27.078729] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:27.079174] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:27.079448] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:27.079873] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:27.080105] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:27.080429] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:42.152150] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:42.233109] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:42.233149] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:42.233818] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:42.269022] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:42.347310] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:42.347347] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:42.347771] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:27:42.347798] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:42.383260] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:42.460650] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:42.460741] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:42.461348] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:42.461780] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:42.462085] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:42.462493] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:42.462711] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:42.462990] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:57.534379] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:57.614068] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:57.614115] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:57.614835] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:57.650436] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:57.729556] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:57.729652] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:57.730164] I [cli-rpc-ops.c:1297:gf_cli3_1_uuid_get_cbk] 0-cli: Received resp to uuid get
[2020-04-07 13:27:57.730204] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:27:57.768743] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:27:57.848062] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:27:57.848153] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:27:57.848830] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:57.849277] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:57.849554] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:57.849983] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:57.850217] I [cli-rpc-ops.c:867:gf_cli_get_volume_cbk] 0-cli: Received resp to get vol: 0
[2020-04-07 13:27:57.850522] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:28:12.934596] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
[2020-04-07 13:28:13.015126] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 0
[2020-04-07 13:28:13.015167] I [MSGID: 101190] [event-epoll.c:688:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2020-04-07 13:28:13.015948] I [input.c:31:cli_batch] 0-: Exiting with: 0
[2020-04-07 13:28:13.054894] I [cli.c:844:main] 0-cli: Started running /usr/sbin/gluster with version 6.8
- glusterd.log
[2020-04-07 13:25:22.884457] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume vmstore
The message "I [MSGID: 106487] [glusterd-handler.c:1516:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req" repeated 7 times between [2020-04-07 13:24:06.750320] and [2020-04-07 13:25:54.449004]
The message "I [MSGID: 106488] [glusterd-handler.c:1577:__glusterd_handle_cli_get_volume] 0-management: Received get vol req" repeated 39 times between [2020-04-07 13:24:06.989378] and [2020-04-07 13:25:54.688025]
[2020-04-07 13:26:09.832058] I [MSGID: 106487] [glusterd-handler.c:1516:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2020-04-07 13:26:10.074287] I [MSGID: 106488] [glusterd-handler.c:1577:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2020-04-07 13:26:23.017801] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume big
[2020-04-07 13:26:23.018753] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume data
[2020-04-07 13:26:23.019190] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2020-04-07 13:26:23.019628] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume vmstore
[2020-04-07 13:27:23.143376] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume data
[2020-04-07 13:27:23.143718] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2020-04-07 13:27:23.142636] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume big
[2020-04-07 13:27:23.144070] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume vmstore
The message "I [MSGID: 106487] [glusterd-handler.c:1516:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req" repeated 7 times between [2020-04-07 13:26:09.832058] and [2020-04-07 13:27:57.614518]
The message "I [MSGID: 106488] [glusterd-handler.c:1577:__glusterd_handle_cli_get_volume] 0-management: Received get vol req" repeated 39 times between [2020-04-07 13:26:10.074287] and [2020-04-07 13:27:57.850194]
[2020-04-07 13:28:13.015558] I [MSGID: 106487] [glusterd-handler.c:1516:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2020-04-07 13:28:13.256713] I [MSGID: 106488] [glusterd-handler.c:1577:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2020-04-07 13:28:23.264138] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume big
[2020-04-07 13:28:23.264884] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume data
[2020-04-07 13:28:23.265316] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2020-04-07 13:28:23.265644] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume vmstore
[2020-04-07 13:29:23.389341] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume data
[2020-04-07 13:29:23.389702] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume engine
[2020-04-07 13:29:23.388761] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume big
[2020-04-07 13:29:23.390058] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume vmstore
The message "I [MSGID: 106487] [glusterd-handler.c:1516:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req" repeated 6 times between [2020-04-07 13:28:13.015558] and [2020-04-07 13:29:45.291552]
The message "I [MSGID: 106488] [glusterd-handler.c:1577:__glusterd_handle_cli_get_volume] 0-management: Received get vol req" repeated 34 times between [2020-04-07 13:28:13.256713] and [2020-04-07 13:29:45.524677]
[2020-04-07 13:30:00.664512] I [MSGID: 106487] [glusterd-handler.c:1516:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2020-04-07 13:30:00.899102] I [MSGID: 106488] [glusterd-handler.c:1577:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2020-04-07 13:30:23.505881] I [MSGID: 106499] [glusterd-handler.c:4429:__glusterd_handle_status_volume] 0-management: Received status volume req for volume big
- events.log
[2020-04-07 15:21:30,978] INFO [utils - 66:log_event] - {'event': 'CLIENT_CONNECT', 'message': {'client_identifier': '
10.50.50.11:49135', 'server_identifier': '
10.50.50.11:49153', 'client_uid': 'CTX_ID:815a9519-d88e-402d-9473-19015cca1db8-GRAPH_ID:0-PID:131714-HOST:ovirt.mydomain.local-PC_NAME:engine-client-0-RECON_NO:-0', 'brick_path': '/gluster_bricks/engine/engine', 'subdir_mount': '(null)'}, 'nodeid': 'ea83cff6-4cc6-4399-a096-5892476721f0', 'ts': 1586265690}
[2020-04-07 15:21:30,980] INFO [utils - 66:log_event] - {'event': 'CLIENT_DISCONNECT', 'message': {'client_identifier': '
10.50.50.11:49135', 'server_identifier': '
10.50.50.11:49153', 'client_uid': 'CTX_ID:815a9519-d88e-402d-9473-19015cca1db8-GRAPH_ID:0-PID:131714-HOST:ovirt.mydomain.local-PC_NAME:engine-client-0-RECON_NO:-0', 'brick_path': '/gluster_bricks/engine/engine'}, 'nodeid': 'ea83cff6-4cc6-4399-a096-5892476721f0', 'ts': 1586265690}
[2020-04-07 15:36:33,598] INFO [utils - 66:log_event] - {'event': 'CLIENT_CONNECT', 'message': {'client_identifier': '
10.50.50.11:49144', 'server_identifier': '
10.50.50.11:49155', 'client_uid': 'CTX_ID:e1d09577-50d6-4679-b0e8-0fe554f9fd49-GRAPH_ID:0-PID:139160-HOST:ovirt.mydomain.local-PC_NAME:big-client-0-RECON_NO:-0', 'brick_path': '/gluster_bricks/big/big', 'subdir_mount': '(null)'}, 'nodeid': 'ea83cff6-4cc6-4399-a096-5892476721f0', 'ts': 1586266593}
[2020-04-07 15:36:33,599] INFO [utils - 66:log_event] - {'event': 'CLIENT_DISCONNECT', 'message': {'client_identifier': '
10.50.50.11:49144', 'server_identifier': '
10.50.50.11:49155', 'client_uid': 'CTX_ID:e1d09577-50d6-4679-b0e8-0fe554f9fd49-GRAPH_ID:0-PID:139160-HOST:ovirt.mydomain.local-PC_NAME:big-client-0-RECON_NO:-0', 'brick_path': '/gluster_bricks/big/big'}, 'nodeid': 'ea83cff6-4cc6-4399-a096-5892476721f0', 'ts': 1586266593}
- rhev-data-center-mnt-glusterSD-ovirtst.mydomain.storage:_vmstore.log
[2020-04-07 13:24:16.070237] W [fuse-bridge.c:2918:fuse_writev_cbk] 0-glusterfs-fuse: 4197133: WRITE => -1 gfid=aa68cd63-f3d3-47bd-a177-c7f48bef3c59 fd=0x7f3c5019f348 (Invalid argument)
[2020-04-07 13:24:16.074521] I [MSGID: 133022] [shard.c:3674:shard_delete_shards] 0-vmstore-shard: Deleted shards of gfid=aa68cd63-f3d3-47bd-a177-c7f48bef3c59 from backend
[2020-04-07 13:27:28.721262] E [MSGID: 133010] [shard.c:2327:shard_common_lookup_shards_cbk] 0-vmstore-shard: Lookup on shard 523 failed. Base file gfid = d22530cf-2e50-4059-8924-0aafe38497b1 [No such file or directory]
[2020-04-07 13:27:28.721432] W [fuse-bridge.c:2918:fuse_writev_cbk] 0-glusterfs-fuse: 4435189: WRITE => -1 gfid=d22530cf-2e50-4059-8924-0aafe38497b1 fd=0x7f3c4c07ab38 (No such file or directory)
[2020-04-07 13:29:16.232630] W [MSGID: 114031] [client-rpc-fops_v2.c:680:client4_0_writev_cbk] 0-vmstore-client-0: remote operation failed [Invalid argument]
[2020-04-07 13:29:16.232675] W [fuse-bridge.c:2918:fuse_writev_cbk] 0-glusterfs-fuse: 4496167: WRITE => -1 gfid=54bafbb5-edd6-42a0-aaba-a5295031be17 fd=0x7f3c48125838 (Invalid argument)
[2020-04-07 13:29:16.236352] I [MSGID: 133022] [shard.c:3674:shard_delete_shards] 0-vmstore-shard: Deleted shards of gfid=54bafbb5-edd6-42a0-aaba-a5295031be17 from backend
[2020-04-07 13:34:16.378326] W [MSGID: 114031] [client-rpc-fops_v2.c:680:client4_0_writev_cbk] 0-vmstore-client-0: remote operation failed [Invalid argument]
[2020-04-07 13:34:16.378461] W [fuse-bridge.c:2918:fuse_writev_cbk] 0-glusterfs-fuse: 4698415: WRITE => -1 gfid=6beaa642-e73e-40c7-8adc-64275528b44f fd=0x7f3c48138cc8 (Invalid argument)
[2020-04-07 13:34:16.382824] I [MSGID: 133022] [shard.c:3674:shard_delete_shards] 0-vmstore-shard: Deleted shards of gfid=6beaa642-e73e-40c7-8adc-64275528b44f from backend
[2020-04-07 13:39:16.546919] W [MSGID: 114031] [client-rpc-fops_v2.c:680:client4_0_writev_cbk] 0-vmstore-client-0: remote operation failed [Invalid argument]
[2020-04-07 13:39:16.547022] W [fuse-bridge.c:2918:fuse_writev_cbk] 0-glusterfs-fuse: 4884227: WRITE => -1 gfid=f32843b7-4131-4536-abe2-f6969aa455ae fd=0x7f3c4813fb58 (Invalid argument)
[2020-04-07 13:39:16.550630] I [MSGID: 133022] [shard.c:3674:shard_delete_shards] 0-vmstore-shard: Deleted shards of gfid=f32843b7-4131-4536-abe2-f6969aa455ae from backend
- gluster_bricks-vmstore-vmstore.log
[2020-04-07 13:21:28.128381] I [MSGID: 115036] [server.c:499:server_rpc_notify] 0-vmstore-server: disconnecting connection from CTX_ID:12a5f3
7d-94d3-4b22-aaf6-4ac5bb7567d6-GRAPH_ID:0-PID:131656-HOST:ovirt.mydomain.local-PC_NAME:vmstore-client-0-RECON_NO:-0
[2020-04-07 13:21:28.128565] I [MSGID: 101055] [client_t.c:436:gf_client_unref] 0-vmstore-server: Shutting down connection CTX_ID:12a5f37d-94
d3-4b22-aaf6-4ac5bb7567d6-GRAPH_ID:0-PID:131656-HOST:ovirt.mydomain.local-PC_NAME:vmstore-client-0-RECON_NO:-0
[2020-04-07 13:24:16.069408] E [MSGID: 113072] [posix-inode-fd-ops.c:1886:posix_writev] 0-vmstore-posix: write failed: offset 0, [Invalid arg
ument]
[2020-04-07 13:24:16.070018] E [MSGID: 115067] [server-rpc-fops_v2.c:1373:server4_writev_cbk] 0-vmstore-server: 3528796: WRITEV 18 (aa68cd63-
f3d3-47bd-a177-c7f48bef3c59), client: CTX_ID:20665547-59fd-4b4a-94fb-7fdfdca9f5a9-GRAPH_ID:0-PID:17102-HOST:ovirt.mydomain.local-PC_NAME:vmsto
re-client-0-RECON_NO:-0, error-xlator: vmstore-posix [Invalid argument]
[2020-04-07 13:27:28.719391] W [MSGID: 113020] [posix-helpers.c:1051:posix_gfid_set] 0-vmstore-posix: setting GFID on /gluster_bricks/vmstore
/vmstore/.shard/d22530cf-2e50-4059-8924-0aafe38497b1.523 failed [File exists]
[2020-04-07 13:27:28.719978] E [MSGID: 113020] [posix-entry-ops.c:517:posix_mknod] 0-vmstore-posix: setting gfid on /gluster_bricks/vmstore/v
mstore/.shard/d22530cf-2e50-4059-8924-0aafe38497b1.523 failed [File exists]
[2020-04-07 13:29:16.232263] E [MSGID: 113072] [posix-inode-fd-ops.c:1886:posix_writev] 0-vmstore-posix: write failed: offset 0, [Invalid argument]
[2020-04-07 13:29:16.232578] E [MSGID: 115067] [server-rpc-fops_v2.c:1373:server4_writev_cbk] 0-vmstore-server: 3777485: WRITEV 12 (54bafbb5-edd6-42a0-aaba-a5295031be17), client: CTX_ID:20665547-59fd-4b4a-94fb-7fdfdca9f5a9-GRAPH_ID:0-PID:17102-HOST:ovirt.mydomain.local-PC_NAME:vmstore-client-0-RECON_NO:-0, error-xlator: vmstore-posix [Invalid argument]
[2020-04-07 13:34:16.377622] E [MSGID: 113072] [posix-inode-fd-ops.c:1886:posix_writev] 0-vmstore-posix: write failed: offset 0, [Invalid argument]
- messages
Apr 7 15:25:26 ovirt gluster_bricks-big-big[39632]: [2020-04-07 13:25:26.789943] E [MSGID: 113072] [posix-inode-fd-ops.c:1886:posix_writev]
0-big-posix: write failed: offset 0, [Invalid argument]
Apr 7 15:25:26 ovirt gluster_bricks-big-big[39632]: [2020-04-07 13:25:26.790284] E [MSGID: 115067] [server-rpc-fops_v2.c:1373:server4_writev
_cbk] 0-big-server: 20535: WRITEV 1 (e965b2e9-500b-4520-a17e-d00b93a2e1bf), client: CTX_ID:2530ad70-1cec-43a2-bdf8-8c5a085fecb4-GRAPH_ID:0-PI
D:41137-HOST:ovirt.mydomain.local-PC_NAME:big-client-0-RECON_NO:-0, error-xlator: big-posix [Invalid argument]
Apr 7 15:26:16 ovirt gluster_bricks-engine-engine[10652]: [2020-04-07 13:26:16.183229] E [MSGID: 113072] [posix-inode-fd-ops.c:1886:posix_wr
itev] 0-engine-posix: write failed: offset 0, [Invalid argument]
Apr 7 15:26:16 ovirt gluster_bricks-engine-engine[10652]: [2020-04-07 13:26:16.183318] E [MSGID: 115067] [server-rpc-fops_v2.c:1373:server4_
writev_cbk] 0-engine-server: 359871: WRITEV 5 (cb0ea391-ed2f-4318-91d3-b8e18f6bdfb7), client: CTX_ID:08f296e3-beeb-4658-8e79-d9add812c3d7-GRA
PH_ID:0-PID:11400-HOST:ovirt.mydomain.local-PC_NAME:engine-client-0-RECON_NO:-0, error-xlator: engine-posix [Invalid argument]
Apr 7 15:27:28 ovirt gluster_bricks-vmstore-vmstore[10678]: [2020-04-07 13:27:28.719391] W [MSGID: 113020] [posix-helpers.c:1051:posix_gfid_set] 0-vmstore-posix: setting GFID on /gluster_bricks/vmstore/vmstore/.shard/d22530cf-2e50-4059-8924-0aafe38497b1.523 failed [File exists]
Apr 7 15:27:28 ovirt gluster_bricks-vmstore-vmstore[10678]: [2020-04-07 13:27:28.719978] E [MSGID: 113020] [posix-entry-ops.c:517:posix_mknod] 0-vmstore-posix: setting gfid on /gluster_bricks/vmstore/vmstore/.shard/d22530cf-2e50-4059-8924-0aafe38497b1.523 failed [File exists]
Apr 7 15:29:16 ovirt gluster_bricks-data-data[10609]: [2020-04-07 13:29:16.232149] E [MSGID: 113072] [posix-inode-fd-ops.c:1886:posix_writev] 0-data-posix: write failed: offset 0, [Invalid argument]