On Sat, Mar 28, 2020 at 8:26 PM Nir Soffer <nsoffer@redhat.com> wrote:


Gluster disks are thin (raw-sparse) by default just like any other
file based storage.

If this theory was correct, this would fail consistently on gluster:

1. create raw sparse image

    truncate -s 100g /rhev/data-center/mnt/glusterSD/server:_path/test

2. Fill image quickly with data

    dd if=/dev/zero bs=1M | tr "\0" "U" | dd
of=/rhev/data-center/mnt/glusterSD/server:_path/test bs=1M count=12800
iflag=fullblock oflag=direct conv=notrunc

According to your theory gluster will fail to allocate shards fast
enough and fail the I/O.

Nir

Coming back here as I got some other occurrences of the same error and I'm going to try to enable some logging and see where it is the origin.

In the mean time I confirmed what written by Nir above. No error on gluster side with the command suggested:

[root@ovirt libvirt]# time dd if=/dev/zero bs=1M | tr "\0" "U" | dd of="/rhev/data-center/mnt/glusterSD/ovirtst.mydomain.storage:_vmstore/test" bs=1M count=12800 iflag=fullblock oflag=direct conv=notrunc
12800+0 records in
12800+0 records out
13421772800 bytes (13 GB) copied, 37.8056 s, 355 MB/s

real 0m37.812s
user 0m10.582s
sys 0m24.072s
[root@ovirt libvirt]#
[root@ovirt libvirt]# echo $?
0
[root@ovirt libvirt]#

During the copy a "vmstat 3" produced indeed about 350 MiB/s:

[root@ovirt ~]# vmstat 3
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 8  0      0 1274344448 329220 5752988    0    0     8   142    0    0  4  2 95  0  0
12  0      0 1274341504 329220 5753008    0    0  1930 30079 76558 186828  7  3 90  0  0
 3  0      0 1274340480 329220 5753028    0    0   917 15786 68796 178179  6  3 91  0  0
 2  0      0 1274336128 329220 5753048    0    0    61 14219 69035 172374  6  3 90  0  0
17  2      0 1274334976 329220 5753088    0    0  1931 14814 74888 178985  7  3 89  0  0
 7  0      0 1274317952 329220 5753112    0    0  1556 144318 72745 257632  7  3 89  0  0
11  1      0 1274315136 329220 5753112    0    0    91 350458 80783 410853  8  4 87  1  0
12  0      0 1274310272 329220 5753148    0    0   965 378040 83638 419936  6  4 89  1  0
10  1      0 1274308352 329220 5753188    0    0  1470 379597 82909 429217  7  4 88  1  0
 7  0      0 1274318592 329220 5753192    0    0   608 368145 85389 417066  9  5 86  1  0
 9  1      0 1274323584 329220 5753208    0    0   460 370369 81002 426601  7  5 88  1  0
12  2      0 1274323456 329220 5753232    0    0  2655 369143 79131 415323  7  4 88  1  0
10  2      0 1274306560 329220 5753252    0    0   625 367324 82076 419014  8  4 87  1  0
15  0      0 1274276480 329220 5753272    0    0    99 355281 81662 409003  8  4 87  1  0
12  2      0 1274259072 329220 5753292    0    0   855 358916 82942 416828  7  4 88  1  0
 5  0      0 1274257024 329220 5753316    0    0  1566 374178 81489 415783  7  4 88  1  0
 7  2      0 1274259968 329220 5753324    0    0   527 344149 82577 418192  7  4 88  1  0
 7  0      0 1274254080 329220 5753364    0    0  1596 341075 82839 414259  6  4 89  1  0
 7  1      0 1274267776 329220 5753404    0    0  1431 97078 73923 242802  6  3 90  0  0

Tried 4 times without any error and consistent timing of about 36-37 seconds

Go to next tests
Gianluca