On Sat, Mar 28, 2020 at 8:26 PM Nir Soffer <nsoffer@redhat.com> wrote:
Gluster disks are thin (raw-sparse) by default just like any other file based storage.
If this theory was correct, this would fail consistently on gluster:
1. create raw sparse image
truncate -s 100g /rhev/data-center/mnt/glusterSD/server:_path/test
2. Fill image quickly with data
dd if=/dev/zero bs=1M | tr "\0" "U" | dd of=/rhev/data-center/mnt/glusterSD/server:_path/test bs=1M count=12800 iflag=fullblock oflag=direct conv=notrunc
According to your theory gluster will fail to allocate shards fast enough and fail the I/O.
Nir
Coming back here as I got some other occurrences of the same error and I'm going to try to enable some logging and see where it is the origin. In the mean time I confirmed what written by Nir above. No error on gluster side with the command suggested: [root@ovirt libvirt]# time dd if=/dev/zero bs=1M | tr "\0" "U" | dd of="/rhev/data-center/mnt/glusterSD/ovirtst.mydomain.storage:_vmstore/test" bs=1M count=12800 iflag=fullblock oflag=direct conv=notrunc 12800+0 records in 12800+0 records out 13421772800 bytes (13 GB) copied, 37.8056 s, 355 MB/s real 0m37.812s user 0m10.582s sys 0m24.072s [root@ovirt libvirt]# [root@ovirt libvirt]# echo $? 0 [root@ovirt libvirt]# During the copy a "vmstat 3" produced indeed about 350 MiB/s: [root@ovirt ~]# vmstat 3 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 8 0 0 1274344448 329220 5752988 0 0 8 142 0 0 4 2 95 0 0 12 0 0 1274341504 329220 5753008 0 0 1930 30079 76558 186828 7 3 90 0 0 3 0 0 1274340480 329220 5753028 0 0 917 15786 68796 178179 6 3 91 0 0 2 0 0 1274336128 329220 5753048 0 0 61 14219 69035 172374 6 3 90 0 0 17 2 0 1274334976 329220 5753088 0 0 1931 14814 74888 178985 7 3 89 0 0 7 0 0 1274317952 329220 5753112 0 0 1556 144318 72745 257632 7 3 89 0 0 11 1 0 1274315136 329220 5753112 0 0 91 350458 80783 410853 8 4 87 1 0 12 0 0 1274310272 329220 5753148 0 0 965 378040 83638 419936 6 4 89 1 0 10 1 0 1274308352 329220 5753188 0 0 1470 379597 82909 429217 7 4 88 1 0 7 0 0 1274318592 329220 5753192 0 0 608 368145 85389 417066 9 5 86 1 0 9 1 0 1274323584 329220 5753208 0 0 460 370369 81002 426601 7 5 88 1 0 12 2 0 1274323456 329220 5753232 0 0 2655 369143 79131 415323 7 4 88 1 0 10 2 0 1274306560 329220 5753252 0 0 625 367324 82076 419014 8 4 87 1 0 15 0 0 1274276480 329220 5753272 0 0 99 355281 81662 409003 8 4 87 1 0 12 2 0 1274259072 329220 5753292 0 0 855 358916 82942 416828 7 4 88 1 0 5 0 0 1274257024 329220 5753316 0 0 1566 374178 81489 415783 7 4 88 1 0 7 2 0 1274259968 329220 5753324 0 0 527 344149 82577 418192 7 4 88 1 0 7 0 0 1274254080 329220 5753364 0 0 1596 341075 82839 414259 6 4 89 1 0 7 1 0 1274267776 329220 5753404 0 0 1431 97078 73923 242802 6 3 90 0 0 Tried 4 times without any error and consistent timing of about 36-37 seconds Go to next tests Gianluca