11 a.m.
On April 8, 2020 2:43:01 PM GMT+03:00, Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Tue, Apr 7, 2020 at 8:16 PM Strahil Nikolov
<hunter86_bg(a)yahoo.com>
wrote:
Hi Gianluca,
>
>
> The positive thing is that you can reproduce the issue.
>
> I would ask you to check your gluster version and if there are any
> updates - update the cluster.
>
I'd prefer to stick on oVirt release version of Gluster if possible
This is what I ment, but if you are using Gluster v6.0 - you should update to latest
version.
Also check the gluster's op-version, as this limits some of the
features.
>
What do you mean by thgis?
Check this one:
https://docs.gluster.org/en/latest/Upgrade-Guide/op_version/
> If there are none - enable trace logs in gluster (
>
https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.1...
> ), start volume peofiling, reproduce the issue and then reduce the
log
> level (it's generating a lot of logs) and stop the profiling.
>
> Once that info is collected, some of the Gluster members can check
the
> situation.
>
> Best Regards,
> Strahil Nikolov
>
Ok. I think that the INFO level set on the different layers outlined
problem somehow related to shardin.
Related to this, I found no official docs on Gluster web site after 3.7
version... where is it?
Only information I found was in Red Hat Gluster Storage 3.5
Administration
Guide, but I would expect something more upstream...
Red Hat versioning is different. So far RH documentation for Gluster has never failed
me.
In particular in my case where I have only one host and the gluster
volumes
are single brick based, do you think I can try to disable sharding and
verify if using new disks with it disabled and oVirt thin provisioned
disks
let the problem go away?
Also, I found some information about sharding block size.
Apparently the only supported size on Red Hat Gluster Storage is 512MB,
but
oVirt sets it to 64MB....?
I also found a bugzilla about passing from 128MB to 64MB in oVirt
4.1.5:
https://bugzilla.redhat.com/show_bug.cgi?id=1469436
Now I see that by default and so also in my environment I have:
features.shard on
features.shard-block-size 64MB
features.shard-lru-limit 16384
features.shard-deletion-rate 100
Not clear how to cross-check for all the values above in docs....
Can I safely set
features.shard off
In Red Hat Gluster storage admin guide it is considered only for
replicated
gluster volumes (and so also ditributed-replicated...)
But in my case with single host and single beick for volumes I think it
doesn't give any particular benefit, isn't it?
NEVER DISABLE SHARDING ON A VOLUME!!!
There is a very good reply from Amar Tumballi in gluster users mailing list about that.
The good thing is that you can do storage migration.
The only benefits of sharding are:
1. When using distributed-replicated volume, shards will spread on different bricks and
thus partially increase read performance
2. When a heal is going on, the shard is locked - so there will be no disruption for the
VM
In both cases you cannot benefit of that.
Shard size is 64 MB and this is default from Gluster , not from oVirt. Maybe there is
a reason behind that large shard size, but I have no clue.
In your case, you shokuld consider using VDO, as most of the VMs have almost the same
system files, which will lead to data reduction at a small cost of write speed.
Another option is to create a new volume and disable sharding at all.
I found this interesting post about sharding and connection between
image
files and shard files:
http://opensource-storage.blogspot.com/2016/07/de-mystifying-gluster-shar...
Gianluca
I have one question - you have a single brick in your volume. Why do you use oVirt
instead of plain KVM ?
Best Regards,
Strahil Nikolov