12:40 a.m.
Hello ,
Hello and thank you for the reply.Bellow are the answers to your questions.
Let me ask some questions:
1. What is the scheduler for your PV ?
On the Raid Controller device where the SSD disks are in Raid 0 (device sda) it is set to
"deadline". But on the lvm volume logical volume dm-7, where the logical block
is set for "data" volunr it is set to none.(i think this is ok )
[root@host1 ~]# ls -al /dev/mapper/gluster_vg_sd
v_data ter_l
lrwxrwxrwx. 1 root root 7 Jun 28 14:14 /dev/mapper/gluster_v
g_sda3-gluster_lv_data -> ../dm-7
[root@host1 ~]# cat /sys/block/dm-7/queue/scheduler
none
root@host1:~[root@host1 ~]# cat /sys/block/dm-7/queue/schedu
[root@host1 ~]# cat /sys/block/sda/queue/scheduler
noop [deadline] cfq
2. Have you aligned your PV during the setup 'pvcreate
--dataalignment alignment_value
device'
I did not made other alignment then the default.Bellow are the partitions on /dev/sda.
Can i enable partition alignment now, if yes how ?
sfdisk -d /dev/sda
# partition table of /dev/sda
unit: sectors
/dev/sda1 : start= 2048, size= 487424, Id=83, bootable
/dev/sda2 : start= 489472, size= 95731712, Id=8e
/dev/sda3 : start= 96221184, size=3808675840, Id=83
/dev/sda4 : start= 0, size= 0, Id= 0
3. What is your tuned profile ? Do you use rhgs-random-io from
the ftp://ftp.redhat.com/redhat/linux/enterprise/7Server/en/RHS/SRPMS/red...
?
My tuned active profile is virtual-host
Current active profile: virtual-host
No i dont use any of the rhgs-random-io profiles
4. What is the output of "xfs_info
/path/to/your/gluster/brick" ?
xfs_info /gluster_bricks/data
meta-data=/dev/mapper/gluster_vg_sda3-gluster_lv_data isize=
512 agcount=32, agsize=6553600 blks
= sectsz=512 attr=2, projid
32bit=1
= crc=1 finobt=0 spino
des=0
data = bsize=4096 blocks=2097152
00, imaxpct=25
= sunit=64 swidth=64 blks
naming =version 2 bsize=8192 ascii-ci=0 fty
pe=1
log =internal bsize=4096 blocks=102400,
version=2
= sectsz=512 sunit=64 blks,
lazy-count=1
realtime =none extsz=4096 blocks=0, rtex
tents=0
5. Are you using Jumbo Frames ? Does your infra support them?
Usually MTU of 9k is standard, but some switches and NICs support up to 16k.
Unfortunately I can not enable MTU to 9000 and Jumbo Frames on these Cisco SG350X
switches to specific ports.The switches dont suport Jumbo Frames enable to a single
port, only on all ports .
I have others devices connected to the switches on the remaining 48 ports that have
1Gb/s.
All the options for "optimize for virt...." are located
at /var/lib/glusterd/groups/virt on each gluster node.
I have already looked previously at that file, but not all the volume settings that are
set by "Optime for Virt Store" are stored there.
For example "Optimize for Virt Store " sets network.remote.dio to disable and
in the glusterd/groups/virt is set to enabled.Or cluster.granular-entry-heal: enable is
not present there, bit it is set by "Optimize for Virt Store"
>
> Best Regards,
> Strahil Nikolov
>
>
>
>
> В неделя, 28 юни 2020 г., 22:13:09 Гринуич+3, jury cat
<shadow.emy1(a)gmail.com>
> написа:
>
>
>
>
>
> Hello all,
>
> I am using Ovirt 4.3.10 on Centos 7.8 with glusterfs 6.9 .
> My Gluster setup is of 3 hosts in replica 3 (2 hosts + 1 arbiter).
> All the 3 hosts are Dell R720 with Perc Raid Controller H710 mini(that has maximim
> throughtout 6Gbs) and with 2×1TB samsumg SSD in RAID 0. The volume is partitioned
using
> LVM thin provision and formated XFS.
> The hosts have separate 10GE network cards for storage traffic.
> The Gluster Network is connected to this 10GE network cards and is mounted using
Fuse
> Glusterfs(NFS is disabled).Also Migration Network is activated on the same storage
> network.
>
>
> The problem is that the 10GE network is not used at full potential by the Gluster.
> If i do live Migration of Vms i can see speeds of 7GB/s ~ 9GB/s.
> The same network tests using iperf3 reported 9.9GB/s , these exluding the network
setup
> as a bottleneck(i will not paste all the iperf3 tests here for now).
> I did not enable all the Volume options from "Optimize for Virt Store",
because
> of the bug that cant set volume cluster.granural-heal to enable(this was fixed in
vdsm-4
> 40, but that is working only on Centos 8 with ovirt 4.4 ) .
> i whould be happy to know what are all these "Optimize for Virt Store"
options,
> so i can set them manually.
>
>
> The speed on the disk inside the host using dd is b etween 1GB/s to 700Mbs.
>
>
> [root@host1 ~]# dd if=/dev/zero of=test bs=100M count=40 cou nt=80 status=progress
> 8074035200 bytes (8.1 GB) copied, 11.059372 s, 730 MB/s 80+0 records in 80+0 records
out
> 8388608000 bytes (8.4 GB) copied, 11.9928 s, 699 MB/s
>
>
> The dd write test on the gluster volme inside the host is poor only ~ 120MB/s .
> During the dd test, if i look at Networks->Gluster network ->Hosts at Tx and Rx
the
> network speed barerly reaches over 1Gbs (~1073 Mbs) out of maximum of 10000 Mbs.
>
>
> dd if=/dev/zero of=/rhev/data-center/mnt/glu sterSD/gluster1.domain.local\:
_data/test
> bs=100M count=80 status=progress 8283750400 bytes (8.3 GB) copied, 71.297942 s, 116
MB/s
> 80+0 records in 80+0 records out 8388608000 bytes (8.4 GB) copied, 71.9545 s, 117
MB/s
>
>
> I have attached my Gluster volume settings and mount options.
>
> Thanks,
> Emy
>
>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/7BR6TZQ4EXS...