10:23 p.m.
Hello,
I'm running oVirt with several data centers, some with NFS storage and
some with local storage.
I had problems in the past with a large pool and local storage. The
problem was nodectl showed the pool being too full (I think >80%), but
it was only the images that made the pool "full" -- and this storage was
carefully setup such that there was no chance it would actually fill.
The LVs for oVirt itself were all under 20%, yet nodectl still reported
the pool was too full.
My solution so far has been to use our RAID card tools, so that sda is
the oVirt node install, and sdb is for images. There are probably other
good reasons for me to handle it this way, for example being able to use
different RAID levels, but I'm hoping someone can confirm my
partitioning below doesn't have some risk I'm now yet aware of.
I setup a new volume group for images, as below:
[root@node4-g8-h4 multipath]# pvs
PV VG Fmt Attr PSize
PFree
/dev/mapper/3600508b1001c7e172160824d7b204c3b2 onn_node4-g8-h4 lvm2
a-- <119.00g <22.85g
/dev/sdb1 data lvm2 a--
1.13t <361.30g
[root@node4-g8-h4 multipath]# vgs
VG #PV #LV #SN Attr VSize VFree
data 1 1 0 wz--n- 1.13t <361.30g
onn_node4-g8-h4 1 13 0 wz--n- <119.00g <22.85g
[root@node4-g8-h4 multipath]# lvs
LV VG Attr LSize
Pool Origin Data% Meta% Move Log Cpy%Sync
Convert
images_main data -wi-ao---- 800.00g
home onn_node4-g8-h4 Vwi-aotz--
1.00g pool00 4.79
ovirt-node-ng-4.2.5.1-0.20180816.0 onn_node4-g8-h4 Vwi---tz-k
64.10g pool00 root
ovirt-node-ng-4.2.5.1-0.20180816.0+1 onn_node4-g8-h4 Vwi---tz--
64.10g pool00 ovirt-node-ng-4.2.5.1-0.20180816.0
ovirt-node-ng-4.2.6-0.20180903.0 onn_node4-g8-h4 Vri---tz-k
64.10g pool00
ovirt-node-ng-4.2.6-0.20180903.0+1 onn_node4-g8-h4 Vwi-aotz--
64.10g pool00 ovirt-node-ng-4.2.6-0.20180903.0 4.83
pool00 onn_node4-g8-h4 twi-aotz--
91.10g 8.94 0.49
root onn_node4-g8-h4 Vwi---tz--
64.10g pool00
swap onn_node4-g8-h4 -wi-ao---- 4.00g
tmp onn_node4-g8-h4 Vwi-aotz--
1.00g pool00 4.87
var onn_node4-g8-h4 Vwi-aotz--
15.00g pool00 3.31
var_crash onn_node4-g8-h4 Vwi-aotz--
10.00g pool00 2.86
var_log onn_node4-g8-h4 Vwi-aotz--
8.00g pool00 3.57
var_log_audit onn_node4-g8-h4 Vwi-aotz--
2.00g pool00 4.89
The images_main is setup as "Block device for filesystems" with ext4. Is
there any reason I should consider pool for thinly provisioned volumes?
I don't need to over-allocate storage and it seems to me like a fixed
partition is ideal. Please confirm or let me know if there's anything
else I should consider.
Thanks
Matt
10:58 p.m.
Hello all,
Following this report below, I did a reboot. Now I have a real question.
I added the VG, LV and mount point to this node using the port 9090 web
interface.
Now the volume group isn't active and will not mount, causing the boot
to hang.
I am able to do "vgchange -ay data" and then a manual mount in rescue mode.
Any feedback on the best way to add a new volume group to an empty
partition (sdb) would be appreciated. Prior to using the web interface,
I was having failures using the manual tools to /dev/sdb with an error
"device /dev/sdb excluded by filter" which I suspect is related.
Thanks
Matt
On 09/04/2018 01:23 PM, Matt Simonsen wrote:
Hello,
I'm running oVirt with several data centers, some with NFS storage and
some with local storage.
I had problems in the past with a large pool and local storage. The
problem was nodectl showed the pool being too full (I think >80%), but
it was only the images that made the pool "full" -- and this storage
was carefully setup such that there was no chance it would actually
fill. The LVs for oVirt itself were all under 20%, yet nodectl still
reported the pool was too full.
My solution so far has been to use our RAID card tools, so that sda is
the oVirt node install, and sdb is for images. There are probably
other good reasons for me to handle it this way, for example being
able to use different RAID levels, but I'm hoping someone can confirm
my partitioning below doesn't have some risk I'm now yet aware of.
I setup a new volume group for images, as below:
[root@node4-g8-h4 multipath]# pvs
PV VG Fmt Attr PSize
PFree
/dev/mapper/3600508b1001c7e172160824d7b204c3b2 onn_node4-g8-h4 lvm2
a-- <119.00g <22.85g
/dev/sdb1 data lvm2 a-- 1.13t
<361.30g
[root@node4-g8-h4 multipath]# vgs
VG #PV #LV #SN Attr VSize VFree
data 1 1 0 wz--n- 1.13t <361.30g
onn_node4-g8-h4 1 13 0 wz--n- <119.00g <22.85g
[root@node4-g8-h4 multipath]# lvs
LV VG Attr LSize
Pool Origin Data% Meta% Move Log
Cpy%Sync Convert
images_main data -wi-ao---- 800.00g
home onn_node4-g8-h4 Vwi-aotz--
1.00g pool00 4.79
ovirt-node-ng-4.2.5.1-0.20180816.0 onn_node4-g8-h4 Vwi---tz-k
64.10g pool00 root
ovirt-node-ng-4.2.5.1-0.20180816.0+1 onn_node4-g8-h4 Vwi---tz--
64.10g pool00 ovirt-node-ng-4.2.5.1-0.20180816.0
ovirt-node-ng-4.2.6-0.20180903.0 onn_node4-g8-h4 Vri---tz-k
64.10g pool00
ovirt-node-ng-4.2.6-0.20180903.0+1 onn_node4-g8-h4 Vwi-aotz--
64.10g pool00 ovirt-node-ng-4.2.6-0.20180903.0 4.83
pool00 onn_node4-g8-h4 twi-aotz--
91.10g 8.94 0.49
root onn_node4-g8-h4 Vwi---tz--
64.10g pool00
swap onn_node4-g8-h4 -wi-ao---- 4.00g
tmp onn_node4-g8-h4 Vwi-aotz--
1.00g pool00 4.87
var onn_node4-g8-h4 Vwi-aotz--
15.00g pool00 3.31
var_crash onn_node4-g8-h4 Vwi-aotz--
10.00g pool00 2.86
var_log onn_node4-g8-h4 Vwi-aotz--
8.00g pool00 3.57
var_log_audit onn_node4-g8-h4 Vwi-aotz--
2.00g pool00 4.89
The images_main is setup as "Block device for filesystems" with ext4.
Is there any reason I should consider pool for thinly provisioned
volumes? I don't need to over-allocate storage and it seems to me
like a fixed partition is ideal. Please confirm or let me know if
there's anything else I should consider.
Thanks
Matt
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7N547X6DC7...
11:22 p.m.
On Wed, Sep 5, 2018 at 12:01 AM Matt Simonsen <matt(a)khoza.com> wrote:
Hello all,
Following this report below, I did a reboot. Now I have a real question.
I added the VG, LV and mount point to this node using the port 9090 web
interface.
Now the volume group isn't active and will not mount, causing the boot
to hang.
I am able to do "vgchange -ay data" and then a manual mount in rescue mode.
Any feedback on the best way to add a new volume group to an empty
partition (sdb) would be appreciated. Prior to using the web interface,
I was having failures using the manual tools to /dev/sdb with an error
"device /dev/sdb excluded by filter" which I suspect is related.
Maybe you have lvm filter set, which is highly recommend for an oVirt
hypervisor.
To add /dev/sdb, you need to add it to the lvm filter in /etc/lvm/lvm.conf.
After you configure the device properly, you can generate lvm filter
for the current setup using:
vdsm-tool config-lvm-filter
Here is example run on unconfigued oVirt host:
# vdsm-tool config-lvm-filter
Analyzing host...
Found these mounted logical volumes on this host:
logical volume: /dev/mapper/fedora_voodoo1-root
mountpoint: /
devices: /dev/vda2
logical volume: /dev/mapper/fedora_voodoo1-swap
mountpoint: [SWAP]
devices: /dev/vda2
This is the recommended LVM filter for this host:
filter = [ "a|^/dev/vda2$|", "r|.*|" ]
This filter allows LVM to access the local devices used by the
hypervisor, but not shared storage owned by Vdsm. If you add a new
device to the volume group, you will need to edit the filter manually.
Nir
On 09/04/2018 01:23 PM, Matt Simonsen wrote:
> Hello,
>
> I'm running oVirt with several data centers, some with NFS storage and
> some with local storage.
>
> I had problems in the past with a large pool and local storage. The
> problem was nodectl showed the pool being too full (I think >80%), but
> it was only the images that made the pool "full" -- and this storage
> was carefully setup such that there was no chance it would actually
> fill. The LVs for oVirt itself were all under 20%, yet nodectl still
> reported the pool was too full.
>
> My solution so far has been to use our RAID card tools, so that sda is
> the oVirt node install, and sdb is for images. There are probably
> other good reasons for me to handle it this way, for example being
> able to use different RAID levels, but I'm hoping someone can confirm
> my partitioning below doesn't have some risk I'm now yet aware of.
>
> I setup a new volume group for images, as below:
>
>
> [root@node4-g8-h4 multipath]# pvs
> PV VG Fmt Attr PSize
> PFree
> /dev/mapper/3600508b1001c7e172160824d7b204c3b2 onn_node4-g8-h4 lvm2
> a-- <119.00g <22.85g
> /dev/sdb1 data lvm2 a-- 1.13t
> <361.30g
>
> [root@node4-g8-h4 multipath]# vgs
> VG #PV #LV #SN Attr VSize VFree
> data 1 1 0 wz--n- 1.13t <361.30g
> onn_node4-g8-h4 1 13 0 wz--n- <119.00g <22.85g
>
> [root@node4-g8-h4 multipath]# lvs
> LV VG Attr LSize
> Pool Origin Data% Meta% Move Log
> Cpy%Sync Convert
> images_main data -wi-ao---- 800.00g
> home onn_node4-g8-h4 Vwi-aotz--
> 1.00g pool00 4.79
> ovirt-node-ng-4.2.5.1-0.20180816.0 onn_node4-g8-h4 Vwi---tz-k
> 64.10g pool00 root
> ovirt-node-ng-4.2.5.1-0.20180816.0+1 onn_node4-g8-h4 Vwi---tz--
> 64.10g pool00 ovirt-node-ng-4.2.5.1-0.20180816.0
> ovirt-node-ng-4.2.6-0.20180903.0 onn_node4-g8-h4 Vri---tz-k
> 64.10g pool00
> ovirt-node-ng-4.2.6-0.20180903.0+1 onn_node4-g8-h4 Vwi-aotz--
> 64.10g pool00 ovirt-node-ng-4.2.6-0.20180903.0 4.83
> pool00 onn_node4-g8-h4 twi-aotz--
> 91.10g 8.94 0.49
> root onn_node4-g8-h4 Vwi---tz--
> 64.10g pool00
> swap onn_node4-g8-h4 -wi-ao---- 4.00g
> tmp onn_node4-g8-h4 Vwi-aotz--
> 1.00g pool00 4.87
> var onn_node4-g8-h4 Vwi-aotz--
> 15.00g pool00 3.31
> var_crash onn_node4-g8-h4 Vwi-aotz--
> 10.00g pool00 2.86
> var_log onn_node4-g8-h4 Vwi-aotz--
> 8.00g pool00 3.57
> var_log_audit onn_node4-g8-h4 Vwi-aotz--
> 2.00g pool00 4.89
>
>
>
> The images_main is setup as "Block device for filesystems" with ext4.
> Is there any reason I should consider pool for thinly provisioned
> volumes? I don't need to over-allocate storage and it seems to me
> like a fixed partition is ideal. Please confirm or let me know if
> there's anything else I should consider.
>
>
> Thanks
>
> Matt
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7N547X6DC7...
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJINANK6PAG...
11:45 p.m.
On 09/04/2018 02:22 PM, Nir Soffer wrote:
Maybe you have lvm filter set, which is highly recommend for an oVirt
hypervisor.
Indeed, I do. I am not sure I have the right filter however, so I
appreciate the help.
This is the filter setup initially:
filter = [ "a|^/dev/mapper/3600508b1001c7e172160824d7b204c3b2$|",
"r|.*|" ]
Just to be clear, my intent isn't to add /dev/sdb to the main volume
group, but to make a new volume group to setup a local ext4 mount point.
I changed it to:
filter = [ "a|^/dev/sdb|",
"a|^/dev/mapper/3600508b1001c7e172160824d7b204c3b2$|", "r|.*|" ]
Following this and a reboot, I was able to create a PV, VG, and LV.
# pvcreate /dev/sdb
# vgcreate data /dev/sdb
# lvcreate -L800g /dev/data --name local_images
# mkfs.ext4 /dev/data/local_images
-- adjust fstab
# mount -a
It seems to function as expected now the the filter has been adjusted.
But is the filter doing what it is "supposed" to?
When I run the command "vdsm-tool config-lvm-filter" what I see is:
[root@node4-g8-h4 ~]# vdsm-tool config-lvm-filter
Analyzing host...
LVM filter is already configured for Vdsm
Thanks for the help and confirming how this should work.
Matt
To add /dev/sdb, you need to add it to the lvm filter in
/etc/lvm/lvm.conf.
After you configure the device properly, you can generate lvm filter
for the current setup using:
vdsm-tool config-lvm-filter
Here is example run on unconfigued oVirt host:
# vdsm-tool config-lvm-filter
Analyzing host...
Found these mounted logical volumes on this host:
logical volume: /dev/mapper/fedora_voodoo1-root
mountpoint: /
devices: /dev/vda2
logical volume: /dev/mapper/fedora_voodoo1-swap
mountpoint: [SWAP]
devices: /dev/vda2
This is the recommended LVM filter for this host:
filter = [ "a|^/dev/vda2$|", "r|.*|" ]
This filter allows LVM to access the local devices used by the
hypervisor, but not shared storage owned by Vdsm. If you add a new
device to the volume group, you will need to edit the filter manually.
Nir
On 09/04/2018 01:23 PM, Matt Simonsen wrote:
> Hello,
>
> I'm running oVirt with several data centers, some with NFS
storage and
> some with local storage.
>
> I had problems in the past with a large pool and local storage. The
> problem was nodectl showed the pool being too full (I think
>80%), but
> it was only the images that made the pool "full" -- and this
storage
> was carefully setup such that there was no chance it would actually
> fill. The LVs for oVirt itself were all under 20%, yet nodectl
still
> reported the pool was too full.
>
> My solution so far has been to use our RAID card tools, so that
sda is
> the oVirt node install, and sdb is for images. There are probably
> other good reasons for me to handle it this way, for example being
> able to use different RAID levels, but I'm hoping someone can
confirm
> my partitioning below doesn't have some risk I'm now yet aware of.
>
> I setup a new volume group for images, as below:
>
>
> [root@node4-g8-h4 multipath]# pvs
> PV VG Fmt Attr PSize
> PFree
> /dev/mapper/3600508b1001c7e172160824d7b204c3b2 onn_node4-g8-h4
lvm2
> a-- <119.00g <22.85g
> /dev/sdb1 data lvm2 a--
1.13t
> <361.30g
>
> [root@node4-g8-h4 multipath]# vgs
> VG #PV #LV #SN Attr VSize VFree
> data 1 1 0 wz--n- 1.13t <361.30g
> onn_node4-g8-h4 1 13 0 wz--n- <119.00g <22.85g
>
> [root@node4-g8-h4 multipath]# lvs
> LV VG Attr LSize
> Pool Origin Data% Meta% Move Log
> Cpy%Sync Convert
> images_main data -wi-ao---- 800.00g
> home onn_node4-g8-h4 Vwi-aotz--
> 1.00g pool00 4.79
> ovirt-node-ng-4.2.5.1-0.20180816.0 onn_node4-g8-h4 Vwi---tz-k
> 64.10g pool00 root
> ovirt-node-ng-4.2.5.1-0.20180816.0+1 onn_node4-g8-h4 Vwi---tz--
> 64.10g pool00 ovirt-node-ng-4.2.5.1-0.20180816.0
> ovirt-node-ng-4.2.6-0.20180903.0 onn_node4-g8-h4 Vri---tz-k
> 64.10g pool00
> ovirt-node-ng-4.2.6-0.20180903.0+1 onn_node4-g8-h4 Vwi-aotz--
> 64.10g pool00 ovirt-node-ng-4.2.6-0.20180903.0 4.83
> pool00 onn_node4-g8-h4 twi-aotz--
> 91.10g 8.94 0.49
> root onn_node4-g8-h4 Vwi---tz--
> 64.10g pool00
> swap onn_node4-g8-h4
-wi-ao---- 4.00g
> tmp onn_node4-g8-h4 Vwi-aotz--
> 1.00g pool00 4.87
> var onn_node4-g8-h4 Vwi-aotz--
> 15.00g pool00 3.31
> var_crash onn_node4-g8-h4 Vwi-aotz--
> 10.00g pool00 2.86
> var_log onn_node4-g8-h4 Vwi-aotz--
> 8.00g pool00 3.57
> var_log_audit onn_node4-g8-h4 Vwi-aotz--
> 2.00g pool00 4.89
>
>
>
> The images_main is setup as "Block device for filesystems" with
ext4.
> Is there any reason I should consider pool for thinly provisioned
> volumes? I don't need to over-allocate storage and it seems to me
> like a fixed partition is ideal. Please confirm or let me know if
> there's anything else I should consider.
>
>
> Thanks
>
> Matt
> _______________________________________________
> Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org>
> To unsubscribe send an email to users-leave(a)ovirt.org
<mailto:users-leave@ovirt.org>
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I7N547X6DC7...
_______________________________________________
Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org>
To unsubscribe send an email to users-leave(a)ovirt.org
<mailto:users-leave@ovirt.org>
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJINANK6PAG...
2083
days inactive
2083
days old
3 comments
2 participants
participants (2)
-
Matt Simonsen -
Nir Soffer