I am about to setup a new hyperconverged setup with the following 3 host setup 2 x nvme 240gb hd for OS on each host (raid) 3 x nvme 7 tb hd for vm storage on each host (no raid card) 500gb RAM per each host I am confused during the gluster setup portion using the web ui setup wizard. When asked for the LV size do I input the maximum size of the hard drive on a single host or do I combine the total capacity of the matching hard drives on all 3 host? Example that I would be putting in is hard drive: /dev/nvme1p1 capacity 7tb or should it be 21tb (combined capacity of the matching single HD on the other host). Or is there a better method you recommend? And since I am using nvme hard drive would you recommend using dedup and compression or no? Thanks
Personally I also found this confusing when I setup my cluster a while back. I ended up creating multiple data volumes. One for each drive. You could probably software raid the drives first and present it to the deployment wizard as one block device. I’m not sure if deployment wizard will combine multiple physical drives into one volume for you. Hopefully someone else can shed more light on this. With your host specs it sounds like you should be fine to use dedupe/compression On Wed, Dec 2, 2020 at 8:10 AM <dofranklin@port-orange.org> wrote:
I am about to setup a new hyperconverged setup with the following
3 host setup 2 x nvme 240gb hd for OS on each host (raid) 3 x nvme 7 tb hd for vm storage on each host (no raid card) 500gb RAM per each host
I am confused during the gluster setup portion using the web ui setup wizard. When asked for the LV size do I input the maximum size of the hard drive on a single host or do I combine the total capacity of the matching hard drives on all 3 host? Example that I would be putting in is hard drive: /dev/nvme1p1 capacity 7tb or should it be 21tb (combined capacity of the matching single HD on the other host). Or is there a better method you recommend? And since I am using nvme hard drive would you recommend using dedup and compression or no?
Thanks _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CLIXBOMZ7EEZ54...
Thanks for the reply. I have been told by a couple of different people not use software raid with this gluster setup. Did you end creating your volumes in command line or did you follow the wizard? Thanks
I have two ssds in each host for storage. I ended up using the wizard but in the wizard I simply added two data volumes Ex: storage1 = /dev/sda storage2 = /dev/sdb You can create as many storage volumes as you want. You don’t need to have just one single large volume. You could have just one big volume but I believe you’d need either hardware or software raid to accomplish that. Am I correct that you have three drives (to be used for storage) in EACH host or three drives total across all three hosts? On Wed, Dec 2, 2020 at 8:47 AM cpo cpo <dofranklin@port-orange.org> wrote:
Thanks for the reply. I have been told by a couple of different people not use software raid with this gluster setup. Did you end creating your volumes in command line or did you follow the wizard?
Thanks _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BRYP2KRFBFUYO7...
Ok yeah that is fairly similar to my setup, except I only have two drives in each host. In my case I created completely separate data volume, one per drive. You could do the same, three data volumes for storage @ 7TB each for example. On one of the volumes you'll need to split off 100Gb volume for engine, So you'd end up with something like 2 x 7Tb data volumes, 100Gb engine and 6.9TB data volume or something along those lines. With multiple non-raided drives I'm not sure if this is the best approach or if it makes more sense to software raid the drives first. Personally I like the fact that my volumes are separated. On Wed, Dec 2, 2020 at 9:13 AM cpo cpo <dofranklin@port-orange.org> wrote:
Once again thanks for the reply. I have 3 identical drives in EACH host.
Thanks _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/5E2ODTEBMOJIKA...
Thanks. Trying to figure out if I should use dedup/comp now. If I don't is the total usable space if I follow your setup guide for my storage domain going to be 21tb once I am done with everything (3 vm storage volumes, one per disk)? Thanks, Donnie
On 2020-12-02 09:56, cpo cpo wrote:
Thanks. Trying to figure out if I should use dedup/comp now. If I don't is the total usable space if I follow your setup guide for my storage domain going to be 21tb once I am done with everything (3 vm storage volumes, one per disk)?
Thanks, Donnie _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/5F3VBLSQPRPJPM...
the comp/dedupe features use VDO, which IMO is a fairly large performance hit, at least in my testing on 9x900GB 24 drive raid 10 arrays. My personal recommendation is to just use hardware / mdadm raid and lvm on top (oVirt can do this for you in the UI) If you really need comp/dedupe, we use ZFS in production for that purpose. Granted this is not supported by oVirt, so YMMV.
You are using NVMEs, so you can safely use JBOD mode. I think that the size should be for a single disk, but you can experiment on that. If the LVs are too small, you can just extend them or even recreate them. What is you network speed ? If your network bandwidth matches your NVMEs' combined Read speed - then avoid dedup and compression unless you really need them. Most probably your NVMEs have a 4096 sector size . If yes, you will need vdo with '--emulate512'. Yet, tuning VDO for such fast devices is not a trivial task. Most probably you can configure a VDO per each NVME (or even per pertition/lv) , and configure it with compression and dedup disabled (emulate512 only if needed). Also check the UDS index topic, as it is quite important: https://access.redhat.com/documentation/en_us/red_hat_enterprise_linux/7/htm... If your network is slower than your disks, you can enable dedup and compression. Also gluster has a setting called 'cluster.choose-local' . You can enable it in order to tell the FUSE driver to prefer local brick instead those over the network. Best Regards, Strahil Nikolov В сряда, 2 декември 2020 г., 14:10:21 Гринуич+2, dofranklin@port-orange.org <dofranklin@port-orange.org> написа: I am about to setup a new hyperconverged setup with the following 3 host setup 2 x nvme 240gb hd for OS on each host (raid) 3 x nvme 7 tb hd for vm storage on each host (no raid card) 500gb RAM per each host I am confused during the gluster setup portion using the web ui setup wizard. When asked for the LV size do I input the maximum size of the hard drive on a single host or do I combine the total capacity of the matching hard drives on all 3 host? Example that I would be putting in is hard drive: /dev/nvme1p1 capacity 7tb or should it be 21tb (combined capacity of the matching single HD on the other host). Or is there a better method you recommend? And since I am using nvme hard drive would you recommend using dedup and compression or no? Thanks _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CLIXBOMZ7EEZ54...
Thanks for info. My storage network speed is 10gb. I rolled out a config where I created 2 domains 1 for and engine the other for vmstore. I decided not to use dedup. I kept my 3 disk on each host non raided. For my vmstore domain I created 9 bricks 1 for each physical disk with a LV size of 6.6tb I created the bricks using the gui wizard. The management engine says I have 19tb of useable space and my vmstore volume is distributed replicate. Would it be ok to create more bricks for this domain with 6.6 TB LV size or will I run into issues with physical space? The reason why I ask is the having 40TB of physical space and only getting 20 TB usable space in my current config gives me a little heart burn. Thanks
With 10GBE you can reach no more than 1.25GB , so you should enable cluster.choose-local if your disks can squeeze more than that. With Replica 3 , you got 1/3 of total space so it seems normal. You can always expand a distributed-replicated volume as long as you expand with 3 bricks at a time (but you will have to rebalance your volume). Keep the replica set (the 3 bricks) of the same size or you will waste space. Best Regards, Strahil Nikolov В четвъртък, 3 декември 2020 г., 21:29:16 Гринуич+2, cpo cpo <dofranklin@port-orange.org> написа: Thanks for info. My storage network speed is 10gb. I rolled out a config where I created 2 domains 1 for and engine the other for vmstore. I decided not to use dedup. I kept my 3 disk on each host non raided. For my vmstore domain I created 9 bricks 1 for each physical disk with a LV size of 6.6tb I created the bricks using the gui wizard. The management engine says I have 19tb of useable space and my vmstore volume is distributed replicate. Would it be ok to create more bricks for this domain with 6.6 TB LV size or will I run into issues with physical space? The reason why I ask is the having 40TB of physical space and only getting 20 TB usable space in my current config gives me a little heart burn. Thanks _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KBBDIQ42YIWHJQ...
participants (5)
-
Alex McWhirter -
cpo cpo -
dofranklin＠port-orange.org
-
Jayme -
Strahil Nikolov