On Wed, Sep 4, 2019 at 9:27 PM <thomas(a)hoberg.net&gt; wrote: Hi Thomas, The way we have to write the variables for 2.8 while setting up cache. Currently we are writing something like this: gluster_infra_cache_vars: - vgname: vg_sdb2 cachedisk: /dev/sdb3 cachelvname: cachelv_thinpool_vg_sdb2 cachethinpoolname: thinpool_vg_sdb2 cachelvsize: '10G' cachemetalvsize: '2G' cachemetalvname: cache_thinpool_vg_sdb2 cachemode: writethrough =================== Not that cachedisk is provided as /dev/sdb3 which would be extended with vg vg_sdb2 ... this works well The module will take care of extending the vg with /dev/sdb3. *However with Ansible-2.8 we cannot provide like this but have to be more explicit. And have to mention the pv underlying* *this volume group vg_sdb2. So, with respect to 2.8 we have to write that variable like:* >>>>>>>>>>> gluster_infra_cache_vars: - vgname: vg_sdb2 cachedisk: '/dev/sdb2,/dev/sdb3' cachelvname: cachelv_thinpool_vg_sdb2 cachethinpoolname: thinpool_vg_sdb2 cachelvsize: '10G' cachemetalvsize: '2G' cachemetalvname: cache_thinpool_vg_sdb2 cachemode: writethrough ===================== Note that I have mentioned both /dev/sdb2 and /dev/sdb3. This change is backward compatible, that is it works with 2.7 as well. I have raised an issue with Ansible as well. Which can be found here: https://github.com/ansible/ansible/issues/56501 However, @olafbuitelaar has fixed this in gluster-ansible-infra, and the patch is merged in master. If you can checkout master branch, you should be fine.

(could become a double post because I am using e-mail to attach the logs..) Hi URS, I have tried again using the latest release (4.3.7) and noted that now the more "explicit" variant you quote was generated. The behavior is changed, but it still fails now complaining about /dev/sdb being mounted (or inaccessible in any other way). I am attaching the logs. I have a HDD RAID on /dev/sdb and a SSD partiton on /dev/sda3 with >600GB of space left. I have mostly gone with defaults everywhere, used an arbiter (at least for the vmstore and data volumes) VDO and write-through caching with 550GB size (note that it fails to apply that value beyond the first node). Has anyone else tried a hyperconverged 3-node with SSD caching with success recently? Thanks for your feedback and help so far, Thomas

Hi Thomas, Can you please share "hc_wizard_inventory.yml" file which is under /etc/ansible/ ? On Thu, Nov 28, 2019 at 11:26 PM Thomas Hoberg <thomas(a)hoberg.net&gt; wrote: -- Thanks, Gobinda

When I first deployed my oVirt lab (v4.2.7 was latest and greatest) the ansible playbook didn't work for me.So I decided to stop the gluster processes on one of the nodes, Wipe all LVM and recreate it manually. Finally , I have managed to use my SSD for write-back cache - but I found out that if your Chunk size is larger than the default limit - it will never push it to the spinning disks. For details you can check 1668163 – LVM cache cannot flush buffer,change cache type or lvremove LV (CachePolicy 'cleaner' also doesn't work) As we use either 'replica 2 arbiter 1' (old name replica 3 arbiter 1) or a pure replica 3 , we can afford a gluster node go 'pouf' as long as we have decent bandwidth and we use sharding. So far I have changed my brick layout at least twice (for the cluster) without the VMs being affected - so you can still try to do the caching, but please check the comments in #1668163 about the chunk size of the cache. Best Regards,Strahil Nikolov В неделя, 1 декември 2019 г., 16:02:36 ч. Гринуич+2, Thomas Hoberg <thomas(a)hoberg.net&gt; написа: Hi Gobinda, unfortunately it's long gone, because I went back to an un-cached setup. It was mostly a trial anyway, I had to re-do the 3-node HCI because it had died rather horribly on me (a repeating issue I have so far had on distinct sets of hardware, that I am still trying to hunt down... separate topic). And since it was a blank(ed) set of servers, I just decided to try the SSD cache, to see if the Ansible script generation issue had been sorted out as described from upstream. I was rather encouraged to see that the Ansible script now had these changes included, that URS had described as becoming necessary with a new Ansible version. It doesn't actually make a lot of sense in the setup, because the SSD cache is a single Samsung EVO 860 1TB unit while the storage is a RAID6 out of 7 4TB 2.5" drives (per server): Both have similar bandwidth, IOPS would be very much workload dependent (the 2nd SSD I intended to use as a mirror was unfortunately cut from the budget). It has space left over because the OS doesn't need that much, but I don't dare use a single SSD as a write-back cache, especially because the RAID controller (HP420i) hides all wear information and doesn't seem to pass TRIM either and for write-through I'm not sure it would do noticeably better than the RAID controller (I configured that not to cache the SSD, too). So after it failed, I simply went back to no-cache for now. This HCI cluster is using relatively low-power hardware recalled from retirement that will host functional VMs, not high-performance workloads. They are well equipped with RAM and that's always the fastest cache anyway. I guess you should be able to add and remove the SSD as cache layer at any time during the operation, because it's at a level oVirt doesn't manage and I'd love to see examples as to how it's done. Especially the removal part would be important to know, if your SSD signals unexpected levels of wear and you need to swap them out on the fly. If I hit across another opportunity to test (most likely a single node), I will update here and make sure to collect a full set of log files including the ansible main config file. Thank you for your interest and the follow-up, Thomas _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Y45BAH7PXJN...