You should be able to do that with the POSIX compliant domain ... If nit, it's better to open a bug so this behaviour is investigated further.

Best Regards, Strahil NikolovOn Nov 2, 2019 00:38, TomK <tomkcpr(a)mdevsys.com&gt; wrote: > > On 10/11/2019 12:10 AM, Strahil wrote: >> I'm not sure, but you can try with replica 1 gluster volume. >> Later that gluster volume can be changed to replica 2 arbiter 1 or even better replica 3 volume. >> > > Now that I see this I'm not sure we're on the same page. > > Thinking this might clarify a bit better.  Let's take this scenario, > assume gluster is working perfectly: > > HOST01 > 4TB Local LUN /dev/sdb > 4TB Gluster FS (brick w/ host02,03) /dev/sdc > > > HOST02 > 4TB Local LUN /dev/sdb > 4TB Gluster FS (brick w/ host01,03) /dev/sdc > > > HOST03 > 4TB Local LUN /dev/sdb > 4TB Gluster FS (brick w/ host02,01) /dev/sdc > > > For each host I need the local LUN to be available for VM's with the > GlusterFS storage that's distributed across all three nodes. > > However, when I tried to add a local LUN, the host was removed from the > storage domain rendering it unavailable.  Now when I took out the local > LUN, I was able to readd that host to the GlusterFS storage domain. > > So it appears I can't add a local LUN when a host is already > participating in a GlusterFS storage domain. > > > Cheers, > TK > > >> >> Best Regards, >> Strahil NikolovOn Oct 10, 2019 13:04, Tom <tk(a)mdevsys.com&gt; wrote: >>> >>> I take it then theres no way to do so without NFS? >>> >>> Sent from my iPhone >>> >>>> On Oct 10, 2019, at 2:53 AM, Strahil <hunter86_bg(a)yahoo.com&gt; wrote: >>>> >>>> You can create an NFS on that host and export to itself that local storage... >>>> >>>> Best Regards, >>>> Strahil NikolovOn Oct 9, 2019 07:43, TomK <tomkcpr(a)mdevsys.com&gt; wrote: >>>>> >>>>> Manged to reverse it through the options.  Had to deactivate the custom >>>>> local cluster. >>>>> >>>>> Now I have my GlusterFS back and want to try and add some local storage >>>>> to each host under the same DC, Cluster and Storage Domain.  So >>>>> effectively I will have two storage locations.  One on my GlusterFS and >>>>> the other on the local storage I've just defined. >>>>> >>>>> Appears I can't make them coexist with oVirt.  Either the host is placed >>>>> in a separate cluster with local storage, or I move it back to the >>>>> GlusterFS. >>>>> >>>>> Appears I can't have two usable storage types under one host.  Is this >>>>> correct? >>>>> >>>>> What I'm looking for is in the image and shared earlier. >>>>> >>>>> Cheers, >>>>> TK >>>>> >>>>>> On 10/8/2019 11:03 PM, TomK wrote: >>>>>> I'm working to reverse this scenario. >>>>>> >>>>>> My storage domain still exists.  However I can't activate it.  The >>>>>> Manage button is greyed out without clear indication as to why. >>>>>> Difficult to tell what my next move should be. >>>>>> >>>>>> Anyway to find out? >>>>>> >>>>>> Cheers, >>>>>> TK >>>>>> >>>>>>> On 10/7/2019 8:25 AM, TomK wrote: >>>>>>> Allright. >>>>>>> >>>>>>> So I followed this and configured local storage.  It had some default >>>>>>> names that I changed.  It would have been either the default name, or >>>>>>> one I choose.  In either case it would be a different name then the >>>>>>> Gluster storage domain the host was a part of anyway. >>>>>>> >>>>>>> Once the process completed for both hosts, I noticed the Gluster >>>>>>> volume storage domain was offline and the gluster volume was gone.  I >>>>>>> can't seem to add that back in anymore.  I get: >>>>>>> >>>>>>> "Error while executing action DisconnectStorageServerConnection: Error >>>>>>> storage server disconnection" >>>>>>> >>>>>>> Guessing I can't have local storage in addition to Gluster on the same >>>>>>> hosts and available for VM's? >>>>>>> >>>>>>> Reason why I need that is that Gluster is slow but provides redundancy >>>>>>> and live migration.  But I also wanted direct storage for VM's >>>>>>> requiring faster IO.  So here's my sc

On 11/3/2019 12:52 AM, TomK wrote: > On 11/2/2019 4:07 AM, Strahil wrote: >> You should  be able to do that with the POSIX compliant domain ... >> If nit, it's better to open a bug so this behaviour is investigated >> further. > > I haven't tried the POXIS compliant domain but from what I recall and > remember happening, I couldn't bind a host to more then a single > storage domain.  I could be wrong but I couldn't select more then a > single storage domain a host could be associated with. "Any POSIX compliant file system used as a storage domain in oVirt must be a clustered file system, such as Global File System 2 (GFS2), and must support sparse files and direct I/O. The Common Internet File System (CIFS), for example, does not support direct I/O, making it incompatible with oVirt." Guessing this might not work if I have a Local LUN that's XFS or EXT4?

> > So it was either Local storage or GlusterFS storage, but both couldn't > be bound to a single host. > > Does oVirt have the facilities to test this out to see how it should > be done? > > Thx, > TK > > >> >> Best Regards, >> Strahil NikolovOn Nov 2, 2019 00:38, TomK <tomkcpr(a)mdevsys.com&gt; wrote: >>> >>> On 10/11/2019 12:10 AM, Strahil wrote: >>>> I'm not sure, but you can try with replica 1 gluster volume. >>>> Later that gluster volume can be changed to replica 2 arbiter 1 or >>>> even better replica 3 volume. >>>> >>> >>> Now that I see this I'm not sure we're on the same page. >>> >>> Thinking this might clarify a bit better.  Let's take this scenario, >>> assume gluster is working perfectly: >>> >>> HOST01 >>> 4TB Local LUN /dev/sdb >>> 4TB Gluster FS (brick w/ host02,03) /dev/sdc >>> >>> >>> HOST02 >>> 4TB Local LUN /dev/sdb >>> 4TB Gluster FS (brick w/ host01,03) /dev/sdc >>> >>> >>> HOST03 >>> 4TB Local LUN /dev/sdb >>> 4TB Gluster FS (brick w/ host02,01) /dev/sdc >>> >>> >>> For each host I need the local LUN to be available for VM's with the >>> GlusterFS storage that's distributed across all three nodes. >>> >>> However, when I tried to add a local LUN, the host was removed from the >>> storage domain rendering it unavailable.  Now when I took out the local >>> LUN, I was able to readd that host to the GlusterFS storage domain. >>> >>> So it appears I can't add a local LUN when a host is already >>> participating in a GlusterFS storage domain. >>> >>> >>> Cheers, >>> TK >>> >>> >>>> >>>> Best Regards, >>>> Strahil NikolovOn Oct 10, 2019 13:04, Tom <tk(a)mdevsys.com&gt; wrote: >>>>> >>>>> I take it then theres no way to do so without NFS? >>>>> >>>>> Sent from my iPhone >>>>> >>>>>> On Oct 10, 2019, at 2:53 AM, Strahil <hunter86_bg(a)yahoo.com&gt; wrote: >>>>>> >>>>>> You can create an NFS on that host and export to itself that >>>>>> local storage... >>>>>> >>>>>> Best Regards, >>>>>> Strahil NikolovOn Oct 9, 2019 07:43, TomK <tomkcpr(a)mdevsys.com&gt; >>>>>> wrote: >>>>>>> >>>>>>> Manged to reverse it through the options.  Had to deactivate the >>>>>>> custom >>>>>>> local cluster. >>>>>>> >>>>>>> Now I have my GlusterFS back and want to try and add some local >>>>>>> storage >>>>>>> to each host under the same DC, Cluster and Storage Domain.  So >>>>>>> effectively I will have two storage locations.  One on my >>>>>>> GlusterFS and >>>>>>> the other on the local storage I've just defined. >>>>>>> >>>>>>> Appears I can't make them coexist with oVirt.  Either the host >>>>>>> is placed >>>>>>> in a separate cluster with local storage, or I move it back to the >>>>>>> GlusterFS. >>>>>>> >>>>>>> Appears I can't have two usable storage types under one host. >>>>>>> Is this >>>>>>> correct? >>>>>>> >>>>>>> What I'm looking for is in the image and shared earlier. >>>>>>> >>>>>>> Cheers, >>>>>>> TK >>>>>>> >>>>>>>> On 10/8/2019 11:03 PM, TomK wrote: >>>>>>>> I'm working to reverse this scenario. >>>>>>>> >>>>>>>> My storage domain still exists.  However I can't activate it.  The >>>>>>>> Manage button is greyed out without clear indication as to why. >>>>>>>> Difficult to tell what my next move should be. >>>>>>>> >>>>>>>> Anyway to find out? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> TK >>>>>>>> >>>>>>>>> On 10/7/2019 8:25 AM, TomK wrote: >>>>>>>>> Allright. >>>>>>>>> >>>>>>>>> So I followed this and configured local storage.  It had some >>>>>>>>> default >>>>>>>>> names that I changed.  It would have been either the default >>>>>>>>> name, or >>>>>>>>> one I choose.  In either case it would be a different name >>>>>>>>> then the >>>>>>>>> Gluster storage domain the host was a part of anyway. >>>>>>>>> >>>>>>>>> Once the process completed for both hosts, I noticed the Gluster >>>>>>>>> volume storage domain was offline and the gluster volume was >>>>>>>>> gone.  I >>>>>>>>> can't seem to add that back in anymore.  I get: >>>>>>>>> >>>>>>>>> "Error while executing action >>>>>>>>> DisconnectStorageServerConnection: Error >>>>>>>>> storage server disconnection" >>>>>>>>> >>>>>>>>> Guessing I can't have local storage in addition to Gluster on >>>>>>>>> the same >>>>>>>>> hosts and available for VM's? >>>>>>>>> >>>>>>>>> Reason why I need that is that Gluster is slow but provides >>>>>>>>> redundancy >>>>>>>>> and live migration.  But I also wanted direct storage for VM's >>>>>>>>> requiring faster IO.  So here's my sc > >