[Users] oVirt 3.1 and Glusterfs how-to

[Robert Middleswarth]

On 07/05/2012 02:49 AM, Itamar Heim wrote:
> On 07/05/2012 12:20 AM, Robert Middleswarth wrote:
>> I wrote a basic how to for using glusterfs under oVirt.  If you have any
>> questions or you find mistakes please let me know.
>>
>> Thanks
>> Robert
>>
>> http://www.middleswarth.net/content/installing-ovirt-31-and-glusterfs-using-either-nfs-or-posix-native-file-system 
>>
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
>> oVirt 3.1 is currently in beta and compared to 3.0 it adds a several new
>> features. One of the nice new features is glusterfs support. For it
>> intended use it works well but I personally think it needs to be
>> adjusted to for more generic use. The 2 main limits that effect me are.
>> 1) Only oVirt nodes in the same cluster can be used as bricks to create
>> clusters. So you can't add in storage only nodes in the same oVirt
>> cluster as nodes. However you can create a cluster of just gluster only
>> nodes.
>
> the long term plan would be "dynamic clusters/services" - a subset of 
> hosts with gluster, a subset of hosts with virt, some with both, in 
> overlapping clusters.
> a more simple solution to this would be just allowing to specify at 
> host level if to use the host for gluster service and/or virt service 
> (so still same set of hosts in a cluster, just allowing to choose if 
> they are to be used for the relevant service).
>
A host level option would fix the issue.
> > 2) You can not change what interface / IP gluster uses, it will
>> always use the ovirtmgmt network. This is a weakness as many people have
>> an independent network just for storage and they can't use them with 
>> 3.1.
>
> sorry, i don't understand this one - only the management of gluster is 
> done via this interface. you can define a different logical network in 
> the cluster for storage, configure ip addresses for them, and define 
> the mount point that way.
> (well, apart from potential bugs on network definitions in 3.1 which 
> may still exist)
>

Unless I am missing something if I setup Gluster to use anything other 
then the ovirtmgmt network then I can't create volumes because the 
engine uses the ovirtmgmt networks IP's and Gluster doesn't recognize it.
>>
>> If you can live with these limits then gluster intergration is for you.
>> Before we start you should ask yourself whether you want to use
>> glusterfsover NFS or posix (Native) fs. Both work and are very simlilar
>> in setup but they do work slightly differently. Many prefer the native
>> version and if you are using Fedora 17 with it latest kernal then native
>> (posix) fs actually supports direct IO that will increase the throuput a
>> lot. However I found on CentOS 6.2 nodes NFS ran faster because NFS
>> shares are cached. My testing was over a 1G networks so a faster network
>> will yield different results. Both install methods are pretty much the
>> same I will add notes on there differences.
>
> cache is supposed to be only relevant to read only images like 
> templates, and i'm not even sure this optimization is in.
> so caching is supposed to be disabled (otherwise, shared disk won't 
> work, etc.)
> maybe something else?
>
NFS is faster on my network.  When I looked around to see why it 
suggested that reads were cached.  It was real world testing you never know.
>>
>> A quick word about OS. While there are plans to support EL6 and other
>> Linux based distributions, support is currently limited to Fedora 17.
>> However Andrey Gordeev ( http://www.dreyou.org/ovirt/ ) has created EL6
>> packages. Since Fedora 17 crashes on both my Dells every 10 to 12 hours
>> I use the centos builds right now. Although the CentOS builds work
>> really well there are a few missing features, such as live snapshots. I
>> have done both installs and the steps outlined below work for both.
>>
>
> very nice!