[Users] oVirt 3.1 and Glusterfs how-to

[Robert Middleswarth]

On 07/07/2012 08:44 AM, Itamar Heim wrote:
> On 07/06/2012 05:16 PM, Robert Middleswarth wrote:
>> On 07/06/2012 08:19 AM, Itamar Heim wrote:
>>> On 07/05/2012 10:08 PM, Robert Middleswarth wrote:
>>>> On 07/05/2012 02:19 PM, Itamar Heim wrote:
>>>>> On 07/05/2012 10:06 AM, Robert Middleswarth wrote:
>>>>>>> > 2) You can not change what interface / IP gluster uses, it will
>>>>>>>> always use the ovirtmgmt network. This is a weakness as many 
>>>>>>>> people
>>>>>>>> have
>>>>>>>> an independent network just for storage and they can't use them 
>>>>>>>> with
>>>>>>>> 3.1.
>>>>>>>
>>>>>>> sorry, i don't understand this one - only the management of
>>>>>>> gluster is
>>>>>>> done via this interface. you can define a different logical
>>>>>>> network in
>>>>>>> the cluster for storage, configure ip addresses for them, and 
>>>>>>> define
>>>>>>> the mount point that way.
>>>>>>> (well, apart from potential bugs on network definitions in 3.1 
>>>>>>> which
>>>>>>> may still exist)
>>>>>>>
>>>>>>
>>>>>> Unless I am missing something if I setup Gluster to use anything 
>>>>>> other
>>>>>> then the ovirtmgmt network then I can't create volumes because the
>>>>>> engine uses the ovirtmgmt networks IP's and Gluster doesn't 
>>>>>> recognize
>>>>>> it.
>>>>>
>>>>> engine uses ovirtmgmt to create them.
>>>>> you can define another network to consume them (assuming you have 
>>>>> more
>>>>> than a single nic. otherwise, i'm not sure why it matters).
>>>>>
>>>> Are you suggesting that I add both Network IP into Gluster. That would
>>>> work but how would I know what network Gluster would use to sync up
>>>> with?
>>>
>>> I can think of two items:
>>> 1. network for hosts running VMs to communicate over - I assume will
>>> be based on the url of the export you provide (well, for nfs. for
>>> native/posix, not sure how redirection will work).
>>>
>>> maybe worth adding something similar to the 'display network' -
>>> telling gluster which interface should be provided for clients to
>>> communicate over
>> I use localhost:/volumename so they talk to the localhost no network
>> traffic between the oVirt node and the gluster systems all the traffice
>> is between the gluster nodes.
>>>
>
> I don't think so. it only means the host uses the local gluster 
> service to fetch the data, then uses the network to directly access 
> the node which has the needed volume to run the VM.
I may not have made that clear but that is what I was saying the oVirt 
node talks to gluster either using NFS or Posix FS (Similar code) on the 
same host and allows Gluster to handle what ever network communication 
Gluster needs to do.
>
>>> 2. for communication between gluster nodes for replication, etc.
>>>
>>> maybe worth defining something like 'live migration network' or
>>> 'storage network'[1] - telling nodes which interface they should use
>>> for replication between nodes.
>> I think in the end that is the proper fix.  Defining a storage network.
>> Then when you go to create gluster volumes it would create them using
>> the IP of the storage network.  Also once you get the gluster peer setup
>> cleaned up it would allow the nodes to be joined that way as well.
>
> I think there are two things here:
> - client to gluster communication (for both metadata and data)
> - replication/sync between gluster nodes
>
Correct.  In my setup all the client to gluster communication is done 
over local host.  Gluster node's sync must happen over the ovirtmgnt 
network or I loss the ability to create volumes inside oVirt.  If you 
don't under stand what I mean create a 2 node network.  Setup the 2 
nodes to use Glusterfs.  Configure Gluster using the 2nd set of nics not 
using the ovirtmgmt and then try and create a volume using the volume 
tab.  It will error out because the blocks use the ovirtmgmt interfaces 
IP for creating blocks not the 2nd nic IP so Gluster will error out on 
the block creation.

That is why I said that create of an option for a Gluster Network would 
work.  Since the code would adjust the add brink dialog box to allow a 
diff IP address to be used in Volume creation.

Thanks
Robert
>>>
>>> [1] both storage and live migration network are still not available
>>> for ovirt today, just a concept.
>>> display network is available today.
>>>
>>>>
>>>> Thanks
>>>> Robert
>>>>
>>>
>>>
>>
>>
>
>