[ovirt-users] Using gluster on other hosts?

Fri Jan 9 08:17:17 UTC 2015

On 01/09/2015 01:13 PM, Kaushal M wrote:
> Hey Will,
>
> It seems to me you are trying manage GlusterFS from oVirt, and trying to get your multi-network setup to work. As Sahina mentioned already, this is not currently possible as oVirt doesn't have the required support.
>
> If you want to make this work right now, I suggest you manage GlusterFS manually. You could do the following,
>
> - Install GlusterFS on both the hosts and setup a GlusterFS trusted storage pool using the 'gluster peer probe' commands. Run 'gluster peer probe gfs2' from node1 (and the reverse just for safety)
> - Create a GlusterFS volume, 'gluster volume create <VOLUMENAME> gfs1:<BRICKPATH> gfs2:<BRICKPATH>; and start it, 'gluster volume start <VOLUMENAME>'.
> After this you'll have GlusterFS setup on the particular network and you'll have volume ready to be added as a oVirt storage domain.

To enable, oVirt to use the node1 interface, is it possible to peer 
probe using node1 and node2 interface in steps above - i.e "gluster peer 
probe node2" (This is essentially what happens when a host is added with 
host address node1 or node2)

and then create a GlusterFS volume from CLI using the command you 
mentioned above?

> - Now setup oVirt on the nodes with the node* network.
> - Add the gfs* network to oVirt. I'm not sure if this would be required, but you can try it anyway.
> - Add the created GlusterFS volume as a storage domain using a gfs* address.
>
> You should now be ready to begin using the new storage domain.
>
> If you would want to expand the volume later, you will need to do it manually with an explicit 'gluster volume add-brick' command.
>
> You could possible add the GlusterFS cluster to the oVirt interface, just so you can get stats and monitoring. But even then you shouldn't use the oVirt interface to do any management tasks.
>
> Multi-network support for GlusterFS within oVirt is an upcoming feature, and Sahina can give you more details on when to expect it to be available.
>
> Thanks,
> Kaushal
>
>
> ----- Original Message -----
>> From: "Sahina Bose" <sabose at redhat.com>
>> To: "Will K" <yetanotherwill at yahoo.com>, users at ovirt.org, "Kaushal M" <kaushal at redhat.com>
>> Sent: Friday, 9 January, 2015 11:10:48 AM
>> Subject: Re: [ovirt-users] Using gluster on other hosts?
>>
>>
>> On 01/08/2015 09:41 PM, Will K wrote:
>>> That's what I did, but didn't work for me.
>>>
>>> 1. use the 192.168.x interface to setup gluster. I used hostname in
>>> /etc/hosts.
>>> 2. setup oVirt using the switched network hostnames, let's say 10.10.10.x
>>> 3. oVirt and all that comes up fine.
>>> 4. When try to create a storage domain, it only shows the 10.10.10.x
>>> hostnames available.
>>>
>>>
>>> Tried to add a brick and I would get something like
>>>      Host gfs2 is not in 'Peer in Cluster' state  (while node2 is the
>>> hostname and gfs2 is the 192.168 name)
>>
>> Which version of glusterfs do you have?
>>
>> Kaushal, will this work in glusterfs3.6 and above?
>>
>>
>>> Ran command `gluster probe peer gfs2` or `gluster probe peer
>>> 192.168.x.x` didn't work
>>>      peer probe: failed: Probe returned with unknown errno 107
>>>
>>> Ran probe again with the switched network hostname or IP worked fine.
>>> May be it is not possible with current GlusterFS version?
>>> http://www.gluster.org/community/documentation/index.php/Features/SplitNetwork
>>>
>>>
>>> Will
>>>
>>>
>>> On Thursday, January 8, 2015 3:43 AM, Sahina Bose <sabose at redhat.com>
>>> wrote:
>>>
>>>
>>>
>>> On 01/08/2015 12:07 AM, Will K wrote:
>>>> Hi
>>>>
>>>> I would like to see if anyone has good suggestion.
>>>>
>>>> I have two physical hosts with 1GB connections to switched networks.
>>>> The hosts also have 10GB interface connected directly using Twinax
>>>> cable like copper crossover cable.  The idea was to use the 10GB as a
>>>> "private network" for GlusterFS till the day we want to grow out of
>>>> this 2 node setup.
>>>>
>>>> GlusterFS was setup with the 10GB ports using non-routable IPs and
>>>> hostnames in /etc/hosts, for example, gfs1 192.168.1.1 and gfs2
>>>> 192.168.1.2.  I'm following example from
>>>> community.redhat.com/blog/2014/10/up-and-running-with-ovirt-3-5/
>>>> <http://community.redhat.com/blog/2014/10/up-and-running-with-ovirt-3-5/>
>>>> , Currently I'm only using Gluster volume on node1, but `gluster
>>>> probe peer` test worked fine with node2 through the 10GB connection.
>>>>
>>>> oVirt engine was setup on physical host1 with hosted engine.  Now,
>>>> when I try to create new Gluster storage domain, I can only see the
>>>> host "node1" available.
>>>>
>>>> Is there anyway I can setup oVirt on node1 and node2, while using
>>>> "gfs1" and "gfs2" for GlusterFS? or some way to take advantage of the
>>>> 10GB connection?
>>> If I understand right, you have 2 interfaces on each of your hosts,
>>> and you want oVirt to communicate via 1 interface and glusterfs to use
>>> other?
>>>
>>> While adding the hosts to oVirt you could use ip1 and then.while
>>> creating the volume, add the brick using the other ip address.
>>> For instance, gluster volume create <volname> 192.168.1.2:/bricks/b1
>>>
>>> Currently, there's no way to specify the IP address to use while
>>> adding a brick from oVirt UI (we're working on this for 3.6), but you
>>> could do this from the gluster CLI commands. This would then be
>>> detected in the oVirt UI.
>>>
>>>
>>>
>>>> Thanks
>>>> W
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org  <mailto:Users at ovirt.org>
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>>