Re: [ovirt-users] Using gluster on other hosts?

From: "Sahina Bose" <sabose(a)redhat.com&gt; To: "Kaushal M" <kaushal(a)redhat.com&gt; Cc: "Will K" <yetanotherwill(a)yahoo.com&gt;, users(a)ovirt.org Sent: Friday, 9 January, 2015 1:47:17 PM Subject: Re: [ovirt-users] Using gluster on other hosts? On 01/09/2015 01:13 PM, Kaushal M wrote: > Hey Will, > > It seems to me you are trying manage GlusterFS from oVirt, and trying to > get your multi-network setup to work. As Sahina mentioned already, this is > not currently possible as oVirt doesn't have the required support. > > If you want to make this work right now, I suggest you manage GlusterFS > manually. You could do the following, > > - Install GlusterFS on both the hosts and setup a GlusterFS trusted storage > pool using the 'gluster peer probe' commands. Run 'gluster peer probe > gfs2' from node1 (and the reverse just for safety) > - Create a GlusterFS volume, 'gluster volume create <VOLUMENAME> > gfs1:<BRICKPATH> gfs2:<BRICKPATH>; and start it, 'gluster volume start > <VOLUMENAME>'. > After this you'll have GlusterFS setup on the particular network and you'll > have volume ready to be added as a oVirt storage domain. To enable, oVirt to use the node1 interface, is it possible to peer probe using node1 and node2 interface in steps above - i.e "gluster peer probe node2" (This is essentially what happens when a host is added with host address node1 or node2) and then create a GlusterFS volume from CLI using the command you mentioned above? > - Now setup oVirt on the nodes with the node* network. > - Add the gfs* network to oVirt. I'm not sure if this would be required, > but you can try it anyway. > - Add the created GlusterFS volume as a storage domain using a gfs* > address. > > You should now be ready to begin using the new storage domain. > > If you would want to expand the volume later, you will need to do it > manually with an explicit 'gluster volume add-brick' command. > > You could possible add the GlusterFS cluster to the oVirt interface, just > so you can get stats and monitoring. But even then you shouldn't use the > oVirt interface to do any management tasks. > > Multi-network support for GlusterFS within oVirt is an upcoming feature, > and Sahina can give you more details on when to expect it to be available. > > Thanks, > Kaushal > > > ----- Original Message ----- >> From: "Sahina Bose" <sabose(a)redhat.com&gt; >> To: "Will K" <yetanotherwill(a)yahoo.com&gt;, users(a)ovirt.org, "Kaushal M" >> <kaushal(a)redhat.com&gt; >> Sent: Friday, 9 January, 2015 11:10:48 AM >> Subject: Re: [ovirt-users] Using gluster on other hosts? >> >> >> On 01/08/2015 09:41 PM, Will K wrote: >>> That's what I did, but didn't work for me. >>> >>> 1. use the 192.168.x interface to setup gluster. I used hostname in >>> /etc/hosts. >>> 2. setup oVirt using the switched network hostnames, let's say 10.10.10.x >>> 3. oVirt and all that comes up fine. >>> 4. When try to create a storage domain, it only shows the 10.10.10.x >>> hostnames available. >>> >>> >>> Tried to add a brick and I would get something like >>> Host gfs2 is not in 'Peer in Cluster' state (while node2 is the >>> hostname and gfs2 is the 192.168 name) >> >> Which version of glusterfs do you have? >> >> Kaushal, will this work in glusterfs3.6 and above? >> >> >>> Ran command `gluster probe peer gfs2` or `gluster probe peer >>> 192.168.x.x` didn't work >>> peer probe: failed: Probe returned with unknown errno 107 >>> >>> Ran probe again with the switched network hostname or IP worked fine. >>> May be it is not possible with current GlusterFS version? >>> http://www.gluster.org/community/documentation/index.php/Features/SplitNe... >>> >>> >>> Will >>> >>> >>> On Thursday, January 8, 2015 3:43 AM, Sahina Bose <sabose(a)redhat.com&gt; >>> wrote: >>> >>> >>> >>> On 01/08/2015 12:07 AM, Will K wrote: >>>> Hi >>>> >>>> I would like to see if anyone has good suggestion. >>>> >>>> I have two physical hosts with 1GB connections to switched networks. >>>> The hosts also have 10GB interface connected directly using Twinax >>>> cable like copper crossover cable. The idea was to use the 10GB as a >>>> "private network" for GlusterFS till the day we want to grow out of >>>> this 2 node setup. >>>> >>>> GlusterFS was setup with the 10GB ports using non-routable IPs and >>>> hostnames in /etc/hosts, for example, gfs1 192.168.1.1 and gfs2 >>>> 192.168.1.2. I'm following example from >>>> community.redhat.com/blog/2014/10/up-and-running-with-ovirt-3-5/ >>>> <http://community.redhat.com/blog/2014/10/up-and-running-with-ovirt-3-5/> >>>> , Currently I'm only using Gluster volume on node1, but `gluster >>>> probe peer` test worked fine with node2 through the 10GB connection. >>>> >>>> oVirt engine was setup on physical host1 with hosted engine. Now, >>>> when I try to create new Gluster storage domain, I can only see the >>>> host "node1" available. >>>> >>>> Is there anyway I can setup oVirt on node1 and node2, while using >>>> "gfs1" and "gfs2" for GlusterFS? or some way to take advantage of the >>>> 10GB connection? >>> If I understand right, you have 2 interfaces on each of your hosts, >>> and you want oVirt to communicate via 1 interface and glusterfs to use >>> other? >>> >>> While adding the hosts to oVirt you could use ip1 and then.while >>> creating the volume, add the brick using the other ip address. >>> For instance, gluster volume create <volname> 192.168.1.2:/bricks/b1 >>> >>> Currently, there's no way to specify the IP address to use while >>> adding a brick from oVirt UI (we're working on this for 3.6), but you >>> could do this from the gluster CLI commands. This would then be >>> detected in the oVirt UI. >>> >>> >>> >>>> Thanks >>>> W >>>> >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users(a)ovirt.org <mailto:Users@ovirt.org> >>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>> >>