[ovirt-users] Persisting glusterfs configs on an oVirt node
Doron Fediuck
dfediuck at redhat.com
Wed May 28 15:16:47 UTC 2014
----- Original Message -----
> From: "Simon Barrett" <Simon.Barrett at tradingscreen.com>
> To: "Fabian Deutsch" <fdeutsch at redhat.com>
> Cc: "Ryan Barry" <rbarry at redhat.com>, "Doron Fediuck" <dfediuck at redhat.com>, users at ovirt.org
> Sent: Wednesday, May 28, 2014 5:22:17 PM
> Subject: RE: [ovirt-users] Persisting glusterfs configs on an oVirt node
>
> I just wasn't sure if I was missing something in the configuration to enable
> this.
>
> I'll stick with the workarounds I have for now and see how it goes.
>
> Thanks again.
>
> Simon
>
> -----Original Message-----
> From: Fabian Deutsch [mailto:fdeutsch at redhat.com]
> Sent: 28 May 2014 15:20
> To: Simon Barrett
> Cc: Ryan Barry; Doron Fediuck; users at ovirt.org
> Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt node
>
> Am Mittwoch, den 28.05.2014, 14:14 +0000 schrieb Simon Barrett:
> > I did a "persist /var/lib/glusterd" and things are looking better. The
> > gluster config is now still in place after a reboot.
> >
> > As a workaround to getting glusterd running on boot, I added "service
> > glusterd start" to /etc/rc.local and ran persist /etc/rc.local. It appears
> > to be working but feels like a bit of a hack.
> >
> > Does anyone have any other suggestions as to the correct way to do this?
>
> Hey Simon,
>
> I was also investigating both the steps you did. And was also about to
> recommend them :) They are more a workaround.
>
> We basically need some patches to change the defaults on Node, to let gluster
> work out of the box.
>
> This would include persisting the correct paths and enabling glusterd if
> enabled.
>
> - fabian
>
> > Thanks,
> >
> > Simon
> >
> > -----Original Message-----
> > From: users-bounces at ovirt.org [mailto:users-bounces at ovirt.org] On
> > Behalf Of Simon Barrett
> > Sent: 28 May 2014 14:12
> > To: Ryan Barry; Fabian Deutsch; Doron Fediuck
> > Cc: users at ovirt.org
> > Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt
> > node
> >
> > Thanks for the replies.
> >
> > I cannot get glusterd to start on boot and I lose all gluster config every
> > reboot.
> >
> > The following shows what I did on the node to start glusterd, create a
> > volume etc, followed by the state of the node after a reboot.
> >
> >
> > [root at ovirt_node]# service glusterd status glusterd is stopped
> >
> > [root at ovirt_node]# chkconfig --list glusterd
> > glusterd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
> >
> > [root at ovirt_node]# service glusterd start Starting glusterd:[ OK ]
> >
> > gluster> volume create vmstore 10.22.8.46:/data/glusterfs/vmstore
> > volume create: vmstore: success: please start the volume to access
> > data
> >
> > gluster> vol start vmstore
> > volume start: vmstore: success
> >
> > gluster> vol info
> > Volume Name: vmstore
> > Type: Distribute
> > Volume ID: 5bd01043-1352-4014-88ca-e632e264d088
> > Status: Started
> > Number of Bricks: 1
> > Transport-type: tcp
> > Bricks:
> > Brick1: 10.22.8.46:/data/glusterfs/vmstore
> >
> > [root at ovirt_node]# ls -1 /var/lib/glusterd/vols/vmstore/ bricks
> > node_state.info trusted-vmstore-fuse.vol cksum
> > rbstate
> > vmstore.10.22.8.46.data-glusterfs-vmstore.vol
> > info
> > run
> > vmstore-fuse.vol
> >
> > [root at ovirt_node]# grep gluster /etc/rwtab.d/*
> > /etc/rwtab.d/ovirt:files /var/lib/glusterd
> >
> > [root at ovirt_node]# chkconfig glusterd on [root at ovirt_node]# chkconfig
> > --list glusterd
> > glusterd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
> >
> >
> > ####################################
> > I then reboot the node and see the following:
> > ####################################
> >
> > [root at ovirt_node]# service glusterd status glusterd is stopped
> >
> > [root at ovirt_node]# chkconfig --list glusterd
> > glusterd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
> >
> > [root at ovirt_node]# ls -l /var/lib/glusterd/vols/ total 0
> >
> > No more gluster volume configuration files.
> >
> > I've taken a look through
> > http://www.ovirt.org/Node_Troubleshooting#Making_changes_last_.2F_Persisting_changes
> > but I'm unsure what needs to be done to persist this configuration.
> >
> > To get glusterd to start on boot, do I need to manually persist /etc/rc*
> > files?
> >
> > I see "files /var/lib/glusterd" mentioned in /etc/rwtab.d/ovirt. Is this a
> > list of the files/dirs that should be persisted automatically? If so, is
> > it recursive and should it include everything in /var/lib/glusterd/vols?
> >
> >
> > TIA for any help with this.
> >
> > Simon
> >
> >
> >
> > -----Original Message-----
> > From: Ryan Barry [mailto:rbarry at redhat.com]
> > Sent: 27 May 2014 14:01
> > To: Fabian Deutsch; Doron Fediuck; Simon Barrett
> > Cc: users at ovirt.org
> > Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt
> > node
> >
> > On 05/26/2014 04:14 AM, Fabian Deutsch wrote:
> > > Am Sonntag, den 25.05.2014, 08:18 -0400 schrieb Doron Fediuck:
> > >>
> > >> ----- Original Message -----
> > >>> From: "Simon Barrett" <Simon.Barrett at tradingscreen.com>
> > >>> To: users at ovirt.org
> > >>> Sent: Friday, May 23, 2014 11:29:39 AM
> > >>> Subject: [ovirt-users] Persisting glusterfs configs on an oVirt
> > >>> node
> > >>>
> > >>>
> > >>>
> > >>> I am working through the setup of oVirt node for a 3.4.1 deployment.
> > >>>
> > >>>
> > >>>
> > >>> I setup some glusterfs volumes/bricks on oVirt Node Hypervisor
> > >>> release 3.0.4
> > >>> (1.0.201401291204.el6) and created a storage domain. All was
> > >>> working OK until I rebooted the node and found that the glusterfs
> > >>> configuration had not been retained.
> > >>>
> > >>>
> > >>>
> > >>> Is there something I should be doing to persist any glusterfs
> > >>> configuration so it survives a node reboot?
> > >>>
> > >>>
> > >>>
> > >>> Many thanks,
> > >>>
> > >>>
> > >>>
> > >>> Simon
> > >>>
> > >>
> > >> Hi Simon,
> > >> it actually sounds like a bug to me, as node are supposed to
> > >> support gluster.
> > >>
> > >> Ryan / Fabian- thoughts?
> > >
> > > Hey,
> > >
> > > I vaguely remember that we were seeing a bug like this some time ago.
> > > We fixed /var/lib/glusterd to be writable (using tmpfs), but it can
> > > actually be that we need to persist those contents.
> > >
> > > But Simon, can you give details which configuration files are
> > > missing and why glusterd is not starting?
> > Is glusterd starting? I'm getting the impression that it's starting, but
> > that it has no configuration. As far as I know, Gluster keeps most of the
> > configuration on the brick itself, but it finds brick information in
> > /var/lib/glusterd.
> >
> > The last patch simply opened the firewall, and it's entirely possible that
> > we need to persist this. It may be a good idea to just persist the entire
> > directory from the get-go, unless we want to try to have a thread watching
> > /var/lib/glusterd for relevant files, but then we're stuck trying to keep
> > up with what's happening with gluster itself...
> >
> > Can we
> > >
> > > Thanks
> > > fabian
> > >
> > >> Either way I suggest you take a look in the below link-
> > >> http://www.ovirt.org/Node_Troubleshooting#Making_changes_last_.2F_P
> > >> er
> > >> sisting_changes
> > >>
> > >> Let s know how it works.
> > >>
> > >> Doron
> > >
> > >
I suggest you open a bug so we'll be able to track this issue properly
rather than using hacks.
Thanks,
Doron
More information about the Users
mailing list