Re: [ovirt-users] Persisting glusterfs configs on an oVirt node

From: "Simon Barrett" <Simon.Barrett(a)tradingscreen.com&gt; To: "Fabian Deutsch" <fdeutsch(a)redhat.com&gt; Cc: "Ryan Barry" <rbarry(a)redhat.com&gt;, "Doron Fediuck" <dfediuck(a)redhat.com&gt;, users(a)ovirt.org Sent: Wednesday, May 28, 2014 5:22:17 PM Subject: RE: [ovirt-users] Persisting glusterfs configs on an oVirt node I just wasn't sure if I was missing something in the configuration to enable this. I'll stick with the workarounds I have for now and see how it goes. Thanks again. Simon -----Original Message----- From: Fabian Deutsch [mailto:fdeutsch@redhat.com] Sent: 28 May 2014 15:20 To: Simon Barrett Cc: Ryan Barry; Doron Fediuck; users(a)ovirt.org Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt node Am Mittwoch, den 28.05.2014, 14:14 +0000 schrieb Simon Barrett: > I did a "persist /var/lib/glusterd" and things are looking better. > The gluster config is now still in place after a reboot. > > As a workaround to getting glusterd running on boot, I added > "service glusterd start" to /etc/rc.local and ran persist > /etc/rc.local. It appears to be working but feels like a bit of a hack. > > Does anyone have any other suggestions as to the correct way to do this? Hey Simon, I was also investigating both the steps you did. And was also about to recommend them :) They are more a workaround. We basically need some patches to change the defaults on Node, to let gluster work out of the box. This would include persisting the correct paths and enabling glusterd if enabled. - fabian > Thanks, > > Simon > > -----Original Message----- > From: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] On > Behalf Of Simon Barrett > Sent: 28 May 2014 14:12 > To: Ryan Barry; Fabian Deutsch; Doron Fediuck > Cc: users(a)ovirt.org > Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt > node > > Thanks for the replies. > > I cannot get glusterd to start on boot and I lose all gluster config > every reboot. > > The following shows what I did on the node to start glusterd, create > a volume etc, followed by the state of the node after a reboot. > > > [root@ovirt_node]# service glusterd status glusterd is stopped > > [root@ovirt_node]# chkconfig --list glusterd > glusterd 0:off 1:off 2:off 3:off 4:off 5:off 6:off > > [root@ovirt_node]# service glusterd start Starting glusterd:[ OK ] > > gluster> volume create vmstore 10.22.8.46:/data/glusterfs/vmstore > volume create: vmstore: success: please start the volume to access > data > > gluster> vol start vmstore > volume start: vmstore: success > > gluster> vol info > Volume Name: vmstore > Type: Distribute > Volume ID: 5bd01043-1352-4014-88ca-e632e264d088 > Status: Started > Number of Bricks: 1 > Transport-type: tcp > Bricks: > Brick1: 10.22.8.46:/data/glusterfs/vmstore > > [root@ovirt_node]# ls -1 /var/lib/glusterd/vols/vmstore/ bricks > node_state.info trusted-vmstore-fuse.vol cksum rbstate > vmstore.10.22.8.46.data-glusterfs-vmstore.vol > info > run > vmstore-fuse.vol > > [root@ovirt_node]# grep gluster /etc/rwtab.d/* > /etc/rwtab.d/ovirt:files /var/lib/glusterd > > [root@ovirt_node]# chkconfig glusterd on [root@ovirt_node]# > chkconfig --list glusterd > glusterd 0:off 1:off 2:on 3:on 4:on 5:on 6:off > > > #################################### > I then reboot the node and see the following: > #################################### > > [root@ovirt_node]# service glusterd status glusterd is stopped > > [root@ovirt_node]# chkconfig --list glusterd > glusterd 0:off 1:off 2:off 3:off 4:off 5:off 6:off > > [root@ovirt_node]# ls -l /var/lib/glusterd/vols/ total 0 > > No more gluster volume configuration files. > > I've taken a look through > http://www.ovirt.org/Node_Troubleshooting#Making_changes_last_.2F_Pe > rsisting_changes but I'm unsure what needs to be done to persist > this configuration. > > To get glusterd to start on boot, do I need to manually persist > /etc/rc* files? > > I see "files /var/lib/glusterd" mentioned in /etc/rwtab.d/ovirt. Is > this a list of the files/dirs that should be persisted > automatically? If so, is it recursive and should it include everything in /var/lib/glusterd/vols? > > > TIA for any help with this. > > Simon > > > > -----Original Message----- > From: Ryan Barry [mailto:rbarry@redhat.com] > Sent: 27 May 2014 14:01 > To: Fabian Deutsch; Doron Fediuck; Simon Barrett > Cc: users(a)ovirt.org > Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt > node > > On 05/26/2014 04:14 AM, Fabian Deutsch wrote: > > Am Sonntag, den 25.05.2014, 08:18 -0400 schrieb Doron Fediuck: > >> > >> ----- Original Message ----- > >>> From: "Simon Barrett" <Simon.Barrett(a)tradingscreen.com&gt; > >>> To: users(a)ovirt.org > >>> Sent: Friday, May 23, 2014 11:29:39 AM > >>> Subject: [ovirt-users] Persisting glusterfs configs on an oVirt > >>> node > >>> > >>> > >>> > >>> I am working through the setup of oVirt node for a 3.4.1 deployment. > >>> > >>> > >>> > >>> I setup some glusterfs volumes/bricks on oVirt Node Hypervisor > >>> release 3.0.4 > >>> (1.0.201401291204.el6) and created a storage domain. All was > >>> working OK until I rebooted the node and found that the > >>> glusterfs configuration had not been retained. > >>> > >>> > >>> > >>> Is there something I should be doing to persist any glusterfs > >>> configuration so it survives a node reboot? > >>> > >>> > >>> > >>> Many thanks, > >>> > >>> > >>> > >>> Simon > >>> > >> > >> Hi Simon, > >> it actually sounds like a bug to me, as node are supposed to > >> support gluster. > >> > >> Ryan / Fabian- thoughts? > > > > Hey, > > > > I vaguely remember that we were seeing a bug like this some time ago. > > We fixed /var/lib/glusterd to be writable (using tmpfs), but it > > can actually be that we need to persist those contents. > > > > But Simon, can you give details which configuration files are > > missing and why glusterd is not starting? > Is glusterd starting? I'm getting the impression that it's starting, > but that it has no configuration. As far as I know, Gluster keeps > most of the configuration on the brick itself, but it finds brick > information in /var/lib/glusterd. > > The last patch simply opened the firewall, and it's entirely > possible that we need to persist this. It may be a good idea to just > persist the entire directory from the get-go, unless we want to try > to have a thread watching /var/lib/glusterd for relevant files, but > then we're stuck trying to keep up with what's happening with gluster itself... > > Can we > > > > Thanks > > fabian > > > >> Either way I suggest you take a look in the below link- > >> http://www.ovirt.org/Node_Troubleshooting#Making_changes_last_.2F > >> _P > >> er > >> sisting_changes > >> > >> Let s know how it works. > >> > >> Doron > > > >