Re: [ovirt-users] Persisting glusterfs configs on an oVirt node

I did a "persist /var/lib/glusterd" and things are looking better. The gluster config is now still in place after a reboot. As a workaround to getting glusterd running on boot, I added "service glusterd start" to /etc/rc.local and ran persist /etc/rc.local. It appears to be working but feels like a bit of a hack. Does anyone have any other suggestions as to the correct way to do this?

Thanks, Simon -----Original Message----- From: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] On Behalf Of Simon Barrett Sent: 28 May 2014 14:12 To: Ryan Barry; Fabian Deutsch; Doron Fediuck Cc: users(a)ovirt.org Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt node Thanks for the replies. I cannot get glusterd to start on boot and I lose all gluster config every reboot. The following shows what I did on the node to start glusterd, create a volume etc, followed by the state of the node after a reboot. [root@ovirt_node]# service glusterd status glusterd is stopped [root@ovirt_node]# chkconfig --list glusterd glusterd 0:off 1:off 2:off 3:off 4:off 5:off 6:off [root@ovirt_node]# service glusterd start Starting glusterd:[ OK ] gluster> volume create vmstore 10.22.8.46:/data/glusterfs/vmstore volume create: vmstore: success: please start the volume to access data gluster> vol start vmstore volume start: vmstore: success gluster> vol info Volume Name: vmstore Type: Distribute Volume ID: 5bd01043-1352-4014-88ca-e632e264d088 Status: Started Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: 10.22.8.46:/data/glusterfs/vmstore [root@ovirt_node]# ls -1 /var/lib/glusterd/vols/vmstore/ bricks node_state.info trusted-vmstore-fuse.vol cksum rbstate vmstore.10.22.8.46.data-glusterfs-vmstore.vol info run vmstore-fuse.vol [root@ovirt_node]# grep gluster /etc/rwtab.d/* /etc/rwtab.d/ovirt:files /var/lib/glusterd [root@ovirt_node]# chkconfig glusterd on [root@ovirt_node]# chkconfig --list glusterd glusterd 0:off 1:off 2:on 3:on 4:on 5:on 6:off #################################### I then reboot the node and see the following: #################################### [root@ovirt_node]# service glusterd status glusterd is stopped [root@ovirt_node]# chkconfig --list glusterd glusterd 0:off 1:off 2:off 3:off 4:off 5:off 6:off [root@ovirt_node]# ls -l /var/lib/glusterd/vols/ total 0 No more gluster volume configuration files. I've taken a look through http://www.ovirt.org/Node_Troubleshooting#Making_changes_last_.2F_Persist... but I'm unsure what needs to be done to persist this configuration. To get glusterd to start on boot, do I need to manually persist /etc/rc* files? I see "files /var/lib/glusterd" mentioned in /etc/rwtab.d/ovirt. Is this a list of the files/dirs that should be persisted automatically? If so, is it recursive and should it include everything in /var/lib/glusterd/vols? TIA for any help with this. Simon -----Original Message----- From: Ryan Barry [mailto:rbarry@redhat.com] Sent: 27 May 2014 14:01 To: Fabian Deutsch; Doron Fediuck; Simon Barrett Cc: users(a)ovirt.org Subject: Re: [ovirt-users] Persisting glusterfs configs on an oVirt node On 05/26/2014 04:14 AM, Fabian Deutsch wrote: > Am Sonntag, den 25.05.2014, 08:18 -0400 schrieb Doron Fediuck: >> >> ----- Original Message ----- >>> From: "Simon Barrett" <Simon.Barrett(a)tradingscreen.com&gt; >>> To: users(a)ovirt.org >>> Sent: Friday, May 23, 2014 11:29:39 AM >>> Subject: [ovirt-users] Persisting glusterfs configs on an oVirt node >>> >>> >>> >>> I am working through the setup of oVirt node for a 3.4.1 deployment. >>> >>> >>> >>> I setup some glusterfs volumes/bricks on oVirt Node Hypervisor >>> release 3.0.4 >>> (1.0.201401291204.el6) and created a storage domain. All was working >>> OK until I rebooted the node and found that the glusterfs >>> configuration had not been retained. >>> >>> >>> >>> Is there something I should be doing to persist any glusterfs >>> configuration so it survives a node reboot? >>> >>> >>> >>> Many thanks, >>> >>> >>> >>> Simon >>> >> >> Hi Simon, >> it actually sounds like a bug to me, as node are supposed to support >> gluster. >> >> Ryan / Fabian- thoughts? > > Hey, > > I vaguely remember that we were seeing a bug like this some time ago. > We fixed /var/lib/glusterd to be writable (using tmpfs), but it can > actually be that we need to persist those contents. > > But Simon, can you give details which configuration files are missing > and why glusterd is not starting? Is glusterd starting? I'm getting the impression that it's starting, but that it has no configuration. As far as I know, Gluster keeps most of the configuration on the brick itself, but it finds brick information in /var/lib/glusterd. The last patch simply opened the firewall, and it's entirely possible that we need to persist this. It may be a good idea to just persist the entire directory from the get-go, unless we want to try to have a thread watching /var/lib/glusterd for relevant files, but then we're stuck trying to keep up with what's happening with gluster itself... Can we > > Thanks > fabian > >> Either way I suggest you take a look in the below link- >> http://www.ovirt.org/Node_Troubleshooting#Making_changes_last_.2F_Per >> sisting_changes >> >> Let s know how it works. >> >> Doron > > _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users