Re: [ovirt-users] Cannot activate storage domain

[+Sas - thanks for the link to virt-store usecase article inline] On 12/18/2014 06:56 PM, Brent Hartzell wrote: > Hello, > > I had actually gotten this sorted out, somewhat. If I disable server > quorum > on the volume, the storage domain will activate. The volume is/was > optimized > for virt store via oVirt. The brick in question was not the first brick > added to the volume through oVirt however, it appears that it may > have been > the first brick in the replica being used, but I'm not certain how to > find > this out. The recommended setting is to have both client and server side quorum turned on. But turning on server-side quorum with a 2-way replica volume would mean that your volume goes offline when one of the bricks goes down. "gluster volume info" command will give you information about the volume topology. So will the bricks sub-tab for Volume in oVirt. The order in which the bricks are listed, is the order of the replica sets. > Disabling quorum allowed me to get the VM's affected back online > however, is > this the recommended procedure? I tried to use replace-brick with > another > node but it failed because the failed brick was not available. Would we > leave quorum disabled until that brick gets replaced? IE - rebuild the > server with the same hostname/IP file structure and rebalance the > cluster? http://www.gluster.org/community/documentation/index.php/Virt-store-usecase - for recommendations on volume tunables. You could add another brick to your volume to make it a replica 3 and then turn on quorum? For help on recovering your volume, I suggest you write to gluster-users(a)gluster.org > > //// > > While that happened, I read somewhere about this happening with a > replica 2 > - I've created a new volume with replica 3 and plan to test this > again. Is > there any info you can point me to for how to handle this when it > happens or > what the correct procedure is when a "first" brick fails? > > > -----Original Message----- > From: Sahina Bose [mailto:sabose@redhat.com] > Sent: Thursday, December 18, 2014 3:51 AM > To: Vered Volansky; Brent Hartzell > Cc: users(a)ovirt.org > Subject: Re: [ovirt-users] Cannot activate storage domain > > > On 12/18/2014 01:35 PM, Vered Volansky wrote: >> Adding Sahina. >> >> ----- Original Message ----- >>> From: "Brent Hartzell" <brent.hartzell(a)outlook.com&gt; >>> To: users(a)ovirt.org >>> Sent: Thursday, December 18, 2014 3:38:11 AM >>> Subject: [ovirt-users] Cannot activate storage domain >>> >>> >>> >>> Have the following: >>> >>> >>> >>> 6 hosts - virt + Gluster shared >>> >>> >>> >>> Gluster volume is distributed-replicate - replica 2 >>> >>> >>> >>> Shutting down servers one at a time all work except for 1 brick. If >>> we shut down one specific brick (1 brick per host) - we're unable to >>> activate the storage domain. VM's that were actively running from >>> other bricks continue to run. Whatever was running form that specific >>> brick fails to run, gets paused etc. >>> >>> >>> >>> Error log shows the entry below. I'm not certain what it's saying is >>> read only.nothing is read only that I can find. >>> >>> >>> >>> >>> >>> 2014-12-17 19:57:13,362 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] >>> (DefaultQuartzScheduler_Worker-47) [4e9290a2] Command >>> SpmStatusVDSCommand(HostName = U23.domainame.net, HostId = >>> 0db58e46-68a3-4ba0-a8aa-094893c045a1, storagePoolId = >>> 7ccd6ea9-7d80-4170-afa1-64c10c185aa6) execution failed. Exception: >>> VDSErrorException: VDSGenericException: VDSErrorException: Failed to >>> SpmStatusVDS, error = [Errno 30] Read-only file system, code = 100 >>> >>> 2014-12-17 19:57:13,363 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] >>> (DefaultQuartzScheduler_Worker-47) [4e9290a2] >>> hostFromVds::selectedVds - U23.domainname.net, spmStatus returned >>> null! >>> >>> >>> >>> >>> >>> According to Ovirt/Gluster, if a brick goes down, the VM should be >>> able to be restarted from another brick without issue. This does not >>> appear to be the case. If we take other bricks offline, it appears to > work as expected. >>> Something with this specific brick cases everything to break which >>> then makes any VM's that were running from the brick unable to start. > Do you have the recommended options for using volume as virt store > turned > on? Is client-side quorum turned on for the volume? Is the brick that > causes > the issue, the first brick in the replica set? > > >>> _______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >>>