11:58 p.m.
--=-ib3K2M22fij89CwJmOQa
Content-Type: multipart/alternative; boundary="=-NjNRZoVPoPY0Nb552k5L"
--=-NjNRZoVPoPY0Nb552k5L
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Hello,
I am using glusterfs storage and ran into a split-brain issue. One of
the file affected by split-brain was dom_md/ids. In attempts to fix the
split-brain issue I deleted the dom_md/ids file. Is there a method to
recreate or reconstruct this file?
Thanks,
Cameron
--=-NjNRZoVPoPY0Nb552k5L
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
<html><head></head><body><div>Hello,</div><div><br></div><div>I
am using gl=
usterfs storage and ran into a split-brain issue. One of the file affected =
by split-brain was dom_md/ids. In attempts to fix the split-brain issue I d=
eleted the dom_md/ids file. Is there a method to recreate or reconstruct th=
is
file?</div><div><br></div><div>Thanks,</div><div><br></div><div>Cameron<=
/div></body></html>
--=-NjNRZoVPoPY0Nb552k5L--
--=-ib3K2M22fij89CwJmOQa
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJWx4HpAAoJEM1PCzopIAOtyogH/2COr1uz+wnbcJIYDwj+B/pI
LeTdieUk/JxDLzp3eYetcPFTjJ2xxNNKIY4UirRPKxx2cIPIsjyWeTgdPqebkP4n
L9Ol7/y3fmgtoG9vQJ0SPKpXJi1kkZgeCZ9MytFv/nsxPqt43GYllIoV+sya2XbJ
L/Kg801158O6c/AZ8l/enErrT5HgqTqVKVBEnqxKUpM2AjMR8a74qX85IwMtgpyR
cJI18PCYb1/+lJibjK9Q7mdIsmG5mTlHrBnYxjk7MfxdipYmq6S+KW/rXHhypNY5
W8+3z4QChIO0/fmb6gM6ullOmjxvfLkYmOKxI3ktp6nTcB4v5q488Sngc7BAR7s=
=6R5n
-----END PGP SIGNATURE-----
--=-ib3K2M22fij89CwJmOQa--
12:34 a.m.
On Fri, Feb 19, 2016 at 10:58 PM, Cameron Christensen
<cameron.christensen(a)uk2group.com> wrote:
Hello,
I am using glusterfs storage and ran into a split-brain issue. One of the
file affected by split-brain was dom_md/ids. In attempts to fix the
split-brain issue I deleted the dom_md/ids file. Is there a method to
recreate or reconstruct this file?
You can do this:
1. Put the gluster domain to maintenance (via engine)
No host should access it while you reconstruct the ids file
2. Mount the gluster volume manually
mkdir repair
mount -t glusterfs <server>:/<path> repair/
3. Create the file:
touch repair/<sd_uuid>/dom_md/ids
4. Initialize the lockspace
sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0
5. Unmount the gluster volume
umount repair
6. Activate the gluster domain (via engine)
The domain should become active after a while.
David: can you confirm this is the best way to reconstruct the ids file?
Nir
12:46 a.m.
On Fri, Feb 19, 2016 at 11:34:28PM +0200, Nir Soffer wrote:
On Fri, Feb 19, 2016 at 10:58 PM, Cameron Christensen
<cameron.christensen(a)uk2group.com> wrote:
> Hello,
>
> I am using glusterfs storage and ran into a split-brain issue. One of the
> file affected by split-brain was dom_md/ids. In attempts to fix the
> split-brain issue I deleted the dom_md/ids file. Is there a method to
> recreate or reconstruct this file?
You can do this:
1. Put the gluster domain to maintenance (via engine)
No host should access it while you reconstruct the ids file
2. Mount the gluster volume manually
mkdir repair
mount -t glusterfs <server>:/<path> repair/
3. Create the file:
touch repair/<sd_uuid>/dom_md/ids
4. Initialize the lockspace
sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0
5. Unmount the gluster volume
umount repair
6. Activate the gluster domain (via engine)
The domain should become active after a while.
David: can you confirm this is the best way to reconstruct the ids file?
Yes, that looks right.
12:54 a.m.
--=-9iJKPIn2UEV1t7IWMWhA
Content-Type: multipart/alternative; boundary="=-Lc9OxRAgLPK5zoxTEAOY"
--=-Lc9OxRAgLPK5zoxTEAOY
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
I cannot put the gluster domain into maintenance. I'm believe this is
because the data center has a status of Non responsive (because a host
cannot connect to storage or start SPM). The only option available on
the gluster storage is activate. I have put all the hosts into
maintenance. Is this enough to continue with the initialize lockspace
step?
On Fri, 2016-02-19 at 23:34 +0200, Nir Soffer wrote:
On Fri, Feb 19, 2016 at 10:58 PM, Cameron Christensen
<cameron.christensen(a)uk2group.com> wrote:
> Hello,
>=20
> I am using glusterfs storage and ran into a split-brain issue. One
> of the
> file affected by split-brain was dom_md/ids. In attempts to fix the
> split-brain issue I deleted the dom_md/ids file. Is there a method
> to
> recreate or reconstruct this file?
=20
You can do this:
=20
1. Put the gluster domain to maintenance (via engine)
=20
No host should access it while you reconstruct the ids file
=20
2. Mount the gluster volume manually
=20
mkdir repair
mount -t glusterfs <server>:/<path> repair/
=20
3. Create the file:
=20
touch repair/<sd_uuid>/dom_md/ids
=20
4. Initialize the lockspace
=20
sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0
=20
5. Unmount the gluster volume
=20
umount repair
=20
6. Activate the gluster domain (via engine)
=20
The domain should become active after a while.
=20
=20
David: can you confirm this is the best way to reconstruct the ids
file?
=20
Nir
--=-Lc9OxRAgLPK5zoxTEAOY
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
<html><head></head><body><div>I cannot put the gluster
domain into maintena=
nce. I'm believe this is because the data center has a status of Non respon=
sive (because a host cannot connect to storage or start SPM). The only opti=
on available on the gluster storage is activate. I have put all the hosts i=
nto maintenance. Is this enough to continue with the initialize lockspace s=
tep?</div><div><br></div><div>On Fri, 2016-02-19 at 23:34
+0200, Nir Soffer=
wrote:</div><blockquote type=3D"cite"><pre>On Fri, Feb 19,
2016 at 10:58 P=
M, Cameron Christensen
<<a
href=3D"mailto:cameron.christensen@uk2group.com">cameron.christensen=
@uk2group.com</a>> wrote:
<blockquote type=3D"cite">
Hello,
I am using glusterfs storage and ran into a split-brain issue. One of the
file affected by split-brain was dom_md/ids. In attempts to fix the
split-brain issue I deleted the dom_md/ids file. Is there a method to
recreate or reconstruct this file?
</blockquote>
You can do this:
1. Put the gluster domain to maintenance (via engine)
No host should access it while you reconstruct the ids file
2. Mount the gluster volume manually
mkdir repair
mount -t glusterfs <server>:/<path> repair/
3. Create the file:
touch repair/<sd_uuid>/dom_md/ids
4. Initialize the lockspace
sanlock direct init -s
<sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:=
0
5. Unmount the gluster volume
umount repair
6. Activate the gluster domain (via engine)
The domain should become active after a while.
David: can you confirm this is the best way to reconstruct the ids file?
Nir
</pre></blockquote></body></html>
--=-Lc9OxRAgLPK5zoxTEAOY--
--=-9iJKPIn2UEV1t7IWMWhA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJWx48dAAoJEM1PCzopIAOtRCoIAMMmTSeTYTzn6Mn9gSXuCjkI
9MYwuEVDQxyEzb0tnfz0GKOfRrL+h5ICTnk4EcXSxWj3DdD2X6EM5iYpqKkGS/wB
xUwQ8XS9mZqf9LN+kriV7l8bMSLcMw4RCgl36mcrmA4HY5TYj5ZwLdj00K/fMsDk
eE/7nExgnozlnmb83FqZolXkUndi9zItmB+iloVLLmkCjqK2hznrbtdH0g0f4XOm
EZkdqeWSKCJ0g05brn07XqjhGA/XrVDbo4gZoGi0QzQ55nJbRsljgI37o82yFiWY
/o3GQ6YhuvISu0AVeNci/4ntCslMr3wXXOY5IadYIj1HMxLDklR+R38CM5ncDdc=
=8sGo
-----END PGP SIGNATURE-----
--=-9iJKPIn2UEV1t7IWMWhA--
1:24 a.m.
On Fri, Feb 19, 2016 at 11:54 PM, Cameron Christensen
<cameron.christensen(a)uk2group.com> wrote:
I cannot put the gluster domain into maintenance. I'm believe
this is
because the data center has a status of Non responsive (because a host
cannot connect to storage or start SPM). The only option available on the
gluster storage is activate. I have put all the hosts into maintenance. Is
this enough to continue with the initialize lockspace step?
Yes, if all hosts are in maintenance, no host will access the gluster
storage domain, and you can repair is safely.
On Fri, 2016-02-19 at 23:34 +0200, Nir Soffer wrote:
On Fri, Feb 19, 2016 at 10:58 PM, Cameron Christensen
<cameron.christensen(a)uk2group.com> wrote:
Hello,
I am using glusterfs storage and ran into a split-brain issue. One of the
file affected by split-brain was dom_md/ids. In attempts to fix the
split-brain issue I deleted the dom_md/ids file. Is there a method to
recreate or reconstruct this file?
You can do this:
1. Put the gluster domain to maintenance (via engine)
No host should access it while you reconstruct the ids file
2. Mount the gluster volume manually
mkdir repair
mount -t glusterfs <server>:/<path> repair/
3. Create the file:
touch repair/<sd_uuid>/dom_md/ids
4. Initialize the lockspace
sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0
5. Unmount the gluster volume
umount repair
6. Activate the gluster domain (via engine)
The domain should become active after a while.
David: can you confirm this is the best way to reconstruct the ids file?
Nir
1:47 a.m.
--=-Y64HBS9PyTFj3Id7Uol1
Content-Type: multipart/alternative; boundary="=-VOwXxmsV44gjJMcDeOV9"
--=-VOwXxmsV44gjJMcDeOV9
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
I was able to recreate the dom_md/ids file using the provided
instructions. The data center and gluster storage are active. Thank
you.
Cameron
On Sat, 2016-02-20 at 00:24 +0200, Nir Soffer wrote:
On Fri, Feb 19, 2016 at 11:54 PM, Cameron Christensen
<cameron.christensen(a)uk2group.com> wrote:
> I cannot put the gluster domain into maintenance. I'm believe this
> is
> because the data center has a status of Non responsive (because a
> host
> cannot connect to storage or start SPM). The only option available
> on the
> gluster storage is activate. I have put all the hosts into
> maintenance. Is
> this enough to continue with the initialize lockspace step?
=20
Yes, if all hosts are in maintenance, no host will access the gluster
storage domain, and you can repair is safely.
=20
>=20
> On Fri, 2016-02-19 at 23:34 +0200, Nir Soffer wrote:
>=20
> On Fri, Feb 19, 2016 at 10:58 PM, Cameron Christensen
> <cameron.christensen(a)uk2group.com> wrote:
>=20
> Hello,
>=20
> I am using glusterfs storage and ran into a split-brain issue. One
> of the
> file affected by split-brain was dom_md/ids. In attempts to fix the
> split-brain issue I deleted the dom_md/ids file. Is there a method
> to
> recreate or reconstruct this file?
>=20
>=20
> You can do this:
>=20
> 1. Put the gluster domain to maintenance (via engine)
>=20
> No host should access it while you reconstruct the ids file
>=20
> 2. Mount the gluster volume manually
>=20
> mkdir repair
> mount -t glusterfs <server>:/<path> repair/
>=20
> 3. Create the file:
>=20
> touch repair/<sd_uuid>/dom_md/ids
>=20
> 4. Initialize the lockspace
>=20
> sanlock direct init -s <sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:0
>=20
> 5. Unmount the gluster volume
>=20
> umount repair
>=20
> 6. Activate the gluster domain (via engine)
>=20
> The domain should become active after a while.
>=20
>=20
> David: can you confirm this is the best way to reconstruct the ids
> file?
>=20
> Nir
--=-VOwXxmsV44gjJMcDeOV9
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
<html><head></head><body><div>I was able to recreate the
dom_md/ids file us=
ing the provided instructions. The data center and gluster storage are acti=
ve. Thank
you.</div><div><br></div><div>Cameron</div><div><br></div><div>On=
Sat, 2016-02-20 at 00:24 +0200, Nir Soffer wrote:</div><blockquote type=3D=
"cite"><pre>On Fri, Feb 19, 2016 at 11:54 PM, Cameron Christensen
<<a
href=3D"mailto:cameron.christensen@uk2group.com">cameron.christensen=
@uk2group.com</a>> wrote:
<blockquote type=3D"cite">
I cannot put the gluster domain into maintenance. I'm believe this is
because the data center has a status of Non responsive (because a host
cannot connect to storage or start SPM). The only option available on the
gluster storage is activate. I have put all the hosts into maintenance. Is
this enough to continue with the initialize lockspace step?
</blockquote>
Yes, if all hosts are in maintenance, no host will access the gluster
storage domain, and you can repair is safely.
<blockquote type=3D"cite">
On Fri, 2016-02-19 at 23:34 +0200, Nir Soffer wrote:
On Fri, Feb 19, 2016 at 10:58 PM, Cameron Christensen
<<a
href=3D"mailto:cameron.christensen@uk2group.com">cameron.christensen=
@uk2group.com</a>> wrote:
Hello,
I am using glusterfs storage and ran into a split-brain issue. One of the
file affected by split-brain was dom_md/ids. In attempts to fix the
split-brain issue I deleted the dom_md/ids file. Is there a method to
recreate or reconstruct this file?
You can do this:
1. Put the gluster domain to maintenance (via engine)
No host should access it while you reconstruct the ids file
2. Mount the gluster volume manually
mkdir repair
mount -t glusterfs <server>:/<path> repair/
3. Create the file:
touch repair/<sd_uuid>/dom_md/ids
4. Initialize the lockspace
sanlock direct init -s
<sd_uuid>:0:repair/<sd_uuid>/dom_md/ids:=
0
5. Unmount the gluster volume
umount repair
6. Activate the gluster domain (via engine)
The domain should become active after a while.
David: can you confirm this is the best way to reconstruct the ids file?
Nir
</blockquote></pre></blockquote></body></html>
--=-VOwXxmsV44gjJMcDeOV9--
--=-Y64HBS9PyTFj3Id7Uol1
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJWx5t3AAoJEM1PCzopIAOt3FQH/jN7Db9O1r83C4up1XaEZpvn
Iq1cex1iH0TtOtwWmFElA6iW/9wUL3IfD3rgxHFwk+NOMYvidH1towCKww6UucVO
nqF/9mvAiKjS1KD7JG7VOxKrcww2Q7KQMFYSTiaqSOFQGrTlD4Zt2767aYc4Xeln
H9F4tnk21A3ceU9NAWCVRwQi2xiNaRHJCqzCRK1M/wM1EBzCQDRhLBtJka5mbOd9
RrvAuor7zEu64i9bMpnKZc0BwzFFDahNxlmUqT2rs4cU3lwqpgRx66UgoSuQUppx
3Zt/CObzMFqLrqSlDDf39JDyTBwKaGSkJDtHKgXLnPZLArECbMmyPsr+Oted8S8=
=pWig
-----END PGP SIGNATURE-----
--=-Y64HBS9PyTFj3Id7Uol1--
3199
days inactive
3199
days old
5 comments
3 participants
participants (3)
-
Cameron Christensen -
David Teigland -
Nir Soffer