GlusterFS Centos 7 unable to mount NFS on gluster members

13 Jan 2015

      This is a multipart message in MIME format.

------=_NextPart_000_0007_01D02EA2.D9347DE0
Content-Type: text/plain;
	charset="koi8-r"
Content-Transfer-Encoding: 7bit

Here is a quick rundown of the system, and the problem.

All hosts on centOS 7 fully up to date

GlusterFS is running on 6 servers, 3x2 distribute/replicate. 

CTDB is running on all hosts

I am unable to mount via nfs the exported volume on any of the gluster
server members.

I am able to mount, read, write, umount from any server that is not a
gluster member. 

Topology - all are hostnames that are resolvable 

Gluster Members

Node1

Node2

Node3

Node4

Node5

Node6

CTDB Virtual IP/Hostname

SharedNFS

Test Machine

Test1

Gluster Volumes

Engine

Data

I am trying to bring up hosted-engine using nfs using the gluster members

I run hosted-engine -deploy

Nfsv3

Host:/path sharednfs:/engine

Error while mounting specified storage path: mount.nfs: an incorrect mount
option was specified

Ok well lets try that without using the hosted-engine script

Mount -v -t nfs -o vers=3 sharednfs:/engine /tmp

mount.nfs: timeout set for Mon Jan 12 22:00:08 2015

mount.nfs: trying text-based options 'vers=3,lock=Flase,addr=192.168.0.240

mount.nfs: prog 100003, trying vers=3, prot=6

mount.nfs: trying 192.168.0.240 prog 100003 vers 3 prot TCP port 2049

mount.nfs: prog 100005, trying vers=3, prot=17

mount.nfs: portmap query retrying: RPC: Program not registered

mount.nfs: prog 100005, trying vers=3, prot=6

mount.nfs: trying 192.168.0.240 prog 100005 vers 3 prot TCP port 38465

mount.nfs: mount(2): Invalid argument

mount.nfs: an incorrect mount option was specified

[root@node4 ~]# systemctl status rpcbind

rpcbind.service - RPC bind service

   Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; enabled)

   Active: active (running) since Mon 2015-01-12 20:01:13 EST; 1h 57min ago

  Process: 1349 ExecStart=/sbin/rpcbind -w ${RPCBIND_ARGS} (code=exited,
status=0/SUCCESS)

Main PID: 1353 (rpcbind)

   CGroup: /system.slice/rpcbind.service

           L-1353 /sbin/rpcbind -w

Jan 12 20:01:13 node4 systemd[1]: Starting RPC bind service...

Jan 12 20:01:13 node4 systemd[1]: Started RPC bind service.

Jan 12 21:19:22 node4 systemd[1]: Started RPC bind service.

Ummm: this makes no sense:. 

[root@test1 ~]# mount -v -o vers=3 -t nfs 192.168.0.240:/engine /tmp

mount.nfs: timeout set for Mon Jan 12 20:02:58 2015

mount.nfs: trying text-based options 'vers=3,addr=192.168.0.240

mount.nfs: prog 100003, trying vers=3, prot=6

mount.nfs: trying 192.168.0.240 prog 100003 vers 3 prot TCP port 2049

mount.nfs: prog 100005, trying vers=3, prot=17

mount.nfs: portmap query retrying: RPC: Program not registered

mount.nfs: prog 100005, trying vers=3, prot=6

mount.nfs: trying 192.168.0.240 prog 100005 vers 3 prot TCP port 38465

192.168.0.240:/engine on /tmp type nfs (rw,vers=3)

On the test machine mounts the nfs share with no problems. I have confirmed
this does not work on a single machine that is part of the gluster.  And any
other machine is able to mount the exact same share, with the exact same
parameters: on the exact same OS:.

I am at a loss

Donny D

------=_NextPart_000_0007_01D02EA2.D9347DE0
Content-Type: text/html;
	charset="koi8-r"
Content-Transfer-Encoding: quoted-printable

<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dkoi8-r">
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta name=3DGenerator =
content=3D"Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DEN-US link=3Dblue =
vlink=3Dpurple><div class=3DWordSection1><p class=3DMsoNormal>Here is a =
quick rundown of the system, and the problem.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>All hosts on =
centOS 7 fully up to date<o:p></o:p></p><p class=3DMsoNormal>GlusterFS =
is running on 6 servers, 3x2 distribute/replicate. <o:p></o:p></p><p =
class=3DMsoNormal>CTDB is running on all hosts<o:p></o:p></p><p =
class=3DMsoNormal>I am unable to mount via nfs the exported volume on =
any of the gluster server members.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>I am able to =
mount, read, write, umount from any server that is not a gluster member. =
<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>Topology – all are hostnames that are resolvable =
<o:p></o:p></p><p class=3DMsoNormal>Gluster Members<o:p></o:p></p><p =
class=3DMsoNormal>Node1<o:p></o:p></p><p =
class=3DMsoNormal>Node2<o:p></o:p></p><p =
class=3DMsoNormal>Node3<o:p></o:p></p><p =
class=3DMsoNormal>Node4<o:p></o:p></p><p =
class=3DMsoNormal>Node5<o:p></o:p></p><p =
class=3DMsoNormal>Node6<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>CTDB Virtual =
IP/Hostname<o:p></o:p></p><p =
class=3DMsoNormal>SharedNFS<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Test =
Machine<o:p></o:p></p><p class=3DMsoNormal>Test1<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Gluster =
Volumes<o:p></o:p></p><p class=3DMsoNormal>Engine<o:p></o:p></p><p =
class=3DMsoNormal>Data<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>I am trying =
to bring up hosted-engine using nfs using the gluster =
members<o:p></o:p></p><p class=3DMsoNormal>I run hosted-engine =
–deploy<o:p></o:p></p><p class=3DMsoNormal>Nfsv3<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Host:/path =
sharednfs:/engine<o:p></o:p></p><p class=3DMsoNormal>Error while =
mounting specified storage path: mount.nfs: an incorrect mount option =
was specified<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>Ok well lets try that without using the hosted-engine =
script<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>Mount –v –t nfs –o vers=3D3 =
sharednfs:/engine /tmp<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: =
timeout set for Mon Jan 12 22:00:08 2015<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: trying text-based options =
'vers=3D3,lock=3DFlase,addr=3D192.168.0.240<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: prog 100003, trying vers=3D3, =
prot=3D6<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: trying =
192.168.0.240 prog 100003 vers 3 prot TCP port 2049<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: prog 100005, trying vers=3D3, =
prot=3D17<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: portmap query =
retrying: RPC: Program not registered<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: prog 100005, trying vers=3D3, =
prot=3D6<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: trying =
192.168.0.240 prog 100005 vers 3 prot TCP port 38465<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: mount(2): Invalid argument<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: an incorrect mount option was =
specified<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>[root@node4 =
~]# systemctl status rpcbind<o:p></o:p></p><p =
class=3DMsoNormal>rpcbind.service - RPC bind service<o:p></o:p></p><p =
class=3DMsoNormal>=9A=9A Loaded: loaded =
(/usr/lib/systemd/system/rpcbind.service; enabled)<o:p></o:p></p><p =
class=3DMsoNormal>=9A=9A Active: active (running) since Mon 2015-01-12 =
20:01:13 EST; 1h 57min ago<o:p></o:p></p><p class=3DMsoNormal>=9A =
Process: 1349 ExecStart=3D/sbin/rpcbind -w ${RPCBIND_ARGS} =
(code=3Dexited, status=3D0/SUCCESS)<o:p></o:p></p><p class=3DMsoNormal> =
Main PID: 1353 (rpcbind)<o:p></o:p></p><p class=3DMsoNormal>=9A=9A =
CGroup: /system.slice/rpcbind.service<o:p></o:p></p><p =
class=3DMsoNormal>=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A └─1353 =
/sbin/rpcbind -w<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Jan 12 =
20:01:13 node4 systemd[1]: Starting RPC bind service...<o:p></o:p></p><p =
class=3DMsoNormal>Jan 12 20:01:13 node4 systemd[1]: Started RPC bind =
service.<o:p></o:p></p><p class=3DMsoNormal>Jan 12 21:19:22 node4 =
systemd[1]: Started RPC bind service.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Ummm… =
this makes no sense…. <o:p></o:p></p><p =
class=3DMsoNormal>[root@test1 ~]# mount -v -o vers=3D3 -t nfs =
192.168.0.240:/engine /tmp<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: =
timeout set for Mon Jan 12 20:02:58 2015<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: trying text-based options =
'vers=3D3,addr=3D192.168.0.240<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: prog 100003, trying vers=3D3, =
prot=3D6<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: trying =
192.168.0.240 prog 100003 vers 3 prot TCP port 2049<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: prog 100005, trying vers=3D3, =
prot=3D17<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: portmap query =
retrying: RPC: Program not registered<o:p></o:p></p><p =
class=3DMsoNormal>mount.nfs: prog 100005, trying vers=3D3, =
prot=3D6<o:p></o:p></p><p class=3DMsoNormal>mount.nfs: trying =
192.168.0.240 prog 100005 vers 3 prot TCP port 38465<o:p></o:p></p><p =
class=3DMsoNormal>192.168.0.240:/engine on /tmp type nfs =
(rw,vers=3D3)<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>On the test =
machine mounts the nfs share with no problems. I have confirmed this =
does not work on a single machine that is part of the gluster. =9AAnd =
any other machine is able to mount the exact same share, with the exact =
same parameters… on the exact same OS….<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>I am at a =
loss<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>Donny D<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p></div></body></html>
------=_NextPart_000_0007_01D02EA2.D9347DE0--

Donny Davis

Karli Sjöberg

Donny Davis

tags

participants (2)