1:16 a.m.
Hi
I can't seem to get a gluster storage domain added I am using Fedora
17 on both the nodes management machine. I have the gluster volumes
showing in ovirt and I can manually mount the gluster volume both
locally on the node and on the management machine.
If I manually mount the volume on the node with /usr/bin/mount -t
glusterfs -o vers=3 localhost:/bsdvol1
/rhev/data-center/mnt/localhost:_bsdvol1 as root then add the domain
in the web interface it works.
Although I have the domain done with the manual mounting, I am
wondering what is going on.
Thread-1934::INFO::2012-10-29
13:27:09,793::logUtils::37::dispatcher::(wrapper) Run and protect:
validateStorageServerConnection(domType=6,
spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port':
'',
'connection': 'localhost:/bsdvol1', 'iqn': '',
'portal': '', 'user':
'', 'vfs_type': 'glusterfs', 'password': '******',
'id':
'00000000-0000-0000-0000-000000000000'}], options=None)
Thread-1934::INFO::2012-10-29
13:27:09,793::logUtils::39::dispatcher::(wrapper) Run and protect:
validateStorageServerConnection, Return response: {'statuslist':
[{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]}
Thread-1934::DEBUG::2012-10-29
13:27:09,793::task::1172::TaskManager.Task::(prepare)
Task=`4f9131eb-45cd-4e82-bc35-e29ed14f0818`::finished: {'statuslist':
[{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]}
Thread-1934::DEBUG::2012-10-29
13:27:09,793::task::588::TaskManager.Task::(_updateState)
Task=`4f9131eb-45cd-4e82-bc35-e29ed14f0818`::moving from state
preparing -> state finished
Thread-1934::DEBUG::2012-10-29
13:27:09,794::resourceManager::809::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}
Thread-1934::DEBUG::2012-10-29
13:27:09,794::resourceManager::844::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-1934::DEBUG::2012-10-29
13:27:09,794::task::978::TaskManager.Task::(_decref)
Task=`4f9131eb-45cd-4e82-bc35-e29ed14f0818`::ref 0 aborting False
Thread-1935::DEBUG::2012-10-29
13:27:09,842::BindingXMLRPC::156::vds::(wrapper) [192.168.1.10]
Thread-1935::DEBUG::2012-10-29
13:27:09,842::task::588::TaskManager.Task::(_updateState)
Task=`ef920d52-3082-495f-bfbc-2eb508c3de18`::moving from state init ->
state preparing
Thread-1935::INFO::2012-10-29
13:27:09,843::logUtils::37::dispatcher::(wrapper) Run and protect:
connectStorageServer(domType=6,
spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port':
'',
'connection': 'localhost:/bsdvol1', 'mnt_options':
'vers=3', 'portal':
'', 'user': '', 'iqn': '', 'vfs_type':
'glusterfs', 'password':
'******', 'id': 'bbe1bbc5-62b8-4115-b409-6ddea910a688'}],
options=None)
Thread-1935::DEBUG::2012-10-29
13:27:09,851::__init__::1249::Storage.Misc.excCmd::(_log)
'/usr/bin/sudo -n /usr/bin/mount -t glusterfs -o vers=3
localhost:/bsdvol1 /rhev/data-center/mnt/localhost:_bsdvol1' (cwd
None)
Thread-1935::ERROR::2012-10-29
13:27:09,930::hsm::1932::Storage.HSM::(connectStorageServer) Could not
connect to storageServer
Traceback (most recent call last):
File "/usr/share/vdsm/storage/hsm.py", line 1929, in connectStorageServer
conObj.connect()
File "/usr/share/vdsm/storage/storageServer.py", line 179, in connect
self._mount.mount(self.options, self._vfsType)
File "/usr/share/vdsm/storage/mount.py", line 190, in mount
return self._runcmd(cmd, timeout)
File "/usr/share/vdsm/storage/mount.py", line 206, in _runcmd
raise MountError(rc, ";".join((out, err)))
MountError: (1, 'unknown option vers (ignored)\nMount failed. Please
check the log file for more details.\n;ERROR: failed to create logfile
"/var/log/glusterfs/rhev-data-center-mnt-localhost:_bsdvol1.log"
(Permission denied)\nERROR: failed to open logfile
/var/log/glusterfs/rhev-data-center-mnt-localhost:_bsdvol1.log\n')
Thread-1935::DEBUG::2012-10-29
13:27:09,931::lvm::457::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-1935::DEBUG::2012-10-29
13:27:09,932::lvm::459::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-1935::DEBUG::2012-10-29
13:27:09,932::lvm::469::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' got the operation mutex
Thread-1935::DEBUG::2012-10-29
13:27:09,932::lvm::471::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' released the operation mutex
Thread-1935::DEBUG::2012-10-29
13:27:09,933::lvm::490::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-1935::DEBUG::2012-10-29
13:27:09,933::lvm::492::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-1935::INFO::2012-10-29
13:27:09,934::logUtils::39::dispatcher::(wrapper) Run and protect:
connectStorageServer, Return response: {'statuslist': [{'status': 477,
'id': 'bbe1bbc5-62b8-4115-b409-6ddea910a688'}]}
Thread-1935::DEBUG::2012-10-29
13:27:09,934::task::1172::TaskManager.Task::(prepare)
Task=`ef920d52-3082-495f-bfbc-2eb508c3de18`::finished: {'statuslist':
[{'status': 477, 'id': 'bbe1bbc5-62b8-4115-b409-6ddea910a688'}]}
Thread-1935::DEBUG::2012-10-29
13:27:09,934::task::588::TaskManager.Task::(_updateState)
Task=`ef920d52-3082-495f-bfbc-2eb508c3de18`::moving from state
preparing -> state finished
Thread-1935::DEBUG::2012-10-29
13:27:09,934::resourceManager::809::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}
Thread-1935::DEBUG::2012-10-29
13:27:09,935::resourceManager::844::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-1935::DEBUG::2012-10-29
13:27:09,935::task::978::TaskManager.Task::(_decref)
Task=`ef920d52-3082-495f-bfbc-2eb508c3de18`::ref 0 aborting False
Thread-1937::DEBUG::2012-10-29
13:27:10,073::BindingXMLRPC::156::vds::(wrapper) [192.168.1.10]
Thread-1937::DEBUG::2012-10-29
13:27:10,073::task::588::TaskManager.Task::(_updateState)
Task=`322f7d1b-0e8d-458b-baf2-5c7f8c5f4303`::moving from state init ->
state preparing
Thread-1937::INFO::2012-10-29
13:27:10,073::logUtils::37::dispatcher::(wrapper) Run and protect:
createStorageDomain(storageType=6,
sdUUID='d8591d07-1611-46cc-9122-fc7badea0f4c',
domainName='DCS_Bis_Sys_gluster',
typeSpecificArg='localhost:/bsdvol1', domClass=1, domVersion='0',
options=None)
Thread-1937::DEBUG::2012-10-29
13:27:10,074::misc::1053::SamplingMethod::(__call__) Trying to enter
sampling method (storage.sdc.refreshStorage)
Thread-1937::DEBUG::2012-10-29
13:27:10,074::misc::1055::SamplingMethod::(__call__) Got in to
sampling method
Thread-1937::DEBUG::2012-10-29
13:27:10,074::misc::1053::SamplingMethod::(__call__) Trying to enter
sampling method (storage.iscsi.rescan)
Thread-1937::DEBUG::2012-10-29
13:27:10,074::misc::1055::SamplingMethod::(__call__) Got in to
sampling method
Thread-1937::DEBUG::2012-10-29
13:27:10,075::__init__::1249::Storage.Misc.excCmd::(_log)
'/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
Thread-1937::DEBUG::2012-10-29
13:27:10,092::__init__::1249::Storage.Misc.excCmd::(_log) FAILED:
<err> = 'iscsiadm: No session found.\n'; <rc> = 21
Thread-1937::DEBUG::2012-10-29
13:27:10,092::misc::1063::SamplingMethod::(__call__) Returning last
result
Thread-1937::DEBUG::2012-10-29
13:27:15,382::__init__::1249::Storage.Misc.excCmd::(_log)
'/usr/bin/sudo -n /sbin/multipath' (cwd None)
Thread-1937::DEBUG::2012-10-29
13:27:15,414::__init__::1249::Storage.Misc.excCmd::(_log) SUCCESS:
<err> = ''; <rc> = 0
Thread-1937::DEBUG::2012-10-29
13:27:15,415::lvm::457::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-1937::DEBUG::2012-10-29
13:27:15,415::lvm::459::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-1937::DEBUG::2012-10-29
13:27:15,415::lvm::469::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' got the operation mutex
Thread-1937::DEBUG::2012-10-29
13:27:15,416::lvm::471::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' released the operation mutex
Thread-1937::DEBUG::2012-10-29
13:27:15,416::lvm::490::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-1937::DEBUG::2012-10-29
13:27:15,416::lvm::492::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-1937::DEBUG::2012-10-29
13:27:15,416::misc::1063::SamplingMethod::(__call__) Returning last
result
Thread-1937::DEBUG::2012-10-29
13:27:15,417::lvm::349::OperationMutex::(_reloadvgs) Operation 'lvm
reload operation' got the operation mutex
Thread-1937::DEBUG::2012-10-29
13:27:15,419::__init__::1249::Storage.Misc.excCmd::(_log)
'/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names =
[\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0
disable_after_error_count=3 filter = [
\\"a%36848f690e0b7860017e62e4257089a6e|36848f690e0b78600181b77200a8142b8%\\",
\\"r%.*%\\" ] } global { locking_type=1 prioritise_write_locks=1
wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } "
--noheadings --units b --nosuffix --separator | -o
uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free
d8591d07-1611-46cc-9122-fc7badea0f4c' (cwd None)
Thread-1937::DEBUG::2012-10-29
13:27:15,536::__init__::1249::Storage.Misc.excCmd::(_log) FAILED:
<err> = ' Volume group "d8591d07-1611-46cc-9122-fc7badea0f4c" not
found\n'; <rc> = 5
Thread-1937::WARNING::2012-10-29
13:27:15,538::lvm::353::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 []
[' Volume group "d8591d07-1611-46cc-9122-fc7badea0f4c" not found']
Thread-1937::DEBUG::2012-10-29
13:27:15,538::lvm::376::OperationMutex::(_reloadvgs) Operation 'lvm
reload operation' released the operation mutex
Thread-1937::INFO::2012-10-29
13:27:15,542::nfsSD::64::Storage.StorageDomain::(create)
sdUUID=d8591d07-1611-46cc-9122-fc7badea0f4c
domainName=DCS_Bis_Sys_gluster remotePath=localhost:/bsdvol1
domClass=1
Thread-1937::ERROR::2012-10-29
13:27:15,550::task::853::TaskManager.Task::(_setError)
Task=`322f7d1b-0e8d-458b-baf2-5c7f8c5f4303`::Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 861, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
res = f(*args, **kwargs)
File "/usr/share/vdsm/storage/hsm.py", line 2136, in createStorageDomain
typeSpecificArg, storageType, domVersion)
File "/usr/share/vdsm/storage/nfsSD.py", line 75, in create
cls._preCreateValidation(sdUUID, mntPoint, remotePath, version)
File "/usr/share/vdsm/storage/nfsSD.py", line 44, in _preCreateValidation
raise se.StorageDomainFSNotMounted(domPath)
StorageDomainFSNotMounted: Storage domain remote path not mounted:
('/rhev/data-center/mnt/localhost:_bsdvol1',)
[root@bsdvmhvnode01 mnt]# cat
/var/log/glusterfs/rhev-data-center-mnt-localhost:_bsdvol1.log
[2012-10-29 13:27:42.311022] I [glusterfsd.c:1666:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version
3.3.1
[2012-10-29 13:27:42.328054] I
[io-cache.c:1549:check_cache_ users(a)ovirt.org. size_ok]
0-bsdvol1-quick-read: Max cache size is 50636935168
[2012-10-29 13:27:42.328157] I [io-cache.c:1549:check_cache_size_ok]
0-bsdvol1-io-cache: Max cache size is 50636935168
[2012-10-29 13:27:42.329037] I [client.c:2142:notify]
0-bsdvol1-client-0: parent translators are ready, attempting connect
on transport
Given volfile:
+------------------------------------------------------------------------------+
1: volume bsdvol1-client-0
2: type protocol/client
3: option remote-host 192.168.1.11
4: option remote-subvolume /data/bsdvol1
5: option transport-type tcp
6: option username ed9b899c-26c1-4d8b-a2f8-ffd2d0b10fed
7: option password 37a8f9ed-e27c-41cb-a049-da6a65042920
8: end-volume
9:
10: volume bsdvol1-write-behind
11: type performance/write-behind
12: subvolumes bsdvol1-client-0
13: end-volume
14:
15: volume bsdvol1-read-ahead
16: type performance/read-ahead
17: subvolumes bsdvol1-write-behind
18: end-volume
19:
20: volume bsdvol1-io-cache
21: type performance/io-cache
22: subvolumes bsdvol1-read-ahead
23: end-volume
24:
25: volume bsdvol1-quick-read
26: type performance/quick-read
27: subvolumes bsdvol1-io-cache
28: end-volume
29:
30: volume bsdvol1-md-cache
31: type performance/md-cache
32: subvolumes bsdvol1-quick-read
33: end-volume
34:
35: volume bsdvol1
36: type debug/io-stats
37: option latency-measurement off
38: option count-fop-hits off
39: subvolumes bsdvol1-md-cache
40: end-volume
+------------------------------------------------------------------------------+
[2012-10-29 13:27:42.334438] I [rpc-clnt.c:1657:rpc_clnt_reconfig]
0-bsdvol1-client-0: changing port to 24009 (from 0)
[2012-10-29 13:27:46.331768] I
[client-handshake.c:1636:select_server_supported_programs]
0-bsdvol1-client-0: Using Program GlusterFS 3.3.1, Num (1298437),
Version (330)
[2012-10-29 13:27:46.332216] I
[client-handshake.c:1433:client_setvolume_cbk] 0-bsdvol1-client-0:
Connected to 192.168.1.11:24009, attached to remote volume
'/data/bsdvol1'.
[2012-10-29 13:27:46.332249] I
[client-handshake.c:1445:client_setvolume_cbk] 0-bsdvol1-client-0:
Server and Client lk-version numbers are not same, reopening the fds
[2012-10-29 13:27:46.337041] I [fuse-bridge.c:4191:fuse_graph_setup]
0-fuse: switched to graph 0
[2012-10-29 13:27:46.337157] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-bsdvol1-client-0:
Server lk version = 1
[2012-10-29 13:27:46.337466] I [fuse-bridge.c:3376:fuse_init]
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13
kernel 7.20