<p dir="ltr">HI,</p>
<p dir="ltr">I have centos 7.1 latest update for hosts and VM that hosts the hosted engine, the hosted engine uses iSCSI storage and so far it is working correctly *as expected* , I have 1 shared DC, 1 Cluster and trying to add GFS2 storage as a master data domain, I am using corosync , pacemaker , pcs to manage the DLM and cLVM that is also working as expected, however whenever I try to add the GFS2 storage I get the following error message from VDSM.log</p>
<p dir="ltr">—————<br>
Thread-884::ERROR::2015-05-08 11:45:59,291::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 2435cdc4-61eb-4ad6-9669-e900f13b7f1d<br>
Thread-884::DEBUG::2015-05-08 11:45:59,291::lvm::371::Storage.OperationMutex::(_reloadvgs) Operation &#39;lvm reload operation&#39; got the operation mutex<br>
Thread-884::DEBUG::2015-05-08 11:45:59,295::lvm::291::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config &#39; devices { preferred_names = [&quot;^/dev/mapper/&quot;] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_co<br>
unt=3 obtain_device_list_from_udev=0 filter = [ &#39;\&#39;&#39;a|/dev/mapper/CETUSRAID1E0A04V01|/dev/mapper/CETUSRAID1E0A04V02|/dev/mapper/CETUSRAID3E0A01V01|&#39;\&#39;&#39;, &#39;\&#39;&#39;r|.*|&#39;\&#39;&#39; ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks<br>
=1  use_lvmetad=0 }  backup {  retain_min = 50  retain_days = 0 } &#39; --noheadings --units b --nosuffix --separator &#39;|&#39; --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_c<br>
ount,pv_count,pv_name 2435cdc4-61eb-4ad6-9669-e900f13b7f1d (cwd None)<br>
Thread-884::DEBUG::2015-05-08 11:45:59,397::lvm::291::Storage.Misc.excCmd::(cmd) FAILED: &lt;err&gt; = &#39;  Volume group &quot;2435cdc4-61eb-4ad6-9669-e900f13b7f1d&quot; not found\n  Cannot process volume group 2435cdc4-61eb-4ad6-9669-e900f13b7f1d\n&#39;; &lt;rc&gt;<br>
 = 5<br>
Thread-884::WARNING::2015-05-08 11:45:59,400::lvm::376::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [&#39;  Volume group &quot;2435cdc4-61eb-4ad6-9669-e900f13b7f1d&quot; not found&#39;, &#39;  Cannot process volume group 2435cdc4-61eb-4ad6-9669-e900f13b7f1d<br>
&#39;]<br>
Thread-884::DEBUG::2015-05-08 11:45:59,401::lvm::416::Storage.OperationMutex::(_reloadvgs) Operation &#39;lvm reload operation&#39; released the operation mutex<br>
Thread-885::DEBUG::2015-05-08 11:45:59,403::__init__::298::IOProcessClient::(_run) Starting IOProcess...<br>
Thread-884::ERROR::2015-05-08 11:45:59,420::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 2435cdc4-61eb-4ad6-9669-e900f13b7f1d not found<br>
Traceback (most recent call last):<br>
  File &quot;/usr/share/vdsm/storage/sdc.py&quot;, line 141, in _findDomain<br>
    dom = findMethod(sdUUID)<br>
  File &quot;/usr/share/vdsm/storage/sdc.py&quot;, line 171, in _findUnfetchedDomain<br>
    raise se.StorageDomainDoesNotExist(sdUUID)<br>
StorageDomainDoesNotExist: Storage domain does not exist: (u&#39;2435cdc4-61eb-4ad6-9669-e900f13b7f1d&#39;,)<br>
Thread-884::INFO::2015-05-08 11:45:59,421::nfsSD::69::Storage.StorageDomain::(create) sdUUID=2435cdc4-61eb-4ad6-9669-e900f13b7f1d domainName=imagesGFS remotePath=/dev/mapper/gfs2_imgs-gfs2_imgs_lvm domClass=1<br>
Thread-884::DEBUG::2015-05-08 11:45:59,434::__init__::298::IOProcessClient::(_run) Starting IOProcess...<br>
Thread-884::ERROR::2015-05-08 11:45:59,446::task::866::Storage.TaskManager.Task::(_setError) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::Unexpected error<br>
Traceback (most recent call last):<br>
  File &quot;/usr/share/vdsm/storage/task.py&quot;, line 873, in _run<br>
    return fn(*args, **kargs)<br>
  File &quot;/usr/share/vdsm/logUtils.py&quot;, line 45, in wrapper<br>
    res = f(*args, **kwargs)<br>
  File &quot;/usr/share/vdsm/storage/hsm.py&quot;, line 2691, in createStorageDomain<br>
    domVersion)<br>
  File &quot;/usr/share/vdsm/storage/nfsSD.py&quot;, line 80, in create<br>
    version)<br>
  File &quot;/usr/share/vdsm/storage/nfsSD.py&quot;, line 49, in _preCreateValidation<br>
    fileSD.validateFileSystemFeatures(sdUUID, domPath)<br>
  File &quot;/usr/share/vdsm/storage/fileSD.py&quot;, line 88, in validateFileSystemFeatures<br>
    oop.getProcessPool(sdUUID).directTouch(testFilePath)<br>
  File &quot;/usr/share/vdsm/storage/outOfProcess.py&quot;, line 351, in directTouch<br>
    ioproc.touch(path, flags, mode)<br>
  File &quot;/usr/lib/python2.7/site-packages/ioprocess/__init__.py&quot;, line 507, in touch<br>
    self.timeout)<br>
  File &quot;/usr/lib/python2.7/site-packages/ioprocess/__init__.py&quot;, line 391, in _sendCommand<br>
    raise OSError(errcode, errstr)<br>
OSError: [Errno 13] Permission denied<br>
Thread-884::DEBUG::2015-05-08 11:45:59,447::task::885::Storage.TaskManager.Task::(_run) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::Task._run: b574d8c6-af2b-4909-be53-248a5caff1c1 (6, u&#39;2435cdc4-61eb-4ad6-9669-e900f13b7f1d&#39;, u&#39;imagesGFS&#39;<br>
, u&#39;/dev/mapper/gfs2_imgs-gfs2_imgs_lvm&#39;, 1, u&#39;3&#39;) {} failed - stopping task<br>
Thread-884::DEBUG::2015-05-08 11:45:59,447::task::1217::Storage.TaskManager.Task::(stop) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::stopping in state preparing (force False)<br>
Thread-884::DEBUG::2015-05-08 11:45:59,448::task::993::Storage.TaskManager.Task::(_decref) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::ref 1 aborting True<br>
Thread-884::INFO::2015-05-08 11:45:59,448::task::1171::Storage.TaskManager.Task::(prepare) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::aborting: Task is aborted: u&#39;[Errno 13] Permission denied&#39; - code 100<br>
Thread-884::DEBUG::2015-05-08 11:45:59,448::task::1176::Storage.TaskManager.Task::(prepare) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::Prepare: aborted: [Errno 13] Permission denied<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::task::993::Storage.TaskManager.Task::(_decref) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::ref 0 aborting True<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::task::928::Storage.TaskManager.Task::(_doAbort) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::Task._doAbort: force False<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::moving from state preparing -&gt; state aborting<br>
Thread-884::DEBUG::2015-05-08 11:45:59,448::task::993::Storage.TaskManager.Task::(_decref) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::ref 1 aborting True<br>
Thread-884::INFO::2015-05-08 11:45:59,448::task::1171::Storage.TaskManager.Task::(prepare) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::aborting: Task is aborted: u&#39;[Errno 13] Permission denied&#39; - code 100<br>
Thread-884::DEBUG::2015-05-08 11:45:59,448::task::1176::Storage.TaskManager.Task::(prepare) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::Prepare: aborted: [Errno 13] Permission denied<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::task::993::Storage.TaskManager.Task::(_decref) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::ref 0 aborting True<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::task::928::Storage.TaskManager.Task::(_doAbort) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::Task._doAbort: force False<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<br>
Thread-884::DEBUG::2015-05-08 11:45:59,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::moving from state preparing -&gt; state aborting<br>
Thread-884::DEBUG::2015-05-08 11:45:59,450::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::_aborting: recover policy none<br>
Thread-884::DEBUG::2015-05-08 11:45:59,450::task::595::Storage.TaskManager.Task::(_updateState) Task=`b574d8c6-af2b-4909-be53-248a5caff1c1`::moving from state aborting -&gt; state failed<br>
Thread-884::DEBUG::2015-05-08 11:45:59,450::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<br>
Thread-884::DEBUG::2015-05-08 11:45:59,450::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<br>
Thread-884::ERROR::2015-05-08 11:45:59,451::dispatcher::79::Storage.Dispatcher::(wrapper) [Errno 13] Permission denied<br>
Traceback (most recent call last):<br>
  File &quot;/usr/share/vdsm/storage/dispatcher.py&quot;, line 71, in wrapper<br>
    result = ctask.prepare(func, *args, **kwargs)<br>
  File &quot;/usr/share/vdsm/storage/task.py&quot;, line 103, in wrapper<br>
    return m(self, *a, **kw)<br>
  File &quot;/usr/share/vdsm/storage/task.py&quot;, line 1179, in prepare<br>
    raise self.error<br>
OSError: [Errno 13] Permission denied<br>
Thread-884::DEBUG::2015-05-08 11:45:59,452::stompReactor::163::yajsonrpc.StompServer::(send) Sending response<br>
JsonRpc (StompReactor)::DEBUG::2015-05-08 11:45:59,918::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message &lt;StompFrame command=&#39;SEND&#39;&gt;<br>
JsonRpcServer::DEBUG::2015-05-08 11:45:59,920::__init__::506::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request<br>
Thread-887::DEBUG::2015-05-08 11:45:59,920::__init__::469::jsonrpc.JsonRpcServer::(_serveRequest) Calling &#39;StoragePool.disconnectStorageServer&#39; in bridge with {u&#39;connectionParams&#39;: [{u&#39;password&#39;: u&#39;&#39;, u&#39;id&#39;: u&#39;a52aaa59-a449-4158-bece-2e0a<br>
0916dc78&#39;, u&#39;connection&#39;: u&#39;/dev/mapper/gfs2_imgs-gfs2_imgs_lvm&#39;, u&#39;mnt_options&#39;: u&#39;noatime,nodiratime&#39;, u&#39;user&#39;: u&#39;&#39;, u&#39;tpgt&#39;: u&#39;1&#39;, u&#39;vfs_type&#39;: u&#39;gfs2&#39;, u&#39;iqn&#39;: u&#39;&#39;, u&#39;port&#39;: u&#39;&#39;}], u&#39;storagepoolID&#39;: u&#39;00000000-0000-0000-0000-000000000<br>
000&#39;, u&#39;domainType&#39;: 6}<br>
Thread-887::DEBUG::2015-05-08 11:45:59,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`dacc3945-1a0d-4b80-9e0b-dd0050179001`::moving from state init -&gt; state preparing<br>
Thread-887::INFO::2015-05-08 11:45:59,925::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=6, spUUID=u&#39;00000000-0000-0000-0000-000000000000&#39;, conList=[{u&#39;iqn&#39;: u&#39;&#39;, u&#39;port&#39;: u&#39;&#39;, u&#39;connection&#39;: u&#39;/dev/<br>
mapper/gfs2_imgs-gfs2_imgs_lvm&#39;, u&#39;mnt_options&#39;: u&#39;noatime,nodiratime&#39;, u&#39;user&#39;: u&#39;&#39;, u&#39;tpgt&#39;: u&#39;1&#39;, u&#39;vfs_type&#39;: u&#39;gfs2&#39;, u&#39;password&#39;: &#39;******&#39;, u&#39;id&#39;: u&#39;a52aaa59-a449-4158-bece-2e0a0916dc78&#39;}], options=None)<br>
Thread-887::DEBUG::2015-05-08 11:45:59,926::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /usr/bin/umount -f -l /rhev/data-center/mnt/_dev_mapper_gfs2__imgs-gfs2__imgs__lvm (cwd None)<br>
Thread-887::DEBUG::2015-05-08 11:45:59,995::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)<br>
Thread-887::DEBUG::2015-05-08 11:45:59,997::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method</p>
<p dir="ltr">————<br>
Any idea what that might be ?</p>
<p dir="ltr">Thank You<br>
<font color="#888888">Sherif</font></p>