<div dir="ltr">Found this in /var/log/vdsm/vdsm.log:<div><br></div><div><div>Thread-1080::INFO::2013-10-15 16:39:43,656::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=7, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': '192.168.1.1:rep2-virt', 'iqn': '', 'portal': '', 'user': '', 'vfs_type': 'glusterfs', 'password': '******', 'id': 'abfea73c-31d5-43be-97f1-998fd184d98f'}], options=None)</div>
</div><div><br></div><div>I'm assuming that UUID shouldn't be zero's...</div><div><br></div><div>messages:</div><div><div>Oct 15 17:17:10 ovirt001 vdsm initSANLock ERROR Cannot initialize SANLock for domain 636abf41-5133-4252-9aef-8a32ec10668e</div>
<div>Oct 15 17:17:11 ovirt001 sanlock[1284]: 2013-10-15 17:17:11-0400 1228 [4075]: write_sectors delta_leader offset 127488 rv -22 /rhev/data-center/mnt/glusterSD/ovirt001:rep2-virt/636abf41-5133-4252-9aef-8a32ec10668e/dom_md/ids</div>
<div>Oct 15 17:17:12 ovirt001 sanlock[1284]: 2013-10-15 17:17:12-0400 1229 [1297]: s3 add_lockspace fail result -22</div><div>Oct 15 17:17:12 ovirt001 vdsm TaskManager.Task ERROR Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::Unexpected error</div>
</div><div><br></div><div><br></div><div>sanlock.log:</div><div><div>2013-10-15 17:17:11-0400 1228 [1297]: s3 lockspace 636abf41-5133-4252-9aef-8a32ec10668e:250:/rhev/data-center/mnt/glusterSD/ovirt001:rep2-virt/636abf41-5133-4252-9aef-8a32ec10668e/dom_md/ids:0</div>
<div>2013-10-15 17:17:11-0400 1228 [4075]: 636abf41 aio collect 1 0x7f41f80008c0:0x7f41f80008d0:0x7f41f8101000 result -22:0 match res</div><div>2013-10-15 17:17:11-0400 1228 [4075]: write_sectors delta_leader offset 127488 rv -22 /rhev/data-center/mnt/glusterSD/ovirt001:rep2-virt/636abf41-5133-4252-9aef-8a32ec10668e/dom_md/ids</div>
<div>2013-10-15 17:17:12-0400 1229 [1297]: s3 add_lockspace fail result -22</div></div><div><br></div><div>vdsm.log:</div><div><div>Thread-488::DEBUG::2013-10-15 17:17:07,316::BindingXMLRPC::177::vds::(wrapper) client [10.0.6.22]</div>
<div>Thread-488::DEBUG::2013-10-15 17:17:07,317::task::579::TaskManager.Task::(_updateState) Task=`6a1db668-e38b-4698-8a34-554f55182850`::moving from state init -> state preparing</div><div>Thread-488::INFO::2013-10-15 17:17:07,318::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=7, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'ovirt001:rep2-virt', 'iqn': '', 'portal': '', 'user': '', 'vfs_type': 'glusterfs', 'password': '******', 'id': '00000000-0000-0000-0000-000000000000'}], options=None)</div>
<div>Thread-488::DEBUG::2013-10-15 17:17:07,323::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n /usr/bin/mount -t glusterfs ovirt001:rep2-virt /rhev/data-center/mnt/glusterSD/ovirt001:rep2-virt' (cwd None)</div>
<div>Thread-488::DEBUG::2013-10-15 17:17:07,423::hsm::2333::Storage.HSM::(__prefetchDomains) glusterDomPath: glusterSD/*</div><div>Thread-488::DEBUG::2013-10-15 17:17:07,506::hsm::2345::Storage.HSM::(__prefetchDomains) Found SD uuids: ()</div>
<div>Thread-488::DEBUG::2013-10-15 17:17:07,507::hsm::2396::Storage.HSM::(connectStorageServer) knownSDs: {4a81a0e0-a95a-4e02-853f-68df420a7ce4: storage.glusterSD.findDomain}</div><div>Thread-488::INFO::2013-10-15 17:17:07,507::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]}</div>
<div>Thread-488::DEBUG::2013-10-15 17:17:07,507::task::1168::TaskManager.Task::(prepare) Task=`6a1db668-e38b-4698-8a34-554f55182850`::finished: {'statuslist': [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]}</div>
<div>Thread-488::DEBUG::2013-10-15 17:17:07,507::task::579::TaskManager.Task::(_updateState) Task=`6a1db668-e38b-4698-8a34-554f55182850`::moving from state preparing -> state finished</div><div>Thread-488::DEBUG::2013-10-15 17:17:07,507::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div>
<div>Thread-488::DEBUG::2013-10-15 17:17:07,508::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-488::DEBUG::2013-10-15 17:17:07,508::task::974::TaskManager.Task::(_decref) Task=`6a1db668-e38b-4698-8a34-554f55182850`::ref 0 aborting False</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:07,709::BindingXMLRPC::177::vds::(wrapper) client [10.0.6.22]</div><div>Thread-490::DEBUG::2013-10-15 17:17:07,710::task::579::TaskManager.Task::(_updateState) Task=`95b7da65-b45f-4836-9103-b64c6b5fc15a`::moving from state init -> state preparing</div>
<div>Thread-490::INFO::2013-10-15 17:17:07,710::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=7, sdUUID='636abf41-5133-4252-9aef-8a32ec10668e', domainName='rep2-virt', typeSpecificArg='ovirt001:rep2-virt', domClass=1, domVersion='3', options=None)</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:07,710::misc::807::SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)</div><div>Thread-490::DEBUG::2013-10-15 17:17:07,710::misc::809::SamplingMethod::(__call__) Got in to sampling method</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:07,711::misc::807::SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)</div><div>Thread-490::DEBUG::2013-10-15 17:17:07,711::misc::809::SamplingMethod::(__call__) Got in to sampling method</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:07,711::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)</div><div>Thread-490::DEBUG::2013-10-15 17:17:07,727::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED: <err> = 'iscsiadm: No session found.\n'; <rc> = 21</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:07,727::misc::817::SamplingMethod::(__call__) Returning last result</div><div>Thread-490::DEBUG::2013-10-15 17:17:09,750::multipath::111::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo -n /sbin/multipath' (cwd None)</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:09,943::multipath::111::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0</div><div>Thread-490::DEBUG::2013-10-15 17:17:09,944::lvm::483::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:09,944::lvm::485::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex</div><div>Thread-490::DEBUG::2013-10-15 17:17:09,944::lvm::494::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:09,945::lvm::496::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex</div><div>Thread-490::DEBUG::2013-10-15 17:17:09,945::lvm::514::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:09,945::lvm::516::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex</div><div>Thread-490::DEBUG::2013-10-15 17:17:09,945::misc::817::SamplingMethod::(__call__) Returning last result</div>
<div>Thread-490::ERROR::2013-10-15 17:17:09,945::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 636abf41-5133-4252-9aef-8a32ec10668e</div><div>Thread-490::ERROR::2013-10-15 17:17:09,945::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 636abf41-5133-4252-9aef-8a32ec10668e</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:09,946::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex</div><div>Thread-490::DEBUG::2013-10-15 17:17:09,947::lvm::311::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings --units b --nosuffix --separator | -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free 636abf41-5133-4252-9aef-8a32ec10668e' (cwd None)</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:09,974::lvm::311::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "636abf41-5133-4252-9aef-8a32ec10668e" not found\n'; <rc> = 5</div><div>Thread-490::WARNING::2013-10-15 17:17:09,975::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "636abf41-5133-4252-9aef-8a32ec10668e" not found']</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:09,975::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex</div><div>Thread-490::ERROR::2013-10-15 17:17:09,981::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 636abf41-5133-4252-9aef-8a32ec10668e not found</div>
<div>Traceback (most recent call last):</div><div> File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain</div><div> dom = findMethod(sdUUID)</div><div> File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain</div>
<div> raise se.StorageDomainDoesNotExist(sdUUID)</div><div>StorageDomainDoesNotExist: Storage domain does not exist: ('636abf41-5133-4252-9aef-8a32ec10668e',)</div><div>Thread-490::INFO::2013-10-15 17:17:09,982::nfsSD::69::Storage.StorageDomain::(create) sdUUID=636abf41-5133-4252-9aef-8a32ec10668e domainName=rep2-virt remotePath=ovirt001:rep2-virt domClass=1</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,104::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,115::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=[]</div>
<div>Thread-490::WARNING::2013-10-15 17:17:10,115::persistentDict::256::Storage.PersistentDict::(refresh) data has no embedded checksum - trust it as it is</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,115::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,115::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,115::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=ovirt001:rep2-virt', 'ROLE=Regular', 'SDUUID=636abf41-5133-4252-9aef-8a32ec10668e', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=a38d3de67a9f2b5bc3979eb5f4dfbce7d48986d7']</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,119::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,122::fileSD::153::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/glusterSD/ovirt001:rep2-virt/636abf41-5133-4252-9aef-8a32ec10668e</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,122::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,133::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=ovirt001:rep2-virt', 'ROLE=Regular', 'SDUUID=636abf41-5133-4252-9aef-8a32ec10668e', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=a38d3de67a9f2b5bc3979eb5f4dfbce7d48986d7']</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,135::fileSD::535::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,136::resourceManager::420::ResourceManager::(registerNamespace) Registering namespace '636abf41-5133-4252-9aef-8a32ec10668e_imageNS'</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,136::resourceManager::420::ResourceManager::(registerNamespace) Registering namespace '636abf41-5133-4252-9aef-8a32ec10668e_volumeNS'</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,136::clusterlock::137::initSANLock::(initSANLock) Initializing SANLock for domain 636abf41-5133-4252-9aef-8a32ec10668e</div>
<div>Thread-490::ERROR::2013-10-15 17:17:10,731::clusterlock::145::initSANLock::(initSANLock) Cannot initialize SANLock for domain 636abf41-5133-4252-9aef-8a32ec10668e</div><div>Traceback (most recent call last):</div><div>
File "/usr/share/vdsm/storage/clusterlock.py", line 140, in initSANLock</div><div> sanlock.init_lockspace(sdUUID, idsPath)</div><div>SanlockException: (22, 'Sanlock lockspace init failure', 'Invalid argument')</div>
<div>Thread-490::WARNING::2013-10-15 17:17:10,732::sd::428::Storage.StorageDomain::(initSPMlease) lease did not initialize successfully</div><div>Traceback (most recent call last):</div><div> File "/usr/share/vdsm/storage/sd.py", line 423, in initSPMlease</div>
<div> self._clusterLock.initLock()</div><div> File "/usr/share/vdsm/storage/clusterlock.py", line 163, in initLock</div><div> initSANLock(self._sdUUID, self._idsPath, self._leasesPath)</div><div> File "/usr/share/vdsm/storage/clusterlock.py", line 146, in initSANLock</div>
<div> raise se.ClusterLockInitError()</div><div>ClusterLockInitError: Could not initialize cluster lock: ()</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,732::hsm::2624::Storage.HSM::(createStorageDomain) knownSDs: {636abf41-5133-4252-9aef-8a32ec10668e: storage.glusterSD.findDomain, 4a81a0e0-a95a-4e02-853f-68df420a7ce4: storage.glusterSD.findDomain}</div>
<div>Thread-490::INFO::2013-10-15 17:17:10,732::logUtils::47::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,732::task::1168::TaskManager.Task::(prepare) Task=`95b7da65-b45f-4836-9103-b64c6b5fc15a`::finished: None</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,733::task::579::TaskManager.Task::(_updateState) Task=`95b7da65-b45f-4836-9103-b64c6b5fc15a`::moving from state preparing -> state finished</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,733::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div>
<div>Thread-490::DEBUG::2013-10-15 17:17:10,733::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-490::DEBUG::2013-10-15 17:17:10,733::task::974::TaskManager.Task::(_decref) Task=`95b7da65-b45f-4836-9103-b64c6b5fc15a`::ref 0 aborting False</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,739::BindingXMLRPC::177::vds::(wrapper) client [10.0.6.22]</div><div>Thread-494::DEBUG::2013-10-15 17:17:10,740::task::579::TaskManager.Task::(_updateState) Task=`7b5b467e-a1e5-413c-934d-18e337cf6dd8`::moving from state init -> state preparing</div>
<div>Thread-494::INFO::2013-10-15 17:17:10,740::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID='636abf41-5133-4252-9aef-8a32ec10668e', options=None)</div><div>Thread-494::DEBUG::2013-10-15 17:17:10,741::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.636abf41-5133-4252-9aef-8a32ec10668e`ReqID=`0602d0b9-a7ff-4f3f-80bd-c886fe4e3270`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2784' at 'getStorageDomainStats'</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,741::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' for lock type 'shared'</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,741::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' is free. Now locking as 'shared' (1 active user)</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,741::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.636abf41-5133-4252-9aef-8a32ec10668e`ReqID=`0602d0b9-a7ff-4f3f-80bd-c886fe4e3270`::Granted request</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,742::task::811::TaskManager.Task::(resourceAcquired) Task=`7b5b467e-a1e5-413c-934d-18e337cf6dd8`::_resourcesAcquired: Storage.636abf41-5133-4252-9aef-8a32ec10668e (shared)</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,742::task::974::TaskManager.Task::(_decref) Task=`7b5b467e-a1e5-413c-934d-18e337cf6dd8`::ref 1 aborting False</div><div>Thread-494::INFO::2013-10-15 17:17:10,743::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '14144560824320', 'disktotal': '14199600185344', 'mdafree': 0}}</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,744::task::1168::TaskManager.Task::(prepare) Task=`7b5b467e-a1e5-413c-934d-18e337cf6dd8`::finished: {'stats': {'mdasize': 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '14144560824320', 'disktotal': '14199600185344', 'mdafree': 0}}</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,744::task::579::TaskManager.Task::(_updateState) Task=`7b5b467e-a1e5-413c-934d-18e337cf6dd8`::moving from state preparing -> state finished</div><div>Thread-494::DEBUG::2013-10-15 17:17:10,744::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.636abf41-5133-4252-9aef-8a32ec10668e': < ResourceRef 'Storage.636abf41-5133-4252-9aef-8a32ec10668e', isValid: 'True' obj: 'None'>}</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,744::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-494::DEBUG::2013-10-15 17:17:10,744::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e'</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,745::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' (0 active users)</div><div>Thread-494::DEBUG::2013-10-15 17:17:10,745::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' is free, finding out if anyone is waiting for it.</div>
<div>Thread-494::DEBUG::2013-10-15 17:17:10,745::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e', Clearing records.</div><div>
Thread-494::DEBUG::2013-10-15 17:17:10,745::task::974::TaskManager.Task::(_decref) Task=`7b5b467e-a1e5-413c-934d-18e337cf6dd8`::ref 0 aborting False</div><div>Thread-495::DEBUG::2013-10-15 17:17:11,113::BindingXMLRPC::177::vds::(wrapper) client [10.0.6.22]</div>
<div>Thread-495::DEBUG::2013-10-15 17:17:11,113::task::579::TaskManager.Task::(_updateState) Task=`d217edaf-c57b-4c28-aceb-5d68ec306531`::moving from state init -> state preparing</div><div>Thread-495::INFO::2013-10-15 17:17:11,114::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=7, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '', 'connection': 'ovirt001:rep2-virt', 'iqn': '', 'portal': '', 'user': '', 'vfs_type': 'glusterfs', 'password': '******', 'id': '648e518f-11eb-48d1-b7cd-64c7b7cf6204'}], options=None)</div>
<div>Thread-495::DEBUG::2013-10-15 17:17:11,119::hsm::2333::Storage.HSM::(__prefetchDomains) glusterDomPath: glusterSD/*</div><div>Thread-495::DEBUG::2013-10-15 17:17:11,125::hsm::2345::Storage.HSM::(__prefetchDomains) Found SD uuids: ('636abf41-5133-4252-9aef-8a32ec10668e',)</div>
<div>Thread-495::DEBUG::2013-10-15 17:17:11,126::hsm::2396::Storage.HSM::(connectStorageServer) knownSDs: {636abf41-5133-4252-9aef-8a32ec10668e: storage.glusterSD.findDomain, 4a81a0e0-a95a-4e02-853f-68df420a7ce4: storage.glusterSD.findDomain}</div>
<div>Thread-495::INFO::2013-10-15 17:17:11,126::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '648e518f-11eb-48d1-b7cd-64c7b7cf6204'}]}</div>
<div>Thread-495::DEBUG::2013-10-15 17:17:11,126::task::1168::TaskManager.Task::(prepare) Task=`d217edaf-c57b-4c28-aceb-5d68ec306531`::finished: {'statuslist': [{'status': 0, 'id': '648e518f-11eb-48d1-b7cd-64c7b7cf6204'}]}</div>
<div>Thread-495::DEBUG::2013-10-15 17:17:11,126::task::579::TaskManager.Task::(_updateState) Task=`d217edaf-c57b-4c28-aceb-5d68ec306531`::moving from state preparing -> state finished</div><div>Thread-495::DEBUG::2013-10-15 17:17:11,126::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div>
<div>Thread-495::DEBUG::2013-10-15 17:17:11,127::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-495::DEBUG::2013-10-15 17:17:11,127::task::974::TaskManager.Task::(_decref) Task=`d217edaf-c57b-4c28-aceb-5d68ec306531`::ref 0 aborting False</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,133::BindingXMLRPC::177::vds::(wrapper) client [10.0.6.22]</div><div>Thread-497::DEBUG::2013-10-15 17:17:11,133::task::579::TaskManager.Task::(_updateState) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::moving from state init -> state preparing</div>
<div>Thread-497::INFO::2013-10-15 17:17:11,134::logUtils::44::dispatcher::(wrapper) Run and protect: createStoragePool(poolType=None, spUUID='5849b030-626e-47cb-ad90-3ce782d831b3', poolName='Default', masterDom='636abf41-5133-4252-9aef-8a32ec10668e', domList=['636abf41-5133-4252-9aef-8a32ec10668e'], masterVersion=12, lockPolicy=None, lockRenewalIntervalSec=5, leaseTimeSec=60, ioOpTimeoutSec=10, leaseRetries=3, options=None)</div>
<div>Thread-497::INFO::2013-10-15 17:17:11,134::fileSD::315::Storage.StorageDomain::(validate) sdUUID=636abf41-5133-4252-9aef-8a32ec10668e</div><div>Thread-497::DEBUG::2013-10-15 17:17:11,144::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=ovirt001:rep2-virt', 'ROLE=Regular', 'SDUUID=636abf41-5133-4252-9aef-8a32ec10668e', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=a38d3de67a9f2b5bc3979eb5f4dfbce7d48986d7']</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,145::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.5849b030-626e-47cb-ad90-3ce782d831b3`ReqID=`938eb5db-1766-43df-8a91-ef34759cfc06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '954' at 'createStoragePool'</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,145::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3' for lock type 'exclusive'</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,146::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3' is free. Now locking as 'exclusive' (1 active user)</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,146::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.5849b030-626e-47cb-ad90-3ce782d831b3`ReqID=`938eb5db-1766-43df-8a91-ef34759cfc06`::Granted request</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,146::task::811::TaskManager.Task::(resourceAcquired) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::_resourcesAcquired: Storage.5849b030-626e-47cb-ad90-3ce782d831b3 (exclusive)</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,146::task::974::TaskManager.Task::(_decref) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::ref 1 aborting False</div><div>Thread-497::DEBUG::2013-10-15 17:17:11,147::resourceManager::197::ResourceManager.Request::(__init__) ResName=`Storage.636abf41-5133-4252-9aef-8a32ec10668e`ReqID=`79bba45b-37b7-417f-bfbf-fc96603ed997`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '956' at 'createStoragePool'</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,147::resourceManager::541::ResourceManager::(registerResource) Trying to register resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' for lock type 'exclusive'</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,147::resourceManager::600::ResourceManager::(registerResource) Resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' is free. Now locking as 'exclusive' (1 active user)</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,147::resourceManager::237::ResourceManager.Request::(grant) ResName=`Storage.636abf41-5133-4252-9aef-8a32ec10668e`ReqID=`79bba45b-37b7-417f-bfbf-fc96603ed997`::Granted request</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,147::task::811::TaskManager.Task::(resourceAcquired) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::_resourcesAcquired: Storage.636abf41-5133-4252-9aef-8a32ec10668e (exclusive)</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,148::task::974::TaskManager.Task::(_decref) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::ref 1 aborting False</div><div>Thread-497::INFO::2013-10-15 17:17:11,148::sp::592::Storage.StoragePool::(create) spUUID=5849b030-626e-47cb-ad90-3ce782d831b3 poolName=Default master_sd=636abf41-5133-4252-9aef-8a32ec10668e domList=['636abf41-5133-4252-9aef-8a32ec10668e'] masterVersion=12 {'LEASETIMESEC': 60, 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3, 'LOCKRENEWALINTERVALSEC': 5}</div>
<div>Thread-497::INFO::2013-10-15 17:17:11,148::fileSD::315::Storage.StorageDomain::(validate) sdUUID=636abf41-5133-4252-9aef-8a32ec10668e</div><div>Thread-497::DEBUG::2013-10-15 17:17:11,158::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=ovirt001:rep2-virt', 'ROLE=Regular', 'SDUUID=636abf41-5133-4252-9aef-8a32ec10668e', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=a38d3de67a9f2b5bc3979eb5f4dfbce7d48986d7']</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,168::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt', 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=ovirt001:rep2-virt', 'ROLE=Regular', 'SDUUID=636abf41-5133-4252-9aef-8a32ec10668e', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=a38d3de67a9f2b5bc3979eb5f4dfbce7d48986d7']</div>
<div>Thread-497::WARNING::2013-10-15 17:17:11,168::fileUtils::167::Storage.fileUtils::(createdir) Dir /rhev/data-center/5849b030-626e-47cb-ad90-3ce782d831b3 already exists</div><div>Thread-497::DEBUG::2013-10-15 17:17:11,168::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,169::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes</div><div>Thread-497::DEBUG::2013-10-15 17:17:11,169::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=ovirt001:rep2-virt', 'ROLE=Regular', 'SDUUID=636abf41-5133-4252-9aef-8a32ec10668e', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=acf2b6bfff3dc3de8e33a5b08af5fc959a01bddc']</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:11,184::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction</div><div>Thread-497::INFO::2013-10-15 17:17:11,184::clusterlock::174::SANLock::(acquireHostId) Acquiring host id for domain 636abf41-5133-4252-9aef-8a32ec10668e (id: 250)</div>
<div>Thread-498::DEBUG::2013-10-15 17:17:11,697::task::579::TaskManager.Task::(_updateState) Task=`7f1a4313-262f-4458-96af-8e5a0d3626d1`::moving from state init -> state preparing</div><div>Thread-498::INFO::2013-10-15 17:17:11,698::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)</div>
<div>Thread-498::INFO::2013-10-15 17:17:11,698::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}</div><div>Thread-498::DEBUG::2013-10-15 17:17:11,698::task::1168::TaskManager.Task::(prepare) Task=`7f1a4313-262f-4458-96af-8e5a0d3626d1`::finished: {}</div>
<div>Thread-498::DEBUG::2013-10-15 17:17:11,698::task::579::TaskManager.Task::(_updateState) Task=`7f1a4313-262f-4458-96af-8e5a0d3626d1`::moving from state preparing -> state finished</div><div>Thread-498::DEBUG::2013-10-15 17:17:11,698::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div>
<div>Thread-498::DEBUG::2013-10-15 17:17:11,698::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-498::DEBUG::2013-10-15 17:17:11,699::task::974::TaskManager.Task::(_decref) Task=`7f1a4313-262f-4458-96af-8e5a0d3626d1`::ref 0 aborting False</div>
<div>Thread-497::ERROR::2013-10-15 17:17:12,186::task::850::TaskManager.Task::(_setError) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::Unexpected error</div><div>Traceback (most recent call last):</div><div> File "/usr/share/vdsm/storage/task.py", line 857, in _run</div>
<div> return fn(*args, **kargs)</div><div> File "/usr/share/vdsm/logUtils.py", line 45, in wrapper</div><div> res = f(*args, **kwargs)</div><div> File "/usr/share/vdsm/storage/hsm.py", line 960, in createStoragePool</div>
<div> masterVersion, leaseParams)</div><div> File "/usr/share/vdsm/storage/sp.py", line 617, in create</div><div> self._acquireTemporaryClusterLock(msdUUID, leaseParams)</div><div> File "/usr/share/vdsm/storage/sp.py", line 559, in _acquireTemporaryClusterLock</div>
<div> msd.acquireHostId(<a href="http://self.id">self.id</a>)</div><div> File "/usr/share/vdsm/storage/sd.py", line 458, in acquireHostId</div><div> self._clusterLock.acquireHostId(hostId, async)</div><div>
File "/usr/share/vdsm/storage/clusterlock.py", line 189, in acquireHostId</div><div> raise se.AcquireHostIdFailure(self._sdUUID, e)</div><div>AcquireHostIdFailure: Cannot acquire host id: ('636abf41-5133-4252-9aef-8a32ec10668e', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument'))</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,186::task::869::TaskManager.Task::(_run) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::Task._run: e53f26ec-66b9-4f01-aa10-053a738780b8 (None, '5849b030-626e-47cb-ad90-3ce782d831b3', 'Default', '636abf41-5133-4252-9aef-8a32ec10668e', ['636abf41-5133-4252-9aef-8a32ec10668e'], 12, None, 5, 60, 10, 3) {} failed - stopping task</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,187::task::1194::TaskManager.Task::(stop) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::stopping in state preparing (force False)</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,187::task::974::TaskManager.Task::(_decref) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::ref 1 aborting True</div>
<div>Thread-497::INFO::2013-10-15 17:17:12,187::task::1151::TaskManager.Task::(prepare) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::aborting: Task is aborted: 'Cannot acquire host id' - code 661</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,187::task::1156::TaskManager.Task::(prepare) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::Prepare: aborted: Cannot acquire host id</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,187::task::974::TaskManager.Task::(_decref) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::ref 0 aborting True</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,187::task::909::TaskManager.Task::(_doAbort) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::Task._doAbort: force False</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,188::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,188::task::579::TaskManager.Task::(_updateState) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::moving from state preparing -> state aborting</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,188::task::534::TaskManager.Task::(__state_aborting) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::_aborting: recover policy none</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,188::task::579::TaskManager.Task::(_updateState) Task=`e53f26ec-66b9-4f01-aa10-053a738780b8`::moving from state aborting -> state failed</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,188::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.5849b030-626e-47cb-ad90-3ce782d831b3': < ResourceRef 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3', isValid: 'True' obj: 'None'>, 'Storage.636abf41-5133-4252-9aef-8a32ec10668e': < ResourceRef 'Storage.636abf41-5133-4252-9aef-8a32ec10668e', isValid: 'True' obj: 'None'>}</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,188::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,189::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3'</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,189::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3' (0 active users)</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,189::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3' is free, finding out if anyone is waiting for it.</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,189::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3', Clearing records.</div><div>
Thread-497::DEBUG::2013-10-15 17:17:12,189::resourceManager::615::ResourceManager::(releaseResource) Trying to release resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e'</div><div>Thread-497::DEBUG::2013-10-15 17:17:12,190::resourceManager::634::ResourceManager::(releaseResource) Released resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' (0 active users)</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,190::resourceManager::640::ResourceManager::(releaseResource) Resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e' is free, finding out if anyone is waiting for it.</div>
<div>Thread-497::DEBUG::2013-10-15 17:17:12,190::resourceManager::648::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.636abf41-5133-4252-9aef-8a32ec10668e', Clearing records.</div><div>
Thread-497::ERROR::2013-10-15 17:17:12,190::dispatcher::67::Storage.Dispatcher.Protect::(run) {'status': {'message': "Cannot acquire host id: ('636abf41-5133-4252-9aef-8a32ec10668e', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument'))", 'code': 661}}</div>
<div>Thread-504::DEBUG::2013-10-15 17:17:27,018::task::579::TaskManager.Task::(_updateState) Task=`4df542ea-0b2a-489a-b3d7-9299841ca2d6`::moving from state init -> state preparing</div><div>Thread-504::INFO::2013-10-15 17:17:27,019::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)</div>
<div>Thread-504::INFO::2013-10-15 17:17:27,019::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}</div><div>Thread-504::DEBUG::2013-10-15 17:17:27,019::task::1168::TaskManager.Task::(prepare) Task=`4df542ea-0b2a-489a-b3d7-9299841ca2d6`::finished: {}</div>
<div>Thread-504::DEBUG::2013-10-15 17:17:27,019::task::579::TaskManager.Task::(_updateState) Task=`4df542ea-0b2a-489a-b3d7-9299841ca2d6`::moving from state preparing -> state finished</div><div>Thread-504::DEBUG::2013-10-15 17:17:27,019::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div>
<div>Thread-504::DEBUG::2013-10-15 17:17:27,019::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-504::DEBUG::2013-10-15 17:17:27,019::task::974::TaskManager.Task::(_decref) Task=`4df542ea-0b2a-489a-b3d7-9299841ca2d6`::ref 0 aborting False</div>
<div>Thread-510::DEBUG::2013-10-15 17:17:42,194::task::579::TaskManager.Task::(_updateState) Task=`03b081b7-430b-4207-9272-ba5785f63cb1`::moving from state init -> state preparing</div><div>Thread-510::INFO::2013-10-15 17:17:42,194::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)</div>
<div>Thread-510::INFO::2013-10-15 17:17:42,194::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}</div><div>Thread-510::DEBUG::2013-10-15 17:17:42,195::task::1168::TaskManager.Task::(prepare) Task=`03b081b7-430b-4207-9272-ba5785f63cb1`::finished: {}</div>
<div>Thread-510::DEBUG::2013-10-15 17:17:42,195::task::579::TaskManager.Task::(_updateState) Task=`03b081b7-430b-4207-9272-ba5785f63cb1`::moving from state preparing -> state finished</div><div>Thread-510::DEBUG::2013-10-15 17:17:42,195::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div>
<div>Thread-510::DEBUG::2013-10-15 17:17:42,195::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-510::DEBUG::2013-10-15 17:17:42,195::task::974::TaskManager.Task::(_decref) Task=`03b081b7-430b-4207-9272-ba5785f63cb1`::ref 0 aborting False</div>
<div>Thread-516::DEBUG::2013-10-15 17:17:57,328::task::579::TaskManager.Task::(_updateState) Task=`535b1567-e5de-4a2d-9a34-080f3cea5de6`::moving from state init -> state preparing</div><div>Thread-516::INFO::2013-10-15 17:17:57,328::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)</div>
<div>Thread-516::INFO::2013-10-15 17:17:57,328::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}</div><div>Thread-516::DEBUG::2013-10-15 17:17:57,328::task::1168::TaskManager.Task::(prepare) Task=`535b1567-e5de-4a2d-9a34-080f3cea5de6`::finished: {}</div>
<div>Thread-516::DEBUG::2013-10-15 17:17:57,329::task::579::TaskManager.Task::(_updateState) Task=`535b1567-e5de-4a2d-9a34-080f3cea5de6`::moving from state preparing -> state finished</div><div>Thread-516::DEBUG::2013-10-15 17:17:57,329::resourceManager::939::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div>
<div>Thread-516::DEBUG::2013-10-15 17:17:57,329::resourceManager::976::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>Thread-516::DEBUG::2013-10-15 17:17:57,329::task::974::TaskManager.Task::(_decref) Task=`535b1567-e5de-4a2d-9a34-080f3cea5de6`::ref 0 aborting False</div>
</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><span style="font-family:arial,sans-serif;font-size:16px"><strong>Steve Dainard </strong></span><span style="font-size:12px"></span><br>
<span style="font-family:arial,sans-serif;font-size:12px">IT Infrastructure Manager<br>
<a href="http://miovision.com/" target="_blank">Miovision</a> | <em>Rethink Traffic</em><br>
519-513-2407 ex.250<br>
877-646-8476 (toll-free)<br>
<br>
<strong style="font-family:arial,sans-serif;font-size:13px;color:#999999"><a href="http://miovision.com/blog" target="_blank">Blog</a> | </strong><font color="#999999" style="font-family:arial,sans-serif;font-size:13px"><strong><a href="https://www.linkedin.com/company/miovision-technologies" target="_blank">LinkedIn</a> | <a href="https://twitter.com/miovision" target="_blank">Twitter</a> | <a href="https://www.facebook.com/miovision" target="_blank">Facebook</a></strong></font> </span>
<hr style="font-family:arial,sans-serif;font-size:13px;color:#333333;clear:both">
<div style="color:#999999;font-family:arial,sans-serif;font-size:13px;padding-top:5px">
<span style="font-family:arial,sans-serif;font-size:12px">Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, Canada | N2C 1L3</span><br>
<span style="font-family:arial,sans-serif;font-size:12px">This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.</span></div>
</div></div>
<br><br><div class="gmail_quote">On Thu, Oct 3, 2013 at 8:02 AM, Allon Mureinik <span dir="ltr"><<a href="mailto:amureini@redhat.com" target="_blank">amureini@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The root cause seems to be a SANlock issue:<br>
Thread-32382::ERROR:: 2013-09-20 13:16:34,126::clusterlock::145::initSANLock: initSANLock) Cannot initialize SANLock for domain 17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
<div class="im">Traceback (most recent call last):<br>
File "/usr/share/vdsm/storage/clusterlock.py", line 140, in initSANLock<br>
sanlock.init_lockspace(sdUUID, idsPath)<br>
SanlockException: (22, 'Sanlock lockspace init failure', 'Invalid argument')<br>
</div>Thread-32382::WARNING:: 2013-09-20 13:16:34,127::sd::428::Storage.StorageDomain: initSPMlease) lease did not initialize successfully<br>
<div class="im">Traceback (most recent call last):<br>
File "/usr/share/vdsm/storage/sd.py", line 423, in initSPMlease<br>
self._clusterLock.initLock()<br>
File "/usr/share/vdsm/storage/clusterlock.py", line 163, in initLock<br>
initSANLock(self._sdUUID, self._idsPath, self._leasesPath)<br>
File "/usr/share/vdsm/storage/clusterlock.py", line 146, in initSANLock<br>
raise se.ClusterLockInitError()<br>
ClusterLockInitError: Could not initialize cluster lock: ()<br>
<br>
</div>Can you include /var/log/sanlock.log and /var/log/messages please?<br>
<br>
----- Original Message -----<br>
<br>
> From: "Steve Dainard" <<a href="mailto:sdainard@miovision.com">sdainard@miovision.com</a>><br>
> To: "Deepak C Shetty" <<a href="mailto:deepakcs@linux.vnet.ibm.com">deepakcs@linux.vnet.ibm.com</a>><br>
> Cc: "users" <<a href="mailto:users@ovirt.org">users@ovirt.org</a>><br>
> Sent: Friday, September 20, 2013 8:30:59 PM<br>
> Subject: Re: [Users] Ovirt 3.3 Fedora 19 add gluster storage permissions<br>
> error<br>
<div class="HOEnZb"><div class="h5"><br>
> Awesome, thanks guys. Its weird that that article tells you to set with<br>
> 'key=value' rather than 'key value' must be some legacy stuff.<br>
<br>
> Once those changes are in place I hit a different error. Deepak, maybe you've<br>
> seen this one on new storage domain add:<br>
<br>
> [root@ovirt-manager2 ~]# tail -f /var/log/ovirt-engine/engine.log<br>
> 2013-09-20 13:16:36,226 ERROR<br>
> [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> (ajp--127.0.0.1-8702-9) Command CreateStoragePoolVDS execution failed.<br>
> Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed<br>
> to CreateStoragePoolVDS, error = Cannot acquire host id:<br>
> ('17d21ac7-5859-4f25-8de7-2a9433d50c11', SanlockException(22, 'Sanlock<br>
> lockspace add failure', 'Invalid argument'))<br>
> 2013-09-20 13:16:36,229 INFO<br>
> [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> (ajp--127.0.0.1-8702-9) FINISH, CreateStoragePoolVDSCommand, log id:<br>
> 672635cc<br>
> 2013-09-20 13:16:36,231 ERROR<br>
> [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> (ajp--127.0.0.1-8702-9) Command<br>
> org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand throw<br>
> Vdc Bll exception. With error message VdcBLLException:<br>
> org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:<br>
> VDSGenericException: VDSErrorException: Failed to CreateStoragePoolVDS,<br>
> error = Cannot acquire host id: ('17d21ac7-5859-4f25-8de7-2a9433d50c11',<br>
> SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument'))<br>
> (Failed with VDSM error AcquireHostIdFailure and code 661)<br>
> 2013-09-20 13:16:36,296 INFO<br>
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]<br>
> (ajp--127.0.0.1-8702-9) Correlation ID: 11070337, Call Stack: null, Custom<br>
> Event ID: -1, Message: Failed to attach Storage Domains to Data Center<br>
> Default. (User: admin@internal)<br>
> 2013-09-20 13:16:36,299 INFO<br>
> [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> (ajp--127.0.0.1-8702-9) Lock freed to object EngineLock [exclusiveLocks=<br>
> key: 5849b030-626e-47cb-ad90-3ce782d831b3 value: POOL<br>
> , sharedLocks= ]<br>
> 2013-09-20 13:16:36,387 INFO<br>
> [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand]<br>
> (ajp--127.0.0.1-8702-9) Command [id=293a1e97-e949-4c17-92c6-c01f2221204e]:<br>
> Compensating CHANGED_ENTITY of<br>
> org.ovirt.engine.core.common.businessentities.StoragePool; snapshot:<br>
> id=5849b030-626e-47cb-ad90-3ce782d831b3.<br>
> 2013-09-20 13:16:36,398 INFO<br>
> [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand]<br>
> (ajp--127.0.0.1-8702-9) Command [id=293a1e97-e949-4c17-92c6-c01f2221204e]:<br>
> Compensating NEW_ENTITY_ID of<br>
> org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot:<br>
> storagePoolId = 5849b030-626e-47cb-ad90-3ce782d831b3, storageId =<br>
> 17d21ac7-5859-4f25-8de7-2a9433d50c11.<br>
> 2013-09-20 13:16:36,425 INFO<br>
> [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand]<br>
> (ajp--127.0.0.1-8702-9) Command [id=293a1e97-e949-4c17-92c6-c01f2221204e]:<br>
> Compensating CHANGED_ENTITY of<br>
> org.ovirt.engine.core.common.businessentities.StorageDomainStatic; snapshot:<br>
> id=17d21ac7-5859-4f25-8de7-2a9433d50c11.<br>
> 2013-09-20 13:16:36,464 INFO<br>
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]<br>
> (ajp--127.0.0.1-8702-9) Correlation ID: 302ae6eb, Job ID:<br>
> 014ec59b-e6d7-4e5e-b588-4fb0dfa8f1c8, Call Stack: null, Custom Event ID: -1,<br>
> Message: Failed to attach Storage Domain rep2-virt to Data Center Default.<br>
> (User: admin@internal)<br>
<br>
> [root@ovirt001 ~]# tail -f /var/log/vdsm/vdsm.log<br>
> Thread-32374::DEBUG::2013-09-20<br>
> 13:16:18,107::task::579::TaskManager.Task::(_updateState)<br>
> Task=`f4cab975-d5fa-463a-990e-ab32686c6806`::moving from state init -> state<br>
> preparing<br>
> Thread-32374::INFO::2013-09-20<br>
> 13:16:18,107::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> repoStats(options=None)<br>
> Thread-32374::INFO::2013-09-20<br>
> 13:16:18,107::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> repoStats, Return response: {}<br>
> Thread-32374::DEBUG::2013-09-20<br>
> 13:16:18,108::task::1168::TaskManager.Task::(prepare)<br>
> Task=`f4cab975-d5fa-463a-990e-ab32686c6806`::finished: {}<br>
> Thread-32374::DEBUG::2013-09-20<br>
> 13:16:18,108::task::579::TaskManager.Task::(_updateState)<br>
> Task=`f4cab975-d5fa-463a-990e-ab32686c6806`::moving from state preparing -><br>
> state finished<br>
> Thread-32374::DEBUG::2013-09-20<br>
> 13:16:18,108::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> Owner.releaseAll requests {} resources {}<br>
> Thread-32374::DEBUG::2013-09-20<br>
> 13:16:18,108::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32374::DEBUG::2013-09-20<br>
> 13:16:18,108::task::974::TaskManager.Task::(_decref)<br>
> Task=`f4cab975-d5fa-463a-990e-ab32686c6806`::ref 0 aborting False<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,509::BindingXMLRPC::177::vds::(wrapper) client [10.0.0.34]<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,510::task::579::TaskManager.Task::(_updateState)<br>
> Task=`1ad55ba1-afaa-4524-b3fa-3d55a421e8bc`::moving from state init -> state<br>
> preparing<br>
> Thread-32379::INFO::2013-09-20<br>
> 13:16:29,510::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> connectStorageServer(domType=7,<br>
> spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '',<br>
> 'connection': '192.168.1.1:rep2-virt', 'iqn': '', 'portal': '', 'user': '',<br>
> 'vfs_type': 'glusterfs', 'password': '******', 'id':<br>
> '00000000-0000-0000-0000-000000000000'}], options=None)<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,516::hsm::2333::Storage.HSM::(__prefetchDomains) glusterDomPath:<br>
> glusterSD/*<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,523::hsm::2345::Storage.HSM::(__prefetchDomains) Found SD uuids: ()<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,523::hsm::2396::Storage.HSM::(connectStorageServer) knownSDs:<br>
> {b72b61d1-e11c-496d-ad3a-6f566a1f0ad1: storage.glusterSD.findDomain,<br>
> 983c4aa1-7b00-4d3b-b6ad-1fd2cf9297ce: storage.glusterSD.findDomain,<br>
> b91afb39-f96e-4eb3-bc6c-9f08fa16869c: storage.glusterSD.findDomain}<br>
> Thread-32379::INFO::2013-09-20<br>
> 13:16:29,523::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id':<br>
> '00000000-0000-0000-0000-000000000000'}]}<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,523::task::1168::TaskManager.Task::(prepare)<br>
> Task=`1ad55ba1-afaa-4524-b3fa-3d55a421e8bc`::finished: {'statuslist':<br>
> [{'status': 0, 'id': '00000000-0000-0000-0000-000000000000'}]}<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,524::task::579::TaskManager.Task::(_updateState)<br>
> Task=`1ad55ba1-afaa-4524-b3fa-3d55a421e8bc`::moving from state preparing -><br>
> state finished<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,524::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> Owner.releaseAll requests {} resources {}<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,524::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32379::DEBUG::2013-09-20<br>
> 13:16:29,524::task::974::TaskManager.Task::(_decref)<br>
> Task=`1ad55ba1-afaa-4524-b3fa-3d55a421e8bc`::ref 0 aborting False<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,888::BindingXMLRPC::177::vds::(wrapper) client [10.0.0.34]<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,888::task::579::TaskManager.Task::(_updateState)<br>
> Task=`a3ba925b-65ee-42a6-8506-927a06f63995`::moving from state init -> state<br>
> preparing<br>
> Thread-32382::INFO::2013-09-20<br>
> 13:16:29,889::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> createStorageDomain(storageType=7,<br>
> sdUUID='17d21ac7-5859-4f25-8de7-2a9433d50c11', domainName='rep2-virt',<br>
> typeSpecificArg='192.168.1.1:rep2-virt', domClass=1, domVersion='3',<br>
> options=None)<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,889::misc::807::SamplingMethod::(__call__) Trying to enter sampling<br>
> method (storage.sdc.refreshStorage)<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,889::misc::809::SamplingMethod::(__call__) Got in to sampling<br>
> method<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,889::misc::807::SamplingMethod::(__call__) Trying to enter sampling<br>
> method (storage.iscsi.rescan)<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,889::misc::809::SamplingMethod::(__call__) Got in to sampling<br>
> method<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,889::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo -n<br>
> /sbin/iscsiadm -m session -R' (cwd None)<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,904::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED: <err> =<br>
> 'iscsiadm: No session found.\n'; <rc> = 21<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:29,904::misc::817::SamplingMethod::(__call__) Returning last result<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:32,931::multipath::111::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo<br>
> -n /sbin/multipath' (cwd None)<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,255::multipath::111::Storage.Misc.excCmd::(rescan) SUCCESS: <err> =<br>
> ''; <rc> = 0<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,256::lvm::483::OperationMutex::(_invalidateAllPvs) Operation 'lvm<br>
> invalidate operation' got the operation mutex<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,256::lvm::485::OperationMutex::(_invalidateAllPvs) Operation 'lvm<br>
> invalidate operation' released the operation mutex<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,256::lvm::494::OperationMutex::(_invalidateAllVgs) Operation 'lvm<br>
> invalidate operation' got the operation mutex<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,256::lvm::496::OperationMutex::(_invalidateAllVgs) Operation 'lvm<br>
> invalidate operation' released the operation mutex<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,257::lvm::514::OperationMutex::(_invalidateAllLvs) Operation 'lvm<br>
> invalidate operation' got the operation mutex<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,257::lvm::516::OperationMutex::(_invalidateAllLvs) Operation 'lvm<br>
> invalidate operation' released the operation mutex<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,257::misc::817::SamplingMethod::(__call__) Returning last result<br>
> Thread-32382::ERROR::2013-09-20<br>
> 13:16:33,257::sdc::137::Storage.StorageDomainCache::(_findDomain) looking<br>
> for unfetched domain 17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> Thread-32382::ERROR::2013-09-20<br>
> 13:16:33,257::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain)<br>
> looking for domain 17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,258::lvm::374::OperationMutex::(_reloadvgs) Operation 'lvm reload<br>
> operation' got the operation mutex<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,259::lvm::311::Storage.Misc.excCmd::(cmd) '/usr/bin/sudo -n<br>
> /sbin/lvm vgs --config " devices { preferred_names = [\\"^/dev/mapper/\\"]<br>
> ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3<br>
> filter = [ \'r|.*|\' ] } global { locking_type=1 prioritise_write_locks=1<br>
> wait_for_locks=1 } backup { retain_min = 50 retain_days = 0 } " --noheadings<br>
> --units b --nosuffix --separator | -o<br>
> uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free<br>
> 17d21ac7-5859-4f25-8de7-2a9433d50c11' (cwd None)<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,285::lvm::311::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume<br>
> group "17d21ac7-5859-4f25-8de7-2a9433d50c11" not found\n'; <rc> = 5<br>
> Thread-32382::WARNING::2013-09-20<br>
> 13:16:33,286::lvm::379::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] ['<br>
> Volume group "17d21ac7-5859-4f25-8de7-2a9433d50c11" not found']<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,286::lvm::403::OperationMutex::(_reloadvgs) Operation 'lvm reload<br>
> operation' released the operation mutex<br>
> Thread-32382::ERROR::2013-09-20<br>
> 13:16:33,295::sdc::143::Storage.StorageDomainCache::(_findDomain) domain<br>
> 17d21ac7-5859-4f25-8de7-2a9433d50c11 not found<br>
> Traceback (most recent call last):<br>
> File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain<br>
> dom = findMethod(sdUUID)<br>
> File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain<br>
> raise se.StorageDomainDoesNotExist(sdUUID)<br>
> StorageDomainDoesNotExist: Storage domain does not exist:<br>
> ('17d21ac7-5859-4f25-8de7-2a9433d50c11',)<br>
> Thread-32382::INFO::2013-09-20<br>
> 13:16:33,295::nfsSD::69::Storage.StorageDomain::(create)<br>
> sdUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11 domainName=rep2-virt<br>
> remotePath=192.168.1.1:rep2-virt domClass=1<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,430::persistentDict::192::Storage.PersistentDict::(__init__)<br>
> Created a persistent dict with FileMetadataRW backend<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,445::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> lines (FileMetadataRW)=[]<br>
> Thread-32382::WARNING::2013-09-20<br>
> 13:16:33,446::persistentDict::256::Storage.PersistentDict::(refresh) data<br>
> has no embedded checksum - trust it as it is<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,447::persistentDict::167::Storage.PersistentDict::(transaction)<br>
> Starting transaction<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,448::persistentDict::173::Storage.PersistentDict::(transaction)<br>
> Flushing changes<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,449::persistentDict::299::Storage.PersistentDict::(flush) about to<br>
> write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt',<br>
> 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=',<br>
> 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=',<br>
> 'REMOTE_PATH=192.168.1.1:rep2-virt', 'ROLE=Regular',<br>
> 'SDUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11', 'TYPE=GLUSTERFS',<br>
> 'VERSION=3', '_SHA_CKSUM=2b07fbc8c65e20eef5180ab785016bde543c6746']<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,454::persistentDict::175::Storage.PersistentDict::(transaction)<br>
> Finished transaction<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,457::fileSD::153::Storage.StorageDomain::(__init__) Reading domain<br>
> in path<br>
> /rhev/data-center/mnt/glusterSD/192.168.1.1:rep2-virt/17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,457::persistentDict::192::Storage.PersistentDict::(__init__)<br>
> Created a persistent dict with FileMetadataRW backend<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,469::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt',<br>
> 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=',<br>
> 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=',<br>
> 'REMOTE_PATH=192.168.1.1:rep2-virt', 'ROLE=Regular',<br>
> 'SDUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11', 'TYPE=GLUSTERFS',<br>
> 'VERSION=3', '_SHA_CKSUM=2b07fbc8c65e20eef5180ab785016bde543c6746']<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,472::fileSD::535::Storage.StorageDomain::(imageGarbageCollector)<br>
> Removing remnants of deleted images []<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,472::resourceManager::420::ResourceManager::(registerNamespace)<br>
> Registering namespace '17d21ac7-5859-4f25-8de7-2a9433d50c11_imageNS'<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,473::resourceManager::420::ResourceManager::(registerNamespace)<br>
> Registering namespace '17d21ac7-5859-4f25-8de7-2a9433d50c11_volumeNS'<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:33,473::clusterlock::137::initSANLock::(initSANLock) Initializing<br>
> SANLock for domain 17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> Thread-32387::DEBUG::2013-09-20<br>
> 13:16:33,717::task::579::TaskManager.Task::(_updateState)<br>
> Task=`0e11c8e5-4e40-4d28-9eaf-129db67b2f4d`::moving from state init -> state<br>
> preparing<br>
> Thread-32387::INFO::2013-09-20<br>
> 13:16:33,718::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> repoStats(options=None)<br>
> Thread-32387::INFO::2013-09-20<br>
> 13:16:33,718::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> repoStats, Return response: {}<br>
> Thread-32387::DEBUG::2013-09-20<br>
> 13:16:33,718::task::1168::TaskManager.Task::(prepare)<br>
> Task=`0e11c8e5-4e40-4d28-9eaf-129db67b2f4d`::finished: {}<br>
> Thread-32387::DEBUG::2013-09-20<br>
> 13:16:33,718::task::579::TaskManager.Task::(_updateState)<br>
> Task=`0e11c8e5-4e40-4d28-9eaf-129db67b2f4d`::moving from state preparing -><br>
> state finished<br>
> Thread-32387::DEBUG::2013-09-20<br>
> 13:16:33,718::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> Owner.releaseAll requests {} resources {}<br>
> Thread-32387::DEBUG::2013-09-20<br>
> 13:16:33,718::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32387::DEBUG::2013-09-20<br>
> 13:16:33,719::task::974::TaskManager.Task::(_decref)<br>
> Task=`0e11c8e5-4e40-4d28-9eaf-129db67b2f4d`::ref 0 aborting False<br>
> Thread-32382::ERROR::2013-09-20<br>
> 13:16:34,126::clusterlock::145::initSANLock::(initSANLock) Cannot initialize<br>
> SANLock for domain 17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> Traceback (most recent call last):<br>
> File "/usr/share/vdsm/storage/clusterlock.py", line 140, in initSANLock<br>
> sanlock.init_lockspace(sdUUID, idsPath)<br>
> SanlockException: (22, 'Sanlock lockspace init failure', 'Invalid argument')<br>
> Thread-32382::WARNING::2013-09-20<br>
> 13:16:34,127::sd::428::Storage.StorageDomain::(initSPMlease) lease did not<br>
> initialize successfully<br>
> Traceback (most recent call last):<br>
> File "/usr/share/vdsm/storage/sd.py", line 423, in initSPMlease<br>
> self._clusterLock.initLock()<br>
> File "/usr/share/vdsm/storage/clusterlock.py", line 163, in initLock<br>
> initSANLock(self._sdUUID, self._idsPath, self._leasesPath)<br>
> File "/usr/share/vdsm/storage/clusterlock.py", line 146, in initSANLock<br>
> raise se.ClusterLockInitError()<br>
> ClusterLockInitError: Could not initialize cluster lock: ()<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:34,127::hsm::2624::Storage.HSM::(createStorageDomain) knownSDs:<br>
> {b72b61d1-e11c-496d-ad3a-6f566a1f0ad1: storage.glusterSD.findDomain,<br>
> 983c4aa1-7b00-4d3b-b6ad-1fd2cf9297ce: storage.glusterSD.findDomain,<br>
> b91afb39-f96e-4eb3-bc6c-9f08fa16869c: storage.glusterSD.findDomain,<br>
> 17d21ac7-5859-4f25-8de7-2a9433d50c11: storage.glusterSD.findDomain}<br>
> Thread-32382::INFO::2013-09-20<br>
> 13:16:34,128::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> createStorageDomain, Return response: None<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:34,128::task::1168::TaskManager.Task::(prepare)<br>
> Task=`a3ba925b-65ee-42a6-8506-927a06f63995`::finished: None<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:34,128::task::579::TaskManager.Task::(_updateState)<br>
> Task=`a3ba925b-65ee-42a6-8506-927a06f63995`::moving from state preparing -><br>
> state finished<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:34,128::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> Owner.releaseAll requests {} resources {}<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:34,128::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32382::DEBUG::2013-09-20<br>
> 13:16:34,129::task::974::TaskManager.Task::(_decref)<br>
> Task=`a3ba925b-65ee-42a6-8506-927a06f63995`::ref 0 aborting False<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,219::BindingXMLRPC::177::vds::(wrapper) client [10.0.0.34]<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,219::task::579::TaskManager.Task::(_updateState)<br>
> Task=`a0d5d4d6-dcb7-4293-bf8a-cf1e2204f586`::moving from state init -> state<br>
> preparing<br>
> Thread-32389::INFO::2013-09-20<br>
> 13:16:34,220::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> getStorageDomainStats(sdUUID='17d21ac7-5859-4f25-8de7-2a9433d50c11',<br>
> options=None)<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,220::resourceManager::197::ResourceManager.Request::(__init__)<br>
> ResName=`Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11`ReqID=`9f37d808-9ad2-4c06-99ef-449b43049e80`::Request<br>
> was made in '/usr/share/vdsm/storage/hsm.py' line '2784' at<br>
> 'getStorageDomainStats'<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,220::resourceManager::541::ResourceManager::(registerResource)<br>
> Trying to register resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11'<br>
> for lock type 'shared'<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,221::resourceManager::600::ResourceManager::(registerResource)<br>
> Resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11' is free. Now locking<br>
> as 'shared' (1 active user)<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,221::resourceManager::237::ResourceManager.Request::(grant)<br>
> ResName=`Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11`ReqID=`9f37d808-9ad2-4c06-99ef-449b43049e80`::Granted<br>
> request<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,221::task::811::TaskManager.Task::(resourceAcquired)<br>
> Task=`a0d5d4d6-dcb7-4293-bf8a-cf1e2204f586`::_resourcesAcquired:<br>
> Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11 (shared)<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,221::task::974::TaskManager.Task::(_decref)<br>
> Task=`a0d5d4d6-dcb7-4293-bf8a-cf1e2204f586`::ref 1 aborting False<br>
> Thread-32389::INFO::2013-09-20<br>
> 13:16:34,223::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> getStorageDomainStats, Return response: {'stats': {'mdasize': 0,<br>
> 'mdathreshold': True, 'mdavalid': True, 'diskfree': '14182986809344',<br>
> 'disktotal': '14199600185344', 'mdafree': 0}}<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,223::task::1168::TaskManager.Task::(prepare)<br>
> Task=`a0d5d4d6-dcb7-4293-bf8a-cf1e2204f586`::finished: {'stats': {'mdasize':<br>
> 0, 'mdathreshold': True, 'mdavalid': True, 'diskfree': '14182986809344',<br>
> 'disktotal': '14199600185344', 'mdafree': 0}}<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,223::task::579::TaskManager.Task::(_updateState)<br>
> Task=`a0d5d4d6-dcb7-4293-bf8a-cf1e2204f586`::moving from state preparing -><br>
> state finished<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,223::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> Owner.releaseAll requests {} resources<br>
> {'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11': < ResourceRef<br>
> 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11', isValid: 'True' obj:<br>
> 'None'>}<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,224::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,224::resourceManager::615::ResourceManager::(releaseResource)<br>
> Trying to release resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11'<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,224::resourceManager::634::ResourceManager::(releaseResource)<br>
> Released resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11' (0 active<br>
> users)<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,224::resourceManager::640::ResourceManager::(releaseResource)<br>
> Resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11' is free, finding out<br>
> if anyone is waiting for it.<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,224::resourceManager::648::ResourceManager::(releaseResource) No<br>
> one is waiting for resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11',<br>
> Clearing records.<br>
> Thread-32389::DEBUG::2013-09-20<br>
> 13:16:34,225::task::974::TaskManager.Task::(_decref)<br>
> Task=`a0d5d4d6-dcb7-4293-bf8a-cf1e2204f586`::ref 0 aborting False<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,099::BindingXMLRPC::177::vds::(wrapper) client [10.0.0.34]<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,099::task::579::TaskManager.Task::(_updateState)<br>
> Task=`089c5f71-9cbb-4626-8f2d-3ed3547a98cd`::moving from state init -> state<br>
> preparing<br>
> Thread-32390::INFO::2013-09-20<br>
> 13:16:35,099::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> connectStorageServer(domType=7,<br>
> spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '',<br>
> 'connection': '192.168.1.1:rep2-virt', 'iqn': '', 'portal': '', 'user': '',<br>
> 'vfs_type': 'glusterfs', 'password': '******', 'id':<br>
> 'cecee482-87e1-4ecc-8bda-0e0ec84d7792'}], options=None)<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,105::hsm::2333::Storage.HSM::(__prefetchDomains) glusterDomPath:<br>
> glusterSD/*<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,112::hsm::2345::Storage.HSM::(__prefetchDomains) Found SD uuids:<br>
> ('17d21ac7-5859-4f25-8de7-2a9433d50c11',<br>
> '17d21ac7-5859-4f25-8de7-2a9433d50c11')<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,113::hsm::2396::Storage.HSM::(connectStorageServer) knownSDs:<br>
> {b72b61d1-e11c-496d-ad3a-6f566a1f0ad1: storage.glusterSD.findDomain,<br>
> 983c4aa1-7b00-4d3b-b6ad-1fd2cf9297ce: storage.glusterSD.findDomain,<br>
> b91afb39-f96e-4eb3-bc6c-9f08fa16869c: storage.glusterSD.findDomain,<br>
> 17d21ac7-5859-4f25-8de7-2a9433d50c11: storage.glusterSD.findDomain}<br>
> Thread-32390::INFO::2013-09-20<br>
> 13:16:35,113::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id':<br>
> 'cecee482-87e1-4ecc-8bda-0e0ec84d7792'}]}<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,113::task::1168::TaskManager.Task::(prepare)<br>
> Task=`089c5f71-9cbb-4626-8f2d-3ed3547a98cd`::finished: {'statuslist':<br>
> [{'status': 0, 'id': 'cecee482-87e1-4ecc-8bda-0e0ec84d7792'}]}<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,113::task::579::TaskManager.Task::(_updateState)<br>
> Task=`089c5f71-9cbb-4626-8f2d-3ed3547a98cd`::moving from state preparing -><br>
> state finished<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,113::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> Owner.releaseAll requests {} resources {}<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,114::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32390::DEBUG::2013-09-20<br>
> 13:16:35,114::task::974::TaskManager.Task::(_decref)<br>
> Task=`089c5f71-9cbb-4626-8f2d-3ed3547a98cd`::ref 0 aborting False<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,148::BindingXMLRPC::177::vds::(wrapper) client [10.0.0.34]<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,148::task::579::TaskManager.Task::(_updateState)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::moving from state init -> state<br>
> preparing<br>
> Thread-32393::INFO::2013-09-20<br>
> 13:16:35,148::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> createStoragePool(poolType=None,<br>
> spUUID='5849b030-626e-47cb-ad90-3ce782d831b3', poolName='Default',<br>
> masterDom='17d21ac7-5859-4f25-8de7-2a9433d50c11',<br>
> domList=['17d21ac7-5859-4f25-8de7-2a9433d50c11'], masterVersion=9,<br>
> lockPolicy=None, lockRenewalIntervalSec=5, leaseTimeSec=60,<br>
> ioOpTimeoutSec=10, leaseRetries=3, options=None)<br>
> Thread-32393::INFO::2013-09-20<br>
> 13:16:35,149::fileSD::315::Storage.StorageDomain::(validate)<br>
> sdUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,161::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt',<br>
> 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=',<br>
> 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=',<br>
> 'REMOTE_PATH=192.168.1.1:rep2-virt', 'ROLE=Regular',<br>
> 'SDUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11', 'TYPE=GLUSTERFS',<br>
> 'VERSION=3', '_SHA_CKSUM=2b07fbc8c65e20eef5180ab785016bde543c6746']<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,162::resourceManager::197::ResourceManager.Request::(__init__)<br>
> ResName=`Storage.5849b030-626e-47cb-ad90-3ce782d831b3`ReqID=`31060ad0-6633-4bbf-a859-b3f0c27af760`::Request<br>
> was made in '/usr/share/vdsm/storage/hsm.py' line '954' at<br>
> 'createStoragePool'<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,162::resourceManager::541::ResourceManager::(registerResource)<br>
> Trying to register resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3'<br>
> for lock type 'exclusive'<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,162::resourceManager::600::ResourceManager::(registerResource)<br>
> Resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3' is free. Now locking<br>
> as 'exclusive' (1 active user)<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,162::resourceManager::237::ResourceManager.Request::(grant)<br>
> ResName=`Storage.5849b030-626e-47cb-ad90-3ce782d831b3`ReqID=`31060ad0-6633-4bbf-a859-b3f0c27af760`::Granted<br>
> request<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,163::task::811::TaskManager.Task::(resourceAcquired)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::_resourcesAcquired:<br>
> Storage.5849b030-626e-47cb-ad90-3ce782d831b3 (exclusive)<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,163::task::974::TaskManager.Task::(_decref)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::ref 1 aborting False<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,163::resourceManager::197::ResourceManager.Request::(__init__)<br>
> ResName=`Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11`ReqID=`db5f52a0-d455-419c-b8a5-86fc6b695571`::Request<br>
> was made in '/usr/share/vdsm/storage/hsm.py' line '956' at<br>
> 'createStoragePool'<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,164::resourceManager::541::ResourceManager::(registerResource)<br>
> Trying to register resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11'<br>
> for lock type 'exclusive'<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,164::resourceManager::600::ResourceManager::(registerResource)<br>
> Resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11' is free. Now locking<br>
> as 'exclusive' (1 active user)<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,164::resourceManager::237::ResourceManager.Request::(grant)<br>
> ResName=`Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11`ReqID=`db5f52a0-d455-419c-b8a5-86fc6b695571`::Granted<br>
> request<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,165::task::811::TaskManager.Task::(resourceAcquired)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::_resourcesAcquired:<br>
> Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11 (exclusive)<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,165::task::974::TaskManager.Task::(_decref)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::ref 1 aborting False<br>
> Thread-32393::INFO::2013-09-20<br>
> 13:16:35,166::sp::592::Storage.StoragePool::(create)<br>
> spUUID=5849b030-626e-47cb-ad90-3ce782d831b3 poolName=Default<br>
> master_sd=17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> domList=['17d21ac7-5859-4f25-8de7-2a9433d50c11'] masterVersion=9<br>
> {'LEASETIMESEC': 60, 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3,<br>
> 'LOCKRENEWALINTERVALSEC': 5}<br>
> Thread-32393::INFO::2013-09-20<br>
> 13:16:35,166::fileSD::315::Storage.StorageDomain::(validate)<br>
> sdUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,177::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt',<br>
> 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=',<br>
> 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=',<br>
> 'REMOTE_PATH=192.168.1.1:rep2-virt', 'ROLE=Regular',<br>
> 'SDUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11', 'TYPE=GLUSTERFS',<br>
> 'VERSION=3', '_SHA_CKSUM=2b07fbc8c65e20eef5180ab785016bde543c6746']<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,188::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt',<br>
> 'IOOPTIMEOUTSEC=1', 'LEASERETRIES=3', 'LEASETIMESEC=5', 'LOCKPOLICY=',<br>
> 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=',<br>
> 'REMOTE_PATH=192.168.1.1:rep2-virt', 'ROLE=Regular',<br>
> 'SDUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11', 'TYPE=GLUSTERFS',<br>
> 'VERSION=3', '_SHA_CKSUM=2b07fbc8c65e20eef5180ab785016bde543c6746']<br>
> Thread-32393::WARNING::2013-09-20<br>
> 13:16:35,189::fileUtils::167::Storage.fileUtils::(createdir) Dir<br>
> /rhev/data-center/5849b030-626e-47cb-ad90-3ce782d831b3 already exists<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,189::persistentDict::167::Storage.PersistentDict::(transaction)<br>
> Starting transaction<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,189::persistentDict::173::Storage.PersistentDict::(transaction)<br>
> Flushing changes<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,189::persistentDict::299::Storage.PersistentDict::(flush) about to<br>
> write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=rep2-virt',<br>
> 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=',<br>
> 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=',<br>
> 'REMOTE_PATH=192.168.1.1:rep2-virt', 'ROLE=Regular',<br>
> 'SDUUID=17d21ac7-5859-4f25-8de7-2a9433d50c11', 'TYPE=GLUSTERFS',<br>
> 'VERSION=3', '_SHA_CKSUM=61b814a65ea3ede1f0ae1d58e139adc06bf9eda5']<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:35,194::persistentDict::175::Storage.PersistentDict::(transaction)<br>
> Finished transaction<br>
> Thread-32393::INFO::2013-09-20<br>
> 13:16:35,194::clusterlock::174::SANLock::(acquireHostId) Acquiring host id<br>
> for domain 17d21ac7-5859-4f25-8de7-2a9433d50c11 (id: 250)<br>
> Thread-32393::ERROR::2013-09-20<br>
> 13:16:36,196::task::850::TaskManager.Task::(_setError)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::Unexpected error<br>
> Traceback (most recent call last):<br>
> File "/usr/share/vdsm/storage/task.py", line 857, in _run<br>
> return fn(*args, **kargs)<br>
> File "/usr/share/vdsm/logUtils.py", line 45, in wrapper<br>
> res = f(*args, **kwargs)<br>
> File "/usr/share/vdsm/storage/hsm.py", line 960, in createStoragePool<br>
> masterVersion, leaseParams)<br>
> File "/usr/share/vdsm/storage/sp.py", line 617, in create<br>
> self._acquireTemporaryClusterLock(msdUUID, leaseParams)<br>
> File "/usr/share/vdsm/storage/sp.py", line 559, in<br>
> _acquireTemporaryClusterLock<br>
> msd.acquireHostId( <a href="http://self.id" target="_blank">self.id</a> )<br>
> File "/usr/share/vdsm/storage/sd.py", line 458, in acquireHostId<br>
> self._clusterLock.acquireHostId(hostId, async)<br>
> File "/usr/share/vdsm/storage/clusterlock.py", line 189, in acquireHostId<br>
> raise se.AcquireHostIdFailure(self._sdUUID, e)<br>
> AcquireHostIdFailure: Cannot acquire host id:<br>
> ('17d21ac7-5859-4f25-8de7-2a9433d50c11', SanlockException(22, 'Sanlock<br>
> lockspace add failure', 'Invalid argument'))<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,196::task::869::TaskManager.Task::(_run)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::Task._run:<br>
> 72348d40-8442-4dbf-bc66-1d354da5fc31 (None,<br>
> '5849b030-626e-47cb-ad90-3ce782d831b3', 'Default',<br>
> '17d21ac7-5859-4f25-8de7-2a9433d50c11',<br>
> ['17d21ac7-5859-4f25-8de7-2a9433d50c11'], 9, None, 5, 60, 10, 3) {} failed -<br>
> stopping task<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,197::task::1194::TaskManager.Task::(stop)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::stopping in state preparing<br>
> (force False)<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,197::task::974::TaskManager.Task::(_decref)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::ref 1 aborting True<br>
> Thread-32393::INFO::2013-09-20<br>
> 13:16:36,197::task::1151::TaskManager.Task::(prepare)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::aborting: Task is aborted:<br>
> 'Cannot acquire host id' - code 661<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,197::task::1156::TaskManager.Task::(prepare)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::Prepare: aborted: Cannot<br>
> acquire host id<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,197::task::974::TaskManager.Task::(_decref)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::ref 0 aborting True<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,197::task::909::TaskManager.Task::(_doAbort)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::Task._doAbort: force False<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,198::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,198::task::579::TaskManager.Task::(_updateState)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::moving from state preparing -><br>
> state aborting<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,198::task::534::TaskManager.Task::(__state_aborting)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::_aborting: recover policy none<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,198::task::579::TaskManager.Task::(_updateState)<br>
> Task=`72348d40-8442-4dbf-bc66-1d354da5fc31`::moving from state aborting -><br>
> state failed<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,198::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> Owner.releaseAll requests {} resources<br>
> {'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11': < ResourceRef<br>
> 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11', isValid: 'True' obj:<br>
> 'None'>, 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3': < ResourceRef<br>
> 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3', isValid: 'True' obj:<br>
> 'None'>}<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,198::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> Owner.cancelAll requests {}<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,199::resourceManager::615::ResourceManager::(releaseResource)<br>
> Trying to release resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11'<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,199::resourceManager::634::ResourceManager::(releaseResource)<br>
> Released resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11' (0 active<br>
> users)<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,199::resourceManager::640::ResourceManager::(releaseResource)<br>
> Resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11' is free, finding out<br>
> if anyone is waiting for it.<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,199::resourceManager::648::ResourceManager::(releaseResource) No<br>
> one is waiting for resource 'Storage.17d21ac7-5859-4f25-8de7-2a9433d50c11',<br>
> Clearing records.<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,199::resourceManager::615::ResourceManager::(releaseResource)<br>
> Trying to release resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3'<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,200::resourceManager::634::ResourceManager::(releaseResource)<br>
> Released resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3' (0 active<br>
> users)<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,200::resourceManager::640::ResourceManager::(releaseResource)<br>
> Resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3' is free, finding out<br>
> if anyone is waiting for it.<br>
> Thread-32393::DEBUG::2013-09-20<br>
> 13:16:36,200::resourceManager::648::ResourceManager::(releaseResource) No<br>
> one is waiting for resource 'Storage.5849b030-626e-47cb-ad90-3ce782d831b3',<br>
> Clearing records.<br>
> Thread-32393::ERROR::2013-09-20<br>
> 13:16:36,200::dispatcher::67::Storage.Dispatcher.Protect::(run) {'status':<br>
> {'message': "Cannot acquire host id:<br>
> ('17d21ac7-5859-4f25-8de7-2a9433d50c11', SanlockException(22, 'Sanlock<br>
> lockspace add failure', 'Invalid argument'))", 'code': 661}}<br>
> Thread-32398::DEBUG::2013-09-20<br>
> 13:16:48,921::task::579::TaskManager.Task::(_updateState)<br>
> Task=`a5bce432-622b-499b-a216-d9a1f876e3ca`::moving from state init -> state<br>
> preparing<br>
> Thread-32398::INFO::2013-09-20<br>
> 13:16:48,922::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> repoStats(options=None)<br>
> Thread-32398::INFO::2013-09-20<br>
> 13:16:48,922::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> repoStats, Return response: {}<br>
> Thread-32398::DEBUG::2013-09-20<br>
> 13:16:48,922::task::1168::TaskManager.Task::(prepare)<br>
> Task=`a5bce432-622b-499b-a216-d9a1f876e3ca`::finished: {}<br>
> Thread-32398::DEBUG::2013-09-20<br>
> 13:16:48,922::task::579::TaskManager.Task::(_updateState)<br>
> Task=`a5bce432-622b-499b-a216-d9a1f876e3ca`::moving from state preparing -><br>
> state finished<br>
<br>
> Steve Dainard<br>
> Infrastructure Manager<br>
> Miovision | Rethink Traffic<br>
> 519-513-2407 ex.250<br>
> 877-646-8476 (toll-free)<br>
<br>
> Blog | LinkedIn | Twitter | Facebook<br>
</div></div><div class="HOEnZb"><div class="h5">> Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON,<br>
> Canada | N2C 1L3<br>
> This e-mail may contain information that is privileged or confidential. If<br>
> you are not the intended recipient, please delete the e-mail and any<br>
> attachments and notify us immediately.<br>
<br>
> On Fri, Sep 20, 2013 at 12:23 PM, Deepak C Shetty <<br>
> <a href="mailto:deepakcs@linux.vnet.ibm.com">deepakcs@linux.vnet.ibm.com</a> > wrote:<br>
<br>
> > Either you can use the volume set .. option as mentioned in the wikipage<br>
><br>
> > --or --<br>
><br>
> > If the Gluster volume is added / managed to the oVirt UI.. go to "Volumes"<br>
> > tab, select the gluster volume and<br>
><br>
> > click on "Optimize for virt. store". That should also set the volume<br>
> > options<br>
> > in addition to few other things<br>
><br>
<br>
> > thanx,<br>
><br>
> > deepak<br>
><br>
<br>
> > ______________________________ _________________<br>
><br>
> > Users mailing list<br>
><br>
> > <a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
><br>
> > <a href="http://lists.ovirt.org/" target="_blank">http://lists.ovirt.org/</a> mailman/listinfo/users<br>
><br>
<br>
> _______________________________________________<br>
> Users mailing list<br>
> <a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
> <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</div></div></blockquote></div><br></div>