<div dir="ltr">Happens every time I try to add a POSIX SD type glusterfs.<div><br></div><div>Logs attached.</div><div><br></div><div>Thanks,</div><div>Steve</div><div class="gmail_extra"><br clear="all"><div><div dir="ltr">
<br></div></div>
<br><br><div class="gmail_quote">On Tue, Feb 4, 2014 at 3:26 PM, Nir Soffer <span dir="ltr"><<a href="mailto:nsoffer@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=nsoffer@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">nsoffer@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">----- Original Message -----<br>
> From: "Steve Dainard" <<a href="mailto:sdainard@miovision.com" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=sdainard@miovision.com&cc=&bcc=&su=&body=','_blank');return false;">sdainard@miovision.com</a>><br>
</div><div class="im">> To: "Elad Ben Aharon" <<a href="mailto:ebenahar@redhat.com" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=ebenahar@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">ebenahar@redhat.com</a>><br>
> Cc: "Nir Soffer" <<a href="mailto:nsoffer@redhat.com" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=nsoffer@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">nsoffer@redhat.com</a>>, "users" <<a href="mailto:users@ovirt.org" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">users@ovirt.org</a>>, "Aharon Canan" <<a href="mailto:acanan@redhat.com" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=acanan@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">acanan@redhat.com</a>><br>
> Sent: Tuesday, February 4, 2014 10:14:10 PM<br>
> Subject: Re: [Users] Ovirt 3.3.2 Cannot attach POSIX (gluster) storage domain<br>
><br>
> I should be able to provide any logs required, I've reverted to my NFS<br>
> storage domain but can move a host over to POSIX whenever necessary.<br>
<br>
</div>Hi Steve,<br>
<br>
Can you reproduce this issue or it happens only once?<br>
<br>
If you can reproduce it, can you send vdsm and engine logs that document<br>
how the storage domain was created what happened before failure?<br>
<br>
Thanks,<br>
Nir<br>
<br>
><br>
> *Steve Dainard *<br>
> IT Infrastructure Manager<br>
> Miovision <<a href="http://miovision.com/" target="_blank">http://miovision.com/</a>> | *Rethink Traffic*<br>
> <a href="tel:519-513-2407" value="+15195132407">519-513-2407</a> ex.250<br>
> <a href="tel:877-646-8476" value="+18776468476">877-646-8476</a> (toll-free)<br>
><br>
> *Blog <<a href="http://miovision.com/blog" target="_blank">http://miovision.com/blog</a>> | **LinkedIn<br>
> <<a href="https://www.linkedin.com/company/miovision-technologies" target="_blank">https://www.linkedin.com/company/miovision-technologies</a>> | Twitter<br>
> <<a href="https://twitter.com/miovision" target="_blank">https://twitter.com/miovision</a>> | Facebook<br>
> <<a href="https://www.facebook.com/miovision" target="_blank">https://www.facebook.com/miovision</a>>*<br>
> ------------------------------<br>
<div class="HOEnZb"><div class="h5">> Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON,<br>
> Canada | N2C 1L3<br>
> This e-mail may contain information that is privileged or confidential. If<br>
> you are not the intended recipient, please delete the e-mail and any<br>
> attachments and notify us immediately.<br>
><br>
><br>
> On Tue, Feb 4, 2014 at 10:20 AM, Elad Ben Aharon <<a href="mailto:ebenahar@redhat.com" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=ebenahar@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">ebenahar@redhat.com</a>>wrote:<br>
><br>
> > Nir,<br>
> > Can you take a look? the user gets the same Sanlock exception as reported<br>
> > here: <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1046430" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1046430</a><br>
> ><br>
> > ----- Original Message -----<br>
> > From: "Steve Dainard" <<a href="mailto:sdainard@miovision.com" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=sdainard@miovision.com&cc=&bcc=&su=&body=','_blank');return false;">sdainard@miovision.com</a>><br>
> > To: "users" <<a href="mailto:users@ovirt.org" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">users@ovirt.org</a>><br>
> > Sent: Tuesday, February 4, 2014 5:09:43 PM<br>
> > Subject: [Users] Ovirt 3.3.2 Cannot attach POSIX (gluster) storage domain<br>
> ><br>
> > I can successfully create a POSIX storage domain backed by gluster, but at<br>
> > the end of creation I get an error message "failed to acquire host id".<br>
> ><br>
> > Note that I have successfully created/activated NFS DC/SD on the same<br>
> > ovirt/hosts.<br>
> ><br>
> > I have some logs when I tried to attach to the DC after failure:<br>
> ><br>
> > engine.log<br>
> ><br>
> > 2014-02-04 09:54:04,324 INFO<br>
> > [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> > (ajp--127.0.0.1-8702-3) [1dd40406] Lock Acquired to object EngineLock [ex<br>
> > clusiveLocks= key: 8c4e8898-c91a-4d49-98e8-b6467791a9cc value: POOL<br>
> > , sharedLocks= ]<br>
> > 2014-02-04 09:54:04,473 INFO<br>
> > [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> > (pool-6-thread-42) [1dd40406] Running command: AddStoragePoolWithStorages<br>
> > Command internal: false. Entities affected : ID:<br>
> > 8c4e8898-c91a-4d49-98e8-b6467791a9cc Type: StoragePool<br>
> > 2014-02-04 09:54:04,673 INFO<br>
> > [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Running command: ConnectStorageToVdsCommand<br>
> > intern<br>
> > al: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa<br>
> > Type: System<br>
> > 2014-02-04 09:54:04,682 INFO<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] START, ConnectStorageServerVDSCommand(<br>
> > HostName = ovirt001, HostId = 48f13d47-8346-4ff6-81ca-4f4324069db3,<br>
> > storagePoolId = 00000000-0000-0000-0000-000000000000, storageType =<br>
> > POSIXFS, connectionList = [{ id: 87f9<br>
> > ff74-93c4-4fe5-9a56-ed5338290af9, connection: 10.0.10.3:/rep2, iqn: null,<br>
> > vfsType: glusterfs, mountOptions: null, nfsVersion: null, nfsRetrans: null,<br>
> > nfsTimeo: null };]), lo<br>
> > g id: 332ff091<br>
> > 2014-02-04 09:54:05,089 INFO<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] FINISH, ConnectStorageServerVDSCommand<br>
> > , return: {87f9ff74-93c4-4fe5-9a56-ed5338290af9=0}, log id: 332ff091<br>
> > 2014-02-04 09:54:05,093 INFO<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] START, CreateStoragePoolVDSCommand(HostNa<br>
> > me = ovirt001, HostId = 48f13d47-8346-4ff6-81ca-4f4324069db3,<br>
> > storagePoolId=8c4e8898-c91a-4d49-98e8-b6467791a9cc, storageType=POSIXFS,<br>
> > storagePoolName=IT, masterDomainId=471<br>
> > 487ed-2946-4dfc-8ec3-96546006be12,<br>
> > domainsIdList=[471487ed-2946-4dfc-8ec3-96546006be12], masterVersion=3), log<br>
> > id: 1be84579<br>
> > 2014-02-04 09:54:08,833 ERROR<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorag<br>
> > ePoolVDSCommand] (pool-6-thread-42) [3f86c31b] Failed in<br>
> > CreateStoragePoolVDS method<br>
> > 2014-02-04 09:54:08,834 ERROR<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Error code AcquireHostIdFailure and error<br>
> > message VDSGenericException: VDSErrorException: Failed to<br>
> > CreateStoragePoolVDS, error = Cannot acquire host id:<br>
> > ('471487ed-2946-4dfc-8ec3-96546006be12', SanlockException(22, 'Sanlock<br>
> > lockspace add failure', 'Invalid argument'))<br>
> > 2014-02-04 09:54:08,835 INFO<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Command<br>
> > org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand<br>
> > return value<br>
> > StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=661,<br>
> > mMessage=Cannot acquire host id: ('471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > SanlockException(22, 'Sanlock lockspace add failure', 'Invalid<br>
> > argument'))]]<br>
> > 2014-02-04 09:54:08,836 INFO<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] HostName = ovirt001<br>
> > 2014-02-04 09:54:08,840 ERROR<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Command CreateStoragePoolVDS execution<br>
> > failed. Exception: VDSErrorException: VDSGenericException:<br>
> > VDSErrorException: Failed to CreateStoragePoolVDS, error = Cannot acquire<br>
> > host id: ('471487ed-2946-4dfc-8ec3-96546006be12', SanlockException(22,<br>
> > 'Sanlock lockspace add failure', 'Invalid argument'))<br>
> > 2014-02-04 09:54:08,840 INFO<br>
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand]<br>
> > (pool-6-thread-42) [3f86c31b] FINISH, CreateStoragePoolVDSCommand, log id:<br>
> > 1be84579<br>
> > 2014-02-04 09:54:08,841 ERROR<br>
> > [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Command<br>
> > org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand throw<br>
> > Vdc Bll exception. With error message VdcBLLException:<br>
> > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:<br>
> > VDSGenericException: VDSErrorException: Failed to CreateStoragePoolVDS,<br>
> > error = Cannot acquire host id: ('471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument'))<br>
> > (Failed with error AcquireHostIdFailure and code 661)<br>
> > 2014-02-04 09:54:08,867 INFO<br>
> > [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Command<br>
> > [id=373987cb-b54d-4174-b4a9-195be631f0d7]: Compensating CHANGED_ENTITY of<br>
> > org.ovirt.engine.core.common.businessentities.StoragePool; snapshot:<br>
> > id=8c4e8898-c91a-4d49-98e8-b6467791a9cc.<br>
> > 2014-02-04 09:54:08,871 INFO<br>
> > [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Command<br>
> > [id=373987cb-b54d-4174-b4a9-195be631f0d7]: Compensating NEW_ENTITY_ID of<br>
> > org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot:<br>
> > storagePoolId = 8c4e8898-c91a-4d49-98e8-b6467791a9cc, storageId =<br>
> > 471487ed-2946-4dfc-8ec3-96546006be12.<br>
> > 2014-02-04 09:54:08,879 INFO<br>
> > [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Command<br>
> > [id=373987cb-b54d-4174-b4a9-195be631f0d7]: Compensating CHANGED_ENTITY of<br>
> > org.ovirt.engine.core.common.businessentities.StorageDomainStatic;<br>
> > snapshot: id=471487ed-2946-4dfc-8ec3-96546006be12.<br>
> > 2014-02-04 09:54:08,951 INFO<br>
> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]<br>
> > (pool-6-thread-42) [3f86c31b] Correlation ID: 1dd40406, Job ID:<br>
> > 07003dff-9e0e-42ae-8f88-6b055b45f797, Call Stack: null, Custom Event ID:<br>
> > -1, Message: Failed to attach Storage Domains to Data Center IT. (User:<br>
> > admin@internal)<br>
> > 2014-02-04 09:54:08,975 INFO<br>
> > [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand]<br>
> > (pool-6-thread-42) [3f86c31b] Lock freed to object EngineLock<br>
> > [exclusiveLocks= key: 8c4e8898-c91a-4d49-98e8-b6467791a9cc value: POOL<br>
> > , sharedLocks= ]<br>
> ><br>
> > vdsm.log<br>
> ><br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:04,692::BindingXMLRPC::167::vds::(wrapper) client [10.0.10.2] flowID<br>
> > [3f86c31b]<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:04,692::task::579::TaskManager.Task::(_updateState)<br>
> > Task=`218dcde9-bbc7-4d5a-ad53-0bab556c6261`::moving from state init -><br>
> > state preparing<br>
> > Thread-30::INFO::2014-02-04<br>
> > 09:54:04,693::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> > connectStorageServer(domType=6,<br>
> > spUUID='00000000-0000-0000-0000-000000000000', conList=[{'port': '',<br>
> > 'connection': '10.0.10.3:/rep2', 'iqn': '', 'portal': '', 'user': '',<br>
> > 'vfs_type': 'glusterfs', 'password': '******', 'id':<br>
> > '87f9ff74-93c4-4fe5-9a56-ed5338290af9'}], options=None)<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:04,698::mount::226::Storage.Misc.excCmd::(_runcmd) '/usr/bin/sudo -n<br>
> > /bin/mount -t glusterfs 10.0.10.3:/rep2<br>
> > /rhev/data-center/mnt/10.0.10.3:_rep2'<br>
> > (cwd None)<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,067::hsm::2315::Storage.HSM::(__prefetchDomains) posix local path:<br>
> > /rhev/data-center/mnt/10.0.10.3:_rep2<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,078::hsm::2333::Storage.HSM::(__prefetchDomains) Found SD uuids:<br>
> > ('471487ed-2946-4dfc-8ec3-96546006be12',)<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,078::hsm::2389::Storage.HSM::(connectStorageServer) knownSDs:<br>
> > {471487ed-2946-4dfc-8ec3-96546006be12: storage.nfsSD.findDomain}<br>
> > Thread-30::INFO::2014-02-04<br>
> > 09:54:05,078::logUtils::47::dispatcher::(wrapper) Run and protect:<br>
> > connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id':<br>
> > '87f9ff74-93c4-4fe5-9a56-ed5338290af9'}]}<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,079::task::1168::TaskManager.Task::(prepare)<br>
> > Task=`218dcde9-bbc7-4d5a-ad53-0bab556c6261`::finished: {'statuslist':<br>
> > [{'status': 0, 'id': '87f9ff74-93c4-4fe5-9a56-ed5338290af9'}]}<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,079::task::579::TaskManager.Task::(_updateState)<br>
> > Task=`218dcde9-bbc7-4d5a-ad53-0bab556c6261`::moving from state preparing -><br>
> > state finished<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,079::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> > Owner.releaseAll requests {} resources {}<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,079::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> > Owner.cancelAll requests {}<br>
> > Thread-30::DEBUG::2014-02-04<br>
> > 09:54:05,079::task::974::TaskManager.Task::(_decref)<br>
> > Task=`218dcde9-bbc7-4d5a-ad53-0bab556c6261`::ref 0 aborting False<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,098::BindingXMLRPC::167::vds::(wrapper) client [10.0.10.2] flowID<br>
> > [3f86c31b]<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,099::task::579::TaskManager.Task::(_updateState)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::moving from state init -><br>
> > state preparing<br>
> > Thread-31::INFO::2014-02-04<br>
> > 09:54:05,099::logUtils::44::dispatcher::(wrapper) Run and protect:<br>
> > createStoragePool(poolType=None,<br>
> > spUUID='8c4e8898-c91a-4d49-98e8-b6467791a9cc', poolName='IT',<br>
> > masterDom='471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > domList=['471487ed-2946-4dfc-8ec3-96546006be12'], masterVersion=3,<br>
> > lockPolicy=None, lockRenewalIntervalSec=5, leaseTimeSec=60,<br>
> > ioOpTimeoutSec=10, leaseRetries=3, options=None)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,099::misc::809::SamplingMethod::(__call__) Trying to enter<br>
> > sampling method (storage.sdc.refreshStorage)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,100::misc::811::SamplingMethod::(__call__) Got in to sampling<br>
> > method<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,100::misc::809::SamplingMethod::(__call__) Trying to enter<br>
> > sampling method (storage.iscsi.rescan)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,100::misc::811::SamplingMethod::(__call__) Got in to sampling<br>
> > method<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,100::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) '/usr/bin/sudo<br>
> > -n /sbin/iscsiadm -m session -R' (cwd None)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,114::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED: <err> =<br>
> > 'iscsiadm: No session found.\n'; <rc> = 21<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:05,115::misc::819::SamplingMethod::(__call__) Returning last result<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,144::multipath::112::Storage.Misc.excCmd::(rescan) '/usr/bin/sudo<br>
> > -n /sbin/multipath -r' (cwd None)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,331::multipath::112::Storage.Misc.excCmd::(rescan) SUCCESS: <err><br>
> > = ''; <rc> = 0<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,332::lvm::510::OperationMutex::(_invalidateAllPvs) Operation 'lvm<br>
> > invalidate operation' got the operation mutex<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,333::lvm::512::OperationMutex::(_invalidateAllPvs) Operation 'lvm<br>
> > invalidate operation' released the operation mutex<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,333::lvm::521::OperationMutex::(_invalidateAllVgs) Operation 'lvm<br>
> > invalidate operation' got the operation mutex<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,333::lvm::523::OperationMutex::(_invalidateAllVgs) Operation 'lvm<br>
> > invalidate operation' released the operation mutex<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,333::lvm::541::OperationMutex::(_invalidateAllLvs) Operation 'lvm<br>
> > invalidate operation' got the operation mutex<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,334::lvm::543::OperationMutex::(_invalidateAllLvs) Operation 'lvm<br>
> > invalidate operation' released the operation mutex<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,334::misc::819::SamplingMethod::(__call__) Returning last result<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,499::fileSD::137::Storage.StorageDomain::(__init__) Reading domain<br>
> > in path /rhev/data-center/mnt/<a href="http://10.0.10.3" target="_blank">10.0.10.3</a>:<br>
> > _rep2/471487ed-2946-4dfc-8ec3-96546006be12<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,605::persistentDict::192::Storage.PersistentDict::(__init__)<br>
> > Created a persistent dict with FileMetadataRW backend<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,647::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> > lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=gluster-store-rep2',<br>
> > 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=',<br>
> > 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.0.10.3:/rep2',<br>
> > 'ROLE=Regular', 'SDUUID=471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > 'TYPE=POSIXFS', 'VERSION=3',<br>
> > '_SHA_CKSUM=469191aac3fb8ef504b6a4d301b6d8be6fffece1']<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,683::fileSD::558::Storage.StorageDomain::(imageGarbageCollector)<br>
> > Removing remnants of deleted images []<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,684::resourceManager::420::ResourceManager::(registerNamespace)<br>
> > Registering namespace '471487ed-2946-4dfc-8ec3-96546006be12_imageNS'<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,684::resourceManager::420::ResourceManager::(registerNamespace)<br>
> > Registering namespace '471487ed-2946-4dfc-8ec3-96546006be12_volumeNS'<br>
> > Thread-31::INFO::2014-02-04<br>
> > 09:54:07,684::fileSD::299::Storage.StorageDomain::(validate)<br>
> > sdUUID=471487ed-2946-4dfc-8ec3-96546006be12<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,692::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> > lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=gluster-store-rep2',<br>
> > 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=',<br>
> > 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.0.10.3:/rep2',<br>
> > 'ROLE=Regular', 'SDUUID=471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > 'TYPE=POSIXFS', 'VERSION=3',<br>
> > '_SHA_CKSUM=469191aac3fb8ef504b6a4d301b6d8be6fffece1']<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,693::resourceManager::197::ResourceManager.Request::(__init__)<br>
> > ResName=`Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc`ReqID=`e0a3d477-b953-49d9-ab78-67695a6bc6d5`::Request<br>
> > was made in '/usr/share/vdsm/storage/hsm.py' line '971' at<br>
> > 'createStoragePool'<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,693::resourceManager::541::ResourceManager::(registerResource)<br>
> > Trying to register resource 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc'<br>
> > for lock type 'exclusive'<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,693::resourceManager::600::ResourceManager::(registerResource)<br>
> > Resource 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc' is free. Now<br>
> > locking as 'exclusive' (1 active user)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,693::resourceManager::237::ResourceManager.Request::(grant)<br>
> > ResName=`Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc`ReqID=`e0a3d477-b953-49d9-ab78-67695a6bc6d5`::Granted<br>
> > request<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,694::task::811::TaskManager.Task::(resourceAcquired)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::_resourcesAcquired:<br>
> > Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc (exclusive)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,694::task::974::TaskManager.Task::(_decref)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::ref 1 aborting False<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,694::resourceManager::197::ResourceManager.Request::(__init__)<br>
> > ResName=`Storage.471487ed-2946-4dfc-8ec3-96546006be12`ReqID=`bc20dd7e-d351-47c5-8ed3-78b1b11d703a`::Request<br>
> > was made in '/usr/share/vdsm/storage/hsm.py' line '973' at<br>
> > 'createStoragePool'<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,694::resourceManager::541::ResourceManager::(registerResource)<br>
> > Trying to register resource 'Storage.471487ed-2946-4dfc-8ec3-96546006be12'<br>
> > for lock type 'exclusive'<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,695::resourceManager::600::ResourceManager::(registerResource)<br>
> > Resource 'Storage.471487ed-2946-4dfc-8ec3-96546006be12' is free. Now<br>
> > locking as 'exclusive' (1 active user)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,695::resourceManager::237::ResourceManager.Request::(grant)<br>
> > ResName=`Storage.471487ed-2946-4dfc-8ec3-96546006be12`ReqID=`bc20dd7e-d351-47c5-8ed3-78b1b11d703a`::Granted<br>
> > request<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,695::task::811::TaskManager.Task::(resourceAcquired)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::_resourcesAcquired:<br>
> > Storage.471487ed-2946-4dfc-8ec3-96546006be12 (exclusive)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,695::task::974::TaskManager.Task::(_decref)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::ref 1 aborting False<br>
> > Thread-31::INFO::2014-02-04<br>
> > 09:54:07,696::sp::593::Storage.StoragePool::(create)<br>
> > spUUID=8c4e8898-c91a-4d49-98e8-b6467791a9cc poolName=IT<br>
> > master_sd=471487ed-2946-4dfc-8ec3-96546006be12<br>
> > domList=['471487ed-2946-4dfc-8ec3-96546006be12'] masterVersion=3<br>
> > {'LEASETIMESEC': 60, 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3,<br>
> > 'LOCKRENEWALINTERVALSEC': 5}<br>
> > Thread-31::INFO::2014-02-04<br>
> > 09:54:07,696::fileSD::299::Storage.StorageDomain::(validate)<br>
> > sdUUID=471487ed-2946-4dfc-8ec3-96546006be12<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,703::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> > lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=gluster-store-rep2',<br>
> > 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=',<br>
> > 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.0.10.3:/rep2',<br>
> > 'ROLE=Regular', 'SDUUID=471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > 'TYPE=POSIXFS', 'VERSION=3',<br>
> > '_SHA_CKSUM=469191aac3fb8ef504b6a4d301b6d8be6fffece1']<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,710::persistentDict::234::Storage.PersistentDict::(refresh) read<br>
> > lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=gluster-store-rep2',<br>
> > 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=',<br>
> > 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.0.10.3:/rep2',<br>
> > 'ROLE=Regular', 'SDUUID=471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > 'TYPE=POSIXFS', 'VERSION=3',<br>
> > '_SHA_CKSUM=469191aac3fb8ef504b6a4d301b6d8be6fffece1']<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,711::persistentDict::167::Storage.PersistentDict::(transaction)<br>
> > Starting transaction<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:07,711::persistentDict::175::Storage.PersistentDict::(transaction)<br>
> > Finished transaction<br>
> > Thread-31::INFO::2014-02-04<br>
> > 09:54:07,711::clusterlock::174::SANLock::(acquireHostId) Acquiring host id<br>
> > for domain 471487ed-2946-4dfc-8ec3-96546006be12 (id: 250)<br>
> > Thread-31::ERROR::2014-02-04<br>
> > 09:54:08,722::task::850::TaskManager.Task::(_setError)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::Unexpected error<br>
> > Traceback (most recent call last):<br>
> > File "/usr/share/vdsm/storage/task.py", line 857, in _run<br>
> > return fn(*args, **kargs)<br>
> > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper<br>
> > res = f(*args, **kwargs)<br>
> > File "/usr/share/vdsm/storage/hsm.py", line 977, in createStoragePool<br>
> > masterVersion, leaseParams)<br>
> > File "/usr/share/vdsm/storage/sp.py", line 618, in create<br>
> > self._acquireTemporaryClusterLock(msdUUID, leaseParams)<br>
> > File "/usr/share/vdsm/storage/sp.py", line 560, in<br>
> > _acquireTemporaryClusterLock<br>
> > msd.acquireHostId( <a href="http://self.id" target="_blank">self.id</a> )<br>
> > File "/usr/share/vdsm/storage/sd.py", line 458, in acquireHostId<br>
> > self._clusterLock.acquireHostId(hostId, async)<br>
> > File "/usr/share/vdsm/storage/clusterlock.py", line 189, in acquireHostId<br>
> > raise se.AcquireHostIdFailure(self._sdUUID, e)<br>
> > AcquireHostIdFailure: Cannot acquire host id:<br>
> > ('471487ed-2946-4dfc-8ec3-96546006be12', SanlockException(22, 'Sanlock<br>
> > lockspace add failure', 'Invalid argument'))<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,826::task::869::TaskManager.Task::(_run)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::Task._run:<br>
> > 66924dbf-5a1c-473e-a158-d038aae38dc3 (None,<br>
> > '8c4e8898-c91a-4d49-98e8-b6467791a9cc', 'IT',<br>
> > '471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > ['471487ed-2946-4dfc-8ec3-96546006be12'], 3, None, 5, 60, 10, 3) {} failed<br>
> > - stopping task<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,826::task::1194::TaskManager.Task::(stop)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::stopping in state preparing<br>
> > (force False)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,826::task::974::TaskManager.Task::(_decref)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::ref 1 aborting True<br>
> > Thread-31::INFO::2014-02-04<br>
> > 09:54:08,826::task::1151::TaskManager.Task::(prepare)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::aborting: Task is aborted:<br>
> > 'Cannot acquire host id' - code 661<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,826::task::1156::TaskManager.Task::(prepare)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::Prepare: aborted: Cannot<br>
> > acquire host id<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,827::task::974::TaskManager.Task::(_decref)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::ref 0 aborting True<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,827::task::909::TaskManager.Task::(_doAbort)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::Task._doAbort: force False<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,827::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> > Owner.cancelAll requests {}<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,827::task::579::TaskManager.Task::(_updateState)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::moving from state preparing -><br>
> > state aborting<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,827::task::534::TaskManager.Task::(__state_aborting)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::_aborting: recover policy none<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,827::task::579::TaskManager.Task::(_updateState)<br>
> > Task=`66924dbf-5a1c-473e-a158-d038aae38dc3`::moving from state aborting -><br>
> > state failed<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,827::resourceManager::939::ResourceManager.Owner::(releaseAll)<br>
> > Owner.releaseAll requests {} resources<br>
> > {'Storage.471487ed-2946-4dfc-8ec3-96546006be12': < ResourceRef<br>
> > 'Storage.471487ed-2946-4dfc-8ec3-96546006be12', isValid: 'True' obj:<br>
> > 'None'>, 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc': < ResourceRef<br>
> > 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc', isValid: 'True' obj:<br>
> > 'None'>}<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,828::resourceManager::976::ResourceManager.Owner::(cancelAll)<br>
> > Owner.cancelAll requests {}<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,828::resourceManager::615::ResourceManager::(releaseResource)<br>
> > Trying to release resource 'Storage.471487ed-2946-4dfc-8ec3-96546006be12'<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,828::resourceManager::634::ResourceManager::(releaseResource)<br>
> > Released resource 'Storage.471487ed-2946-4dfc-8ec3-96546006be12' (0 active<br>
> > users)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,828::resourceManager::640::ResourceManager::(releaseResource)<br>
> > Resource 'Storage.471487ed-2946-4dfc-8ec3-96546006be12' is free, finding<br>
> > out if anyone is waiting for it.<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,828::resourceManager::648::ResourceManager::(releaseResource) No<br>
> > one is waiting for resource 'Storage.471487ed-2946-4dfc-8ec3-96546006be12',<br>
> > Clearing records.<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,828::resourceManager::615::ResourceManager::(releaseResource)<br>
> > Trying to release resource 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc'<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,829::resourceManager::634::ResourceManager::(releaseResource)<br>
> > Released resource 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc' (0 active<br>
> > users)<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,829::resourceManager::640::ResourceManager::(releaseResource)<br>
> > Resource 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc' is free, finding<br>
> > out if anyone is waiting for it.<br>
> > Thread-31::DEBUG::2014-02-04<br>
> > 09:54:08,829::resourceManager::648::ResourceManager::(releaseResource) No<br>
> > one is waiting for resource 'Storage.8c4e8898-c91a-4d49-98e8-b6467791a9cc',<br>
> > Clearing records.<br>
> > Thread-31::ERROR::2014-02-04<br>
> > 09:54:08,829::dispatcher::67::Storage.Dispatcher.Protect::(run) {'status':<br>
> > {'message': "Cannot acquire host id:<br>
> > ('471487ed-2946-4dfc-8ec3-96546006be12', SanlockException(22, 'Sanlock<br>
> > lockspace add failure', 'Invalid argument'))", 'code': 661}}<br>
> ><br>
> ><br>
> > Storage domain metadata file:<br>
> ><br>
> > CLASS=Data<br>
> > DESCRIPTION=gluster-store-rep2<br>
> > IOOPTIMEOUTSEC=10<br>
> > LEASERETRIES=3<br>
> > LEASETIMESEC=60<br>
> > LOCKPOLICY=<br>
> > LOCKRENEWALINTERVALSEC=5<br>
> > POOL_UUID=<br>
> > REMOTE_PATH=10.0.10.3:/rep2<br>
> > ROLE=Regular<br>
> > SDUUID=471487ed-2946-4dfc-8ec3-96546006be12<br>
> > TYPE=POSIXFS<br>
> > VERSION=3<br>
> > _SHA_CKSUM=469191aac3fb8ef504b6a4d301b6d8be6fffece1<br>
> ><br>
> ><br>
> ><br>
> > Steve Dainard<br>
> > IT Infrastructure Manager<br>
> > Miovision | Rethink Traffic<br>
> > 519-513-2407 ex.250<br>
> > 877-646-8476 (toll-free)<br>
> ><br>
> > Blog | LinkedIn | Twitter | Facebook<br>
> > Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON,<br>
> > Canada | N2C 1L3<br>
> > This e-mail may contain information that is privileged or confidential. If<br>
> > you are not the intended recipient, please delete the e-mail and any<br>
> > attachments and notify us immediately.<br>
> ><br>
> > _______________________________________________<br>
> > Users mailing list<br>
> > <a href="mailto:Users@ovirt.org" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a><br>
> > <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
> ><br>
><br>
</div></div></blockquote></div><br></div></div>