Re: [Users] Failed to attach Storage Domain export to Data Center

</o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US =
</p><p class=3DMsoNormal><span lang=3DEN-US =
<pre>_______________________________________________<o:p></o:p></pre><pr= e>Users mailing list<o:p></o:p></pre><pre><a =
This is a multipart message in MIME format. ------=_NextPart_000_0063_01CD15C2.E4EF9E40 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I found the metadata in the file ./dom_md/metadata Changed =E2=80=9Cb23c7ab6-b1d4-4888-8d4a-adc78e61db38=E2=80=9D to = =E2=80=9Ce3e565d6-8104-11e1-afd2-001cc43ceea0=E2=80=9D (SDUUID) The data I gave the command:=20 [root@noc-3-synt ~]# vdsClient -s 0 getConnectedStoragePoolsList e3e565d6-8104-11e1-afd2-001cc43ceea0 =20 Found in checksum vdsm.log: Thread-36836::WARNING::2012-04-08 = 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) = data seal is broken metadata declares = `4df87518878fd0d74932917b83aba9672f92dff2` should be = `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines=3D{'VERSION': '0', = 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', = 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', = 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', = 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', = 'CLASS': 'Backup'}) MetaDataSealIsBroken: Meta Data seal is broken (checksum mismatch): = 'cksum =3D 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum =3D = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59' =20 Changed it. As a result, the metadata file: =20 [root@noc-2-synt dom_md]# cat metadata CLASS=3DBackup DESCRIPTION=3Dexport IOOPTIMEOUTSEC=3D1 LEASERETRIES=3D3 LEASETIMESEC=3D5 LOCKPOLICY=3D LOCKRENEWALINTERVALSEC=3D5 MASTER_VERSION=3D0 POOL_UUID=3D801e35d4-7405-11e1-b152-001cc43ceea0 REMOTE_PATH=3D10.1.20.2:/home/nfs4 ROLE=3DRegular SDUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 TYPE=3DNFS VERSION=3D0 _SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd2b59 =20 =20 attachment to the ovirt fails: Thread-35311::INFO::2012-04-08 = 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or = directory: = '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc= 43ceea0' Thread-35311::ERROR::2012-04-08 = 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected error =20 I renamed the folder that contains the data to = e3e565d6-8104-11e1-afd2-001cc43ceea0 for = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID) =20 attachment to the ovirt fails =3D( messages: Error while executing action AttachStorageDomainToPool: Error in storage = domain action =20 =20 =20 engine.log: 2012-04-08 11:53:10,557 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage 2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 3dbf104d 2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 4ac74ee6 2012-04-08 11:53:10,685 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 4ac74ee6 2012-04-08 11:53:10,686 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 11:53:10,689 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 53bd874a 2012-04-08 11:53:10,690 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 3dbf104d 2012-04-08 11:53:10,691 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 11:53:10,693 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 3129fd68 2012-04-08 11:53:10,714 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 53bd874a 2012-04-08 11:53:10,716 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 3129fd68 2012-04-08 11:53:10,718 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 2b559a22 2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Failed in AttachStorageDomainVDS method 2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-08 11:53:10,738 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-08 11:53:10,752 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) START, SpmStopVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 6b2f2dd2 2012-04-08 11:53:10,764 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0 2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: 6b2f2dd2 2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) Irs placed on server null failed. Proceed Failover 2012-04-08 11:53:11,850 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-49) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool 2012-04-08 11:53:11,881 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, = storage pool Default 2012-04-08 11:53:11,886 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, = prevId -1, LVER 8 2012-04-08 11:53:11,888 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) START, SpmStartVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D8, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 382e702 2012-04-08 11:53:11,905 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) spmStart polling started: taskId =3D = f65c40e4-956f-4c01-ab3f-5c43f2853806 =20 =20 Vdsm.log: Thread-38311::INFO::2012-04-08 = 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getStoragePoolInfo, Return response: {'status': {'message': = 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': = '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': = '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', = 'master_ver': 1, 'lver': 10}, 'dominfo': = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', = 'diskfree': '490079256576', 'alerts': [], 'disktotal': '516524343296'}}} Thread-38312::DEBUG::2012-04-08 = 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38312::INFO::2012-04-08 = 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-38312::DEBUG::2012-04-08 = 11:54:32,953::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state = preparing Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-38312::DEBUG::2012-04-08 = 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd None) Thread-38312::DEBUG::2012-04-08 = 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = =3D ''; <rc> =3D 0 Thread-38312::DEBUG::2012-04-08 = 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd None) Thread-38312::DEBUG::2012-04-08 = 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-38312::DEBUG::2012-04-08 = 11:54:33,022::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> = state finished Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting False Thread-38312::INFO::2012-04-08 = 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38313::DEBUG::2012-04-08 = 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38313::INFO::2012-04-08 = 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-38313::DEBUG::2012-04-08 = 11:54:33,038::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state = preparing Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> = state finished Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting False Thread-38313::INFO::2012-04-08 = 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38314::DEBUG::2012-04-08 = 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38314::INFO::2012-04-08 = 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state = preparing Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain' Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Granted request Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting False Thread-38314::ERROR::2012-04-08 = 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run return fn(*args, **kargs) File "/usr/share/vdsm/storage/spm.py", line 115, in run return self.func(*args, **kwargs) File "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain vars.task.getExclusiveLock(STORAGE, sdUUID) File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout) File "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire raise ValueError("Owner %s: acquire: resource %s is already = acquired" % (str(self), fullName)) ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: = 2b306544-6fcc-443b-9ea6-23ae4b71d422 = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force = False) Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting True Thread-38314::INFO::2012-04-08 = 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100 Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting True Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force False Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> = state aborting Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy none Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> = state failed Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-38314::DEBUG::2012-04-08 = 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run result =3D ctask.prepare(self.func, *args, **kwargs) File "/usr/share/vdsm/storage/task.py", line 1178, in prepare raise self.error ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired =20 Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38315::INFO::2012-04-08 = 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state = preparing Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D10 spmId=3D2 Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 10}} Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> = state finished Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting False Thread-38315::INFO::2012-04-08 = 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}} Thread-38316::DEBUG::2012-04-08 = 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38316::INFO::2012-04-08 = 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: () Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state = preparing Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) Entry. Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': {}} Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> = state finished Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting False Thread-38316::INFO::2012-04-08 = 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}} Thread-38317::DEBUG::2012-04-08 = 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38317::INFO::2012-04-08 = 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state = preparing Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) Entry. Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: {} Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop' Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Granted request Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting False Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at '_stop' Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive' Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user) Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted request Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users) Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it. Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records. Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) Entry. Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) Return Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D10', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', '_SHA_CKSUM=3D1d1cd722cad65ebb2edfa75641c30a5b97302f84'] Thread-38317::DEBUG::2012-04-08 = 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction Thread-38317::INFO::2012-04-08 = 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099 Thread-38317::DEBUG::2012-04-08 = 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/) Thread-38317::DEBUG::2012-04-08 = 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock = released successfully Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: None Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> = state finished Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-38317::DEBUG::2012-04-08 = 11:54:34,168::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting False Thread-38317::INFO::2012-04-08 = 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}} Thread-38318::INFO::2012-04-08 = 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: () Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state = preparing Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', = 'lastCheck': 1333900465.46615, 'valid': True, 'code': 0}} Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> = state finished Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38318::DEBUG::2012-04-08 = 11:54:34,397::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting False =20 =20 =20 From: Rami Vaknin [mailto:rvaknin@redhat.com]=20 Sent: Sunday, April 08, 2012 7:02 PM To: =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 Subject: Re: [Users] Failed to attach Storage Domain export to Data = Center =20 On 04/08/2012 03:48 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote: 1. Change the spuuid in the domain's metadata file to be the spuuid of = the new datacenter (the metadata can be found in dom_md/ dir somewhere = inside the stoarge domain's directories tree), you can take the new = spuuid from any host running on the new data center by running: vdsClient -s 0 getConnectedStoragePoolsList 2. Try to attach the storage domain again, this time it will fail due to = broken checksum on the metadata file, go to vdsm.log of the spm and look = for "checksum", it will tell you what the right checksum should be like, = copy it to the metadata file. This time the attachment should work. Hi.=20 Thank you for the help you provided me in the mailing list. I dropped the settings using the ovirt - engine-cleanup Ovirt reinstalled and tried to restore the virtual machine from the = export of storage domain. If you try an attachment domain ovirt error: Error while executing action AttachStorageDomainToPool: The Domain is = already used by another Data Center.<br/>In order to use it, detach it = from the Data Center it is currently attached to. =20 =20 In the logs: =20 2012-04-08 08:39:49,898 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: Storage 2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 1985db9c 2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 17f02ea3 2012-04-08 08:39:50,004 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 17f02ea3 2012-04-08 08:39:50,005 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 08:39:50,008 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 6f4db784 2012-04-08 08:39:50,009 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 1985db9c 2012-04-08 08:39:50,010 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 08:39:50,013 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 42baf149 2012-04-08 08:39:50,031 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 6f4db784 2012-04-08 08:39:50,035 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 42baf149 2012-04-08 08:39:50,037 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: 6f22bcac 2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IrsOperationFailedNoFailoverException: IRSGenericException: = IRSErrorException: Storage domain already attached to pool: = 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0' 2012-04-08 08:39:53,186 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: = 6f22bcac 2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command = org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw = Vdc Bll exception. With error message VdcBLLException: = org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverExc= eption: IRSGenericException: IRSErrorException: Storage domain already = attached to pool: 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0' 2012-04-08 08:39:53,190 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command [id=3Db717db6a-a582-49ba-92c1-6bb09340789a]: = Compensating NEW_ENTITY_ID of = org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; = snapshot: storagePoolId =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, = storageId =3D b23c7ab6-b1d4-4888-8d4a-adc78e61db38. =20 I did not deactivate the export domain before clearing the = configuration. How can I be in this case? I can manually remove the flag = of the domain? _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users --=20 =20 Thanks, =20 Rami Vaknin, QE @ Red Hat, TLV, IL. ------=_NextPart_000_0063_01CD15C2.E4EF9E40 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"><head> <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; = charset=3Dutf-8"> <meta name=3DGenerator content=3D"Microsoft Word 14 (filtered = medium)"><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:"Arial Unicode MS"; panose-1:2 11 6 4 2 2 2 2 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} @font-face {font-family:Consolas; panose-1:2 11 6 9 2 2 4 3 2 4;} @font-face {font-family:"\@Arial Unicode MS"; panose-1:2 11 6 4 2 2 2 2 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"; color:black; mso-fareast-language:EN-US;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} pre {mso-style-priority:99; = mso-style-link:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML =D0=97=D0=BD=D0=B0=D0=BA"; margin:0cm; margin-bottom:.0001pt; font-size:10.0pt; font-family:"Courier New"; color:black;} span.EmailStyle17 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:windowtext;} span.HTML = {mso-style-name:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML =D0=97=D0=BD=D0=B0=D0=BA"; mso-style-priority:99; = mso-style-link:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML"; font-family:Consolas; color:black; mso-fareast-language:EN-US;} span.EmailStyle20 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body bgcolor=3Dwhite lang=3DRU = link=3Dblue vlink=3Dpurple><div class=3DWordSection1><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>I found the = metadata in the file ./dom_md/metadata<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Changed = =E2=80=9Cb23c7ab6-b1d4-4888-8d4a-adc78e61db38=E2=80=9D to = =E2=80=9Ce3e565d6-8104-11e1-afd2-001cc43ceea0=E2=80=9D=C2=A0 = (SDUUID)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>The data I gave the command: = <o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>[root@noc-3-synt ~]# vdsClient -s 0 = getConnectedStoragePoolsList<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></= span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Found in = </span><span lang=3DEN-US>checksum</span><span lang=3DEN-US = style=3D'color:#1F497D'> vdsm.log:<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-36836::WARNING::2012-04-08 = 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) = data seal is broken metadata declares = `4df87518878fd0d74932917b83aba9672f92dff2` should be = `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines=3D{'VERSION': '0', = 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', = 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', = 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', = 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', = 'CLASS': 'Backup'})<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>MetaDataSealIsBroken: Meta Data = seal is broken (checksum mismatch): 'cksum =3D = 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum =3D = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59'<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Changed = it.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>As a result, the metadata = file:<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>[root@noc-2-synt dom_md]# cat = metadata<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>CLASS=3DBackup<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>DESCRIPTION=3Dexport<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>IOOPTIMEOUTSEC=3D1<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASERETRIES=3D3<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASETIMESEC=3D5<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKPOLICY=3D<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKRENEWALINTERVALSEC=3D5<o:p></o:p></span></p><= p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>MASTER_VERSION=3D0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>POOL_UUID=3D801e35d4-7405-11e1-b152-001cc43ceea0<= o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>REMOTE_PATH=3D10.1.20.2:/home/nfs4<o:p></o:p></sp= an></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ROLE=3DRegular<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>SDUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0<o:p= style=3D'color:#1F497D'>TYPE=3DNFS<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>VERSION=3D0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>_SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd= 2b59<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>attachment = to the ovirt fails:<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-35311::INFO::2012-04-08 = 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or = directory: = '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc= 43ceea0'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-35311::ERROR::2012-04-08 = 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected = error<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>I renamed = the folder that contains the data to = e3e565d6-8104-11e1-afd2-001cc43ceea0 for = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>attachment = to the ovirt fails=C2=A0 =3D(<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>messages:<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'font-size:12.0pt;font-family:"Arial Unicode = MS","sans-serif";mso-fareast-language:RU'>Error while executing action = AttachStorageDomainToPool: Error in storage domain = action<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 = <o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>engine.log:<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,557 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected :=C2=A0 ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = Storage<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,583 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 3dbf104d<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,583 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 4ac74ee6<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,685 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 4ac74ee6<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,686 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected :=C2=A0 ID: aaa00000-0000-0000-0000-123456789aaa = Type: System<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:10,689 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 53bd874a<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,690 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 3dbf104d<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,691 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected :=C2=A0 ID: aaa00000-0000-0000-0000-123456789aaa = Type: System<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:10,693 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 3129fd68<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,714 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 53bd874a<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,716 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 3129fd68<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,718 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 2b559a22<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Failed in AttachStorageDomainVDS = method<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,738 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,752 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) START, SpmStopVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 6b2f2dd2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,764 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:11,843 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: = 6b2f2dd2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,843 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) Irs placed on server null failed. Proceed = Failover<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,850 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-49) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected :=C2=A0 ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = StoragePool<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,881 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, = storage pool Default<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,886 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, = prevId -1, LVER 8<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,888 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) START, SpmStartVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D8, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 382e702<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,905 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) spmStart polling started: taskId =3D = f65c40e4-956f-4c01-ab3f-5c43f2853806<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Vdsm.log:<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38311::INFO::2012-04-08 = 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getStoragePoolInfo, Return response: {'status': {'message': = 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': = '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': = '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', = 'master_ver': 1, 'lver': 10}, 'dominfo': = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', = 'diskfree': '490079256576', 'alerts': [], 'disktotal': = '516524343296'}}}<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,953::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd = None)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: = <err> =3D ''; <rc> =3D 0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd = None)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,022::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,038::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::INFO::2012-04-08 = 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain'<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Granted request<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected = error<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Traceback (most recent call = last):<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/task.py", line 876, in = _run<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 return fn(*args, = **kargs)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/spm.py", line 115, in = run<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 return self.func(*args, = **kwargs)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 = vars.task.getExclusiveLock(STORAGE, sdUUID)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/task.py", line 1311, in = getExclusiveLock<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 = self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 raise ValueError("Owner = %s: acquire: resource %s is already acquired" % (str(self), = fullName))<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: = 2b306544-6fcc-443b-9ea6-23ae4b71d422 = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping = task<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force = False)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting = True<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::INFO::2012-04-08 = 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting = True<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> = state aborting<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy = none<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> = state failed<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last):<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/dispatcher.py", line 96, in = run<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 result =3D = ctask.prepare(self.func, *args, **kwargs)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/task.py", line 1178, in = prepare<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 raise = self.error<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::INFO::2012-04-08 = 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D10 spmId=3D2<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 10}}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::INFO::2012-04-08 = 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': = 10}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::INFO::2012-04-08 = 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: ()<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) = Entry.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': = {}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::INFO::2012-04-08 = 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) = Entry.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: = {}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at = 'public_spmStop'<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Granted request<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at = '_stop'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted = request<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span><= /p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it.<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) = Entry.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) = Return<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D10', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', = '_SHA_CKSUM=3D1d1cd722cad65ebb2edfa75641c30a5b97302f84']<o:p></o:p></span= style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain = 4f1199ec-9cdb-4efe-91eb-587faeec2099<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock = released successfully<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: = None<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,168::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::INFO::2012-04-08 = 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: ()<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', = 'lastCheck': 1333900465.46615, 'valid': True, 'code': = 0}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,397::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><div><div = style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm = 0cm 0cm'><p class=3DMsoNormal><b><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext;mso-fareast-language:RU'>From:</span></b><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext;mso-fareast-language:RU'> Rami Vaknin [mailto:rvaknin@redhat.com] = <br><b>Sent:</b> Sunday, April 08, 2012 7:02 PM<br><b>To:</b> = </span><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext;mso-fareast-language:RU'>=D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87</span><= span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext;mso-fareast-language:RU'><br><b>Subject:</b> Re: [Users] Failed to = attach Storage Domain export to Data = Center<o:p></o:p></span></p></div></div><p class=3DMsoNormal><span = lang=3DEN-US><o:p> </o:p></span></p><p class=3DMsoNormal>On = 04/08/2012 03:48 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 = wrote:<br><br>1. Change the spuuid in the domain's metadata file to be = the spuuid of the new datacenter (the metadata can be found in dom_md/ = dir somewhere inside the stoarge domain's directories tree), you can = take the new spuuid from any host running on the new data center by = running:<br>vdsClient -s 0 getConnectedStoragePoolsList<br><br>2. Try to = attach the storage domain again, this time it will fail due to broken = checksum on the metadata file, go to vdsm.log of the spm and look for = "checksum", it will tell you what the right checksum should be = like, copy it to the metadata file.<br>This time the attachment should = work.<br><br><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Hi. = </span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Thank you = for the help you provided me in the mailing = list.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>I = dropped the settings using the ovirt - = engine-cleanup</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>Ovirt reinstalled and tried to restore the virtual machine = from the export of storage domain.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>If you try an attachment domain = ovirt error:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'font-size:12.0pt'>Error while executing action = AttachStorageDomainToPool: The Domain is already used by another Data = Center.<br/>In order to use it, detach it from the Data Center it = is currently attached to.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>In the = logs:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,898 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: = Storage</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 1985db9c</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 17f02ea3</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,004 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 17f02ea3</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,005 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,008 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 6f4db784</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,009 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 1985db9c</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,010 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,013 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 42baf149</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,031 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 6f4db784</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,035 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 42baf149</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,037 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: = 6f22bcac</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IrsOperationFailedNoFailoverException: IRSGenericException: = IRSErrorException: Storage domain already attached to pool: = 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>2012-04-08 08:39:53,186 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: = 6f22bcac</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command = org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw = Vdc Bll exception. With error message VdcBLLException: = org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverExc= eption: IRSGenericException: IRSErrorException: Storage domain already = attached to pool: 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>2012-04-08 08:39:53,190 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command [id=3Db717db6a-a582-49ba-92c1-6bb09340789a]: = Compensating NEW_ENTITY_ID of = org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; = snapshot: storagePoolId =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, = storageId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>I did not deactivate the export = domain before clearing the configuration. How can I be in this case? I = can manually remove the flag of the domain?</span><o:p></o:p></p><p = class=3DMsoNormal><span style=3D'font-size:12.0pt;font-family:"Times New = Roman","serif";mso-fareast-language:RU'><br><br><br><o:p></o:p></span></p= href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><o:p></o:p></pre><pre>= <a = href=3D"http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt= .org/mailman/listinfo/users</a><o:p></o:p></pre><p = class=3DMsoNormal><span style=3D'font-size:12.0pt;font-family:"Times New = Roman","serif";mso-fareast-language:RU'><br><br><br><o:p></o:p></span></p=
<pre>-- = <o:p></o:p></pre><pre><o:p> </o:p></pre><pre>Thanks,<o:p></o:p></pre= <pre><o:p> </o:p></pre><pre>Rami Vaknin, QE @ Red Hat, TLV, = IL.<o:p></o:p></pre></div></body></html> ------=_NextPart_000_0063_01CD15C2.E4EF9E40--

This is a multi-part message in MIME format. --------------000509020904070208080405 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 04/08/2012 07:05 PM, зоррыч wrote:
I found the metadata in the file ./dom_md/metadata
Changed “b23c7ab6-b1d4-4888-8d4a-adc78e61db38” to “e3e565d6-8104-11e1-afd2-001cc43ceea0” (SDUUID)
It's wrong, you should change that in POOL_UUID and not in SDUUID (my fault, by spuuid I meant to the POOL_UUID).
The data I gave the command:
[root@noc-3-synt ~]# vdsClient -s 0 getConnectedStoragePoolsList
e3e565d6-8104-11e1-afd2-001cc43ceea0
Found in checksumvdsm.log:
Thread-36836::WARNING::2012-04-08 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) data seal is broken metadata declares `4df87518878fd0d74932917b83aba9672f92dff2` should be `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines={'VERSION': '0', 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', 'CLASS': 'Backup'})
MetaDataSealIsBroken: Meta Data seal is broken (checksum mismatch): 'cksum = 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59'
Changed it.
As a result, the metadata file:
[root@noc-2-synt dom_md]# cat metadata
CLASS=Backup
DESCRIPTION=export
IOOPTIMEOUTSEC=1
LEASERETRIES=3
LEASETIMESEC=5
LOCKPOLICY=
LOCKRENEWALINTERVALSEC=5
MASTER_VERSION=0
POOL_UUID=801e35d4-7405-11e1-b152-001cc43ceea0
REMOTE_PATH=10.1.20.2:/home/nfs4
ROLE=Regular
SDUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0
TYPE=NFS
VERSION=0
_SHA_CKSUM=d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59
attachment to the ovirt fails:
Thread-35311::INFO::2012-04-08 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or directory: '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-35311::ERROR::2012-04-08 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected error
I renamed the folder that contains the data to e3e565d6-8104-11e1-afd2-001cc43ceea0 for b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID)
attachment to the ovirt fails =(
messages:
Error while executing action AttachStorageDomainToPool: Error in storage domain action
engine.log:
2012-04-08 11:53:10,557 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage
2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3dbf104d
2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 4ac74ee6
2012-04-08 11:53:10,685 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 4ac74ee6
2012-04-08 11:53:10,686 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 11:53:10,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 53bd874a
2012-04-08 11:53:10,690 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3dbf104d
2012-04-08 11:53:10,691 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 11:53:10,693 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3129fd68
2012-04-08 11:53:10,714 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 53bd874a
2012-04-08 11:53:10,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3129fd68
2012-04-08 11:53:10,718 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 2b559a22
2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Failed in AttachStorageDomainVDS method
2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Error code StorageDomainActionError and error message IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)
2012-04-08 11:53:10,738 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)
2012-04-08 11:53:10,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) START, SpmStopVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 6b2f2dd2
2012-04-08 11:53:10,764 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, pool id e3e565d6-8104-11e1-afd2-001cc43ceea0
2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: 6b2f2dd2
2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) Irs placed on server null failed. Proceed Failover
2012-04-08 11:53:11,850 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-49) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool
2012-04-08 11:53:11,881 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, storage pool Default
2012-04-08 11:53:11,886 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, prevId -1, LVER 8
2012-04-08 11:53:11,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) START, SpmStartVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=-1, prevLVER=8, storagePoolFormatType=V1, recoveryMode=Manual, SCSIFencing=false), log id: 382e702
2012-04-08 11:53:11,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) spmStart polling started: taskId = f65c40e4-956f-4c01-ab3f-5c43f2853806
Vdsm.log:
Thread-38311::INFO::2012-04-08 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 10}, 'dominfo': {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', 'diskfree': '490079256576', 'alerts': [], 'disktotal': '516524343296'}}}
Thread-38312::DEBUG::2012-04-08 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38312::INFO::2012-04-08 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])
Thread-38312::DEBUG::2012-04-08 11:54:32,953::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state preparing
Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server
Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]
Thread-38312::DEBUG::2012-04-08 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd None)
Thread-38312::DEBUG::2012-04-08 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
Thread-38312::DEBUG::2012-04-08 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd None)
Thread-38312::DEBUG::2012-04-08 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
Thread-38312::DEBUG::2012-04-08 11:54:33,022::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38312::DEBUG::2012-04-08 11:54:33,023::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> state finished
Thread-38312::DEBUG::2012-04-08 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38312::DEBUG::2012-04-08 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38312::DEBUG::2012-04-08 11:54:33,024::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting False
Thread-38312::INFO::2012-04-08 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38313::DEBUG::2012-04-08 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38313::INFO::2012-04-08 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])
Thread-38313::DEBUG::2012-04-08 11:54:33,038::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state preparing
Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server
Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]
Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> state finished
Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38313::DEBUG::2012-04-08 11:54:33,051::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting False
Thread-38313::INFO::2012-04-08 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38314::DEBUG::2012-04-08 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38314::INFO::2012-04-08 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-38314::DEBUG::2012-04-08 11:54:33,063::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state preparing
Thread-38314::DEBUG::2012-04-08 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1072' at 'public_attachStorageDomain'
Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Granted request
Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)
Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting False
Thread-38314::ERROR::2012-04-08 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 876, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/storage/spm.py", line 115, in run
return self.func(*args, **kwargs)
File "/usr/share/vdsm/storage/spm.py", line 1073, in public_attachStorageDomain
vars.task.getExclusiveLock(STORAGE, sdUUID)
File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock
self.resOwner.acquire(namespace, resName, resourceManager.LockType.exclusive, timeout)
File "/usr/share/vdsm/storage/resourceManager.py", line 673, in acquire
raise ValueError("Owner %s: acquire: resource %s is already acquired" % (str(self), fullName))
ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: 2b306544-6fcc-443b-9ea6-23ae4b71d422 ('e3e565d6-8104-11e1-afd2-001cc43ceea0', 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task
Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force False)
Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting True
Thread-38314::INFO::2012-04-08 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code 100
Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting True
Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force False
Thread-38314::DEBUG::2012-04-08 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> state aborting
Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy none
Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> state failed
Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}
Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-38314::DEBUG::2012-04-08 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
result = ctask.prepare(self.func, *args, **kwargs)
File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
raise self.error
ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38315::DEBUG::2012-04-08 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38315::INFO::2012-04-08 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-38315::DEBUG::2012-04-08 11:54:33,086::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state preparing
Thread-38315::DEBUG::2012-04-08 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=SPM spmLver=10 spmId=2
Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}
Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> state finished
Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38315::DEBUG::2012-04-08 11:54:33,088::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting False
Thread-38315::INFO::2012-04-08 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}
Thread-38316::DEBUG::2012-04-08 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38316::INFO::2012-04-08 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
Thread-38316::DEBUG::2012-04-08 11:54:33,102::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state preparing
Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': {}}
Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> state finished
Thread-38316::DEBUG::2012-04-08 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38316::DEBUG::2012-04-08 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38316::DEBUG::2012-04-08 11:54:33,104::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting False
Thread-38316::INFO::2012-04-08 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
Thread-38317::DEBUG::2012-04-08 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38317::INFO::2012-04-08 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-38317::DEBUG::2012-04-08 11:54:33,114::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state preparing
Thread-38317::DEBUG::2012-04-08 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0
Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) Entry.
Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: {}
Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'
Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Granted request
Thread-38317::DEBUG::2012-04-08 11:54:33,116::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)
Thread-38317::DEBUG::2012-04-08 11:54:33,117::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting False
Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted request
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) Entry.
Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) Return
Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/master` is not mounted, skipping
Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: e3e565d6-8104-11e1-afd2-001cc43ceea0
Thread-38317::DEBUG::2012-04-08 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=main', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=10', 'POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH=nfs.corp:/share', 'ROLE=Master', 'SDUUID=4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=1d1cd722cad65ebb2edfa75641c30a5b97302f84']
Thread-38317::DEBUG::2012-04-08 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
Thread-38317::INFO::2012-04-08 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099
Thread-38317::DEBUG::2012-04-08 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd /usr/libexec/vdsm/)
Thread-38317::DEBUG::2012-04-08 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
Thread-38317::DEBUG::2012-04-08 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock released successfully
Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: None
Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> state finished
Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}
Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-38317::DEBUG::2012-04-08 11:54:34,168::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting False
Thread-38317::INFO::2012-04-08 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}
Thread-38318::INFO::2012-04-08 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state preparing
Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', 'lastCheck': 1333900465.46615, 'valid': True, 'code': 0}}
Thread-38318::DEBUG::2012-04-08 11:54:34,396::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> state finished
Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38318::DEBUG::2012-04-08 11:54:34,397::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting False
*From:*Rami Vaknin [mailto:rvaknin@redhat.com] *Sent:* Sunday, April 08, 2012 7:02 PM *To:* зоррыч *Subject:* Re: [Users] Failed to attach Storage Domain export to Data Center
On 04/08/2012 03:48 PM, зоррыч wrote:
1. Change the spuuid in the domain's metadata file to be the spuuid of the new datacenter (the metadata can be found in dom_md/ dir somewhere inside the stoarge domain's directories tree), you can take the new spuuid from any host running on the new data center by running: vdsClient -s 0 getConnectedStoragePoolsList
2. Try to attach the storage domain again, this time it will fail due to broken checksum on the metadata file, go to vdsm.log of the spm and look for "checksum", it will tell you what the right checksum should be like, copy it to the metadata file. This time the attachment should work.
Hi.
Thank you for the help you provided me in the mailing list.
I dropped the settings using the ovirt - engine-cleanup
Ovirt reinstalled and tried to restore the virtual machine from the export of storage domain.
If you try an attachment domain ovirt error:
Error while executing action AttachStorageDomainToPool: The Domain is already used by another Data Center.<br/>In order to use it, detach it from the Data Center it is currently attached to.
In the logs:
2012-04-08 08:39:49,898 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: Storage
2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 1985db9c
2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 17f02ea3
2012-04-08 08:39:50,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 17f02ea3
2012-04-08 08:39:50,005 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 08:39:50,008 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 6f4db784
2012-04-08 08:39:50,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 1985db9c
2012-04-08 08:39:50,010 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 08:39:50,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 42baf149
2012-04-08 08:39:50,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 6f4db784
2012-04-08 08:39:50,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 42baf149
2012-04-08 08:39:50,037 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: 6f22bcac
2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'
2012-04-08 08:39:53,186 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: 6f22bcac
2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'
2012-04-08 08:39:53,190 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command [id=b717db6a-a582-49ba-92c1-6bb09340789a]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, storageId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38.
I did not deactivate the export domain before clearing the configuration. How can I be in this case? I can manually remove the flag of the domain?
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
--
Thanks,
Rami Vaknin, QE @ Red Hat, TLV, IL.
-- Thanks, Rami Vaknin, QE @ Red Hat, TLV, IL. --------------000509020904070208080405 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> On 04/08/2012 07:05 PM, зоррыч wrote: <blockquote cite="mid:006201cd15a1$5ddcc5c0$19965140$@megatrone.ru" type="cite"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta name="Generator" content="Microsoft Word 14 (filtered medium)"> <style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:"Arial Unicode MS"; panose-1:2 11 6 4 2 2 2 2 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} @font-face {font-family:Consolas; panose-1:2 11 6 9 2 2 4 3 2 4;} @font-face {font-family:"\@Arial Unicode MS"; panose-1:2 11 6 4 2 2 2 2 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"; color:black; mso-fareast-language:EN-US;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} pre {mso-style-priority:99; mso-style-link:"Стандартный HTML Знак"; margin:0cm; margin-bottom:.0001pt; font-size:10.0pt; font-family:"Courier New"; color:black;} span.EmailStyle17 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:windowtext;} span.HTML {mso-style-name:"Стандартный HTML Знак"; mso-style-priority:99; mso-style-link:"Стандартный HTML"; font-family:Consolas; color:black; mso-fareast-language:EN-US;} span.EmailStyle20 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext="edit" spidmax="1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext="edit"> <o:idmap v:ext="edit" data="1" /> </o:shapelayout></xml><![endif]--> <div class="WordSection1"> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">I found the metadata in the file ./dom_md/metadata<o:p></o:p></span></p> <p class="MsoNormal"><span style="color: rgb(31, 73, 125);" lang="EN-US">Changed “b23c7ab6-b1d4-4888-8d4a-adc78e61db38” to “e3e565d6-8104-11e1-afd2-001cc43ceea0” (SDUUID)</span></p> </div> </blockquote> <br> It's wrong, you should change that in POOL_UUID and not in SDUUID (my fault, by spuuid I meant to the POOL_UUID)<span style="color:#1F497D" lang="EN-US"></span>.<br> <br> <blockquote cite="mid:006201cd15a1$5ddcc5c0$19965140$@megatrone.ru" type="cite"> <div class="WordSection1"> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">The data I gave the command: <o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">[root@noc-3-synt ~]# vdsClient -s 0 getConnectedStoragePoolsList<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Found in </span><span lang="EN-US">checksum</span><span style="color:#1F497D" lang="EN-US"> vdsm.log:<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-36836::WARNING::2012-04-08 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) data seal is broken metadata declares `4df87518878fd0d74932917b83aba9672f92dff2` should be `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines={'VERSION': '0', 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', 'CLASS': 'Backup'})<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">MetaDataSealIsBroken: Meta Data seal is broken (checksum mismatch): 'cksum = 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Changed it.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">As a result, the metadata file:<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">[root@noc-2-synt dom_md]# cat metadata<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">CLASS=Backup<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">DESCRIPTION=export<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">IOOPTIMEOUTSEC=1<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LEASERETRIES=3<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LEASETIMESEC=5<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LOCKPOLICY=<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LOCKRENEWALINTERVALSEC=5<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">MASTER_VERSION=0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">POOL_UUID=801e35d4-7405-11e1-b152-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">REMOTE_PATH=10.1.20.2:/home/nfs4<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ROLE=Regular<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">SDUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">TYPE=NFS<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">VERSION=0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">_SHA_CKSUM=d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">attachment to the ovirt fails:<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-35311::INFO::2012-04-08 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or directory: '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-35311::ERROR::2012-04-08 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected error<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">I renamed the folder that contains the data to e3e565d6-8104-11e1-afd2-001cc43ceea0 for b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">attachment to the ovirt fails =(<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">messages:<o:p></o:p></span></p> <p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Arial Unicode MS","sans-serif";mso-fareast-language:RU" lang="EN-US">Error while executing action AttachStorageDomainToPool: Error in storage domain action<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> <o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">engine.log:<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,557 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3dbf104d<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 4ac74ee6<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,685 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 4ac74ee6<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,686 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 53bd874a<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,690 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3dbf104d<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,691 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,693 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3129fd68<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,714 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 53bd874a<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3129fd68<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,718 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 2b559a22<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Failed in AttachStorageDomainVDS method<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Error code StorageDomainActionError and error message IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,738 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) START, SpmStopVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 6b2f2dd2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,764 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, pool id e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: 6b2f2dd2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) Irs placed on server null failed. Proceed Failover<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,850 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-49) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,881 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, storage pool Default<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,886 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, prevId -1, LVER 8<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) START, SpmStartVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=-1, prevLVER=8, storagePoolFormatType=V1, recoveryMode=Manual, SCSIFencing=false), log id: 382e702<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) spmStart polling started: taskId = f65c40e4-956f-4c01-ab3f-5c43f2853806<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Vdsm.log:<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38311::INFO::2012-04-08 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 10}, 'dominfo': {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', 'diskfree': '490079256576', 'alerts': [], 'disktotal': '516524343296'}}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,953::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd None)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd None)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,022::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,023::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,024::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,038::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,051::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::INFO::2012-04-08 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,063::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1072' at 'public_attachStorageDomain'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Granted request<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::ERROR::2012-04-08 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected error<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Traceback (most recent call last):<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 876, in _run<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> return fn(*args, **kargs)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/spm.py", line 115, in run<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> return self.func(*args, **kwargs)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/spm.py", line 1073, in public_attachStorageDomain<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> vars.task.getExclusiveLock(STORAGE, sdUUID)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> self.resOwner.acquire(namespace, resName, resourceManager.LockType.exclusive, timeout)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/resourceManager.py", line 673, in acquire<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> raise ValueError("Owner %s: acquire: resource %s is already acquired" % (str(self), fullName))<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: 2b306544-6fcc-443b-9ea6-23ae4b71d422 ('e3e565d6-8104-11e1-afd2-001cc43ceea0', 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force False)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting True<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::INFO::2012-04-08 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code 100<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting True<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> state aborting<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy none<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> state failed<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> result = ctask.prepare(self.func, *args, **kwargs)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 1178, in prepare<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> raise self.error<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::INFO::2012-04-08 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,086::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=SPM spmLver=10 spmId=2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,088::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::INFO::2012-04-08 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::INFO::2012-04-08 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,102::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': {}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,104::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::INFO::2012-04-08 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::INFO::2012-04-08 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) Entry.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Granted request<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,116::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,117::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted request<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) Entry.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) Return<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/master` is not mounted, skipping<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=main', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=10', 'POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH=nfs.corp:/share', 'ROLE=Master', 'SDUUID=4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=1d1cd722cad65ebb2edfa75641c30a5b97302f84']<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::INFO::2012-04-08 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd /usr/libexec/vdsm/)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock released successfully<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: None<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,168::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::INFO::2012-04-08 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::INFO::2012-04-08 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', 'lastCheck': 1333900465.46615, 'valid': True, 'code': 0}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,396::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,397::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:RU" lang="EN-US">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:RU" lang="EN-US"> Rami Vaknin [<a class="moz-txt-link-freetext" href="mailto:rvaknin@redhat.com">mailto:rvaknin@redhat.com</a>] <br> <b>Sent:</b> Sunday, April 08, 2012 7:02 PM<br> <b>To:</b> </span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:RU">зоррыч</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:RU" lang="EN-US"><br> <b>Subject:</b> Re: [Users] Failed to attach Storage Domain export to Data Center<o:p></o:p></span></p> </div> </div> <p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal">On 04/08/2012 03:48 PM, зоррыч wrote:<br> <br> 1. Change the spuuid in the domain's metadata file to be the spuuid of the new datacenter (the metadata can be found in dom_md/ dir somewhere inside the stoarge domain's directories tree), you can take the new spuuid from any host running on the new data center by running:<br> vdsClient -s 0 getConnectedStoragePoolsList<br> <br> 2. Try to attach the storage domain again, this time it will fail due to broken checksum on the metadata file, go to vdsm.log of the spm and look for "checksum", it will tell you what the right checksum should be like, copy it to the metadata file.<br> This time the attachment should work.<br> <br> <o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">Hi. </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">Thank you for the help you provided me in the mailing list.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">I dropped the settings using the ovirt - engine-cleanup</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">Ovirt reinstalled and tried to restore the virtual machine from the export of storage domain.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">If you try an attachment domain ovirt error:</span><o:p></o:p></p> <p class="MsoNormal"><span style="font-size:12.0pt" lang="EN-US">Error while executing action AttachStorageDomainToPool: The Domain is already used by another Data Center.<br/>In order to use it, detach it from the Data Center it is currently attached to.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">In the logs:</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:49,898 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: Storage</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 1985db9c</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 17f02ea3</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 17f02ea3</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,005 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,008 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 6f4db784</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 1985db9c</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,010 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 42baf149</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 6f4db784</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 42baf149</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,037 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: 6f22bcac</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,186 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: 6f22bcac</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,190 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command [id=b717db6a-a582-49ba-92c1-6bb09340789a]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, storageId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">I did not deactivate the export domain before clearing the configuration. How can I be in this case? I can manually remove the flag of the domain?</span><o:p></o:p></p> <p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:RU"><br> <br> <br> <o:p></o:p></span></p> <pre>_______________________________________________<o:p></o:p></pre> <pre>Users mailing list<o:p></o:p></pre> <pre><a moz-do-not-send="true" href="mailto:Users@ovirt.org">Users@ovirt.org</a><o:p></o:p></pre> <pre><a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><o:p></o:p></pre> <p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:RU"><br> <br> <br> <o:p></o:p></span></p> <pre>-- <o:p></o:p></pre> <pre><o:p> </o:p></pre> <pre>Thanks,<o:p></o:p></pre> <pre><o:p> </o:p></pre> <pre>Rami Vaknin, QE @ Red Hat, TLV, IL.<o:p></o:p></pre> </div> </blockquote> <br> <br> <pre class="moz-signature" cols="72">-- Thanks, Rami Vaknin, QE @ Red Hat, TLV, IL. </pre> </body> </html> --------------000509020904070208080405--

<p class=3DMsoNormal><span lang=3DEN-US =
</o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US =
</p><p class=3DMsoNormal><span lang=3DEN-US =
<pre> <o:p></o:p></pre><pre>Rami Vaknin, QE @ Red Hat, TLV, = IL.<o:p></o:p></pre><p class=3DMsoNormal><span =
This is a multi-part message in MIME format. ------=_NextPart_000_011F_01CD165B.CD354DB0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable In an attempt to connect ovirt storage domain fail [root@noc-2-synt nfs4]# cat = b23c7ab6-b1d4-4888-8d4a-adc78e61db38/dom_md/metadata CLASS=3DBackup DESCRIPTION=3Dexport IOOPTIMEOUTSEC=3D1 LEASERETRIES=3D3 LEASETIMESEC=3D5 LOCKPOLICY=3D LOCKRENEWALINTERVALSEC=3D5 MASTER_VERSION=3D0 POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 REMOTE_PATH=3D10.1.20.2:/home/nfs4 ROLE=3DRegular SDUUID=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38 TYPE=3DNFS VERSION=3D0 _SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd2b59 =20 =20 logs: =20 2012-04-09 06:11:38,628 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage 2012-04-09 06:11:38,658 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 651832d2 2012-04-09 06:11:38,658 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 69324bf2 2012-04-09 06:11:38,747 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 69324bf2 2012-04-09 06:11:38,748 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-09 06:11:38,750 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 1b5bf8d2 2012-04-09 06:11:38,762 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 651832d2 2012-04-09 06:11:38,763 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-49) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-09 06:11:38,765 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 59f6a186 2012-04-09 06:11:38,775 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 1b5bf8d2 2012-04-09 06:11:38,791 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 59f6a186 2012-04-09 06:11:38,792 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 58a14aa 2012-04-09 06:11:38,816 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Failed in AttachStorageDomainVDS method 2012-04-09 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-09 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-09 06:11:38,836 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) START, SpmStopVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: fbb0af9 2012-04-09 06:11:38,848 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) SpmStopVDSCommand::Stopping SPM on vds 10.2.20.8, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0 2012-04-09 06:11:39,915 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) FINISH, SpmStopVDSCommand, log id: fbb0af9 2012-04-09 06:11:39,915 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) Irs placed on server null failed. Proceed Failover 2012-04-09 06:11:39,923 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-50) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool 2012-04-09 06:11:39,948 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) hostFromVds::selectedVds - 10.1.20.7, spmStatus Free, = storage pool Default 2012-04-09 06:11:39,953 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) starting spm on vds 10.1.20.7, storage pool Default, = prevId -1, LVER 13 2012-04-09 06:11:39,955 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) START, SpmStartVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D13, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 3eff326c 2012-04-09 06:11:39,971 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) spmStart polling started: taskId =3D = e6db13d3-1c55-4d11-9fbc-8abf50c635ae ^C =20 From: Rami Vaknin [mailto:rvaknin@redhat.com]=20 Sent: Monday, April 09, 2012 1:27 PM To: =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 Cc: users@ovirt.org Subject: Re: [Users] Failed to attach Storage Domain export to Data = Center =20 On 04/08/2012 07:05 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote:=20 I found the metadata in the file ./dom_md/metadata Changed =E2=80=9Cb23c7ab6-b1d4-4888-8d4a-adc78e61db38=E2=80=9D to = =E2=80=9Ce3e565d6-8104-11e1-afd2-001cc43ceea0=E2=80=9D (SDUUID) It's wrong, you should change that in POOL_UUID and not in SDUUID (my = fault, by spuuid I meant to the POOL_UUID). The data I gave the command:=20 [root@noc-3-synt ~]# vdsClient -s 0 getConnectedStoragePoolsList e3e565d6-8104-11e1-afd2-001cc43ceea0 =20 Found in checksum vdsm.log: Thread-36836::WARNING::2012-04-08 = 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) = data seal is broken metadata declares = `4df87518878fd0d74932917b83aba9672f92dff2` should be = `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines=3D{'VERSION': '0', = 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', = 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', = 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', = 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', = 'CLASS': 'Backup'}) MetaDataSealIsBroken: Meta Data seal is broken (checksum mismatch): = 'cksum =3D 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum =3D = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59' =20 Changed it. As a result, the metadata file: =20 [root@noc-2-synt dom_md]# cat metadata CLASS=3DBackup DESCRIPTION=3Dexport IOOPTIMEOUTSEC=3D1 LEASERETRIES=3D3 LEASETIMESEC=3D5 LOCKPOLICY=3D LOCKRENEWALINTERVALSEC=3D5 MASTER_VERSION=3D0 POOL_UUID=3D801e35d4-7405-11e1-b152-001cc43ceea0 REMOTE_PATH=3D10.1.20.2:/home/nfs4 ROLE=3DRegular SDUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 TYPE=3DNFS VERSION=3D0 _SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd2b59 =20 =20 attachment to the ovirt fails: Thread-35311::INFO::2012-04-08 = 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or = directory: = '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc= 43ceea0' Thread-35311::ERROR::2012-04-08 = 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected error =20 I renamed the folder that contains the data to = e3e565d6-8104-11e1-afd2-001cc43ceea0 for = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID) =20 attachment to the ovirt fails =3D( messages: Error while executing action AttachStorageDomainToPool: Error in storage = domain action =20 =20 =20 engine.log: 2012-04-08 11:53:10,557 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage 2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 3dbf104d 2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 4ac74ee6 2012-04-08 11:53:10,685 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 4ac74ee6 2012-04-08 11:53:10,686 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 11:53:10,689 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 53bd874a 2012-04-08 11:53:10,690 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 3dbf104d 2012-04-08 11:53:10,691 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 11:53:10,693 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 3129fd68 2012-04-08 11:53:10,714 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 53bd874a 2012-04-08 11:53:10,716 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 3129fd68 2012-04-08 11:53:10,718 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 2b559a22 2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Failed in AttachStorageDomainVDS method 2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-08 11:53:10,738 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-08 11:53:10,752 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) START, SpmStopVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 6b2f2dd2 2012-04-08 11:53:10,764 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0 2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: 6b2f2dd2 2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) Irs placed on server null failed. Proceed Failover 2012-04-08 11:53:11,850 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-49) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool 2012-04-08 11:53:11,881 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, = storage pool Default 2012-04-08 11:53:11,886 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, = prevId -1, LVER 8 2012-04-08 11:53:11,888 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) START, SpmStartVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D8, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 382e702 2012-04-08 11:53:11,905 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) spmStart polling started: taskId =3D = f65c40e4-956f-4c01-ab3f-5c43f2853806 =20 =20 Vdsm.log: Thread-38311::INFO::2012-04-08 = 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getStoragePoolInfo, Return response: {'status': {'message': = 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': = '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': = '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', = 'master_ver': 1, 'lver': 10}, 'dominfo': = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', = 'diskfree': '490079256576', 'alerts': [], 'disktotal': '516524343296'}}} Thread-38312::DEBUG::2012-04-08 = 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38312::INFO::2012-04-08 = 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-38312::DEBUG::2012-04-08 = 11:54:32,953::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state = preparing Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-38312::DEBUG::2012-04-08 = 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd None) Thread-38312::DEBUG::2012-04-08 = 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = =3D ''; <rc> =3D 0 Thread-38312::DEBUG::2012-04-08 = 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd None) Thread-38312::DEBUG::2012-04-08 = 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-38312::DEBUG::2012-04-08 = 11:54:33,022::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> = state finished Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting False Thread-38312::INFO::2012-04-08 = 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38313::DEBUG::2012-04-08 = 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38313::INFO::2012-04-08 = 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-38313::DEBUG::2012-04-08 = 11:54:33,038::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state = preparing Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> = state finished Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting False Thread-38313::INFO::2012-04-08 = 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38314::DEBUG::2012-04-08 = 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38314::INFO::2012-04-08 = 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state = preparing Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain' Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Granted request Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting False Thread-38314::ERROR::2012-04-08 = 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run return fn(*args, **kargs) File "/usr/share/vdsm/storage/spm.py", line 115, in run return self.func(*args, **kwargs) File "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain vars.task.getExclusiveLock(STORAGE, sdUUID) File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout) File "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire raise ValueError("Owner %s: acquire: resource %s is already = acquired" % (str(self), fullName)) ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: = 2b306544-6fcc-443b-9ea6-23ae4b71d422 = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force = False) Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting True Thread-38314::INFO::2012-04-08 = 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100 Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting True Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force False Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> = state aborting Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy none Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> = state failed Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-38314::DEBUG::2012-04-08 = 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run result =3D ctask.prepare(self.func, *args, **kwargs) File "/usr/share/vdsm/storage/task.py", line 1178, in prepare raise self.error ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired =20 Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38315::INFO::2012-04-08 = 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state = preparing Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D10 spmId=3D2 Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 10}} Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> = state finished Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting False Thread-38315::INFO::2012-04-08 = 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}} Thread-38316::DEBUG::2012-04-08 = 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38316::INFO::2012-04-08 = 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: () Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state = preparing Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) Entry. Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': {}} Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> = state finished Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting False Thread-38316::INFO::2012-04-08 = 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}} Thread-38317::DEBUG::2012-04-08 = 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38317::INFO::2012-04-08 = 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state = preparing Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) Entry. Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: {} Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop' Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Granted request Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting False Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at '_stop' Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive' Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user) Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted request Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users) Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it. Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records. Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) Entry. Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) Return Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D10', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', '_SHA_CKSUM=3D1d1cd722cad65ebb2edfa75641c30a5b97302f84'] Thread-38317::DEBUG::2012-04-08 = 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction Thread-38317::INFO::2012-04-08 = 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099 Thread-38317::DEBUG::2012-04-08 = 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/) Thread-38317::DEBUG::2012-04-08 = 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock = released successfully Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: None Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> = state finished Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-38317::DEBUG::2012-04-08 = 11:54:34,168::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting False Thread-38317::INFO::2012-04-08 = 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}} Thread-38318::INFO::2012-04-08 = 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: () Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state = preparing Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', = 'lastCheck': 1333900465.46615, 'valid': True, 'code': 0}} Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> = state finished Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38318::DEBUG::2012-04-08 = 11:54:34,397::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting False =20 =20 =20 From: Rami Vaknin [mailto:rvaknin@redhat.com]=20 Sent: Sunday, April 08, 2012 7:02 PM To: =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 Subject: Re: [Users] Failed to attach Storage Domain export to Data = Center =20 On 04/08/2012 03:48 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote: 1. Change the spuuid in the domain's metadata file to be the spuuid of = the new datacenter (the metadata can be found in dom_md/ dir somewhere = inside the stoarge domain's directories tree), you can take the new = spuuid from any host running on the new data center by running: vdsClient -s 0 getConnectedStoragePoolsList 2. Try to attach the storage domain again, this time it will fail due to = broken checksum on the metadata file, go to vdsm.log of the spm and look = for "checksum", it will tell you what the right checksum should be like, = copy it to the metadata file. This time the attachment should work. Hi.=20 Thank you for the help you provided me in the mailing list. I dropped the settings using the ovirt - engine-cleanup Ovirt reinstalled and tried to restore the virtual machine from the = export of storage domain. If you try an attachment domain ovirt error: Error while executing action AttachStorageDomainToPool: The Domain is = already used by another Data Center.<br/>In order to use it, detach it = from the Data Center it is currently attached to. =20 =20 In the logs: =20 2012-04-08 08:39:49,898 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: Storage 2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 1985db9c 2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 17f02ea3 2012-04-08 08:39:50,004 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 17f02ea3 2012-04-08 08:39:50,005 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 08:39:50,008 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 6f4db784 2012-04-08 08:39:50,009 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 1985db9c 2012-04-08 08:39:50,010 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 08:39:50,013 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 42baf149 2012-04-08 08:39:50,031 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 6f4db784 2012-04-08 08:39:50,035 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 42baf149 2012-04-08 08:39:50,037 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: 6f22bcac 2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IrsOperationFailedNoFailoverException: IRSGenericException: = IRSErrorException: Storage domain already attached to pool: = 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0' 2012-04-08 08:39:53,186 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: = 6f22bcac 2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command = org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw = Vdc Bll exception. With error message VdcBLLException: = org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverExc= eption: IRSGenericException: IRSErrorException: Storage domain already = attached to pool: 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0' 2012-04-08 08:39:53,190 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command [id=3Db717db6a-a582-49ba-92c1-6bb09340789a]: = Compensating NEW_ENTITY_ID of = org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; = snapshot: storagePoolId =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, = storageId =3D b23c7ab6-b1d4-4888-8d4a-adc78e61db38. =20 I did not deactivate the export domain before clearing the = configuration. How can I be in this case? I can manually remove the flag = of the domain? _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users --=20 =20 Thanks, =20 Rami Vaknin, QE @ Red Hat, TLV, IL. --=20 =20 Thanks, =20 Rami Vaknin, QE @ Red Hat, TLV, IL. ------=_NextPart_000_011F_01CD165B.CD354DB0 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"><head> <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; = charset=3DUTF-8"> <meta name=3DGenerator content=3D"Microsoft Word 12 (filtered = medium)"><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} @font-face {font-family:Consolas; panose-1:2 11 6 9 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"; color:black;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} pre {mso-style-priority:99; = mso-style-link:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML =D0=97=D0=BD=D0=B0=D0=BA"; margin:0cm; margin-bottom:.0001pt; font-size:10.0pt; font-family:"Courier New"; color:black;} p.MsoAcetate, li.MsoAcetate, div.MsoAcetate {mso-style-priority:99; mso-style-link:"=D0=A2=D0=B5=D0=BA=D1=81=D1=82 = =D0=B2=D1=8B=D0=BD=D0=BE=D1=81=D0=BA=D0=B8 =D0=97=D0=BD=D0=B0=D0=BA"; margin:0cm; margin-bottom:.0001pt; font-size:8.0pt; font-family:"Tahoma","sans-serif"; color:black;} span.HTML = {mso-style-name:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML =D0=97=D0=BD=D0=B0=D0=BA"; mso-style-priority:99; = mso-style-link:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML"; font-family:Consolas; color:black;} span.EmailStyle19 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:windowtext;} span.EmailStyle20 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:#1F497D;} span.EmailStyle21 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} span.a {mso-style-name:"=D0=A2=D0=B5=D0=BA=D1=81=D1=82 = =D0=B2=D1=8B=D0=BD=D0=BE=D1=81=D0=BA=D0=B8 =D0=97=D0=BD=D0=B0=D0=BA"; mso-style-priority:99; mso-style-link:"=D0=A2=D0=B5=D0=BA=D1=81=D1=82 = =D0=B2=D1=8B=D0=BD=D0=BE=D1=81=D0=BA=D0=B8"; font-family:"Tahoma","sans-serif"; color:black;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body bgcolor=3Dwhite lang=3DRU = link=3Dblue vlink=3Dpurple><div class=3DWordSection1><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>In an = attempt to connect ovirt storage domain fail<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> = [root@noc-2-synt nfs4]# cat = b23c7ab6-b1d4-4888-8d4a-adc78e61db38/dom_md/metadata<o:p></o:p></span></p= style=3D'color:#1F497D'>CLASS=3DBackup<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>DESCRIPTION=3Dexport<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>IOOPTIMEOUTSEC=3D1<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASERETRIES=3D3<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASETIMESEC=3D5<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKPOLICY=3D<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKRENEWALINTERVALSEC=3D5<o:p></o:p></span></p><= p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>MASTER_VERSION=3D0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0<= o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>REMOTE_PATH=3D10.1.20.2:/home/nfs4<o:p></o:p></sp= an></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ROLE=3DRegular<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>SDUUID=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38<o:p= style=3D'color:#1F497D'>TYPE=3DNFS<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>VERSION=3D0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>_SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd= 2b59<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span style=3D'color:#1F497D'>logs:</span><span = lang=3DEN-US style=3D'color:#1F497D'><o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:38,628 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected :=C2=A0 ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = Storage<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,658 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 651832d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,658 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 69324bf2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,747 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 69324bf2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,748 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected :=C2=A0 ID: aaa00000-0000-0000-0000-123456789aaa = Type: System<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:38,750 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 1b5bf8d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,762 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 651832d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,763 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-49) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected :=C2=A0 ID: aaa00000-0000-0000-0000-123456789aaa = Type: System<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:38,765 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 59f6a186<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,775 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 1b5bf8d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,791 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 59f6a186<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,792 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 58a14aa<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,816 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Failed in AttachStorageDomainVDS = method<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:38,836 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) START, SpmStopVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = fbb0af9<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,848 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) SpmStopVDSCommand::Stopping SPM on vds 10.2.20.8, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:39,915 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) FINISH, SpmStopVDSCommand, log id: = fbb0af9<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:39,915 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) Irs placed on server null failed. Proceed = Failover<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:39,923 INFO=C2=A0 = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-50) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected :=C2=A0 ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = StoragePool<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:39,948 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) hostFromVds::selectedVds - 10.1.20.7, spmStatus Free, = storage pool Default<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:39,953 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) starting spm on vds 10.1.20.7, storage pool Default, = prevId -1, LVER 13<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:39,955 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) START, SpmStartVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D13, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 3eff326c<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:39,971 INFO=C2=A0 = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) spmStart polling started: taskId =3D = e6db13d3-1c55-4d11-9fbc-8abf50c635ae<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>^C<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'color:#1F497D'><o:p> </o:p></span></p><div><div = style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm = 0cm 0cm'><p class=3DMsoNormal><b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'>From:</span></b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'> Rami Vaknin [mailto:rvaknin@redhat.com] <br><b>Sent:</b> Monday, = April 09, 2012 1:27 PM<br><b>To:</b> = =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87<br><b>Cc:</b> = users@ovirt.org<br><b>Subject:</b> Re: [Users] Failed to attach Storage = Domain export to Data Center<o:p></o:p></span></p></div></div><p = class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>On = 04/08/2012 07:05 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote: = <o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>I found the metadata in the file = ./dom_md/metadata</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Changed = =E2=80=9Cb23c7ab6-b1d4-4888-8d4a-adc78e61db38=E2=80=9D to = =E2=80=9Ce3e565d6-8104-11e1-afd2-001cc43ceea0=E2=80=9D = (SDUUID)</span><o:p></o:p></p><p class=3DMsoNormal><span = style=3D'font-size:12.0pt;font-family:"Times New = Roman","serif"'><br>It's wrong, you should change that in POOL_UUID and = not in SDUUID (my fault, by spuuid I meant to the = POOL_UUID).<br><br><br><o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>The data I gave the command: = </span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>[root@noc-3-synt ~]# vdsClient -s 0 = getConnectedStoragePoolsList</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p><= /o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Found in = </span><span lang=3DEN-US>checksum</span><span lang=3DEN-US = style=3D'color:#1F497D'> vdsm.log:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-36836::WARNING::2012-04-08 = 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) = data seal is broken metadata declares = `4df87518878fd0d74932917b83aba9672f92dff2` should be = `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines=3D{'VERSION': '0', = 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', = 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', = 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', = 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', = 'CLASS': 'Backup'})</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>MetaDataSealIsBroken: Meta Data = seal is broken (checksum mismatch): 'cksum =3D = 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum =3D = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Changed = it.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>As a result, the metadata = file:</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>[root@noc-2-synt dom_md]# cat = metadata</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>CLASS=3DBackup</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>DESCRIPTION=3Dexport</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>IOOPTIMEOUTSEC=3D1</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASERETRIES=3D3</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASETIMESEC=3D5</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKPOLICY=3D</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKRENEWALINTERVALSEC=3D5</span><o:p></o:p></p><= p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>MASTER_VERSION=3D0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>POOL_UUID=3D801e35d4-7405-11e1-b152-001cc43ceea0<= /span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>REMOTE_PATH=3D10.1.20.2:/home/nfs4</span><o:p></o= :p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ROLE=3DRegular</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>SDUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0</sp= an><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>TYPE=3DNFS</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>VERSION=3D0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>_SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd= 2b59</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>attachment = to the ovirt fails:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-35311::INFO::2012-04-08 = 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or = directory: = '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc= 43ceea0'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-35311::ERROR::2012-04-08 = 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected = error</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>I renamed = the folder that contains the data to = e3e565d6-8104-11e1-afd2-001cc43ceea0 for = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>attachment = to the ovirt fails =3D(</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>messages:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:12.0pt'>Error = while executing action AttachStorageDomainToPool: Error in storage = domain action</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> = </span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>engine.log:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,557 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = Storage</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 3dbf104d</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 4ac74ee6</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,685 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 4ac74ee6</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,686 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:10,689 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 53bd874a</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,690 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 3dbf104d</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,691 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:10,693 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 3129fd68</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,714 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 53bd874a</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,716 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 3129fd68</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,718 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 2b559a22</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Failed in AttachStorageDomainVDS = method</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,738 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,752 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) START, SpmStopVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 6b2f2dd2</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,764 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: = 6b2f2dd2</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) Irs placed on server null failed. Proceed = Failover</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,850 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-49) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = StoragePool</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,881 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, = storage pool Default</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,886 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, = prevId -1, LVER 8</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,888 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) START, SpmStartVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D8, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 382e702</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,905 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) spmStart polling started: taskId =3D = f65c40e4-956f-4c01-ab3f-5c43f2853806</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Vdsm.log:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38311::INFO::2012-04-08 = 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getStoragePoolInfo, Return response: {'status': {'message': = 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': = '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': = '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', = 'master_ver': 1, 'lver': 10}, 'dominfo': = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', = 'diskfree': '490079256576', 'alerts': [], 'disktotal': = '516524343296'}}}</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,953::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd = None)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: = <err> =3D ''; <rc> =3D 0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd = None)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,022::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,038::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::INFO::2012-04-08 = 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Granted request</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected = error</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Traceback (most recent call = last):</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/task.py", line 876, in = _run</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> return fn(*args, = **kargs)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/spm.py", line 115, in = run</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> return self.func(*args, = **kwargs)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> = vars.task.getExclusiveLock(STORAGE, sdUUID)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/task.py", line 1311, in = getExclusiveLock</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'> = self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> raise ValueError("Owner = %s: acquire: resource %s is already acquired" % (str(self), = fullName))</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: = 2b306544-6fcc-443b-9ea6-23ae4b71d422 = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping = task</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force = False)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting = True</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::INFO::2012-04-08 = 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting = True</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> = state aborting</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy = none</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> = state failed</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last):</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/dispatcher.py", line 96, in = run</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> result =3D = ctask.prepare(self.func, *args, **kwargs)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/task.py", line 1178, in = prepare</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> raise = self.error</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::INFO::2012-04-08 = 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D10 spmId=3D2</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 10}}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::INFO::2012-04-08 = 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': = 10}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::INFO::2012-04-08 = 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: ()</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) = Entry.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': = {}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::INFO::2012-04-08 = 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) = Entry.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: = {}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at = 'public_spmStop'</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Granted request</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at = '_stop'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted = request</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p><= /p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) = Entry.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) = Return</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D10', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', = '_SHA_CKSUM=3D1d1cd722cad65ebb2edfa75641c30a5b97302f84']</span><o:p></o:p= style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain = 4f1199ec-9cdb-4efe-91eb-587faeec2099</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock = released successfully</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: = None</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,168::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::INFO::2012-04-08 = 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: ()</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', = 'lastCheck': 1333900465.46615, 'valid': True, 'code': = 0}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,397::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><div><div = style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm = 0cm 0cm'><p class=3DMsoNormal><b><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'>From:</span></b><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'> Rami Vaknin [<a = href=3D"mailto:rvaknin@redhat.com">mailto:rvaknin@redhat.com</a>] = <br><b>Sent:</b> Sunday, April 08, 2012 7:02 PM<br><b>To:</b> = </span><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'>=D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87</span><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'><br><b>Subject:</b> Re: [Users] Failed to attach Storage Domain = export to Data Center</span><o:p></o:p></p></div></div><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal>On 04/08/2012 03:48 PM, = =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote:<br><br>1. Change the spuuid = in the domain's metadata file to be the spuuid of the new datacenter = (the metadata can be found in dom_md/ dir somewhere inside the stoarge = domain's directories tree), you can take the new spuuid from any host = running on the new data center by running:<br>vdsClient -s 0 = getConnectedStoragePoolsList<br><br>2. Try to attach the storage domain = again, this time it will fail due to broken checksum on the metadata = file, go to vdsm.log of the spm and look for "checksum", it = will tell you what the right checksum should be like, copy it to the = metadata file.<br>This time the attachment should = work.<br><br><br><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>Hi. </span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>Thank you for the help you provided me in the mailing = list.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>I = dropped the settings using the ovirt - = engine-cleanup</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>Ovirt reinstalled and tried to restore the virtual machine = from the export of storage domain.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>If you try an attachment domain = ovirt error:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'font-size:12.0pt'>Error while executing action = AttachStorageDomainToPool: The Domain is already used by another Data = Center.<br/>In order to use it, detach it from the Data Center it = is currently attached to.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>In the = logs:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,898 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: = Storage</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 1985db9c</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 17f02ea3</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,004 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 17f02ea3</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,005 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,008 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 6f4db784</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,009 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 1985db9c</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,010 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,013 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 42baf149</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,031 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 6f4db784</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,035 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 42baf149</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,037 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: = 6f22bcac</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IrsOperationFailedNoFailoverException: IRSGenericException: = IRSErrorException: Storage domain already attached to pool: = 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>2012-04-08 08:39:53,186 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: = 6f22bcac</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command = org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw = Vdc Bll exception. With error message VdcBLLException: = org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverExc= eption: IRSGenericException: IRSErrorException: Storage domain already = attached to pool: 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>2012-04-08 08:39:53,190 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command [id=3Db717db6a-a582-49ba-92c1-6bb09340789a]: = Compensating NEW_ENTITY_ID of = org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; = snapshot: storagePoolId =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, = storageId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>I did not deactivate the export = domain before clearing the configuration. How can I be in this case? I = can manually remove the flag of the domain?</span><o:p></o:p></p><p = class=3DMsoNormal><span = style=3D'font-size:12.0pt'><br><br><br><br></span><o:p></o:p></p><pre>___= ____________________________________________<o:p></o:p></pre><pre>Users = mailing list<o:p></o:p></pre><pre><a = href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><o:p></o:p></pre><pre>= <a = href=3D"http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt= .org/mailman/listinfo/users</a><o:p></o:p></pre><p = class=3DMsoNormal><span = style=3D'font-size:12.0pt'><br><br><br><br></span><o:p></o:p></p><pre>-- = <o:p></o:p></pre><pre> <o:p></o:p></pre><pre>Thanks,<o:p></o:p></pre= style=3D'font-size:12.0pt;font-family:"Times New = Roman","serif"'><br><br><br><o:p></o:p></span></p><pre>-- = <o:p></o:p></pre><pre><o:p> </o:p></pre><pre>Thanks,<o:p></o:p></pre=
<pre><o:p> </o:p></pre><pre>Rami Vaknin, QE @ Red Hat, TLV, = IL.<o:p></o:p></pre></div></body></html> ------=_NextPart_000_011F_01CD165B.CD354DB0--

</p><p class=3DMsoNormal><span lang=3DEN-US =
<p class=3DMsoNormal><span lang=3DEN-US =
</o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US =
</p><p class=3DMsoNormal><span lang=3DEN-US =
<pre> <o:p></o:p></pre><pre>Rami Vaknin, QE @ Red Hat, TLV, = IL.<o:p></o:p></pre><p class=3DMsoNormal =
This is a multi-part message in MIME format. ------=_NextPart_000_012A_01CD165E.3D454860 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Vdsm log (spm): =20 =20 Thread-79073::DEBUG::2012-04-09 = 06:28:05,189::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-79073::INFO::2012-04-09 = 06:28:05,190::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-79073::DEBUG::2012-04-09 = 06:28:05,190::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state init -> state = preparing Thread-79073::INFO::2012-04-09 = 06:28:05,190::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server Thread-79073::INFO::2012-04-09 = 06:28:05,191::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-79073::DEBUG::2012-04-09 = 06:28:05,192::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmp2rKh29' (cwd None) Thread-79073::DEBUG::2012-04-09 = 06:28:05,228::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = =3D ''; <rc> =3D 0 Thread-79073::DEBUG::2012-04-09 = 06:28:05,235::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmp2rKh29' (cwd None) Thread-79073::DEBUG::2012-04-09 = 06:28:05,270::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-79073::DEBUG::2012-04-09 = 06:28:05,271::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-79073::DEBUG::2012-04-09 = 06:28:05,272::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state preparing -> = state finished Thread-79073::DEBUG::2012-04-09 = 06:28:05,272::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-79073::DEBUG::2012-04-09 = 06:28:05,272::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79073::DEBUG::2012-04-09 = 06:28:05,273::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: ref 0 aborting False Thread-79073::INFO::2012-04-09 = 06:28:05,273::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-79074::DEBUG::2012-04-09 = 06:28:05,297::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-79074::INFO::2012-04-09 = 06:28:05,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-79074::DEBUG::2012-04-09 = 06:28:05,298::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state init -> state = preparing Thread-79074::INFO::2012-04-09 = 06:28:05,298::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server Thread-79074::INFO::2012-04-09 = 06:28:05,298::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-79074::DEBUG::2012-04-09 = 06:28:05,308::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-79074::DEBUG::2012-04-09 = 06:28:05,308::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-79074::DEBUG::2012-04-09 = 06:28:05,309::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation mutex Thread-79074::DEBUG::2012-04-09 = 06:28:05,309::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation mutex Thread-79074::DEBUG::2012-04-09 = 06:28:05,310::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-79074::DEBUG::2012-04-09 = 06:28:05,310::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-79074::DEBUG::2012-04-09 = 06:28:05,310::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state preparing -> = state finished Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: ref 0 aborting False Thread-79074::INFO::2012-04-09 = 06:28:05,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-79075::DEBUG::2012-04-09 = 06:28:05,328::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-79075::INFO::2012-04-09 = 06:28:05,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-79075::DEBUG::2012-04-09 = 06:28:05,329::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state init -> state = preparing Thread-79075::DEBUG::2012-04-09 = 06:28:05,329::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`af876ee5= -24f4-43f1-a161-d495903c82b5`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain' Thread-79075::DEBUG::2012-04-09 = 06:28:05,330::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-79075::DEBUG::2012-04-09 = 06:28:05,330::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-79075::DEBUG::2012-04-09 = 06:28:05,330::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`af876ee5= -24f4-43f1-a161-d495903c82b5`::Granted request Thread-79075::DEBUG::2012-04-09 = 06:28:05,331::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-79075::DEBUG::2012-04-09 = 06:28:05,331::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting False Thread-79075::ERROR::2012-04-09 = 06:28:05,331::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run return fn(*args, **kargs) File "/usr/share/vdsm/storage/spm.py", line 115, in run return self.func(*args, **kwargs) File "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain vars.task.getExclusiveLock(STORAGE, sdUUID) File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout) File "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire raise ValueError("Owner %s: acquire: resource %s is already = acquired" % (str(self), fullName)) ValueError: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired Thread-79075::DEBUG::2012-04-09 = 06:28:05,332::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._run: = b0ef8b95-fabe-43e0-944c-eb959cf9930a = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task Thread-79075::DEBUG::2012-04-09 = 06:28:05,332::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: stopping in state preparing (force = False) Thread-79075::DEBUG::2012-04-09 = 06:28:05,333::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting True Thread-79075::INFO::2012-04-09 = 06:28:05,333::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100 Thread-79075::DEBUG::2012-04-09 = 06:28:05,333::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: Prepare: aborted: Owner = b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-79075::DEBUG::2012-04-09 = 06:28:05,333::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 0 aborting True Thread-79075::DEBUG::2012-04-09 = 06:28:05,334::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._doAbort: force False Thread-79075::DEBUG::2012-04-09 = 06:28:05,334::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79075::DEBUG::2012-04-09 = 06:28:05,334::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state preparing -> = state aborting Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: _aborting: recover policy none Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state aborting -> = state failed Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79075::DEBUG::2012-04-09 = 06:28:05,336::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-79075::DEBUG::2012-04-09 = 06:28:05,336::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-79075::DEBUG::2012-04-09 = 06:28:05,336::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-79075::DEBUG::2012-04-09 = 06:28:05,337::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-79075::ERROR::2012-04-09 = 06:28:05,337::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-79075::ERROR::2012-04-09 = 06:28:05,337::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run result =3D ctask.prepare(self.func, *args, **kwargs) File "/usr/share/vdsm/storage/task.py", line 1178, in prepare raise self.error ValueError: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired =20 Thread-79076::DEBUG::2012-04-09 = 06:28:05,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-79076::INFO::2012-04-09 = 06:28:05,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state init -> state = preparing Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D18 spmId=3D2 Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 18}} Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state preparing -> = state finished Thread-79076::DEBUG::2012-04-09 = 06:28:05,351::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-79076::DEBUG::2012-04-09 = 06:28:05,351::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79076::DEBUG::2012-04-09 = 06:28:05,351::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: ref 0 aborting False Thread-79076::INFO::2012-04-09 = 06:28:05,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18}} Thread-79077::DEBUG::2012-04-09 = 06:28:05,362::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-79077::INFO::2012-04-09 = 06:28:05,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: () Thread-79077::DEBUG::2012-04-09 = 06:28:05,363::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state init -> state = preparing Thread-79077::DEBUG::2012-04-09 = 06:28:05,363::taskManager::92::TaskManager::(getAllTasksStatuses) Entry. Thread-79077::DEBUG::2012-04-09 = 06:28:05,363::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {} Thread-79077::DEBUG::2012-04-09 = 06:28:05,364::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: finished: {'allTasksStatus': {}} Thread-79077::DEBUG::2012-04-09 = 06:28:05,364::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state preparing -> = state finished Thread-79077::DEBUG::2012-04-09 = 06:28:05,364::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-79077::DEBUG::2012-04-09 = 06:28:05,365::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79077::DEBUG::2012-04-09 = 06:28:05,365::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: ref 0 aborting False Thread-79077::INFO::2012-04-09 = 06:28:05,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}} Thread-79078::DEBUG::2012-04-09 = 06:28:05,373::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-79078::INFO::2012-04-09 = 06:28:05,373::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-79078::DEBUG::2012-04-09 = 06:28:05,374::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state init -> state = preparing Thread-79078::DEBUG::2012-04-09 = 06:28:05,374::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-79078::DEBUG::2012-04-09 = 06:28:05,374::taskManager::105::TaskManager::(getAllTasks) Entry. Thread-79078::DEBUG::2012-04-09 = 06:28:05,375::taskManager::110::TaskManager::(getAllTasks) Return: {} Thread-79078::DEBUG::2012-04-09 = 06:28:05,375::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`0f0e6998= -87a6-40e8-8ded-9f600cb0ca2d`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop' Thread-79078::DEBUG::2012-04-09 = 06:28:05,375::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-79078::DEBUG::2012-04-09 = 06:28:05,376::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-79078::DEBUG::2012-04-09 = 06:28:05,376::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`0f0e6998= -87a6-40e8-8ded-9f600cb0ca2d`::Granted request Thread-79078::DEBUG::2012-04-09 = 06:28:05,376::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-79078::DEBUG::2012-04-09 = 06:28:05,377::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 1 aborting False Thread-79078::DEBUG::2012-04-09 = 06:28:05,377::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= eb290e92-2fbb-4e12-bde2-83292aade8e4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at '_stop' Thread-79078::DEBUG::2012-04-09 = 06:28:05,378::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive' Thread-79078::DEBUG::2012-04-09 = 06:28:05,378::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user) Thread-79078::DEBUG::2012-04-09 = 06:28:05,378::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= eb290e92-2fbb-4e12-bde2-83292aade8e4`::Granted request Thread-79078::DEBUG::2012-04-09 = 06:28:05,379::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-79078::DEBUG::2012-04-09 = 06:28:05,379::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users) Thread-79078::DEBUG::2012-04-09 = 06:28:05,379::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it. Thread-79078::DEBUG::2012-04-09 = 06:28:05,380::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records. Thread-79078::DEBUG::2012-04-09 = 06:28:05,380::taskManager::118::TaskManager::(unloadTasks) Entry. Thread-79078::DEBUG::2012-04-09 = 06:28:05,380::taskManager::122::TaskManager::(unloadTasks) Return Thread-79078::DEBUG::2012-04-09 = 06:28:05,381::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping Thread-79078::DEBUG::2012-04-09 = 06:28:05,381::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-79078::DEBUG::2012-04-09 = 06:28:05,381::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction Thread-79078::DEBUG::2012-04-09 = 06:28:05,382::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes Thread-79078::DEBUG::2012-04-09 = 06:28:05,382::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D18', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', '_SHA_CKSUM=3D5a3d228ad230c9d61a754c22abaa6b5f11b6c721'] Thread-79078::DEBUG::2012-04-09 = 06:28:05,397::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction Thread-79078::INFO::2012-04-09 = 06:28:05,398::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099 Thread-79078::DEBUG::2012-04-09 = 06:28:05,398::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/) Thread-79078::DEBUG::2012-04-09 = 06:28:06,433::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-79078::DEBUG::2012-04-09 = 06:28:06,434::safelease::87::ClusterLock::(release) Cluster lock = released successfully Thread-79078::DEBUG::2012-04-09 = 06:28:06,434::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: finished: None Thread-79078::DEBUG::2012-04-09 = 06:28:06,434::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state preparing -> = state finished Thread-79078::DEBUG::2012-04-09 = 06:28:06,435::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-79078::DEBUG::2012-04-09 = 06:28:06,435::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79078::DEBUG::2012-04-09 = 06:28:06,435::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-79078::DEBUG::2012-04-09 = 06:28:06,436::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-79078::DEBUG::2012-04-09 = 06:28:06,436::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-79078::DEBUG::2012-04-09 = 06:28:06,436::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-79078::DEBUG::2012-04-09 = 06:28:06,437::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 0 aborting False Thread-79078::INFO::2012-04-09 = 06:28:06,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}} Thread-79080::INFO::2012-04-09 = 06:28:08,375::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: () Thread-79080::DEBUG::2012-04-09 = 06:28:08,375::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state init -> state = preparing Thread-79080::DEBUG::2012-04-09 = 06:28:08,376::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00202488899231', = 'lastCheck': 1333967286.1603661, 'valid': True, 'code': 0}} Thread-79080::DEBUG::2012-04-09 = 06:28:08,376::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state preparing -> = state finished Thread-79080::DEBUG::2012-04-09 = 06:28:08,376::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-79080::DEBUG::2012-04-09 = 06:28:08,377::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-79080::DEBUG::2012-04-09 = 06:28:08,377::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: ref 0 aborting False Thread-79080::INFO::2012-04-09 = 06:28:08,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, Return response: {'status': {'message': 'OK', = 'code': 0}, '4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': = '0.00202488899231', 'lastCheck': 1333967286.1603661, 'valid': True, = 'code': 0}} ^C =20 From: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] On Behalf = Of =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 Sent: Monday, April 09, 2012 2:20 PM To: rvaknin@redhat.com Cc: users@ovirt.org Subject: Re: [Users] Failed to attach Storage Domain export to Data = Center =20 In an attempt to connect ovirt storage domain fail [root@noc-2-synt nfs4]# cat = b23c7ab6-b1d4-4888-8d4a-adc78e61db38/dom_md/metadata CLASS=3DBackup DESCRIPTION=3Dexport IOOPTIMEOUTSEC=3D1 LEASERETRIES=3D3 LEASETIMESEC=3D5 LOCKPOLICY=3D LOCKRENEWALINTERVALSEC=3D5 MASTER_VERSION=3D0 POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 REMOTE_PATH=3D10.1.20.2:/home/nfs4 ROLE=3DRegular SDUUID=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38 TYPE=3DNFS VERSION=3D0 _SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd2b59 =20 =20 logs: =20 2012-04-09 06:11:38,628 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage 2012-04-09 06:11:38,658 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 651832d2 2012-04-09 06:11:38,658 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 69324bf2 2012-04-09 06:11:38,747 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 69324bf2 2012-04-09 06:11:38,748 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-09 06:11:38,750 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 1b5bf8d2 2012-04-09 06:11:38,762 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 651832d2 2012-04-09 06:11:38,763 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-49) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-09 06:11:38,765 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 59f6a186 2012-04-09 06:11:38,775 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 1b5bf8d2 2012-04-09 06:11:38,791 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 59f6a186 2012-04-09 06:11:38,792 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 58a14aa 2012-04-09 06:11:38,816 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Failed in AttachStorageDomainVDS method 2012-04-09 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-09 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-09 06:11:38,836 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) START, SpmStopVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: fbb0af9 2012-04-09 06:11:38,848 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) SpmStopVDSCommand::Stopping SPM on vds 10.2.20.8, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0 2012-04-09 06:11:39,915 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) FINISH, SpmStopVDSCommand, log id: fbb0af9 2012-04-09 06:11:39,915 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) Irs placed on server null failed. Proceed Failover 2012-04-09 06:11:39,923 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-50) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool 2012-04-09 06:11:39,948 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) hostFromVds::selectedVds - 10.1.20.7, spmStatus Free, = storage pool Default 2012-04-09 06:11:39,953 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) starting spm on vds 10.1.20.7, storage pool Default, = prevId -1, LVER 13 2012-04-09 06:11:39,955 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) START, SpmStartVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D13, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 3eff326c 2012-04-09 06:11:39,971 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) spmStart polling started: taskId =3D = e6db13d3-1c55-4d11-9fbc-8abf50c635ae ^C =20 From: Rami Vaknin [mailto:rvaknin@redhat.com]=20 Sent: Monday, April 09, 2012 1:27 PM To: =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 Cc: users@ovirt.org Subject: Re: [Users] Failed to attach Storage Domain export to Data = Center =20 On 04/08/2012 07:05 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote:=20 I found the metadata in the file ./dom_md/metadata Changed =E2=80=9Cb23c7ab6-b1d4-4888-8d4a-adc78e61db38=E2=80=9D to = =E2=80=9Ce3e565d6-8104-11e1-afd2-001cc43ceea0=E2=80=9D (SDUUID) It's wrong, you should change that in POOL_UUID and not in SDUUID (my = fault, by spuuid I meant to the POOL_UUID). The data I gave the command:=20 [root@noc-3-synt ~]# vdsClient -s 0 getConnectedStoragePoolsList e3e565d6-8104-11e1-afd2-001cc43ceea0 =20 Found in checksum vdsm.log: Thread-36836::WARNING::2012-04-08 = 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) = data seal is broken metadata declares = `4df87518878fd0d74932917b83aba9672f92dff2` should be = `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines=3D{'VERSION': '0', = 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', = 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', = 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', = 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', = 'CLASS': 'Backup'}) MetaDataSealIsBroken: Meta Data seal is broken (checksum mismatch): = 'cksum =3D 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum =3D = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59' =20 Changed it. As a result, the metadata file: =20 [root@noc-2-synt dom_md]# cat metadata CLASS=3DBackup DESCRIPTION=3Dexport IOOPTIMEOUTSEC=3D1 LEASERETRIES=3D3 LEASETIMESEC=3D5 LOCKPOLICY=3D LOCKRENEWALINTERVALSEC=3D5 MASTER_VERSION=3D0 POOL_UUID=3D801e35d4-7405-11e1-b152-001cc43ceea0 REMOTE_PATH=3D10.1.20.2:/home/nfs4 ROLE=3DRegular SDUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 TYPE=3DNFS VERSION=3D0 _SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd2b59 =20 =20 attachment to the ovirt fails: Thread-35311::INFO::2012-04-08 = 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or = directory: = '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc= 43ceea0' Thread-35311::ERROR::2012-04-08 = 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected error =20 I renamed the folder that contains the data to = e3e565d6-8104-11e1-afd2-001cc43ceea0 for = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID) =20 attachment to the ovirt fails =3D( messages: Error while executing action AttachStorageDomainToPool: Error in storage = domain action =20 =20 =20 engine.log: 2012-04-08 11:53:10,557 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage 2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 3dbf104d 2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 4ac74ee6 2012-04-08 11:53:10,685 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 4ac74ee6 2012-04-08 11:53:10,686 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 11:53:10,689 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 53bd874a 2012-04-08 11:53:10,690 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 3dbf104d 2012-04-08 11:53:10,691 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 11:53:10,693 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 3129fd68 2012-04-08 11:53:10,714 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 53bd874a 2012-04-08 11:53:10,716 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 3129fd68 2012-04-08 11:53:10,718 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 2b559a22 2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Failed in AttachStorageDomainVDS method 2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-08 11:53:10,738 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',) 2012-04-08 11:53:10,752 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) START, SpmStopVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 6b2f2dd2 2012-04-08 11:53:10,764 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0 2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: 6b2f2dd2 2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) Irs placed on server null failed. Proceed Failover 2012-04-08 11:53:11,850 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-49) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool 2012-04-08 11:53:11,881 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, = storage pool Default 2012-04-08 11:53:11,886 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, = prevId -1, LVER 8 2012-04-08 11:53:11,888 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) START, SpmStartVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D8, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 382e702 2012-04-08 11:53:11,905 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) spmStart polling started: taskId =3D = f65c40e4-956f-4c01-ab3f-5c43f2853806 =20 =20 Vdsm.log: Thread-38311::INFO::2012-04-08 = 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getStoragePoolInfo, Return response: {'status': {'message': = 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': = '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': = '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', = 'master_ver': 1, 'lver': 10}, 'dominfo': = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', = 'diskfree': '490079256576', 'alerts': [], 'disktotal': '516524343296'}}} Thread-38312::DEBUG::2012-04-08 = 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38312::INFO::2012-04-08 = 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-38312::DEBUG::2012-04-08 = 11:54:32,953::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state = preparing Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-38312::DEBUG::2012-04-08 = 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd None) Thread-38312::DEBUG::2012-04-08 = 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = =3D ''; <rc> =3D 0 Thread-38312::DEBUG::2012-04-08 = 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd None) Thread-38312::DEBUG::2012-04-08 = 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-38312::DEBUG::2012-04-08 = 11:54:33,022::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> = state finished Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting False Thread-38312::INFO::2012-04-08 = 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38313::DEBUG::2012-04-08 = 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38313::INFO::2012-04-08 = 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]) Thread-38313::DEBUG::2012-04-08 = 11:54:33,038::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state = preparing Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}] Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation mutex Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': = [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> = state finished Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting False Thread-38313::INFO::2012-04-08 = 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]} Thread-38314::DEBUG::2012-04-08 = 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38314::INFO::2012-04-08 = 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state = preparing Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain' Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Granted request Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting False Thread-38314::ERROR::2012-04-08 = 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 876, in _run return fn(*args, **kargs) File "/usr/share/vdsm/storage/spm.py", line 115, in run return self.func(*args, **kwargs) File "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain vars.task.getExclusiveLock(STORAGE, sdUUID) File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout) File "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire raise ValueError("Owner %s: acquire: resource %s is already = acquired" % (str(self), fullName)) ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: = 2b306544-6fcc-443b-9ea6-23ae4b71d422 = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force = False) Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting True Thread-38314::INFO::2012-04-08 = 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100 Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting True Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force False Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> = state aborting Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy none Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> = state failed Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-38314::DEBUG::2012-04-08 = 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run result =3D ctask.prepare(self.func, *args, **kwargs) File "/usr/share/vdsm/storage/task.py", line 1178, in prepare raise self.error ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: = resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired =20 Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38315::INFO::2012-04-08 = 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state = preparing Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D10 spmId=3D2 Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 10}} Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> = state finished Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting False Thread-38315::INFO::2012-04-08 = 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}} Thread-38316::DEBUG::2012-04-08 = 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38316::INFO::2012-04-08 = 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: () Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state = preparing Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) Entry. Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': {}} Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> = state finished Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting False Thread-38316::INFO::2012-04-08 = 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}} Thread-38317::DEBUG::2012-04-08 = 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2] Thread-38317::INFO::2012-04-08 = 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0) Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state = preparing Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) Entry. Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: {} Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop' Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive' Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user) Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Granted request Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive) Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting False Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at '_stop' Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive' Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user) Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted request Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users) Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it. Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records. Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) Entry. Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) Return Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D10', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', '_SHA_CKSUM=3D1d1cd722cad65ebb2edfa75641c30a5b97302f84'] Thread-38317::DEBUG::2012-04-08 = 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction Thread-38317::INFO::2012-04-08 = 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099 Thread-38317::DEBUG::2012-04-08 = 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/) Thread-38317::DEBUG::2012-04-08 = 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0 Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock = released successfully Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: None Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> = state finished Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>} Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users) Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it. Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records. Thread-38317::DEBUG::2012-04-08 = 11:54:34,168::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting False Thread-38317::INFO::2012-04-08 = 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}} Thread-38318::INFO::2012-04-08 = 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: () Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state = preparing Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', = 'lastCheck': 1333900465.46615, 'valid': True, 'code': 0}} Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> = state finished Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {} Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {} Thread-38318::DEBUG::2012-04-08 = 11:54:34,397::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting False =20 =20 =20 From: Rami Vaknin [mailto:rvaknin@redhat.com]=20 Sent: Sunday, April 08, 2012 7:02 PM To: =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 Subject: Re: [Users] Failed to attach Storage Domain export to Data = Center =20 On 04/08/2012 03:48 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote: 1. Change the spuuid in the domain's metadata file to be the spuuid of = the new datacenter (the metadata can be found in dom_md/ dir somewhere = inside the stoarge domain's directories tree), you can take the new = spuuid from any host running on the new data center by running: vdsClient -s 0 getConnectedStoragePoolsList 2. Try to attach the storage domain again, this time it will fail due to = broken checksum on the metadata file, go to vdsm.log of the spm and look = for "checksum", it will tell you what the right checksum should be like, = copy it to the metadata file. This time the attachment should work. Hi.=20 Thank you for the help you provided me in the mailing list. I dropped the settings using the ovirt - engine-cleanup Ovirt reinstalled and tried to restore the virtual machine from the = export of storage domain. If you try an attachment domain ovirt error: Error while executing action AttachStorageDomainToPool: The Domain is = already used by another Data Center.<br/>In order to use it, detach it = from the Data Center it is currently attached to. =20 =20 In the logs: =20 2012-04-08 08:39:49,898 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: Storage 2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 1985db9c 2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 17f02ea3 2012-04-08 08:39:50,004 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 17f02ea3 2012-04-08 08:39:50,005 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 08:39:50,008 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 6f4db784 2012-04-08 08:39:50,009 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 1985db9c 2012-04-08 08:39:50,010 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System 2012-04-08 08:39:50,013 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: 42baf149 2012-04-08 08:39:50,031 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 6f4db784 2012-04-08 08:39:50,035 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: 42baf149 2012-04-08 08:39:50,037 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: 6f22bcac 2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IrsOperationFailedNoFailoverException: IRSGenericException: = IRSErrorException: Storage domain already attached to pool: = 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0' 2012-04-08 08:39:53,186 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: = 6f22bcac 2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command = org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw = Vdc Bll exception. With error message VdcBLLException: = org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverExc= eption: IRSGenericException: IRSErrorException: Storage domain already = attached to pool: 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0' 2012-04-08 08:39:53,190 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command [id=3Db717db6a-a582-49ba-92c1-6bb09340789a]: = Compensating NEW_ENTITY_ID of = org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; = snapshot: storagePoolId =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, = storageId =3D b23c7ab6-b1d4-4888-8d4a-adc78e61db38. =20 I did not deactivate the export domain before clearing the = configuration. How can I be in this case? I can manually remove the flag = of the domain? _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users --=20 =20 Thanks, =20 Rami Vaknin, QE @ Red Hat, TLV, IL. --=20 =20 Thanks, =20 Rami Vaknin, QE @ Red Hat, TLV, IL. ------=_NextPart_000_012A_01CD165E.3D454860 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"><head> <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; = charset=3DUTF-8"> <meta name=3DGenerator content=3D"Microsoft Word 12 (filtered = medium)"><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} @font-face {font-family:Consolas; panose-1:2 11 6 9 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"; color:black;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} pre {mso-style-priority:99; = mso-style-link:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML =D0=97=D0=BD=D0=B0=D0=BA"; margin:0cm; margin-bottom:.0001pt; font-size:10.0pt; font-family:"Courier New"; color:black;} p.MsoAcetate, li.MsoAcetate, div.MsoAcetate {mso-style-priority:99; mso-style-link:"=D0=A2=D0=B5=D0=BA=D1=81=D1=82 = =D0=B2=D1=8B=D0=BD=D0=BE=D1=81=D0=BA=D0=B8 =D0=97=D0=BD=D0=B0=D0=BA"; margin:0cm; margin-bottom:.0001pt; font-size:8.0pt; font-family:"Tahoma","sans-serif"; color:black;} span.HTML = {mso-style-name:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML =D0=97=D0=BD=D0=B0=D0=BA"; mso-style-priority:99; = mso-style-link:"=D0=A1=D1=82=D0=B0=D0=BD=D0=B4=D0=B0=D1=80=D1=82=D0=BD=D1= =8B=D0=B9 HTML"; font-family:Consolas; color:black;} span.a {mso-style-name:"=D0=A2=D0=B5=D0=BA=D1=81=D1=82 = =D0=B2=D1=8B=D0=BD=D0=BE=D1=81=D0=BA=D0=B8 =D0=97=D0=BD=D0=B0=D0=BA"; mso-style-priority:99; mso-style-link:"=D0=A2=D0=B5=D0=BA=D1=81=D1=82 = =D0=B2=D1=8B=D0=BD=D0=BE=D1=81=D0=BA=D0=B8"; font-family:"Tahoma","sans-serif"; color:black;} span.EmailStyle21 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:windowtext;} span.EmailStyle22 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:#1F497D;} span.EmailStyle23 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:#1F497D;} span.EmailStyle24 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body bgcolor=3Dwhite lang=3DRU = link=3Dblue vlink=3Dpurple><div class=3DWordSection1><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Vdsm log = (spm):<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,189::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::INFO::2012-04-09 = 06:28:05,190::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,190::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::INFO::2012-04-09 = 06:28:05,190::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::INFO::2012-04-09 = 06:28:05,191::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,192::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmp2rKh29' (cwd = None)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,228::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: = <err> =3D ''; <rc> =3D 0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,235::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmp2rKh29' (cwd = None)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,270::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,271::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,272::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,272::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,272::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::DEBUG::2012-04-09 = 06:28:05,273::task::495::TaskManager.Task::(_debug) Task = 3634fc46-5104-4ebd-8ed6-1d4428beed92: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79073::INFO::2012-04-09 = 06:28:05,273::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,297::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::INFO::2012-04-09 = 06:28:05,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,298::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::INFO::2012-04-09 = 06:28:05,298::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::INFO::2012-04-09 = 06:28:05,298::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,308::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,308::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,309::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,309::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,310::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,310::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation = mutex<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,310::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::DEBUG::2012-04-09 = 06:28:05,311::task::495::TaskManager.Task::(_debug) Task = 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79074::INFO::2012-04-09 = 06:28:05,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,328::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::INFO::2012-04-09 = 06:28:05,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,329::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,329::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`af876ee5= -24f4-43f1-a161-d495903c82b5`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain'<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,330::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,330::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,330::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`af876ee5= -24f4-43f1-a161-d495903c82b5`::Granted request<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,331::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,331::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::ERROR::2012-04-09 = 06:28:05,331::task::868::TaskManager.Task::(_setError) Unexpected = error<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Traceback (most recent call = last):<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/task.py", line 876, in = _run<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 return fn(*args, = **kargs)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/spm.py", line 115, in = run<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 return self.func(*args, = **kwargs)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 = vars.task.getExclusiveLock(STORAGE, sdUUID)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/task.py", line 1311, in = getExclusiveLock<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 = self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 raise ValueError("Owner = %s: acquire: resource %s is already acquired" % (str(self), = fullName))<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,332::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._run: = b0ef8b95-fabe-43e0-944c-eb959cf9930a = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping = task<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,332::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: stopping in state preparing (force = False)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,333::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting = True<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::INFO::2012-04-09 = 06:28:05,333::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,333::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: Prepare: aborted: Owner = b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,333::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 0 aborting = True<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,334::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._doAbort: force = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,334::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,334::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state preparing -> = state aborting<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: _aborting: recover policy = none<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::task::495::TaskManager.Task::(_debug) Task = b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state aborting -> = state failed<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,335::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,336::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,336::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,336::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::DEBUG::2012-04-09 = 06:28:05,337::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::ERROR::2012-04-09 = 06:28:05,337::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79075::ERROR::2012-04-09 = 06:28:05,337::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last):<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/dispatcher.py", line 96, in = run<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 result =3D = ctask.prepare(self.func, *args, **kwargs)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>=C2=A0 File = "/usr/share/vdsm/storage/task.py", line 1178, in = prepare<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>=C2=A0=C2=A0=C2=A0 raise = self.error<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::INFO::2012-04-09 = 06:28:05,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D18 spmId=3D2<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 18}}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,350::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,351::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,351::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::DEBUG::2012-04-09 = 06:28:05,351::task::495::TaskManager.Task::(_debug) Task = 13ba6f56-88a9-4bab-a677-11b84d1170dd: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79076::INFO::2012-04-09 = 06:28:05,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': = 18}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,362::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::INFO::2012-04-09 = 06:28:05,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: ()<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,363::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,363::taskManager::92::TaskManager::(getAllTasksStatuses) = Entry.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,363::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,364::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: finished: {'allTasksStatus': = {}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,364::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,364::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,365::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::DEBUG::2012-04-09 = 06:28:05,365::task::495::TaskManager.Task::(_debug) Task = d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79077::INFO::2012-04-09 = 06:28:05,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,373::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::INFO::2012-04-09 = 06:28:05,373::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,374::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,374::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,374::taskManager::105::TaskManager::(getAllTasks) = Entry.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,375::taskManager::110::TaskManager::(getAllTasks) Return: = {}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,375::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`0f0e6998= -87a6-40e8-8ded-9f600cb0ca2d`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at = 'public_spmStop'<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,375::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,376::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,376::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`0f0e6998= -87a6-40e8-8ded-9f600cb0ca2d`::Granted request<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,376::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,377::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 1 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,377::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= eb290e92-2fbb-4e12-bde2-83292aade8e4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at = '_stop'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,378::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,378::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,378::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= eb290e92-2fbb-4e12-bde2-83292aade8e4`::Granted = request<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,379::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span><= /p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,379::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,379::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it.<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,380::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,380::taskManager::118::TaskManager::(unloadTasks) = Entry.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,380::taskManager::122::TaskManager::(unloadTasks) = Return<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,381::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,381::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,381::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,382::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,382::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D18', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', = '_SHA_CKSUM=3D5a3d228ad230c9d61a754c22abaa6b5f11b6c721']<o:p></o:p></span= style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,397::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::INFO::2012-04-09 = 06:28:05,398::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain = 4f1199ec-9cdb-4efe-91eb-587faeec2099<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:05,398::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,433::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,434::safelease::87::ClusterLock::(release) Cluster lock = released successfully<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,434::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: finished: = None<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,434::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,435::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,435::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,435::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,436::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,436::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,436::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::DEBUG::2012-04-09 = 06:28:06,437::task::495::TaskManager.Task::(_debug) Task = 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79078::INFO::2012-04-09 = 06:28:06,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79080::INFO::2012-04-09 = 06:28:08,375::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: ()<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79080::DEBUG::2012-04-09 = 06:28:08,375::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state init -> state = preparing<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79080::DEBUG::2012-04-09 = 06:28:08,376::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00202488899231', = 'lastCheck': 1333967286.1603661, 'valid': True, 'code': = 0}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79080::DEBUG::2012-04-09 = 06:28:08,376::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state preparing -> = state finished<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-79080::DEBUG::2012-04-09 = 06:28:08,376::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79080::DEBUG::2012-04-09 = 06:28:08,377::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79080::DEBUG::2012-04-09 = 06:28:08,377::task::495::TaskManager.Task::(_debug) Task = 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: ref 0 aborting = False<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-79080::INFO::2012-04-09 = 06:28:08,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, Return response: {'status': {'message': 'OK', = 'code': 0}, '4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': = '0.00202488899231', 'lastCheck': 1333967286.1603661, 'valid': True, = 'code': 0}}<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>^C<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'color:#1F497D'><o:p> </o:p></span></p><div><div = style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm = 0cm 0cm'><p class=3DMsoNormal><b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'>From:</span></b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'> users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] <b>On = Behalf Of </b>=D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87<br><b>Sent:</b> = Monday, April 09, 2012 2:20 PM<br><b>To:</b> = rvaknin@redhat.com<br><b>Cc:</b> users@ovirt.org<br><b>Subject:</b> Re: = [Users] Failed to attach Storage Domain export to Data = Center<o:p></o:p></span></p></div></div><p = class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>In an attempt to connect ovirt = storage domain fail<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>[root@noc-2-synt nfs4]# cat = b23c7ab6-b1d4-4888-8d4a-adc78e61db38/dom_md/metadata<o:p></o:p></span></p= style=3D'color:#1F497D'>CLASS=3DBackup<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>DESCRIPTION=3Dexport<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>IOOPTIMEOUTSEC=3D1<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASERETRIES=3D3<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASETIMESEC=3D5<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKPOLICY=3D<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKRENEWALINTERVALSEC=3D5<o:p></o:p></span></p><= p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>MASTER_VERSION=3D0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0<= o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>REMOTE_PATH=3D10.1.20.2:/home/nfs4<o:p></o:p></sp= an></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ROLE=3DRegular<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>SDUUID=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38<o:p= style=3D'color:#1F497D'>TYPE=3DNFS<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>VERSION=3D0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>_SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd= 2b59<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span style=3D'color:#1F497D'>logs:</span><span = lang=3DEN-US style=3D'color:#1F497D'><o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:38,628 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = Storage<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,658 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 651832d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,658 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 69324bf2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,747 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 69324bf2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,748 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:38,750 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 1b5bf8d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,762 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-49) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 651832d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,763 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-49) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:38,765 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 59f6a186<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,775 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 1b5bf8d2<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,791 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-49) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 59f6a186<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,792 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 58a14aa<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,816 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Failed in AttachStorageDomainVDS = method<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-50) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:38,817 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:38,836 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) START, SpmStopVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = fbb0af9<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:38,848 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) SpmStopVDSCommand::Stopping SPM on vds 10.2.20.8, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 = 06:11:39,915 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-50) FINISH, SpmStopVDSCommand, log id: = fbb0af9<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:39,915 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) Irs placed on server null failed. Proceed = Failover<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:39,923 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-50) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = StoragePool<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-09 06:11:39,948 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) hostFromVds::selectedVds - 10.1.20.7, spmStatus Free, = storage pool Default<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:39,953 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) starting spm on vds 10.1.20.7, storage pool Default, = prevId -1, LVER 13<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:39,955 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) START, SpmStartVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D13, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 3eff326c<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-09 06:11:39,971 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-50) spmStart polling started: taskId =3D = e6db13d3-1c55-4d11-9fbc-8abf50c635ae<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>^C<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'color:#1F497D'><o:p> </o:p></span></p><div><div = style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm = 0cm 0cm'><p class=3DMsoNormal><b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'>From:</span></b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'> Rami Vaknin [<a = href=3D"mailto:rvaknin@redhat.com">mailto:rvaknin@redhat.com</a>] = <br><b>Sent:</b> Monday, April 09, 2012 1:27 PM<br><b>To:</b> = =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87<br><b>Cc:</b> <a = href=3D"mailto:users@ovirt.org">users@ovirt.org</a><br><b>Subject:</b> = Re: [Users] Failed to attach Storage Domain export to Data = Center<o:p></o:p></span></p></div></div><p = class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>On = 04/08/2012 07:05 PM, =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote: = <o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>I found the metadata in the file = ./dom_md/metadata</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Changed = =E2=80=9Cb23c7ab6-b1d4-4888-8d4a-adc78e61db38=E2=80=9D to = =E2=80=9Ce3e565d6-8104-11e1-afd2-001cc43ceea0=E2=80=9D = (SDUUID)</span><o:p></o:p></p><p class=3DMsoNormal = style=3D'margin-bottom:12.0pt'><span = style=3D'font-size:12.0pt;font-family:"Times New = Roman","serif"'><br>It's wrong, you should change that in POOL_UUID and = not in SDUUID (my fault, by spuuid I meant to the = POOL_UUID).<br><br><o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>The data I gave the command: = </span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>[root@noc-3-synt ~]# vdsClient -s 0 = getConnectedStoragePoolsList</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p><= /o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Found in = </span><span lang=3DEN-US>checksum</span><span lang=3DEN-US = style=3D'color:#1F497D'> vdsm.log:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-36836::WARNING::2012-04-08 = 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) = data seal is broken metadata declares = `4df87518878fd0d74932917b83aba9672f92dff2` should be = `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines=3D{'VERSION': '0', = 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', = 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', = 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', = 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', = 'CLASS': 'Backup'})</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>MetaDataSealIsBroken: Meta Data = seal is broken (checksum mismatch): 'cksum =3D = 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum =3D = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Changed = it.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>As a result, the metadata = file:</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>[root@noc-2-synt dom_md]# cat = metadata</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>CLASS=3DBackup</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>DESCRIPTION=3Dexport</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>IOOPTIMEOUTSEC=3D1</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASERETRIES=3D3</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LEASETIMESEC=3D5</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKPOLICY=3D</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>LOCKRENEWALINTERVALSEC=3D5</span><o:p></o:p></p><= p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>MASTER_VERSION=3D0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>POOL_UUID=3D801e35d4-7405-11e1-b152-001cc43ceea0<= /span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>REMOTE_PATH=3D10.1.20.2:/home/nfs4</span><o:p></o= :p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ROLE=3DRegular</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>SDUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0</sp= an><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>TYPE=3DNFS</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>VERSION=3D0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>_SHA_CKSUM=3Dd91770b5a0f1956cbfb79a6a59b5291dc5bd= 2b59</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>attachment = to the ovirt fails:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-35311::INFO::2012-04-08 = 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or = directory: = '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc= 43ceea0'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-35311::ERROR::2012-04-08 = 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected = error</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>I renamed = the folder that contains the data to = e3e565d6-8104-11e1-afd2-001cc43ceea0 for = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>attachment = to the ovirt fails =3D(</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>messages:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'font-size:12.0pt'>Error = while executing action AttachStorageDomainToPool: Error in storage = domain action</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> = </span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>engine.log:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,557 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = Storage</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 3dbf104d</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,583 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 4ac74ee6</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,685 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-50) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 4ac74ee6</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,686 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:10,689 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 53bd874a</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,690 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-46) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 3dbf104d</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,691 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:10,693 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 3129fd68</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,714 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 53bd874a</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,716 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 3129fd68</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,718 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 2b559a22</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Failed in AttachStorageDomainVDS = method</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,737 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] = (pool-5-thread-49) Error code StorageDomainActionError and error message = IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,738 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: = IRSErrorException: IRSGenericException: IRSErrorException: Failed to = AttachStorageDomainVDS, error =3D Error in storage domain action: = ('sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0, = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0',)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:10,752 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) START, SpmStopVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: = 6b2f2dd2</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:10,764 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, = pool id e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 = 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] = (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: = 6b2f2dd2</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,843 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) Irs placed on server null failed. Proceed = Failover</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,850 INFO = [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] = (pool-5-thread-49) Running command: SetStoragePoolStatusCommand = internal: true. Entities affected : ID: = e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: = StoragePool</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>2012-04-08 11:53:11,881 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, = storage pool Default</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,886 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, = prevId -1, LVER 8</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,888 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) START, SpmStartVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=3D-1, prevLVER=3D8, = storagePoolFormatType=3DV1, recoveryMode=3DManual, SCSIFencing=3Dfalse), = log id: 382e702</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>2012-04-08 11:53:11,905 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] = (pool-5-thread-49) spmStart polling started: taskId =3D = f65c40e4-956f-4c01-ab3f-5c43f2853806</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Vdsm.log:</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38311::INFO::2012-04-08 = 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getStoragePoolInfo, Return response: {'status': {'message': = 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': = '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': = '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', = 'master_ver': 1, 'lver': 10}, 'dominfo': = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', = 'diskfree': '490079256576', 'alerts': [], 'disktotal': = '516524343296'}}}</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,953::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validat= e) Request to validate NFS storage server</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo = -n /bin/mount -o soft,timeo=3D600,retrans=3D6,nosharecache,vers=3D3 -t = nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd = None)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: = <err> =3D ''; <rc> =3D 0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) = '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd = None)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: = <err> =3D ''; <rc> =3D 0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,022::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::DEBUG::2012-04-08 = 11:54:33,024::task::495::TaskManager.Task::(_debug) Task = 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38312::INFO::2012-04-08 = 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: validateStorageServerConnection, Return response: {'status': = {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, args: (domType=3D1, = spUUID=3D00000000-0000-0000-0000-000000000000, conList=3D[{'connection': = '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': = '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}])</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,038::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect)= Request to connect NFS storage server</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__valida= teConnectionParams) conList=3D[{'connection': '10.1.20.2:/home/nfs4', = 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': = ''}]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' got the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation = 'lvm invalidate operation' released the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' got the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation = 'lvm invalidate operation' released the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' got the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation = 'lvm invalidate operation' released the operation = mutex</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': = [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,050::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::DEBUG::2012-04-08 = 11:54:33,051::task::495::TaskManager.Task::(_debug) Task = 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38313::INFO::2012-04-08 = 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: connectStorageServer, Return response: {'status': {'message': = 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': = 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::INFO::2012-04-08 = 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: attachStorageDomain, args: ( = sdUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0 = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '1072' at = 'public_attachStorageDomain'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`a97b6096= -7b1a-4f93-b0c0-322399c2ac3a`::Granted request</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,065::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected = error</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Traceback (most recent call = last):</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/task.py", line 876, in = _run</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> return fn(*args, = **kargs)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/spm.py", line 115, in = run</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> return self.func(*args, = **kwargs)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/spm.py", line 1073, in = public_attachStorageDomain</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> = vars.task.getExclusiveLock(STORAGE, sdUUID)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/task.py", line 1311, in = getExclusiveLock</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'> = self.resOwner.acquire(namespace, resName, = resourceManager.LockType.exclusive, timeout)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/resourceManager.py", line 673, in = acquire</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> raise ValueError("Owner = %s: acquire: resource %s is already acquired" % (str(self), = fullName))</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: = 2b306544-6fcc-443b-9ea6-23ae4b71d422 = ('e3e565d6-8104-11e1-afd2-001cc43ceea0', = 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping = task</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,066::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force = False)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting = True</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::INFO::2012-04-08 = 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is = aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code = 100</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,067::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting = True</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,068::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> = state aborting</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy = none</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::task::495::TaskManager.Task::(_debug) Task = 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> = state failed</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::DEBUG::2012-04-08 = 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38314::ERROR::2012-04-08 = 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) = Traceback (most recent call last):</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/dispatcher.py", line 96, in = run</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> result =3D = ctask.prepare(self.func, *args, **kwargs)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'> File = "/usr/share/vdsm/storage/task.py", line 1178, in = prepare</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> raise = self.error</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>ValueError: Owner = 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already = acquired</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::INFO::2012-04-08 = 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,086::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=3DSPM = spmLver=3D10 spmId=3D2</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, = 'spmStatus': 'SPM', 'spmLver': 10}}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,087::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::DEBUG::2012-04-08 = 11:54:33,088::task::495::TaskManager.Task::(_debug) Task = 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38315::INFO::2012-04-08 = 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getSpmStatus, Return response: {'status': {'message': 'OK', = 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': = 10}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::INFO::2012-04-08 = 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, args: ()</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) = Entry.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) = Return: {}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': = {}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::DEBUG::2012-04-08 = 11:54:33,104::task::495::TaskManager.Task::(_debug) Task = 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38316::INFO::2012-04-08 = 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: getAllTasksStatuses, Return response: {'status': {'message': = 'OK', 'code': 0}, 'allTasksStatus': {}}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) = [10.1.20.2]</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, args: ( = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) = spUUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) = Entry.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: = {}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '573' at = 'public_spmStop'</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now = locking as 'exclusive' (1 active user)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`417fa7c5= -958c-4001-8944-d64df2b44f43`::Granted request</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,116::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: = Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 = (exclusive)</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in = '/usr/share/vdsm/storage/spm.py' line '584' at = '_stop'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) = Trying to register resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type = 'exclusive'</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. = Now locking as 'exclusive' (1 active user)</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) = ResName=3D`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=3D`= b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted = request</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p><= /p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' = (0 active users)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, = finding out if anyone is waiting for it.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) = Entry.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) = Return</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master = `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/maste= r` is not mounted, skipping</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: = e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) = Starting transaction</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) = Flushing changes</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about = to write lines (FileMetadataRW)=3D['CLASS=3DData', 'DESCRIPTION=3Dmain', = 'IOOPTIMEOUTSEC=3D10', 'LEASERETRIES=3D3', 'LEASETIMESEC=3D60', = 'LOCKPOLICY=3D', 'LOCKRENEWALINTERVALSEC=3D5', 'MASTER_VERSION=3D1', = 'POOL_DESCRIPTION=3DDefault', = 'POOL_DOMAINS=3D4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', = 'POOL_SPM_ID=3D-1', 'POOL_SPM_LVER=3D10', = 'POOL_UUID=3De3e565d6-8104-11e1-afd2-001cc43ceea0', = 'REMOTE_PATH=3Dnfs.corp:/share', 'ROLE=3DMaster', = 'SDUUID=3D4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=3DNFS', = 'VERSION=3D0', = '_SHA_CKSUM=3D1d1cd722cad65ebb2edfa75641c30a5b97302f84']</span><o:p></o:p= style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) = Finished transaction</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster = lock for domain = 4f1199ec-9cdb-4efe-91eb-587faeec2099</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) = '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd = /usr/libexec/vdsm/)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: = <err> =3D ''; <rc> =3D 0</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock = released successfully</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: = None</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,165::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources = {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: = 'None'>}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) = Trying to release resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) = Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 = active users)</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) = Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding = out if anyone is waiting for it.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) = No one is waiting for resource = 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing = records.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::DEBUG::2012-04-08 = 11:54:34,168::task::495::TaskManager.Task::(_debug) Task = 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38317::INFO::2012-04-08 = 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and = protect: spmStop, Return response: {'status': {'message': 'OK', 'code': = 0}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::INFO::2012-04-08 = 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and = protect: repoStats, args: ()</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state = preparing</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,395::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: = {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', = 'lastCheck': 1333900465.46615, 'valid': True, 'code': = 0}}</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> = state finished</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) = Owner.releaseAll requests {} resources {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) = Owner.cancelAll requests {}</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'>Thread-38318::DEBUG::2012-04-08 = 11:54:34,397::task::495::TaskManager.Task::(_debug) Task = 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting = False</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'> </span><o:p></o:p></p><div><div = style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm = 0cm 0cm'><p class=3DMsoNormal><b><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'>From:</span></b><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'> Rami Vaknin [<a = href=3D"mailto:rvaknin@redhat.com">mailto:rvaknin@redhat.com</a>] = <br><b>Sent:</b> Sunday, April 08, 2012 7:02 PM<br><b>To:</b> = </span><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'>=D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87</span><span lang=3DEN-US = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowt= ext'><br><b>Subject:</b> Re: [Users] Failed to attach Storage Domain = export to Data Center</span><o:p></o:p></p></div></div><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal style=3D'margin-bottom:12.0pt'>On 04/08/2012 03:48 PM, = =D0=B7=D0=BE=D1=80=D1=80=D1=8B=D1=87 wrote:<br><br>1. Change the spuuid = in the domain's metadata file to be the spuuid of the new datacenter = (the metadata can be found in dom_md/ dir somewhere inside the stoarge = domain's directories tree), you can take the new spuuid from any host = running on the new data center by running:<br>vdsClient -s 0 = getConnectedStoragePoolsList<br><br>2. Try to attach the storage domain = again, this time it will fail due to broken checksum on the metadata = file, go to vdsm.log of the spm and look for "checksum", it = will tell you what the right checksum should be like, copy it to the = metadata file.<br>This time the attachment should = work.<br><br><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Hi. = </span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>Thank you = for the help you provided me in the mailing = list.</span><o:p></o:p></p><p class=3DMsoNormal><span lang=3DEN-US>I = dropped the settings using the ovirt - = engine-cleanup</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>Ovirt reinstalled and tried to restore the virtual machine = from the export of storage domain.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>If you try an attachment domain = ovirt error:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'font-size:12.0pt'>Error while executing action = AttachStorageDomainToPool: The Domain is already used by another Data = Center.<br/>In order to use it, detach it from the Data Center it = is currently attached to.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>In the = logs:</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,898 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand = internal: false. Entities affected : ID: = b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: = Storage</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 1985db9c</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:49,925 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) START, = ValidateStorageServerConnectionVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 17f02ea3</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,004 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-47) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 17f02ea3</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,005 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,008 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId =3D = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 6f4db784</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,009 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectio= nVDSCommand] (pool-5-thread-48) FINISH, = ValidateStorageServerConnectionVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 1985db9c</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,010 INFO = [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] = (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: = true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa = Type: System</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,013 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId =3D = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId =3D = 00000000-0000-0000-0000-000000000000, storageType =3D NFS, = connectionList =3D [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, = connection: 10.1.20.2:/home/nfs4 };]), log id: = 42baf149</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,031 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 6f4db784</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,035 INFO = [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand= ] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: = {aef7de23-f271-45f3-853b-9e7b7be6607f=3D0}, log id: = 42baf149</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:50,037 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit =3D false, = compatabilityVersion =3D null, storageDomainId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: = 6f22bcac</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] = (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: = IrsOperationFailedNoFailoverException: IRSGenericException: = IRSErrorException: Storage domain already attached to pool: = 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>2012-04-08 08:39:53,186 INFO = [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]= (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: = 6f22bcac</span><o:p></o:p></p><p class=3DMsoNormal><span = lang=3DEN-US>2012-04-08 08:39:53,186 ERROR = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command = org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw = Vdc Bll exception. With error message VdcBLLException: = org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverExc= eption: IRSGenericException: IRSErrorException: Storage domain already = attached to pool: 'domain=3Db23c7ab6-b1d4-4888-8d4a-adc78e61db38, = pool=3D801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>2012-04-08 08:39:53,190 INFO = [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] = (pool-5-thread-50) Command [id=3Db717db6a-a582-49ba-92c1-6bb09340789a]: = Compensating NEW_ENTITY_ID of = org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; = snapshot: storagePoolId =3D e3e565d6-8104-11e1-afd2-001cc43ceea0, = storageId =3D = b23c7ab6-b1d4-4888-8d4a-adc78e61db38.</span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US> </span><o:p></o:p></p><p = class=3DMsoNormal><span lang=3DEN-US>I did not deactivate the export = domain before clearing the configuration. How can I be in this case? I = can manually remove the flag of the domain?</span><o:p></o:p></p><p = class=3DMsoNormal style=3D'margin-bottom:12.0pt'><span = style=3D'font-size:12.0pt'><br><br><br></span><o:p></o:p></p><pre>_______= ________________________________________<o:p></o:p></pre><pre>Users = mailing list<o:p></o:p></pre><pre><a = href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><o:p></o:p></pre><pre>= <a = href=3D"http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt= .org/mailman/listinfo/users</a><o:p></o:p></pre><p class=3DMsoNormal = style=3D'margin-bottom:12.0pt'><span = style=3D'font-size:12.0pt'><br><br><br></span><o:p></o:p></p><pre>-- = <o:p></o:p></pre><pre> <o:p></o:p></pre><pre>Thanks,<o:p></o:p></pre= style=3D'margin-bottom:12.0pt'><span = style=3D'font-size:12.0pt;font-family:"Times New = Roman","serif"'><br><br><o:p></o:p></span></p><pre>-- = <o:p></o:p></pre><pre><o:p> </o:p></pre><pre>Thanks,<o:p></o:p></pre=
<pre><o:p> </o:p></pre><pre>Rami Vaknin, QE @ Red Hat, TLV, = IL.<o:p></o:p></pre></div></body></html> ------=_NextPart_000_012A_01CD165E.3D454860--

This is a multi-part message in MIME format. --------------070000070304040100060303 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit All commands in this both logs you've attached (see example line below) indicate that the sdUUID is still e3e565d6-8104-11e1-afd2-001cc43ceea0 which is the same as the spUUID. Are you sure your metadata file contains the "SDUUID=b23c7ab6-b1d4-4888-8d4a-adc78e61db38" line and not the wrong "SDUUID=3e565d6-8104-11e1-afd2-001cc43ceea0"line? Thread-79075::INFO::2012-04-09 06:28:05,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0) On 04/09/2012 01:37 PM, зоррыч wrote:
Vdsm log (spm):
Thread-79073::DEBUG::2012-04-09 06:28:05,189::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-79073::INFO::2012-04-09 06:28:05,190::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])
Thread-79073::DEBUG::2012-04-09 06:28:05,190::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state init -> state preparing
Thread-79073::INFO::2012-04-09 06:28:05,190::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server
Thread-79073::INFO::2012-04-09 06:28:05,191::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]
Thread-79073::DEBUG::2012-04-09 06:28:05,192::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs 10.1.20.2:/home/nfs4 /tmp/tmp2rKh29' (cwd None)
Thread-79073::DEBUG::2012-04-09 06:28:05,228::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
Thread-79073::DEBUG::2012-04-09 06:28:05,235::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmp2rKh29' (cwd None)
Thread-79073::DEBUG::2012-04-09 06:28:05,270::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
Thread-79073::DEBUG::2012-04-09 06:28:05,271::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-79073::DEBUG::2012-04-09 06:28:05,272::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state preparing -> state finished
Thread-79073::DEBUG::2012-04-09 06:28:05,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-79073::DEBUG::2012-04-09 06:28:05,272::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79073::DEBUG::2012-04-09 06:28:05,273::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: ref 0 aborting False
Thread-79073::INFO::2012-04-09 06:28:05,273::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-79074::DEBUG::2012-04-09 06:28:05,297::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-79074::INFO::2012-04-09 06:28:05,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])
Thread-79074::DEBUG::2012-04-09 06:28:05,298::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state init -> state preparing
Thread-79074::INFO::2012-04-09 06:28:05,298::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server
Thread-79074::INFO::2012-04-09 06:28:05,298::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]
Thread-79074::DEBUG::2012-04-09 06:28:05,308::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
Thread-79074::DEBUG::2012-04-09 06:28:05,308::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
Thread-79074::DEBUG::2012-04-09 06:28:05,309::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
Thread-79074::DEBUG::2012-04-09 06:28:05,309::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
Thread-79074::DEBUG::2012-04-09 06:28:05,310::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
Thread-79074::DEBUG::2012-04-09 06:28:05,310::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
Thread-79074::DEBUG::2012-04-09 06:28:05,310::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-79074::DEBUG::2012-04-09 06:28:05,311::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state preparing -> state finished
Thread-79074::DEBUG::2012-04-09 06:28:05,311::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-79074::DEBUG::2012-04-09 06:28:05,311::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79074::DEBUG::2012-04-09 06:28:05,311::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: ref 0 aborting False
Thread-79074::INFO::2012-04-09 06:28:05,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-79075::DEBUG::2012-04-09 06:28:05,328::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-79075::INFO::2012-04-09 06:28:05,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-79075::DEBUG::2012-04-09 06:28:05,329::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state init -> state preparing
Thread-79075::DEBUG::2012-04-09 06:28:05,329::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`af876ee5-24f4-43f1-a161-d495903c82b5`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1072' at 'public_attachStorageDomain'
Thread-79075::DEBUG::2012-04-09 06:28:05,330::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-79075::DEBUG::2012-04-09 06:28:05,330::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-79075::DEBUG::2012-04-09 06:28:05,330::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`af876ee5-24f4-43f1-a161-d495903c82b5`::Granted request
Thread-79075::DEBUG::2012-04-09 06:28:05,331::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)
Thread-79075::DEBUG::2012-04-09 06:28:05,331::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting False
Thread-79075::ERROR::2012-04-09 06:28:05,331::task::868::TaskManager.Task::(_setError) Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 876, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/storage/spm.py", line 115, in run
return self.func(*args, **kwargs)
File "/usr/share/vdsm/storage/spm.py", line 1073, in public_attachStorageDomain
vars.task.getExclusiveLock(STORAGE, sdUUID)
File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock
self.resOwner.acquire(namespace, resName, resourceManager.LockType.exclusive, timeout)
File "/usr/share/vdsm/storage/resourceManager.py", line 673, in acquire
raise ValueError("Owner %s: acquire: resource %s is already acquired" % (str(self), fullName))
ValueError: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-79075::DEBUG::2012-04-09 06:28:05,332::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._run: b0ef8b95-fabe-43e0-944c-eb959cf9930a ('e3e565d6-8104-11e1-afd2-001cc43ceea0', 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task
Thread-79075::DEBUG::2012-04-09 06:28:05,332::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: stopping in state preparing (force False)
Thread-79075::DEBUG::2012-04-09 06:28:05,333::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting True
Thread-79075::INFO::2012-04-09 06:28:05,333::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: 'Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code 100
Thread-79075::DEBUG::2012-04-09 06:28:05,333::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: Prepare: aborted: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-79075::DEBUG::2012-04-09 06:28:05,333::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 0 aborting True
Thread-79075::DEBUG::2012-04-09 06:28:05,334::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._doAbort: force False
Thread-79075::DEBUG::2012-04-09 06:28:05,334::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79075::DEBUG::2012-04-09 06:28:05,334::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state preparing -> state aborting
Thread-79075::DEBUG::2012-04-09 06:28:05,335::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: _aborting: recover policy none
Thread-79075::DEBUG::2012-04-09 06:28:05,335::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state aborting -> state failed
Thread-79075::DEBUG::2012-04-09 06:28:05,335::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}
Thread-79075::DEBUG::2012-04-09 06:28:05,335::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79075::DEBUG::2012-04-09 06:28:05,336::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-79075::DEBUG::2012-04-09 06:28:05,336::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-79075::DEBUG::2012-04-09 06:28:05,336::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-79075::DEBUG::2012-04-09 06:28:05,337::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-79075::ERROR::2012-04-09 06:28:05,337::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-79075::ERROR::2012-04-09 06:28:05,337::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
result = ctask.prepare(self.func, *args, **kwargs)
File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
raise self.error
ValueError: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-79076::DEBUG::2012-04-09 06:28:05,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-79076::INFO::2012-04-09 06:28:05,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-79076::DEBUG::2012-04-09 06:28:05,350::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state init -> state preparing
Thread-79076::DEBUG::2012-04-09 06:28:05,350::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=SPM spmLver=18 spmId=2
Thread-79076::DEBUG::2012-04-09 06:28:05,350::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18}}
Thread-79076::DEBUG::2012-04-09 06:28:05,350::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state preparing -> state finished
Thread-79076::DEBUG::2012-04-09 06:28:05,351::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-79076::DEBUG::2012-04-09 06:28:05,351::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79076::DEBUG::2012-04-09 06:28:05,351::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: ref 0 aborting False
Thread-79076::INFO::2012-04-09 06:28:05,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18}}
Thread-79077::DEBUG::2012-04-09 06:28:05,362::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-79077::INFO::2012-04-09 06:28:05,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
Thread-79077::DEBUG::2012-04-09 06:28:05,363::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state init -> state preparing
Thread-79077::DEBUG::2012-04-09 06:28:05,363::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
Thread-79077::DEBUG::2012-04-09 06:28:05,363::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
Thread-79077::DEBUG::2012-04-09 06:28:05,364::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: finished: {'allTasksStatus': {}}
Thread-79077::DEBUG::2012-04-09 06:28:05,364::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state preparing -> state finished
Thread-79077::DEBUG::2012-04-09 06:28:05,364::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-79077::DEBUG::2012-04-09 06:28:05,365::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79077::DEBUG::2012-04-09 06:28:05,365::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: ref 0 aborting False
Thread-79077::INFO::2012-04-09 06:28:05,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
Thread-79078::DEBUG::2012-04-09 06:28:05,373::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-79078::INFO::2012-04-09 06:28:05,373::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-79078::DEBUG::2012-04-09 06:28:05,374::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state init -> state preparing
Thread-79078::DEBUG::2012-04-09 06:28:05,374::spm::547::Storage.SPM::(public_spmStop) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0
Thread-79078::DEBUG::2012-04-09 06:28:05,374::taskManager::105::TaskManager::(getAllTasks) Entry.
Thread-79078::DEBUG::2012-04-09 06:28:05,375::taskManager::110::TaskManager::(getAllTasks) Return: {}
Thread-79078::DEBUG::2012-04-09 06:28:05,375::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`0f0e6998-87a6-40e8-8ded-9f600cb0ca2d`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'
Thread-79078::DEBUG::2012-04-09 06:28:05,375::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-79078::DEBUG::2012-04-09 06:28:05,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-79078::DEBUG::2012-04-09 06:28:05,376::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`0f0e6998-87a6-40e8-8ded-9f600cb0ca2d`::Granted request
Thread-79078::DEBUG::2012-04-09 06:28:05,376::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)
Thread-79078::DEBUG::2012-04-09 06:28:05,377::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 1 aborting False
Thread-79078::DEBUG::2012-04-09 06:28:05,377::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`eb290e92-2fbb-4e12-bde2-83292aade8e4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
Thread-79078::DEBUG::2012-04-09 06:28:05,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-79078::DEBUG::2012-04-09 06:28:05,378::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-79078::DEBUG::2012-04-09 06:28:05,378::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`eb290e92-2fbb-4e12-bde2-83292aade8e4`::Granted request
Thread-79078::DEBUG::2012-04-09 06:28:05,379::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-79078::DEBUG::2012-04-09 06:28:05,379::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-79078::DEBUG::2012-04-09 06:28:05,379::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-79078::DEBUG::2012-04-09 06:28:05,380::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-79078::DEBUG::2012-04-09 06:28:05,380::taskManager::118::TaskManager::(unloadTasks) Entry.
Thread-79078::DEBUG::2012-04-09 06:28:05,380::taskManager::122::TaskManager::(unloadTasks) Return
Thread-79078::DEBUG::2012-04-09 06:28:05,381::spm::201::Storage.SPM::(__cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/master` is not mounted, skipping
Thread-79078::DEBUG::2012-04-09 06:28:05,381::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: e3e565d6-8104-11e1-afd2-001cc43ceea0
Thread-79078::DEBUG::2012-04-09 06:28:05,381::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
Thread-79078::DEBUG::2012-04-09 06:28:05,382::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
Thread-79078::DEBUG::2012-04-09 06:28:05,382::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=main', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=18', 'POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH=nfs.corp:/share', 'ROLE=Master', 'SDUUID=4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=5a3d228ad230c9d61a754c22abaa6b5f11b6c721']
Thread-79078::DEBUG::2012-04-09 06:28:05,397::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
Thread-79078::INFO::2012-04-09 06:28:05,398::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099
Thread-79078::DEBUG::2012-04-09 06:28:05,398::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd /usr/libexec/vdsm/)
Thread-79078::DEBUG::2012-04-09 06:28:06,433::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
Thread-79078::DEBUG::2012-04-09 06:28:06,434::safelease::87::ClusterLock::(release) Cluster lock released successfully
Thread-79078::DEBUG::2012-04-09 06:28:06,434::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: finished: None
Thread-79078::DEBUG::2012-04-09 06:28:06,434::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state preparing -> state finished
Thread-79078::DEBUG::2012-04-09 06:28:06,435::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}
Thread-79078::DEBUG::2012-04-09 06:28:06,435::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79078::DEBUG::2012-04-09 06:28:06,435::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-79078::DEBUG::2012-04-09 06:28:06,436::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-79078::DEBUG::2012-04-09 06:28:06,436::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-79078::DEBUG::2012-04-09 06:28:06,436::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-79078::DEBUG::2012-04-09 06:28:06,437::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 0 aborting False
Thread-79078::INFO::2012-04-09 06:28:06,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}
Thread-79080::INFO::2012-04-09 06:28:08,375::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
Thread-79080::DEBUG::2012-04-09 06:28:08,375::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state init -> state preparing
Thread-79080::DEBUG::2012-04-09 06:28:08,376::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: finished: {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00202488899231', 'lastCheck': 1333967286.1603661, 'valid': True, 'code': 0}}
Thread-79080::DEBUG::2012-04-09 06:28:08,376::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state preparing -> state finished
Thread-79080::DEBUG::2012-04-09 06:28:08,376::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-79080::DEBUG::2012-04-09 06:28:08,377::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-79080::DEBUG::2012-04-09 06:28:08,377::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: ref 0 aborting False
Thread-79080::INFO::2012-04-09 06:28:08,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, '4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00202488899231', 'lastCheck': 1333967286.1603661, 'valid': True, 'code': 0}}
^C
*From:*users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] *On Behalf Of *зоррыч *Sent:* Monday, April 09, 2012 2:20 PM *To:* rvaknin@redhat.com *Cc:* users@ovirt.org *Subject:* Re: [Users] Failed to attach Storage Domain export to Data Center
In an attempt to connect ovirt storage domain fail
[root@noc-2-synt nfs4]# cat b23c7ab6-b1d4-4888-8d4a-adc78e61db38/dom_md/metadata
CLASS=Backup
DESCRIPTION=export
IOOPTIMEOUTSEC=1
LEASERETRIES=3
LEASETIMESEC=5
LOCKPOLICY=
LOCKRENEWALINTERVALSEC=5
MASTER_VERSION=0
POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0
REMOTE_PATH=10.1.20.2:/home/nfs4
ROLE=Regular
SDUUID=b23c7ab6-b1d4-4888-8d4a-adc78e61db38
TYPE=NFS
VERSION=0
_SHA_CKSUM=d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59
logs:
2012-04-09 06:11:38,628 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage
2012-04-09 06:11:38,658 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-49) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 651832d2
2012-04-09 06:11:38,658 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 69324bf2
2012-04-09 06:11:38,747 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 69324bf2
2012-04-09 06:11:38,748 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-09 06:11:38,750 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 1b5bf8d2
2012-04-09 06:11:38,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-49) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 651832d2
2012-04-09 06:11:38,763 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-49) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-09 06:11:38,765 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-49) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 59f6a186
2012-04-09 06:11:38,775 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 1b5bf8d2
2012-04-09 06:11:38,791 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-49) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 59f6a186
2012-04-09 06:11:38,792 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 58a14aa
2012-04-09 06:11:38,816 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-50) Failed in AttachStorageDomainVDS method
2012-04-09 06:11:38,817 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-50) Error code StorageDomainActionError and error message IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)
2012-04-09 06:11:38,817 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)
2012-04-09 06:11:38,836 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-50) START, SpmStopVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: fbb0af9
2012-04-09 06:11:38,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-50) SpmStopVDSCommand::Stopping SPM on vds 10.2.20.8, pool id e3e565d6-8104-11e1-afd2-001cc43ceea0
2012-04-09 06:11:39,915 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-50) FINISH, SpmStopVDSCommand, log id: fbb0af9
2012-04-09 06:11:39,915 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) Irs placed on server null failed. Proceed Failover
2012-04-09 06:11:39,923 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-50) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool
2012-04-09 06:11:39,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) hostFromVds::selectedVds - 10.1.20.7, spmStatus Free, storage pool Default
2012-04-09 06:11:39,953 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) starting spm on vds 10.1.20.7, storage pool Default, prevId -1, LVER 13
2012-04-09 06:11:39,955 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-50) START, SpmStartVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=-1, prevLVER=13, storagePoolFormatType=V1, recoveryMode=Manual, SCSIFencing=false), log id: 3eff326c
2012-04-09 06:11:39,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-50) spmStart polling started: taskId = e6db13d3-1c55-4d11-9fbc-8abf50c635ae
^C
*From:*Rami Vaknin [mailto:rvaknin@redhat.com] *Sent:* Monday, April 09, 2012 1:27 PM *To:* зоррыч *Cc:* users@ovirt.org <mailto:users@ovirt.org> *Subject:* Re: [Users] Failed to attach Storage Domain export to Data Center
On 04/08/2012 07:05 PM, зоррыч wrote:
I found the metadata in the file ./dom_md/metadata
Changed “b23c7ab6-b1d4-4888-8d4a-adc78e61db38” to “e3e565d6-8104-11e1-afd2-001cc43ceea0” (SDUUID)
It's wrong, you should change that in POOL_UUID and not in SDUUID (my fault, by spuuid I meant to the POOL_UUID).
The data I gave the command:
[root@noc-3-synt ~]# vdsClient -s 0 getConnectedStoragePoolsList
e3e565d6-8104-11e1-afd2-001cc43ceea0
Found in checksumvdsm.log:
Thread-36836::WARNING::2012-04-08 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) data seal is broken metadata declares `4df87518878fd0d74932917b83aba9672f92dff2` should be `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines={'VERSION': '0', 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', 'CLASS': 'Backup'})
MetaDataSealIsBroken: Meta Data seal is broken (checksum mismatch): 'cksum = 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59'
Changed it.
As a result, the metadata file:
[root@noc-2-synt dom_md]# cat metadata
CLASS=Backup
DESCRIPTION=export
IOOPTIMEOUTSEC=1
LEASERETRIES=3
LEASETIMESEC=5
LOCKPOLICY=
LOCKRENEWALINTERVALSEC=5
MASTER_VERSION=0
POOL_UUID=801e35d4-7405-11e1-b152-001cc43ceea0
REMOTE_PATH=10.1.20.2:/home/nfs4
ROLE=Regular
SDUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0
TYPE=NFS
VERSION=0
_SHA_CKSUM=d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59
attachment to the ovirt fails:
Thread-35311::INFO::2012-04-08 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or directory: '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-35311::ERROR::2012-04-08 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected error
I renamed the folder that contains the data to e3e565d6-8104-11e1-afd2-001cc43ceea0 for b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID)
attachment to the ovirt fails =(
messages:
Error while executing action AttachStorageDomainToPool: Error in storage domain action
engine.log:
2012-04-08 11:53:10,557 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage
2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3dbf104d
2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 4ac74ee6
2012-04-08 11:53:10,685 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 4ac74ee6
2012-04-08 11:53:10,686 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 11:53:10,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 53bd874a
2012-04-08 11:53:10,690 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3dbf104d
2012-04-08 11:53:10,691 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 11:53:10,693 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3129fd68
2012-04-08 11:53:10,714 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 53bd874a
2012-04-08 11:53:10,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3129fd68
2012-04-08 11:53:10,718 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 2b559a22
2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Failed in AttachStorageDomainVDS method
2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Error code StorageDomainActionError and error message IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)
2012-04-08 11:53:10,738 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)
2012-04-08 11:53:10,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) START, SpmStopVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 6b2f2dd2
2012-04-08 11:53:10,764 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, pool id e3e565d6-8104-11e1-afd2-001cc43ceea0
2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: 6b2f2dd2
2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) Irs placed on server null failed. Proceed Failover
2012-04-08 11:53:11,850 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-49) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool
2012-04-08 11:53:11,881 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, storage pool Default
2012-04-08 11:53:11,886 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, prevId -1, LVER 8
2012-04-08 11:53:11,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) START, SpmStartVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=-1, prevLVER=8, storagePoolFormatType=V1, recoveryMode=Manual, SCSIFencing=false), log id: 382e702
2012-04-08 11:53:11,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) spmStart polling started: taskId = f65c40e4-956f-4c01-ab3f-5c43f2853806
Vdsm.log:
Thread-38311::INFO::2012-04-08 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 10}, 'dominfo': {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', 'diskfree': '490079256576', 'alerts': [], 'disktotal': '516524343296'}}}
Thread-38312::DEBUG::2012-04-08 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38312::INFO::2012-04-08 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])
Thread-38312::DEBUG::2012-04-08 11:54:32,953::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state preparing
Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server
Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]
Thread-38312::DEBUG::2012-04-08 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd None)
Thread-38312::DEBUG::2012-04-08 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0
Thread-38312::DEBUG::2012-04-08 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd None)
Thread-38312::DEBUG::2012-04-08 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0
Thread-38312::DEBUG::2012-04-08 11:54:33,022::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38312::DEBUG::2012-04-08 11:54:33,023::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> state finished
Thread-38312::DEBUG::2012-04-08 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38312::DEBUG::2012-04-08 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38312::DEBUG::2012-04-08 11:54:33,024::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting False
Thread-38312::INFO::2012-04-08 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38313::DEBUG::2012-04-08 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38313::INFO::2012-04-08 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])
Thread-38313::DEBUG::2012-04-08 11:54:33,038::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state preparing
Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server
Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]
Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex
Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> state finished
Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38313::DEBUG::2012-04-08 11:54:33,051::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting False
Thread-38313::INFO::2012-04-08 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}
Thread-38314::DEBUG::2012-04-08 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38314::INFO::2012-04-08 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-38314::DEBUG::2012-04-08 11:54:33,063::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state preparing
Thread-38314::DEBUG::2012-04-08 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1072' at 'public_attachStorageDomain'
Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Granted request
Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)
Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting False
Thread-38314::ERROR::2012-04-08 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 876, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/storage/spm.py", line 115, in run
return self.func(*args, **kwargs)
File "/usr/share/vdsm/storage/spm.py", line 1073, in public_attachStorageDomain
vars.task.getExclusiveLock(STORAGE, sdUUID)
File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock
self.resOwner.acquire(namespace, resName, resourceManager.LockType.exclusive, timeout)
File "/usr/share/vdsm/storage/resourceManager.py", line 673, in acquire
raise ValueError("Owner %s: acquire: resource %s is already acquired" % (str(self), fullName))
ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: 2b306544-6fcc-443b-9ea6-23ae4b71d422 ('e3e565d6-8104-11e1-afd2-001cc43ceea0', 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task
Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force False)
Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting True
Thread-38314::INFO::2012-04-08 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code 100
Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting True
Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force False
Thread-38314::DEBUG::2012-04-08 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> state aborting
Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy none
Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> state failed
Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}
Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-38314::DEBUG::2012-04-08 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run
result = ctask.prepare(self.func, *args, **kwargs)
File "/usr/share/vdsm/storage/task.py", line 1178, in prepare
raise self.error
ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired
Thread-38315::DEBUG::2012-04-08 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38315::INFO::2012-04-08 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-38315::DEBUG::2012-04-08 11:54:33,086::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state preparing
Thread-38315::DEBUG::2012-04-08 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=SPM spmLver=10 spmId=2
Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}
Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> state finished
Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38315::DEBUG::2012-04-08 11:54:33,088::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting False
Thread-38315::INFO::2012-04-08 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}
Thread-38316::DEBUG::2012-04-08 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38316::INFO::2012-04-08 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()
Thread-38316::DEBUG::2012-04-08 11:54:33,102::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state preparing
Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.
Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}
Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': {}}
Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> state finished
Thread-38316::DEBUG::2012-04-08 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38316::DEBUG::2012-04-08 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38316::DEBUG::2012-04-08 11:54:33,104::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting False
Thread-38316::INFO::2012-04-08 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}
Thread-38317::DEBUG::2012-04-08 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]
Thread-38317::INFO::2012-04-08 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)
Thread-38317::DEBUG::2012-04-08 11:54:33,114::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state preparing
Thread-38317::DEBUG::2012-04-08 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0
Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) Entry.
Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: {}
Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'
Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Granted request
Thread-38317::DEBUG::2012-04-08 11:54:33,116::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)
Thread-38317::DEBUG::2012-04-08 11:54:33,117::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting False
Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'
Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'
Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)
Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted request
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) Entry.
Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) Return
Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/master` is not mounted, skipping
Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: e3e565d6-8104-11e1-afd2-001cc43ceea0
Thread-38317::DEBUG::2012-04-08 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction
Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes
Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=main', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=10', 'POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH=nfs.corp:/share', 'ROLE=Master', 'SDUUID=4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=1d1cd722cad65ebb2edfa75641c30a5b97302f84']
Thread-38317::DEBUG::2012-04-08 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction
Thread-38317::INFO::2012-04-08 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099
Thread-38317::DEBUG::2012-04-08 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd /usr/libexec/vdsm/)
Thread-38317::DEBUG::2012-04-08 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0
Thread-38317::DEBUG::2012-04-08 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock released successfully
Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: None
Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> state finished
Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}
Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'
Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)
Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.
Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.
Thread-38317::DEBUG::2012-04-08 11:54:34,168::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting False
Thread-38317::INFO::2012-04-08 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}
Thread-38318::INFO::2012-04-08 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()
Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state preparing
Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', 'lastCheck': 1333900465.46615, 'valid': True, 'code': 0}}
Thread-38318::DEBUG::2012-04-08 11:54:34,396::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> state finished
Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-38318::DEBUG::2012-04-08 11:54:34,397::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting False
*From:*Rami Vaknin [mailto:rvaknin@redhat.com] *Sent:* Sunday, April 08, 2012 7:02 PM *To:* зоррыч *Subject:* Re: [Users] Failed to attach Storage Domain export to Data Center
On 04/08/2012 03:48 PM, зоррыч wrote:
1. Change the spuuid in the domain's metadata file to be the spuuid of the new datacenter (the metadata can be found in dom_md/ dir somewhere inside the stoarge domain's directories tree), you can take the new spuuid from any host running on the new data center by running: vdsClient -s 0 getConnectedStoragePoolsList
2. Try to attach the storage domain again, this time it will fail due to broken checksum on the metadata file, go to vdsm.log of the spm and look for "checksum", it will tell you what the right checksum should be like, copy it to the metadata file. This time the attachment should work.
Hi.
Thank you for the help you provided me in the mailing list.
I dropped the settings using the ovirt - engine-cleanup
Ovirt reinstalled and tried to restore the virtual machine from the export of storage domain.
If you try an attachment domain ovirt error:
Error while executing action AttachStorageDomainToPool: The Domain is already used by another Data Center.<br/>In order to use it, detach it from the Data Center it is currently attached to.
In the logs:
2012-04-08 08:39:49,898 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: Storage
2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 1985db9c
2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 17f02ea3
2012-04-08 08:39:50,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 17f02ea3
2012-04-08 08:39:50,005 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 08:39:50,008 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 6f4db784
2012-04-08 08:39:50,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 1985db9c
2012-04-08 08:39:50,010 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System
2012-04-08 08:39:50,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 42baf149
2012-04-08 08:39:50,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 6f4db784
2012-04-08 08:39:50,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 42baf149
2012-04-08 08:39:50,037 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: 6f22bcac
2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'
2012-04-08 08:39:53,186 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: 6f22bcac
2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'
2012-04-08 08:39:53,190 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command [id=b717db6a-a582-49ba-92c1-6bb09340789a]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, storageId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38.
I did not deactivate the export domain before clearing the configuration. How can I be in this case? I can manually remove the flag of the domain?
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
--
Thanks,
Rami Vaknin, QE @ Red Hat, TLV, IL.
--
Thanks,
Rami Vaknin, QE @ Red Hat, TLV, IL.
-- Thanks, Rami Vaknin, QE @ Red Hat, TLV, IL. --------------070000070304040100060303 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <font color="#000000"><span style="color: rgb(31, 73, 125);" lang="EN-US">All commands in this both logs you've attached (see example line below) indicate that the sdUUID is still </span><span style="color:#1F497D" lang="EN-US">e3e565d6-8104-11e1-afd2-001cc43ceea0 which is the same as the spUUID</span>. Are you sure your metadata file contains the "SDUUID=b23c7ab6-b1d4-4888-8d4a-adc78e61db38" line and not </font><font color="#000000">the wrong "SDUUID=</font><font color="#000000"><span style="color:#1F497D" lang="EN-US">3e565d6-8104-11e1-afd2-001cc43ceea0"</span></font><font color="#000000"> line?<br> <span style="color:#1F497D" lang="EN-US"><br> Thread-79075::INFO::2012-04-09 06:28:05,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)</span></font><br> <br> On 04/09/2012 01:37 PM, зоррыч wrote: <blockquote cite="mid:012901cd163c$b633a860$229af920$@ru" type="cite"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta name="Generator" content="Microsoft Word 12 (filtered medium)"> <style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} @font-face {font-family:Consolas; panose-1:2 11 6 9 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri","sans-serif"; color:black;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} pre {mso-style-priority:99; mso-style-link:"Стандартный HTML Знак"; margin:0cm; margin-bottom:.0001pt; font-size:10.0pt; font-family:"Courier New"; color:black;} p.MsoAcetate, li.MsoAcetate, div.MsoAcetate {mso-style-priority:99; mso-style-link:"Текст выноски Знак"; margin:0cm; margin-bottom:.0001pt; font-size:8.0pt; font-family:"Tahoma","sans-serif"; color:black;} span.HTML {mso-style-name:"Стандартный HTML Знак"; mso-style-priority:99; mso-style-link:"Стандартный HTML"; font-family:Consolas; color:black;} span.a {mso-style-name:"Текст выноски Знак"; mso-style-priority:99; mso-style-link:"Текст выноски"; font-family:"Tahoma","sans-serif"; color:black;} span.EmailStyle21 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:windowtext;} span.EmailStyle22 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:#1F497D;} span.EmailStyle23 {mso-style-type:personal; font-family:"Calibri","sans-serif"; color:#1F497D;} span.EmailStyle24 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page WordSection1 {size:612.0pt 792.0pt; margin:2.0cm 42.5pt 2.0cm 3.0cm;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext="edit" spidmax="1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext="edit"> <o:idmap v:ext="edit" data="1" /> </o:shapelayout></xml><![endif]--> <div class="WordSection1"> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Vdsm log (spm):<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,189::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::INFO::2012-04-09 06:28:05,190::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,190::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::INFO::2012-04-09 06:28:05,190::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::INFO::2012-04-09 06:28:05,191::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,192::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs 10.1.20.2:/home/nfs4 /tmp/tmp2rKh29' (cwd None)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,228::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,235::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmp2rKh29' (cwd None)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,270::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,271::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,272::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,272::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,272::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::DEBUG::2012-04-09 06:28:05,273::task::495::TaskManager.Task::(_debug) Task 3634fc46-5104-4ebd-8ed6-1d4428beed92: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79073::INFO::2012-04-09 06:28:05,273::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,297::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::INFO::2012-04-09 06:28:05,298::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,298::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::INFO::2012-04-09 06:28:05,298::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::INFO::2012-04-09 06:28:05,298::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,308::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,308::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,309::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,309::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,310::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,310::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,310::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,311::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,311::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,311::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::DEBUG::2012-04-09 06:28:05,311::task::495::TaskManager.Task::(_debug) Task 4cf78074-142b-418d-ae9d-aa1a3afbd0b8: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79074::INFO::2012-04-09 06:28:05,312::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,328::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::INFO::2012-04-09 06:28:05,328::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,329::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,329::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`af876ee5-24f4-43f1-a161-d495903c82b5`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1072' at 'public_attachStorageDomain'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,330::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,330::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,330::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`af876ee5-24f4-43f1-a161-d495903c82b5`::Granted request<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,331::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,331::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::ERROR::2012-04-09 06:28:05,331::task::868::TaskManager.Task::(_setError) Unexpected error<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Traceback (most recent call last):<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 876, in _run<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> return fn(*args, **kargs)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/spm.py", line 115, in run<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> return self.func(*args, **kwargs)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/spm.py", line 1073, in public_attachStorageDomain<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> vars.task.getExclusiveLock(STORAGE, sdUUID)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> self.resOwner.acquire(namespace, resName, resourceManager.LockType.exclusive, timeout)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/resourceManager.py", line 673, in acquire<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> raise ValueError("Owner %s: acquire: resource %s is already acquired" % (str(self), fullName))<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ValueError: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,332::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._run: b0ef8b95-fabe-43e0-944c-eb959cf9930a ('e3e565d6-8104-11e1-afd2-001cc43ceea0', 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,332::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: stopping in state preparing (force False)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,333::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 1 aborting True<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::INFO::2012-04-09 06:28:05,333::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: 'Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code 100<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,333::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: Prepare: aborted: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,333::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: ref 0 aborting True<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,334::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: Task._doAbort: force False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,334::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,334::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state preparing -> state aborting<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,335::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: _aborting: recover policy none<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,335::task::495::TaskManager.Task::(_debug) Task b0ef8b95-fabe-43e0-944c-eb959cf9930a: moving from state aborting -> state failed<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,335::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,335::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,336::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,336::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,336::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::DEBUG::2012-04-09 06:28:05,337::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::ERROR::2012-04-09 06:28:05,337::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79075::ERROR::2012-04-09 06:28:05,337::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> result = ctask.prepare(self.func, *args, **kwargs)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 1178, in prepare<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> raise self.error<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ValueError: Owner b0ef8b95-fabe-43e0-944c-eb959cf9930a: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,349::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::INFO::2012-04-09 06:28:05,349::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,350::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,350::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=SPM spmLver=18 spmId=2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,350::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,350::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,351::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,351::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::DEBUG::2012-04-09 06:28:05,351::task::495::TaskManager.Task::(_debug) Task 13ba6f56-88a9-4bab-a677-11b84d1170dd: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79076::INFO::2012-04-09 06:28:05,352::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 18}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,362::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::INFO::2012-04-09 06:28:05,363::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,363::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,363::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,363::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,364::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: finished: {'allTasksStatus': {}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,364::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,364::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,365::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::DEBUG::2012-04-09 06:28:05,365::task::495::TaskManager.Task::(_debug) Task d87eafd3-24e9-4aaf-ba81-9c52dbfb5a1c: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79077::INFO::2012-04-09 06:28:05,365::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,373::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::INFO::2012-04-09 06:28:05,373::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,374::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,374::spm::547::Storage.SPM::(public_spmStop) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,374::taskManager::105::TaskManager::(getAllTasks) Entry.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,375::taskManager::110::TaskManager::(getAllTasks) Return: {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,375::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`0f0e6998-87a6-40e8-8ded-9f600cb0ca2d`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,375::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,376::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,376::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`0f0e6998-87a6-40e8-8ded-9f600cb0ca2d`::Granted request<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,376::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,377::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 1 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,377::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`eb290e92-2fbb-4e12-bde2-83292aade8e4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,378::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,378::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,378::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`eb290e92-2fbb-4e12-bde2-83292aade8e4`::Granted request<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,379::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,379::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,379::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,380::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,380::taskManager::118::TaskManager::(unloadTasks) Entry.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,380::taskManager::122::TaskManager::(unloadTasks) Return<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,381::spm::201::Storage.SPM::(__cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/master` is not mounted, skipping<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,381::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,381::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,382::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,382::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=main', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=18', 'POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH=nfs.corp:/share', 'ROLE=Master', 'SDUUID=4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=5a3d228ad230c9d61a754c22abaa6b5f11b6c721']<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,397::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::INFO::2012-04-09 06:28:05,398::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:05,398::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd /usr/libexec/vdsm/)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,433::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,434::safelease::87::ClusterLock::(release) Cluster lock released successfully<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,434::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: finished: None<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,434::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,435::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,435::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,435::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,436::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,436::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,436::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::DEBUG::2012-04-09 06:28:06,437::task::495::TaskManager.Task::(_debug) Task 0b1d8a06-7441-4614-ab6a-e5ae6b7f1891: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79078::INFO::2012-04-09 06:28:06,437::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::INFO::2012-04-09 06:28:08,375::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::DEBUG::2012-04-09 06:28:08,375::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state init -> state preparing<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::DEBUG::2012-04-09 06:28:08,376::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: finished: {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00202488899231', 'lastCheck': 1333967286.1603661, 'valid': True, 'code': 0}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::DEBUG::2012-04-09 06:28:08,376::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: moving from state preparing -> state finished<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::DEBUG::2012-04-09 06:28:08,376::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::DEBUG::2012-04-09 06:28:08,377::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::DEBUG::2012-04-09 06:28:08,377::task::495::TaskManager.Task::(_debug) Task 6ef7ea52-d656-4f05-a80d-10026a9cf6a7: ref 0 aborting False<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-79080::INFO::2012-04-09 06:28:08,377::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, Return response: {'status': {'message': 'OK', 'code': 0}, '4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00202488899231', 'lastCheck': 1333967286.1603661, 'valid': True, 'code': 0}}<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">^C<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> <a class="moz-txt-link-abbreviated" href="mailto:users-bounces@ovirt.org">users-bounces@ovirt.org</a> [<a class="moz-txt-link-freetext" href="mailto:users-bounces@ovirt.org">mailto:users-bounces@ovirt.org</a>] <b>On Behalf Of </b>зоррыч<br> <b>Sent:</b> Monday, April 09, 2012 2:20 PM<br> <b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:rvaknin@redhat.com">rvaknin@redhat.com</a><br> <b>Cc:</b> <a class="moz-txt-link-abbreviated" href="mailto:users@ovirt.org">users@ovirt.org</a><br> <b>Subject:</b> Re: [Users] Failed to attach Storage Domain export to Data Center<o:p></o:p></span></p> </div> </div> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">In an attempt to connect ovirt storage domain fail<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">[root@noc-2-synt nfs4]# cat b23c7ab6-b1d4-4888-8d4a-adc78e61db38/dom_md/metadata<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">CLASS=Backup<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">DESCRIPTION=export<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">IOOPTIMEOUTSEC=1<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LEASERETRIES=3<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LEASETIMESEC=5<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LOCKPOLICY=<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LOCKRENEWALINTERVALSEC=5<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">MASTER_VERSION=0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">REMOTE_PATH=10.1.20.2:/home/nfs4<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ROLE=Regular<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">SDUUID=b23c7ab6-b1d4-4888-8d4a-adc78e61db38<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">TYPE=NFS<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">VERSION=0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">_SHA_CKSUM=d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D">logs:</span><span style="color:#1F497D" lang="EN-US"><o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,628 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,658 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-49) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 651832d2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,658 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 69324bf2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,747 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 69324bf2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,748 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,750 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 1b5bf8d2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-49) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 651832d2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,763 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-49) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,765 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-49) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 59f6a186<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,775 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 1b5bf8d2<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,791 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-49) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 59f6a186<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,792 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 58a14aa<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,816 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-50) Failed in AttachStorageDomainVDS method<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,817 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-50) Error code StorageDomainActionError and error message IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,817 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,836 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-50) START, SpmStopVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: fbb0af9<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:38,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-50) SpmStopVDSCommand::Stopping SPM on vds 10.2.20.8, pool id e3e565d6-8104-11e1-afd2-001cc43ceea0<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:39,915 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-50) FINISH, SpmStopVDSCommand, log id: fbb0af9<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:39,915 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) Irs placed on server null failed. Proceed Failover<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:39,923 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-50) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:39,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) hostFromVds::selectedVds - 10.1.20.7, spmStatus Free, storage pool Default<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:39,953 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) starting spm on vds 10.1.20.7, storage pool Default, prevId -1, LVER 13<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:39,955 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-50) START, SpmStartVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=-1, prevLVER=13, storagePoolFormatType=V1, recoveryMode=Manual, SCSIFencing=false), log id: 3eff326c<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-09 06:11:39,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-50) spmStart polling started: taskId = e6db13d3-1c55-4d11-9fbc-8abf50c635ae<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">^C<o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> Rami Vaknin [<a moz-do-not-send="true" href="mailto:rvaknin@redhat.com">mailto:rvaknin@redhat.com</a>] <br> <b>Sent:</b> Monday, April 09, 2012 1:27 PM<br> <b>To:</b> зоррыч<br> <b>Cc:</b> <a moz-do-not-send="true" href="mailto:users@ovirt.org">users@ovirt.org</a><br> <b>Subject:</b> Re: [Users] Failed to attach Storage Domain export to Data Center<o:p></o:p></span></p> </div> </div> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">On 04/08/2012 07:05 PM, зоррыч wrote: <o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">I found the metadata in the file ./dom_md/metadata</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Changed “b23c7ab6-b1d4-4888-8d4a-adc78e61db38” to “e3e565d6-8104-11e1-afd2-001cc43ceea0” (SDUUID)</span><o:p></o:p></p> <p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt;font-family:"Times New Roman","serif""><br> It's wrong, you should change that in POOL_UUID and not in SDUUID (my fault, by spuuid I meant to the POOL_UUID).<br> <br> <o:p></o:p></span></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">The data I gave the command: </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">[root@noc-3-synt ~]# vdsClient -s 0 getConnectedStoragePoolsList</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Found in </span><span lang="EN-US">checksum</span><span style="color:#1F497D" lang="EN-US"> vdsm.log:</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-36836::WARNING::2012-04-08 11:19:47,945::persistentDict::242::Storage.PersistentDict::(refresh) data seal is broken metadata declares `4df87518878fd0d74932917b83aba9672f92dff2` should be `d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59` (lines={'VERSION': '0', 'LEASETIMESEC': '5', 'DESCRIPTION': 'export', 'LOCKPOLICY': '', 'LEASERETRIES': '3', 'SDUUID': 'e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH': '10.1.20.2:/home/nfs4', 'MASTER_VERSION': '0', 'IOOPTIMEOUTSEC': '1', 'ROLE': 'Regular', 'LOCKRENEWALINTERVALSEC': '5', 'POOL_UUID': '801e35d4-7405-11e1-b152-001cc43ceea0', 'TYPE': 'NFS', 'CLASS': 'Backup'})</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">MetaDataSealIsBroken: Meta Data seal is broken (checksum mismatch): 'cksum = 4df87518878fd0d74932917b83aba9672f92dff2, computed_cksum = d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Changed it.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">As a result, the metadata file:</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">[root@noc-2-synt dom_md]# cat metadata</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">CLASS=Backup</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">DESCRIPTION=export</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">IOOPTIMEOUTSEC=1</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LEASERETRIES=3</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LEASETIMESEC=5</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LOCKPOLICY=</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">LOCKRENEWALINTERVALSEC=5</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">MASTER_VERSION=0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">POOL_UUID=801e35d4-7405-11e1-b152-001cc43ceea0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">REMOTE_PATH=10.1.20.2:/home/nfs4</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ROLE=Regular</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">SDUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">TYPE=NFS</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">VERSION=0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">_SHA_CKSUM=d91770b5a0f1956cbfb79a6a59b5291dc5bd2b59</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">attachment to the ovirt fails:</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-35311::INFO::2012-04-08 11:25:54,668::fileSD::244::Storage.StorageDomain::(getStats) sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 [Errno 2] No such file or directory: '/rhev/data-center/mnt/10.1.20.2:_home_nfs4/e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-35311::ERROR::2012-04-08 11:25:54,669::task::868::TaskManager.Task::(_setError) Unexpected error</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">I renamed the folder that contains the data to e3e565d6-8104-11e1-afd2-001cc43ceea0 for b23c7ab6-b1d4-4888-8d4a-adc78e61db38 (SDUUID)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">attachment to the ovirt fails =(</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">messages:</span><o:p></o:p></p> <p class="MsoNormal"><span style="font-size:12.0pt" lang="EN-US">Error while executing action AttachStorageDomainToPool: Error in storage domain action</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">engine.log:</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,557 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-49) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: Storage</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3dbf104d</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 4ac74ee6</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,685 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-50) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 4ac74ee6</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,686 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-50) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 53bd874a</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,690 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-46) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3dbf104d</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,691 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-46) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,693 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 3129fd68</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,714 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-50) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 53bd874a</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-46) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 3129fd68</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,718 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-49) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 2b559a22</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Failed in AttachStorageDomainVDS method</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-5-thread-49) Error code StorageDomainActionError and error message IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,738 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) IrsBroker::Failed::AttachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Error in storage domain action: ('sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0, spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0',)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) START, SpmStopVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0), log id: 6b2f2dd2</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:10,764 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) SpmStopVDSCommand::Stopping SPM on vds 10.1.20.7, pool id e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-5-thread-49) FINISH, SpmStopVDSCommand, log id: 6b2f2dd2</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) Irs placed on server null failed. Proceed Failover</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,850 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-5-thread-49) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: e3e565d6-8104-11e1-afd2-001cc43ceea0 Type: StoragePool</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,881 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) hostFromVds::selectedVds - 10.2.20.8, spmStatus Free, storage pool Default</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,886 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-49) starting spm on vds 10.2.20.8, storage pool Default, prevId -1, LVER 8</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) START, SpmStartVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, prevId=-1, prevLVER=8, storagePoolFormatType=V1, recoveryMode=Manual, SCSIFencing=false), log id: 382e702</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">2012-04-08 11:53:11,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (pool-5-thread-49) spmStart polling started: taskId = f65c40e4-956f-4c01-ab3f-5c43f2853806</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Vdsm.log:</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38311::INFO::2012-04-08 11:54:32,427::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getStoragePoolInfo, Return response: {'status': {'message': 'OK', 'code': 0}, 'info': {'spm_id': 2, 'master_uuid': '4f1199ec-9cdb-4efe-91eb-587faeec2099', 'name': 'Default', 'version': '0', 'domains': '4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'pool_status': 'connected', 'isoprefix': '', 'type': 'NFS', 'master_ver': 1, 'lver': 10}, 'dominfo': {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'status': 'Active', 'diskfree': '490079256576', 'alerts': [], 'disktotal': '516524343296'}}}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,952::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:32,952::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,953::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state init -> state preparing</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::127::Storage.ServerConnection::(validate) Request to validate NFS storage server</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:32,953::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,955::fileUtils::83::Storage.Misc.excCmd::(mount) '/usr/bin/sudo -n /bin/mount -o soft,timeo=600,retrans=6,nosharecache,vers=3 -t nfs 10.1.20.2:/home/nfs4 /tmp/tmplHltd0' (cwd None)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,987::fileUtils::83::Storage.Misc.excCmd::(mount) SUCCESS: <err> = ''; <rc> = 0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:32,994::fileUtils::112::Storage.Misc.excCmd::(umount) '/usr/bin/sudo -n /bin/umount -t nfs -f /tmp/tmplHltd0' (cwd None)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,021::fileUtils::112::Storage.Misc.excCmd::(umount) SUCCESS: <err> = ''; <rc> = 0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,022::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,023::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: moving from state preparing -> state finished</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,023::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,024::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::DEBUG::2012-04-08 11:54:33,024::task::495::TaskManager.Task::(_debug) Task 7899228e-1fc3-4fe0-bbcd-84cb4aa1ef11: ref 0 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38312::INFO::2012-04-08 11:54:33,024::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: validateStorageServerConnection, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,037::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,037::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, args: (domType=1, spUUID=00000000-0000-0000-0000-000000000000, conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}])</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,038::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state init -> state preparing</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::95::Storage.ServerConnection::(connect) Request to connect NFS storage server</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,038::storage_connection::53::Storage.ServerConnection::(__validateConnectionParams) conList=[{'connection': '10.1.20.2:/home/nfs4', 'iqn': '', 'portal': '', 'user': '', 'password': '******', 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f', 'port': ''}]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::547::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,048::lvm::549::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::559::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::561::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,049::lvm::580::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,050::lvm::582::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: finished: {'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,050::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: moving from state preparing -> state finished</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,051::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::DEBUG::2012-04-08 11:54:33,051::task::495::TaskManager.Task::(_debug) Task 83b36131-7378-4f69-841d-b82637362295: ref 0 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38313::INFO::2012-04-08 11:54:33,051::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: connectStorageServer, Return response: {'status': {'message': 'OK', 'code': 0}, 'statuslist': [{'status': 0, 'id': 'aef7de23-f271-45f3-853b-9e7b7be6607f'}]}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,062::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::INFO::2012-04-08 11:54:33,062::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: attachStorageDomain, args: ( sdUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0 spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,063::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state init -> state preparing</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,063::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Request was made in '/usr/share/vdsm/storage/spm.py' line '1072' at 'public_attachStorageDomain'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,064::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`a97b6096-7b1a-4f93-b0c0-322399c2ac3a`::Granted request</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,065::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::ERROR::2012-04-08 11:54:33,065::task::868::TaskManager.Task::(_setError) Unexpected error</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Traceback (most recent call last):</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 876, in _run</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> return fn(*args, **kargs)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/spm.py", line 115, in run</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> return self.func(*args, **kwargs)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/spm.py", line 1073, in public_attachStorageDomain</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> vars.task.getExclusiveLock(STORAGE, sdUUID)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 1311, in getExclusiveLock</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> self.resOwner.acquire(namespace, resName, resourceManager.LockType.exclusive, timeout)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/resourceManager.py", line 673, in acquire</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> raise ValueError("Owner %s: acquire: resource %s is already acquired" % (str(self), fullName))</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._run: 2b306544-6fcc-443b-9ea6-23ae4b71d422 ('e3e565d6-8104-11e1-afd2-001cc43ceea0', 'e3e565d6-8104-11e1-afd2-001cc43ceea0') {} failed - stopping task</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,066::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: stopping in state preparing (force False)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 1 aborting True</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::INFO::2012-04-08 11:54:33,067::task::1171::TaskManager.Task::(prepare) aborting: Task is aborted: 'Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired' - code 100</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Prepare: aborted: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,067::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: ref 0 aborting True</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: Task._doAbort: force False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,068::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,068::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state preparing -> state aborting</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: _aborting: recover policy none</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::task::495::TaskManager.Task::(_debug) Task 2b306544-6fcc-443b-9ea6-23ae4b71d422: moving from state aborting -> state failed</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,069::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,070::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::DEBUG::2012-04-08 11:54:33,071::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::106::Storage.Dispatcher.Protect::(run) Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38314::ERROR::2012-04-08 11:54:33,071::dispatcher::107::Storage.Dispatcher.Protect::(run) Traceback (most recent call last):</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/dispatcher.py", line 96, in run</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> result = ctask.prepare(self.func, *args, **kwargs)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> File "/usr/share/vdsm/storage/task.py", line 1178, in prepare</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> raise self.error</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">ValueError: Owner 2b306544-6fcc-443b-9ea6-23ae4b71d422: acquire: resource Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 is already acquired</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,086::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::INFO::2012-04-08 11:54:33,086::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,086::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state init -> state preparing</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,087::spm::688::Storage.SPM::(public_getSpmStatus) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0: spmStatus=SPM spmLver=10 spmId=2</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: finished: {'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,087::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: moving from state preparing -> state finished</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,088::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::DEBUG::2012-04-08 11:54:33,088::task::495::TaskManager.Task::(_debug) Task 04c6f0a0-6010-4c40-a888-8c59b1e69de3: ref 0 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38315::INFO::2012-04-08 11:54:33,089::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getSpmStatus, Return response: {'status': {'message': 'OK', 'code': 0}, 'spm_st': {'spmId': 2, 'spmStatus': 'SPM', 'spmLver': 10}}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,101::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::INFO::2012-04-08 11:54:33,102::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, args: ()</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,102::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state init -> state preparing</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::92::TaskManager::(getAllTasksStatuses) Entry.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,102::taskManager::97::TaskManager::(getAllTasksStatuses) Return: {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: finished: {'allTasksStatus': {}}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,103::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: moving from state preparing -> state finished</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,103::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,104::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::DEBUG::2012-04-08 11:54:33,104::task::495::TaskManager.Task::(_debug) Task 6714a525-c7c6-4132-90a3-da21b312d7fd: ref 0 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38316::INFO::2012-04-08 11:54:33,104::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: getAllTasksStatuses, Return response: {'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,113::clientIF::239::Storage.Dispatcher.Protect::(wrapper) [10.1.20.2]</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::INFO::2012-04-08 11:54:33,113::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, args: ( spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state init -> state preparing</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::spm::547::Storage.SPM::(public_spmStop) spUUID=e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::105::TaskManager::(getAllTasks) Entry.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,114::taskManager::110::TaskManager::(getAllTasks) Return: {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Request was made in '/usr/share/vdsm/storage/spm.py' line '573' at 'public_spmStop'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,115::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,116::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`417fa7c5-958c-4001-8944-d64df2b44f43`::Granted request</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,116::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: _resourcesAcquired: Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0 (exclusive)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,117::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 1 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::155::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Request was made in '/usr/share/vdsm/storage/spm.py' line '584' at '_stop'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,117::resourceManager::463::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' for lock type 'exclusive'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::505::ResourceManager::(registerResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free. Now locking as 'exclusive' (1 active user)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,118::resourceManager::192::ResourceManager.Request::(grant) ResName=`Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0`ReqID=`b1954479-bdb7-47d3-a42c-b672fe788ca4`::Granted request</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,119::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::118::TaskManager::(unloadTasks) Entry.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,120::taskManager::122::TaskManager::(unloadTasks) Return</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::201::Storage.SPM::(__cleanupMasterMount) master `/rhev/data-center/mnt/blockSD/e5a63624-716e-4bb4-ae60-cd4d7aae9ed2/master` is not mounted, skipping</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,121::spm::290::Storage.SPM::(__cleanupSPM) cleaning up SPM: e3e565d6-8104-11e1-afd2-001cc43ceea0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,121::persistentDict::140::Storage.PersistentDict::(transaction) Starting transaction</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::146::Storage.PersistentDict::(transaction) Flushing changes</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,122::persistentDict::265::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=main', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=4f1199ec-9cdb-4efe-91eb-587faeec2099:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=10', 'POOL_UUID=e3e565d6-8104-11e1-afd2-001cc43ceea0', 'REMOTE_PATH=nfs.corp:/share', 'ROLE=Master', 'SDUUID=4f1199ec-9cdb-4efe-91eb-587faeec2099', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=1d1cd722cad65ebb2edfa75641c30a5b97302f84']</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,128::persistentDict::148::Storage.PersistentDict::(transaction) Finished transaction</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::INFO::2012-04-08 11:54:33,129::safelease::83::ClusterLock::(release) Releasing cluster lock for domain 4f1199ec-9cdb-4efe-91eb-587faeec2099</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:33,129::safelease::84::Storage.Misc.excCmd::(release) '/usr/libexec/vdsm/spmstop.sh 4f1199ec-9cdb-4efe-91eb-587faeec2099' (cwd /usr/libexec/vdsm/)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,164::safelease::84::Storage.Misc.excCmd::(release) SUCCESS: <err> = ''; <rc> = 0</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,165::safelease::87::ClusterLock::(release) Cluster lock released successfully</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: finished: None</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,165::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: moving from state preparing -> state finished</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0': < ResourceRef 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', isValid: 'True' obj: 'None'>}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,166::resourceManager::515::ResourceManager::(releaseResource) Trying to release resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::530::ResourceManager::(releaseResource) Released resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' (0 active users)</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::535::ResourceManager::(releaseResource) Resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0' is free, finding out if anyone is waiting for it.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,167::resourceManager::542::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.e3e565d6-8104-11e1-afd2-001cc43ceea0', Clearing records.</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::DEBUG::2012-04-08 11:54:34,168::task::495::TaskManager.Task::(_debug) Task 6bec12b5-5e5b-49d8-a6d3-e8f2a5ef06bf: ref 0 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38317::INFO::2012-04-08 11:54:34,168::dispatcher::100::Storage.Dispatcher.Protect::(run) Run and protect: spmStop, Return response: {'status': {'message': 'OK', 'code': 0}}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::INFO::2012-04-08 11:54:34,395::dispatcher::94::Storage.Dispatcher.Protect::(run) Run and protect: repoStats, args: ()</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state init -> state preparing</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,395::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: finished: {'4f1199ec-9cdb-4efe-91eb-587faeec2099': {'delay': '0.00149178504944', 'lastCheck': 1333900465.46615, 'valid': True, 'code': 0}}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,396::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: moving from state preparing -> state finished</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::786::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,396::resourceManager::821::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thread-38318::DEBUG::2012-04-08 11:54:34,397::task::495::TaskManager.Task::(_debug) Task 612d6da8-5aa1-4f72-89b4-14c3101b0cec: ref 0 aborting False</span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"> </span><o:p></o:p></p> <div> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext" lang="EN-US">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext" lang="EN-US"> Rami Vaknin [<a moz-do-not-send="true" href="mailto:rvaknin@redhat.com">mailto:rvaknin@redhat.com</a>] <br> <b>Sent:</b> Sunday, April 08, 2012 7:02 PM<br> <b>To:</b> </span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">зоррыч</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext" lang="EN-US"><br> <b>Subject:</b> Re: [Users] Failed to attach Storage Domain export to Data Center</span><o:p></o:p></p> </div> </div> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal" style="margin-bottom:12.0pt">On 04/08/2012 03:48 PM, зоррыч wrote:<br> <br> 1. Change the spuuid in the domain's metadata file to be the spuuid of the new datacenter (the metadata can be found in dom_md/ dir somewhere inside the stoarge domain's directories tree), you can take the new spuuid from any host running on the new data center by running:<br> vdsClient -s 0 getConnectedStoragePoolsList<br> <br> 2. Try to attach the storage domain again, this time it will fail due to broken checksum on the metadata file, go to vdsm.log of the spm and look for "checksum", it will tell you what the right checksum should be like, copy it to the metadata file.<br> This time the attachment should work.<br> <br> <o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">Hi. </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">Thank you for the help you provided me in the mailing list.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">I dropped the settings using the ovirt - engine-cleanup</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">Ovirt reinstalled and tried to restore the virtual machine from the export of storage domain.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">If you try an attachment domain ovirt error:</span><o:p></o:p></p> <p class="MsoNormal"><span style="font-size:12.0pt" lang="EN-US">Error while executing action AttachStorageDomainToPool: The Domain is already used by another Data Center.<br/>In order to use it, detach it from the Data Center it is currently attached to.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">In the logs:</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:49,898 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: b23c7ab6-b1d4-4888-8d4a-adc78e61db38 Type: Storage</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) START, ValidateStorageServerConnectionVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 1985db9c</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:49,925 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) START, ValidateStorageServerConnectionVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 17f02ea3</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-47) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 17f02ea3</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,005 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-47) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,008 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) START, ConnectStorageServerVDSCommand(vdsId = 270e3720-8105-11e1-a3ed-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 6f4db784</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (pool-5-thread-48) FINISH, ValidateStorageServerConnectionVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 1985db9c</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,010 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (pool-5-thread-48) Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: System</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) START, ConnectStorageServerVDSCommand(vdsId = 1bb3653a-8105-11e1-9eda-001cc43ceea0, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: aef7de23-f271-45f3-853b-9e7b7be6607f, connection: 10.1.20.2:/home/nfs4 };]), log id: 42baf149</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-47) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 6f4db784</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (pool-5-thread-48) FINISH, ConnectStorageServerVDSCommand, return: {aef7de23-f271-45f3-853b-9e7b7be6607f=0}, log id: 42baf149</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:50,037 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) START, AttachStorageDomainVDSCommand(storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38), log id: 6f22bcac</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-5-thread-50) IrsBroker::Failed::AttachStorageDomainVDS due to: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,186 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (pool-5-thread-50) FINISH, AttachStorageDomainVDSCommand, log id: 6f22bcac</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,186 ERROR [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: 'domain=b23c7ab6-b1d4-4888-8d4a-adc78e61db38, pool=801e35d4-7405-11e1-b152-001cc43ceea0'</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">2012-04-08 08:39:53,190 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (pool-5-thread-50) Command [id=b717db6a-a582-49ba-92c1-6bb09340789a]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.storage_pool_iso_map; snapshot: storagePoolId = e3e565d6-8104-11e1-afd2-001cc43ceea0, storageId = b23c7ab6-b1d4-4888-8d4a-adc78e61db38.</span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal"><span lang="EN-US">I did not deactivate the export domain before clearing the configuration. How can I be in this case? I can manually remove the flag of the domain?</span><o:p></o:p></p> <p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt"><br> <br> <br> </span><o:p></o:p></p> <pre>_______________________________________________<o:p></o:p></pre> <pre>Users mailing list<o:p></o:p></pre> <pre><a moz-do-not-send="true" href="mailto:Users@ovirt.org">Users@ovirt.org</a><o:p></o:p></pre> <pre><a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a><o:p></o:p></pre> <p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt"><br> <br> <br> </span><o:p></o:p></p> <pre>-- <o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>Thanks,<o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>Rami Vaknin, QE @ Red Hat, TLV, IL.<o:p></o:p></pre> <p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt;font-family:"Times New Roman","serif""><br> <br> <o:p></o:p></span></p> <pre>-- <o:p></o:p></pre> <pre><o:p> </o:p></pre> <pre>Thanks,<o:p></o:p></pre> <pre><o:p> </o:p></pre> <pre>Rami Vaknin, QE @ Red Hat, TLV, IL.<o:p></o:p></pre> </div> </blockquote> <br> <br> <pre class="moz-signature" cols="72">-- Thanks, Rami Vaknin, QE @ Red Hat, TLV, IL. </pre> </body> </html> --------------070000070304040100060303--
participants (2)
-
Rami Vaknin
-
зоррыч