Which version are you using? Can you provide the whole log? For some reason, it looks like the Vdsm thinks that the Storage Domain is not part of the pool. On Wed, May 2, 2018 at 11:20 AM, Callum Smith <callum@well.ox.ac.uk> wrote:
State is maintenance for the ISOs storage. I've extracted what is hopefully the relevant bits of the log.
VDSM.log (SPM)
2018-05-02 09:16:03,455+0100 INFO (ioprocess communication (179084)) [IOProcess] Starting ioprocess (__init__:447) 2018-05-02 09:16:03,456+0100 INFO (ioprocess communication (179091)) [IOProcess] Starting ioprocess (__init__:447) 2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [vdsm.api] FINISH activateStorageDomain error=Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-000000000304' from=::ffff:192.168.64.254,58968, flow_id=93433989-8e26-48a9-bd3a-2ab95f296c08, task_id=7f21f911-348f-45a3-b79c-e3cb11642035 (api:50) 2018-05-02 09:16:03,461+0100 ERROR (jsonrpc/0) [storage.TaskManager.Task] (Task='7f21f911-348f-45a3-b79c-e3cb11642035') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in activateStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1256, in activateStorageDomain pool.activateSD(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1130, in activateSD self.validateAttachedDomain(dom) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 557, in validateAttachedDomain raise se.StorageDomainNotInPool(self.spUUID, dom.sdUUID) StorageDomainNotInPool: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-000000000304' 2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [storage.TaskManager.Task] (Task='7f21f911-348f-45a3-b79c-e3cb11642035') aborting: Task is aborted: "Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-000000000304'" - code 353 (task:1181) 2018-05-02 09:16:03,462+0100 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH activateStorageDomain error=Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-000000000304' (dispatcher:82)
engine.log
2018-05-02 09:16:02,326+01 INFO [org.ovirt.engine.core.bll. storage.domain.ActivateStorageDomainCommand] (default task-20) [93433989-8e26-48a9-bd3a-2ab95f296c08] Lock Acquired to object 'EngineLock:{exclusiveLocks='[f5914df0-f46c-4cc0-b666-c929aa0225ae=STORAGE]', sharedLocks=''}' 2018-05-02 09:16:02,376+01 INFO [org.ovirt.engine.core.bll. storage.domain.ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: f5914df0-f46c-4cc0-b666-c929aa0225ae Type: StorageAction group MANIPULATE_STORAGE_DOMA IN with role type ADMIN 2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll. storage.domain.ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] Lock freed to object 'EngineLock:{exclusiveLocks='[f5914df0-f46c-4cc0-b666-c929aa0225ae=STORAGE]', sharedLocks=''}' 2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll. storage.domain.ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] ActivateStorage Domain. Before Connect all hosts to pool. Time: Wed May 02 09:16:02 BST 2018 2018-05-02 09:16:02,407+01 INFO [org.ovirt.engine.core.bll. storage.connection.ConnectStorageToVdsCommand] (EE-ManagedThreadFactory-engine-Thread-33456) [40a82b47] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN 2018-05-02 09:16:02,421+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33456) [40a82b47] START, ConnectStorageServerVDSCommand(HostName = virtA003, StorageServerConnectionManagementVDSParameters:{hostId=' fe2861fc-2b47-4807-b054-470198eda473', storagePoolId='00000000-0000- 0000-0000-000000 000000', storageType='NFS', connectionList='[ StorageServerConnections:{id='da392861-aedc-4f1e-97f4-6919fb01f1e9', connection='backoffice01.cluster:/vm-iso', iqn='null', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]'}), log id: 23ce648f 2018-05-02 09:16:02,446+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33456) [40a82b47] FINISH, ConnectStorageServerVDSCommand, return: {da392861-aedc-4f1e-97f4-6919fb01f1e9=0}, log id: 23ce648f 2018-05-02 09:16:02,450+01 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] START, ActivateStorageDomainVDSCommand( ActivateStorageDomainVDSComman dParameters:{storagePoolId='5a54bf81-0228-02bc-0358-000000000304', ignoreFailoverLimit='false', stor ageDomainId='f5914df0-f46c-4cc0-b666-c929aa0225ae'}), log id: 5c864594 2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command ActivateStorageDomainVDS failed: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae, pool=5a5 4bf81-0228-02bc-0358-000000000304' 2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core. vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] Command 'ActivateStorageDomainVDSCommand( ActivateStorageDomainVDSComman dParameters:{storagePoolId='5a54bf81-0228-02bc-0358-000000000304', ignoreFailoverLimit='false', st orageDomainId='f5914df0-f46c-4cc0-b666-c929aa0225ae'})' execution failed: IRSGenericException: IRSErrorException: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-000000000304' 2018-05-02 09:16:02,635+01 INFO [org.ovirt.engine.core. vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] FINISH, ActivateStorageDomainVDSCommand, log id: 5c864594 2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.bll. storage.domain.ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] Command 'org.ovirt.engine.core.bll. storage.domain.ActivateStorageDomainCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailove rException: IRSGenericException: IRSErrorException: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-000000000304' (Failed with error StorageDomainNotInPool and code 353) 2018-05-02 09:16:02,636+01 INFO [org.ovirt.engine.core.bll. storage.domain.ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] Command [id=22b0f3c1-9a09-4e26-8096-d83465c8f4ee]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatus Snapshot:{id='StoragePoolIsoMapId:{storagePoolId='5a54bf81-0228-02bc-0358-000000000304', storageId='f5914df0-f46c-4cc0-b666-c929aa0225ae'}', status='Maintenance'}. 2018-05-02 09:16:02,660+01 ERROR [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID: USER_ACTIVATE_STORAGE_DOMAIN_FAILED(967), Failed to activate Storage Domain VMISOs (Data Center Default) by admin@internal-authz
Regards, Callum
--
Callum Smith Research Computing Core Wellcome Trust Centre for Human Genetics University of Oxford e. callum@well.ox.ac.uk
On 2 May 2018, at 08:44, Fred Rolland <frolland@redhat.com> wrote:
Hi,
Can you provide logs from engine and Vdsm(SPM)? What is the state now?
Thanks, Fred
On Tue, May 1, 2018 at 4:11 PM, Callum Smith <callum@well.ox.ac.uk> wrote:
Dear All,
It appears that clicking "detach" on the ISO storage domain is a really bad idea. This has gotten half way through the procedure and now can't be recovered from. Is there any advice for re-attaching the ISO storage domain manually? An NFS mount didn't add it back to the "pool" unfortunately.
On a separate note, is it possible to migrate this storage to a new location? And if so how.
Regards, Callum
--
Callum Smith Research Computing Core Wellcome Trust Centre for Human Genetics University of Oxford e. callum@well.ox.ac.uk
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users