12:43 p.m.
Which version are you using?
Can you provide the whole log?
For some reason, it looks like the Vdsm thinks that the Storage Domain is
not part of the pool.
On Wed, May 2, 2018 at 11:20 AM, Callum Smith <callum(a)well.ox.ac.uk> wrote:
State is maintenance for the ISOs storage. I've extracted what
is
hopefully the relevant bits of the log.
VDSM.log (SPM)
2018-05-02 09:16:03,455+0100 INFO (ioprocess communication (179084))
[IOProcess] Starting ioprocess (__init__:447)
2018-05-02 09:16:03,456+0100 INFO (ioprocess communication (179091))
[IOProcess] Starting ioprocess (__init__:447)
2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [vdsm.api] FINISH
activateStorageDomain error=Storage domain not in pool:
u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae,
pool=5a54bf81-0228-02bc-0358-000000000304' from=::ffff:192.168.64.254,58968,
flow_id=93433989-8e26-48a9-bd3a-2ab95f296c08,
task_id=7f21f911-348f-45a3-b79c-e3cb11642035
(api:50)
2018-05-02 09:16:03,461+0100 ERROR (jsonrpc/0) [storage.TaskManager.Task]
(Task='7f21f911-348f-45a3-b79c-e3cb11642035') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
in _run
return fn(*args, **kargs)
File "<string>", line 2, in activateStorageDomain
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in
method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1256,
in activateStorageDomain
pool.activateSD(sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line
79, in wrapper
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1130,
in activateSD
self.validateAttachedDomain(dom)
File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line
79, in wrapper
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 557,
in validateAttachedDomain
raise se.StorageDomainNotInPool(self.spUUID, dom.sdUUID)
StorageDomainNotInPool: Storage domain not in pool:
u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae,
pool=5a54bf81-0228-02bc-0358-000000000304'
2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [storage.TaskManager.Task]
(Task='7f21f911-348f-45a3-b79c-e3cb11642035') aborting: Task is aborted:
"Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae,
pool=5a54bf81-0228-02bc-0358-000000000304'" - code 353 (task:1181)
2018-05-02 09:16:03,462+0100 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH
activateStorageDomain error=Storage domain not in pool:
u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae,
pool=5a54bf81-0228-02bc-0358-000000000304' (dispatcher:82)
engine.log
2018-05-02 09:16:02,326+01 INFO [org.ovirt.engine.core.bll.
storage.domain.ActivateStorageDomainCommand] (default task-20)
[93433989-8e26-48a9-bd3a-2ab95f296c08] Lock Acquired to object
'EngineLock:{exclusiveLocks='[f5914df0-f46c-4cc0-b666-c929aa0225ae=STORAGE]',
sharedLocks=''}'
2018-05-02 09:16:02,376+01 INFO [org.ovirt.engine.core.bll.
storage.domain.ActivateStorageDomainCommand]
(EE-ManagedThreadFactory-engine-Thread-33455)
[93433989-8e26-48a9-bd3a-2ab95f296c08] Running command:
ActivateStorageDomainCommand internal: false. Entities affected : ID:
f5914df0-f46c-4cc0-b666-c929aa0225ae Type: StorageAction group
MANIPULATE_STORAGE_DOMA
IN with role type ADMIN
2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll.
storage.domain.ActivateStorageDomainCommand]
(EE-ManagedThreadFactory-engine-Thread-33455)
[93433989-8e26-48a9-bd3a-2ab95f296c08] Lock freed to object
'EngineLock:{exclusiveLocks='[f5914df0-f46c-4cc0-b666-c929aa0225ae=STORAGE]',
sharedLocks=''}'
2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll.
storage.domain.ActivateStorageDomainCommand]
(EE-ManagedThreadFactory-engine-Thread-33455)
[93433989-8e26-48a9-bd3a-2ab95f296c08] ActivateStorage Domain. Before
Connect all hosts to pool. Time: Wed May 02 09:16:02 BST 2018
2018-05-02 09:16:02,407+01 INFO [org.ovirt.engine.core.bll.
storage.connection.ConnectStorageToVdsCommand]
(EE-ManagedThreadFactory-engine-Thread-33456)
[40a82b47] Running command: ConnectStorageToVdsCommand internal: true.
Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type:
SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN
2018-05-02 09:16:02,421+01 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-33456) [40a82b47] START,
ConnectStorageServerVDSCommand(HostName = virtA003,
StorageServerConnectionManagementVDSParameters:{hostId='
fe2861fc-2b47-4807-b054-470198eda473', storagePoolId='00000000-0000-
0000-0000-000000
000000', storageType='NFS', connectionList='[
StorageServerConnections:{id='da392861-aedc-4f1e-97f4-6919fb01f1e9',
connection='backoffice01.cluster:/vm-iso', iqn='null',
vfsType='null',
mountOptions='null', nfsVersion='null', nfsRetrans='null',
nfsTimeo='null',
iface='null', netIfaceName='null'}]'}), log id: 23ce648f
2018-05-02 09:16:02,446+01 INFO [org.ovirt.engine.core.
vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-33456) [40a82b47] FINISH,
ConnectStorageServerVDSCommand, return: {da392861-aedc-4f1e-97f4-6919fb01f1e9=0},
log id: 23ce648f
2018-05-02 09:16:02,450+01 INFO [org.ovirt.engine.core.
vdsbroker.irsbroker.ActivateStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08]
START, ActivateStorageDomainVDSCommand( ActivateStorageDomainVDSComman
dParameters:{storagePoolId='5a54bf81-0228-02bc-0358-000000000304',
ignoreFailoverLimit='false', stor
ageDomainId='f5914df0-f46c-4cc0-b666-c929aa0225ae'}), log id: 5c864594
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.dal.
dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-33455)
[93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID:
IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command ActivateStorageDomainVDS
failed: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae,
pool=5a5
4bf81-0228-02bc-0358-000000000304'
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.
vdsbroker.irsbroker.ActivateStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08]
Command 'ActivateStorageDomainVDSCommand( ActivateStorageDomainVDSComman
dParameters:{storagePoolId='5a54bf81-0228-02bc-0358-000000000304',
ignoreFailoverLimit='false', st
orageDomainId='f5914df0-f46c-4cc0-b666-c929aa0225ae'})' execution failed:
IRSGenericException: IRSErrorException: Storage domain not in pool:
u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae,
pool=5a54bf81-0228-02bc-0358-000000000304'
2018-05-02 09:16:02,635+01 INFO [org.ovirt.engine.core.
vdsbroker.irsbroker.ActivateStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-33455) [93433989-8e26-48a9-bd3a-2ab95f296c08]
FINISH, ActivateStorageDomainVDSCommand, log id: 5c864594
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.bll.
storage.domain.ActivateStorageDomainCommand]
(EE-ManagedThreadFactory-engine-Thread-33455)
[93433989-8e26-48a9-bd3a-2ab95f296c08] Command 'org.ovirt.engine.core.bll.
storage.domain.ActivateStorageDomainCommand' failed: EngineException:
org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailove
rException: IRSGenericException: IRSErrorException: Storage domain not in
pool: u'domain=f5914df0-f46c-4cc0-b666-c929aa0225ae,
pool=5a54bf81-0228-02bc-0358-000000000304' (Failed with error
StorageDomainNotInPool and code 353)
2018-05-02 09:16:02,636+01 INFO [org.ovirt.engine.core.bll.
storage.domain.ActivateStorageDomainCommand]
(EE-ManagedThreadFactory-engine-Thread-33455)
[93433989-8e26-48a9-bd3a-2ab95f296c08] Command
[id=22b0f3c1-9a09-4e26-8096-d83465c8f4ee]: Compensating
CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap;
snapshot: EntityStatus
Snapshot:{id='StoragePoolIsoMapId:{storagePoolId='5a54bf81-0228-02bc-0358-000000000304',
storageId='f5914df0-f46c-4cc0-b666-c929aa0225ae'}',
status='Maintenance'}.
2018-05-02 09:16:02,660+01 ERROR [org.ovirt.engine.core.dal.
dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-33455)
[93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID:
USER_ACTIVATE_STORAGE_DOMAIN_FAILED(967), Failed to activate Storage
Domain VMISOs (Data Center Default) by admin@internal-authz
Regards,
Callum
--
Callum Smith
Research Computing Core
Wellcome Trust Centre for Human Genetics
University of Oxford
e. callum(a)well.ox.ac.uk
On 2 May 2018, at 08:44, Fred Rolland <frolland(a)redhat.com> wrote:
Hi,
Can you provide logs from engine and Vdsm(SPM)?
What is the state now?
Thanks,
Fred
On Tue, May 1, 2018 at 4:11 PM, Callum Smith <callum(a)well.ox.ac.uk> wrote:
> Dear All,
>
> It appears that clicking "detach" on the ISO storage domain is a really
> bad idea. This has gotten half way through the procedure and now can't be
> recovered from. Is there any advice for re-attaching the ISO storage domain
> manually? An NFS mount didn't add it back to the "pool" unfortunately.
>
> On a separate note, is it possible to migrate this storage to a new
> location? And if so how.
>
> Regards,
> Callum
>
> --
>
> Callum Smith
> Research Computing Core
> Wellcome Trust Centre for Human Genetics
> University of Oxford
> e. callum(a)well.ox.ac.uk
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>