<div dir="ltr"><div>Which version are you using?<br>Can you provide the whole log?<br><br></div>For some reason, it looks like the Vdsm thinks that the Storage Domain is not part of the pool.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 2, 2018 at 11:20 AM, Callum Smith <span dir="ltr"><<a href="mailto:callum@well.ox.ac.uk" target="_blank">callum@well.ox.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;line-break:after-white-space">
State is maintenance for the ISOs storage. I've extracted what is hopefully the relevant bits of the log.
<div>
<div><br>
</div>
<div>VDSM.log (SPM)</div>
<div><br>
</div>
<div>
<div>2018-05-02 09:16:03,455+0100 INFO (ioprocess communication (179084)) [IOProcess] Starting ioprocess (__init__:447)</div>
<div>2018-05-02 09:16:03,456+0100 INFO (ioprocess communication (179091)) [IOProcess] Starting ioprocess (__init__:447)</div>
<div>2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [vdsm.api] FINISH activateStorageDomain error=Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-<wbr>000000000304' from=::ffff:192.168.64.254,<wbr>58968,
flow_id=93433989-8e26-48a9-<wbr>bd3a-2ab95f296c08, task_id=7f21f911-348f-45a3-<wbr>b79c-e3cb11642035 (api:50)</div>
<div>2018-05-02 09:16:03,461+0100 ERROR (jsonrpc/0) [storage.TaskManager.Task] (Task='7f21f911-348f-45a3-<wbr>b79c-e3cb11642035') Unexpected error (task:875)</div>
<div>Traceback (most recent call last):</div>
<div> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/storage/task.py"<wbr>, line 882, in _run</div>
<div> return fn(*args, **kargs)</div>
<div> File "<string>", line 2, in activateStorageDomain</div>
<div> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/common/api.py", line 48, in method</div>
<div> ret = func(*args, **kwargs)</div>
<div> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/storage/hsm.py", line 1256, in activateStorageDomain</div>
<div> pool.activateSD(sdUUID)</div>
<div> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/storage/<wbr>securable.py", line 79, in wrapper</div>
<div> return method(self, *args, **kwargs)</div>
<div> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/storage/sp.py", line 1130, in activateSD</div>
<div> self.validateAttachedDomain(<wbr>dom)</div>
<div> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/storage/<wbr>securable.py", line 79, in wrapper</div>
<div> return method(self, *args, **kwargs)</div>
<div> File "/usr/lib/python2.7/site-<wbr>packages/vdsm/storage/sp.py", line 557, in validateAttachedDomain</div>
<div> raise se.StorageDomainNotInPool(<wbr>self.spUUID, dom.sdUUID)</div>
<div>StorageDomainNotInPool: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-<wbr>000000000304'</div>
<div>2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [storage.TaskManager.Task] (Task='7f21f911-348f-45a3-<wbr>b79c-e3cb11642035') aborting: Task is aborted: "Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-<wbr>000000000304'"
- code 353 (task:1181)</div>
<div>2018-05-02 09:16:03,462+0100 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH activateStorageDomain error=Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-<wbr>000000000304' (dispatcher:82)</div>
</div>
<div><br>
</div>
<div>engine.log</div>
<div><br>
</div>
<div>
<div>2018-05-02 09:16:02,326+01 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ActivateStorageDomainCommand] (default task-20) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] Lock Acquired to object 'EngineLock:{exclusiveLocks='[<wbr>f5914df0-f46c-4cc0-b666-<wbr>c929aa0225ae=STORAGE]',
sharedLocks=''}'</div>
<div>2018-05-02 09:16:02,376+01 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ActivateStorageDomainCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] Running command: ActivateStorageDomainCommand internal:
false. Entities affected : ID: f5914df0-f46c-4cc0-b666-<wbr>c929aa0225ae Type: StorageAction group MANIPULATE_STORAGE_DOMA</div>
<div>IN with role type ADMIN</div>
<div>2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ActivateStorageDomainCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] Lock freed to object 'EngineLock:{exclusiveLocks='[<wbr>f5914df0-f46c-4cc0-b666-<wbr>c929aa0225ae=STORAGE]',
sharedLocks=''}'</div>
<div>2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ActivateStorageDomainCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] ActivateStorage Domain. Before Connect all hosts to pool.
Time: Wed May 02 09:16:02 BST 2018</div>
<div>2018-05-02 09:16:02,407+01 INFO [org.ovirt.engine.core.bll.<wbr>storage.connection.<wbr>ConnectStorageToVdsCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-33456) [40a82b47] Running command: ConnectStorageToVdsCommand internal: true. Entities affected :
ID: aaa00000-0000-0000-0000-<wbr>123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN</div>
<div>2018-05-02 09:16:02,421+01 INFO [org.ovirt.engine.core.<wbr>vdsbroker.vdsbroker.<wbr>ConnectStorageServerVDSCommand<wbr>] (EE-ManagedThreadFactory-<wbr>engine-Thread-33456) [40a82b47] START, ConnectStorageServerVDSCommand<wbr>(HostName = virtA003, StorageServerConnectionManagem<wbr>entVDSParameters:{hostId='<wbr>fe2861fc-2b47-4807-b054-<wbr>470198eda473',
storagePoolId='00000000-0000-<wbr>0000-0000-000000</div>
<div>000000', storageType='NFS', connectionList='[<wbr>StorageServerConnections:{id='<wbr>da392861-aedc-4f1e-97f4-<wbr>6919fb01f1e9', connection='backoffice01.<wbr>cluster:/vm-iso', iqn='null', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]'}), log id: 23ce648f</div>
<div>2018-05-02 09:16:02,446+01 INFO [org.ovirt.engine.core.<wbr>vdsbroker.vdsbroker.<wbr>ConnectStorageServerVDSCommand<wbr>] (EE-ManagedThreadFactory-<wbr>engine-Thread-33456) [40a82b47] FINISH, ConnectStorageServerVDSCommand<wbr>, return: {da392861-aedc-4f1e-97f4-<wbr>6919fb01f1e9=0},
log id: 23ce648f</div>
<div>2018-05-02 09:16:02,450+01 INFO [org.ovirt.engine.core.<wbr>vdsbroker.irsbroker.<wbr>ActivateStorageDomainVDSComman<wbr>d] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] START, ActivateStorageDomainVDSComman<wbr>d( ActivateStorageDomainVDSComman<wbr>dParameters:{storagePoolId='<wbr>5a54bf81-0228-02bc-0358-<wbr>000000000304',
ignoreFailoverLimit='false', stor</div>
<div>ageDomainId='f5914df0-f46c-<wbr>4cc0-b666-c929aa0225ae'}), log id: 5c864594</div>
<div>2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,<wbr>803), VDSM command
ActivateStorageDomainVDS failed: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae, pool=5a5</div>
<div>4bf81-0228-02bc-0358-<wbr>000000000304'</div>
<div>2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.<wbr>vdsbroker.irsbroker.<wbr>ActivateStorageDomainVDSComman<wbr>d] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] Command '<wbr>ActivateStorageDomainVDSComman<wbr>d( ActivateStorageDomainVDSComman<wbr>dParameters:{storagePoolId='<wbr>5a54bf81-0228-02bc-0358-<wbr>000000000304',
ignoreFailoverLimit='false', st</div>
<div>orageDomainId='f5914df0-f46c-<wbr>4cc0-b666-c929aa0225ae'})' execution failed: IRSGenericException: IRSErrorException: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-<wbr>000000000304'</div>
<div>2018-05-02 09:16:02,635+01 INFO [org.ovirt.engine.core.<wbr>vdsbroker.irsbroker.<wbr>ActivateStorageDomainVDSComman<wbr>d] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] FINISH, ActivateStorageDomainVDSComman<wbr>d, log id:
5c864594</div>
<div>2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ActivateStorageDomainCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] Command 'org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ActivateStorageDomainCommand'
failed: EngineException: org.ovirt.engine.core.<wbr>vdsbroker.irsbroker.<wbr>IrsOperationFailedNoFailove</div>
<div>rException: IRSGenericException: IRSErrorException: Storage domain not in pool: u'domain=f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae, pool=5a54bf81-0228-02bc-0358-<wbr>000000000304' (Failed with error StorageDomainNotInPool and code 353)</div>
<div>2018-05-02 09:16:02,636+01 INFO [org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>ActivateStorageDomainCommand] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] Command [id=22b0f3c1-9a09-4e26-8096-<wbr>d83465c8f4ee]: Compensating
CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.<wbr>businessentities.<wbr>StoragePoolIsoMap; snapshot: EntityStatus</div>
<div>Snapshot:{id='<wbr>StoragePoolIsoMapId:{<wbr>storagePoolId='5a54bf81-0228-<wbr>02bc-0358-000000000304', storageId='f5914df0-f46c-4cc0-<wbr>b666-c929aa0225ae'}', status='Maintenance'}.</div>
<div>2018-05-02 09:16:02,660+01 ERROR [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (EE-ManagedThreadFactory-<wbr>engine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr>2ab95f296c08] EVENT_ID: USER_ACTIVATE_STORAGE_DOMAIN_<wbr>FAILED(967), Failed
to activate Storage Domain VMISOs (Data Center Default) by admin@internal-authz</div>
</div>
<div><span class=""><br>
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="word-wrap:break-word">
<div style="color:rgb(0,0,0);font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Regards,</div>
<div style="color:rgb(0,0,0);font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Callum</div>
<div style="color:rgb(0,0,0);font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<br>
</div>
<div style="color:rgb(0,0,0);font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
--<br>
<br>
Callum Smith</div>
<div style="color:rgb(0,0,0);font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Research Computing Core</div>
<div style="color:rgb(0,0,0);font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Wellcome Trust Centre for Human Genetics<br>
University of Oxford<br>
e. <a href="mailto:callum@well.ox.ac.uk" target="_blank">callum@well.ox.ac.uk</a></div>
</div>
</div>
</div>
</span><div><div class="h5"><div><br>
<blockquote type="cite">
<div>On 2 May 2018, at 08:44, Fred Rolland <<a href="mailto:frolland@redhat.com" target="_blank">frolland@redhat.com</a>> wrote:</div>
<br class="m_-2093273250354586386Apple-interchange-newline">
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>Hi,<br>
<br>
</div>
Can you provide logs from engine and Vdsm(SPM)?<br>
</div>
What is the state now?<br>
<br>
</div>
Thanks,<br>
</div>
Fred<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, May 1, 2018 at 4:11 PM, Callum Smith <span dir="ltr">
<<a href="mailto:callum@well.ox.ac.uk" target="_blank">callum@well.ox.ac.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;line-break:after-white-space">Dear All,
<div><br>
</div>
<div>It appears that clicking "detach" on the ISO storage domain is a really bad idea. This has gotten half way through the procedure and now can't be recovered from. Is there any advice for re-attaching the ISO storage domain manually? An NFS mount
didn't add it back to the "pool" unfortunately.</div>
<div><br>
</div>
<div>On a separate note, is it possible to migrate this storage to a new location? And if so how.</div>
<div><br>
<div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="word-wrap:break-word">
<div style="font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Regards,</div>
<div style="font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Callum</div>
<div style="font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<br>
</div>
<div style="font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
--<br>
<br>
Callum Smith</div>
<div style="font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Research Computing Core</div>
<div style="font-family:Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
Wellcome Trust Centre for Human Genetics<br>
University of Oxford<br>
e. <a href="mailto:callum@well.ox.ac.uk" target="_blank">callum@well.ox.ac.uk</a></div>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div></div></div>
</div>
</div>
</blockquote></div><br></div>