Re: [ovirt-users] Re-attaching ISOs and moving ISOs storage

2 May 2018

      --_000_288C5B8ED9BC48E5A130EA2F9FA9E942welloxacuk_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

This is on 4.2.0.2-1, I've linked the main logs to dropbox simply because t=
hey're big, full of noise right now.
https://www.dropbox.com/s/f8q3m5amro2a1b2/engine.log?dl=3D0
https://www.dropbox.com/s/uods85jk65halo3/vdsm.log?dl=3D0

Regards,
Callum

--

Callum Smith
Research Computing Core
Wellcome Trust Centre for Human Genetics
University of Oxford
e. callum@well.ox.ac.uk<mailto:callum@well.ox.ac.uk>

On 2 May 2018, at 10:43, Fred Rolland <frolland@redhat.com<mailto:frolland@=
redhat.com>> wrote:

Which version are you using?
Can you provide the whole log?

For some reason, it looks like the Vdsm thinks that the Storage Domain is n=
ot part of the pool.

On Wed, May 2, 2018 at 11:20 AM, Callum Smith <callum@well.ox.ac.uk<mailto:=
callum@well.ox.ac.uk>> wrote:
State is maintenance for the ISOs storage. I've extracted what is hopefully=
 the relevant bits of the log.

VDSM.log (SPM)

2018-05-02 09:16:03,455+0100 INFO  (ioprocess communication (179084)) [IOPr=
ocess] Starting ioprocess (__init__:447)
2018-05-02 09:16:03,456+0100 INFO  (ioprocess communication (179091)) [IOPr=
ocess] Starting ioprocess (__init__:447)
2018-05-02 09:16:03,461+0100 INFO  (jsonrpc/0) [vdsm.api] FINISH activateSt=
orageDomain error=3DStorage domain not in pool: u'domain=3Df5914df0-f46c-4c=
c0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-000000000304' from=3D:=
:ffff:192.168.64.254,58968, flow_id=3D93433989-8e26-48a9-bd3a-2ab95f296c08,=
 task_id=3D7f21f911-348f-45a3-b79c-e3cb11642035 (api:50)
2018-05-02 09:16:03,461+0100 ERROR (jsonrpc/0) [storage.TaskManager.Task] (=
Task=3D'7f21f911-348f-45a3-b79c-e3cb11642035') Unexpected error (task:875)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, i=
n _run
    return fn(*args, **kargs)
  File "<string>", line 2, in activateStorageDomain
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in m=
ethod
    ret =3D func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1256, i=
n activateStorageDomain
    pool.activateSD(sdUUID)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 7=
9, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1130, in=
 activateSD
    self.validateAttachedDomain(dom)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 7=
9, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 557, in =
validateAttachedDomain
    raise se.StorageDomainNotInPool(self.spUUID, dom.sdUUID)
StorageDomainNotInPool: Storage domain not in pool: u'domain=3Df5914df0-f46=
c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-000000000304'
2018-05-02 09:16:03,461+0100 INFO  (jsonrpc/0) [storage.TaskManager.Task] (=
Task=3D'7f21f911-348f-45a3-b79c-e3cb11642035') aborting: Task is aborted: "=
Storage domain not in pool: u'domain=3Df5914df0-f46c-4cc0-b666-c929aa0225ae=
, pool=3D5a54bf81-0228-02bc-0358-000000000304'" - code 353 (task:1181)
2018-05-02 09:16:03,462+0100 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH =
activateStorageDomain error=3DStorage domain not in pool: u'domain=3Df5914d=
f0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-000000000304=
' (dispatcher:82)

engine.log

2018-05-02 09:16:02,326+01 INFO  [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (default task-20) [93433989-8e26-48a9-bd3a-2a=
b95f296c08] Lock Acquired to object 'EngineLock:{exclusiveLocks=3D'[f5914df=
0-f46c-4cc0-b666-c929aa0225ae=3DSTORAGE]', sharedLocks=3D''}'
2018-05-02 09:16:02,376+01 INFO  [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
 [93433989-8e26-48a9-bd3a-2ab95f296c08] Running command: ActivateStorageDom=
ainCommand internal: false. Entities affected :  ID: f5914df0-f46c-4cc0-b66=
6-c929aa0225ae Type: StorageAction group MANIPULATE_STORAGE_DOMA
IN with role type ADMIN
2018-05-02 09:16:02,385+01 INFO  [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
 [93433989-8e26-48a9-bd3a-2ab95f296c08] Lock freed to object 'EngineLock:{e=
xclusiveLocks=3D'[f5914df0-f46c-4cc0-b666-c929aa0225ae=3DSTORAGE]', sharedL=
ocks=3D''}'
2018-05-02 09:16:02,385+01 INFO  [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
 [93433989-8e26-48a9-bd3a-2ab95f296c08] ActivateStorage Domain. Before Conn=
ect all hosts to pool. Time: Wed May 02 09:16:02 BST 2018
2018-05-02 09:16:02,407+01 INFO  [org.ovirt.engine.core.bll.storage.connect=
ion.ConnectStorageToVdsCommand] (EE-ManagedThreadFactory-engine-Thread-3345=
6) [40a82b47] Running command: ConnectStorageToVdsCommand internal: true. E=
ntities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAc=
tion group CREATE_STORAGE_DOMAIN with role type ADMIN
2018-05-02 09:16:02,421+01 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-334=
56) [40a82b47] START, ConnectStorageServerVDSCommand(HostName =3D virtA003,=
 StorageServerConnectionManagementVDSParameters:{hostId=3D'fe2861fc-2b47-48=
07-b054-470198eda473', storagePoolId=3D'00000000-0000-0000-0000-000000
000000', storageType=3D'NFS', connectionList=3D'[StorageServerConnections:{=
id=3D'da392861-aedc-4f1e-97f4-6919fb01f1e9', connection=3D'backoffice01.clu=
ster:/vm-iso', iqn=3D'null', vfsType=3D'null', mountOptions=3D'null', nfsVe=
rsion=3D'null', nfsRetrans=3D'null', nfsTimeo=3D'null', iface=3D'null', net=
IfaceName=3D'null'}]'}), log id: 23ce648f
2018-05-02 09:16:02,446+01 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-334=
56) [40a82b47] FINISH, ConnectStorageServerVDSCommand, return: {da392861-ae=
dc-4f1e-97f4-6919fb01f1e9=3D0}, log id: 23ce648f
2018-05-02 09:16:02,450+01 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker=
.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33=
455) [93433989-8e26-48a9-bd3a-2ab95f296c08] START, ActivateStorageDomainVDS=
Command( ActivateStorageDomainVDSCommandParameters:{storagePoolId=3D'5a54bf=
81-0228-02bc-0358-000000000304', ignoreFailoverLimit=3D'false', stor
ageDomainId=3D'f5914df0-f46c-4cc0-b666-c929aa0225ae'}), log id: 5c864594
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-33455) =
[93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID: IRS_BROKER_COMMAND_FAILURE=
(10,803), VDSM command ActivateStorageDomainVDS failed: Storage domain not =
in pool: u'domain=3Df5914df0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a5
4bf81-0228-02bc-0358-000000000304'
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker=
.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33=
455) [93433989-8e26-48a9-bd3a-2ab95f296c08] Command 'ActivateStorageDomainV=
DSCommand( ActivateStorageDomainVDSCommandParameters:{storagePoolId=3D'5a54=
bf81-0228-02bc-0358-000000000304', ignoreFailoverLimit=3D'false', st
orageDomainId=3D'f5914df0-f46c-4cc0-b666-c929aa0225ae'})' execution failed:=
 IRSGenericException: IRSErrorException: Storage domain not in pool: u'doma=
in=3Df5914df0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-0=
00000000304'
2018-05-02 09:16:02,635+01 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker=
.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33=
455) [93433989-8e26-48a9-bd3a-2ab95f296c08] FINISH, ActivateStorageDomainVD=
SCommand, log id: 5c864594
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
 [93433989-8e26-48a9-bd3a-2ab95f296c08] Command 'org.ovirt.engine.core.bll.=
storage.domain.ActivateStorageDomainCommand' failed: EngineException: org.o=
virt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailove
rException: IRSGenericException: IRSErrorException: Storage domain not in p=
ool: u'domain=3Df5914df0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-=
02bc-0358-000000000304' (Failed with error StorageDomainNotInPool and code =
353)
2018-05-02 09:16:02,636+01 INFO  [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
 [93433989-8e26-48a9-bd3a-2ab95f296c08] Command [id=3D22b0f3c1-9a09-4e26-80=
96-d83465c8f4ee]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core=
.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatus
Snapshot:{id=3D'StoragePoolIsoMapId:{storagePoolId=3D'5a54bf81-0228-02bc-03=
58-000000000304', storageId=3D'f5914df0-f46c-4cc0-b666-c929aa0225ae'}', sta=
tus=3D'Maintenance'}.
2018-05-02 09:16:02,660+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-33455) =
[93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID: USER_ACTIVATE_STORAGE_DOMA=
IN_FAILED(967), Failed to activate Storage Domain VMISOs (Data Center Defau=
lt) by admin@internal-authz

Regards,
Callum

--

Callum Smith
Research Computing Core
Wellcome Trust Centre for Human Genetics
University of Oxford
e. callum@well.ox.ac.uk<mailto:callum@well.ox.ac.uk>

On 2 May 2018, at 08:44, Fred Rolland <frolland@redhat.com<mailto:frolland@=
redhat.com>> wrote:

Hi,

Can you provide logs from engine and Vdsm(SPM)?
What is the state now?

Thanks,
Fred

On Tue, May 1, 2018 at 4:11 PM, Callum Smith <callum@well.ox.ac.uk<mailto:c=
allum@well.ox.ac.uk>> wrote:
Dear All,

It appears that clicking "detach" on the ISO storage domain is a really bad=
 idea. This has gotten half way through the procedure and now can't be reco=
vered from. Is there any advice for re-attaching the ISO storage domain man=
ually? An NFS mount didn't add it back to the "pool" unfortunately.

On a separate note, is it possible to migrate this storage to a new locatio=
n? And if so how.

Regards,
Callum

--

Callum Smith
Research Computing Core
Wellcome Trust Centre for Human Genetics
University of Oxford
e. callum@well.ox.ac.uk<mailto:callum@well.ox.ac.uk>

_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users

--_000_288C5B8ED9BC48E5A130EA2F9FA9E942welloxacuk_
Content-Type: text/html; charset="us-ascii"
Content-ID: <995C5595881C87499E993EB0AD84590E@well.ox.ac.uk>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
...
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; line-break:=
 after-white-space;" class=3D"">
This is on 4.2.0.2-1, I've linked the main logs to dropbox simply because t=
hey're big, full of noise right now.
<div class=3D""><a href=3D"https://www.dropbox.com/s/f8q3m5amro2a1b2/engine=
.log?dl=3D0" class=3D"">https://www.dropbox.com/s/f8q3m5amro2a1b2/engine.lo=
g?dl=3D0</a></div>
<div class=3D""><a href=3D"https://www.dropbox.com/s/uods85jk65halo3/vdsm.l=
og?dl=3D0" class=3D"">https://www.dropbox.com/s/uods85jk65halo3/vdsm.log?dl=
=3D0</a></div>
<div class=3D""><br class=3D"">
<div class=3D"">
<div class=3D"">
<div class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-align: star=
t; text-indent: 0px; text-transform: none; white-space: normal; word-spacin=
g: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp=
-mode: space; -webkit-line-break: after-white-space;" class=3D"">
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line=
-break: after-white-space;" class=3D"">
<div style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Regards,</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Callum</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br class=3D"">
</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
--<br class=3D"">
<br class=3D"">
Callum Smith</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Research Computing Core</div>
<div style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Wellcome Trust Centre for Human Genetics<br class=3D"">
University of Oxford<br class=3D"">
e. <a href=3D"mailto:callum@well.ox.ac.uk" class=3D"">callum@well.ox.ac.uk<=
/a></div>
</div>
</div>
</div>
<div><br class=3D"">
<blockquote type=3D"cite" class=3D"">
<div class=3D"">On 2 May 2018, at 10:43, Fred Rolland <<a href=3D"mailto=
:frolland@redhat.com" class=3D"">frolland@redhat.com</a>> wrote:</div>
<br class=3D"Apple-interchange-newline">
<div class=3D"">
<div dir=3D"ltr" class=3D"">
<div class=3D"">Which version are you using?<br class=3D"">
Can you provide the whole log?<br class=3D"">
<br class=3D"">
</div>
For some reason, it looks like the Vdsm thinks that the Storage Domain is n=
ot part of the pool.<br class=3D"">
</div>
<div class=3D"gmail_extra"><br class=3D"">
<div class=3D"gmail_quote">On Wed, May 2, 2018 at 11:20 AM, Callum Smith <s=
pan dir=3D"ltr" class=3D"">
<<a href=3D"mailto:callum@well.ox.ac.uk" target=3D"_blank" class=3D"">ca=
llum@well.ox.ac.uk</a>></span> wrote:<br class=3D"">
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word;line-break:after-white-space" class=3D""=
...
State is maintenance for the ISOs storage. I've extracted what is hopefull=
y the relevant bits of the log.
<div class=3D"">
<div class=3D""><br class=3D"">
</div>
<div class=3D"">VDSM.log (SPM)</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">
<div class=3D"">2018-05-02 09:16:03,455+0100 INFO  (ioprocess comm=
unication (179084)) [IOProcess] Starting ioprocess (__init__:447)</div>
<div class=3D"">2018-05-02 09:16:03,456+0100 INFO  (ioprocess comm=
unication (179091)) [IOProcess] Starting ioprocess (__init__:447)</div>
<div class=3D"">2018-05-02 09:16:03,461+0100 INFO  (jsonrpc/0) [vd=
sm.api] FINISH activateStorageDomain error=3DStorage domain not in pool: u'=
domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae, pool=3D5a54b=
f81-0228-02bc-0358-<wbr class=3D"">000000000304' from=3D::ffff:192.168.64.2=
54,<wbr class=3D"">58968,
 flow_id=3D93433989-8e26-48a9-<wbr class=3D"">bd3a-2ab95f296c08, task_id=3D=
7f21f911-348f-45a3-<wbr class=3D"">b79c-e3cb11642035 (api:50)</div>
<div class=3D"">2018-05-02 09:16:03,461+0100 ERROR (jsonrpc/0) [storage=
.TaskManager.Task] (Task=3D'7f21f911-348f-45a3-<wbr class=3D"">b79c-e3cb116=
42035') Unexpected error (task:875)</div>
<div class=3D"">Traceback (most recent call last):</div>
<div class=3D"">  File "/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/storage/task.py"<wbr class=3D"">, line 882, in _run</div>
<div class=3D"">    return fn(*args, **kargs)</div>
<div class=3D"">  File "<string>", line 2, in activate=
StorageDomain</div>
<div class=3D"">  File "/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/common/api.py", line 48, in method</div>
<div class=3D"">    ret =3D func(*args, **kwargs)</div>
<div class=3D"">  File "/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/storage/hsm.py", line 1256, in activateStorageDomain</div=
...
businessentities.<wbr class=3D"">StoragePoolIsoMap; snapshot: EntityStatus=
</div>
<div class=3D"">Snapshot:{id=3D'<wbr class=3D"">StoragePoolIsoMapId:{<wbr c=
lass=3D"">storagePoolId=3D'5a54bf81-0228-<wbr class=3D"">02bc-0358-00000000=
0304', storageId=3D'f5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae'}'=
, status=3D'Maintenance'}.</div>
<div class=3D"">2018-05-02 09:16:02,660+01 ERROR [org.ovirt.engine.core=
.dal.<wbr class=3D"">dbbroker.auditloghandling.<wbr class=3D"">AuditLogDire=
ctor] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455) [934339=
89-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 EVENT_ID: USER_ACTIVATE_STORAGE_DOMAIN_<wbr class=3D"">FAILED(967), Failed=
 to activate Storage Domain VMISOs (Data Center Default) by admin@internal-=
authz</div>
</div>
<div class=3D""><span class=3D""><br class=3D"">
<div class=3D"">
<div style=3D"letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: br=
eak-word;" class=3D"">
<div style=3D"word-wrap:break-word" class=3D"">
<div style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"">
Regards,</div>
<div style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"">
Callum</div>
<div style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"">
<br class=3D"">
</div>
<div style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"">
--<br class=3D"">
<br class=3D"">
Callum Smith</div>
<div style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"">
Research Computing Core</div>
<div style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"">
Wellcome Trust Centre for Human Genetics<br class=3D"">
University of Oxford<br class=3D"">
e. <a href=3D"mailto:callum@well.ox.ac.uk" target=3D"_blank" class=3D"">cal=
lum@well.ox.ac.uk</a></div>
</div>
</div>
</div>
</span>
<div class=3D"">
<div class=3D"h5">
<div class=3D""><br class=3D"">
<blockquote type=3D"cite" class=3D"">
<div class=3D"">On 2 May 2018, at 08:44, Fred Rolland <<a href=3D"mailto=
:frolland@redhat.com" target=3D"_blank" class=3D"">frolland@redhat.com</a>&=
gt; wrote:</div>
<br class=3D"m_-2093273250354586386Apple-interchange-newline">
<div class=3D"">
<div dir=3D"ltr" class=3D"">
<div class=3D"">
<div class=3D"">
<div class=3D"">
<div class=3D"">Hi,<br class=3D"">
<br class=3D"">
</div>
Can you provide logs from engine and Vdsm(SPM)?<br class=3D"">
</div>
What is the state now?<br class=3D"">
<br class=3D"">
</div>
Thanks,<br class=3D"">
</div>
Fred<br class=3D"">
</div>
<div class=3D"gmail_extra"><br class=3D"">
<div class=3D"gmail_quote">On Tue, May 1, 2018 at 4:11 PM, Callum Smith <sp=
an dir=3D"ltr" class=3D"">
<<a href=3D"mailto:callum@well.ox.ac.uk" target=3D"_blank" class=3D"">ca=
llum@well.ox.ac.uk</a>></span> wrote:<br class=3D"">
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div style=3D"word-wrap:break-word;line-break:after-white-space" class=3D""=
Dear All,
<div class=3D""><br class=3D"">
</div>
<div class=3D"">It appears that clicking "detach" on the ISO stor=
age domain is a really bad idea. This has gotten half way through the proce=
dure and now can't be recovered from. Is there any advice for re-attaching =
<div class=3D"">    pool.activateSD(sdUUID)</div>
<div class=3D"">  File "/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/storage/<wbr class=3D"">securable.py", line 79, in wrappe=
r</div>
<div class=3D"">    return method(self, *args, **kwargs)</div>
<div class=3D"">  File "/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/storage/sp.py", line 1130, in activateSD</div>
<div class=3D"">    self.validateAttachedDomain(<wbr class=3D"">d=
om)</div>
<div class=3D"">  File "/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/storage/<wbr class=3D"">securable.py", line 79, in wrappe=
r</div>
<div class=3D"">    return method(self, *args, **kwargs)</div>
<div class=3D"">  File "/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/storage/sp.py", line 557, in validateAttachedDomain</div>
<div class=3D"">    raise se.StorageDomainNotInPool(<wbr class=3D=
"">self.spUUID, dom.sdUUID)</div>
<div class=3D"">StorageDomainNotInPool: Storage domain not in pool: u'domai=
n=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae, pool=3D5a54bf81-0=
228-02bc-0358-<wbr class=3D"">000000000304'</div>
<div class=3D"">2018-05-02 09:16:03,461+0100 INFO  (jsonrpc/0) [st=
orage.TaskManager.Task] (Task=3D'7f21f911-348f-45a3-<wbr class=3D"">b79c-e3=
cb11642035') aborting: Task is aborted: "Storage domain not in pool: u=
'domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae,
 pool=3D5a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304'" - code =
353 (task:1181)</div>
<div class=3D"">2018-05-02 09:16:03,462+0100 ERROR (jsonrpc/0) [storage=
.Dispatcher] FINISH activateStorageDomain error=3DStorage domain not in poo=
l: u'domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae, pool=3D=
5a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304'
 (dispatcher:82)</div>
</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">engine.log</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">
<div class=3D"">2018-05-02 09:16:02,326+01 INFO  [org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr class=3D"">ActivateStorageDom=
ainCommand] (default task-20) [93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab9=
5f296c08] Lock Acquired to object 'EngineLock:{exclusiveLocks=3D'[<wbr clas=
s=3D"">f5914df0-f46c-4cc0-b666-<wbr class=3D"">c929aa0225ae=3DSTORAGE]',
 sharedLocks=3D''}'</div>
<div class=3D"">2018-05-02 09:16:02,376+01 INFO  [org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455) [=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 Running command: ActivateStorageDomainCommand internal: false. Entities af=
fected :  ID: f5914df0-f46c-4cc0-b666-<wbr class=3D"">c929aa0225ae Typ=
e: StorageAction group MANIPULATE_STORAGE_DOMA</div>
<div class=3D"">IN with role type ADMIN</div>
<div class=3D"">2018-05-02 09:16:02,385+01 INFO  [org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455) [=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 Lock freed to object 'EngineLock:{exclusiveLocks=3D'[<wbr class=3D"">f5914=
df0-f46c-4cc0-b666-<wbr class=3D"">c929aa0225ae=3DSTORAGE]', sharedLocks=3D=
''}'</div>
<div class=3D"">2018-05-02 09:16:02,385+01 INFO  [org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455) [=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 ActivateStorage Domain. Before Connect all hosts to pool. Time: Wed May 02=
 09:16:02 BST 2018</div>
<div class=3D"">2018-05-02 09:16:02,407+01 INFO  [org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.connection.<wbr class=3D"">ConnectStorage=
ToVdsCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33456)=
 [40a82b47] Running command: ConnectStorageToVdsCommand
 internal: true. Entities affected :  ID: aaa00000-0000-0000-0000-<wbr=
 class=3D"">123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN wit=
h role type ADMIN</div>
<div class=3D"">2018-05-02 09:16:02,421+01 INFO  [org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.vdsbroker.<wbr class=3D"">ConnectStorageSer=
verVDSCommand<wbr class=3D"">] (EE-ManagedThreadFactory-<wbr class=3D"">eng=
ine-Thread-33456) [40a82b47] START, ConnectStorageServerVDSCommand<wbr clas=
s=3D"">(HostName
 =3D virtA003, StorageServerConnectionManagem<wbr class=3D"">entVDSParamete=
rs:{hostId=3D'<wbr class=3D"">fe2861fc-2b47-4807-b054-<wbr class=3D"">47019=
8eda473', storagePoolId=3D'00000000-0000-<wbr class=3D"">0000-0000-000000</=
div>
<div class=3D"">000000', storageType=3D'NFS', connectionList=3D'[<wbr class=
=3D"">StorageServerConnections:{id=3D'<wbr class=3D"">da392861-aedc-4f1e-97=
f4-<wbr class=3D"">6919fb01f1e9', connection=3D'backoffice01.<wbr class=3D"=
">cluster:/vm-iso', iqn=3D'null', vfsType=3D'null', mountOptions=3D'null',
 nfsVersion=3D'null', nfsRetrans=3D'null', nfsTimeo=3D'null', iface=3D'null=
', netIfaceName=3D'null'}]'}), log id: 23ce648f</div>
<div class=3D"">2018-05-02 09:16:02,446+01 INFO  [org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.vdsbroker.<wbr class=3D"">ConnectStorageSer=
verVDSCommand<wbr class=3D"">] (EE-ManagedThreadFactory-<wbr class=3D"">eng=
ine-Thread-33456) [40a82b47] FINISH, ConnectStorageServerVDSCommand<wbr cla=
ss=3D"">,
 return: {da392861-aedc-4f1e-97f4-<wbr class=3D"">6919fb01f1e9=3D0}, log id=
: 23ce648f</div>
<div class=3D"">2018-05-02 09:16:02,450+01 INFO  [org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.irsbroker.<wbr class=3D"">ActivateStorageDo=
mainVDSComman<wbr class=3D"">d] (EE-ManagedThreadFactory-<wbr class=3D"">en=
gine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 START, ActivateStorageDomainVDSComman<wbr class=3D"">d( ActivateStorageDom=
ainVDSComman<wbr class=3D"">dParameters:{storagePoolId=3D'<wbr class=3D"">5=
a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304', ignoreFailoverLimit=
=3D'false', stor</div>
<div class=3D"">ageDomainId=3D'f5914df0-f46c-<wbr class=3D"">4cc0-b666-c929=
aa0225ae'}), log id: 5c864594</div>
<div class=3D"">2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core=
.dal.<wbr class=3D"">dbbroker.auditloghandling.<wbr class=3D"">AuditLogDire=
ctor] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455) [934339=
89-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,<wbr class=3D"">803), VDSM command=
 ActivateStorageDomainVDS failed: Storage domain not in pool: u'domain=3Df5=
914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae, pool=3D5a5</div>
<div class=3D"">4bf81-0228-02bc-0358-<wbr class=3D"">000000000304'</div>
<div class=3D"">2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core=
.<wbr class=3D"">vdsbroker.irsbroker.<wbr class=3D"">ActivateStorageDomainV=
DSComman<wbr class=3D"">d] (EE-ManagedThreadFactory-<wbr class=3D"">engine-=
Thread-33455) [93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 Command '<wbr class=3D"">ActivateStorageDomainVDSComman<wbr class=3D"">d( =
ActivateStorageDomainVDSComman<wbr class=3D"">dParameters:{storagePoolId=3D=
'<wbr class=3D"">5a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304', ign=
oreFailoverLimit=3D'false', st</div>
<div class=3D"">orageDomainId=3D'f5914df0-f46c-<wbr class=3D"">4cc0-b666-c9=
29aa0225ae'})' execution failed: IRSGenericException: IRSErrorException: St=
orage domain not in pool: u'domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b66=
6-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304=
'</div>
<div class=3D"">2018-05-02 09:16:02,635+01 INFO  [org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.irsbroker.<wbr class=3D"">ActivateStorageDo=
mainVDSComman<wbr class=3D"">d] (EE-ManagedThreadFactory-<wbr class=3D"">en=
gine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 FINISH, ActivateStorageDomainVDSComman<wbr class=3D"">d, log id: 5c864594<=
/div>
<div class=3D"">2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core=
.bll.<wbr class=3D"">storage.domain.<wbr class=3D"">ActivateStorageDomainCo=
mmand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455) [93433=
989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 Command 'org.ovirt.engine.core.bll.<wbr class=3D"">storage.domain.<wbr cla=
ss=3D"">ActivateStorageDomainCommand' failed: EngineException: org.ovirt.en=
gine.core.<wbr class=3D"">vdsbroker.irsbroker.<wbr class=3D"">IrsOperationF=
ailedNoFailove</div>
<div class=3D"">rException: IRSGenericException: IRSErrorException: Storage=
 domain not in pool: u'domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c92=
9aa0225ae, pool=3D5a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304' (Fa=
iled with error StorageDomainNotInPool and
 code 353)</div>
<div class=3D"">2018-05-02 09:16:02,636+01 INFO  [org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455) [=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
 Command [id=3D22b0f3c1-9a09-4e26-8096-<wbr class=3D"">d83465c8f4ee]: Compe=
nsating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.<wbr class=3D""=
the ISO storage domain manually? An NFS mount
 didn't add it back to the "pool" unfortunately.</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">On a separate note, is it possible to migrate this storage =
to a new location? And if so how.</div>
<div class=3D""><br class=3D"">
<div class=3D"">
<div style=3D"letter-spacing:normal;text-align:start;text-indent:0px;text-t=
ransform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" cla=
ss=3D"">
<div style=3D"word-wrap:break-word" class=3D"">
<div style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"">
Regards,</div>
<div style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"">
Callum</div>
<div style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"">
<br class=3D"">
</div>
<div style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"">
--<br class=3D"">
<br class=3D"">
Callum Smith</div>
<div style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"">
Research Computing Core</div>
<div style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"">
Wellcome Trust Centre for Human Genetics<br class=3D"">
University of Oxford<br class=3D"">
e. <a href=3D"mailto:callum@well.ox.ac.uk" target=3D"_blank" class=3D"">cal=
lum@well.ox.ac.uk</a></div>
</div>
</div>
</div>
<br class=3D"">
</div>
</div>
<br class=3D"">
______________________________<wbr class=3D"">_________________<br class=3D=
"">
Users mailing list<br class=3D"">
<a href=3D"mailto:Users@ovirt.org" target=3D"_blank" class=3D"">Users@ovirt=
.org</a><br class=3D"">
<a href=3D"http://lists.ovirt.org/mailman/listinfo/users" rel=3D"noreferrer=
" target=3D"_blank" class=3D"">http://lists.ovirt.org/mailman<wbr class=3D"=
">/listinfo/users</a><br class=3D"">
<br class=3D"">
</blockquote>
</div>
<br class=3D"">
</div>
</div>
</blockquote>
</div>
<br class=3D"">
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class=3D"">
</div>
</div>
</blockquote>
</div>
<br class=3D"">
</div>
</div>
</div>
</body>
</html>

--_000_288C5B8ED9BC48E5A130EA2F9FA9E942welloxacuk_--