12:53 p.m.
--_000_288C5B8ED9BC48E5A130EA2F9FA9E942welloxacuk_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
This is on 4.2.0.2-1, I've linked the main logs to dropbox simply because t=
hey're big, full of noise right now.
https://www.dropbox.com/s/f8q3m5amro2a1b2/engine.log?dl=3D0
https://www.dropbox.com/s/uods85jk65halo3/vdsm.log?dl=3D0
Regards,
Callum
--
Callum Smith
Research Computing Core
Wellcome Trust Centre for Human Genetics
University of Oxford
e. callum@well.ox.ac.uk<mailto:callum@well.ox.ac.uk
On 2 May 2018, at 10:43, Fred Rolland <frolland@redhat.com<mailto:frolland@=
redhat.com>> wrote:
Which version are you using?
Can you provide the whole log?
For some reason, it looks like the Vdsm thinks that the Storage Domain is n=
ot part of the pool.
On Wed, May 2, 2018 at 11:20 AM, Callum Smith <callum@well.ox.ac.uk<mailto:=
callum(a)well.ox.ac.uk>> wrote:
State is maintenance for the ISOs storage. I've extracted what is hopefully=
the relevant bits of the log.
VDSM.log (SPM)
2018-05-02 09:16:03,455+0100 INFO (ioprocess communication (179084)) [IOPr=
ocess] Starting ioprocess (__init__:447)
2018-05-02 09:16:03,456+0100 INFO (ioprocess communication (179091)) [IOPr=
ocess] Starting ioprocess (__init__:447)
2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [vdsm.api] FINISH activateSt=
orageDomain error=3DStorage domain not in pool: u'domain=3Df5914df0-f46c-4c=
c0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-000000000304' from=3D:=
:ffff:192.168.64.254,58968, flow_id=3D93433989-8e26-48a9-bd3a-2ab95f296c08,=
task_id=3D7f21f911-348f-45a3-b79c-e3cb11642035 (api:50)
2018-05-02 09:16:03,461+0100 ERROR (jsonrpc/0) [storage.TaskManager.Task] (=
Task=3D'7f21f911-348f-45a3-b79c-e3cb11642035') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, i=
n _run
return fn(*args, **kargs)
File "<string>", line 2, in activateStorageDomain
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in m=
ethod
ret =3D func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1256, i=
n activateStorageDomain
pool.activateSD(sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 7=
9, in wrapper
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1130, in=
activateSD
self.validateAttachedDomain(dom)
File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 7=
9, in wrapper
return method(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 557, in =
validateAttachedDomain
raise se.StorageDomainNotInPool(self.spUUID, dom.sdUUID)
StorageDomainNotInPool: Storage domain not in pool: u'domain=3Df5914df0-f46=
c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-000000000304'
2018-05-02 09:16:03,461+0100 INFO (jsonrpc/0) [storage.TaskManager.Task] (=
Task=3D'7f21f911-348f-45a3-b79c-e3cb11642035') aborting: Task is aborted: "=
Storage domain not in pool: u'domain=3Df5914df0-f46c-4cc0-b666-c929aa0225ae=
, pool=3D5a54bf81-0228-02bc-0358-000000000304'" - code 353 (task:1181)
2018-05-02 09:16:03,462+0100 ERROR (jsonrpc/0) [storage.Dispatcher] FINISH =
activateStorageDomain error=3DStorage domain not in pool: u'domain=3Df5914d=
f0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-000000000304=
' (dispatcher:82)
engine.log
2018-05-02 09:16:02,326+01 INFO [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (default task-20) [93433989-8e26-48a9-bd3a-2a=
b95f296c08] Lock Acquired to object 'EngineLock:{exclusiveLocks=3D'[f5914df=
0-f46c-4cc0-b666-c929aa0225ae=3DSTORAGE]', sharedLocks=3D''}'
2018-05-02 09:16:02,376+01 INFO [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
[93433989-8e26-48a9-bd3a-2ab95f296c08] Running command: ActivateStorageDom=
ainCommand internal: false. Entities affected : ID: f5914df0-f46c-4cc0-b66=
6-c929aa0225ae Type: StorageAction group MANIPULATE_STORAGE_DOMA
IN with role type ADMIN
2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
[93433989-8e26-48a9-bd3a-2ab95f296c08] Lock freed to object 'EngineLock:{e=
xclusiveLocks=3D'[f5914df0-f46c-4cc0-b666-c929aa0225ae=3DSTORAGE]', sharedL=
ocks=3D''}'
2018-05-02 09:16:02,385+01 INFO [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
[93433989-8e26-48a9-bd3a-2ab95f296c08] ActivateStorage Domain. Before Conn=
ect all hosts to pool. Time: Wed May 02 09:16:02 BST 2018
2018-05-02 09:16:02,407+01 INFO [org.ovirt.engine.core.bll.storage.connect=
ion.ConnectStorageToVdsCommand] (EE-ManagedThreadFactory-engine-Thread-3345=
6) [40a82b47] Running command: ConnectStorageToVdsCommand internal: true. E=
ntities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAc=
tion group CREATE_STORAGE_DOMAIN with role type ADMIN
2018-05-02 09:16:02,421+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-334=
56) [40a82b47] START, ConnectStorageServerVDSCommand(HostName =3D virtA003,=
StorageServerConnectionManagementVDSParameters:{hostId=3D'fe2861fc-2b47-48=
07-b054-470198eda473', storagePoolId=3D'00000000-0000-0000-0000-000000
000000', storageType=3D'NFS',
connectionList=3D'[StorageServerConnections:{=
id=3D'da392861-aedc-4f1e-97f4-6919fb01f1e9', connection=3D'backoffice01.clu=
ster:/vm-iso', iqn=3D'null', vfsType=3D'null',
mountOptions=3D'null', nfsVe=
rsion=3D'null', nfsRetrans=3D'null', nfsTimeo=3D'null',
iface=3D'null', net=
IfaceName=3D'null'}]'}), log id: 23ce648f
2018-05-02 09:16:02,446+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker=
.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-334=
56) [40a82b47] FINISH, ConnectStorageServerVDSCommand, return: {da392861-ae=
dc-4f1e-97f4-6919fb01f1e9=3D0}, log id: 23ce648f
2018-05-02 09:16:02,450+01 INFO [org.ovirt.engine.core.vdsbroker.irsbroker=
.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33=
455) [93433989-8e26-48a9-bd3a-2ab95f296c08] START, ActivateStorageDomainVDS=
Command( ActivateStorageDomainVDSCommandParameters:{storagePoolId=3D'5a54bf=
81-0228-02bc-0358-000000000304', ignoreFailoverLimit=3D'false', stor
ageDomainId=3D'f5914df0-f46c-4cc0-b666-c929aa0225ae'}), log id: 5c864594
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-33455) =
[93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID: IRS_BROKER_COMMAND_FAILURE=
(10,803), VDSM command ActivateStorageDomainVDS failed: Storage domain not =
in pool: u'domain=3Df5914df0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a5
4bf81-0228-02bc-0358-000000000304'
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker=
.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33=
455) [93433989-8e26-48a9-bd3a-2ab95f296c08] Command 'ActivateStorageDomainV=
DSCommand( ActivateStorageDomainVDSCommandParameters:{storagePoolId=3D'5a54=
bf81-0228-02bc-0358-000000000304', ignoreFailoverLimit=3D'false', st
orageDomainId=3D'f5914df0-f46c-4cc0-b666-c929aa0225ae'})' execution failed:=
IRSGenericException: IRSErrorException: Storage domain not in pool: u'doma=
in=3Df5914df0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-0=
00000000304'
2018-05-02 09:16:02,635+01 INFO [org.ovirt.engine.core.vdsbroker.irsbroker=
.ActivateStorageDomainVDSCommand] (EE-ManagedThreadFactory-engine-Thread-33=
455) [93433989-8e26-48a9-bd3a-2ab95f296c08] FINISH, ActivateStorageDomainVD=
SCommand, log id: 5c864594
2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
[93433989-8e26-48a9-bd3a-2ab95f296c08] Command 'org.ovirt.engine.core.bll.=
storage.domain.ActivateStorageDomainCommand' failed: EngineException: org.o=
virt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailove
rException: IRSGenericException: IRSErrorException: Storage domain not in p=
ool: u'domain=3Df5914df0-f46c-4cc0-b666-c929aa0225ae, pool=3D5a54bf81-0228-=
02bc-0358-000000000304' (Failed with error StorageDomainNotInPool and code =
353)
2018-05-02 09:16:02,636+01 INFO [org.ovirt.engine.core.bll.storage.domain.=
ActivateStorageDomainCommand] (EE-ManagedThreadFactory-engine-Thread-33455)=
[93433989-8e26-48a9-bd3a-2ab95f296c08] Command [id=3D22b0f3c1-9a09-4e26-80=
96-d83465c8f4ee]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core=
.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatus
Snapshot:{id=3D'StoragePoolIsoMapId:{storagePoolId=3D'5a54bf81-0228-02bc-03=
58-000000000304', storageId=3D'f5914df0-f46c-4cc0-b666-c929aa0225ae'}',
sta=
tus=3D'Maintenance'}.
2018-05-02 09:16:02,660+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditl=
oghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-33455) =
[93433989-8e26-48a9-bd3a-2ab95f296c08] EVENT_ID: USER_ACTIVATE_STORAGE_DOMA=
IN_FAILED(967), Failed to activate Storage Domain VMISOs (Data Center Defau=
lt) by admin@internal-authz
Regards,
Callum
--
Callum Smith
Research Computing Core
Wellcome Trust Centre for Human Genetics
University of Oxford
e. callum@well.ox.ac.uk<mailto:callum@well.ox.ac.uk
On 2 May 2018, at 08:44, Fred Rolland <frolland@redhat.com<mailto:frolland@=
redhat.com>> wrote:
Hi,
Can you provide logs from engine and Vdsm(SPM)?
What is the state now?
Thanks,
Fred
On Tue, May 1, 2018 at 4:11 PM, Callum Smith <callum@well.ox.ac.uk<mailto:c=
allum(a)well.ox.ac.uk>> wrote:
Dear All,
It appears that clicking "detach" on the ISO storage domain is a really bad=
idea. This has gotten half way through the procedure and now can't be reco=
vered from. Is there any advice for re-attaching the ISO storage domain man=
ually? An NFS mount didn't add it back to the "pool" unfortunately.
On a separate note, is it possible to migrate this storage to a new locatio=
n? And if so how.
Regards,
Callum
--
Callum Smith
Research Computing Core
Wellcome Trust Centre for Human Genetics
University of Oxford
e. callum@well.ox.ac.uk<mailto:callum@well.ox.ac.uk
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--_000_288C5B8ED9BC48E5A130EA2F9FA9E942welloxacuk_
Content-Type: text/html; charset="us-ascii"
Content-ID: <995C5595881C87499E993EB0AD84590E(a)well.ox.ac.uk Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Dus-ascii"=
</head <body
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; line-break:=
after-white-space;" class=3D"" This is
on 4.2.0.2-1, I've linked the main logs to dropbox simply because t=
hey're big, full of noise right now.
<div class=3D""><a
href=3D"https://www.dropbox.com/s/f8q3m5amro2a1b2/engine=
.log?dl=3D0"
class=3D"">https://www.dropbox.com/s/f8q3m5amro2a1b2/engine.lo=
g?dl=3D0</a></div <div
class=3D""><a
href=3D"https://www.dropbox.com/s/uods85jk65halo3/vdsm.l=
og?dl=3D0"
class=3D"">https://www.dropbox.com/s/uods85jk65halo3/vdsm.lo...
=3D0</a></div <div
class=3D""><br class=3D"" <div
class=3D"" <div class=3D"" <div class=3D"" <div
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-align: star=
t; text-indent: 0px; text-transform: none; white-space: normal; word-spacin=
g: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp=
-mode: space; -webkit-line-break: after-white-space;" class=3D"" <div style=3D"word-wrap: break-word; -webkit-nbsp-mode:
space; -webkit-line=
-break: after-white-space;" class=3D"" <div
style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Regards,</div <div
style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Callum</div <div
style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" <br class=3D""
</div <div style=3D"color: rgb(0,
0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" --<br class=3D"" <br
class=3D"" Callum Smith</div <div style=3D"color: rgb(0, 0, 0); font-family: Arial;
font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Research Computing Core</div <div
style=3D"color: rgb(0, 0, 0); font-family: Arial; font-size: 13px; fon=
t-style: normal; font-variant-caps: normal; font-weight: normal; letter-spa=
cing: normal; text-align: start; text-indent: 0px; text-transform: none; wh=
ite-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" Wellcome Trust Centre for Human Genetics<br
class=3D"" University of Oxford<br
class=3D"" e. <a
href=3D"mailto:callum@well.ox.ac.uk"
class=3D"">callum(a)well.ox.ac.uk<=
/a></div </div </div
</div <div><br
class=3D"" <blockquote
type=3D"cite" class=3D"" <div
class=3D"">On 2 May 2018, at 10:43, Fred Rolland <<a
href=3D"mailto=
:frolland@redhat.com" class=3D"">frolland(a)redhat.com</a>&gt;
wrote:</div <br
class=3D"Apple-interchange-newline" <div
class=3D"" <div dir=3D"ltr"
class=3D"" <div
class=3D"">Which version are you using?<br class=3D"" Can you provide the whole log?<br class=3D"" <br class=3D""
</div For some reason, it looks like the
Vdsm thinks that the Storage Domain is n=
ot part of the pool.<br class=3D""
</div <div
class=3D"gmail_extra"><br class=3D"" <div class=3D"gmail_quote">On Wed, May 2, 2018
at 11:20 AM, Callum Smith <s=
pan dir=3D"ltr" class=3D""
<<a href=3D"mailto:callum@well.ox.ac.uk"
target=3D"_blank" class=3D"">ca=
llum(a)well.ox.ac.uk</a>&gt;</span> wrote:<br class=3D"" <blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex" <div
style=3D"word-wrap:break-word;line-break:after-white-space"
class=3D""=
<div
class=3D""><br class=3D""
</div <div
class=3D"">VDSM.log (SPM)</div <div
class=3D""><br class=3D""
</div <div class=3D"" <div class=3D"">2018-05-02
09:16:03,455+0100 INFO (ioprocess comm=
unication (179084)) [IOProcess] Starting ioprocess (__init__:447)</div <div class=3D"">2018-05-02
09:16:03,456+0100 INFO (ioprocess comm=
unication (179091)) [IOProcess] Starting ioprocess (__init__:447)</div <div class=3D"">2018-05-02
09:16:03,461+0100 INFO (jsonrpc/0) [vd=
sm.api] FINISH activateStorageDomain error=3DStorage domain not in pool: u'=
domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae,
pool=3D5a54b=
f81-0228-02bc-0358-<wbr class=3D"">000000000304'
from=3D::ffff:192.168.64.2=
54,<wbr class=3D"">58968,
flow_id=3D93433989-8e26-48a9-<wbr class=3D"">bd3a-2ab95f296c08,
task_id=3D=
7f21f911-348f-45a3-<wbr class=3D"">b79c-e3cb11642035 (api:50)</div <div class=3D"">2018-05-02
09:16:03,461+0100 ERROR (jsonrpc/0) [storage=
.TaskManager.Task] (Task=3D'7f21f911-348f-45a3-<wbr
class=3D"">b79c-e3cb116=
42035') Unexpected error (task:875)</div <div
class=3D"">Traceback (most recent call last):</div <div class=3D""> File
"/usr/lib/python2.7/site-<wbr class=3D"">p=
ackages/vdsm/storage/task.py"<wbr class=3D"">, line 882, in
_run</div <div
class=3D""> return fn(*args, **kargs)</div <div class=3D""> File
"<string>", line 2, in activate=
StorageDomain</div <div
class=3D""> File "/usr/lib/python2.7/site-<wbr
class=3D"">p=
ackages/vdsm/common/api.py", line 48, in method</div <div class=3D""> ret =3D
func(*args, **kwargs)</div <div
class=3D""> File "/usr/lib/python2.7/site-<wbr
class=3D"">p=
ackages/vdsm/storage/hsm.py", line 1256, in activateStorageDomain</div=
<div class=3D"">
pool.activateSD(sdUUID)</div <div
class=3D""> File "/usr/lib/python2.7/site-<wbr
class=3D"">p=
ackages/vdsm/storage/<wbr class=3D"">securable.py", line 79, in
wrappe=
r</div <div
class=3D""> return method(self, *args,
**kwargs)</div <div
class=3D""> File "/usr/lib/python2.7/site-<wbr
class=3D"">p=
ackages/vdsm/storage/sp.py", line 1130, in activateSD</div <div class=3D"">
self.validateAttachedDomain(<wbr class=3D"">d=
om)</div <div
class=3D""> File "/usr/lib/python2.7/site-<wbr
class=3D"">p=
ackages/vdsm/storage/<wbr class=3D"">securable.py", line 79, in
wrappe=
r</div <div
class=3D""> return method(self, *args,
**kwargs)</div <div
class=3D""> File "/usr/lib/python2.7/site-<wbr
class=3D"">p=
ackages/vdsm/storage/sp.py", line 557, in validateAttachedDomain</div <div class=3D""> raise
se.StorageDomainNotInPool(<wbr class=3D=
"">self.spUUID, dom.sdUUID)</div <div
class=3D"">StorageDomainNotInPool: Storage domain not in pool: u'domai=
n=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae,
pool=3D5a54bf81-0=
228-02bc-0358-<wbr class=3D"">000000000304'</div <div class=3D"">2018-05-02
09:16:03,461+0100 INFO (jsonrpc/0) [st=
orage.TaskManager.Task] (Task=3D'7f21f911-348f-45a3-<wbr
class=3D"">b79c-e3=
cb11642035') aborting: Task is aborted: "Storage domain not in pool: u=
'domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae,
pool=3D5a54bf81-0228-02bc-0358-<wbr
class=3D"">000000000304'" - code =
353 (task:1181)</div <div
class=3D"">2018-05-02 09:16:03,462+0100 ERROR (jsonrpc/0) [storage=
.Dispatcher] FINISH activateStorageDomain error=3DStorage domain not in poo=
l: u'domain=3Df5914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae,
pool=3D=
5a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304'
(dispatcher:82)</div </div <div class=3D""><br class=3D"" </div <div
class=3D"">engine.log</div <div
class=3D""><br class=3D""
</div <div class=3D"" <div class=3D"">2018-05-02
09:16:02,326+01 INFO [org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr
class=3D"">ActivateStorageDom=
ainCommand] (default task-20) [93433989-8e26-48a9-bd3a-<wbr
class=3D"">2ab9=
5f296c08] Lock Acquired to object 'EngineLock:{exclusiveLocks=3D'[<wbr clas=
s=3D"">f5914df0-f46c-4cc0-b666-<wbr
class=3D"">c929aa0225ae=3DSTORAGE]',
sharedLocks=3D''}'</div <div
class=3D"">2018-05-02 09:16:02,376+01 INFO
[org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr
class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455)
[=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
Running command: ActivateStorageDomainCommand internal: false. Entities af=
fected : ID: f5914df0-f46c-4cc0-b666-<wbr
class=3D"">c929aa0225ae Typ=
e: StorageAction group MANIPULATE_STORAGE_DOMA</div <div
class=3D"">IN with role type ADMIN</div <div
class=3D"">2018-05-02 09:16:02,385+01 INFO
[org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr
class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455)
[=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
Lock freed to object 'EngineLock:{exclusiveLocks=3D'[<wbr
class=3D"">f5914=
df0-f46c-4cc0-b666-<wbr class=3D"">c929aa0225ae=3DSTORAGE]',
sharedLocks=3D=
''}'</div <div
class=3D"">2018-05-02 09:16:02,385+01 INFO
[org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr
class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455)
[=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
ActivateStorage Domain. Before Connect all hosts to pool. Time: Wed May 02=
09:16:02 BST 2018</div <div
class=3D"">2018-05-02 09:16:02,407+01 INFO
[org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.connection.<wbr
class=3D"">ConnectStorage=
ToVdsCommand] (EE-ManagedThreadFactory-<wbr
class=3D"">engine-Thread-33456)=
[40a82b47] Running command: ConnectStorageToVdsCommand
internal: true. Entities affected : ID: aaa00000-0000-0000-0000-<wbr=
class=3D"">123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN wit=
h role type ADMIN</div <div
class=3D"">2018-05-02 09:16:02,421+01 INFO
[org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.vdsbroker.<wbr
class=3D"">ConnectStorageSer=
verVDSCommand<wbr class=3D"">] (EE-ManagedThreadFactory-<wbr
class=3D"">eng=
ine-Thread-33456) [40a82b47] START, ConnectStorageServerVDSCommand<wbr clas=
s=3D"">(HostName
=3D virtA003, StorageServerConnectionManagem<wbr
class=3D"">entVDSParamete=
rs:{hostId=3D'<wbr class=3D"">fe2861fc-2b47-4807-b054-<wbr
class=3D"">47019=
8eda473', storagePoolId=3D'00000000-0000-<wbr
class=3D"">0000-0000-000000</=
div <div class=3D"">000000',
storageType=3D'NFS', connectionList=3D'[<wbr class=
=3D"">StorageServerConnections:{id=3D'<wbr
class=3D"">da392861-aedc-4f1e-97=
f4-<wbr class=3D"">6919fb01f1e9',
connection=3D'backoffice01.<wbr class=3D"=
">cluster:/vm-iso', iqn=3D'null', vfsType=3D'null',
mountOptions=3D'null',
nfsVersion=3D'null', nfsRetrans=3D'null', nfsTimeo=3D'null',
iface=3D'null=
', netIfaceName=3D'null'}]'}), log id: 23ce648f</div <div class=3D"">2018-05-02
09:16:02,446+01 INFO [org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.vdsbroker.<wbr
class=3D"">ConnectStorageSer=
verVDSCommand<wbr class=3D"">] (EE-ManagedThreadFactory-<wbr
class=3D"">eng=
ine-Thread-33456) [40a82b47] FINISH, ConnectStorageServerVDSCommand<wbr cla=
ss=3D"">,
return: {da392861-aedc-4f1e-97f4-<wbr class=3D"">6919fb01f1e9=3D0}, log
id=
: 23ce648f</div <div
class=3D"">2018-05-02 09:16:02,450+01 INFO
[org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.irsbroker.<wbr
class=3D"">ActivateStorageDo=
mainVDSComman<wbr class=3D"">d] (EE-ManagedThreadFactory-<wbr
class=3D"">en=
gine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
START, ActivateStorageDomainVDSComman<wbr class=3D"">d(
ActivateStorageDom=
ainVDSComman<wbr class=3D"">dParameters:{storagePoolId=3D'<wbr
class=3D"">5=
a54bf81-0228-02bc-0358-<wbr class=3D"">000000000304',
ignoreFailoverLimit=
=3D'false', stor</div <div
class=3D"">ageDomainId=3D'f5914df0-f46c-<wbr
class=3D"">4cc0-b666-c929=
aa0225ae'}), log id: 5c864594</div <div
class=3D"">2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core=
.dal.<wbr class=3D"">dbbroker.auditloghandling.<wbr
class=3D"">AuditLogDire=
ctor] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455)
[934339=
89-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,<wbr class=3D"">803), VDSM
command=
ActivateStorageDomainVDS failed: Storage domain not in pool: u'domain=3Df5=
914df0-f46c-4cc0-<wbr class=3D"">b666-c929aa0225ae, pool=3D5a5</div <div class=3D"">4bf81-0228-02bc-0358-<wbr
class=3D"">000000000304'</div <div
class=3D"">2018-05-02 09:16:02,635+01 ERROR [org.ovirt.engine.core=
.<wbr class=3D"">vdsbroker.irsbroker.<wbr
class=3D"">ActivateStorageDomainV=
DSComman<wbr class=3D"">d] (EE-ManagedThreadFactory-<wbr
class=3D"">engine-=
Thread-33455) [93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
Command '<wbr class=3D"">ActivateStorageDomainVDSComman<wbr
class=3D"">d( =
ActivateStorageDomainVDSComman<wbr
class=3D"">dParameters:{storagePoolId=3D=
'<wbr class=3D"">5a54bf81-0228-02bc-0358-<wbr
class=3D"">000000000304', ign=
oreFailoverLimit=3D'false', st</div <div
class=3D"">orageDomainId=3D'f5914df0-f46c-<wbr
class=3D"">4cc0-b666-c9=
29aa0225ae'})' execution failed: IRSGenericException: IRSErrorException: St=
orage domain not in pool: u'domain=3Df5914df0-f46c-4cc0-<wbr
class=3D"">b66=
6-c929aa0225ae, pool=3D5a54bf81-0228-02bc-0358-<wbr
class=3D"">000000000304=
'</div <div
class=3D"">2018-05-02 09:16:02,635+01 INFO
[org.ovirt.engine=
.core.<wbr class=3D"">vdsbroker.irsbroker.<wbr
class=3D"">ActivateStorageDo=
mainVDSComman<wbr class=3D"">d] (EE-ManagedThreadFactory-<wbr
class=3D"">en=
gine-Thread-33455) [93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
FINISH, ActivateStorageDomainVDSComman<wbr class=3D"">d, log id:
5c864594<=
/div <div class=3D"">2018-05-02
09:16:02,635+01 ERROR [org.ovirt.engine.core=
.bll.<wbr class=3D"">storage.domain.<wbr
class=3D"">ActivateStorageDomainCo=
mmand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455)
[93433=
989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
Command 'org.ovirt.engine.core.bll.<wbr
class=3D"">storage.domain.<wbr cla=
ss=3D"">ActivateStorageDomainCommand' failed: EngineException:
org.ovirt.en=
gine.core.<wbr class=3D"">vdsbroker.irsbroker.<wbr
class=3D"">IrsOperationF=
ailedNoFailove</div <div
class=3D"">rException: IRSGenericException: IRSErrorException: Storage=
domain not in pool: u'domain=3Df5914df0-f46c-4cc0-<wbr
class=3D"">b666-c92=
9aa0225ae, pool=3D5a54bf81-0228-02bc-0358-<wbr
class=3D"">000000000304' (Fa=
iled with error StorageDomainNotInPool and
code 353)</div <div
class=3D"">2018-05-02 09:16:02,636+01 INFO
[org.ovirt.engine=
.core.bll.<wbr class=3D"">storage.domain.<wbr
class=3D"">ActivateStorageDom=
ainCommand] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455)
[=
93433989-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
Command [id=3D22b0f3c1-9a09-4e26-8096-<wbr class=3D"">d83465c8f4ee]:
Compe=
nsating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.<wbr class=3D""=
<div
class=3D"">Snapshot:{id=3D'<wbr
class=3D"">StoragePoolIsoMapId:{<wbr c=
lass=3D"">storagePoolId=3D'5a54bf81-0228-<wbr
class=3D"">02bc-0358-00000000=
0304', storageId=3D'f5914df0-f46c-4cc0-<wbr
class=3D"">b666-c929aa0225ae'}'=
, status=3D'Maintenance'}.</div <div
class=3D"">2018-05-02 09:16:02,660+01 ERROR [org.ovirt.engine.core=
.dal.<wbr class=3D"">dbbroker.auditloghandling.<wbr
class=3D"">AuditLogDire=
ctor] (EE-ManagedThreadFactory-<wbr class=3D"">engine-Thread-33455)
[934339=
89-8e26-48a9-bd3a-<wbr class=3D"">2ab95f296c08]
EVENT_ID: USER_ACTIVATE_STORAGE_DOMAIN_<wbr class=3D"">FAILED(967),
Failed=
to activate Storage Domain VMISOs (Data Center Default) by admin@internal-=
authz</div </div <div class=3D""><span
class=3D""><br class=3D"" <div
class=3D"" <div
style=3D"letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: br=
eak-word;" class=3D"" <div
style=3D"word-wrap:break-word" class=3D"" <div
style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D""
Regards,</div <div style=3D"font-family:
Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D""
Callum</div <div style=3D"font-family:
Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"" <br
class=3D"" </div <div style=3D"font-family: Arial; font-size: 13px;
font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"" --<br
class=3D"" <br class=3D"" Callum Smith</div <div
style=3D"font-family: Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"" Research
Computing Core</div <div style=3D"font-family:
Arial; font-size: 13px; font-style: normal; font=
-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-al=
ign: start; text-indent: 0px; text-transform: none; white-space: normal; wo=
rd-spacing: 0px;" class=3D"" Wellcome
Trust Centre for Human Genetics<br class=3D""
University of Oxford<br class=3D"" e. <a
href=3D"mailto:callum@well.ox.ac.uk" target=3D"_blank"
class=3D"">cal=
lum(a)well.ox.ac.uk</a></div
</div </div </div
</span <div class=3D"" <div class=3D"h5" <div
class=3D""><br class=3D""
<blockquote type=3D"cite" class=3D"" <div class=3D"">On 2 May 2018, at 08:44, Fred
Rolland <<a href=3D"mailto=
:frolland@redhat.com" target=3D"_blank"
class=3D"">frolland(a)redhat.com</a>&=
gt; wrote:</div <br
class=3D"m_-2093273250354586386Apple-interchange-newline" <div class=3D"" <div
dir=3D"ltr" class=3D"" <div
class=3D"" <div class=3D"" <div class=3D"" <div
class=3D"">Hi,<br class=3D"" <br
class=3D"" </div Can you provide logs from engine and Vdsm(SPM)?<br
class=3D"" </div What is the state now?<br class=3D"" <br class=3D""
</div Thanks,<br
class=3D"" </div Fred<br class=3D""
</div <div
class=3D"gmail_extra"><br class=3D"" <div class=3D"gmail_quote">On Tue, May 1, 2018
at 4:11 PM, Callum Smith <sp=
an dir=3D"ltr" class=3D""
<<a href=3D"mailto:callum@well.ox.ac.uk"
target=3D"_blank" class=3D"">ca=
llum(a)well.ox.ac.uk</a>&gt;</span> wrote:<br class=3D"" <blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex" <div
style=3D"word-wrap:break-word;line-break:after-white-space"
class=3D""=
</div <div class=3D"">It appears that clicking
"detach" on the ISO stor=
age domain is a really bad idea. This has gotten half way through the proce=
dure and now can't be recovered from. Is there any advice for re-attaching =
the ISO storage domain manually? An NFS mount
didn't add it back to the "pool" unfortunately.</div <div class=3D""><br class=3D"" </div <div
class=3D"">On a separate note, is it possible to migrate this storage =
to a new location? And if so how.</div <div
class=3D""><br class=3D"" <div
class=3D"" <div
style=3D"letter-spacing:normal;text-align:start;text-indent:0px;text-t=
ransform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" cla=
ss=3D"" <div
style=3D"word-wrap:break-word" class=3D"" <div
style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"" Regards,</div <div
style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"" Callum</div <div
style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"" <br class=3D"" </div <div
style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"" --<br class=3D"" <br class=3D"" Callum
Smith</div <div
style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"" Research Computing Core</div <div
style=3D"font-family:Arial;font-size:13px;font-style:normal;font-varia=
nt-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" clas=
s=3D"" Wellcome Trust Centre for Human
Genetics<br class=3D""
University of Oxford<br class=3D"" e. <a
href=3D"mailto:callum@well.ox.ac.uk" target=3D"_blank"
class=3D"">cal=
lum(a)well.ox.ac.uk</a></div
</div </div </div <br
class=3D"" </div </div <br
class=3D""
______________________________<wbr
class=3D"">_________________<br class=3D=
"" Users mailing list<br
class=3D"" <a
href=3D"mailto:Users@ovirt.org" target=3D"_blank"
class=3D"">Users@ovirt=
.org</a><br class=3D"" <a
href=3D"http://lists.ovirt.org/mailman/listinfo/users" rel=3D"noreferrer=
" target=3D"_blank"
class=3D"">http://lists.ovirt.org/mailman<wbr class=3D"=
">/listinfo/users</a><br class=3D"" <br class=3D""
</blockquote </div <br class=3D""
</div </div </blockquote
</div <br class=3D"" </div
</div </div </div
</div </blockquote </div <br
class=3D"" </div </div
</blockquote </div <br class=3D""
</div </div </div
</body </html
--_000_288C5B8ED9BC48E5A130EA2F9FA9E942welloxacuk_--
State is maintenance for the ISOs storage. I've extracted what is
hopefull=
y the relevant bits of the log.
<div class=3D""businessentities.<wbr class=3D"">StoragePoolIsoMap;
snapshot: EntityStatus=
</divDear All,
<div class=3D""><br
class=3D""