9:25 p.m.
--_000_140965710620221419edigitalresearchcom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
?Hi All!
I decommissioned a NFS export domain this morning, ended up 'destroying' it=
through the web interface as detaching kept failing.? Now one of my hosts=
keeps flipping between 'Non Operational' and 'Unassigned'. All of the
VMs=
on this host are still running. I am in global-maintenance to prevent mig=
rations etc.
vdsm.log seems to indicate that it is related to connecting to the destroye=
d storage domain:
Thread-206::WARNING::2014-09-02 12:19:05,692::fileSD::673::scanDomains::(co=
llectMetaFiles) Metadata collection for domain path /rhev/data-center/mnt/1=
0.0.0.30:_mnt_kvm1_export timedout
Traceback (most recent call last):
File "/usr/share/vdsm/storage/fileSD.py", line 662, in collectMetaFiles
sd.DOMAIN_META_DATA))
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 297, in callCra=
bRPCFunction
*args, **kwargs)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 184, in callCra=
bRPCFunction
rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH, timeout)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 150, in _recvAl=
l
raise Timeout()
Timeout
Thread-206::DEBUG::2014-09-02 12:19:05,695::remoteFileHandler::260::RepoFil=
eHelper.PoolHandler::(stop) Pool handler existed, OUT: '' ERR: ''
Thread-210::WARNING::2014-09-02 12:19:05,745::fileSD::673::scanDomains::(co=
llectMetaFiles) Metadata collection for domain path /rhev/data-center/mnt/1=
0.0.0.30:_mnt_kvm1_export timedout
Traceback (most recent call last):
File "/usr/share/vdsm/storage/fileSD.py", line 662, in collectMetaFiles
sd.DOMAIN_META_DATA))
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 297, in callCra=
bRPCFunction
*args, **kwargs)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 184, in callCra=
bRPCFunction
rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH, timeout)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 150, in _recvAl=
l
raise Timeout()
Timeout
Given that this storage domain doesn't exist any more and is not visible in=
the web interface, how can I get this host to stop trying to connect to it=
, initialize and become online?
James
Watch How to Turn Data into Action to Improve Customer Experiences in an Ag=
ile World<http://www.edigitalresearch.com/news/item/nid/878290232
We are delighted to be ranked 32nd in 'The Sunday Times Top 100 Best Small =
Companies to Work For 2014' list.
This message is sent in confidence for the addressee only. The contents are=
not to be disclosed to anyone other than the addressee.
Unauthorised recipients must preserve this confidentiality and should pleas=
e advise the sender immediately of any error in transmission.
Any attachment(s) to this message have been checked for viruses, but please=
rely on your own virus checker and procedures.
Please note that Internet email is not a secure communications medium. We a=
dvise that you understand and observe this when emailing us.
eDigitalResearch plc is a public limited company registered at the Registra=
r Of Companies for England and Wales. Company registration number: 5424597
Registered Office: Vanbrugh House, Hedge End, Hampshire, SO30 2AF
PS: Save paper - do you really need to print this email?
--_000_140965710620221419edigitalresearchcom_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Diso-8859-=
1" <style type=3D"text/css"
style=3D"display:none"><!-- p { margin-top: 0px; m=
argin-bottom: 0px; }--></style
</head <body dir=3D"ltr"
style=3D"font-size:12pt;color:#000000;background-color:#F=
FFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;" <div
dir=3D"ltr" style=3D"font-size:12pt; color:#000000; background-color:#=
FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif"
<div <p>​Hi
All!<br </p
<p><br </p
<p>I decommissioned a NFS export domain this morning, ended up
'destroying'=
it through the web interface as detaching kept failing.​ <span=
style=3D"font-size:12pt">Now one of my hosts keeps flipping between 'Non
O=
perational' and 'Unassigned'. All of the VMs
on this host are still running. I am in global-maintenance to preven=
t migrations etc.</span></p
<p><span style=3D"font-size:12pt"><br </span></p
<p><span style=3D"font-size:12pt">vdsm.log seems to indicate
that it is&nbs=
p;related to connecting to the destroyed storage domain:</span></p <p><br
</p
<div>Thread-206::WARNING::2014-09-02 12:19:05,692::fileSD::673::scanDomains=
::(collectMetaFiles) Metadata collection for domain path /rhev/data-center/=
mnt/10.0.0.30:_mnt_kvm1_export timedout</div
<div>Traceback (most recent call last):</div
<div> File "/usr/share/vdsm/storage/fileSD.py",
line 662, i=
n collectMetaFiles</div <div>
sd.DOMAIN_META_DATA))</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 297, in callCrabRPCFunction</div
<div> *args, **kwargs)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 184, in callCrabRPCFunction</div
<div> rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH,
timeou=
t)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 150, in _recvAll</div <div>
raise Timeout()</div <div>Timeout</div <div>Thread-206::DEBUG::2014-09-02
12:19:05,695::remoteFileHandler::260::Re=
poFileHelper.PoolHandler::(stop) Pool handler existed, OUT: '' ERR:
''</div=
<div>Thread-210::WARNING::2014-09-02
12:19:05,745::fileSD::673::scanDomains=
::(collectMetaFiles) Metadata collection for domain path /rhev/data-center/=
mnt/10.0.0.30:_mnt_kvm1_export timedout</div
<div>Traceback (most recent call last):</div
<div> File "/usr/share/vdsm/storage/fileSD.py",
line 662, i=
n collectMetaFiles</div <div>
sd.DOMAIN_META_DATA))</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 297, in callCrabRPCFunction</div
<div> *args, **kwargs)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 184, in callCrabRPCFunction</div
<div> rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH,
timeou=
t)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 150, in _recvAll</div <div>
raise Timeout()</div <div>Timeout</div <div><br
<br </div <div>Given that this storage domain doesn't exist any
more and is not visib=
le in the web interface, how can I get this host to stop trying to connect =
to it, initialize and become online?<br
</div <p><br </p <p><br </p <div
id=3D"Signature" <div
name=3D"divtagdefaultwrapper" style=3D"font-family:Calibri,Arial,Helve=
tica,sans-serif; font-size:; margin:0" <span
style=3D"color:rgb(34,34,34); font-family:Arial,Helvetica,sans-serif;=
font-size:12px; background-color:rgb(255,255,255)">James</span><br </div
</div </div </div <p
style=3D"font-family:Arial,Helvetica,sans-serif;font-size:12px;color:#18=
52a0;line-height:1.2em" <i><a
href=3D"http://www.edigitalresearch.com/news/item/nid/878290232"...
h How to Turn Data into Action to Improve Customer Experiences in an Agile =
World</a <br
<br We are delighted to be ranked 32nd
in ‘The Sunday Times Top 100 Best =
Small Companies to Work For 2014‘ list.
</i></p <p style=3D"font-family:
Arial,Helvetica,sans-serif;font-size:12px;color:#0=
00;line-height:1.2em" This message is sent in confidence
for the addressee only. The contents are=
not to be disclosed to anyone other than the addressee.
<br Unauthorised recipients must preserve this confidentiality and
should pleas=
e advise the sender immediately of any error in transmission.
<br <br Any attachment(s) to this message
have been checked for viruses, but please=
rely on your own virus checker and procedures.
<br <br Please note that Internet email is
not a secure communications medium. We a=
dvise that you understand and observe this when emailing us.
</p <p style=3D"font-family: Arial, Helvetica,
sans-serif;font-size: 12px; colo=
r:#000;line-height:1.2em" eDigitalResearch plc is a public
limited company registered at the Registra=
r Of Companies for England and Wales. Company registration number: 5424597
<br Registered Office: Vanbrugh House, Hedge End, Hampshire, SO30
2AF </p <p style=3D"color:#31CC5A;
font-size:10px;font-family: Arial, Helvetica, sa=
ns-serif;" PS: Save paper - do you really need
to print this email?</p </body </html
--_000_140965710620221419edigitalresearchcom_--
9:36 p.m.
New subject: hosted engine - can't contact destroyed storage
--_000_140965776275196045edigitalresearchcom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Might as well admit to fixing it myself. SSH'd to the host and saw that th=
is share was mounted. Forced an umount and now the host is up.
Weird, but working.
Thanks,
James
-----------------------------------------------------------
From: James Clarke
Sent: 02 September 2014 12:25
To: users(a)ovirt.org
Subject: FW: hosted engine - can't contact destroyed storage
?Hi All!
I decommissioned a NFS export domain this morning, ended up 'destroying' it=
through the web interface as detaching kept failing.? Now one of my hosts=
keeps flipping between 'Non Operational' and 'Unassigned'. All of the
VMs=
on this host are still running. I am in global-maintenance to prevent mig=
rations etc.
vdsm.log seems to indicate that it is related to connecting to the destroye=
d storage domain:
Thread-206::WARNING::2014-09-02 12:19:05,692::fileSD::673::scanDomains::(co=
llectMetaFiles) Metadata collection for domain path /rhev/data-center/mnt/1=
0.0.0.30:_mnt_kvm1_export timedout
Traceback (most recent call last):
File "/usr/share/vdsm/storage/fileSD.py", line 662, in collectMetaFiles
sd.DOMAIN_META_DATA))
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 297, in callCra=
bRPCFunction
*args, **kwargs)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 184, in callCra=
bRPCFunction
rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH, timeout)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 150, in _recvAl=
l
raise Timeout()
Timeout
Thread-206::DEBUG::2014-09-02 12:19:05,695::remoteFileHandler::260::RepoFil=
eHelper.PoolHandler::(stop) Pool handler existed, OUT: '' ERR: ''
Thread-210::WARNING::2014-09-02 12:19:05,745::fileSD::673::scanDomains::(co=
llectMetaFiles) Metadata collection for domain path /rhev/data-center/mnt/1=
0.0.0.30:_mnt_kvm1_export timedout
Traceback (most recent call last):
File "/usr/share/vdsm/storage/fileSD.py", line 662, in collectMetaFiles
sd.DOMAIN_META_DATA))
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 297, in callCra=
bRPCFunction
*args, **kwargs)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 184, in callCra=
bRPCFunction
rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH, timeout)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 150, in _recvAl=
l
raise Timeout()
Timeout
Given that this storage domain doesn't exist any more and is not visible in=
the web interface, how can I get this host to stop trying to connect to it=
, initialize and become online?
James
Watch How to Turn Data into Action to Improve Customer Experiences in an Ag=
ile World<http://www.edigitalresearch.com/news/item/nid/878290232
We are delighted to be ranked 32nd in 'The Sunday Times Top 100 Best Small =
Companies to Work For 2014' list.
This message is sent in confidence for the addressee only. The contents are=
not to be disclosed to anyone other than the addressee.
Unauthorised recipients must preserve this confidentiality and should pleas=
e advise the sender immediately of any error in transmission.
Any attachment(s) to this message have been checked for viruses, but please=
rely on your own virus checker and procedures.
Please note that Internet email is not a secure communications medium. We a=
dvise that you understand and observe this when emailing us.
eDigitalResearch plc is a public limited company registered at the Registra=
r Of Companies for England and Wales. Company registration number: 5424597
Registered Office: Vanbrugh House, Hedge End, Hampshire, SO30 2AF
PS: Save paper - do you really need to print this email?
--_000_140965776275196045edigitalresearchcom_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3Diso-8859-=
1" <style type=3D"text/css"
style=3D"display:none"><!-- p { margin-top: 0px; m=
argin-bottom: 0px; }--></style
</head <body dir=3D"ltr"
style=3D"font-size:12pt;color:#000000;background-color:#F=
FFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;"
<p>Might as well admit to fixing it myself. SSH'd to the host
and saw=
that this share was mounted. Forced an umount and now the host is up=
.</p <p><br
</p <p>Weird, but
working.</p <p><br </p <p>Thanks,<br </p <p>James<br </p
<p>-----------------------------------------------------------</p <p><b style=3D"font-size: 11pt; font-family:
Calibri, sans-serif;"><br
</b></p <p><b
style=3D"font-size: 11pt; font-family: Calibri, sans-serif;">From:</b=
<div
dir=3D"ltr" style=3D"font-size:12pt; color:#000000; background-color:#=
FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif" <div
id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri,
sans-serif" co=
lor=3D"#000000" style=3D"font-size:11pt"><b>Sent:</b>
02 September 2014 12:=
25<br <b>To:</b>
users(a)ovirt.org<br <b>Subject:</b> FW:
hosted engine - can't contact destroyed storage</font
<div> </div
</div <div <div
dir=3D"ltr" style=3D"font-size:12pt; color:#000000; background-color:#=
FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif"
<div <p>​Hi
All!<br </p
<p><br </p
<p>I decommissioned a NFS export domain this morning, ended up
'destroying'=
it through the web interface as detaching kept failing.​ <span=
style=3D"font-size:12pt">Now one of my hosts keeps flipping between 'Non
O=
perational' and 'Unassigned'. All of the VMs
on this host are still running. I am in global-maintenance to preven=
t migrations etc.</span></p
<p><span style=3D"font-size:12pt"><br </span></p
<p><span style=3D"font-size:12pt">vdsm.log seems to indicate
that it is&nbs=
p;related to connecting to the destroyed storage domain:</span></p <p><br
</p
<div>Thread-206::WARNING::2014-09-02 12:19:05,692::fileSD::673::scanDomains=
::(collectMetaFiles) Metadata collection for domain path /rhev/data-center/=
mnt/10.0.0.30:_mnt_kvm1_export timedout</div
<div>Traceback (most recent call last):</div
<div> File "/usr/share/vdsm/storage/fileSD.py",
line 662, i=
n collectMetaFiles</div <div>
sd.DOMAIN_META_DATA))</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 297, in callCrabRPCFunction</div
<div> *args, **kwargs)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 184, in callCrabRPCFunction</div
<div> rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH,
timeou=
t)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 150, in _recvAll</div <div>
raise Timeout()</div <div>Timeout</div <div>Thread-206::DEBUG::2014-09-02
12:19:05,695::remoteFileHandler::260::Re=
poFileHelper.PoolHandler::(stop) Pool handler existed, OUT: '' ERR:
''</div=
<div>Thread-210::WARNING::2014-09-02
12:19:05,745::fileSD::673::scanDomains=
::(collectMetaFiles) Metadata collection for domain path /rhev/data-center/=
mnt/10.0.0.30:_mnt_kvm1_export timedout</div
<div>Traceback (most recent call last):</div
<div> File "/usr/share/vdsm/storage/fileSD.py",
line 662, i=
n collectMetaFiles</div <div>
sd.DOMAIN_META_DATA))</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 297, in callCrabRPCFunction</div
<div> *args, **kwargs)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 184, in callCrabRPCFunction</div
<div> rawLength =3D self._recvAll(LENGTH_STRUCT_LENGTH,
timeou=
t)</div <div> File
"/usr/share/vdsm/storage/remoteFileHandler.py", =
line 150, in _recvAll</div <div>
raise Timeout()</div <div>Timeout</div <div><br
<br </div <div>Given that this storage domain doesn't exist any
more and is not visib=
le in the web interface, how can I get this host to stop trying to connect =
to it, initialize and become online?<br
</div <p><br </p <p><br </p <div
id=3D"Signature" <div
name=3D"divtagdefaultwrapper" style=3D"font-family:Calibri,Arial,Helve=
tica,sans-serif; font-size:; margin:0" <span
style=3D"color:rgb(34,34,34); font-family:Arial,Helvetica,sans-serif;=
font-size:12px; background-color:rgb(255,255,255)">James</span><br </div
</div </div </div
</div </div <p
style=3D"font-family:Arial,Helvetica,sans-serif;font-size:12px;color:#18=
52a0;line-height:1.2em" <i><a
href=3D"http://www.edigitalresearch.com/news/item/nid/878290232"...
h How to Turn Data into Action to Improve Customer Experiences in an Agile =
World</a <br
<br We are delighted to be ranked 32nd
in ‘The Sunday Times Top 100 Best =
Small Companies to Work For 2014‘ list.
</i></p <p style=3D"font-family:
Arial,Helvetica,sans-serif;font-size:12px;color:#0=
00;line-height:1.2em" This message is sent in confidence
for the addressee only. The contents are=
not to be disclosed to anyone other than the addressee.
<br Unauthorised recipients must preserve this confidentiality and
should pleas=
e advise the sender immediately of any error in transmission.
<br <br Any attachment(s) to this message
have been checked for viruses, but please=
rely on your own virus checker and procedures.
<br <br Please note that Internet email is
not a secure communications medium. We a=
dvise that you understand and observe this when emailing us.
</p <p style=3D"font-family: Arial, Helvetica,
sans-serif;font-size: 12px; colo=
r:#000;line-height:1.2em" eDigitalResearch plc is a public
limited company registered at the Registra=
r Of Companies for England and Wales. Company registration number: 5424597
<br Registered Office: Vanbrugh House, Hedge End, Hampshire, SO30
2AF </p <p style=3D"color:#31CC5A;
font-size:10px;font-family: Arial, Helvetica, sa=
ns-serif;" PS: Save paper - do you really need
to print this email?</p </body </html
--_000_140965776275196045edigitalresearchcom_--
<span style=3D"font-size: 11pt; font-family: Calibri,
sans-serif;"> James =
Clarke</span></p
5:31 p.m.
New subject: hosted engine - can't contact destroyed storage
I'd just like to add a note that this problem is not directly related
with the fact that it's hosted engine.
--Jirka
On 09/02/2014 01:36 PM, James Clarke wrote:
Might as well admit to fixing it myself. SSH'd to the host and
saw that
this share was mounted. Forced an umount and now the host is up.
Weird, but working.
Thanks,
James
-----------------------------------------------------------
*
*
*From:*James Clarke
*Sent:* 02 September 2014 12:25
*To:* users(a)ovirt.org
*Subject:* FW: hosted engine - can't contact destroyed storage
Hi All!
I decommissioned a NFS export domain this morning, ended up 'destroying'
it through the web interface as detaching kept failing. Now one of my
hosts keeps flipping between 'Non Operational' and 'Unassigned'. All of
the VMs on this host are still running. I am in global-maintenance to
prevent migrations etc.
vdsm.log seems to indicate that it is related to connecting to the
destroyed storage domain:
Thread-206::WARNING::2014-09-02
12:19:05,692::fileSD::673::scanDomains::(collectMetaFiles) Metadata
collection for domain path
/rhev/data-center/mnt/10.0.0.30:_mnt_kvm1_export timedout
Traceback (most recent call last):
File "/usr/share/vdsm/storage/fileSD.py", line 662, in collectMetaFiles
sd.DOMAIN_META_DATA))
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 297, in
callCrabRPCFunction
*args, **kwargs)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 184, in
callCrabRPCFunction
rawLength = self._recvAll(LENGTH_STRUCT_LENGTH, timeout)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 150, in
_recvAll
raise Timeout()
Timeout
Thread-206::DEBUG::2014-09-02
12:19:05,695::remoteFileHandler::260::RepoFileHelper.PoolHandler::(stop)
Pool handler existed, OUT: '' ERR: ''
Thread-210::WARNING::2014-09-02
12:19:05,745::fileSD::673::scanDomains::(collectMetaFiles) Metadata
collection for domain path
/rhev/data-center/mnt/10.0.0.30:_mnt_kvm1_export timedout
Traceback (most recent call last):
File "/usr/share/vdsm/storage/fileSD.py", line 662, in collectMetaFiles
sd.DOMAIN_META_DATA))
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 297, in
callCrabRPCFunction
*args, **kwargs)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 184, in
callCrabRPCFunction
rawLength = self._recvAll(LENGTH_STRUCT_LENGTH, timeout)
File "/usr/share/vdsm/storage/remoteFileHandler.py", line 150, in
_recvAll
raise Timeout()
Timeout
Given that this storage domain doesn't exist any more and is not visible
in the web interface, how can I get this host to stop trying to connect
to it, initialize and become online?
James
/Watch How to Turn Data into Action to Improve Customer Experiences in
an Agile World <http://www.edigitalresearch.com/news/item/nid/878290232>
We are delighted to be ranked 32nd in ‘The Sunday Times Top 100 Best
Small Companies to Work For 2014‘ list. /
This message is sent in confidence for the addressee only. The contents
are not to be disclosed to anyone other than the addressee.
Unauthorised recipients must preserve this confidentiality and should
please advise the sender immediately of any error in transmission.
Any attachment(s) to this message have been checked for viruses, but
please rely on your own virus checker and procedures.
Please note that Internet email is not a secure communications medium.
We advise that you understand and observe this when emailing us.
eDigitalResearch plc is a public limited company registered at the
Registrar Of Companies for England and Wales. Company registration
number: 5424597
Registered Office: Vanbrugh House, Hedge End, Hampshire, SO30 2AF
PS: Save paper - do you really need to print this email?
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
3525
days inactive
3526
days old
2 comments
2 participants
participants (2)
-
James Clarke -
Jiri Moskovcak