3:08 a.m.
--_000_CO2PR0801MB0743D5BED04E03821D47364FA6700CO2PR0801MB0743_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
Spoke too soon. Some hosts came back up but the storage domain is still loc=
ked so no vm=92s can be started. What is the proper way to force this to be=
unlocked? Each time we look to move into production after successful testi=
ng, something like this always seems to pop up at the last minute rending o=
Virt questionable in terms of reliability for some unknown issue.
From: Bill Bill<mailto:jax2568@outlook.com>
Sent: Friday, January 20, 2017 7:54 PM
To: users<mailto:users@ovirt.org>
Subject: RE: master storage domain stuck in locked state
So apparently something didn=92t change the metadata to master before conne=
ction was lost. I changed the metadata role to master and it came backup. S=
eems emailing in helped because every time I can=92t figure something out, =
email in a find it shortly after.
From: Bill Bill<mailto:jax2568@outlook.com>
Sent: Friday, January 20, 2017 7:43 PM
To: users<mailto:users@ovirt.org>
Subject: master storage domain stuck in locked state
No clue how to get this out. I can mount all storage manually on the hyperv=
isors. It seems like after a reboot oVirt is now having some issue and the =
storage domain is stuck in locked state. Because of this, can=92t activate =
any other storage either, so the other domains are in maintenance and the m=
aster sits in locked state, has been for hours.
This sticks out on a hypervisor:
StoragePoolWrongMaster: Wrong Master domain or its version: u'SD=3Dd8a0172e=
-837f-4552-92c7-566dc4e548e4, pool=3D3fd2ad92-e1eb-49c2-906d-00ec233f610a'
Not sure, nothing changed other than a reboot of the storage.
Engine log shows:
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzSche=
duler8) [5696732b] START, SetVdsStatusVDSCommand(HostName =3D U31U32NodeA, =
SetVdsStatusVDSCommandParameters:{runAsync=3D'true', hostId=3D'70e2b8e4-075=
2-47a8-884c-837a00013e79', status=3D'NonOperational', nonOperationalReason=
=3D'STORAGE_DOMAIN_UNREACHABLE', stopSpmFailureLogged=3D'false',
maintenanc=
eReason=3D'null'}), log id: 6db9820a
No idea why it says unreachable, it certainly is because I can manually mou=
nt ALL storage to the hypervisor.
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=3D550986> for Window=
s 10
--_000_CO2PR0801MB0743D5BED04E03821D47364FA6700CO2PR0801MB0743_
Content-Type: text/html; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
<html xmlns:o=3D"urn:schemas-microsoft-com:office:office"
xmlns:w=3D"urn:sc=
hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of=
fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html;
charset=3DWindows-1=
252">
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang=3D"EN-US" link=3D"blue" vlink=3D"#954F72">
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Spoke too soon. Some hosts came back up but the
stor=
age domain is still locked so no vm=92s can be started. What is the proper =
way to force this to be unlocked? Each time we look to move into production=
after successful testing, something
like this always seems to pop up at the last minute rending oVirt question=
able in terms of reliability for some unknown issue.</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<div style=3D"mso-element:para-border-div;border:none;border-top:solid #E1E=
1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class=3D"MsoNormal"
style=3D"border:none;padding:0in"><b>From: </b><a hr=
ef=3D"mailto:jax2568@outlook.com">Bill Bill</a><br>
<b>Sent: </b>Friday, January 20, 2017 7:54 PM<br>
<b>To: </b><a
href=3D"mailto:users@ovirt.org">users</a><br>
<b>Subject: </b>RE: master storage domain stuck in locked state</p>
</div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
<div>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">So apparently something didn=92t change the
metadata=
to master before connection was lost. I changed the metadata role to maste=
r and it came backup. Seems emailing in helped because every time I can=92t=
figure something out, email in a find
it shortly after.</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<div style=3D"mso-element:para-border-div;border:none;border-top:solid #E1E=
1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class=3D"MsoNormal"
style=3D"border:none;padding:0in"><b>From: </b><a hr=
ef=3D"mailto:jax2568@outlook.com">Bill Bill</a><br>
<b>Sent: </b>Friday, January 20, 2017 7:43 PM<br>
<b>To: </b><a
href=3D"mailto:users@ovirt.org">users</a><br>
<b>Subject: </b>master storage domain stuck in locked state</p>
</div>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div class=3D"WordSection1">
<p class=3D"MsoNormal">No clue how to get this out. I can mount all
storage=
manually on the hypervisors. It seems like after a reboot oVirt is now hav=
ing some issue and the storage domain is stuck in locked state. Because of =
this, can=92t activate any other storage
either, so the other domains are in maintenance and the master sits in loc=
ked state, has been for hours.</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">This sticks out on a hypervisor:</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">StoragePoolWrongMaster: Wrong Master domain or its
v=
ersion: u'SD=3Dd8a0172e-837f-4552-92c7-566dc4e548e4, pool=3D3fd2ad92-e1eb-4=
9c2-906d-00ec233f610a'</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Not sure, nothing changed other than a reboot of
the=
storage.</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Engine log shows:</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p
class=3D"MsoNormal">[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSComm=
and] (DefaultQuartzScheduler8) [5696732b] START, SetVdsStatusVDSCommand(Hos=
tName =3D U31U32NodeA, SetVdsStatusVDSCommandParameters:{runAsync=3D'true',=
hostId=3D'70e2b8e4-0752-47a8-884c-837a00013e79',
status=3D'NonOperational', nonOperationalReason=3D'STORAGE_DOMAIN_UNREACHA=
BLE', stopSpmFailureLogged=3D'false', maintenanceReason=3D'null'}),
log id:=
6db9820a</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">No idea why it says unreachable, it certainly is
bec=
ause I can manually mount ALL storage to the hypervisor.</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Sent from <a
href=3D"https://go.microsoft.com/fwlink=
/?LinkId=3D550986">
Mail</a> for Windows 10</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>
--_000_CO2PR0801MB0743D5BED04E03821D47364FA6700CO2PR0801MB0743_--