<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><meta name="Generator" content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#44546A;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang="EN-US" link="#0563C1" vlink="#954F72"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">(sorry: resending as I wasn’t part of the list, yet)</span></p><p class="MsoNormal"><span style="color:#44546a"> </span></p><p class="MsoNormal">hi,</p><p class="MsoNormal">this is my first post so hallo all and thank you for reading.</p><p class="MsoNormal">I have an issue with my production Ovirt environment (3.5.1.1-1.el6).</p><p class="MsoNormal"> </p><p class="MsoNormal">My system consists of several datancers.</p><p class="MsoNormal">2 of them are connected to an iSCSI SAN and they were working fine.</p><p class="MsoNormal">Until the moment I had the bad idea of deleting a SAN volume from the SAN manager before deleting the associated storage on Ovirt. From that moment, the DC where this storage was mounted became not responsive: it cannot attach the master storage (or any other).</p><p class="MsoNormal">I tried to</p><p class="MsoNormal">1) manually destroy the offending storage (select -> destroy) but still cannot recover the situation.</p><p class="MsoNormal">2) right click on master storage and activate it</p><p class="MsoNormal">3) re-initialize the datacenter using a NFS storage from the working sister DC.</p><p class="MsoNormal"> </p><p class="MsoNormal">All Hosts are still running even though their status is "unknown".</p><p class="MsoNormal">All VM are still running even though their status is "not responding".</p><p class="MsoNormal"> </p><p class="MsoNormal"><span style="color:#44546a">I half resolved the issue by manually restarting the host where the datastore was originally mounted. This cleared the orphaned multipath.</span></p><p class="MsoNormal"><span style="color:#44546a">However, the SPM does not come up still.</span></p><p class="MsoNormal"><span style="color:#44546a">This is an extract of the log</span></p><p class="MsoNormal"><i><span style="color:#44546a">2015-04-16 03:51:48,069 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-14) [61a44b19] could not stop spm of pool </span><b><span style="color:red">00000002-0002-0002-0002-00000000009c</span></b><span style="color:#44546a"> on vds </span><b><span style="color:red">89254f23-8748-402a-afc9-08438dca0975</span></b><span style="color:#44546a"> - reason: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues</span></i></p><p class="MsoNormal"><i><span style="color:#44546a">2015-04-16 03:51:48,072 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-14) [61a44b19] FINISH, SpmStopVDSCommand, log id: 4354cf46</span></i></p><p class="MsoNormal"><i><span style="color:#44546a">2015-04-16 03:51:48,072 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-14) [61a44b19] spm stop on spm failed, stopping spm selection!</span></i></p><p class="MsoNormal"><i><span style="color:#44546a">2015-04-16 03:51:58,223 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) [4ca2d938] hostFromVds::selectedVds - Brachetto, spmStatus Free, storage pool IRDC-INTEL</span></i></p><p class="MsoNormal"><i><span style="color:#44546a">2015-04-16 03:51:58,225 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) [4ca2d938] SPM Init: could not find reported vds or not up - pool:IRDC-INTEL vds_spm_id: 3</span></i></p><p class="MsoNormal"><i><span style="color:#44546a">2015-04-16 03:51:58,239 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) [4ca2d938] SPM selection - vds seems as spm sovana</span></i></p><p class="MsoNormal"><i><span style="color:#44546a">2015-04-16 03:51:58,252 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-4) [4ca2d938] START, SpmStopVDSCommand(HostName = sovana, HostId = 89254f23-8748-402a-afc9-08438dca0975, storagePoolId = 00000002-0002-0002-0002-00000000009c), log id: 63a17687</span></i></p><p class="MsoNormal"><span style="color:#44546a"> </span></p><p class="MsoNormal"><span style="color:#44546a">storagePoolId = 00000002-0002-0002-0002-00000000009c is (was) hertz-dstore2 which does not exists anymore on SAN adn ovirt</span></p><p class="MsoNormal"><span style="color:#44546a">hostid 89254f23-8748-402a-afc9-08438dca0975 is sovana server (current SPM)</span></p><p class="MsoNormal"><span style="color:#44546a"> </span></p><p class="MsoNormal"><span style="color:#44546a"> </span></p><p class="MsoNormal"><span style="color:#44546a"> </span></p><p class="MsoNormal"><span style="color:#44546a"> </span></p><p class="MsoNormal"><span style="color:#44546a">I’m thinking about </span></p><p class="MsoNormal" style="background:white"><i><span style="color:#1f497d">Put the hosted engine host into Maintenance</span></i><i><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#222222"></span></i></p><p class="MsoNormal" style="background:white"><i><span style="color:#1f497d">Shutdown Ovirt Manager</span></i><i><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#222222"></span></i></p><p class="MsoNormal" style="background:white"><i><span style="color:#1f497d">Rebooted SPM server</span></i><i><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#222222"></span></i></p><p class="MsoNormal" style="background:white"><i><span style="color:#1f497d">Restarted Ovirt Manager</span></i><i><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#222222"></span></i></p><p class="MsoNormal" style="background:white"><i><span style="color:#1f497d">Took hosted engine host out of Maintenance</span></i><i><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#222222"></span></i></p><p class="MsoNormal"><span style="color:#44546a"> </span></p><p class="MsoNormal"> </p><p class="MsoNormal">any help or clue is highly welcomed with cheers and beers</p><p class="MsoNormal">thank you!</p><p class="MsoNormal">Andrea</p></div></body></html>