<div dir="ltr"><div>Same here!<br><br></div>An reboot of the nfs machine hosting Export domain, caused the same..Upgrading to 3.4.1 on F19 and then some tweking on my storage domain Gluster based , which mean time became " split-brain" - some brick remove then add and so on - finally solved it.<br>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, May 9, 2014 at 4:55 PM, Nicolas Ecarnot <span dir="ltr"><<a href="mailto:nicolas@ecarnot.net" target="_blank">nicolas@ecarnot.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
On our second oVirt setup in 3.4.0-1.el6 (that was running fine), I did a yum upgrade on the engine (...sigh...).<br>
Then rebooted the engine.<br>
This machine is hosting the NFS export domain.<br>
Though the VM are still running, the storage domain is in invalid status. You'll find below the engine.log.<br>
<br>
At first sight, I thought it was the same issue as :<br>
<a href="http://lists.ovirt.org/pipermail/users/2014-March/022161.html" target="_blank">http://lists.ovirt.org/<u></u>pipermail/users/2014-March/<u></u>022161.html</a><br>
because it looked very similar.<br>
But the NFS export domain connection seemed OK (tested).<br>
I did try every trick I could thought of, restarting, checking anything...<br>
Our cluster stayed in a broken state.<br>
<br>
On second sight, I saw that when rebooting the engine, then NFS export domain was not mounted correctly (I wrote a static /dev/sd-something in fstab, and the iscsi manager changed the letter. Next time, I'll use LVM or a label).<br>
So the NFS served was void/empty/black hole.<br>
<br>
I just realized all the above, and spent my afternoon in cold sweat.<br>
Correcting the NFS mounting and restarting the engine did the trick.<br>
What still disturbs me is that the unavailability of the NFS export domain should NOT be a reason for the MASTER storage domain to break!<br>
<br>
Following the URL above and the BZ opened by the user (<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1072900" target="_blank">https://bugzilla.redhat.com/<u></u>show_bug.cgi?id=1072900</a>), I see this has been corrected in 3.4.1. What gives a perfectly connected NFS export domain, but empty?<br>
<br>
PS : I see no 3.4.1 update on CentOS repo.<br>
<br>
Regards,<br>
<br>
--------------------------<br>
<br>
<br>
The engine log :<br>
<br>
<br>
2014-05-09 14:40:37,767 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>SpmStartVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] spmStart polling started: taskId = 6d612398-fdad-49f2-9874-<u></u>5f32a9bf87e2 <br>
│<br>
20│2014-05-09 14:40:40,848 ERROR [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>HSMGetTaskStatusVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] Failed in HSMGetTaskStatusVDS method <br>
│<br>
20│2014-05-09 14:40:40,850 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>SpmStartVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] spmStart polling ended: taskId = 6d612398-fdad-49f2-9874-<u></u>5f32a9bf87e2 task status = finished <br>
│<br>
20│2014-05-09 14:40:40,850 ERROR [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>SpmStartVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] Start SPM Task failed - result: cleanSuccess, message: VDSGenericException: VDSErrorException: Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist, code = 358 │<br>
20│2014-05-09 14:40:40,913 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>SpmStartVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] spmStart polling ended, spm status: Free <br>
│<br>
20│2014-05-09 14:40:40,932 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>HSMClearTaskVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] START, HSMClearTaskVDSCommand(<u></u>HostName = serv-vm-adm17, HostId = 049943eb-2bcc-4167-a780-<u></u>7ef76a1f95e9, taskId=6d612398-fdad-49f2-<u></u>9874-5f32a9bf87e2), log id: 5cfdc8ce │<br>
20│2014-05-09 14:40:40,982 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>HSMClearTaskVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] FINISH, HSMClearTaskVDSCommand, log id: 5cfdc8ce <br>
<br>
│<br>
20│2014-05-09 14:40:40,983 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>SpmStartVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [f685ea4] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.<u></u>businessentities.<u></u>SpmStatusResult@39471ba9, log id: 58ec77ee │<br>
20│2014-05-09 14:40:40,985 INFO [org.ovirt.engine.core.bll.<u></u>storage.<u></u>SetStoragePoolStatusCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 5849b030-626e-47cb-ad90-<u></u>3ce782d831b3 Type: StoragePool │<br>
20│2014-05-09 14:40:41,009 INFO [org.ovirt.engine.core.dal.<u></u>dbbroker.auditloghandling.<u></u>AuditLogDirector] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] Correlation ID: 6b69119f, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Etat-Major3. Setting status to Non Responsive. │<br>
20│2014-05-09 14:40:41,017 ERROR [org.ovirt.engine.core.<u></u>vdsbroker.irsbroker.<u></u>IrsBrokerCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] IrsBroker::Failed::<u></u>GetStoragePoolInfoVDS due to: IrsSpmStartFailedException: IRSGenericException: IRSErrorException: SpmStart failed │al<br>
se│2014-05-09 14:40:41,112 INFO [org.ovirt.engine.core.<u></u>vdsbroker.irsbroker.<u></u>IrsBrokerCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] Irs placed on server 049943eb-2bcc-4167-a780-<u></u>7ef76a1f95e9 failed. Proceed Failover <br>
│<br>
20│2014-05-09 14:40:41,206 INFO [org.ovirt.engine.core.<u></u>vdsbroker.irsbroker.<u></u>IrsBrokerCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] hostFromVds::selectedVds - serv-vm-adm16, spmStatus Free, storage pool Etat-Major3 <br>
│<br>
20│2014-05-09 14:40:41,209 INFO [org.ovirt.engine.core.<u></u>vdsbroker.irsbroker.<u></u>IrsBrokerCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] starting spm on vds serv-vm-adm16, storage pool Etat-Major3, prevId -1, LVER -1 <br>
│<br>
20│2014-05-09 14:40:41,227 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>SpmStartVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] START, SpmStartVDSCommand(HostName = serv-vm-adm16, HostId = 13a2bc0a-979a-4fcd-8597-<u></u>06131030d9a0, storagePoolId = 5849b030-626e-47cb-ad90-<u></u>3ce782d831b3, prevId=-1, prevLVER=-1, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFenci│<br>
20│ng=false), log id: 67d013a4 <br>
<br>
<br>
<br>
│<br>
20│2014-05-09 14:40:41,292 INFO [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>SpmStartVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] spmStart polling started: taskId = 1046fd3e-71e4-4fcd-bbd0-<u></u>f17cd6dc08e4 <br>
│<br>
20│2014-05-09 14:40:44,438 ERROR [org.ovirt.engine.core.<u></u>vdsbroker.vdsbroker.<u></u>HSMGetTaskStatusVDSCommand] (DefaultQuartzScheduler_<u></u>Worker-28) [6b69119f] Failed in HSMGetTaskStatusVDS method <br><span class="HOEnZb"><font color="#888888">
<br>
<br>
-- <br>
Nicolas Ecarnot<br>
______________________________<u></u>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<u></u>mailman/listinfo/users</a><br>
</font></span></blockquote></div><br></div>