<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif"><span style="font-family:arial">On Mon, Feb 3, 2014 at 11:23 PM, Itamar Heim </span><span dir="ltr" style="font-family:arial">&lt;<a href="mailto:iheim@redhat.com" target="_blank">iheim@redhat.com</a>&gt;</span><span style="font-family:arial"> wrote:</span><br>

</div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 02/03/2014 01:19 PM, Andrew Lau wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The issue was a split-brain issue on the dom_md/ids file causing an<br>
input/output error, thanks!<br>
</blockquote>
<br></div>
is this with gluster?<br></blockquote><div><br></div><div><div class="gmail_default" style="font-family:tahoma,sans-serif">​Yup a 2 brick gluster replicated instance serving the NFS server, sorry was meant to say I resolved it too.​</div>

<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
<br>
On Mon, Feb 3, 2014 at 10:43 PM, Andrew Lau &lt;<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a><br></div><div class="im">
&lt;mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>&gt;<u></u>&gt; wrote:<br>
<br>
    On Mon, Feb 3, 2014 at 10:40 PM, Doron Fediuck &lt;<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a><br></div><div class="im">
    &lt;mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>&gt;&gt; wrote:<br>
<br>
<br>
<br>
        ----- Original Message -----<br>
         &gt; From: &quot;Andrew Lau&quot; &lt;<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a><br></div><div class="im">
        &lt;mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>&gt;<u></u>&gt;<br>
         &gt; To: &quot;Doron Fediuck&quot; &lt;<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a><br></div><div class="im">
        &lt;mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>&gt;&gt;<br>
         &gt; Cc: &quot;users&quot; &lt;<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a> &lt;mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>&gt;&gt;, &quot;Jiri<br>
        Moskovcak&quot; &lt;<a href="mailto:jmoskovc@redhat.com" target="_blank">jmoskovc@redhat.com</a> &lt;mailto:<a href="mailto:jmoskovc@redhat.com" target="_blank">jmoskovc@redhat.com</a>&gt;&gt;,<br>
        &quot;Greg Padgett&quot; &lt;<a href="mailto:gpadgett@redhat.com" target="_blank">gpadgett@redhat.com</a> &lt;mailto:<a href="mailto:gpadgett@redhat.com" target="_blank">gpadgett@redhat.com</a>&gt;&gt;<br>
         &gt; Sent: Monday, February 3, 2014 1:35:01 PM<br>
         &gt; Subject: Re: [Users] Hosted Engine always reports &quot;unknown<br>
        stale-data&quot;<br>
         &gt;<br>
         &gt; On Mon, Feb 3, 2014 at 9:53 PM, Doron Fediuck<br></div><div class="im">
        &lt;<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a> &lt;mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>&gt;&gt; wrote:<br>
         &gt;<br>
         &gt; &gt;<br>
         &gt; &gt;<br>
         &gt; &gt; ----- Original Message -----<br>
         &gt; &gt; &gt; From: &quot;Andrew Lau&quot; &lt;<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a><br></div><div><div class="h5">
        &lt;mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>&gt;<u></u>&gt;<br>
         &gt; &gt; &gt; To: &quot;users&quot; &lt;<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a> &lt;mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>&gt;&gt;<br>
         &gt; &gt; &gt; Sent: Monday, February 3, 2014 12:32:45 PM<br>
         &gt; &gt; &gt; Subject: [Users] Hosted Engine always reports &quot;unknown<br>
        stale-data&quot;<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; Hi,<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; I was wondering if anyone has this same notice when they run:<br>
         &gt; &gt; &gt; hosted-engine --vm-status<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; The &quot;engine status&quot; will always be &quot;unknown stale-data&quot;<br>
        even when the VM<br>
         &gt; &gt; is<br>
         &gt; &gt; &gt; powered on and the engine is online. engine-health will<br>
        actually report<br>
         &gt; &gt; the<br>
         &gt; &gt; &gt; correct status.<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; eg.<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; --== Host 1 status ==--<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; Status up-to-date : False<br>
         &gt; &gt; &gt; Hostname : 172.16.0.11<br>
         &gt; &gt; &gt; Host ID : 1<br>
         &gt; &gt; &gt; Engine status : unknown stale-data<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; Is it some sort of blocked port causing this or is this<br>
        by design?<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; Thanks,<br>
         &gt; &gt; &gt; Andrew<br>
         &gt; &gt; &gt;<br>
         &gt; &gt; &gt; ______________________________<u></u>_________________<br>
         &gt; &gt; &gt; Users mailing list<br></div></div>
         &gt; &gt; &gt; <a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a> &lt;mailto:<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a>&gt;<div><div class="h5"><br>
         &gt; &gt; &gt; <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<u></u>mailman/listinfo/users</a><br>
         &gt; &gt; &gt;<br>
         &gt; &gt;<br>
         &gt; &gt; Hi Andrew,<br>
         &gt; &gt; it looks like an issue with the time stamp.<br>
         &gt; &gt; Which time stamp do you have? How relevant is it?<br>
         &gt; &gt;<br>
         &gt;<br>
         &gt; timestamps seem to be outdated by a lot, interesting error in<br>
        the broker.log<br>
         &gt;<br>
         &gt; Thread-24::INFO::2014-02-03<br>
         &gt;<br>
        22:33:14,801::engine_health::<u></u>90::engine_health.<u></u>CpuLoadNoEngine::(action)<br>
        VM<br>
         &gt; not running on this host, status down<br>
         &gt; Thread-22::INFO::2014-02-03<br>
         &gt; 22:33:14,834::mem_free::53::<u></u>mem_free.MemFree::(action)<br>
        memFree: 27382<br>
         &gt; Thread-23::ERROR::2014-02-03<br>
         &gt;<br>
        22:33:14,922::cpu_load_no_<u></u>engine::156::cpu_load_no_<u></u>engine.EngineHealth::(update_<u></u>stat_file)<br>
         &gt; Failed to getVmStats: &#39;pid&#39;<br>
         &gt; Thread-23::INFO::2014-02-03<br>
         &gt;<br>
        22:33:14,923::cpu_load_no_<u></u>engine::121::cpu_load_no_<u></u>engine.EngineHealth::(<u></u>calculate_load)<br>
         &gt; System load total=0.0124, engine=0.0000, non-engine=0.0124<br>
         &gt;<br>
         &gt; I&#39;m assuming that update_stat_file is the metadata file the<br>
        vm-status is<br>
         &gt; getting pulled from?<br>
         &gt;<br>
<br>
        Yep.<br>
        Can you please verify the time your host actually has?<br>
        ie- we have a known issue with time, since we assume all<br>
        hosts are in sync. So if one of your hosts has a time sync<br>
        issue, this can explain the problem you see.<br>
<br>
<br>
    --== Host 1 status ==--<br>
<br>
    Status up-to-date                  : False<br>
    Hostname                           : 172.16.0.11<br>
    Host ID                            : 1<br>
    Engine status                      : unknown stale-data<br>
    Score                              : 0<br>
    Local maintenance                  : False<br>
    Host timestamp                     : 1391417611<br>
<br>
    --== Host 2 status ==--<br>
<br>
    Status up-to-date                  : False<br>
    Hostname                           : 172.16.0.12<br>
    Host ID                            : 2<br>
    Engine status                      : unknown stale-data<br>
    Score                              : 0<br>
    Local maintenance                  : False<br>
    Host timestamp                     : 1391417171<br>
<br>
<br>
    ​<br>
    [root@hv01 ~]# date +%s<br>
                             │[root@hv02 ~]# date +%s<br>
    ​​<br>
    1391427754<br>
                            │139142775<br>
    ​5​<br>
<br>
    ​​<br>
<br>
<br>
<br>
<br></div></div>
______________________________<u></u>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<u></u>mailman/listinfo/users</a><br>
<br>
</blockquote>
<br>
</blockquote></div><br></div></div>