<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif"><span style="font-family:arial">On Mon, Feb 3, 2014 at 11:27 PM, Itamar Heim </span><span dir="ltr" style="font-family:arial"><<a href="mailto:iheim@redhat.com" target="_blank">iheim@redhat.com</a>></span><span style="font-family:arial"> wrote:</span><br>
</div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 02/03/2014 01:25 PM, Andrew Lau wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
On Mon, Feb 3, 2014 at 11:23 PM, Itamar Heim <<a href="mailto:iheim@redhat.com" target="_blank">iheim@redhat.com</a><br></div>
<mailto:<a href="mailto:iheim@redhat.com" target="_blank">iheim@redhat.com</a>>><u></u>wrote:<div class="im"><br>
<br>
On 02/03/2014 01:19 PM, Andrew Lau wrote:<br>
<br>
The issue was a split-brain issue on the dom_md/ids file causing an<br>
input/output error, thanks!<br>
<br>
<br>
is this with gluster?<br>
<br>
<br>
Yup a 2 brick gluster replicated instance serving the NFS server, sorry<br>
was meant to say I resolved it too.<br>
</div></blockquote>
<br>
you have to use a gluster with quorum, or this will happen often<div class="gmail_default" style="font-family:tahoma,sans-serif;display:inline"></div></blockquote><div><br></div><div><div class="gmail_default" style="font-family:tahoma,sans-serif">
Yeah, I disabled quorum temporarily because I'm only using a two host scenario and I need to have the case scenario where one is to be shutdown the VMs won't end up in a paused state.</div></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_default" style="font-family:tahoma,sans-serif;display:inline"></div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
On Mon, Feb 3, 2014 at 10:43 PM, Andrew Lau<br>
<<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a> <mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>><br>
<mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a> <mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>><u></u>>__><br>
wrote:<br>
<br>
On Mon, Feb 3, 2014 at 10:40 PM, Doron Fediuck<br>
<<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a> <mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>><br>
<mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a> <mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>>>><br>
wrote:<br>
<br>
<br>
<br>
----- Original Message -----<br>
> From: "Andrew Lau" <<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a><br>
<mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>><br>
<mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a><br>
<mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>><u></u>>__><br>
> To: "Doron Fediuck" <<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a><br>
<mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>><br>
<mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a> <mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>>>><br>
> Cc: "users" <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>
<mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>> <mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>
<mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>>>>, "Jiri<br>
Moskovcak" <<a href="mailto:jmoskovc@redhat.com" target="_blank">jmoskovc@redhat.com</a><br>
<mailto:<a href="mailto:jmoskovc@redhat.com" target="_blank">jmoskovc@redhat.com</a>> <mailto:<a href="mailto:jmoskovc@redhat.com" target="_blank">jmoskovc@redhat.com</a><br>
<mailto:<a href="mailto:jmoskovc@redhat.com" target="_blank">jmoskovc@redhat.com</a>>>><u></u>,<br>
"Greg Padgett" <<a href="mailto:gpadgett@redhat.com" target="_blank">gpadgett@redhat.com</a><br>
<mailto:<a href="mailto:gpadgett@redhat.com" target="_blank">gpadgett@redhat.com</a>> <mailto:<a href="mailto:gpadgett@redhat.com" target="_blank">gpadgett@redhat.com</a><br>
<mailto:<a href="mailto:gpadgett@redhat.com" target="_blank">gpadgett@redhat.com</a>>>><br>
> Sent: Monday, February 3, 2014 1:35:01 PM<br>
> Subject: Re: [Users] Hosted Engine always reports<br>
"unknown<br>
stale-data"<br>
><br>
> On Mon, Feb 3, 2014 at 9:53 PM, Doron Fediuck<br>
<<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a> <mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>><br>
<mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a> <mailto:<a href="mailto:dfediuck@redhat.com" target="_blank">dfediuck@redhat.com</a>>>> wrote:<br>
><br>
> ><br>
> ><br>
> > ----- Original Message -----<br>
> > > From: "Andrew Lau" <<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a><br>
<mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>><br>
<mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a><br>
<mailto:<a href="mailto:andrew@andrewklau.com" target="_blank">andrew@andrewklau.com</a>><u></u>>__><br>
> > > To: "users" <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>
<mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>> <mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a><br>
<mailto:<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>>>><br>
> > > Sent: Monday, February 3, 2014 12:32:45 PM<br>
> > > Subject: [Users] Hosted Engine always reports<br>
"unknown<br>
stale-data"<br>
> > ><br>
> > > Hi,<br>
> > ><br>
> > > I was wondering if anyone has this same notice<br>
when they run:<br>
> > > hosted-engine --vm-status<br>
> > ><br>
> > > The "engine status" will always be "unknown<br>
stale-data"<br>
even when the VM<br>
> > is<br>
> > > powered on and the engine is online.<br>
engine-health will<br>
actually report<br>
> > the<br>
> > > correct status.<br>
> > ><br>
> > > eg.<br>
> > ><br>
> > > --== Host 1 status ==--<br>
> > ><br>
> > > Status up-to-date : False<br>
> > > Hostname : 172.16.0.11<br>
> > > Host ID : 1<br>
> > > Engine status : unknown stale-data<br>
> > ><br>
> > > Is it some sort of blocked port causing this or<br>
is this<br>
by design?<br>
> > ><br>
> > > Thanks,<br>
> > > Andrew<br>
> > ><br>
> > > ______________________________<u></u>___________________<br>
> > > Users mailing list<br>
> > > <a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a>><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a>>><br>
<br>
> > > <a href="http://lists.ovirt.org/__mailman/listinfo/users" target="_blank">http://lists.ovirt.org/__<u></u>mailman/listinfo/users</a><br>
<<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<u></u>mailman/listinfo/users</a>><br>
> > ><br>
> ><br>
> > Hi Andrew,<br>
> > it looks like an issue with the time stamp.<br>
> > Which time stamp do you have? How relevant is it?<br>
> ><br>
><br>
> timestamps seem to be outdated by a lot, interesting<br>
error in<br>
the broker.log<br>
><br>
> Thread-24::INFO::2014-02-03<br>
><br>
<br>
22:33:14,801::engine_health::_<u></u>_90::engine_health.__<u></u>CpuLoadNoEngine::(action)<br>
VM<br>
> not running on this host, status down<br>
> Thread-22::INFO::2014-02-03<br>
> 22:33:14,834::mem_free::53::__<u></u>mem_free.MemFree::(action)<br>
memFree: 27382<br>
> Thread-23::ERROR::2014-02-03<br>
><br>
<br>
22:33:14,922::cpu_load_no___<u></u>engine::156::cpu_load_no___<u></u>engine.EngineHealth::(update__<u></u>_stat_file)<br>
> Failed to getVmStats: 'pid'<br>
> Thread-23::INFO::2014-02-03<br>
><br>
<br>
22:33:14,923::cpu_load_no___<u></u>engine::121::cpu_load_no___<u></u>engine.EngineHealth::(__<u></u>calculate_load)<br>
> System load total=0.0124, engine=0.0000,<br>
non-engine=0.0124<br>
><br>
> I'm assuming that update_stat_file is the metadata<br>
file the<br>
vm-status is<br>
> getting pulled from?<br>
><br>
<br>
Yep.<br>
Can you please verify the time your host actually has?<br>
ie- we have a known issue with time, since we assume all<br>
hosts are in sync. So if one of your hosts has a time sync<br>
issue, this can explain the problem you see.<br>
<br>
<br>
--== Host 1 status ==--<br>
<br>
Status up-to-date : False<br>
Hostname : 172.16.0.11<br>
Host ID : 1<br>
Engine status : unknown stale-data<br>
Score : 0<br>
Local maintenance : False<br>
Host timestamp : 1391417611<br>
<br>
--== Host 2 status ==--<br>
<br>
Status up-to-date : False<br>
Hostname : 172.16.0.12<br>
Host ID : 2<br>
Engine status : unknown stale-data<br>
Score : 0<br>
Local maintenance : False<br>
Host timestamp : 1391417171<br>
<br>
<br>
<br>
[root@hv01 ~]# date +%s<br>
│[root@hv02 ~]# date +%s<br>
<br>
1391427754<br>
│139142775<br>
5<br>
<br>
<br>
<br>
<br>
<br>
<br>
______________________________<u></u>___________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a>><br>
<a href="http://lists.ovirt.org/__mailman/listinfo/users" target="_blank">http://lists.ovirt.org/__<u></u>mailman/listinfo/users</a><br>
<<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<u></u>mailman/listinfo/users</a>><br>
<br>
<br>
<br>
</blockquote>
<br>
</blockquote></div><br></div></div>