<div dir="auto"><div><br><div class="gmail_extra"><br><div class="gmail_quote">Il 13 dic 2017 8:19 PM, "Yaniv Kaul" <<a href="mailto:ykaul@redhat.com">ykaul@redhat.com</a>> ha scritto:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div class="elided-text">On Wed, Dec 13, 2017 at 4:15 PM, Luca 'remix_tj' Lorenzetto <span dir="ltr"><<a href="mailto:lorenzetto.luca@gmail.com" target="_blank">lorenzetto.luca@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div>Today i started troubleshooting more in depth on dns requests and exactly while i was looking at tcpdump an event of EngineUp -> EngineBadHealth happened.</div><div><br></div><div>Looking at the dns requests i see this:</div><div><br></div><div>[...]</div><div><div>14:30:35.909201 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>55654 > dns.company.it.53: 34102+ A? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:30:35.909215 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>55654 > dns.company.it.53: 6242+ AAAA? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:30:40.914285 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>55654 > dns.company.it.53: 34102+ A? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:30:40.914316 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>55654 > dns.company.it.53: 6242+ AAAA? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:30:45.918306 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>54885 > dns.company.it.53: 60263+ A? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:30:45.918329 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>54885 > dns.company.it.53: 18681+ AAAA? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:30:50.920376 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>54885 > dns.company.it.53: 60263+ A? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:30:50.920411 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>54885 > dns.company.it.53: 18681+ AAAA? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:30:56.<a href="tel:0442%2042" value="+39044242" target="_blank">044242</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58319 > dns.company.it.53: 28413+ A? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:30:56.<a href="tel:0442%2067" value="+39044267" target="_blank">044267</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58319 > dns.company.it.53: 29680+ AAAA? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:31:01.<a href="tel:049%20761" value="+39049761" target="_blank">049761</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58319 > dns.company.it.53: 28413+ A? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:31:01.<a href="tel:049%20777" value="+39049777" target="_blank">049777</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58319 > dns.company.it.53: 29680+ AAAA? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:31:<a href="tel:06%20052635" value="+3906052635" target="_blank">06.052635</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58093 > dns.company.it.53: 24807+ A? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:31:<a href="tel:06%20052649" value="+3906052649" target="_blank">06.052649</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58093 > dns.company.it.53: 53745+ AAAA? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:31:11.<a href="tel:0577%2024" value="+39057724" target="_blank">057724</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58093 > dns.company.it.53: 24807+ A? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:31:11.<a href="tel:0577%2045" value="+39057745" target="_blank">057745</a> IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>58093 > dns.company.it.53: 53745+ AAAA? <a href="http://engine01.intranet.company.it.intranet.company.it" target="_blank">engine01.intranet.company.it.i<wbr>ntranet.company.it</a>. (74)</div><div>14:31:16.175204 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>44950 > dns.company.it.53: 63680+ A? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:31:16.175225 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>44950 > dns.company.it.53: 15726+ AAAA? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:31:19.670746 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>54689 > dns.company.it.53: 40999+ A? <a href="http://kvmsvilca01.intranet.company.it" target="_blank">kvmsvilca01.intranet.company.i<wbr>t</a>. (49)</div><div>14:31:21.180295 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>44950 > dns.company.it.53: 63680+ A? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:31:21.180337 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>44950 > dns.company.it.53: 15726+ AAAA? <a href="http://engine01.intranet.company.it" target="_blank">engine01.intranet.company.it</a>. (54)</div><div>14:31:23.771959 IP <a href="http://kvmhost01.intranet.company.it" target="_blank">kvmhost01.intranet.company.it</a>.<wbr>53741 > dns.company.it.53: 1707+ A? <a href="http://internalmx.intranet.company.it" target="_blank">internalmx.intranet.company.it</a><wbr>. (48)</div></div><div>[...]</div><div><br></div><div>The last dns requests has success and gets the MX address and immediately after i get the email reporting the status change.</div></div></blockquote><div><br></div></div><div>Can you ensure it doesn't have multiple IPs registered for it in DNS?</div><div>dig or so should help.</div><font color="#888888"><div>Y.</div><div> </div></font></div></div></div></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">No, it's not. A single ip is registered. It's for sure a dns query missing its replies.</div><div dir="auto"><br></div><div dir="auto">I'm debugging with network team on what's happening.</div><div dir="auto"><br></div><div dir="auto">Anyway, i think that Broker log in debug Mode should help identifying the source if this errors.</div><div dir="auto">Maybe explaining better why liveness check has failed will reduce the troubleshooting experiments.</div><div dir="auto"><br></div><div dir="auto">Luca</div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="quoted-text"><div dir="ltr"><div><br></div><div>This is clearly an issue with name resolution, but that's not clear to me from the broker.log file. The only message about it that i get is:</div><div><br></div><div><div>Thread-16::DEBUG::2017-12-13 14:31:23,657::monitor::126::ov<wbr><a href="http://irt_hosted_engine_ha.broker.mo">irt_hosted_engine_ha.broker.mo</a><wbr>nitor.Monitor::(get_value) Submonitor engine-health id 139653</div><div>412040592 current value: {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "up"}</div><div>Thread-16::DEBUG::2017-12-13 14:31:23,657::listener::170::o<wbr>virt_hosted_engine_ha.broker.l<wbr>istener.ConnectionHandler::(ha<wbr>ndle) Response: success {"reaso</div><div>n": "failed liveliness check", "health": "bad", "vm": "up", "detail": "up"}</div></div><div><br></div><div><br></div><div>But around that messages i get no signals of error on dns queries or similar. Do i need to check on other log files?</div><span class="m_-3470669771316353166HOEnZb"><font color="#888888"><div><br></div><div>Luca</div><div><br></div></font></span><div class="gmail_extra"><span><br><div class="gmail_quote">On Mon, Dec 11, 2017 at 3:34 PM, Luca 'remix_tj' Lorenzetto <span dir="ltr"><<a href="mailto:lorenzetto.luca@gmail.com" target="_blank">lorenzetto.luca@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Martin, Hi all,<div><br></div><div>*some minutes* has passed and i've the piece of log i'm looking at.</div><div><br></div><div><br><div class="gmail_chip gmail_drive_chip" style="width:396px;height:18px;max-height:18px;background-color:rgb(245,245,245);padding:5px;color:rgb(34,34,34);font-family:arial;font-style:normal;font-weight:bold;font-size:13px;border:1px solid rgb(221,221,221);line-height:1"><a href="https://drive.google.com/file/d/1wlWZPuhgtJRBWt4xUZC-Jis8vLWM1jYD/view?usp=drive_web" style="display:inline-block;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;text-decoration:none;padding:1px 0px;border:none;width:100%" target="_blank"><img style="vertical-align:bottom;border:none" src="https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png"> <span dir="ltr" style="color:rgb(17,85,204);text-decoration:none;vertical-align:bottom">broker.log-upbadup</span></a><div class="chip_delete_button"><div class="chip_delete_button_icon"></div></div></div><br></div><div><br></div></div></blockquote><div><br></div><div> </div></div><div><br></div></span><span>-- <br><div class="m_-3470669771316353166m_5560473697925200975gmail_signature">"E' assurdo impiegare gli uomini di intelligenza eccellente per fare<br>calcoli che potrebbero essere affidati a chiunque se si usassero delle<br>macchine"<br>Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716)<br><br>"Internet è la più grande biblioteca del mondo.<br>Ma il problema è che i libri sono tutti sparsi sul pavimento"<br>John Allen Paulos, Matematico (1945-vivente)<br> <br>Luca 'remix_tj' Lorenzetto, <a href="http://www.remixtj.net" target="_blank">http://www.remixtj.net</a> , <<a href="mailto:lorenzetto.luca@gmail.com" target="_blank">lorenzetto.luca@gmail.com</a>></div>
</span></div></div>
<br></div><div class="quoted-text">______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br></div></blockquote></div><br></div></div>
</blockquote></div><br></div></div></div>