<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Simone,<div class=""><br class=""></div><div class="">Yes... I guess it was not clear in my original email. I changed the numbers myself to lower the timeout and retries. With them set as they were set by ovirt (timeout=3600 retry=5) things were not working for me. <br class=""><div class="">
<div id="signature" class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div>
</div>
<br class=""><div><blockquote type="cite" class=""><div class="">On Sep 29, 2016, at 10:04 AM, Simone Tiraboschi <<a href="mailto:stirabos@redhat.com" class="">stirabos@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Thu, Sep 29, 2016 at 12:47 PM, Martin Perina <span dir="ltr" class=""><<a href="mailto:mperina@redhat.com" target="_blank" class="">mperina@redhat.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class="">Hi,<br class=""><br class=""></div><div class="">please take a look at my inline comments:<br class=""></div><div class="gmail_extra"><br class=""><div class="gmail_quote"><span class="gmail-">On Tue, Sep 27, 2016 at 7:23 PM, Gervais de Montbrun <span dir="ltr" class=""><<a href="mailto:gervais@demontbrun.com" target="_blank" class="">gervais@demontbrun.com</a>></span> wrote:<br class=""><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class="">Hey All,<div class=""><br class=""></div><div class="">Since updating to 4.0.x of oVirt, I have had an issue with my hosted engine. After a some poking around, I think I have figured out my issue and thought I would share to see what others think.</div><div class="">The issue has existed with 4.0, 4.0.1, 4.0.2, 4.0.3, and still exists in 4.0.4.</div><div class=""><br class=""></div><div class="">Description:</div><div class="">When my hosted engine starts it reports that it is in a degraded state with 7 or 8 services still not started when I run systemctl status. It takes about 6 or 7 minutes to eventually start all the services and come online. If I don't set my cluster to Global-Maintenance mode it eventually thinks that my hosted-engine needs to be rebooted and restarts it before it can start everything.</div></div></blockquote></span><div class=""><br class=""><div class="">Could you please share with us logs gathered by ovirt-log-collector?<br class=""><br class="">It's just a guess but could you please take a look if you HE VM has enough entropy?<br class=""><br class=""> cat /proc/sys/kernel/random/<wbr class="">entropy_avail<br class=""><br class=""></div><div class="">If the value is low (below or around 200), you really need to install and configure some entropy generator such as haveged<br class=""><br class=""></div></div><span class="gmail-"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class=""><div class=""><br class=""></div><div class="">Solution:</div><div class="">I realized that Apache was the culprit and found that the proxy to the ovirt-engine in /etc/httpd/conf.d/z-ovirt-e<wbr class="">ngine-proxy.conf has a super long timeout with many retries. I changed the settings and now everything works for me.</div><div class=""><br class=""></div><div class="">-> Before change:</div><blockquote style="margin:0px 0px 0px 40px;border-width:medium;border-style:none;padding:0px" class=""><div class=""><div class=""> <LocationMatch ^/(ovirt-engine($|/)|api($|/)|<wbr class="">RHEVManagerWeb/|OvirtEngineWeb<wbr class="">/|ca.crt$|engine.ssh.key.txt$|<wbr class="">rhevm.ssh.key.txt$)></div><div class=""> ProxyPassMatch <a class="">ajp://127.0.0.1:8702</a> timeout=3600 retry=5</div><div class=""><br class=""></div><div class=""> <IfModule deflate_module></div><div class=""> AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml</div><div class=""> </IfModule></div><div class=""> </LocationMatch></div></div></blockquote><div class=""><br class=""></div>-> After change:<blockquote style="margin:0px 0px 0px 40px;border-width:medium;border-style:none;padding:0px" class=""><div class=""><div class=""> <LocationMatch ^/ovirt-engine($|/)></div><div class=""> ProxyPassMatch <a class="">ajp://127.0.0.1:8702</a> timeout=5 retry=2</div><div class=""><br class=""></div><div class=""> <IfModule deflate_module></div><div class=""> AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml</div><div class=""> </IfModule></div><div class=""> </LocationMatch></div></div></blockquote></div></blockquote></span><div class=""><br class=""><div style="font-family:arial,helvetica,sans-serif;display:inline" class="">This one is correct for 4.0</div><div style="font-family:arial,helvetica,sans-serif;display:inline" class="">, not sure why it was not updated during upgrade from 3.6. @Simone?<br class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Honestly it's</div><div class=""><div class=""> <LocationMatch ^/ovirt-engine($|/)></div><div class=""> ProxyPassMatch ajp://<a href="http://127.0.0.1:8702/" class="">127.0.0.1:8702</a> timeout=3600 retry=5</div><div class=""><br class=""></div><div class=""> <IfModule deflate_module></div><div class=""> AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml</div><div class=""> </IfModule></div><div class=""> </LocationMatch></div></div><div class="">also on a fresh 4.0 engine from our latest engine-appliance.</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class=""><div class=""><br class=""></div>If I read the timeout settings correctly, it will wait 60 minutes with 5 retries. 5 hours is way too long for my little server to hold onto all those apache processes. </div></blockquote><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class="">The change I made allows for there to be an error, and also releases apache's hold on the process. Once everything is ready, apache is ready to serve requests and everything/everyone is happy. Before making the change, I just get a whitescreen in my browser and then nothing works until I restart Apache (or I end up in an endless loop of ovirt-ha services restarting my hosted-engine.<br class=""></div></blockquote></span><div class=""><br class=""><div style="font-family:arial,helvetica,sans-serif;display:inline" class="">Well, if you have an issue with too many apache processes waiting for engine to respond, then there's some issue in engine. As I wrote above please share the logs with us and check entropy.<br class=""><br class=""></div><div style="font-family:arial,helvetica,sans-serif;display:inline" class="">Thanks<br class=""><br class=""></div><div style="font-family:arial,helvetica,sans-serif;display:inline" class="">Martin Perina<br class=""></div> </div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><span class="gmail-"><div style="word-wrap:break-word" class=""><div class=""><div class=""><div class=""><br class=""></div><div class="">I noticed that this setting reverts to the original setting, so oVirt must be writing this file. Perhaps these number can be changed in oVirt? If not, I will just setup and ansible play to revert the settings with working values and restart apache on my engine.</div><div class="">:-)</div><div class="">
<div class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div>
</div>
<br class=""></div></div></div><br class=""></span>______________________________<wbr class="">_________________<br class="">
Users mailing list<br class="">
<a href="mailto:Users@ovirt.org" target="_blank" class="">Users@ovirt.org</a><br class="">
<a rel="noreferrer" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" class="">http://lists.ovirt.org/mailman<wbr class="">/listinfo/users</a><br class="">
<br class=""></blockquote></div><br class=""></div></div>
</blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""></div></body></html>