<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Martin,<div class=""><br class=""></div><div class="">The entropy was super low. Somewhere around 140. I installed and configured haveged.service to start at bootup, reverted my apache changes... After a reboot, my systemctl status still says that there are 7 services queued (note that I erroneously said degraded in my previous email - the services are, in fact, queued), but the oVirt GUI comes up almost immediately and everything seems to be great.</div><div class=""><br class=""></div><div class="">Thank you for the tip. You solved my issue.<br class=""><div class="">
<div id="signature" class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div>
</div>
<br class=""><div><blockquote type="cite" class=""><div class="">On Sep 29, 2016, at 7:47 AM, Martin Perina <<a href="mailto:mperina@redhat.com" class="">mperina@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">Hi,<br class=""><br class=""></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">please take a look at my inline comments:<br class=""></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Sep 27, 2016 at 7:23 PM, Gervais de Montbrun <span dir="ltr" class=""><<a target="_blank" href="mailto:gervais@demontbrun.com" class="">gervais@demontbrun.com</a>></span> wrote:<br class=""><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class="">Hey All,<div class=""><br class=""></div><div class="">Since updating to 4.0.x of oVirt, I have had an issue with my hosted engine. After a some poking around, I think I have figured out my issue and thought I would share to see what others think.</div><div class="">The issue has existed with 4.0, 4.0.1, 4.0.2, 4.0.3, and still exists in 4.0.4.</div><div class=""><br class=""></div><div class="">Description:</div><div class="">When my hosted engine starts it reports that it is in a degraded state with 7 or 8 services still not started when I run systemctl status. It takes about 6 or 7 minutes to eventually start all the services and come online. If I don't set my cluster to Global-Maintenance mode it eventually thinks that my hosted-engine needs to be rebooted and restarts it before it can start everything.</div></div></blockquote><div class=""><br class=""><div style="font-family:arial,helvetica,sans-serif;display:inline" class="gmail_default">Could you please share with us logs gathered by ovirt-log-collector?<br class=""><br class="">It's just a guess but could you please take a look if you HE VM has enough entropy?<br class=""><br class=""> cat /proc/sys/kernel/random/entropy_avail<br class=""><br class=""></div><div style="font-family:arial,helvetica,sans-serif;display:inline" class="gmail_default">If the value is low (below or around 200), you really need to install and configure some entropy generator such as haveged<br class=""><br class=""></div></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class=""><div class=""><br class=""></div><div class="">Solution:</div><div class="">I realized that Apache was the culprit and found that the proxy to the ovirt-engine in /etc/httpd/conf.d/z-ovirt-<wbr class="">engine-proxy.conf has a super long timeout with many retries. I changed the settings and now everything works for me.</div><div class=""><br class=""></div><div class="">-> Before change:</div><blockquote style="margin:0px 0px 0px 40px;border-width:medium;border-style:none;border-color:-moz-use-text-color;padding:0px" class=""><div class=""><div class=""> <LocationMatch ^/(ovirt-engine($|/)|api($|/)|<wbr class="">RHEVManagerWeb/|<wbr class="">OvirtEngineWeb/|ca.crt$|<wbr class="">engine.ssh.key.txt$|rhevm.ssh.<wbr class="">key.txt$)></div><div class=""> ProxyPassMatch <a class="">ajp://127.0.0.1:8702</a> timeout=3600 retry=5</div><div class=""><br class=""></div><div class=""> <IfModule deflate_module></div><div class=""> AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml</div><div class=""> </IfModule></div><div class=""> </LocationMatch></div></div></blockquote><div class=""><br class=""></div>-> After change:<blockquote style="margin:0px 0px 0px 40px;border-width:medium;border-style:none;border-color:-moz-use-text-color;padding:0px" class=""><div class=""><div class=""> <LocationMatch ^/ovirt-engine($|/)></div><div class=""> ProxyPassMatch <a class="">ajp://127.0.0.1:8702</a> timeout=5 retry=2</div><div class=""><br class=""></div><div class=""> <IfModule deflate_module></div><div class=""> AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml</div><div class=""> </IfModule></div><div class=""> </LocationMatch></div></div></blockquote></div></blockquote><div class=""><br class=""><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">This one is correct for 4.0</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">, not sure why it was not updated during upgrade from 3.6. @Simone?<br class=""></div> </div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class=""><div class=""><br class=""></div>If I read the timeout settings correctly, it will wait 60 minutes with 5 retries. 5 hours is way too long for my little server to hold onto all those apache processes. </div></blockquote><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class="">The change I made allows for there to be an error, and also releases apache's hold on the process. Once everything is ready, apache is ready to serve requests and everything/everyone is happy. Before making the change, I just get a whitescreen in my browser and then nothing works until I restart Apache (or I end up in an endless loop of ovirt-ha services restarting my hosted-engine.<br class=""></div></blockquote><div class=""><br class=""><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Well, if you have an issue with too many apache processes waiting for engine to respond, then there's some issue in engine. As I wrote above please share the logs with us and check entropy.<br class=""><br class=""></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Thanks<br class=""><br class=""></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Martin Perina<br class=""></div> </div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="word-wrap:break-word" class=""><div class=""><div class=""><div class=""><br class=""></div><div class="">I noticed that this setting reverts to the original setting, so oVirt must be writing this file. Perhaps these number can be changed in oVirt? If not, I will just setup and ansible play to revert the settings with working values and restart apache on my engine.</div><div class="">:-)</div><div class="">
<div class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div>
</div>
<br class=""></div></div></div><br class="">______________________________<wbr class="">_________________<br class="">
Users mailing list<br class="">
<a href="mailto:Users@ovirt.org" class="">Users@ovirt.org</a><br class="">
<a target="_blank" rel="noreferrer" href="http://lists.ovirt.org/mailman/listinfo/users" class="">http://lists.ovirt.org/<wbr class="">mailman/listinfo/users</a><br class="">
<br class=""></blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""></div></body></html>