<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Simone,<div class=""><br class=""></div><div class="">Thanks for the info. I'll look at the solution that you suggested.<br class=""><div class="">
<div id="signature" class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div>
</div>
<br class=""><div><blockquote type="cite" class=""><div class="">On Sep 29, 2016, at 10:01 AM, Simone Tiraboschi <<a href="mailto:stirabos@redhat.com" class="">stirabos@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><br class="Apple-interchange-newline"><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">On Thu, Sep 29, 2016 at 2:51 PM, Gervais de Montbrun<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:gervais@demontbrun.com" target="_blank" class="">gervais@demontbrun.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class="">Hi Martin,<div class=""><br class=""></div><div class="">The entropy was super low. Somewhere around 140. I installed and configured haveged.service to start at bootup, reverted my apache changes... After a reboot, my systemctl status still says that there are 7 services queued (note that I erroneously said degraded in my previous email - the services are, in fact, queued), but the oVirt GUI comes up almost immediately and everything seems to be great.</div><div class=""><br class=""></div></div></blockquote><div class="gmail_quote"><br class=""></div>Take care that using havaged on a VM should not be considered a good source of entropy and the oVirt PKi is managed by the engine.<br class=""><a href="http://security.stackexchange.com/questions/34523/is-it-appropriate-to-use-haveged-as-a-source-of-entropy-on-virtual-machines" target="_blank" class="">http://security.stackexchange.<wbr class="">com/questions/34523/is-it-<wbr class="">appropriate-to-use-haveged-as-<wbr class="">a-source-of-entropy-on-<wbr class="">virtual-machines</a><br class=""><br class="">A better approach is the virtio-rng paravirtualised rng driver as for patch <a href="https://gerrit.ovirt.org/#/c/62334/" class="">https://gerrit.ovirt.org/#/c/62334/</a></div><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""></div><div class="">Thank you for the tip. You solved my issue.<br class=""><div class=""><div class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div></div><div class=""><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Sep 29, 2016, at 7:47 AM, Martin Perina <<a href="mailto:mperina@redhat.com" target="_blank" class="">mperina@redhat.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div class="">Hi,<br class=""><br class=""></div><div class="">please take a look at my inline comments:<br class=""></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Sep 27, 2016 at 7:23 PM, Gervais de Montbrun<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:gervais@demontbrun.com" target="_blank" class="">gervais@demontbrun.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class="">Hey All,<div class=""><br class=""></div><div class="">Since updating to 4.0.x of oVirt, I have had an issue with my hosted engine. After a some poking around, I think I have figured out my issue and thought I would share to see what others think.</div><div class="">The issue has existed with 4.0, 4.0.1, 4.0.2, 4.0.3, and still exists in 4.0.4.</div><div class=""><br class=""></div><div class="">Description:</div><div class="">When my hosted engine starts it reports that it is in a degraded state with 7 or 8 services still not started when I run systemctl status. It takes about 6 or 7 minutes to eventually start all the services and come online. If I don't set my cluster to Global-Maintenance mode it eventually thinks that my hosted-engine needs to be rebooted and restarts it before it can start everything.</div></div></blockquote><div class=""><br class=""><div class="">Could you please share with us logs gathered by ovirt-log-collector?<br class=""><br class="">It's just a guess but could you please take a look if you HE VM has enough entropy?<br class=""><br class=""> <span class="Apple-converted-space"> </span>cat /proc/sys/kernel/random/entrop<wbr class="">y_avail<br class=""><br class=""></div><div class="">If the value is low (below or around 200), you really need to install and configure some entropy generator such as haveged<br class=""><br class=""></div></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><br class=""></div><div class="">Solution:</div><div class="">I realized that Apache was the culprit and found that the proxy to the ovirt-engine in /etc/httpd/conf.d/z-ovirt-e<wbr class="">ngine-proxy.conf has a super long timeout with many retries. I changed the settings and now everything works for me.</div><div class=""><br class=""></div><div class="">-> Before change:</div><blockquote style="margin: 0px 0px 0px 40px; border-width: medium; border-style: none; padding: 0px;" class=""><div class=""><div class=""> <span class="Apple-converted-space"> </span><LocationMatch ^/(ovirt-engine($|/)|api($|/)|<wbr class="">RHEVManagerWeb/|OvirtEngineWeb<wbr class="">/|ca.crt$|engine.ssh.key.txt$|<wbr class="">rhevm.ssh.key.txt$)></div><div class=""> <span class="Apple-converted-space"> </span>ProxyPassMatch<span class="Apple-converted-space"> </span><a class="">ajp://127.0.0.1:8702</a><span class="Apple-converted-space"> </span>timeout=3600 retry=5</div><div class=""><br class=""></div><div class=""> <span class="Apple-converted-space"> </span><IfModule deflate_module></div><div class=""> <span class="Apple-converted-space"> </span>AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml</div><div class=""> <span class="Apple-converted-space"> </span></IfModule></div><div class=""> <span class="Apple-converted-space"> </span></LocationMatch></div></div></blockquote><div class=""><br class=""></div>-> After change:<blockquote style="margin: 0px 0px 0px 40px; border-width: medium; border-style: none; padding: 0px;" class=""><div class=""><div class=""> <span class="Apple-converted-space"> </span><LocationMatch ^/ovirt-engine($|/)></div><div class=""> <span class="Apple-converted-space"> </span>ProxyPassMatch<span class="Apple-converted-space"> </span><a class="">ajp://127.0.0.1:8702</a><span class="Apple-converted-space"> </span>timeout=5 retry=2</div><div class=""><br class=""></div><div class=""> <span class="Apple-converted-space"> </span><IfModule deflate_module></div><div class=""> <span class="Apple-converted-space"> </span>AddOutputFilterByType DEFLATE text/javascript text/css text/html text/xml text/json application/xml application/json application/x-yaml</div><div class=""> <span class="Apple-converted-space"> </span></IfModule></div><div class=""> <span class="Apple-converted-space"> </span></LocationMatch></div></div></blockquote></div></blockquote><div class=""><br class=""><div style="font-family: arial, helvetica, sans-serif; display: inline;" class="">This one is correct for 4.0</div><div style="font-family: arial, helvetica, sans-serif; display: inline;" class="">, not sure why it was not updated during upgrade from 3.6. @Simone?<br class=""></div> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><br class=""></div>If I read the timeout settings correctly, it will wait 60 minutes with 5 retries. 5 hours is way too long for my little server to hold onto all those apache processes.</div></blockquote><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class="">The change I made allows for there to be an error, and also releases apache's hold on the process. Once everything is ready, apache is ready to serve requests and everything/everyone is happy. Before making the change, I just get a whitescreen in my browser and then nothing works until I restart Apache (or I end up in an endless loop of ovirt-ha services restarting my hosted-engine.<br class=""></div></blockquote><div class=""><br class=""><div style="font-family: arial, helvetica, sans-serif; display: inline;" class="">Well, if you have an issue with too many apache processes waiting for engine to respond, then there's some issue in engine. As I wrote above please share the logs with us and check entropy.<br class=""><br class=""></div><div style="font-family: arial, helvetica, sans-serif; display: inline;" class="">Thanks<br class=""><br class=""></div><div style="font-family: arial, helvetica, sans-serif; display: inline;" class="">Martin Perina<br class=""></div> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><div class=""><div class=""><br class=""></div><div class="">I noticed that this setting reverts to the original setting, so oVirt must be writing this file. Perhaps these number can be changed in oVirt? If not, I will just setup and ansible play to revert the settings with working values and restart apache on my engine.</div><div class="">:-)</div><div class=""><div class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div></div><br class=""></div></div></div><br class="">______________________________<wbr class="">_________________<br class="">Users mailing list<br class=""><a href="mailto:Users@ovirt.org" target="_blank" class="">Users@ovirt.org</a><br class=""><a rel="noreferrer" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" class="">http://lists.ovirt.org/mailman<wbr class="">/listinfo/users</a></blockquote></div></div></div></div></blockquote></div></div></div></div></div></blockquote></div></div></blockquote></div><br class=""></div></body></html>