--Apple-Mail=_36EF5248-AA3D-484E-8A71-55F0C8DF8023 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hey All, Since updating to 4.0.x of oVirt, I have had an issue with my hosted = engine. After a some poking around, I think I have figured out my issue = and thought I would share to see what others think. The issue has existed with 4.0, 4.0.1, 4.0.2, 4.0.3, and still exists in = 4.0.4. Description: When my hosted engine starts it reports that it is in a degraded state = with 7 or 8 services still not started when I run systemctl status. It = takes about 6 or 7 minutes to eventually start all the services and come = online. If I don't set my cluster to Global-Maintenance mode it = eventually thinks that my hosted-engine needs to be rebooted and = restarts it before it can start everything. Solution: I realized that Apache was the culprit and found that the proxy to the = ovirt-engine in /etc/httpd/conf.d/z-ovirt-engine-proxy.conf has a super = long timeout with many retries. I changed the settings and now = everything works for me. -> Before change: <LocationMatch = ^/(ovirt-engine($|/)|api($|/)|RHEVManagerWeb/|OvirtEngineWeb/|ca.crt$|engi= ne.ssh.key.txt$|rhevm.ssh.key.txt$)> ProxyPassMatch ajp://127.0.0.1:8702 timeout=3D3600 retry=3D5 <IfModule deflate_module> AddOutputFilterByType DEFLATE text/javascript text/css = text/html text/xml text/json application/xml application/json = application/x-yaml </IfModule> </LocationMatch> -> After change: <LocationMatch ^/ovirt-engine($|/)> ProxyPassMatch ajp://127.0.0.1:8702 timeout=3D5 retry=3D2 <IfModule deflate_module> AddOutputFilterByType DEFLATE text/javascript text/css = text/html text/xml text/json application/xml application/json = application/x-yaml </IfModule> </LocationMatch> If I read the timeout settings correctly, it will wait 60 minutes with 5 = retries. 5 hours is way too long for my little server to hold onto all = those apache processes. The change I made allows for there to be an = error, and also releases apache's hold on the process. Once everything = is ready, apache is ready to serve requests and everything/everyone is = happy. Before making the change, I just get a whitescreen in my browser = and then nothing works until I restart Apache (or I end up in an endless = loop of ovirt-ha services restarting my hosted-engine. I noticed that this setting reverts to the original setting, so oVirt = must be writing this file. Perhaps these number can be changed in oVirt? = If not, I will just setup and ansible play to revert the settings with = working values and restart apache on my engine. :-) Cheers, Gervais --Apple-Mail=_36EF5248-AA3D-484E-8A71-55F0C8DF8023 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D"">Hey All,<div class=3D""><br class=3D""></div><div = class=3D"">Since updating to 4.0.x of oVirt, I have had an issue with my = hosted engine. After a some poking around, I think I have figured out my = issue and thought I would share to see what others think.</div><div = class=3D"">The issue has existed with 4.0, 4.0.1, 4.0.2, 4.0.3, and = still exists in 4.0.4.</div><div class=3D""><br class=3D""></div><div = class=3D"">Description:</div><div class=3D"">When my hosted engine = starts it reports that it is in a degraded state with 7 or 8 services = still not started when I run systemctl status. It takes about 6 or 7 = minutes to eventually start all the services and come online. If I don't = set my cluster to Global-Maintenance mode it eventually thinks that my = hosted-engine needs to be rebooted and restarts it before it can start = everything.</div><div class=3D""><br class=3D""></div><div = class=3D"">Solution:</div><div class=3D"">I realized that Apache was the = culprit and found that the proxy to the ovirt-engine = in /etc/httpd/conf.d/z-ovirt-engine-proxy.conf has a super long = timeout with many retries. I changed the settings and now everything = works for me.</div><div class=3D""><br class=3D""></div><div = class=3D"">-> Before change:</div><blockquote style=3D"margin: 0 0 0 = 40px; border: none; padding: 0px;" class=3D""><div class=3D""><div = class=3D""> <LocationMatch = ^/(ovirt-engine($|/)|api($|/)|RHEVManagerWeb/|OvirtEngineWeb/|ca.crt$|engi= ne.ssh.key.txt$|rhevm.ssh.key.txt$)></div><div class=3D""> = ProxyPassMatch <a href=3D"ajp://127.0.0.1:8702" = class=3D"">ajp://127.0.0.1:8702</a> timeout=3D3600 retry=3D5</div><div = class=3D""><br class=3D""></div><div class=3D""> = <IfModule deflate_module></div><div class=3D""> = AddOutputFilterByType DEFLATE = text/javascript text/css text/html text/xml text/json application/xml = application/json application/x-yaml</div><div class=3D""> = </IfModule></div><div class=3D""> = </LocationMatch></div></div></blockquote><div class=3D""><br = class=3D""></div>-> After change:<blockquote style=3D"margin: 0 0 0 = 40px; border: none; padding: 0px;" class=3D""><div class=3D""><div = class=3D""> <LocationMatch = ^/ovirt-engine($|/)></div><div class=3D""> = ProxyPassMatch <a href=3D"ajp://127.0.0.1:8702" = class=3D"">ajp://127.0.0.1:8702</a> timeout=3D5 retry=3D2</div><div = class=3D""><br class=3D""></div><div class=3D""> = <IfModule deflate_module></div><div class=3D""> = AddOutputFilterByType DEFLATE = text/javascript text/css text/html text/xml text/json application/xml = application/json application/x-yaml</div><div class=3D""> = </IfModule></div><div class=3D""> = </LocationMatch></div></div></blockquote><div class=3D""><br = class=3D""></div>If I read the timeout settings correctly, it will wait = 60 minutes with 5 retries. 5 hours is way too long for my little server = to hold onto all those apache processes. The change I made allows for = there to be an error, and also releases apache's hold on the process. = Once everything is ready, apache is ready to serve requests and = everything/everyone is happy. Before making the change, I just get a = whitescreen in my browser and then nothing works until I restart Apache = (or I end up in an endless loop of ovirt-ha services restarting my = hosted-engine.<br class=3D""><div class=3D""><div class=3D""><div = class=3D""><br class=3D"webkit-block-placeholder"></div><div class=3D"">I = noticed that this setting reverts to the original setting, so oVirt must = be writing this file. Perhaps these number can be changed in oVirt? If = not, I will just setup and ansible play to revert the settings with = working values and restart apache on my engine.</div><div = class=3D"">:-)</div><div class=3D""> <div id=3D"signature" class=3D""><br class=3D"">Cheers,<br = class=3D"">Gervais<br class=3D""><br class=3D""><br class=3D""></div> </div> <br class=3D""></div></div></body></html>= --Apple-Mail=_36EF5248-AA3D-484E-8A71-55F0C8DF8023--