Safe to upgrade HE hosts from GUI?
This is a multi-part message in MIME format. --------------D1B075435FBB036A716F94AD Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, I used to follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... when upgrading Hosted Engine (HE) but always fail to make the engine VM migrate to the fresh upgraded host as described in step 7. Furthermore, the update available icon never disappeared from the GUI. So I though using the GUI might be better for an amateur like me. Can I just follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI? Thank you, -- Wee --------------D1B075435FBB036A716F94AD Content-Type: multipart/related; boundary="------------1B6AE34A5F145C09C4C13AB6" --------------1B6AE34A5F145C09C4C13AB6 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p>Hi,</p> <p>I used to follow <a class="moz-txt-link-freetext" href="http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engine">http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engine</a> when upgrading Hosted Engine (HE) but always fail to make the engine VM migrate to the fresh upgraded host as described in step 7. Furthermore, the update available icon never disappeared from the GUI.</p> <p><img src="cid:part1.0C5A5B3F.762B762F@forest.go.th" alt=""></p> <p>So I though using the GUI might be better for an amateur like me.</p> <p>Can I just follow <a class="moz-txt-link-freetext" href="http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engine">http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engine</a> until step 3 and do everything else via GUI?</p> <p>Thank you,<br> </p> <br> <pre class="moz-signature" cols="72">-- Wee</pre> </body> </html> --------------1B6AE34A5F145C09C4C13AB6 Content-Type: image/png; name="bbdplhjmfhdddbdj.png" Content-Transfer-Encoding: base64 Content-ID: <part1.0C5A5B3F.762B762F@forest.go.th> Content-Disposition: inline; filename="bbdplhjmfhdddbdj.png" iVBORw0KGgoAAAANSUhEUgAAAQYAAAB0CAIAAADoygsVAAAZd0lEQVR4nO2c/V9TV57HM/MX 7Gt2Z/eHrc7O7My8Zt2Z7XbYUV+dzbTrCnaAgsjT5RZb6gNjU0XG2kGdVm1dH0ZLfQhSEJ9q sbaolfCgRCOgIDaKgEBICAFCgCQQEkISkhtivPvDzb05N/cmJJCQGM739fnh5uTcc7/33O/7 nO+5gcMxBmhHAMOhQYs640AkoEEDDSIBDRrNIBLQoNGME+gJEAlo0W0QCWjQaAaRgAaNZr6Q UCqVdXV1p0+fPuLFFsxLaNAWzLwioVQqL1y44A0GiAS0aDWvSNTV1fnmASIBLSrNKxI+8iWI BLQoNq9IzMqDX0ioBLwYnkBFfaCOg21ifozbPK6iEvBi+GLq+mK+/14YZTWlgIlHQuD6vGxE XFojM85ShahhlNXMVtXzlLmZSsCLieGLWT8xarI/i0CeUbAtACSYhbM3rxLwYqgOCTESXrrd jQT1KSAk3MHhR/gtsBllNWKZrMYnqQGQMI9TQFMJeDweD3jo1IdAWoh4JHwU+jKVgMcTCFzI AzdKDepEX3l8MweE2JAgeIzh8fnuWYLP57FNJN6MHhwjYnKeGBHT5g2jrKZGLK4pLS0trZGN yFwHrqGZqMgcqd3HHq35bYQ/Rg8m3DMb8SV5eaOspkY2wuYA7QzwlLn5Rjx0AdHDYj5fIAD4 YHnuKgGPz+e7y8laxDNihgqtspdmGXUYlRgtk+YXEqyF/iOhUhF9QoY4HQC+GAeKxHwej8cg aHYDEyfqdKrtGCBxmv8sARS6IDHKaqivwDpGWY0rlJjnkkfM1vw1srqXqcx1cTDwSTC8OMBo cG6+qQQ8nkAs4AtURAYkdiHh7nn6Y3fP4oxwYIaKj8pgiUcdXMwnR0HXlyxB6LIFQcKVG7o/ AHl/DO3exXyeQEUkkoFNnoxZAjzd3ZtzQAJYSngO6lQhM4aIAyCOqENXnk6N7czW/HaNquy+ DjPnYfjm6QDtNumQzM03oofFfL4YVwn41Ijo+gp87J5PhDFoMkOFWdlbs6wNUsYWhIQtROLk Cnm+wI0EM7d0DS08vpgaWgJJJ0OHhJsD7wNmIEgQRx4f/b5PyujRypqZefON6YC3qWxuvlHD mytlIpFw58TkmO0PEh6hwqzsrVnWBinzvvgM/fKatnwg/fNMbnDaOozIBQNaYLH2nbungpE4 UVkQlR1RRd6QYCZOxHGNWEwN0czW/DHPYAWjGEAXzJRAvEEHgPnLc5aYm2+uHlYJ+DwyYSaR cD8Q/5BghgorEqzNgnWAxMl1yB6EOL4QL2HBd7AxND48VjbA97R3t34ZfR6kVlnErMjzeAk7 j+U1+6LTGxJASgLGk3tcdn8OaAnruaJ2NUI5Q2tuRAwsrylYgGuRlxeLyWLylLn4xgxsKnEi HxKf72WQoidM3pbXrJWZzdJrB2F5DX+qg7Y4Df5BBzRoNIN/9gcNGs3gH4dDg0Yz+C9E0KDR DCIBDRrNWJAoLCxEvVsiYD6qQQu1nTx5cuHDZTEYCxK+nwREInJs4cNlMRhE4gW2hQ+XxWAc CcPSfRpEInIs3METnQZniRfYFj5cFoNBJF5gW/hwWQwGkXiBbeHDZTHYvJBoLMms/TSmev8r dYdevXXkVeGRPwiPvHb76OuiwtV3P19z9/iahpMJosK45pK0wo83LEyULCpb+HBZDDZ3JL7b 82/PLG2lO1/bxF36wZtLd8QvzY9fuv2NpXlxS9+PXfr+6qXvrVr63qqlWS8v3Z269Jnz+Zld q7PXr89EsqCCpWG9FSromiMSW/60qfcy4hy4Vn4w9bPtr5f+de2Zj1PL9maePfDW2U+zz//f 2xcOvfvloU0XD28q2ffWlRPvO9UPLM0fCc5+hCCoN6WnZ/j41kMiiWZUUu9//RApPT3DtyeS UZNEVOxRWCSSGkel87+60YRDzVNS6ZBHyRyRuFDy2Z2dP6o+9daYcKdTcdbZfcopK3JKTztl xU5pibO31NlX5lScc/afcyrPO7s+H+qqF5cm1R1aCTzRPaNGk6hoD1VyUtjlf6D4DsTDl6u/ 3L89pDAIWgeGOu9CJF50BQ2JLRvSTxzYJm6uH6/KcT7aM3lnq6bmvfFbWywNeabmj8fq92pu 7xq59YGyKm+w8j1J2Rs3Lx1AEHTLppyFQaJNqQVbDp0gEi+6goYEYQ1n3xu5kS0sP1J3q1Yo ul9Xd6fy/MG7x14b/CpeUfF2z1cZ3ReSO0rj206/zs9/jfFEvSJRJJKODfSO6M02m92OTbdV lxEVCo5V6s12zGY3T00MDOuJQNyYX6w2WDDMbrc7NIonGxH0Ya/WZnfYpq1m81Tt8QI055MB 7SSG2THMSjXljs7KJyabnbjQ48piNKd00m6+lL+Z+PZSs2JyVIoimx8rtDZsxoY5zJNjRfmb KSbpSOxREBeyO/TagV0kEiMDCoPZZsMcJr36VMEWOhLZdW1KKzZjw2a0Ax25EIkXHYmze9ZU fHf7sUwv6R0eUI3pTY5pJ94mNxfvjP12x08r9r1esY9bsY/79V9e5n+4JiAknjksV/fmIwh6 6MIDMzZ5KicbQXYoJ8y9Td+iCJpbcMlgdZCBuKPoi8MogubkluoxS+X+7QiCSkb1VMvN8nF1 74McBM0tKDdYzYLDO0A38g8VHszbgiDoF3US04RqG5LdMWSQN5cjCIogm+Vas0RUiiDosVPF uTnZCLKha2RS2VblBYnNp744mYOgaM7uYb2lrbKQQMIyqd6Xk40g2fUS9cRQJwogsf9ii8mo PZS7AUG2SNVGZVs1ROIFRiIvL6/yVou4teu28PaD5gfdXZLBQeWEbsLpxFUmfBf6m9Pb/pOf v+pU/qpDyE/Wr38nICRMYwMbXeWbFWOmtsrCbfurTJjhGFm5RaFjpCvuBgEkDo9ZTJXkukIy apI2ek4UhHYV1RuNml0Ieqq6c1Kr2IigaE6pwWo4lZMNVqMAYEOCJV+SjJoULVeIwo35V4yY 4RiARItCRwBG3PWUuheFSEQaEiPGmcTUrD+mIMlpSFaWLyROnDjRI5E21Dfevi3qkyumprFJ g9E6jenGxmeceH1Dc8OhV8Rlqd+Xpdd9+lu2J+oLCTDVJsKLClmP0Cw4dk1rMGN2p9OJO50Y A4lioxN3AqKHb3ZFo3TKbHM4cacTtxo1uxAUzSmasE5dyt/8hUhKDOpozm6JanzaNgO2wEQi t6BoSDtpwxxENQoJYC1RbHSaioAblIyaQN+swA1CJCICiZ4xbMWZyVVJWauSslYno2vWIgiS lZWFgmxQPNTW1g4NDXd0dCoHB8eMM80POww2fMKgb3vSodFqFTr87ql1qm/W9X2TLT692gsS 5sbSfVTJpSb5mOJ7xiyxXTlhYc4SjVLtqKQeQT7VmmxENuV9ljBeJtcGHtp/sWUam7qyNx+c JRAEbVPqJaIyyaixo7oIQdBm+fikui8vJ9vnLLFZrjUTGVpAswT17RwU9niKAs2CxKXWyTGT A0XRdRlIXHLWmhQ0PgVJSMlMSMn0QOL69euPHz/W6fSqoWGTA6/56rNHB/5FULRlUGuXSWV9 8r4RA15bVtBfniSreKftbDrrExVJNIZRGRFqeXvPjOqNLZePIK61hLW+9BOEvpYY0lukjeUI gubkHh8zzYxK6hGkcAKbbnTVbLI4bCQSpoFH14h5oEUxTkVqTu7WXCAROlbZYTZq9uVkI8iG e1LNNInEscoOg1qtM+mPuQgxjErvowiak3tkdNLqBYntygmLtPE8gqB5e68YrA4KCWot0SjV 6OlriUOXvyfXEiiCbMjL2wKRiCwkwLXEunQkPiUrMRVNTEWTUxEPJFpaWmQy2fS0TT+htzrx W2U7tMU/rStM6tc6JBKpRNLTr3VUl+1WXn5TfmW9/Oom1ieK5uQ/6VPb7A7rtN1ms7TWnkPJ rNqkUw9pJ602ux2ztFQUEfWpN07T5sknnUoiNC+Kemx2h81m16mkkqExAol9pfUmbGZ62nTr eAGas1ui0mGYfdpmnzbpL9N+r9jSrdLb7TM2m03S2KSZGCXzlsNjlpkJZQdRLbfgks51Xf29 Jumwl8RpX2m9CXPYbPYp/WhL1zCJhF76pEtvmvb2xqmiUWrGZqbNNgyz9dwLbMYIezxFgXwh 0aOzacwOCom1aUh8CpKUhianoSlpnrNEW1tba2ur04nbbPaZZ06pQiP46niXZGBEM97S8qCr q6dNrheX55nu5E7c+UBXt9WfB0z9eh2s1/YLIJZf3NPXxnK53IQ0BEGRdYlcblxyeqiuHvZ4 igJ5RaJLY7smcRSJZ3bctKxZiyBZ6BvJmfFrkTdTs95MzUrP8JwlEhMTb9y44XTiGDajUAyM jKh1Frx/cEQovN3QcE+hUF6/UVN9NPH++dz75zd/dzhh0SHBTUzJcCGRlIpmQiQiVV6RuNM3 XdWLn2/H/3p7em26G4DMTBoPIBJisXhgQOl04lrtuEIx8PRp58OHYqHwzoMHDxsb7+/du3/O 4RUFSMTGxsWnoJkAEusSuC6LXZtOVktISKQKk8kKscmZrqbWkd9yuQnrIBILi8Q37VOnxc7P HjgLbjvWpdMY8PYSNjEx8dy5C1JpL5E+jY9PDA4OyWTy69dvHDhwcMsWXhDCK+LlDYn4pLWx 8WkZFBLrErmxa9My0IwUV4lrMolPy8hAU5PiuFxubFJmpus4MSWTZGYdmkGdlQaRWEAkjokm Dt0xfCI0bqrC45KzCCoykayklAwfSIhE9VeufFtRce3hQzExRXz11eXCwuMFBbuDE14RL69I pKAp8XFJSUDilEYkVFwulxufgmaS1TJc+RU3PgXNyETTk+OIU9KS47h0i0/xTMDCHk9RIK9I WDCHdcapN89sujq5OhldlZT1P28iqxIz0zM9f64DkSAeTFtbR3//YHd3T2Pj/SCHV8TLBxIZ 6xJjY8H4jktOJ8Z7bnwKmpFGVst0LzkYSMQlpaIZGYAyIRILhQRlFuxZUiryRjKSkIJkIiw/ XzORCGF4Rbx8IZGZlsDlEmGdkuBacBNjvzckMhE3EsQ0EpuUSRYmrmNcPezxFAUK8p/9QSR8 IkEsqd3xzeVyuQmJCVxubFLm7Egg4PI6LjkNzciAiRNEIuLlzWcqycnMcIcyMwVy50KZQMQz jjMyWHiASEAkIlHh9Tns8RQF8gsJ/7dJjqbweuF8PvbZybDHUxTILyR8G7iZfqidm4PKL1/z P6rKL1+LBJ+hwqjoR8LoNxXz5yGIPkOFS4sCCaMfVASFh+D6DBUWLRYkjD6pCBYPQfcZauG1 iJAweqEiiDyEwmeoBdbiQsLIoCK4PITIZ6iF1KJDwmjCz5RdIni4/t3NBehQqBdLixEJowk/ U3YpFDyE1GeohVGQkWjqUEJBRZkgElBQNEEkoKBogkhAQdEUhUjcOvr61T//07d5P/7uw59c L/jJjV3/emPXzwV7flH98bKavb+p2f8fNz/9bfXHvxYd/e+WO5fD7i1UpCnakKgu+Gd8+vvP N/5i3c85G1dwNsRwcmI4b7/CWf8yB/0NB/01B1nGQZZxYn/M4f2Bg+O4qPTdpvb+sLsNFTmK KiS+75SPfJuCK74885ff713/y8L3V36+7ffH814/uXP1yQ9i+R++UbQroXhX0undyZ9tX33u QAY+ene6nieuORl2z6EiR1GFhLT7yf0//+DqwdXaqndx+ed4xwFcchDvPoRLDuPdR/GeY7is EJcfx/tO4AMn8fa9g+21zYW/qyxYEnbPoSJHkYUE3pqLl8bgpTFzO735YfPDu1cH5JKxinj8 wZbJGkR9LW3sRup0HWquf197a6u6erPqRs5ABdp/Je3p8Zfbbpc2dSgftHaF/TFARY6iCglC 7Vfzh7+OqzpTILhxtapWKKisusLfefPjn/eXviK/+EZX6WsdRStaC18R/+2X1SfeCkGf6mbw 55oh9VxPV1sdTr9Pn+e1ZtEU5rRazEHtGWdvaFwNoqIQifJ9sRcvV7ZIxp/2DCqUar1pxorj YunU0Y3/fuHdH36Z/8uL+T+7uP1nZzf/qLbkTyHo03mGaUCnhxaJIKpXj+E4jkMkmjqUy4Qc /72ZPxJPZCOVQnHzwyfVVZUN9Xc72tv7+xUT42M4jitN+Ja4v/vbW39/6J1fHXz7VwX/y3nw dCgEfaqbwZ9PWexEd81gGFDuMmrobR8yP6MVqq2O58THKb2uqUOpsTjI793BRBVaLZhvJKYw J1HzmcPRTl5xxuGwkOXEVZrcIYs/czgsmJMoJ2cJtdXhZLsjt7dAIYt69RiOO4eGzM8gEsuE HA7O8Z+K+SNRJ7rX1dlZd+tWdXW1TCqdstomDQab1Tau1TzD8Vt1orqCf2g+/vumE1zBB/8Y mj7VzZAh0j5kfuYKWd0MEH9kqIGF1LF74CciqReIql564RTmxL0jobE4KBKoYwJCgkmqKcBP FxsMJJ6TTbl9nsKcFAngsTe1QyQIHgj5ScU8kZApx+7cuTM4ONja2jrQ36c12uvvPTJg+IRB J/5erNaMysfxmwdfVZ5/VXY+rvlvy0LTp2Ay41oY9OoxKjqbOpTkRzDUWE6np/Jqq+P5lF4H FoKhzJCrvkez9Lh0JfcaiwMM6CmWWeI5yLMHuk3+hftiRwLkwX8q5olEj3ygu7tbp9MpBwfN Tvxa6UcPdv7g28Op/RpM0t0lk/aoDPj1E5v6zvxOcvGPj07+ITR96oHE8ym9rlePgTEHBIc7 9wCGYeJ091eUeSDhcy3hztNIY0XiuWZIrbE4wGX0rEhYLWYw5SMNIuEdCSYPflLRmhtDaG43 I+uVDw8PW61W/YTOhuM3TryjOfrDyn2/69PMdD592vm0o08zc/X4nwbK/kt6bo30UnJo+pQd CbZZAjxLbXU8J7Mp9ywBDPMu+Y0E+5srb0gENEuQfgYW34sXCW88BJRBzU1DqmGJRILjOIZh z547u+Uj35Tsb3/aO6zWNDbebW/veCQdbz6DmmpSJmpydAIkNG6wIOHHWgKsybKWIEblKb0O DCyNxeHnWqJXj/mYJfxZSzCcp60fNBYHnCXYkVgm5HBaZ5EPKua/vK6pqcFx3G63y3t7h4eH ddN4X7+yqkpQV1fXK1eUf11x9aMYET9FxF97efdvQ9OnrEgoZ33jBC5VqTrAGyeWt0P+v3Gi yGFFomn2N04sSNBTu9ljfZEiMU/NH4nOzk6FQoHjuEajkcvlT5603rt3r6qqqqGhQSgU1t5t CXuPR7w81uWLTpGFxDzXEoROnTrV3d2N4ziGYWNjY/39/RKJpLy8vOxC+cPOUPwQEXYxV9J4 QL86MzK0Of78R001gL0YvySCiiwkgiKFcvTcuXMXL168f//+/fv3q6qqSkpKvq64Ie4Ov28R K2aKtWgVhUgQ0k9Nj6o18j7FgGpRP2CoQBW1SEBBzU0QCSgomiASUFA0zQsJvcEIBRVlmhcS ZpsDCirKBJGAgqIJIgEFRRNEAgqKpihEwv/d/oae1obdW6hIU7QhEehuf0++fn/KYgu721CR o6hCwjBlnMNuf5L6krB7DhU5iiokdKOyue32F3bPoSJHkYUE9f8ScztdOyLra7upVcn93O1P 3nzRbHNMGnRhfwxQkaOoQoKQrOpDP3f7az6/OQR9Kl/OWSLs0cz1dA1vxUq/T5/ntWZR8daV vJKGIPWJy5ZvLQuRt8FSFCJx9eAf/dzt7+HXfw5Bn84zTAM6PbRIBEka3oolxSI58TF4mIVK IUdimZDjvzdBQMJqr77z2M/d/kLTp/LlnCXFJQcZg6J7pKRiQtnT8BKtUMNbsYT4SMSQsGQb +f3KLvISVCGvpMw3EsVbVxI1X1qxTUlecfmKbZ+Q5VSkdonKqJqfbF1JlJPhq+GtWMl2R25v fYz9yp6GlwDnu0RlET5RhBYJarc/P72ZPxIPxY/83+0vNH0qX06GiLKn4SVXyMqXA/FHhhpY SB27B/4uURlFAnUMFhZvXcnxjoSwZBtFAnVMQEgwSTUF+Olig4HEErIpt8/FW1dSwQ0e+5SG t2LJ4p0lPHb788eb+SJhxQLa7S80fQomM66FQZeojIpOs81BfgRDjeV0eo7hSj/AQjCUGaKl K1Sz9DFbvpyzssvmEJZsAwO6mGWWoGU+HuiaGVOBNxVvXcm434hTqJBg3e1vVm/micTIqDqg 3f5C06ceSCwpFsk9sgUggNy5BzAME6e7v6LMAwmfawl3nkYaKxJLhD0aYck2cOSeFQleSQOY 8jFTOzax8h+JCgkSPnb78+0NtUPH3G5GoegPaLe/0PQpOxJss4RnxJDZlHuWAIZ5l/xGgv3N lTckApolSD9nnxYAP1+Ad02Ego/ErLv9he5mRkfVAe32Fxo3WJDwYy0B1mRZSxCjcrFIDsa0 sGSbn2uJLlGZj1nCn7UE85URuH4QlmzzPku8AOsHUEFGws/d/rx5M//ldUC7/YWmT1mRcMz6 xglcqlJ1gDdOLG+H/H/jRJHDioR59jdOrG9RwdTO64wB3oLH66/IVLT9LuF7t7+OLknYezzi 5bEuX3SKLCTmuZYg5G23v4pr16emsbD3eAjEXElzAkpUGBnaHH/+o6YajzV9uPsnMEUWEkGR Tm9g7vZXe6tu0jQddt8iVswUa9EqCpEg5HjmHBsbHxgYnNAbwu4M1AukqEUCCmpugkhAQdEE kYCCognu9gcFRdO8kAj0XGjQIt/+H1HE0CDFdrxpAAAAAElFTkSuQmCC --------------1B6AE34A5F145C09C4C13AB6-- --------------D1B075435FBB036A716F94AD--
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
Hi,
I used to follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... when upgrading Hosted Engine (HE) but always fail to make the engine VM migrate to the fresh upgraded host as described in step 7. Furthermore, the update available icon never disappeared from the GUI.
Yes, you are right on that: it will happen only upgrading from 3.5, where maximum score for an hosted-engine hosts was 2400 points, to 3.6 where the maximum score is 3400. On 3.6.z upgrades all the hosts are already at 3400 points and so the VM will not migrate fro that.
So I though using the GUI might be better for an amateur like me.
Can I just follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI?
Yes, absolutely.
Thank you,
-- Wee
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
This is a multi-part message in MIME format. --------------8F3B0F02ECFB4A9BFFA6F4AF Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th=20 <mailto:wee.s@forest.go.th>> wrote:
Can I just follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-ho= sted-engine until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted=20 the services but the result is still the same. Kinda lost now. What=20 should I do next? [root@host02 ~]# service vdsmd restart Redirecting to /bin/systemctl restart vdsmd.service [root@host02 ~]# systemctl restart ovirt-ha-broker && systemctl restart=20 ovirt-ha-agent [root@host02 ~]# systemctl status ovirt-ha-broker =E2=97=8F ovirt-ha-broker.service - oVirt Hosted Engine High Availability= =20 Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service;=20 enabled; vendor preset: disabled) Active: active (running) since Thu 2016-07-28 15:09:38 ICT; 20min ago Main PID: 4614 (ovirt-ha-broker) CGroup: /system.slice/ovirt-ha-broker.service =E2=94=94=E2=94=804614 /usr/bin/python=20 /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker --no-daemon Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 established Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 closed Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 established Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 closed Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 established Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 closed Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 established Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 closed Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 established Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]:=20 INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection=20 closed [root@host02 ~]# systemctl status ovirt-ha-agent =E2=97=8F ovirt-ha-agent.service - oVirt Hosted Engine High Availability=20 Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service;=20 enabled; vendor preset: disabled) Active: active (running) since Thu 2016-07-28 15:28:34 ICT; 1min 19s = ago Main PID: 11488 (ovirt-ha-agent) CGroup: /system.slice/ovirt-ha-agent.service =E2=94=94=E2=94=8011488 /usr/bin/python=20 /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent --no-daemon Jul 28 15:29:52 host02.ovirt.forest.go.th ovirt-ha-agent[11488]:=20 /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352:=20 DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:52 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending=20 =3D getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]:=20 /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352:=20 DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending=20 =3D getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]:=20 /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352:=20 DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending=20 =3D getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]:=20 /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352:=20 DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending=20 =3D getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]:=20 ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Error:=20 'Attempt to call functi...rt agent Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]:=20 ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Error: 'Attempt to call=20 function: teardownIma...rt agent Hint: Some lines were ellipsized, use -l to show in full. [root@host01 ~]# hosted-engine --vm-status --=3D=3D Host 1 status =3D=3D-- Status up-to-date : True Hostname : host01.ovirt.forest.go.th Host ID : 1 Engine status : {"health": "good", "vm": "up",=20 "detail": "up"} Score : 3400 stopped : False Local maintenance : False crc32 : d9521f2f Host timestamp : 1467916 Extra metadata (valid at timestamp): metadata_parse_version=3D1 metadata_feature_version=3D1 timestamp=3D1467916 (Thu Jul 28 15:29:52 2016) host-id=3D1 score=3D3400 maintenance=3DFalse state=3DEngineUp stopped=3DFalse --=3D=3D Host 2 status =3D=3D-- Status up-to-date : False Hostname : host02.ovirt.forest.go.th Host ID : 2 Engine status : unknown stale-data Score : 0 stopped : True Local maintenance : False crc32 : 0e1c66ec Host timestamp : 341176 Extra metadata (valid at timestamp): metadata_parse_version=3D1 metadata_feature_version=3D1 timestamp=3D341176 (Fri Jul 15 14:30:36 2016) host-id=3D2 score=3D0 maintenance=3DFalse state=3DAgentStopped stopped=3DTrue P.S. In case if anyone is wondering, hosted03 doesn't appear when I run=20 hosted-engine --vm-status because I installed it before upgrading=20 hosted01 and hosted02 to the latest version, so it might be broken. --=20 Wee --------------8F3B0F02ECFB4A9BFFA6F4AF Content-Type: multipart/related; boundary="------------DDB619A20B0C9AD190B68535" --------------DDB619A20B0C9AD190B68535 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html> <head> <meta content=3D"text/html; charset=3Dutf-8" http-equiv=3D"Content-Ty= pe"> </head> <body bgcolor=3D"#FFFFFF" text=3D"#000000"> On 21/7/2559 16:53, Simone Tiraboschi wrote:<br> <blockquote cite=3D"mid:CAN8-ONp_V58QUt3uB-_-pXtUWpxhc7iZRTNyTSA6ZpDtsSF_=3Dg@mail.gm= ail.com" type=3D"cite"> <div dir=3D"ltr">On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <s= pan dir=3D"ltr"><<a moz-do-not-send=3D"true" href=3D"mailto:wee.s@forest.go.th" target=3D"_blank">wee.s@fo= rest.go.th</a>></span> wrote:<br> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <div>=C2=A0</div> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor=3D"#FFFFFF" text=3D"#000000">Can I just follow <a moz-do-not-send=3D"true" href=3D"http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-= hosted-engine" target=3D"_blank">http://www.ovirt.org/documentation/ho= w-to/hosted-engine/#upgrade-hosted-engine</a> until step 3 and do everything else via GUI?</div> </blockquote> <div>Yes, absolutely.</div> <div bgcolor=3D"#FFFFFF" text=3D"#000000">=C2=A0<span class=3D= "HOEnZb"><font color=3D"#888888"><br> </font></span></div> </div> </div> </div> </blockquote> Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?<br> <br> [root@host02 ~]# service vdsmd restart<br> Redirecting to /bin/systemctl restart=C2=A0 vdsmd.service<br> [root@host02 ~]# systemctl restart ovirt-ha-broker && systemctl restart ovirt-ha-agent<br> [root@host02 ~]# systemctl status ovirt-ha-broker<br> =E2=97=8F ovirt-ha-broker.service - oVirt Hosted Engine High Availabi= lity Communications Broker<br> =C2=A0=C2=A0 Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.= service; enabled; vendor preset: disabled)<br> =C2=A0=C2=A0 Active: active (running) since Thu 2016-07-28 15:09:38 I= CT; 20min ago<br> =C2=A0Main PID: 4614 (ovirt-ha-broker)<br> =C2=A0=C2=A0 CGroup: /system.slice/ovirt-ha-broker.service<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =E2=94=94= =E2=94=804614 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker --no-daemon<br> <br> Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion established<br> Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion closed<br> Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion established<br> Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion closed<br> Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion established<br> Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion closed<br> Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion established<br> Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion closed<br> Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion established<br> Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connect= ion closed<br> [root@host02 ~]# systemctl status ovirt-ha-agent<br> =E2=97=8F ovirt-ha-agent.service - oVirt Hosted Engine High Availabil= ity Monitoring Agent<br> =C2=A0=C2=A0 Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.s= ervice; enabled; vendor preset: disabled)<br> =C2=A0=C2=A0 Active: active (running) since Thu 2016-07-28 15:28:34 I= CT; 1min 19s ago<br> =C2=A0Main PID: 11488 (ovirt-ha-agent)<br> =C2=A0=C2=A0 CGroup: /system.slice/ovirt-ha-agent.service<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =E2=94=94= =E2=94=8011488 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent --no-daemon<br> <br> Jul 28 15:29:52 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead.<br> Jul 28 15:29:52 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending =3D getattr(dispatcher, 'pending', lambda: 0)<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead.<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending =3D getattr(dispatcher, 'pending', lambda: 0)<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead.<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending =3D getattr(dispatcher, 'pending', lambda: 0)<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead.<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending =3D getattr(dispatcher, 'pending', lambda: 0)<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Error: 'Attempt to call functi...rt agent<br> Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Error: 'Attempt to call function: teardownIma...rt agent<br> Hint: Some lines were ellipsized, use -l to show in full.<br> [root@host01 ~]# hosted-engine --vm-status<br> <br> <br> --=3D=3D Host 1 status =3D=3D--<br> <br> Status up-to-date=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : True<br> Hostname=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 : host01.ovirt.forest.go.th<br> Host ID=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 : 1<br> Engine status=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : {= "health": "good", "vm": "up", "detail": "up"}<br> Score=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : 3400<br> stopped=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 : False<br> Local maintenance=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : False<br> crc32=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : d9521f2f<br> Host timestamp=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : 1467916= <br> Extra metadata (valid at timestamp):<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 metadata_parse_version=3D1= <br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 metadata_feature_version=3D= 1<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 timestamp=3D1467916 (Thu J= ul 28 15:29:52 2016)<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 host-id=3D1<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 score=3D3400<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 maintenance=3DFalse<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 state=3DEngineUp<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 stopped=3DFalse<br> <br> <br> --=3D=3D Host 2 status =3D=3D--<br> <br> Status up-to-date=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : False<br> Hostname=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 : host02.ovirt.forest.go.th<br> Host ID=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 : 2<br> Engine status=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : u= nknown stale-data<br> Score=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : 0<br> stopped=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 : True<br> Local maintenance=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : False<br> crc32=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : 0e1c66ec<br> Host timestamp=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : 341176<= br> Extra metadata (valid at timestamp):<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 metadata_parse_version=3D1= <br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 metadata_feature_version=3D= 1<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 timestamp=3D341176 (Fri Ju= l 15 14:30:36 2016)<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 host-id=3D2<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 score=3D0<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 maintenance=3DFalse<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 state=3DAgentStopped<br> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 stopped=3DTrue<br> <br> <img src=3D"cid:part3.FD9D1747.A6CF6D68@forest.go.th" alt=3D""><br> <br> P.S. In case if anyone is wondering, hosted03 doesn't appear when I run hosted-engine --vm-status because I installed it before upgrading hosted01 and hosted02 to the latest version, so it might be broken.<br> <br> <pre class=3D"moz-signature" cols=3D"72">--=20 Wee</pre> </body> </html> --------------DDB619A20B0C9AD190B68535 Content-Type: image/png; name="cbmafmakdomfeobe.png" Content-Transfer-Encoding: base64 Content-ID: <part3.FD9D1747.A6CF6D68@forest.go.th> Content-Disposition: inline; filename="cbmafmakdomfeobe.png" iVBORw0KGgoAAAANSUhEUgAAAiAAAAG9CAIAAACj+1pLAAAgAElEQVR4nO29f8wdVXrnWVkh 1DvxpGkWZbuBu+ps6Kzml7olQ0OtUIts8gfyZP/Y2axtbGCMsXmx6fadWDE9Gb/xeBLMGwIv BsxrPOCKXwz2+8YO+MeNekYZWZGuNI1a7d4W6Z6mFfUdTw8QobxqXaAjmPU7cPePunXqOec8 p6rufW+9Pz8fXcF13XNOnXPq1POt55zz1hO1AAAAaiBa6goAAMDqBIEBAIBaQGAAAKAWEBgA AKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAAqAUEZpVz4cLFf/34q0de+NOLF5e6Kq1Wq9U6fe70 kTNHnpp56tS5UxcvXmy1Wsf/7Pi5i+eWul4AMHoqCszssQOPpExOZwcOHJtd0Klnjx14xCmj f2j22AFzov65zb+q11RWd5i6LayEZcSzR2YbX5l4/KnTS6sxr194fcfxHdcfuj46GEUHo8/8 4We2vrj12Nljdzx7x8uvvVy9nOifRupnxNU9Mb7hllvGJr1/nRjfsGH8xMDFTY4Nk0vi3YOj wrmXQ3ffkLf89GSaawQWw2GxOqTWcw1ck4WVVtGuVjyvY2wfeeQR16JXEhipBNOT6deRtDwr TJ5ocjot/MABIWXmH5Uqm5fpnqByEXm26cnF1pjTp08nSXLu3Mge6s+cPX/LbY9//h8eeunE n42qzEH5swt/9htHfiM6GP3SY7+0/un1tz9z+y8//svRweiaf3PNtX9w7Suvv1K9qOifRodb h+UneqQegdmwYUNfYax/DFvaQgRGuwdHhS8wQ999pYWPjEXskFrPtZhUv7IDXrWC5BVuSzv3 7Oz0rBAYo2H9JLPHDkxOTjpaPz0ZUH+n6Oxfs8cOHDh27Fj6y/Tk5LFjQwpMa3rSKJapX1Zv v55aCaYMPYv/ZBNsbCXOnTs3NTV19OjRV1999eKIPI6Z2XNfuu3x63714Nad//a11y5Uz3jk K186/NnomRuu/aPrr/3D66/9/c9d+83PXbv3c9fu+cVrm7/891/+1/urF/Xg8Qejg9GvPflr E6cn0iOPn378s499NjoYXfsH1776+qvVi0oFJpqO8o8QmM2/f6p6UUWcGN+wYXx8PFWFybGx 8fENtgczOXbLLbfcckv2w4nxDRvGxjbckjo6J8Y33JIi/2VlFP5RKeo96I++4YaoIjDa3WeS 9bNnP8weO3BgcvJAvzj7bNm/0mmJ6hZj+XVIwblCDZH58u/KuUzv5TXzzdWoOk21q0q/mfOm RwrNmvtUL1JWEJjMwCpFWuf3a+R2s19Snlw+G8weO3Dg2PSxyWOzqYc9PazA+P6W2nN24e6B gixWlfsjJNzYUs6fP3/06NGpqannn3/++eefH5XGPHd09oYv/f7/9KUDX7nrj8+fH6DA86dP JV/50oXro3PXR6c/F534bHT0s9HT/2P05Bc+98r4N1uD1O36Q9dff+j6ydm+Sb1w8cL2l7b/ wsFfSKfLhvBgQgKz5cAAWlXEifENG8Ynx8fGT6TzW5O2wEyOCbnIRCRzUnJ/ZXLMmVgTrkz2 WwUK7kH583BDVLOnyt3XT2YKzHx7+XyvVklOkVWxGMutQ0LnGsD0CRvin+sR32TYJYyq00qu rHfebHaz1Kz5DTUpFyYw4qegr2AETSqwUrooKyt8cro1e2zy2OzsYALjn67oSnvuXYHAhIdL nreosUWcO3fu2LFjR48eTTXmyJEjR44cOXXq1IULA/gcPrNnzv3G//nHn/3i711/y/j6/+OJ 188NpljnXn3lhS/fMvO56OR10YufjZ79e9HTX/jsqQP/8uKAtYoORl+e/LL558XWxT957U+O njn64tkXXzz74oWLA5TWF5hHIuuTC8zpgSoWJFWCybGxydaJ8bHxEycsgfF1wpkEy12YDeMn fL9H/laBkBlxHnmHG6LqA7t/9xljV6pPapWqW4zl1iHquao1pG+ijQErPpepvHS2RthpBXbV 7zdZjzKzpvhnWcqBp8gGbnmZxKa/S1cmf1bqO3EDCkzeVvXBqkxg3AOVnkectINy7ty5F198 8dixY0Zjnn/+eaMxA/kx589fSNOfO3fh1OnX7/6//ujv37T3hv/1X/7S//LNh/YcG1RgWq3W +bNnDv/jLybXRS/8YvTcF9adGv/mEH5VdDD6J5P/5GLLzfjkzJP7Xt538vWTAxRVuMgvBeYL X/51/1P1NMZT6c+BDSIwmVfTMo6OFJghlnLUezBfHiz01EuHqCow/t1XRWAKqlSHrVzUDvHO VaUh05OPTE77nklRQzJbn5c54k7zrmyg36Y9x6akYC1lBYFxvTghvKrv5l/RR8wQ0+o5m85C im7qZ509Nnkgm+UdaorMnND6UiYwVntF5ytZhIalX0sbq3Hu3LmjR4+mcvLcc889++yzzzzz zOHDh59++umnn3768OHDr7zySnWbfvbsuUf+xfPf2Dt1zz9/6qZf2/uL//M3rmvs/cUv7L35 H+6fnR1y48D5P539o3/8q0caN5zev2+4Eq4/dP26P1x34NUD8uDzZ57/3KHPRQejo2eODlGm unnsnt+3psiGVJeWkYQT42MbsjWWsikyKTBGXxyByWfSToxvqK412j0onf/gqK4wRHV76t19 xjz5U2TybglVqbrFWG4dEjKAlRoyPXlgctJyFILnkiajX+hoOy10ZYP9JmfkisyaVR07ZRWB aUnPR5EP6dWFZDy8QNQvQ3tkEP8fbg0m77F+FYySlVyh3GP0/KqWNyjsdIMv8l+8ePFcGRWL Snnq6ZM3/eo3/od1239h3Y5rr3943ef3/G/r9x976U8HKsThtVdfffXQHw607iJ5+E8ejg5G X3ziiwdePfD6hddbrdYfz/7xP3rqH0UHo3927J+dv3h+iDJVgdnqTZENoy6tXBLE//VF/jFl iiybCBsby3yZybGFLPK3Wso9aI5MTnq2fpAhGnhgd+8+aaKzs3rm2KtSeuSAsshfZDGWVYfo 56rakPwJtcK53NOMttOCdjXcb9bjRNCsqUOg8iI/rEBmZ1878Acv7tj19EOPPHPwsWR29rWl rc+5i+d+/civp3/+8g+e+gdfnvzydY9dFx2MvjL5ldPnBl41KfgjGHUX2cDqMgAL3YC8YhlU D2AtgsDAIvHahde+ceIbN//RzenOsesPXb8j2TF7fsQmamTblEvIF+vXmLwoz/IAIRAYWFXc M6pdZACwYBAYAACohVoEZnp6+tChQ3v27Nm2bds9AACwDNi2bduePXsOHTo0PT3MX1MMwYgF 5vjx47t3794EAADLmN27dx8/fny09t9nZAJz9uzZvXv3LnWnAQBAVfbu3Xv27NlRqYDPaATm 5MmT27dvN5XeunVrq9V67733rl69+ikAACwDrl69+t5777Vara1btxpzvX379pMnB3iVxkCM QGBOnjx57733mupeunRpfn6+BwAAy5L5+flLly4Zo33vvffWpDELFZgzZ8488MADaS0feuih 999/37Th008//fnPf3758uUkSR588MH6vT0AAHB58MEHkyS5fPnyz3/+808//dSY6Pfff/+h hx5K0zzwwANnzpwZiahIFiowZt1lbGzs6tWrRlo6nc6WLVuWrksBAMBly5YtnU7HyMzVq1fH xsbSn/bu3TsSUZEsSGCOHz9u6v3BBx+kNf74449feOGFpek8AAAo44UXXvj4449Ti/3BBx+Y 4yPfV7YggTE7ki9dumTUZf/+/UvUaQAAUIn9+/cbjTHrMbt37x6VtKQMLzAzMzNpnbZu3Zqu 6n/66af4LgAAK4IXXnghnSubn583+8pmZmZGKTAnTr823Gf84GNphU6ePJnKYKfTWbq+AgCA weh0Oqn1PnnyZHpk/OBjQ4uC/4lmz7Sczwsv/Il/0P888vU9aYXeeeed1H1hVR8AYAWxZcuW 1Il555130iOPfH1PFftf8TO8wIyN7Uor9NFHH/V6vQ8//HDpegkAYE2za9eu8fHxJ554Ympq ampq6oknnhgfH9+1a1dpxg8//LDX63300UfpP8fGdi0LgTH+yieffNLr9S5fvlylFy6vj8xn 6K4EAICU+++/f//+/VNTU/v27du5c+fWrVu3bt26c+fOffv2TU1N7d+///777y/Ifvny5V6v 98knn6T/3LJly7IQmHvuuSetUOphPfvss1X64vL6qLc+6lUVmPNX5J+fdi8/WeUco+P8ld6V 87Im2b9GVvzSNm6IFp2/shR1BQCV+++//+DBg48++qi6QrFly5ZHH3304MGDBRrz7LPPpmsc 6T/vueee5Sgw8l1kBQwgME9e7irvOFg821aHpLilO9R1soIaIDAAK5j9+/c/+uijxWkeffTR gr8e2b59+woQmIrdUVlgMnkRBrBvk/vWLU1w5XxuqS1bKeVJ/JAe7l650s3LUhwJW9yunPfN sUwh7G1JrZzcJmNWWnZA922KCw+0OPhDUYvsSue/9M+NwAAsA3bt2jU1NWV8F7MTzPnnli1b pqamCtZj1p7A9IXAMWTyqObgqPbZ+sXO1b38pJ+2e/nJMoFR3I/+L1qtfIlRWvfk5a4zHefX vqBw/6dgVe1OKk1W1M8AsISMj4/v27cv/Z6+rdKIivPPffv2jY+Ph8rprTmBCczfiMOOE9C3 kFfO++Y7e/DPv4dmhhSbq6zBuO5H+pMle/3f5C/eeapZaVmLcOH9X7yaF3WF1yI1mV2y23QA WDqeeOKJnTt3mn8aUXHUZdOmTTt37nziiSdC5SAw/mFpKzdtkkZSXd6wLHHA5PcpExivbp49 trVnSIHxqxQuPFRgUVd4LVKTuY1ligxguTA1NSUju2zatOnkyZOXL1++fPmyM122devWqamp UDmrSmAqbVOuOkW2YIFRZoAWLjDmt4A5Lp4iC1YpXHhowR6BAVi9rCaBmd5357qMr+07NqTA VKbiIn+VKTK/WHcyyC5y4VNkZQLjNMVe5A9XKVy4NpHllaXWQZ0i868DU2QAy45VM0U2ve/O devuHD+ctF56qfXkI19bt+5rmzbX6cFsKt2mPNgiv22whYEMr9fbe7UGWuQvFZjCbcrB0gsK H2CR317DKU3GIj/AMmW1LPJPbF+37mvNp1ovJa1TZ1qzZ1qnplubhxWYQf7Q0t/05fwk9uza Vi/wR4zKs7oswDLh5uT5vFvlbcqy7KA5Dv6hZaBKJYWL4vQG2r+ok18FXZ0evlzYIgBYPFbJ NuUX9n1t3bqvPfJka/p0frDmv4MpxV2DAQBYayz8Dy03rTKBGdG7yBAYAFjrLPxVMZuWXGBm J7avW7du4/gIBGZ0IDAAAAt92eWmpReYM9P7/vd1624ff/qV3KEZbpEfAABGztCv69+0DASm NXtmunlHvk157+EhF/kBAGBZsRwEpjV7pjWdtF56qf9Z6ikyAAAYActFYOQHgQEAWAUgMAAA UAsrQGAAAGCFgsAAAEAtrACB+X9/fIUPHz58+Kygz0oSmKWQXgAAGAYEBgAAagGBAQCAWkBg AACgFhAYAACohdoF5onnkuE+zrvIeggMAMCKwheYzZvvGVoU/A8CAwCwRkFgAACgFhAYAACo BQQGAABqAYEBAIBaQGAAAKAWEBgAAKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAAqIVVJjCdOHJp xM25+vpPMNdpN6K44x1uxo2k3en1OnHUaHcWpy6V6LSTuJmkXxerbp0466KkGTeTdv1nBIAl Y/UJzJIZ8YDAGJadwLSTZiYwi0RZFwHAqmKtCEw7aTaTJI6iKGpcaF+I4+ZEM46iqJm0Uycj dXeMwU2a8USSNKIoEgZxrtNu5OV34ihK2h2TfmOz2YjiJGmmRWWP54oH02knxr9Sa+snSJqx qUk7aUbZqZNm7Dtq7awOWRZTh35Vm0nbnKIRN+fyunVipQlpw/sFJknT9zzEGSMhWk7HGv+y 0e7MCQ9G6X9xvfJLYPc/ACx31pDA2HbKaMNcM24Yi2ke6pNmrM6tGbOYGmgxxRS/2Wk3MqPc aSeece/XTVpJ1WKGEqRV6ogj7aRpKmm+i1ObVisC07M8GCkw/UaJU+dSOifamHe6OKNT4az8 uWbciJuJ9GCyauj9306akVIOAKwkVp/AuHjG1JqocSZtzD9DKwSddpLa8XbSnEiSNHG6mGEX ZRYbXIFxJqb8E4UT9Fvn+UbmjH3PQBzs+SnLBMYI3lwzjtudOdNkU73CtZN+Lrnckv/mCUyo /2UniLUiAFhJrD6BCXowIYGJLU8lN9MBM5om6DTjOPvvXNLcmLkmUmDSmrgCYya1HAk0FCSQ z/WamqYVa4xOYBpJu+PY94DA5NNc6fSdutyiCoza/47ALNpODQAYIQjMYB5Mr/9TsjHzYzY2 m6mJrCgwZR5A0EVIZ5/iuJFZW+Mr+NVzskuByaekqgtMsQfTaSeRl0u9FkN7MAgMwEpkrQtM wRpMSAbk6kv6PU1ZUWD8nQLOiQIJTD3zLHINxlh5uSJivjtLRwMJTOkajCMGcgOC3DQx6BoM AgOw0ll9AuPjPhF7szf6LrKwnyH3j+UrDRUFpmdtEnONdT+zm2CumTsulkiI+bRcWcWeLrlZ oN9A07TsYNwpEZjSXWR5t8fNiXQ93+lY4XU1UgUq3UXmCwy7yABWFqtMYKB2+ANJAKgIAgMl FPz1DwBAAQgMlCP/jhJ1AYCKIDAAAFALtQvM6ZnZ4T733IPAAACsYHyBueeee4YWBf8TtYYF gQEAWNGoAjO0KPggMAAAaxQEBgAAagGBAQCAWkBgAACgFhAYAACoBQQGAABqAYEBAIBaQGAA AKAWEBgAAKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAAqAUEBgAAagGBAQCAWkBgAACgFhAYAACo BQQGAABqAYEBAIBaQGAAAKAWEBgAAKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAAqAUEBgAAagGB AQCAWkBgAACgFlaZwHTiqNHuzIkjc824kbQ7A3VK0owrZzHlzzXjRuQSD3biXi9pxs2kPWAm AIDlCALjM1AWS2AGVTIAgFXMmhKY3MmIm4nIEsmDSbN/IPUk2knTd0c67STLMlEqMHOddhw3 J7JiTZq5TjtzeeIkaaanyzyYuWYcJ8mEV9u8eo24OZeX47QaAGDpWTsCM9eMG2b2qZ0042Zi HzTfc6notBOjK+a7NOip/JQKTCOTK1FgJ87ERiYQAtPIJCRP2U6aRlfkdwCAZcjqExiFpN2Z 67QbwgUx/0yasbdSkkuFvSLSP56JU37G0BpMmsw+dSeO4k6v12knUh7aigeTy1W2JuRomK+m AADLiNUnMLoHk85TzWkpzaRTpjRF6/aewJSvwXgC02h35jrtRE58lQpMM2lr8onAAMDyZQ0J jOrBSJJmbObNjAfja8aoBGYgD8YszKAoALBSWCsCU2ENRjHucg0mXSlJ2h15lnS1fwiBqbIG 4wmMte7SaSd4MACwnFk7AtMr3UUm189NGrGLTNkA5uwi81Z/Gu3OXEBgSneRKQLTsyb0ZDko DQAsO1aZwKxU+PtKAFh9IDBLg+125NNlAACrBgRmyVAn3wAAVg0IDAAA1AICAwAAtYDAAABA LSAwAABQCwgMAADUAgIDAAC1gMAAAEAtIDAAAFALCAwAANQCAgMAALWAwIwM8XbkyIuSOXpM CJmBWPBbNavGpGknTb8TsvAH6ZunlfdeF/RbGhnBfVt13JwLB0pYIGrQoNGxwICk/dCoDiaC 0UCVV+MeVafiSbN3IzUutC/U2bGjRe/nagw5MsuunQz5sdxfo47AlPPWW28dPny41WrNz8+H 0rSTprSYixCsZYlewFx9TFuxduwj/dAG8tdOO2nEcRW748RqG/o2XmoWZB0CNijvikEEZoEd WDX7Snxl+MIeMuoWmBUAAlPCzMzMNddckz4v33XXXQGNUS55Fh9T92w67SSNJeNEdonj5kQW 8UUMzTxojYk85gW26Z80ab8pzLdrysVNbspsJMlEWk7SjCeSpJHVU8SesUozdRPv61TuBxke rWfdNnPNuDGRJDKIdXbqIQUmq3bUsMuM7IPtpNlMktgK1RN836iocN7/Tof3BrlqTjviqJEk E16xSi7pvWUh7yI5cpz2NpN2Wvkku0Ay1F4+FrOLrlZSdk7F7AU9ZpLFzUQaUHUIeVHM/eti XTUZZNYMD3mLJe12nPWGUx8Zwc+7JfV+tqud/+S1JTQy1cBUeZXMjRCOVWh9CY1AM2wacXNi YU7q0CAwJdx8882RYGZmxk/jmTzrR/Eq/vyhXoTCzA2xDG0pgmlaL/M3CqE+DybN2L433OiZ MqCZVBpz10lbbEfPNJWR0TxzvdRsgaW7wgqkT3Zviue7ThzFb1Z7WlQFJjuSt1TW33y3J+5k xyoRE7LbW/aV75YNdtXszonEI4h7pUwuGdnBfK/mwbi18iK0psWqD9pK55RmL+4x0wmm8uoQ Ui+cWjETZDYkMJEdEUPKpH+K8C0Zmum1vsuD2TDTR6Z59JTfRZXyueLqAuNfazlsZNMWGQSm iPn5eeO+pDz55JN+skKBsTALJzJL+gDijeb+5K9TuPlnSGA0Z8WVJee26bQTIzCBSQwzE51r hp1Ynw1wHCYptNIupBWoOB1RPEWWrQc49elXW7VEKfKnfrlZfZJmXLA4NNBVk+0Q6ttf2Qrk yqUocFKlK9RaeYnjsMBI46i3289e3GO+wKhDqNgamrEqCQmMqL/T28rYqHBLZhUVBl22Tu1D b2RaT11Skm3PY1CBca+1M54XuMw2NAhMCXfddZcUmDfeeMNPUyow/sSCM5oLPGLnjgrcnH2q C0xsG7KAwOQTHZE7uJUQ0QWzTHYX9W8n5+4alcA0k7Zd87z+8q6TF8Wfr+jZl6Ng0magq2b3 rWvywrncGafBBcY6l33JQksFeUp1esfPXtxjnsAEh5A3JZjjPwf0BhYYfWxUuCVF17jzh0GR 1vxRd9w5AlNsECoITH+oy3sZgVmmAnPlypU777wziqIbbrjhpZdeCqRS1mDk46f0zYcQmCXx YLzZJHfjSrUh279zQu5OuvRS8LToU01g9N1ujiXyTZVVda0+cn5DS1Zy1WQ7VIGpkCsKi3GJ wNizRiZx6Vp0f5qrNHtxj6keTPEQ8qde1Yc557mhggejjI2BBCbvmmwmSrsfQ48+RRcudHZ7 HrWSwODBrACBSel2u8UJnG1jZnbYu/YDezCDrsFUEZiCNRg1u9gjp6/BmPb6PdNOmnGzad9U +e2UPqs6dqe0qysIjL+G5N917qxdQDmsdRe/2we6arId2qRNyRpMwKYoXVFqdMRMVOkajDul Gcpe3GPFazByQUVeuII1GLk4JJc6ygRGHxvVBcardmg9SR+Z8hnFPMbJs4sJRut5zrsHiwSG NZiVJDBVsP9QIx/Ncq4g0cZTmcD0pFPv7Cxybr/KAiPLtHaRmSz2JEBi/LC0Oc5mtsLh6y8h yBsvb6azzSy0pbWiwNg93z+F81gn26huD/NnqPzH54Gumt0t6qpAyS4yuXiuziBF3k4tR/ay 9k4YTW2L7VWFnVOSvbjHfIHpBYaQf+FCFXN6I4ridvkUmX4K9ZYs7men2t7WstDIlHODuXqZ 7HI7pTmYtNsDTZHJYcMustUgMCsXdV4bAFYLS/a3YgjM2kTfPA0Aq4PAnvLFBoFZo2izLgCw eiieaVwcEBgAAKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAAqAUEBgAAagGBAQCAWkBgAACgFhAY AACoBQQGAABqAYFZppRGK3ESF0Y8U0NQlCNeNaG8RXggKr6K35x34W+vMe/cXUhRxa8BdeK2 FWTndaKwNkFglikjNEkDGXeDY+W16Oh11WEkAlM9jnUBxVdhoFDZCAysQRCY0SBjVPhhLWR0 ozSQhvMGOie+rAzkMNfrxP1AMmlYDj+McW7p2kkzjd0iQk3kcV+ymIZuPdUGFbzfWxTixlgL lZwKTJIHtzAJlPAnWtAaGZTMCriiRoIRNTSBdpy6zTXjOEkmNOfMnDROhDA4V1OU6Z4iEsE8 HA9G63+rFQCrCQRmJOQ2wgTmk3HrZJxwY1mMTyCdA3lQRnmSoYqkIXbiY4rQk/mvdkAzGd2r 6B2rIqyeRaAQJc6glysSspFHZPKbJoLLFoXyTDtT9JIeW9duqRXoV/Mq8mqkdTYK4V8jJ5Ci H/7AEZiB+h9gFYDAjABtvcSJoKeEajcz+GpAU1tgdEtkzisFxo8578QZrDxXI+PuKU/lKam5 dyaL/BkhZ4pMBAPOD5pC1DI7XqTCdtI0sf/8S6D2iaib7qL59dQSK1dT0k6aqsAM2P8AKx4E ZgRoM+wdNQp9ICyrMnNSIDAysW9zfbfJi6iaKcYgls44NHLl30xeyQiyauGO4U5NsB1kOm+O ERgvvLEMVN5pxnH237mkudHRYHvasO0dryQwUhicmvoCIzvBmRlz/NEh+h9gJYLAjADtSbaS B+Ove5s4dAGBsSZ21If6MoGRNRxo6r9fDcdem3MVm8uQB6Puwir2YLIEycasyRubTb+ckXgw InHsO5HOFJnpFl9XAo8gLL3AKgeBGQnKCkRoDcYXGDm/r5lFKTCu2AwkMN4KeXANQMZb7WW1 jZUlDWNY3UUIzYMpXoPpJc1YejCla/hy2VzVvNI1GNW+O1sM/DWYTjtxPBjfR1QFpnr/A6wO EJjRIDcRSYspZ356AYHpacFNswLjjm2JxDxMI0kmHEunCoxZTUnn6CK3nvqjtL0tKnKcCa8Q K73vzdi7yBzfzj1FwS4ymUsourIZwevqPlIFAw6EWXzSd5FFtr6m/5T7AJOk6bih6i6y4v4H WAUgMAAAUAsIDAAA1AICAwAAtYDAAABALSAwAABQCwgMAADUAgIDAAC1gMAAAEAtIDAAAFAL CAwAANQCAgMAALWAwAAAQC0gMCPAD5ES5ddpB+8AACAASURBVO8YHuKNuQt8z676bvmiEMhV SJpx5ewLPVcxo4tvn0dU81/G7ODH0xxhlUJhCyoiYiUU9bx542f1xpadbrQow17U+ULWNPkq biWGQila9Ar/1J1YiaXdZ3QjsA7yMaDes8UXd+QgMJXodrtVknkXb0kERs2+QKM/UPZ6BWZU iHgK5RUuvi0XaHFCkTErUvHsCzzLoKcbHGXcFkZaGvI2qSIwnXYSNydCo2J5C4xhWdyGCEwJ V65cufPOO6MouuGGG1566aXixKrAJMmE9xL70Fvoi3PlT9zmoBcmIE9jj625ZtyYSJKG9+59 P6aAH3XRpElrK945n9+W5mDBnemktN97b17mr4YAiBMvDnHSjNUWiYgGymv8ZSebyqv6Yc7e iJsTigdjesl6q7/o8LjKRZRt7LglNJxLY8dQ6MeKFmmsp1c1WRTFzSzujmm7U/msjUoT7NOF 4iD0czXi5puZpXZGi3T6vShB/jiJO3nTTDxTa5yroTHMqeVorCAw/TBFISFpJ03RKCs8nT/w /NHuX8ReMJCHMgZUO+AMmzQeld29zsW1An/UF5cIgSnhrrvuigRvvPFGQWJNYKQd8aNsWRG3 KueyQvk6gc4KPBgZftHcll78GLV6ueWSUbPMd3kwHbiFEzVu9kCT83rKeGVSYCJxHwrxk4Go CzTG6nCvwvlBGdNMGN/+GdO6aQLjd7h6EXXfQl6a7Lsb1c3pEHOZZHg0892Jv1kQTUe20c/i hH/2hp8bdDUSGU3UIjt6m4yEHfJgHIGxPJhQe1XTWSowgQB9Oe2kaUqWsQT9gSdLyOrzpnoR VYHRxoB+ULavOKSeCRc7+G0yDAhMCddcc40UmImJiYLEhVNk/SljZ3yXhVvWc2X/lCZAzW6w Rls2OavHdRaGW8nuPNalTbYPFkXQUrOrTfYj3vsCI421tkZSOkvQfxj0nwTts+eGwAR7diJA h+/zfoKk3QlcRF1gpBVT8TvEnFSadbVF0gr7bZcC442ZoC6qsagdu6ktNRmFG1pgnKushCeX OJH0/Kf4Mgve80Z7/67xB54aX9wpqkBgtDGga6FsXxWBsSPM1ggCU8IXv/hFOQzPnj1bkLhM YPKb3xraJQLTSG2TfXNad5ddTiWBycy6HJ3BCZbAT5Hxwe1brsisq9lDHSXv7VKBMdbNKb/K NLR8BO73r22enEV+58bW5lKUDg9dxJApVCcxxEE3MLM8qRxm5ogz1ZZOpITXgdUxY4XpdLq6 VGCcwNJ264YWmI47pMoFJujBaPLjPt17oz2WU3ay20NLd/5FDHWUMwYqVK+SwDgtrU9pEJgS Wq3WZz7zmfQy3H333fPz8wWJqwhMhS1Dei714VdgnrIHEhjFg3Fa5FgN9aGsusAEnul0gRnI gzEHqyiKM/vh92dNAlPdg5Fk+teJhS3w4zGrPW8mQPwWNeLYG0WysSUCE9hOVi4w7SykdJqk YNG+sgejbCcbTmC8Rw1lEjvkwfgDTxsY+kWUtVXnrMQYKJ7OqioweYbATOBIQGDKuXLlyiuv vPKXf/mXpSmrCIwzZBN7xbViLm0Nxr8DJbqxKFuDUWyNPRzN3ZKfVJ2fyetRlt1uSMkajHqj mpvTPKeXXizfg/Ga7KzBuKv9VQRm6DWYNKXvAYQExlnk8NZgellHqavEVQXGH34VPRin5xfs wfiLOkN7MMrsrm/uS9dgzMDzVxwvtC+oF9Fb2HfXYOSwlweH82C82rIGs4z/DsZQTWB60qnX 7oFQrpJdZHLi2HN7dWNhEkeBXWRy1l6688Y/93d8DbKLTN1B5Fr2tFR1F5lvrE1VzTRFqBra 7KJ/Ifq/arvITPbQLjK1w9UtQOmJHPMqp/v697+cukwy45L1ktxqJVO6plCUr/8dSbHAiNOp w69cYOzJmUR7yOh52ZXnJznOK7Q3a3ZYYLTVx57/ACF3kQW2L0ayB5yD6kUUx6UIKWMgcFBe 1nyyIfKeIeSDQrXbZEEgMLAyCE1nw3As8K87AaqAwEBd+Ov5A/2hnz+ZNtxzlrprqL5HtpVC 6QYngIWDwMDyZXG8+LVGqrgj+at+gGIQGAAAqAUEBgAAagGBAQCAWkBgAACgFhAYAACoBQQG AABqAYEBAIBaQGAAAKAWEBgAAKgFBAYAAGoBgVlsAlE0SvADS5QGEfEZ6P2GoXfIe6/1GjiS RCjU+eJkH67/AWAIEJjFZjj7OBKBGShLSGCW/A2JS6tPAFAdBGYkdOJ+aIcosgKx5IFVTOTt wHuFrVi/VULj+RFwnbzhYKtuqAknGGWaJUkmBhGY0h5oJMmEEyAracYTSdLwOkTUXImDJKNx BGIJ57mc2M/h/geA0YPAjAQnMp0SHU8LCJjjRKnzzV+BwNivtc/jDRs7qwbNlWc038XBTqxZ 4UKB8XtABnBUIjCK+E55SrXmTldEIuSUKVMNzSsDpqXf8WAAFg0EZiQ4ARljP2Kr+WepgVNX SuSL6/PVD61AEV7XCWopBcaKHphZZOugqnPhEC9KDzihA83iRygkpRcysqeGfZXtzcp0kkmV dd1BBAZg0UBgRoIS8ddZTJaP1aqBs+NiKY/tIQ/GiRxlezbuBJ3/kzijJTADrsEoPeAoZanA GKlwalYsMI24OedF/zUl+wFrERiARQOBGQm6wFT3YJJm7M/wSIoFxvdgZEozZ2V7MGrs8REL zEAejDlYHFssIDCKB+N0V1Qo8AAwchCYkaCY10HWYCyDKFYmchayBmMMfWgNpp00PeM714wb CxSYKmswvsB4Na/iwZSvwVSfogSAUYHAjATVvPb8XWS9fHrKkhA5k5NurCpdORhkF1lk1ydf gQ9vuwruIvOXgjI1Le4BfReZLzCBmutdITuhbBdZv3qi/3P9s0UaAEYDAgOLBK4DwFoDgYH6 kJOEcroMANYECAzUiJz6Q10A1hoIDAAA1AICAwAAtYDAAABALSAwAABQCwgMAADUAgIDAAC1 gMAAAEAtIDAAAFALCAwAANQCAgMAALWAwIwA9R3DzaTtRFipzHC5DG70reygGyJlIErDtIzw XMWM+qWZane5mBc/qwELFlglNYZpdURou6KeN6/cLn1nTzgog3O60aIMe1HnCyIKhnkdePmF 80nLdN4Unh40ASP894gPhIkBqMZ2GrCqw2dfDiAwJczPz09MTPzmb/7mtm3b3nrrreLE3p25 JAKjZl+g0R8oe70CM0Iy+1Xe28U2d4ECs0CLVvHsC7ebA51ucJRxq9XZJBvyNlEvetKM4zge VbuKR0t1EJjVLzBf//rXjVNy3XXXvf322wWJVYFJkol+lJJ8+CpxYirkyiO4mINeXOQ8jW3i 55pxI400E9mPb8b9Egfz6pnALbK24hWWVgTMLMtEscBo2TtxFCdZQBcvdk4URXGSPRXKcDJq i9TAMD5ZDJg3Q3bKnL0RNycUD8b0UpzoQeTiKhdRtrHjltBwLo0fV9t0phPZU3jVVrIoipsi fGrPDj1njmRjyW2CfTo1EFGeqxE332wn6XhwRot0+kV/WhdC1rljxfFrtDsdZ5z7w7gjTi1H Y2q1EyvQeH/4eR5MfgkmkiTOI5QnJq6SjF1kwh2Zf0qFqD7s/araPdwLGBDFPpT1cz5g/ODi gToPBgJTxIcffvgrv/IrkeC5554rSK8JjLQjeaSvLJn6EvvSXMbIyvfhm+9BD0bES85vSzX8 sFe9fCDK2FzmuzyY3mCFEzVu9kCT83qmhswXmMwo53WWATHl9zChB+H87OltZguME6ws0gTG 73D1IgajU0sz4V3uvCgnDmnS7shQnua7HTBbDhu3i2Qb/SzidOrwm2vGDTmoIpHRDz8qrlGx B+MIjOXBhNqrmsXUar9pm/64mfiXQ1yCvCtM+Fe78tbANnZABpMVQ1E2wR/2TlV9YdANiFPb tEVl/Syjmxfd1ENH5ENgivjwww9vueUWKTBHjhwpSF84RdafMvYj1YcnAYK5sn+qgY3Lp8iy BRUnZf+fWszmPLszQ5I22T5oxYp2ULOrTXZ6pq15MNJYa2skVSbrdIGxz27dxs2k7VwO3wlQ OzxwEXWBaSfN4sdGv0PsgNnuJXCifxor7M8LSYHxxkxQF7PjVi4n7qq21GQUbmiBca5y/3ho SlDeO+YxK1Vlu11WfUzlwzOE+ZqQLzCOZSgY9lpVrVwBA6I8KFToZyu+u9OuwK06GAhMCb/7 u79r1OXzn//83FyRhpcJTH7zS8oEpv+IYd+c1t1ll1NJYLLxLQdlcIIl8FPuettjscisq9lD HSXHd6nAGOvmlD8KgXEX+Z2bXDM6SoeHLmLIFIrpl7yG6pyMv8jvz3j4MbaTdkfduxESGKf/ ZU1EfYoEpmE9xed3QGhNpZrAdNwhVUlg+g1x/tkLmOywwFhDztwjvsDIfhYzEMpslV9Vk6uZ tFUD0gmu1hT3sx7f3ZkYt2/VwUBgSpifnz98+PBv/dZv7dix48qVkgpUEZgKW4ZCPqzy8Csw T9kDCYziwTgtcqxG25q87lNdYNTsIYEZyIMxB8sUpeDU4mg9AlPdg5FkDkcnFje5NPQFu8jM 3IjfokYcq1aposAEtpOVC4ycZVKnvGTlq3kwynayUoExSyB+uyp6MPYsnPUQNpQHUyIwgccU pxtzKvSzOtnQPxi4VQcDgRklVQTGmYhP7BXXirm0NRj/DpToxqJsDUaxNfZsrDF5+UnV+Zm8 HmXZHbddnFRZg1HXM+xp8SE9GK/JzhqMu9pfRWCGXoNxnrWdkxavwZjCnbOkl0md7akoMP7w q+jBOBduwR6Mv6hTyYPp9R/S80IqrsE410X2Z0hgwmswpQJTvAaTGxB/0S7Qz9bDYlpn765s OGsw4lYdDARmlFQTmJ70RrV7IJSrZBeZs3XEroluLEziKLCLTM7am1OogZBNTQbZRVays6V0 F5lvrE1V5ZRFIQW7Xfv9ENhFZi5HaBeZ2uHKRcxO5FRDzr1Ig5h3SHo86yV3XcGeGPFlLPh3 JMUCI06nDr9ygbG3PibaQ0bPy648P8lxXqG9WbPt5X05wj2913eReRcuH/byBnFmrgK7yMo9 mGwXmTPZ4N6h/mx5oJ/zmoi/LtJv6oWHPEdgYGUw3BojhFjgX3euNUb1V0RrDQQG6sJfJBzo FvU99AEXV2Q5LsMVtZoYyQz7qsadiQpsHoMiEBhYvgw42QWVUN+VAj7q/DMMBAIDAAC1gMAA AEAtIDAAAFALCAwAANQCAgMAALWAwAAAQC0gMAAAUAsIDAAA1AICAwAAtYDAjAT5etQlj6Gd v8Sw4guUFviepcBr2xXE60kKXjEZPI/2Ptf++zzUdwPLSJQAsPggMKNEC9Oy+Fhvzl8EgQnH +HOxX3E/qMA47zy3jmSvfXXip8WxEpEaABYJBGYkpObyzezljr7pzF+vbd6Y7b/GvNNO0pd+ R1GUtNtx9qZuJ8CcE4N2IjtuwsaYf6ZnedMOTq6a6YZbjknptsgJdmnSiIjleieYn0wsvySZ 8F705HaU343ypWRGsUwTzK+mcxAYgKUCgRkJMkSP78HI17L253lCAhMFIvzIkErmu4xHJGRD 8WCcYERa0MPIxCAy8an8EElqNdRgX76M9QJBusRbk2VHWe+yNXjROKyWvmmH92gmbV7yD7CE IDAjoUhg1MAbIYGxo36FghGppzPLP4rACCOurBI5lYkzTfKDvKrRKo3RtwPwKQSmyPqLRk7v qRN3Mo0Wt9ENo4vAACwhCMxIKBIY1VAOKDAdN7KKLjBuHFYnNl/cTNQF+VBcWykktptivUJf ehV2ED3XsocFppFO6LmN1FaG1HiL0leTDiICA7CEIDAjocSDKVYduVId9mCU6LaDCEx//i3W ghmXBk7vaZbaVFtd5LfDhfmFKAJTMcZiWttOwN0xSy9OdHQAWHwQmJFQfQ2m/12kTJfliwXG nQ4a3IPpZW6QssktvCBUtAZjcqlrMGpXFAuMH0MwsLetE0eNZnOjVA4vjnoexB6BAVgqEJiR IKf+G1r4xXyOSy62p/NAF9oXyqbIZPp8g1ZAYPopna0EvXCUXFVgeuW7yGQ1IuPNOBNozomy ia/QClPeUQU7p9tJ09ml1rGX92Une5sgCH8LsEggMGsHfZ4NAKAmEJi1QsUVDgCAUYHArAXS ibuB/3IeAGAhIDAAAFALCAwAANQCAgMAALWAwAAAQC0gMAAAUAsIDAAA1AICAwAAtYDAAABA LSAwAABQCwgMAADUAgIzAsQLhnOaSdt/I3I1hstlUF9q6cTEHBg/0HJhBRZ0rmLU8DND0X/1 dfGbm1PMW5nV9/8vsEoLfE2cCCJX1PMmFlxp/ILiGAdqzLpRoAx7UecLWdPcoKWDnkYLJOGf uhOHR8XoRmAd5GNAvWcXOYAFAlPOW2+9dfjw4VarNT8/X5zSu3hLIjBq9gUa/YGy1yswo0Je rFKTUXxbLtDiqDFPq1Px7As8y6CnGxxl3Gp1VuIeDUQVgem0k7g5ERrGy1tgDMviNkRgSpiZ mbnmmmvS59y77rqrWGNUgUmSCScSjIx6opmtUK78iTscojhPY4+tuWbcmEiShvfAbtwvLzRZ fiKTRsRWcePEmIMFd6aTUmTvxFGcZLFkTJ+I1sVJFszG3N5JM1ZbJGLSKNHVZCebyqs+hDl7 I25OKB6M6aU4USzOXDOOq1xE2caOW4LrYMnLHeVx4Uwa6+lVTRZFcdOO5OZHxxFBqd0m2Kez 6pNd8TxXI26+mVlqZ7RIp1+LQeeMk7iTNy1N1nHGuT+MO+LUcjRWEJh+4LuQkLSTpmhUXpQ6 8PzR7l/EXjAmk+pkK3bAGTZJu+Pcs97FlbWt8TW4CEwJN998cySYmZkpSKwJjLQjeVAyJ8Cl XUxpLmNkO3F+85jvQQ8mG6N5LmlcsiGuVi+3XDIWsvkuD6YDt3Cixs0eaHJez/Se9AUmEveh H1jMDspZhBaCLD97aiNsgcnPmNZNExi/w9WLqPsWskrZd3m586KEHexfJhl41HwXZ3GGTSjw qD5mxOnU4TfXjBtyUEUiowmB6scLL/NgHIGxPJhQe1XTWSowJoEa9rtnx7szNVQHniwhq8+b 6kUMhS33xoB+ULZPxocNTZENd5sMAQJTxPz8vHFfUp588smC9IVTZP0pY2d8F04CBHNl/5Qm QM1usEZbNjnrpOz/UxhuJbvzWJc22T4o7Y6Lml1tcigcpxQYaay1NZJKswTtpOnfYPbZc0OQ lu9cjsL7vJ8gaXcCF1EXGD9qp1/tkMBIs662SFph/ylYCow3ZoK6mB23cjl2U1tqMgo3tMA4 V7l/PDQlaDsQBsu7KrTgPW+09+8af+CFYsjKogoERhsDuhbK9lURGPnEVisITAl33XWXHIZv vPFGQeIygclvfmtolwhMI7VN9s3pBmkW5VQSmMysy9EZnGAJ/BQZH9y+5YrMupo91FHy3i4V GGPdnPKLBUZT017Ps57OIr9zY2tzKUqHhy5iyBSqkxgyKHV6Ul9getbMTO6HOVNt6URKeB1Y HTP56WRNRH2KBMaO8O0Y96EFpuMOqXKBCXowmvy4w8Mb7bGcspPdHlq68y9iqKOcMVChepUE xmlpfUqDwJRw5cqVO++8M4qiG2644aWXXipOXEVgKmwZ0nOpD78C85Q9kMAoHozTIsdqqA9l 1QUm8EynC8xAHow5WOqyyHqG1r1rEpjqHowkczg6sbAF0tAX7CIzEyB+ixpx7I0i2dgSgQls JysXGNtlLFq0r+zBKNvJhhMYOdtmmuOM2JAH4w88bWDoF1HWVp2zEmOgeDqrqsDkGQIzgSOh doF5Zlg2b968HAQmpdvtVklWRWCcIZvYK64Vc2lrMP4dKNGNRdkajGJr7OFo7pb8pOr8TF6P sux2Q0rWYNQb1dyc5jk9dLEK9wI5TXbWYNzV/ioCM/QaTJrS9wBCAuMscnhrML2so9RV4qoC 4w+/ih6Mc+EW7MH4izpDezDK7K5v7kvXYMzA81ccL7QvqBfRW9h312DksJcHh/NgvNou3hrM 5s2bhxYFn1UiMBWpJjA96dRr90AoV8kuMjlx7Lm9urEwiaPALjI5ay/deeOf+zu+BtlFpu4g ci17Wqq6i0zdZyznH0LV8CcqtYmyfj8EdpGZyxHaRaZ2uLoFKD2RY17ldF+/bnLqMsmMS9ZL cquVTOmaQlG+/nckxQIjTqcOv3KBsSdnEu0ho+dlV56f5Div0N6s2WGBCcyXug8QchdZYPti JHvAOaheRHFcipAyBgIH5WXNJxsi7xlCPiiU3iYLB4GBlUFoOhuGY4F/3QlQBQQG6sJfzx/o D/38ybThnrPUXUP1PbKtFEo3OAEsHAQGli+L48WvNVLFHclf9QMUg8AAAEAtIDAAAFALCAwA ANQCAgMAALWAwAAAQC0gMAAAUAsIDAAA1AICAwAAtYDAAABALSAwAABQC7ULzOywLJN4MCMn EEWjBD+wRGkQEZ+B3m8Yeoe891qvgSNJhEKdL0724fofAIZAjQcztCj4RP98WDZu3LgqBWY4 +zgSgRkoS0hglvwNiUurTwBQHV9gNm7cOLQo+KwRgenE/dAOUWQFYskDq5jI24H3CluxfquE xvMj4Dp5w8FW3VATTjDKNEuSTAwiMKU90EiSCSdAVtKMJ5Kk4XWIqLkSB0lG4wjEEs5zObGf w/0PAKMHgRkJTmQ6JTqeFhAwx4lS55u/AoGxX2ufxxs2dlYNmivPaL6Lg51Ys8KFAuP3gAzg qERgFPGd8pRqzZ2uiETIKVOmGppXBkxLv+PBACwaCMxIcAIyxn7EVvPPUgOnrpTIF9fnqx9a gSK8rhPUUgqMFT0ws8jWQVXnwiFelB5wQgeaxY9QSEovZGRPDfsq25uV6SSTKuu6gwgMwKJR u8D8m2FZUbvIlIi/zmKyfKxWDZwdF0t5bA95ME7kKNuzcSfo/J/EGS2BGXANRukBRylLBcZI hVOzYoFpxM05L/qvKdkPWIvAACwa6i6yoUXBJzowLJsyVq7AVPdgkmbsz/BIigXG92BkSjNn ZXswauzxEQvMQB6MOVgcWywgMIoH43RXVCjwADByfIHZtGnT0KLgE40Py0oXmEHWYCyDKFYm chayBmMMfWgNpp00PeM714wbCxSYKmswvsB4Na/iwZSvwVSfogSAUaEKzNCi4BNdHJYV9Xcw qnnt+bvIevn0lCUhciYn3VhVunIwyC6yyK5PvgIf3nYV3EXmLwVlalrcA/ouMl9gAjXXu0J2 Qtkusn71RP/n+meLNACMBvXvYIYWBZ/o9LCsKIGBcnAdANYaqsAMLQo+CMxaRk4SyukyAFgT 1C4wM8OCwKwC5NQf6gKw1lAFZmhR8MGDAQBYozBFBgAAtYDAAABALSAwAABQC2UCM7X/vrvv vvu+/VMlxyoIzNT++/ZMDCIwK+hVMQAA4FLqwShqMrGnsr4YgUmLQWAAANYM5VNknjQMoi+n oz0TMzMzR/ffd3fGnomZmZmZiT3mwH37j/a3nKUl33ff3Xffffd9K+ptygAA4FJhm/LRvsJY KrD/aP7LfiMfeyZy4UgzRHbSCbcI/3v2DzwYAIAVTZW/g7HE4ej++4wG9B0T6aS4chL1nR/p B9ku0FRa4JT7AwIDALCiqbSLTBh+IQfO7JlUh/wHRWCmxHyZmDhDYAAAVhXVtikby2/py5AC E17EQWAAAFYPFf8OJjP9E3ukNAwgMOnPVtJ/9fzp06dPP/+vXOVJjyMwAAArm6p/aJnqw549 uQA44hBQjejuu7/x+KlTp06devwb/U1jv3dE/Ovuu02C/sH091OnEBgAgBWNKjCnFIwiGDk4 derI7913ty0gfXXIf4heHRb+kh8AYEWjCszQouATvTwsCAwAwIpGFZihRcEnmh4WBAYAYEWj CszQouATJcOCwAAArGhUgRlaFHyifzssmzezyA8AsILxBWbz5s1Di4JPdHRYEBgAgBWNKjBD i4JPNDUsCAwAwIpGFZihRcEnem5YEBgAgBWNKjBDi4JPtHEBIDAAACsXX2AWogg+UWtY2EUG ALCiUXeRDS0KPggMAMAaBYEBAIBaQGAAAKAWEBgAAKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAA qAUEBgAAagGBAQCAWkBgAACgFhAYAACoBQQGAABqAYEBAIBaQGAAAKAWEBgAAKgFBAYAAGoB gQEAgFpAYAAAoBYQGAAAqAUEBgAAagGBAQCAWkBgAACgFhAYAACoBQQGAABqAYEBAIBaQGAA AKAWEBgAAKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAAqAUEBgAAagGBAQCAWkBgAACgFhAYAACo BQQGAABqAYEBAIBaQGAAAKAWEBgAAKgFBAYAAGoBgQEAgFpAYAAAoBYQGAAAqAUEBgAAagGB AQCAWkBgAACgFhAYAACoBQQGAABqAYEBAIBaQGAAAKAWEBgAAKgFBAYAAGoBgQEAgFpAYAAA oBYQGAAAqAUEBgAAagGBAQCAWkBgAACgFhAYAACohRUmMHz48OHDZwV9VozAAADASgSBAQCA WkBgAACgFhAYAACoBQQGAABqAYEBAIBaQGAAAFYe3SXld37nd6pUEoEBAFh5dLvdJTw7AgMA sGrpdruffvLpUn0QGACAVUu32/3v//2TpfogMAAAq5Zut3t1fn6pPggMAMCqpdvt/rf/9v8t 1QeBAQBYtXS73Y8+/kj7/Kfboz433/7I2x9/9NHHH739o/9we/Z9JB8EBgBg1dLtdv/O46c/ +Iubo5v/4gc/Tf/5F0d333z77p/+3d/99Ad/cfvtu3/qZxgWBAYAYNXS7XY//OBD5zO166tT /+6vxJEru75689S/+6srb/77r35115UsTerffHXXVJrs3z+/K/N5vvpX/Yx/9dXskF1g/4PA AACsWrQ/f/z+bdFt39d++Mn3v3Xb4n1nkAAABmRJREFUbQ//pNv91pGHb7rt4Z90u93uTx6+ 7aaHj3zrJ9//1k1ZriMP33bbw0fMT1mZN33r+z/hDy0BANYK3W73Zy7fuy267Xve0Z/97Gd/ /b0/v+22sb/+2V+P3XbTc3/+Pfvg926LInMwPX6TKOfPnxsbe+7PnQIRGACAVUu32/1bl+/e Gt108bs/9o7/7Y+/e/HWWx/68d/++KFbbzUJfvzdizdFt363/6XPMxe/K/+Z8tAzF50CERgA gFVLt9t9z+PwzlsPX/iOc2Tn4Qs/+s6FW2/d+aP3frTz1htNgu9cOHzjrTt/JBKnR77znQs3 RrdapXggMAAAq5Zut/s3777rfH747ddvjG489+0fpP88N7kjita/8e67P/z26+vX7/jhu++e m9xx4/odP3z33b959wc71t+4Y/L1N16fTNP8zbvvvvH65I3rd/zw3R/sWH/j069/Oy3k6R3r d0y+7pwIgQEAWLV0u92333nb/7z57dduzCe31v/H7OD69Q+++c7bb7/z9lMPru//9uBTaZbX nnrQSf/2O/9xvTmUJZOflSowW7ZsSSv0ySef1H2FAABWKN1u96f/9adL9akoMJ988klqz7ds 2bIsBGbXrl1phT766KO6rxAAwAql2+3+lyv/Zak+FQXmo48+Su35rl27loXA7NmzJ63QO++8 U/cVAgBYoXS73f+8dFQUmHfeeSe153v27FkWAjMxMZFW6OTJk3VfIQCAFUq32+0sHRUF5uTJ k6k9n5iYWBYCMzMzk1Zo69at8/PzdV8kAICVyO8sNaU1nJ+f37p1a2rPZ2ZmloXAtFqt3bt3 p3W6dOnSIlwnAAAYOZcuXUot+e7du0clLSkLEpjjx49vyvjggw+WupcAAGAwPvjgA2PGjx8/ PippSVmQwLRarb1796Y1Gxsbu3r16lL3FQAAVOXq1atjY2OpDd+7d+9IREWyUIE5c+bMAw88 kNbvoYceev/995e6xwAAoJz333//oYceSq33Aw88cObMmZGIimShAtNqtU6ePHnvvfcaJ+vS pUus+QMALFvm5+fNusumTZvuvffekydPLlwLfEYgMK1W6+TJk9u3bzfV3bp1a6vVeu+9965e vfopAAAsA65evfree++1Wi2zZ2zTpk3bt2+vSV1aoxKYVqt19uxZsx4DAADLn7179549e3ZU KuAzMoFJOX78uNm7DAAAy5Pdu3ePfM+Yz4gFJmV6evrQoUN79uzZtm3bPQBQF5v+n/974+al rgQE2Lxp46ZldXW2bdu2Z8+eQ4cOTU9P12H5fWoRGABYBF4evyPa9sxS1wIE0489/NsPP5Za 7+e++du//c3nlrhCSwwCA7ACeXn8jiiKkJflx/RjD/92n0xp1jAIDAAA1AICAwAAtYDAAABA LSAwAABQCwgMAADUAgIDAAC1gMAAAEAtIDAAAFALCAwAANQCAgPLgP7fpVss9I/UXx6/447x l/3yhyh3yFeyPLMtMjXI6qO0VKQJlJKmGt9WJfngFQOoDQQGlgE1vFQr15dntglVeXn8jsE1 ZhQC06+PX5RVvYJCRqgKCAwsEggMLANGLzBGXzz7/fL4HYOa1xEITFYfpahCc2/Sj7KLEBhY JBAYWAaErOfL43fcsW3bHWZiqz9dZEtG4GBAXyxM1sh1cqxjafWyxKI47dT5sW3Cjmf1UT2Y 9ID8Kf0enDn0zlulo7SKITVQKwgMLAN8S5rbXMfoZt/6dlE9KKx2kQUV4vPMtijyFMnyHqTp dgVBnNr+2ZzbUpHAIowvMOoX9bzlHRWoGECdIDCwDCjwYIwptNJkuqEeFPa88iN6lk5Nr1ZD PbWVXauP31JTeEWBCfZDYUcFKgZQKwgMLAOKBEbaXO+5Xz0o7HnJFJmVPVuD1wVGNf3uqe3s cn3e9y4MWRUHEBi1H4o6KlAxgHpBYGAZUFVgvDTqQWHPNYWRBl3OqRV7MGHT75Tt23FZn4JF /mE8mModhQcDSwECA8uAKgIj/6EtSJivL4/fIcuylxzEv5wlGH0NJmz69fooSx1WfbyWhpdJ ytdgAqpTtWIAdYPAwDKgksC0xEYoaR+dgy+LP7D0kkTSncnnku4Yf8Y2ynbaoH+g1SfLfsf4 +LYo/dNK5VeBUqE7xse3lSy92Oet0lFuxfrpkBqoDwQGAABqAYEBAIBa+P8BxD9vASjFZYsA AAAASUVORK5CYII= --------------DDB619A20B0C9AD190B68535-- --------------8F3B0F02ECFB4A9BFFA6F4AF--
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
Can I just follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
[root@host02 ~]# service vdsmd restart Redirecting to /bin/systemctl restart vdsmd.service [root@host02 ~]# systemctl restart ovirt-ha-broker && systemctl restart ovirt-ha-agent [root@host02 ~]# systemctl status ovirt-ha-broker ● ovirt-ha-broker.service - oVirt Hosted Engine High Availability Communications Broker Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-broker.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2016-07-28 15:09:38 ICT; 20min ago Main PID: 4614 (ovirt-ha-broker) CGroup: /system.slice/ovirt-ha-broker.service └─4614 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker --no-daemon
Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection established Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection established Jul 28 15:29:35 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection established Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection established Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection established Jul 28 15:29:48 host02.ovirt.forest.go.th ovirt-ha-broker[4614]: INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed [root@host02 ~]# systemctl status ovirt-ha-agent ● ovirt-ha-agent.service - oVirt Hosted Engine High Availability Monitoring Agent Loaded: loaded (/usr/lib/systemd/system/ovirt-ha-agent.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2016-07-28 15:28:34 ICT; 1min 19s ago Main PID: 11488 (ovirt-ha-agent) CGroup: /system.slice/ovirt-ha-agent.service └─11488 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-agent --no-daemon
Jul 28 15:29:52 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:52 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending = getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending = getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending = getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: /usr/lib/python2.7/site-packages/yajsonrpc/stomp.py:352: DeprecationWarning: Dispatcher.pend...instead. Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: pending = getattr(dispatcher, 'pending', lambda: 0) Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Error: 'Attempt to call functi...rt agent Jul 28 15:29:53 host02.ovirt.forest.go.th ovirt-ha-agent[11488]: ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Error: 'Attempt to call function: teardownIma...rt agent Hint: Some lines were ellipsized, use -l to show in full. [root@host01 ~]# hosted-engine --vm-status
--== Host 1 status ==--
Status up-to-date : True Hostname : host01.ovirt.forest.go.th Host ID : 1 Engine status : {"health": "good", "vm": "up", "detail": "up"} Score : 3400 stopped : False Local maintenance : False crc32 : d9521f2f Host timestamp : 1467916 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=1467916 (Thu Jul 28 15:29:52 2016) host-id=1 score=3400 maintenance=False state=EngineUp stopped=False
--== Host 2 status ==--
Status up-to-date : False Hostname : host02.ovirt.forest.go.th Host ID : 2 Engine status : unknown stale-data Score : 0 stopped : True Local maintenance : False crc32 : 0e1c66ec Host timestamp : 341176 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=341176 (Fri Jul 15 14:30:36 2016) host-id=2 score=0 maintenance=False state=AgentStopped stopped=True
P.S. In case if anyone is wondering, hosted03 doesn't appear when I run hosted-engine --vm-status because I installed it before upgrading hosted01 and hosted02 to the latest version, so it might be broken.
-- Wee
This is a multi-part message in MIME format. --------------D3BA50B968088C500BF7839D Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th <mailto:wee.s@forest.go.th>> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th <mailto:wee.s@forest.go.th>> wrote:
Can I just follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs? Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
-- Wee --------------D3BA50B968088C500BF7839D Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html> <head> <meta content=3D"text/html; charset=3Dutf-8" http-equiv=3D"Content-Ty= pe"> </head> <body bgcolor=3D"#FFFFFF" text=3D"#000000"> On 28/7/2559 15:54, Simone Tiraboschi wrote:<br> <blockquote cite=3D"mid:CAN8-ONo+w739YiOoXMW_13L4xFtEfNiq=3DLWtPVGeVbSvh04Amg@mail.gm= ail.com" type=3D"cite"> <div dir=3D"ltr">On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <s= pan dir=3D"ltr"><<a moz-do-not-send=3D"true" href=3D"mailto:wee.s@forest.go.th" target=3D"_blank">wee.s@fo= rest.go.th</a>></span> wrote:<br> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor=3D"#FFFFFF" text=3D"#000000"><span class=3D"">= On 21/7/2559 16:53, Simone Tiraboschi wrote:<br> </span> <blockquote type=3D"cite"> <div dir=3D"ltr"><span class=3D"">On Thu, Jul 21, 2016 = at 11:43 AM, Wee Sritippho <span dir=3D"ltr"><<a moz-do-not-send=3D"true" href=3D"mailto:wee.s@forest.go.th" target=3D"_blank">wee.s@forest.go.th</a>></s= pan> wrote:<br> </span><span class=3D""> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <div>=C2=A0</div> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div bgcolor=3D"#FFFFFF" text=3D"#000000">Can= I just follow <a moz-do-not-send=3D"true" href=3D"http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-= hosted-engine" target=3D"_blank">http://www.ovirt.org/do= cumentation/how-to/hosted-engine/#upgrade-hosted-engine</a> until step 3 and do everything else via GUI?</div> </blockquote> <div>Yes, absolutely.</div> <div bgcolor=3D"#FFFFFF" text=3D"#000000">=C2=A0= <span><font color=3D"#888888"><br> </font></span></div> </div> </div> </span></div> </blockquote> Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?<br> <br> </div> </blockquote> <div><br> </div> <div>Can you please attach ovirt-ha-agent logs?</div> <div>=C2=A0</div> </div> </div> </div> </blockquote> Yes, here are the logs:<br> <a class=3D"moz-txt-link-freetext" href=3D"https://app.box.com/s/b4ur= jty8dsuj98n3ywygpk3oh5o7pbsh">https://app.box.com/s/b4urjty8dsuj98n3ywygp= k3oh5o7pbsh</a><br> <pre class=3D"moz-signature" cols=3D"72">--=20 Wee</pre> </body> </html> --------------D3BA50B968088C500BF7839D--
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
Can I just follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17 14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path) The hosted-engine storage domain is already mounted on '/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be '/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'. Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20 Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like: mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:host03.ovirt.forest.go.th,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here. Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
Can I just follow http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI? Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17 14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path) The hosted-engine storage domain is already mounted on '/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be '/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like: mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:host03.ovirt.forest.go.th,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here. Weird. The configuration in all hosts are already referring to host01.
Also, in the storage_server_connections table: engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans --------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+---------- -+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
On Fri, Jul 29, 2016 at 11:35 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
Can I just follow
http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17
14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path) The hosted-engine storage domain is already mounted on
'/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be
'/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like:
mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:host03.ovirt.forest.go.th,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here.
Weird. The configuration in all hosts are already referring to host01.
but for sure you have a connection pointing to host02 somewhere, did you try to manually deploy from CLI connecting the gluster volume on host02?
Also, in the storage_server_connections table:
engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans --------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+---------- -+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
Aw crap. I did exactly the same thing and this could explain a lot of the issues I've been pulling out my beard over. Every time I did 'hosted-engine --deploy' on the RHEV-M|NODE host I entered the FQDN of *that* host, not the first host, as the origin of the Gluster FS volume because at the time I didn't realize that a. the manager would key deduplication on the URI of the volume b. that the volume would be mounted on FUSE, not NFS, and therefore no single point of entry is created by using the FQDN of the first host because the GFS client will persist connections with all peers On Fri, Jul 29, 2016 at 6:08 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Fri, Jul 29, 2016 at 11:35 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th>
wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
Can I just follow
http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi...
until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17
14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path)
The hosted-engine storage domain is already mounted on
'/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be
'/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like:
mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:h ost03.ovirt.forest.go.th ,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here.
Weird. The configuration in all hosts are already referring to host01.
but for sure you have a connection pointing to host02 somewhere, did you try to manually deploy from CLI connecting the gluster volume on host02?
Also, in the storage_server_connections table:
engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans
--------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+----------
-+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 30 July 2016 at 02:48, Kenneth Bingham <w@qrk.us> wrote:
Aw crap. I did exactly the same thing and this could explain a lot of the issues I've been pulling out my beard over. Every time I did 'hosted-engine --deploy' on the RHEV-M|NODE host I entered the FQDN of *that* host, not the first host, as the origin of the Gluster FS volume because at the time I didn't realize that a. the manager would key deduplication on the URI of the volume b. that the volume would be mounted on FUSE, not NFS, and therefore no single point of entry is created by using the FQDN of the first host because the GFS client will persist connections with all peers
If you ever want to add an hosted engine host to your setup please do that from UI and not from CLI. That will prevent all this confusion.
On Fri, Jul 29, 2016 at 6:08 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Fri, Jul 29, 2016 at 11:35 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th>
wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
> Can I just follow > >
http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi...
> until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17
14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path)
The hosted-engine storage domain is already mounted on
'/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be
'/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like:
mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:h ost03.ovirt.forest.go.th ,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here.
Weird. The configuration in all hosts are already referring to host01.
but for sure you have a connection pointing to host02 somewhere, did you try to manually deploy from CLI connecting the gluster volume on host02?
Also, in the storage_server_connections table:
engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans
--------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+----------
-+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Roy, would I do that in the Cluster tab, with New button, and then in host configurator select Hosted Engine sidebar? I noticed this option existed, but the RHEV docs and developer blogs I've been referencing specify the 'hosted-engine --deploy' method. On Sun, Jul 31, 2016 at 3:04 AM Roy Golan <rgolan@redhat.com> wrote:
On 30 July 2016 at 02:48, Kenneth Bingham <w@qrk.us> wrote:
Aw crap. I did exactly the same thing and this could explain a lot of the issues I've been pulling out my beard over. Every time I did 'hosted-engine --deploy' on the RHEV-M|NODE host I entered the FQDN of *that* host, not the first host, as the origin of the Gluster FS volume because at the time I didn't realize that a. the manager would key deduplication on the URI of the volume b. that the volume would be mounted on FUSE, not NFS, and therefore no single point of entry is created by using the FQDN of the first host because the GFS client will persist connections with all peers
If you ever want to add an hosted engine host to your setup please do that from UI and not from CLI. That will prevent all this confusion.
On Fri, Jul 29, 2016 at 6:08 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Fri, Jul 29, 2016 at 11:35 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th>
wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote: > > On 21/7/2559 16:53, Simone Tiraboschi wrote: > > On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th
> wrote: > >> Can I just follow >> >> http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... >> until step 3 and do everything else via GUI? > > Yes, absolutely. > > > Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted > the services but the result is still the same. Kinda lost now. What > should I > do next? > Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17
14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path)
The hosted-engine storage domain is already mounted on
'/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be
'/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like:
mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:h ost03.ovirt.forest.go.th ,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here.
Weird. The configuration in all hosts are already referring to host01.
but for sure you have a connection pointing to host02 somewhere, did you try to manually deploy from CLI connecting the gluster volume on host02?
Also, in the storage_server_connections table:
engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans
--------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+----------
-+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Roy, would I do that in the Cluster tab, with New button, and then in host configurator select Hosted Engine sidebar? I noticed this option existed, but the RHEV docs and developer blogs I've been referencing specify the 'hosted-engine --deploy' method.
on 4.0 this is the supported way of doing that. If fixes and prevents many
On 31 July 2016 at 18:26, Kenneth Bingham <w@qrk.us> wrote: troubles and more than that, its simply doing that in 1 place. The doc will be updated for that if not already. Also please refer to this ovirt blog post [1] [1] http://www.ovirt.org/blog/2016/07/Manage-Your-Hosted-Engine-Hosts-Deployment...
On Sun, Jul 31, 2016 at 3:04 AM Roy Golan <rgolan@redhat.com> wrote:
On 30 July 2016 at 02:48, Kenneth Bingham <w@qrk.us> wrote:
Aw crap. I did exactly the same thing and this could explain a lot of the issues I've been pulling out my beard over. Every time I did 'hosted-engine --deploy' on the RHEV-M|NODE host I entered the FQDN of *that* host, not the first host, as the origin of the Gluster FS volume because at the time I didn't realize that a. the manager would key deduplication on the URI of the volume b. that the volume would be mounted on FUSE, not NFS, and therefore no single point of entry is created by using the FQDN of the first host because the GFS client will persist connections with all peers
If you ever want to add an hosted engine host to your setup please do that from UI and not from CLI. That will prevent all this confusion.
On Fri, Jul 29, 2016 at 6:08 AM Simone Tiraboschi <stirabos@redhat.com> wrote:
On Fri, Jul 29, 2016 at 11:35 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th>
wrote:
> > On 28/7/2559 15:54, Simone Tiraboschi wrote: > > On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th
> wrote: >> >> On 21/7/2559 16:53, Simone Tiraboschi wrote: >> >> On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho < wee.s@forest.go.th> >> wrote: >> >>> Can I just follow >>> >>> http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... >>> until step 3 and do everything else via GUI? >> >> Yes, absolutely. >> >> >> Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted >> the services but the result is still the same. Kinda lost now. What >> should I >> do next? >> > Can you please attach ovirt-ha-agent logs? > > > Yes, here are the logs: > https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17
14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path)
The hosted-engine storage domain is already mounted on
'/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be
'/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th: _hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th: _hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like:
mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:h ost03.ovirt.forest.go.th ,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here.
Weird. The configuration in all hosts are already referring to host01.
but for sure you have a connection pointing to host02 somewhere, did you try to manually deploy from CLI connecting the gluster volume on host02?
Also, in the storage_server_connections table:
engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans
--------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+----------
-+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On 29/7/2559 17:07, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 11:35 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
Can I just follow
http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... until step 3 and do everything else via GUI? Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17
14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path) The hosted-engine storage domain is already mounted on
'/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be
'/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like:
mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:host03.ovirt.forest.go.th,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here. Weird. The configuration in all hosts are already referring to host01. but for sure you have a connection pointing to host02 somewhere, did you try to manually deploy from CLI connecting the gluster volume on host02? If I recall correctly, yes. Also, in the storage_server_connections table:
engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans --------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+---------- -+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
-- Wee
On Mon, Aug 1, 2016 at 3:54 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 17:07, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 11:35 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 29/7/2559 15:50, Simone Tiraboschi wrote:
On Fri, Jul 29, 2016 at 6:31 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 28/7/2559 15:54, Simone Tiraboschi wrote:
On Thu, Jul 28, 2016 at 10:41 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
On 21/7/2559 16:53, Simone Tiraboschi wrote:
On Thu, Jul 21, 2016 at 11:43 AM, Wee Sritippho <wee.s@forest.go.th> wrote:
> Can I just follow > > > http://www.ovirt.org/documentation/how-to/hosted-engine/#upgrade-hosted-engi... > until step 3 and do everything else via GUI?
Yes, absolutely.
Hi, I upgrade a host (host02) via GUI and now its score is 0. Restarted the services but the result is still the same. Kinda lost now. What should I do next?
Can you please attach ovirt-ha-agent logs?
Yes, here are the logs: https://app.box.com/s/b4urjty8dsuj98n3ywygpk3oh5o7pbsh
Thanks Wee, your issue is here: MainThread::ERROR::2016-07-17
14:32:45,586::storage_server::143::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(_validate_pre_connected_path) The hosted-engine storage domain is already mounted on
'/rhev/data-center/mnt/glusterSD/host02.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4' with a path that is not supported anymore: the right path should be
'/rhev/data-center/mnt/glusterSD/host01.ovirt.forest.go.th:_hosted__engine/639e689c-8493-479b-a6eb-cc92b6fc4cf4'.
Did you manually tried to avoid the issue of a single entry point for the gluster FS volume using host01.ovirt.forest.go.th:_hosted__engine and host02.ovirt.forest.go.th:_hosted__engine there? This could cause a lot of confusion since the code could not detect that the storage domain is the same and you can end with it mounted twice into different locations and a lot of issues. The correct solution of that issue was this one: https://bugzilla.redhat.com/show_bug.cgi?id=1298693#c20
Now, to have it fixed on your env you have to hack a bit. First step, you have to edit /etc/ovirt-hosted-engine/hosted-engine.conf on all your hosted-engine hosts to ensure that the storage field always point to the same entry point (host01 for instance) Then on each host you can add something like:
mnt_options=backupvolfile-server=host02.ovirt.forest.go.th:host03.ovirt.forest.go.th,fetch-attempts=2,log-level=WARNING,log-file=/var/log/engine_domain.log
Then check the representation of your storage connection in the table storage_server_connections of the engine DB and make sure that connection refers to the entry point you used in hosted-engine.conf on all your hosts, you have lastly to set the value of mount_options also here.
Weird. The configuration in all hosts are already referring to host01.
but for sure you have a connection pointing to host02 somewhere, did you try to manually deploy from CLI connecting the gluster volume on host02?
If I recall correctly, yes.
Ok, so please reboot your host before trying again to make sure that every reference get cleaned.
Also, in the storage_server_connections table:
engine=> SELECT * FROM storage_server_connections; id | connection | user_name | password | iqn | port | portal | storage_type | mount_options | vfs_type | nfs_version | nfs_timeo | nfs_retrans
--------------------------------------+------------------------------------------+-----------+----------+-----+------+--------+--------------+---------------+---------- -+-------------+-----------+------------- bd78d299-c8ff-4251-8aab-432ce6443ae8 | host01.ovirt.forest.go.th:/hosted_engine | | | | | 1 | 7 | | glusterfs | | | (1 row)
Please tune also the value of network.ping-timeout for your glusterFS volume to avoid this: https://bugzilla.redhat.com/show_bug.cgi?id=1319657#c17
-- Wee
-- Wee
participants (4)
-
Kenneth Bingham -
Roy Golan -
Simone Tiraboschi -
Wee Sritippho