From jaicel at asti.dost.gov.ph Thu Oct 30 04:23:03 2014 Content-Type: multipart/mixed; boundary="===============1635798293437166669==" MIME-Version: 1.0 From: Jaicel R. Sabonsolin To: users at ovirt.org Subject: [ovirt-users] Hosted-Engine HA problem Date: Thu, 30 Oct 2014 16:22:52 +0800 Message-ID: <704950385.234529.1414657372047.JavaMail.zimbra@asti.dost.gov.ph> --===============1635798293437166669== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable ------=3D_Part_234528_2091500974.1414657372046 Content-Type: text/plain; charset=3Dutf-8 Content-Transfer-Encoding: 7bit Hi Guys, = I need help with my ovirt Hosted-Engine HA setup. I am running on 2 ovirt h= osts and 2 gluster nodes with replicated volumes. i already have VMs runnin= g on my hosts and they can migrate normally once i for example power off th= e host that they are running on. the problem is that the engine can't migra= te once i switch off the host that hosts the engine. = oVirt 3.4.3-1.el6 = KVM 0.12.1.2 - 2.415.el6_5.10 = LIBVIRT libvirt-0.10.2-29.el6_5.9 = VDSM vdsm-4.14.17-0.el6 = right now, i have this result from hosted-engine --vm-status. = BQ_BEGIN File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main = "__main__", fname, loader, pkg_name) = File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code = exec code in run_globals = File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status.= py", line 111, in = if not status_checker.print_status(): = File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status.= py", line 58, in print_status = all_host_stats =3D ha_cli.get_all_host_stats() = File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client= .py", line 137, in get_all_host_stats = return self.get_all_stats(self.StatModes.HOST) = File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client= .py", line 86, in get_all_stats = constants.SERVICE_TYPE) = File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlin= k.py", line 171, in get_stats_from_storage = result =3D self._checked_communicate(request) = File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlin= k.py", line 199, in _checked_communicate = .format(message or response)) = ovirt_hosted_engine_ha.lib.exceptions.RequestError: Request failed: = BQ_END restarting ha-broker and ha-agent normalizes the status but eventually it w= ould become "false" and then return to the result above. hope you guys coul= d help me with this. = Thanks, = Jaicel = ------=3D_Part_234528_2091500974.1414657372046 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: quoted-printable

Hi Guys,

I need help with my ovirt Hosted-Engine HA setup. I am = ru=3D nning on 2 ovirt hosts and 2 gluster nodes with replicated volumes. i alrea= =3D dy have VMs running on my hosts and they can migrate normally once i for ex= =3D ample power off the host that they are running on. the problem is that the = =3D engine can't migrate once i switch off the host that hosts the engine.

=3D

oVirt = =3D 3.4.3-1.el6

KVM &nbs= =3D p; 0.12.1.2 - 2.415.el6_5.10

LIBVIRT libvi= =3D rt-0.10.2-29.el6_5.9

VDSM &= nb=3D sp; vdsm-4.14.17-0.el6

=

right now, i have this result from= h=3D osted-engine --vm-status.

File "/usr/lib64/python2.6/runpy= .p=3D y", line 122, in _run_module_as_main
    "__main__", fnam= =3D e, loader, pkg_name)
File "/usr/lib64/python2.6/runpy.py", line 3= =3D 4, in _run_code
    exec code in run_globals
Fi= =3D le "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status.py= =3D ", line 111, in <module>
    if not status_checker.= =3D print_status():
File "/usr/lib/python2.6/site-packages/ovirt_host= =3D ed_engine_setup/vm_status.py", line 58, in print_status
  &nbs= =3D p; all_host_stats =3D3D ha_cli.get_all_host_stats()
File "/usr/li= b/=3D python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 137,= =3D in get_all_host_stats
    return self.get_all_stats(self= =3D .StatModes.HOST)
File "/usr/lib/python2.6/site-packages/ovirt_hos= =3D ted_engine_ha/client/client.py", line 86, in get_all_stats
  &= =3D nbsp; constants.SERVICE_TYPE)
File "/usr/lib/python2.6/site-packa= =3D ges/ovirt_hosted_engine_ha/lib/brokerlink.py", line 171, in get_stats_from_= =3D storage
    result =3D3D self._checked_communicate(reques= t)=3D
File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/li= =3D b/brokerlink.py", line 199, in _checked_communicate
    .= =3D format(message or response))
ovirt_hosted_engine_ha.lib.exceptions.Reque= =3D stError: Request failed: <type 'exceptions.OSError'>

restarting ha-broker and ha-ag= en=3D t normalizes the status but eventually it would become "false" and then ret= =3D urn to the result above. hope you guys could help me with this.

Thanks,
Jaicel
------=3D_Part_234528_2091500974.1414657372046-- --===============1635798293437166669== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" LS0tLS0tPV9QYXJ0XzIzNDUyOF8yMDkxNTAwOTc0LjE0MTQ2NTczNzIwNDYKQ29udGVudC1UeXBl OiB0ZXh0L3BsYWluOyBjaGFyc2V0PXV0Zi04CkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdi aXQKCkhpIEd1eXMsIAoKSSBuZWVkIGhlbHAgd2l0aCBteSBvdmlydCBIb3N0ZWQtRW5naW5lIEhB IHNldHVwLiBJIGFtIHJ1bm5pbmcgb24gMiBvdmlydCBob3N0cyBhbmQgMiBnbHVzdGVyIG5vZGVz IHdpdGggcmVwbGljYXRlZCB2b2x1bWVzLiBpIGFscmVhZHkgaGF2ZSBWTXMgcnVubmluZyBvbiBt eSBob3N0cyBhbmQgdGhleSBjYW4gbWlncmF0ZSBub3JtYWxseSBvbmNlIGkgZm9yIGV4YW1wbGUg cG93ZXIgb2ZmIHRoZSBob3N0IHRoYXQgdGhleSBhcmUgcnVubmluZyBvbi4gdGhlIHByb2JsZW0g aXMgdGhhdCB0aGUgZW5naW5lIGNhbid0IG1pZ3JhdGUgb25jZSBpIHN3aXRjaCBvZmYgdGhlIGhv c3QgdGhhdCBob3N0cyB0aGUgZW5naW5lLiAKCgoKCm9WaXJ0IDMuNC4zLTEuZWw2IApLVk0gMC4x Mi4xLjIgLSAyLjQxNS5lbDZfNS4xMCAKTElCVklSVCBsaWJ2aXJ0LTAuMTAuMi0yOS5lbDZfNS45 IApWRFNNIHZkc20tNC4xNC4xNy0wLmVsNiAKCgoKCnJpZ2h0IG5vdywgaSBoYXZlIHRoaXMgcmVz dWx0IGZyb20gaG9zdGVkLWVuZ2luZSAtLXZtLXN0YXR1cy4gCgoKQlFfQkVHSU4KCkZpbGUgIi91 c3IvbGliNjQvcHl0aG9uMi42L3J1bnB5LnB5IiwgbGluZSAxMjIsIGluIF9ydW5fbW9kdWxlX2Fz X21haW4gCiJfX21haW5fXyIsIGZuYW1lLCBsb2FkZXIsIHBrZ19uYW1lKSAKRmlsZSAiL3Vzci9s aWI2NC9weXRob24yLjYvcnVucHkucHkiLCBsaW5lIDM0LCBpbiBfcnVuX2NvZGUgCmV4ZWMgY29k ZSBpbiBydW5fZ2xvYmFscyAKRmlsZSAiL3Vzci9saWIvcHl0aG9uMi42L3NpdGUtcGFja2FnZXMv b3ZpcnRfaG9zdGVkX2VuZ2luZV9zZXR1cC92bV9zdGF0dXMucHkiLCBsaW5lIDExMSwgaW4gPG1v ZHVsZT4gCmlmIG5vdCBzdGF0dXNfY2hlY2tlci5wcmludF9zdGF0dXMoKTogCkZpbGUgIi91c3Iv bGliL3B5dGhvbjIuNi9zaXRlLXBhY2thZ2VzL292aXJ0X2hvc3RlZF9lbmdpbmVfc2V0dXAvdm1f c3RhdHVzLnB5IiwgbGluZSA1OCwgaW4gcHJpbnRfc3RhdHVzIAphbGxfaG9zdF9zdGF0cyA9IGhh X2NsaS5nZXRfYWxsX2hvc3Rfc3RhdHMoKSAKRmlsZSAiL3Vzci9saWIvcHl0aG9uMi42L3NpdGUt cGFja2FnZXMvb3ZpcnRfaG9zdGVkX2VuZ2luZV9oYS9jbGllbnQvY2xpZW50LnB5IiwgbGluZSAx MzcsIGluIGdldF9hbGxfaG9zdF9zdGF0cyAKcmV0dXJuIHNlbGYuZ2V0X2FsbF9zdGF0cyhzZWxm LlN0YXRNb2Rlcy5IT1NUKSAKRmlsZSAiL3Vzci9saWIvcHl0aG9uMi42L3NpdGUtcGFja2FnZXMv b3ZpcnRfaG9zdGVkX2VuZ2luZV9oYS9jbGllbnQvY2xpZW50LnB5IiwgbGluZSA4NiwgaW4gZ2V0 X2FsbF9zdGF0cyAKY29uc3RhbnRzLlNFUlZJQ0VfVFlQRSkgCkZpbGUgIi91c3IvbGliL3B5dGhv bjIuNi9zaXRlLXBhY2thZ2VzL292aXJ0X2hvc3RlZF9lbmdpbmVfaGEvbGliL2Jyb2tlcmxpbmsu cHkiLCBsaW5lIDE3MSwgaW4gZ2V0X3N0YXRzX2Zyb21fc3RvcmFnZSAKcmVzdWx0ID0gc2VsZi5f Y2hlY2tlZF9jb21tdW5pY2F0ZShyZXF1ZXN0KSAKRmlsZSAiL3Vzci9saWIvcHl0aG9uMi42L3Np dGUtcGFja2FnZXMvb3ZpcnRfaG9zdGVkX2VuZ2luZV9oYS9saWIvYnJva2VybGluay5weSIsIGxp bmUgMTk5LCBpbiBfY2hlY2tlZF9jb21tdW5pY2F0ZSAKLmZvcm1hdChtZXNzYWdlIG9yIHJlc3Bv bnNlKSkgCm92aXJ0X2hvc3RlZF9lbmdpbmVfaGEubGliLmV4Y2VwdGlvbnMuUmVxdWVzdEVycm9y OiBSZXF1ZXN0IGZhaWxlZDogPHR5cGUgJ2V4Y2VwdGlvbnMuT1NFcnJvcic+IAoKQlFfRU5ECgoK cmVzdGFydGluZyBoYS1icm9rZXIgYW5kIGhhLWFnZW50IG5vcm1hbGl6ZXMgdGhlIHN0YXR1cyBi dXQgZXZlbnR1YWxseSBpdCB3b3VsZCBiZWNvbWUgImZhbHNlIiBhbmQgdGhlbiByZXR1cm4gdG8g dGhlIHJlc3VsdCBhYm92ZS4gaG9wZSB5b3UgZ3V5cyBjb3VsZCBoZWxwIG1lIHdpdGggdGhpcy4g CgpUaGFua3MsIApKYWljZWwgCgotLS0tLS09X1BhcnRfMjM0NTI4XzIwOTE1MDA5NzQuMTQxNDY1 NzM3MjA0NgpDb250ZW50LVR5cGU6IHRleHQvaHRtbDsgY2hhcnNldD11dGYtOApDb250ZW50LVRy YW5zZmVyLUVuY29kaW5nOiBxdW90ZWQtcHJpbnRhYmxlCgo8aHRtbD48Ym9keT48ZGl2IHN0eWxl PTNEImZvbnQtZmFtaWx5OiBhcmlhbCwgaGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXM9Cml6 ZTogMTBwdDsgY29sb3I6ICMwMDAwMDAiPjxkaXY+SGkgR3V5cyw8YnI+PC9kaXY+PGRpdj48YnIg ZGF0YS1tY2UtYm9ndXM9Cj0zRCIxIj48L2Rpdj48ZGl2PkkgbmVlZCBoZWxwIHdpdGggbXkgb3Zp cnQgSG9zdGVkLUVuZ2luZSBIQSBzZXR1cC4gSSBhbSBydT0Kbm5pbmcgb24gMiBvdmlydCBob3N0 cyBhbmQgMiBnbHVzdGVyIG5vZGVzIHdpdGggcmVwbGljYXRlZCB2b2x1bWVzLiBpIGFscmVhPQpk eSBoYXZlIFZNcyBydW5uaW5nIG9uIG15IGhvc3RzIGFuZCB0aGV5IGNhbiBtaWdyYXRlIG5vcm1h bGx5IG9uY2UgaSBmb3IgZXg9CmFtcGxlIHBvd2VyIG9mZiB0aGUgaG9zdCB0aGF0IHRoZXkgYXJl IHJ1bm5pbmcgb24uIHRoZSBwcm9ibGVtIGlzIHRoYXQgdGhlID0KZW5naW5lIGNhbid0IG1pZ3Jh dGUgb25jZSBpIHN3aXRjaCBvZmYgdGhlIGhvc3QgdGhhdCBob3N0cyB0aGUgZW5naW5lLiA8YnIg PQpkYXRhLW1jZS1ib2d1cz0zRCIxIj48L2Rpdj48ZGl2PjxiciBkYXRhLW1jZS1ib2d1cz0zRCIx Ij48L2Rpdj48YmxvY2txdW90ZT49CjxkaXY+PHNwYW4gZGF0YS1tY2Utc3R5bGU9M0QiY29sb3I6 ICMxMTExMTE7IGZvbnQtZmFtaWx5OiBWZXJkYW5hOyBmb250LXNpej0KZTogMTBwdDsgZm9udC1z dHlsZTogbm9ybWFsOyBmb250LXZhcmlhbnQ6IG5vcm1hbDsgZm9udC13ZWlnaHQ6IG5vcm1hbDsg bGV0PQp0ZXItc3BhY2luZzogbm9ybWFsOyBsaW5lLWhlaWdodDogMjRweDsgdGV4dC1hbGlnbjog bGVmdDsgdGV4dC1pbmRlbnQ6IDBweDs9CiB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3Bh Y2U6IG5vcm1hbDsgd29yZC1zcGFjaW5nOiAwcHg7IGRpc3BsYXk6IGlubD0KaW5lICEgaW1wb3J0 YW50OyBmbG9hdDogbm9uZTsiIHN0eWxlPTNEImNvbG9yOiByZ2IoMTcsIDE3LCAxNyk7IGZvbnQt ZmFtaWx5PQo6IFZlcmRhbmE7IGZvbnQtc2l6ZTogMTBwdDsgZm9udC1zdHlsZTogbm9ybWFsOyBm b250LXZhcmlhbnQ6IG5vcm1hbDsgZm9udC09CndlaWdodDogbm9ybWFsOyBsZXR0ZXItc3BhY2lu Zzogbm9ybWFsOyBsaW5lLWhlaWdodDogMjRweDsgdGV4dC1hbGlnbjogbGVmdD0KOyB0ZXh0LWlu ZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2hpdGUtc3BhY2U6IG5vcm1hbDsgd29y ZC1zcGFjaW5nPQo6IDBweDsgZGlzcGxheTogaW5saW5lICEgaW1wb3J0YW50OyBmbG9hdDogbm9u ZTsiPm9WaXJ0ICZuYnNwOyZuYnNwOyAmbmJzcDs9CiAmbmJzcDsgMy40LjMtMS5lbDY8L3NwYW4+ PC9kaXY+PGRpdj5LVk0gJm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5icz0KcDsmbmJz cDsmbmJzcDswLjEyLjEuMiAtIDIuNDE1LmVsNl81LjEwPC9kaXY+PGRpdj5MSUJWSVJUJm5ic3A7 Jm5ic3A7IGxpYnZpPQpydC0wLjEwLjItMjkuZWw2XzUuOTxiciBkYXRhLW1jZS1ib2d1cz0zRCIx Ij48L2Rpdj48ZGl2PlZEU00mbmJzcDsmbmJzcDsmbmI9CnNwOyAmbmJzcDsgdmRzbS00LjE0LjE3 LTAuZWw2PGJyIGRhdGEtbWNlLWJvZ3VzPTNEIjEiPjwvZGl2PjwvYmxvY2txdW90ZT48ZD0KaXY+ PGJyIGRhdGEtbWNlLWJvZ3VzPTNEIjEiPjwvZGl2PjxkaXY+cmlnaHQgbm93LCBpIGhhdmUgdGhp cyByZXN1bHQgZnJvbSBoPQpvc3RlZC1lbmdpbmUgLS12bS1zdGF0dXMuPGJyIGRhdGEtbWNlLWJv Z3VzPTNEIjEiPjwvZGl2PjxkaXY+PGJyIGRhdGEtbWNlLWI9Cm9ndXM9M0QiMSI+PC9kaXY+PGJs b2NrcXVvdGU+PGRpdj4mbmJzcDsgRmlsZSAiL3Vzci9saWI2NC9weXRob24yLjYvcnVucHkucD0K eSIsIGxpbmUgMTIyLCBpbiBfcnVuX21vZHVsZV9hc19tYWluPGJyPiZuYnNwOyZuYnNwOyZuYnNw OyAiX19tYWluX18iLCBmbmFtPQplLCBsb2FkZXIsIHBrZ19uYW1lKTxicj4mbmJzcDsgRmlsZSAi L3Vzci9saWI2NC9weXRob24yLjYvcnVucHkucHkiLCBsaW5lIDM9CjQsIGluIF9ydW5fY29kZTxi cj4mbmJzcDsmbmJzcDsmbmJzcDsgZXhlYyBjb2RlIGluIHJ1bl9nbG9iYWxzPGJyPiZuYnNwOyBG aT0KbGUgIi91c3IvbGliL3B5dGhvbjIuNi9zaXRlLXBhY2thZ2VzL292aXJ0X2hvc3RlZF9lbmdp bmVfc2V0dXAvdm1fc3RhdHVzLnB5PQoiLCBsaW5lIDExMSwgaW4gJmx0O21vZHVsZSZndDs8YnI+ Jm5ic3A7Jm5ic3A7Jm5ic3A7IGlmIG5vdCBzdGF0dXNfY2hlY2tlci49CnByaW50X3N0YXR1cygp Ojxicj4mbmJzcDsgRmlsZSAiL3Vzci9saWIvcHl0aG9uMi42L3NpdGUtcGFja2FnZXMvb3ZpcnRf aG9zdD0KZWRfZW5naW5lX3NldHVwL3ZtX3N0YXR1cy5weSIsIGxpbmUgNTgsIGluIHByaW50X3N0 YXR1czxicj4mbmJzcDsmbmJzcDsmbmJzPQpwOyBhbGxfaG9zdF9zdGF0cyA9M0QgaGFfY2xpLmdl dF9hbGxfaG9zdF9zdGF0cygpPGJyPiZuYnNwOyBGaWxlICIvdXNyL2xpYi89CnB5dGhvbjIuNi9z aXRlLXBhY2thZ2VzL292aXJ0X2hvc3RlZF9lbmdpbmVfaGEvY2xpZW50L2NsaWVudC5weSIsIGxp bmUgMTM3LD0KIGluIGdldF9hbGxfaG9zdF9zdGF0czxicj4mbmJzcDsmbmJzcDsmbmJzcDsgcmV0 dXJuIHNlbGYuZ2V0X2FsbF9zdGF0cyhzZWxmPQouU3RhdE1vZGVzLkhPU1QpPGJyPiZuYnNwOyBG aWxlICIvdXNyL2xpYi9weXRob24yLjYvc2l0ZS1wYWNrYWdlcy9vdmlydF9ob3M9CnRlZF9lbmdp bmVfaGEvY2xpZW50L2NsaWVudC5weSIsIGxpbmUgODYsIGluIGdldF9hbGxfc3RhdHM8YnI+Jm5i c3A7Jm5ic3A7Jj0KbmJzcDsgY29uc3RhbnRzLlNFUlZJQ0VfVFlQRSk8YnI+Jm5ic3A7IEZpbGUg Ii91c3IvbGliL3B5dGhvbjIuNi9zaXRlLXBhY2thPQpnZXMvb3ZpcnRfaG9zdGVkX2VuZ2luZV9o YS9saWIvYnJva2VybGluay5weSIsIGxpbmUgMTcxLCBpbiBnZXRfc3RhdHNfZnJvbV89CnN0b3Jh Z2U8YnI+Jm5ic3A7Jm5ic3A7Jm5ic3A7IHJlc3VsdCA9M0Qgc2VsZi5fY2hlY2tlZF9jb21tdW5p Y2F0ZShyZXF1ZXN0KT0KPGJyPiZuYnNwOyBGaWxlICIvdXNyL2xpYi9weXRob24yLjYvc2l0ZS1w YWNrYWdlcy9vdmlydF9ob3N0ZWRfZW5naW5lX2hhL2xpPQpiL2Jyb2tlcmxpbmsucHkiLCBsaW5l IDE5OSwgaW4gX2NoZWNrZWRfY29tbXVuaWNhdGU8YnI+Jm5ic3A7Jm5ic3A7Jm5ic3A7IC49CmZv cm1hdChtZXNzYWdlIG9yIHJlc3BvbnNlKSk8YnI+b3ZpcnRfaG9zdGVkX2VuZ2luZV9oYS5saWIu ZXhjZXB0aW9ucy5SZXF1ZT0Kc3RFcnJvcjogUmVxdWVzdCBmYWlsZWQ6ICZsdDt0eXBlICdleGNl cHRpb25zLk9TRXJyb3InJmd0OzwvZGl2PjwvYmxvY2txdW90PQplPjxkaXY+PGJyIGRhdGEtbWNl LWJvZ3VzPTNEIjEiPjwvZGl2PjxkaXY+cmVzdGFydGluZyBoYS1icm9rZXIgYW5kIGhhLWFnZW49 CnQgbm9ybWFsaXplcyB0aGUgc3RhdHVzIGJ1dCBldmVudHVhbGx5IGl0IHdvdWxkIGJlY29tZSAi ZmFsc2UiIGFuZCB0aGVuIHJldD0KdXJuIHRvIHRoZSByZXN1bHQgYWJvdmUuIGhvcGUgeW91IGd1 eXMgY291bGQgaGVscCBtZSB3aXRoIHRoaXMuPGJyIGRhdGEtbWNlPQotYm9ndXM9M0QiMSI+PC9k aXY+PGRpdj48YnIgZGF0YS1tY2UtYm9ndXM9M0QiMSI+PC9kaXY+PGRpdiBkYXRhLW1hcmtlcj0z RCI9Cl9fU0lHX1BSRV9fIj5UaGFua3MsPGJyPkphaWNlbDwvZGl2PjwvZGl2PjwvYm9keT48L2h0 bWw+Ci0tLS0tLT1fUGFydF8yMzQ1MjhfMjA5MTUwMDk3NC4xNDE0NjU3MzcyMDQ2LS0K --===============1635798293437166669==-- From jmoskovc at redhat.com Thu Oct 30 09:15:38 2014 Content-Type: multipart/mixed; boundary="===============5528456198905904949==" MIME-Version: 1.0 From: Jiri Moskovcak To: users at ovirt.org Subject: Re: [ovirt-users] Hosted-Engine HA problem Date: Thu, 30 Oct 2014 14:15:30 +0100 Message-ID: <545239F2.4020502@redhat.com> In-Reply-To: 704950385.234529.1414657372047.JavaMail.zimbra@asti.dost.gov.ph --===============5528456198905904949== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On 10/30/2014 09:22 AM, Jaicel R. Sabonsolin wrote: > Hi Guys, > > I need help with my ovirt Hosted-Engine HA setup. I am running on 2 > ovirt hosts and 2 gluster nodes with replicated volumes. i already have > VMs running on my hosts and they can migrate normally once i for example > power off the host that they are running on. the problem is that the > engine can't migrate once i switch off the host that hosts the engine. > > oVirt 3.4.3-1.el6 > KVM 0.12.1.2 - 2.415.el6_5.10 > LIBVIRT libvirt-0.10.2-29.el6_5.9 > VDSM vdsm-4.14.17-0.el6 > > > right now, i have this result from hosted-engine --vm-status. > > File "/usr/lib64/python2.6/runpy.py", line 122, in > _run_module_as_main > "__main__", fname, loader, pkg_name) > File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code > exec code in run_globals > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status= .py", > line 111, in > if not status_checker.print_status(): > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status= .py", > line 58, in print_status > all_host_stats =3D ha_cli.get_all_host_stats() > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/clien= t.py", > line 137, in get_all_host_stats > return self.get_all_stats(self.StatModes.HOST) > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/clien= t.py", > line 86, in get_all_stats > constants.SERVICE_TYPE) > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerli= nk.py", > line 171, in get_stats_from_storage > result =3D self._checked_communicate(request) > File > "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerli= nk.py", > line 199, in _checked_communicate > .format(message or response)) > ovirt_hosted_engine_ha.lib.exceptions.RequestError: Request failed: > > > > restarting ha-broker and ha-agent normalizes the status but eventually > it would become "false" and then return to the result above. hope you > guys could help me with this. > Hi Jaicel, please attach agent.log and broker.log from the host where you trying to = run hosted-engine --vm-status. I have a feeling that you ran into a = known problem on gluster - stalled file descriptor, in that case the = only known solution at this time is to restart the broker & agent as you = have already found out. Regards, Jirka > Thanks, > Jaicel > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > --===============5528456198905904949==-- From vbellur at redhat.com Thu Oct 30 11:38:03 2014 Content-Type: multipart/mixed; boundary="===============7710819019199758777==" MIME-Version: 1.0 From: Vijay Bellur To: users at ovirt.org Subject: Re: [ovirt-users] Hosted-Engine HA problem Date: Thu, 30 Oct 2014 21:07:24 +0530 Message-ID: <54525B34.2080202@redhat.com> In-Reply-To: 545239F2.4020502@redhat.com --===============7710819019199758777== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On 10/30/2014 06:45 PM, Jiri Moskovcak wrote: > On 10/30/2014 09:22 AM, Jaicel R. Sabonsolin wrote: >> Hi Guys, >> >> I need help with my ovirt Hosted-Engine HA setup. I am running on 2 >> ovirt hosts and 2 gluster nodes with replicated volumes. i already have >> VMs running on my hosts and they can migrate normally once i for example >> power off the host that they are running on. the problem is that the >> engine can't migrate once i switch off the host that hosts the engine. >> >> oVirt 3.4.3-1.el6 >> KVM 0.12.1.2 - 2.415.el6_5.10 >> LIBVIRT libvirt-0.10.2-29.el6_5.9 >> VDSM vdsm-4.14.17-0.el6 >> >> >> right now, i have this result from hosted-engine --vm-status. >> >> File "/usr/lib64/python2.6/runpy.py", line 122, in >> _run_module_as_main >> "__main__", fname, loader, pkg_name) >> File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code >> exec code in run_globals >> File >> >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status.py= ", >> >> line 111, in >> if not status_checker.print_status(): >> File >> >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status.py= ", >> >> line 58, in print_status >> all_host_stats =3D ha_cli.get_all_host_stats() >> File >> >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.p= y", >> >> line 137, in get_all_host_stats >> return self.get_all_stats(self.StatModes.HOST) >> File >> >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.p= y", >> >> line 86, in get_all_stats >> constants.SERVICE_TYPE) >> File >> >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.= py", >> >> line 171, in get_stats_from_storage >> result =3D self._checked_communicate(request) >> File >> >> "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.= py", >> >> line 199, in _checked_communicate >> .format(message or response)) >> ovirt_hosted_engine_ha.lib.exceptions.RequestError: Request failed: >> >> >> >> restarting ha-broker and ha-agent normalizes the status but eventually >> it would become "false" and then return to the result above. hope you >> guys could help me with this. >> > > Hi Jaicel, > please attach agent.log and broker.log from the host where you trying to > run hosted-engine --vm-status. I have a feeling that you ran into a > known problem on gluster - stalled file descriptor, in that case the > only known solution at this time is to restart the broker & agent as you > have already found out. > Adding Niels and gluster-devel to troubleshoot from Gluster NFS perspective. Thanks, Vijay --===============7710819019199758777==-- From ndevos at redhat.com Thu Oct 30 16:11:36 2014 Content-Type: multipart/mixed; boundary="===============2595986334058809138==" MIME-Version: 1.0 From: Niels de Vos To: users at ovirt.org Subject: Re: [ovirt-users] Hosted-Engine HA problem Date: Thu, 30 Oct 2014 21:11:25 +0100 Message-ID: <20141030201125.GJ13542@ndevos-x240.usersys.redhat.com> In-Reply-To: 54525B34.2080202@redhat.com --===============2595986334058809138== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Thu, Oct 30, 2014 at 09:07:24PM +0530, Vijay Bellur wrote: > On 10/30/2014 06:45 PM, Jiri Moskovcak wrote: > >On 10/30/2014 09:22 AM, Jaicel R. Sabonsolin wrote: > >>Hi Guys, > >> > >>I need help with my ovirt Hosted-Engine HA setup. I am running on 2 > >>ovirt hosts and 2 gluster nodes with replicated volumes. i already have > >>VMs running on my hosts and they can migrate normally once i for example > >>power off the host that they are running on. the problem is that the > >>engine can't migrate once i switch off the host that hosts the engine. > >> > >> oVirt 3.4.3-1.el6 > >> KVM 0.12.1.2 - 2.415.el6_5.10 > >> LIBVIRT libvirt-0.10.2-29.el6_5.9 > >> VDSM vdsm-4.14.17-0.el6 > >> > >> > >>right now, i have this result from hosted-engine --vm-status. > >> > >> File "/usr/lib64/python2.6/runpy.py", line 122, in > >> _run_module_as_main > >> "__main__", fname, loader, pkg_name) > >> File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code > >> exec code in run_globals > >> File > >> > >>"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status.p= y", > >> > >> line 111, in > >> if not status_checker.print_status(): > >> File > >> > >>"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_setup/vm_status.p= y", > >> > >> line 58, in print_status > >> all_host_stats =3D ha_cli.get_all_host_stats() > >> File > >> > >>"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.= py", > >> > >> line 137, in get_all_host_stats > >> return self.get_all_stats(self.StatModes.HOST) > >> File > >> > >>"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.= py", > >> > >> line 86, in get_all_stats > >> constants.SERVICE_TYPE) > >> File > >> > >>"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink= .py", > >> > >> line 171, in get_stats_from_storage > >> result =3D self._checked_communicate(request) > >> File > >> > >>"/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink= .py", > >> > >> line 199, in _checked_communicate > >> .format(message or response)) > >> ovirt_hosted_engine_ha.lib.exceptions.RequestError: Request failed: > >> > >> > >> > >>restarting ha-broker and ha-agent normalizes the status but eventually > >>it would become "false" and then return to the result above. hope you > >>guys could help me with this. > >> > > > >Hi Jaicel, > >please attach agent.log and broker.log from the host where you trying to > >run hosted-engine --vm-status. I have a feeling that you ran into a > >known problem on gluster - stalled file descriptor, in that case the > >only known solution at this time is to restart the broker & agent as you > >have already found out. > > > = > Adding Niels and gluster-devel to troubleshoot from Gluster NFS perspecti= ve. I'd welcome any details on this "stalled file descriptor" problem. Is there a bug filed with some details like logs, sysrq-t and maybe even tcpdumps? If there is an easy way to reproduce this behaviour, I can surely look into it and hopefully come up with some advise or fix. Thanks, Niels --===============2595986334058809138==--