VDSM Logs -

ual': True}} from=internal, task_id=b017c5d2-a941-4ccd-af02-3fac0a767df5 (api:54)
2022-02-03 18:53:53,368+0530 ERROR (periodic/3) [ovirt_hosted_engine_ha.client.client.HAClient] Malformed metadata for host 3: received 0 of 512 expected bytes (client:137)
2022-02-03 18:53:54,529+0530 INFO (jsonrpc/5) [api.host] START getStats() from=::1,38990 (api:48)
2022-02-03 18:53:54,543+0530 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=()) from=::1,38990, task_id=3864f22e-98cc-426f-a4d5-2e2254f34aae (api:48)
2022-02-03 18:53:54,543+0530 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={'8ab8b572-39a3-4603-b0e7-2b6fe71c4d73': {'code': 0, 'lastCheck': '6.3', 'delay': '0.000359743', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'f3114b74-8b12-4493-9faf-0f514afa3a16': {'code': 0, 'lastCheck': '2.1', 'delay': '0.000447464', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '26bfbe5d-e427-47a9-9111-cca8a628fea8': {'code': 0, 'lastCheck': '2.0', 'delay': '0.00034851', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '23540fa8-4de2-4715-97e1-4ce9fe4f66cc': {'code': 0, 'lastCheck': '2.1', 'delay': '0.000352501', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '8fa0a28e-480f-444c-9cf2-bef536d3c52f': {'code': 0, 'lastCheck': '2.0', 'delay': '0.000320087', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, 'd1dba7f5-cb30-4571-8fe6-5fb5005bcdb2': {'code': 0, 'lastCheck': '2.6', 'delay': '0.000452836', 'valid': True, 'version': 0, 'acquired': True, 'actual': True}, '88b94c9a-6855-4199-9d37-1f88d21f211c': {'code': 0, 'lastCheck': '2.0', 'delay': '0.000299592', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '2e0ec671-5972-4276-aa49-aefc306f5da4': {'code': 0, 'lastCheck': '2.0', 'delay': '0.000316513', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,38990, task_id=3864f22e-98cc-426f-a4d5-2e2254f34aae (api:54)
2022-02-03 18:53:54,543+0530 INFO (jsonrpc/5) [vdsm.api] START multipath_health() from=::1,38990, task_id=7ecaad0c-aa5d-45c0-96a5-dffc78572248 (api:48)
2022-02-03 18:53:54,543+0530 INFO (jsonrpc/5) [vdsm.api] FINISH multipath_health return={} from=::1,38990, task_id=7ecaad0c-aa5d-45c0-96a5-dffc78572248 (api:54)
2022-02-03 18:53:54,548+0530 ERROR (jsonrpc/5) [ovirt_hosted_engine_ha.client.client.HAClient] Malformed metadata for host 3: received 0 of 512 expected bytes (client:137)
2022-02-03 18:53:54,549+0530 INFO (jsonrpc/5) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::1,38990 (api:54)
2022-02-03 18:53:56,038+0530 INFO (jsonrpc/4) [vdsm.api] START repoStats(domains=['8ab8b572-39a3-4603-b0e7-2b6fe71c4d73']) from=::1,38990, task_id=8855f314-9334-4b06-88e3-7a1a3f577049 (api:48)
2022-02-03 18:53:56,038+0530 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats return={'8ab8b572-39a3-4603-b0e7-2b6fe71c4d73': {'code': 0, 'lastCheck': '1.3', 'delay': '0.000349788', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,38990, task_id=8855f314-9334-4b06-88e3-7a1a3f577049 (api:54)

On Thu, Feb 3, 2022 at 6:42 PM Martin Sivak <msivak@redhat.com> wrote:

On Thu, Feb 3, 2022 at 1:50 PM Ayansh Rocks <shashank123rastogi@gmail.com> wrote:
Do I need to clean it from every host which has the hosted engine ? Sinc the other host still can see it.

No, the metadata "whiteboard" is shared. Just one copy that you removed. Once node 3 start reporting again, it will simply put new content in there.

Not sure about vdsm, maybe some component (vdsm, engine) is still trying to get status of node 3. I have been away for too long, so Didi will have to pitch in.

Martin Sivak

On Thu, Feb 3, 2022 at 6:02 PM Ayansh Rocks <shashank123rastogi@gmail.com> wrote:
Hi Yedidyah,

Thanks, After cleaning it from one host i am getting below error messages on /var/log/messages

Feb 3 18:01:02 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:03 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:13 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:17 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:18 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes
Feb 3 18:01:23 iondelsvr14 vdsm[11893]: ERROR Malformed metadata for host 3: received 0 of 512 expected bytes

On Thu, Feb 3, 2022 at 1:04 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Thu, Feb 3, 2022 at 9:11 AM Ayansh Rocks
<shashank123rastogi@gmail.com> wrote:
>
> Hi Yedidyah,
>
> I am thinking to do below -
>
> # hosted-engine --set-maintenance --mode=global
> # hosted-engine --clean-metadata --host-id=<old_host_id> --force-clean"
>
> I hope it is safe and tested ?

It is tested, and it's "safe" as in "Does what it's designed to do".

It's definitely not "safe" as in "riskless" - if you make a mistake,
you risk damage. I didn't use it myself recently, and do not remember
details anymore. It's definitely not something that you are supposed
to use/need casually.

It should be safe in global maintenance, the only risk is when you remove the report of a node that currently runs the engine. As other nodes might decide that the engine needs restarting.

If unsure, and if your environment is important to you (not just some
test/dev/learn thing), I warmly suggest to try this first on a test
env (can be done with nested-kvm VMs, no need for physical machines
for this).

That said, others on this list (and elsewhere) did use it successfully,
and I recommend again to search for examples/reports/experiences.

Good luck and best regards,

>
> Thanks
>
> On Thu, Feb 3, 2022 at 12:17 PM Yedidyah Bar David <didi@redhat.com> wrote:
>>
>> On Thu, Feb 3, 2022 at 5:53 AM Ayansh Rocks
>> <shashank123rastogi@gmail.com> wrote:
>> >
>> > Hi All,
>> >
>> > How to clean stale hosted engine from the database. I want to remove host id=3
>>
>> Perhaps by using the options '--clean_metadata' and '--host-id'
>> of the command 'hosted-engine'.
>> You can check documentation, list archives, etc. for details.
>>
>> Good luck and best regards,
>> --
>> Didi
>>

--
Didi

_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IR3SMGNMRRGEKXUXTMCZBINAX6XZDRM5/