[ovirt-users] Re: ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(score) Penalizing score by 1600 due to network status
On Mon, Jul 19, 2021 at 8:39 AM Christoph Timm <ovirt(a)timmi.org> wrote:
Hi List,
I'm trying to understand why my hosted engine is moved from one node to
another from time to time.
It is happening sometime multiple times a day. But there are also days
without it.
I can see the following in the ovirt-hosted-engine-ha/agent.log:
ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(score)
Penalizing score by 1600 due to network status
After that the engine will be shutdown and started on another host.
The oVirt Admin portal is showing the following around the same time:
Invalid status on Data Center Default. Setting status to Non Responsive.
But the whole cluster is working normally during that time.
I believe that I have somehow a network issue on my side but I have no
clue what kind of check is causing the network status to penalized.
Does anyone have an idea how to investigate this further?
Please check also broker.log. Do you see 'dig' failures?
This happened several times already on our CI infrastructure, but yours is
the first report from an actual real user. See also:
https://lists.ovirt.org/archives/list/infra@ovirt.org/thread/LIGS5WXGEKWA...
I didn't open a bug for this (yet?), also because I never reproduced on my
own machines and am not sure about the exact failing flow. If this is
reproducible
reliably for you, you might want to test the patch I pushed:
https://gerrit.ovirt.org/c/ovirt-hosted-engine-ha/+/115596
Other ideas/opinions about how to enhance this part of the monitoring
are most welcome.
If this phenomenon is new for you, and you can reliably say it's not due to
a recent "natural" higher network load, I wonder if it's due to some weird
bug/change somewhere.
Thanks and best regards,
--
Didi