[ovirt-devel] Re: Health check endpoint in the engine hanging forever on CheckDBConnection

On 11/27/20 11:24 AM, Martin Perina wrote: > Hi, > > the health status is pretty stupid simple call to database: > > https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules... > < https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules... > > https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules... > < https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules... > > https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules... > < https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules... > > https://github.com/oVirt/ovirt-engine/blob/master/packaging/dbscripts/com... > < https://github.com/oVirt/ovirt-engine/blob/master/packaging/dbscripts/com... > > > So it should definitely not hang forever unless there is some serious > issue in the engine start up or PostgreSQL database. Could you please > share logs? Especially interesting would server.log and engine.log > from /var/log/ovirt-engine Well, after some discussion with Artur and trying some workarounds, the problem magically disappeared on my servers, but there's one OST gating run in CI that suffered from the same problem: https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_g... https://jenkins.ovirt.org/job/ovirt-system-tests_gate/937/artifact/basic-... https://jenkins.ovirt.org/job/ovirt-system-tests_gate/937/artifact/basic-... > > Martin > > > On Thu, Nov 26, 2020 at 4:24 PM Marcin Sobczyk <msobczyk(a)redhat.com > <mailto:msobczyk@redhat.com>> wrote: > > Hi, > > I'm trying out (I think) the latest build of ovirt-engine in OST > [1] and > the basic suite > fails when we do engine reconfiguration and then restart the > service [2]. > After restarting we wait on the health check endpoint status here [3]. > This however ends with a timeout. > I tried running manually: > > curl -D - http://engine/ovirt-engine/services/health > <http://engine/ovirt-engine/services/health> > > and that command also hangs forever. > In the engine log the last related entry seem to be: > > 2020-11-26 16:20:15,761+01 DEBUG > [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default > task-7) [] > Incoming locale 'en-US'. Filter determined locale to be 'en-US' > 2020-11-26 16:20:15,761+01 DEBUG > [org.ovirt.engine.core.services.HealthStatus] (default task-7) [] > Health > Status servlet: entry > 2020-11-26 16:20:15,761+01 DEBUG > [org.ovirt.engine.core.services.HealthStatus] (default task-7) [] > Calling CheckDBConnection query > > Has anyone else also encountered that? > > Regards, Marcin > > [1] > ovirt-engine-4.4.4.3-0.0.master.20201126133903.gitc2c805a2662.el8.noarch > [2] > https://github.com/oVirt/ovirt-system-tests/blob/3e2fc267b376a12eda131fa0... > < https://github.com/oVirt/ovirt-system-tests/blob/3e2fc267b376a12eda131fa0... > > [3] > https://github.com/oVirt/ovirt-system-tests/blob/3e2fc267b376a12eda131fa0... > < https://github.com/oVirt/ovirt-system-tests/blob/3e2fc267b376a12eda131fa0... > > > > > -- > Martin Perina > Manager, Software Engineering > Red Hat Czech s.r.o.