5:04 a.m.
Hi All,
OST currently fails all the time during engine setup.
Here's a piece of ansible log that's seen repeatedly and I think
describes the problem:
11:07:54 E "engine-config",
11:07:54 E "-s",
11:07:54 E "OvfUpdateIntervalInMinutes=10"
11:07:54 E ],
11:07:54 E "delta": "0:00:01.142926",
11:07:54 E "end": "2021-12-24 11:06:37.894810",
11:07:54 E "invocation": {
11:07:54 E "module_args": {
11:07:54 E "_raw_params": "engine-config -s
OvfUpdateIntervalInMinutes='10' ",
11:07:54 E "_uses_shell": false,
11:07:54 E "argv": null,
11:07:54 E "chdir": null,
11:07:54 E "creates": null,
11:07:54 E "executable": null,
11:07:54 E "removes": null,
11:07:54 E "stdin": null,
11:07:54 E "stdin_add_newline": true,
11:07:54 E "strip_empty_ends": true,
11:07:54 E "warn": false
11:07:54 E }
11:07:54 E },
11:07:54 E "item": {
11:07:54 E "key": "OvfUpdateIntervalInMinutes",
11:07:54 E "value": "10"
11:07:54 E },
11:07:54 E "msg": "non-zero return code",
11:07:54 E "rc": 1,
11:07:54 E "start": "2021-12-24 11:06:36.751884",
11:07:54 E "stderr": "Picked up JAVA_TOOL_OPTIONS:
-Dcom.redhat.fips=false",
11:07:54 E "stderr_lines": [
11:07:54 E "Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false"
11:07:54 E ],
11:07:54 E "stdout": "Error loading module from
/usr/share/ovirt-engine/modules/common/org/springframework/main/module.xml",
11:07:54 E "stdout_lines": [
11:07:54 E "Error loading module from
/usr/share/ovirt-engine/modules/common/org/springframework/main/module.xml"
We do set some config values for the engine in OST when running
engine-setup. I tried commenting these out, but then engine failed
health check anyway:
"Status code was 503 and not [200]: HTTP Error 503: Service Unavailable"
The last working set of OST images was the one from Dec 23, 2021 2:05:08
AM. The first broken one is from Dec 24, 2021 2:05:09 AM. The shipped
ovirt-engine RPMs doesn't seem to contain any important changes for
these two sets, but AFAICS the newer ovirt-dependencies RPM did take in
a couple of patches that look suspicious [1][2][3]. The patches were
merged on November 16th, but it seems they were first used in that
broken set from Dec 24 (the one from Dec 23 seems to contain
ovirt-dependencies RPM based on this [4] commit).
I wanted to try out an older version of ovirt-dependencies, but I think
they were wiped out from resources.ovirt.org.
I will disable cyclic el8stream OST runs for now, cause all of them
fail. If there's anyone working and able to make a build with those
patches reverted and test that out, please ping me and I'll re-enable them.
Regards, Marcin
[1] https://gerrit.ovirt.org/c/ovirt-dependencies/+/114699
[2] https://gerrit.ovirt.org/c/ovirt-dependencies/+/113877
[3] https://gerrit.ovirt.org/c/ovirt-dependencies/+/114654
[4] https://gerrit.ovirt.org/c/ovirt-dependencies/+/117459