On Thu, Oct 15, 2020 at 12:25 PM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:

On Thu, Oct 15, 2020 at 10:41 AM Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:

Any feedback on my latest comments?

In the meantime here:

https://drive.google.com/file/d/1iN37znRtCo2vgyGTH_ymLhBJfs-2pWDr/view?usp=sharing

you can find inside the sosreport in tar.gz format, where I have modified some file names and context in respect of hostnames.

The only file I have not put inside is the dump of the database, but I can run any query you like in case.

Gianluca

I have also tried to put debug into the engine.

 

So after huge debugging work with Dana Elfassy and Martin Necas (thank you very much to both!) and coordination of Sandro we found the culprit!

Inside firewall custom rules of my engine I had this (note the double quotes for the comment about Nagios):

[root@ovmgr1 ovirt-engine]# engine-config -g IPTablesConfigSiteCustom
IPTablesConfigSiteCustom: -A INPUT -p tcp --dport 5666 -s 10.4.5.99/32 -m comment --comment "Nagios NRPE daemon" -j ACCEPT version: general
[root@ovmgr1 ovirt-engine]#

So those double quotes  caused a wrong formatted json block that ansible-runner-service was not able to manage in the http post phase

After changing with single quotes, with this command:

engine-config -s IPTablesConfigSiteCustom="-A INPUT -p tcp --dport 5666 -s 10.4.5.99/32 -m comment --comment 'Nagios NRPE daemon' -j ACCEPT"

and restarting the engine so that now I have

[root@ovmgr1 ovirt-engine]# engine-config -g IPTablesConfigSiteCustom
IPTablesConfigSiteCustom: -A INPUT -p tcp --dport 5666 -s 10.4.5.99/32 -m comment --comment 'Nagios NRPE daemon' -j ACCEPT version: general
[root@ovmgr1 ovirt-engine]#

I was able to add the CentOS 8.2 host.

So mind if you have the double quotes in any engine-config key before upgrading from 4.3 to 4.4.

What a nasty thing to detect...

Thanks again guys for your help

Gianluca