- Users - oVirt List Archives

Re: engine-setup fails: "Failed to execute stage 'Misc configuration': Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute"
by Brent Saner 11 Jul '24

11 Jul '24

Whoops, sent before I finished replying- I'll look into first trying the Nightly, to see if that has the same behavior, and then trying the _setupAuth patching et. al. to see if I can narrow down what might be happening here. Brent Saner SENIOR SYSTEMS ENGINEER Follow us on LinkedIn! brent.saner(a)netfire.com 855-696-3834 Ext. 110 www.netfire.com ________________________________ From: Brent Saner <brent.saner(a)netfire.com> Sent: Thursday, July 11, 2024 5:38 PM To: users(a)ovirt.org <users(a)ovirt.org> Subject: Re: [ovirt-users] Re: engine-setup fails: "Failed to execute stage 'Misc configuration': Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute" Looking into this, I can confirm that *after* the packages are installed and *before* engine-setup is run, /etc/ovirt-engine/aaa/ does not even exist: # ls /etc/ovirt-engine/aaa ls: cannot access '/etc/ovirt-engine/aaa': No such file or directory Which may be expected, I'm unclear on that. Further details inline below: ________________________________ From: Yedidyah Bar David <didi(a)redhat.com> Sent: Thursday, July 4, 2024 2:13 AM To: Brent Saner <brent.saner(a)netfire.com> Cc: users(a)ovirt.org <users(a)ovirt.org> Subject: Re: [ovirt-users] Re: engine-setup fails: "Failed to execute stage 'Misc configuration': Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute" You don't often get email from didi(a)redhat.com. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Caution: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance. On Thu, Jul 4, 2024 at 9:11 AM Yedidyah Bar David <didi(a)redhat.com<mailto:didi@redhat.com>> wrote: On Wed, Jun 19, 2024 at 10:38 PM Brent S. <brent.saner(a)netfire.com<mailto:brent.saner@netfire.com>> wrote: As a quick update to this: # ovirt-aaa-jdbc-tool Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false Jun 19, 2024 7:28:14 PM org.ovirt.engine.extension.aaa.jdbc.binding.cli.Cli main SEVERE: Unexpected Exception invoking Cli: Could not read properties from: /etc/ovirt-engine/aaa/internal.properties Which is, of course, the same message in the log. This is probably expected, since *engine-setup never actually created the file*: Are you sure about this? Presumably it was removed, but I do know the directory is not present before engine-setup is run and the file is not present after. # ls -la /etc/ovirt-engine/aaa total 4 drwxr-xr-x. 2 root root 6 Jun 19 19:27 . drwxr-xr-x. 18 root root 4096 Jun 19 19:27 .. # I guess you checked the above only after engine-setup failed/finished, right? At the time, yes; see above inlines; the directory itself does not even exist before engine-setup. And: 2024-06-19 19:27:10,917+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:923 execute-output: ['/usr/share/ovirt-engine-extension-aaa-jdbc/dbscripts/schema.sh', '-s', '[REDACTED_REMOTE_DB_HOST]', '-p', '5432', '-u', '[REDACTED_REMOTE_DB_USER]', '-d', '[REDACTED_REMOTE_DB_NAME]', '-e', 'aaa_jdbc', '-l', '/root/ovirt-engine-setup.log', '-c', 'apply'] stderr: 2024-06-19 19:27:10,917+0000 DEBUG otopi.transaction transaction._prepare:61 preparing 'File transaction for '/etc/ovirt-engine/aaa/internal.properties'' 2024-06-19 19:27:10,917+0000 DEBUG otopi.filetransaction filetransaction.prepare:184 file '/etc/ovirt-engine/aaa/internal.properties' missing Indeed (SNIP) If you grep ovirt-engine sources, you'll find internal.properties in: packaging/setup/ovirt_engine_setup/engine/constants.py: AAA_JDBC_CONFIG_DB = os.path.join( OVIRT_ENGINE_SYSCONFDIR, 'aaa', 'internal.properties' ) If you then grep for AAA_JDBC_CONFIG_DB, you see it in: packaging/setup/plugins/ovirt-engine-setup/ovirt-engine/config/aaajdbc.py: def _setupAuth(self): self.environment[otopicons.CoreEnv.MAIN_TRANSACTION].append( filetransaction.FileTransaction( name=oenginecons.FileLocations.AAA_JDBC_CONFIG_DB, ... visibleButUnsafe=True, Forgot to mention: You can check otopi sources:src/otopi/filetransaction.py to see what this means. ... def _setupAdminUser(self): toolArgs = ( oenginecons.FileLocations.AAA_JDBC_TOOL, '--db-config=%s' % oenginecons.FileLocations.AAA_JDBC_CONFIG_DB, ) ... @plugin.event( stage=plugin.Stages.STAGE_MISC, name=AAA_JDBC_SETUP_ADMIN_USER, after=( oengcommcons.Stages.DB_SCHEMA, oengcommcons.Stages.DB_CONNECTION_AVAILABLE, oenginecons.Stages.CONFIG_EXTENSIONS_UPGRADE, ), before=( oenginecons.Stages.CONFIG_AAA_ADMIN_USER_SETUP, ), condition=lambda self: self.environment[ oenginecons.ConfigEnv.ADMIN_USER_AUTHZ_TYPE ] == self.AAA_JDBC_AUTHZ_TYPE, ) def _misc(self): # TODO: if we knew that aaa-jdbc package was upgraded by engine-setup # TODO: we could display summary note that custom profiles have to be # TODO: upgraded manually self._setupSchema() self._setupAuth() self._setupAdminUser() ... This means that: At STAGE_MISC, _misc calls _setupAuth, which creates this file, and then it calls _setupAdminUser which tries to use it. Latter fails, and engine-setup rolls back the MAIN_TRANSACTION, including removing the file. I'd start debugging this issue by: 1. Patching _setupAuth to wait (e.g. using dialog.queryBoolean, search the source for examples) after it creates the file, so that I can investigate it 2. Patching _setupAdminUser to wait after it runs the tool, so that I can try to investigate the failure - e.g. run it myself under strace, if the existing logging is not enough. You can try using the otopi plugin wait_on_error for this, instead of patching. Good luck and best regards, -- Didi -- Didi

1 0

Recall: Re: Re: engine-setup fails: "Failed to execute stage 'Misc configuration': Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute"
by Brent Saner 11 Jul '24

11 Jul '24

brent.saner(a)netfire.com would like to recall the message, "Re: [ovirt-users] Re: engine-setup fails: "Failed to execute stage 'Misc configuration': Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute"". Brent Saner SENIOR SYSTEMS ENGINEER Follow us on LinkedIn! brent.saner(a)netfire.com 855-696-3834 Ext. 110 www.netfire.com

1 0

engine-setup fails: "Failed to execute stage 'Misc configuration': Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute"
by brent.saner＠netfire.com 11 Jul '24

11 Jul '24

I can find no reason why this would be happening. AlmaLinux 9.4, oVirt 4.5.6, brand new/fresh cluster/install. From the log, these seem to be the only relevant lines: 2024-06-18 22:02:52,720+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:918 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'query', '--what= user', '--pattern=name=admin') stdout: 2024-06-18 22:02:52,720+0000 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:923 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'query', '--what= user', '--pattern=name=admin') stderr: Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false Jun 18, 2024 10:02:52 PM org.ovirt.engine.extension.aaa.jdbc.binding.cli.Cli main SEVERE: Unexpected Exception invoking Cli: com/ongres/scram/common/stringprep/StringPreparation 2024-06-18 22:02:52,720+0000 DEBUG otopi.context context._executeMethod:143 method exception Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/config/aaajdbc.py", line 414, in _misc self._setupAdminUser() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/config/aaajdbc.py", line 298, in _setupAdminUser if not self._userExists( File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/config/aaajdbc.py", line 49, in _userExists rc, stdout, stderr = self.execute( File "/usr/lib/python3.9/site-packages/otopi/plugin.py", line 929, in execute raise RuntimeError( RuntimeError: Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute 2024-06-18 22:02:52,721+0000 ERROR otopi.context context._executeMethod:151 Failed to execute stage 'Misc configuration': Command '/usr/bin/ovirt-aaa-jdbc-tool' failed to execute 2024-06-18 22:02:52,721+0000 DEBUG otopi.transaction transaction.abort:124 aborting 'DNF Transaction' Any ideas?

5 7

Get Host Capabilities failed: Connection timeout for host
by Frederico Madeira 11 Jul '24

11 Jul '24

Hi guys I'm getting these messages from engine-gui interfaces.: Get Host Capabilities failed: Connection timeout for host My hosts are in irresponsible state My VMs are in unknown state My Storage are in inactive state From the engine, I did successfully : ping ip and name and ssh connect to host my hosts, so I didn´t have network connectivity problems. error in engine.log 2024-07-10 03:47:30,088-03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM servernode10 command Get Host Capabilities failed: Connection timeout for host '10.180.18.116', last response arrived 6609 ms ago. 2024-07-10 03:47:30,089-03 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Connection timeout for host '10.180.18.116', last response arrived 6609 ms ago. 2024-07-10 03:49:45,027-03 INFO Any hint how to troubleshoot this issue? Regards,*o Madeira * fred(a)madeira.eng.br www.madeira.eng.br Cisco CCNA, LPIC-1, LPIC-2 Registered GNU/Linux nº 206120 GPG-Key-ID: 1024D/0F0A721D Key fingerprint = C424 D86B 57D5 BE55 767A 6ED1 53F8 254E 0F0A 721D MSN: fttmadeira(a)hotmail.com GTalk:fmadeira@gmail.com SKYPE: fred_madeira

1 0

VM live migration fails after update to qemu-kvm-9.0.0-4.el9.x86_64
by g.vasilopoulos＠uoc.gr 11 Jul '24

11 Jul '24

Hello After updating a hypervisor with qemu-kvm-9.0.0-4.el9.x86_64 live migration stopped working with errors related to the cpu topology. When I downgraded to qemu-kvm-9.0.0-3.el9.x86_64 migration worked again. Is this a known bug? Has anyone else encountered this?

3 3

vm stays locked after failed export
by h aanst 10 Jul '24

10 Jul '24

Hi, Running ovirt for many years, but this baffles me. An export was taking too long, seems to hang. Due to the heat a switch failedt, so export storage is umounted, at this time. Now my VM has a little lock, and can't be restarted. (shutdown worked) Checked all DB entries and vdsm. update vm_static set template_status = 0 where template_status = 1; select * from vm_static where template_status = 1; 0 update vm_dynamic set status = 0 where status = 15; select * from vm_dynamic where status = 15; 0 update images set imagestatus = 1 where imagestatus = 2; select * from images where imagestatus = 2; 0 update snapshots set status = 'OK' where status like 'LOCKED'; select * from snapshots where status like 'LOCKED'; 0 UPDATE images SET imagestatus = 1 WHERE imagestatus = 4; select * from images where imagestatus = 4; 0 restarted the hosted-engine I dont know what to check next. Any hints? thanks

1 0

CentOS9 Stream -- Only way forward?
by eshwayri＠gmail.com 10 Jul '24

10 Jul '24

I've been running 4.5.4-1 on RHEL8 for a while now; been running fine for my needs. I was getting around the unholy dependency issue caused by the centos-release package conflicting with the redhat-release package by upgrading everything else -- not it. It's been necessary to do this since RHEL versions were higher than any CentOS 8 Stream versions. This time round since CentOS has been archived, I made sure to install anything I hadn't and then manually purged centos-release from the rpm database. This allowed RHEL to update to 8.10, including the release package this time. Not expecting an issue since there won't be any future patches coming from that repo. If I decide to stick with oVirt and install daily snapshots, I assume what I will have to do is setup a new CentOS Steam 9 server, create a backup of the old server, and then restore on the new server. Like what I did when I was forced to migrate from CentOS 8 to RHEL8 (with CentOS 8 Stream packages). Is this right? I haven't decided if I want to do this. CentOS Stream 9 is supported until mid 2027; RHEL 8.10 is supported until mid 2029. Barring any new features or major incompatibility, I really don't see a pressing reason to do this. In fact, with no one stepping in to take over the project, there is a better chance of being supported by sticking with RHEL 8.10 (2+ years).

5 5

VM Migration Failed
by KSNull Zero 10 Jul '24

10 Jul '24

Running oVirt 4.4.5 VM cannot migrate between hosts. vdsm.log contains the following error: libvirt.libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+tls://ovhost01.local/system: authentication failed: Failed to verify peer's certificate Certificates on hosts was renewed some time ago. How this issue can be fixed ? Thank you.

4 6

Re: Multiple GPU Passthrough with NVLink (Invalid I/O region)
by Zhengyi Lai 10 Jul '24

10 Jul '24

I noticed this document https://docs.nvidia.com/vgpu/16.0/grid-vgpu-release-notes-generic-linux-kvm… has this to say In pass through mode, all GPUs connected to each other through NVLink must be assigned to the same VM. If a subset of GPUs connected to each other through NVLink is passed through to a VM, unrecoverable error XID 74 occurs when the VM is booted. If a subset of GPUs connected to each other through NVLink is passed through to a VM, unrecoverable error XID 74 occurs when the VM is booted. This error corrupts the NVLink state on the physical GPUs and, as a result, the NVLink bridge between the NVLink and the physical GPUs is not recognized. result, the NVLink bridge between the GPUs is unusable. You may need to passthrough all GPUs in the nvlink to the VM

2 1

problem on dnf update
by g.vasilopoulos＠uoc.gr 09 Jul '24

09 Jul '24

Hi I tried dnf update today and got the error bellow. Is there a solution for this?: [root@*****~]# dnf update Last metadata expiration check: 3:42:36 ago on Mon 17 Jun 2024 08:47:59 AM EEST. Error: Problem 1: package python3-boto3-1.18.58-1.el9s.noarch from @System requires (python3.9dist(botocore) < 1.22 with python3.9dist(botocore) >= 1.21.58), but none of the providers can be installed - cannot install both python3-botocore-1.31.62-1.el9.noarch from appstream and python3-botocore-1.21.58-1.el9s.noarch from @System - cannot install both python3-botocore-1.21.58-1.el9s.noarch from ovirt-master-centos-stream-openstack-yoga-testing and python3-botocore-1.31.62-1.el9.noarch from appstream - cannot install the best update candidate for package python3-botocore-1.21.58-1.el9s.noarch - cannot install the best update candidate for package python3-boto3-1.18.58-1.el9s.noarch Problem 2: package python3-pyngus-2.3.0-8.el9s.noarch from @System requires python3.9dist(python-qpid-proton), but none of the providers can be installed - cannot install both python3-qpid-proton-0.39.0-2.el9s.x86_64 from ovirt-master-centos-stream-opstools-collectd5-testing and python3-qpid-proton-0.35.0-2.el9s.x86_64 from @System - cannot install both python3-qpid-proton-0.35.0-2.el9s.x86_64 from ovirt-master-centos-stream-openstack-yoga-testing and python3-qpid-proton-0.39.0-2.el9s.x86_64 from ovirt-master-centos-stream-opstools-collectd5-testing - cannot install the best update candidate for package python3-qpid-proton-0.35.0-2.el9s.x86_64 - cannot install the best update candidate for package python3-pyngus-2.3.0-8.el9s.noarch Problem 3: package python3-oslo-messaging-12.13.3-1.el9s.noarch from @System requires python3-pyngus, but none of the providers can be installed - package python3-pyngus-2.3.0-8.el9s.noarch from @System requires python3.9dist(python-qpid-proton), but none of the providers can be installed - package python3-pyngus-2.3.0-8.el9s.noarch from ovirt-master-centos-stream-openstack-yoga-testing requires python3.9dist(python-qpid-proton), but none of the providers can be installed - package python3-qpid-proton-0.35.0-2.el9s.x86_64 from @System requires qpid-proton-c(x86-64) = 0.35.0-2.el9s, but none of the providers can be installed - package python3-qpid-proton-0.35.0-2.el9s.x86_64 from ovirt-master-centos-stream-openstack-yoga-testing requires qpid-proton-c(x86-64) = 0.35.0-2.el9s, but none of the providers can be installed - cannot install both qpid-proton-c-0.39.0-2.el9s.x86_64 from ovirt-master-centos-stream-opstools-collectd5-testing and qpid-proton-c-0.35.0-2.el9s.x86_64 from @System - cannot install both qpid-proton-c-0.35.0-2.el9s.x86_64 from ovirt-master-centos-stream-openstack-yoga-testing and qpid-proton-c-0.39.0-2.el9s.x86_64 from ovirt-master-centos-stream-opstools-collectd5-testing - cannot install the best update candidate for package qpid-proton-c-0.35.0-2.el9s.x86_64 - cannot install the best update candidate for package python3-oslo-messaging-12.13.3-1.el9s.noarch (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

2 2