3:33 p.m.
- having a 4,3 ovirt-engine up and running on centos 7
- Create a new engine machine with centos 8 stream, latests updates
- installing packages, restoring ovirt-engine backup from previous host and running
ovirt-setup,
- When started, no access to the admin console at https://<fqdn>/ovirt-engine. The
web proxy will return: 500 Internal error""
- I can access the Grafana login
- ovirt-engine.service is started without any systemd errors
Any Ideas how to proceed?
Log files:
/var/log/ovirt-engine/boot.log # No Errors
/var/log/ovirt-engine/engine.log # No Errors
/var/log/ovirt-engine/server.log # Errors found
If I just grep for "ERROR" in server.log
2022-04-26 13:52:42,220+02 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool
-- 51) MSC000001: Failed to start service
jboss.deployment.subunit."engine.ear"."bll.jar".component.Backend.START:
org.jboss.msc.service.StartException in service
jboss.deployment.subunit."engine.ear"."bll.jar".component.Backend.START:
java.lang.IllegalStateException: WFLYEE0042: Failed to construct component instance
2022-04-26 13:52:42,231+02 ERROR [org.jboss.as.controller.management-operation]
(Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address:
([("deployment" => "engine.ear")]) - failure description:
{"WFLYCTL0080: Failed services" =>
{"jboss.deployment.subunit.\"engine.ear\".\"bll.jar\".component.Backend.START"
=> "java.lang.IllegalStateException: WFLYEE0042: Failed to construct component
instance
2022-04-26 13:52:42,568+02 ERROR [org.jboss.as] (Controller Boot Thread) WFLYSRV0026:
WildFly Full 23.0.2.Final (WildFly Core 15.0.1.Final) started (with errors) in 24096ms -
Started 1668 of 1888 services (6 services failed or missing dependencies, 393 services are
lazy, passive or on-demand)
The complete starting sequence from server.log
2022-04-26 13:52:23,322+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4)
WFLYSRV0027: Starting deployment of "restapi.war" (runtime-name:
"restapi.war")
2022-04-26 13:52:24,074+02 INFO [org.jboss.as.remoting] (MSC service thread 1-2)
WFLYRMT0001: Listening on 127.0.0.1:8707
2022-04-26 13:52:24,247+02 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
service thread 1-2) WFLYJCA0098: Bound non-transactional data source:
java:/ENGINEDataSourceNoJTA
2022-04-26 13:52:24,901+02 INFO [org.jboss.as.ejb3] (MSC service thread 1-4) WFLYEJB0493:
Jakarta Enterprise Beans subsystem suspension complete
2022-04-26 13:52:25,194+02 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
service thread 1-4) WFLYJCA0001: Bound data source [java:/ENGINEDataSource]
2022-04-26 13:52:25,195+02 INFO [org.jboss.as.connector.subsystems.datasources] (MSC
service thread 1-4) WFLYJCA0001: Bound data source [java:/DWHDataSource]
2022-04-26 13:52:25,530+02 INFO [org.wildfly.security] (MSC service thread 1-3) ELY00001:
WildFly Elytron version 1.15.3.Final
2022-04-26 13:52:25,741+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-2)
WFLYWELD0003: Processing weld deployment restapi.war
2022-04-26 13:52:25,999+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1)
WFLYSRV0207: Starting subdeployment (runtime-name: "services.war")
2022-04-26 13:52:26,000+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1)
WFLYSRV0207: Starting subdeployment (runtime-name: "root.war")
2022-04-26 13:52:26,000+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1)
WFLYSRV0207: Starting subdeployment (runtime-name: "docs.war")
2022-04-26 13:52:26,000+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1)
WFLYSRV0207: Starting subdeployment (runtime-name: "welcome.war")
2022-04-26 13:52:26,000+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1)
WFLYSRV0207: Starting subdeployment (runtime-name: "bll.jar")
2022-04-26 13:52:26,001+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1)
WFLYSRV0207: Starting subdeployment (runtime-name: "enginesso.war")
2022-04-26 13:52:26,001+02 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1)
WFLYSRV0207: Starting subdeployment (runtime-name: "webadmin.war")
2022-04-26 13:52:26,368+02 INFO [org.hibernate.validator.internal.util.Version] (MSC
service thread 1-2) HV000001: Hibernate Validator 6.0.22.Final
2022-04-26 13:52:26,752+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 40) WFLYUT0021: Registered web context: '/ovirt-engine/apidoc' for server
'default-server'
2022-04-26 13:52:26,751+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 41) WFLYUT0021: Registered web context: '/ovirt-engine/web-ui' for server
'default-server'
2022-04-26 13:52:26,983+02 INFO [org.jboss.weld.Version] (MSC service thread 1-2)
WELD-000900: 3.1.6 (Final)
2022-04-26 13:52:28,596+02 WARN [org.jboss.as.dependency.deprecated] (MSC service thread
1-1) WFLYSRV0221: Deployment "deployment.engine.ear.bll.jar" is using a
deprecated module ("sun.jdk") which may be removed in future versions without
notice.
2022-04-26 13:52:28,604+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-4)
WFLYWELD0003: Processing weld deployment engine.ear
2022-04-26 13:52:28,713+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-4)
WFLYWELD0003: Processing weld deployment root.war
2022-04-26 13:52:28,747+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-1)
WFLYWELD0003: Processing weld deployment docs.war
2022-04-26 13:52:28,863+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-3)
WFLYWELD0003: Processing weld deployment enginesso.war
2022-04-26 13:52:28,929+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-4)
WFLYWELD0003: Processing weld deployment services.war
2022-04-26 13:52:28,978+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-1)
WFLYWELD0003: Processing weld deployment webadmin.war
2022-04-26 13:52:29,007+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-3)
WFLYWELD0003: Processing weld deployment welcome.war
2022-04-26 13:52:29,042+02 INFO [org.jboss.weld.deployer] (MSC service thread 1-2)
WFLYWELD0003: Processing weld deployment bll.jar
2022-04-26 13:52:29,067+02 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2)
WFLYEJB0473: JNDI bindings for session bean named 'LockManager' in deployment unit
'subdeployment "bll.jar" of deployment "engine.ear"' are as
follows:
2022-04-26 13:52:29,067+02 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2)
WFLYEJB0473: JNDI bindings for session bean named 'Backend' in deployment unit
'subdeployment "bll.jar" of deployment "engine.ear"' are as
follows:
2022-04-26 13:52:29,067+02 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2)
WFLYEJB0473: JNDI bindings for session bean named 'OvirtGlusterSchedulingService'
in deployment unit 'subdeployment "bll.jar" of deployment
"engine.ear"' are as follows:
2022-04-26 13:52:29,067+02 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2)
WFLYEJB0473: JNDI bindings for session bean named
'InitBackendServicesOnStartupBean' in deployment unit 'subdeployment
"bll.jar" of deployment "engine.ear"' are as follows:
2022-04-26 13:52:29,068+02 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2)
WFLYEJB0473: JNDI bindings for session bean named 'ManagedBlockStorageDiskUtil' in
deployment unit 'subdeployment "bll.jar" of deployment
"engine.ear"' are as follows:
2022-04-26 13:52:29,068+02 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2)
WFLYEJB0473: JNDI bindings for session bean named 'MacPoolPerCluster' in
deployment unit 'subdeployment "bll.jar" of deployment
"engine.ear"' are as follows:
2022-04-26 13:52:30,878+02 INFO [org.infinispan.CONTAINER] (ServerService Thread Pool --
51) ISPN000128: Infinispan version: Infinispan 'Corona Extra' 11.0.9.Final
2022-04-26 13:52:31,074+02 INFO [org.infinispan.PERSISTENCE] (ServerService Thread Pool
-- 41) ISPN000556: Starting user marshaller
'org.wildfly.clustering.infinispan.marshalling.jboss.JBossMarshaller'
2022-04-26 13:52:31,462+02 INFO [org.jboss.as.clustering.infinispan] (ServerService
Thread Pool -- 41) WFLYCLINF0002: Started dashboard cache from ovirt-engine container
2022-04-26 13:52:31,465+02 INFO [org.jboss.as.clustering.infinispan] (ServerService
Thread Pool -- 51) WFLYCLINF0002: Started inventory cache from ovirt-engine container
2022-04-26 13:52:35,426+02 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService
Thread Pool -- 45) RESTEASY002225: Deploying javax.ws.rs.core.Application: class
org.ovirt.engine.api.restapi.BackendApplication
2022-04-26 13:52:35,436+02 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService
Thread Pool -- 45) RESTEASY002210: Adding provider singleton
org.ovirt.engine.api.restapi.resource.validation.JsonExceptionMapper from Application
class org.ovirt.engine.api.restapi.BackendApplication
2022-04-26 13:52:35,436+02 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService
Thread Pool -- 45) RESTEASY002220: Adding singleton resource
org.ovirt.engine.api.restapi.resource.BackendApiResource from Application class
org.ovirt.engine.api.restapi.BackendApplication
2022-04-26 13:52:35,436+02 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService
Thread Pool -- 45) RESTEASY002210: Adding provider singleton
org.ovirt.engine.api.restapi.resource.validation.MappingExceptionMapper from Application
class org.ovirt.engine.api.restapi.BackendApplication
2022-04-26 13:52:35,436+02 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService
Thread Pool -- 45) RESTEASY002210: Adding provider singleton
org.ovirt.engine.api.restapi.resource.validation.IOExceptionMapper from Application class
org.ovirt.engine.api.restapi.BackendApplication
2022-04-26 13:52:35,436+02 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService
Thread Pool -- 45) RESTEASY002210: Adding provider singleton
org.ovirt.engine.api.restapi.resource.validation.MalformedIdExceptionMapper from
Application class org.ovirt.engine.api.restapi.BackendApplication
2022-04-26 13:52:35,437+02 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService
Thread Pool -- 45) RESTEASY002210: Adding provider singleton
org.ovirt.engine.api.restapi.resource.validation.ValidationExceptionMapper from
Application class org.ovirt.engine.api.restapi.BackendApplication
2022-04-26 13:52:35,615+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 45) WFLYUT0021: Registered web context: '/ovirt-engine/api' for server
'default-server'
2022-04-26 13:52:40,024+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 1)
WELD-001125: Illegal bean type
org.codehaus.jackson.map.deser.std.StdScalarDeserializer<java.lang.Enum<?>>
ignored on [EnhancedAnnotatedTypeImpl] public class
org.ovirt.engine.core.utils.serialization.json.JsonEnumDeserializer
2022-04-26 13:52:40,031+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 1)
WELD-001125: Illegal bean type class
org.codehaus.jackson.map.JsonDeserializer<java.lang.Enum<?>> ignored on
[EnhancedAnnotatedTypeImpl] public class
org.ovirt.engine.core.utils.serialization.json.JsonEnumDeserializer
2022-04-26 13:52:40,031+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 1)
WELD-001125: Illegal bean type class
org.codehaus.jackson.map.deser.std.StdDeserializer<java.lang.Enum<?>> ignored
on [EnhancedAnnotatedTypeImpl] public class
org.ovirt.engine.core.utils.serialization.json.JsonEnumDeserializer
2022-04-26 13:52:40,350+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 2)
WELD-001125: Illegal bean type
org.ovirt.engine.core.dao.DefaultGenericDao<org.ovirt.engine.core.common.businessentities.Provider<?>,
org.ovirt.engine.core.compat.Guid> ignored on [EnhancedAnnotatedTypeImpl] public @Named
@Singleton class org.ovirt.engine.core.dao.provider.ProviderDaoImpl
2022-04-26 13:52:40,351+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 2)
WELD-001125: Illegal bean type class
org.ovirt.engine.core.dao.DefaultReadDao<org.ovirt.engine.core.common.businessentities.Provider<?>,class
org.ovirt.engine.core.compat.Guid> ignored on [EnhancedAnnotatedTypeImpl] public @Named
@Singleton class org.ovirt.engine.core.dao.provider.ProviderDaoImpl
2022-04-26 13:52:40,351+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 2)
WELD-001125: Illegal bean type interface
org.ovirt.engine.core.dao.ModificationDao<org.ovirt.engine.core.common.businessentities.Provider<?>,class
org.ovirt.engine.core.compat.Guid> ignored on [EnhancedAnnotatedTypeImpl] public @Named
@Singleton class org.ovirt.engine.core.dao.provider.ProviderDaoImpl
2022-04-26 13:52:40,351+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 2)
WELD-001125: Illegal bean type
org.ovirt.engine.core.dao.SearchDao<org.ovirt.engine.core.common.businessentities.Provider<?>>
ignored on [EnhancedAnnotatedTypeImpl] public @Named @Singleton class
org.ovirt.engine.core.dao.provider.ProviderDaoImpl
2022-04-26 13:52:40,351+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 2)
WELD-001125: Illegal bean type interface
org.ovirt.engine.core.dao.ReadDao<org.ovirt.engine.core.common.businessentities.Provider<?>,class
org.ovirt.engine.core.compat.Guid> ignored on [EnhancedAnnotatedTypeImpl] public @Named
@Singleton class org.ovirt.engine.core.dao.provider.ProviderDaoImpl
2022-04-26 13:52:40,352+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 2)
WELD-001125: Illegal bean type
org.ovirt.engine.core.dao.GenericDao<org.ovirt.engine.core.common.businessentities.Provider<?>,
org.ovirt.engine.core.compat.Guid> ignored on [EnhancedAnnotatedTypeImpl] public @Named
@Singleton class org.ovirt.engine.core.dao.provider.ProviderDaoImpl
2022-04-26 13:52:40,424+02 INFO [org.jboss.weld.Bootstrap] (Weld Thread Pool -- 2)
WELD-001125: Illegal bean type
org.springframework.jdbc.core.RowMapper<org.ovirt.engine.core.common.businessentities.Provider<?>>
ignored on [EnhancedAnnotatedTypeImpl] private static class
org.ovirt.engine.core.dao.provider.ProviderDaoImpl$ProviderRowMapper
2022-04-26 13:52:41,564+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 58) WFLYUT0021: Registered web context: '/' for server
'default-server'
2022-04-26 13:52:41,603+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 51) WFLYUT0021: Registered web context: '/ovirt-engine/docs' for server
'default-server'
2022-04-26 13:52:41,659+02 WARN
[org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (ServerService Thread Pool --
47) IJ000407: No lazy enlistment available for ENGINEDataSource
2022-04-26 13:52:41,673+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 58) WFLYUT0021: Registered web context: '/ovirt-engine/services' for
server 'default-server'
2022-04-26 13:52:41,687+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 56) WFLYUT0021: Registered web context: '/ovirt-engine' for server
'default-server'
2022-04-26 13:52:41,825+02 INFO
[org.springframework.beans.factory.xml.XmlBeanDefinitionReader] (ServerService Thread Pool
-- 51) Loading XML bean definitions from class path resource
[org/springframework/jdbc/support/sql-error-codes.xml]
2022-04-26 13:52:41,889+02 WARN
[org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (ServerService Thread Pool --
47) IJ000407: No lazy enlistment available for DWHDataSource
2022-04-26 13:52:41,934+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 47) WFLYUT0021: Registered web context: '/ovirt-engine/webadmin' for
server 'default-server'
2022-04-26 13:52:41,938+02 INFO [org.wildfly.extension.undertow] (ServerService Thread
Pool -- 44) WFLYUT0021: Registered web context: '/ovirt-engine/sso' for server
'default-server'
2022-04-26 13:52:42,093+02 INFO [org.springframework.jdbc.support.SQLErrorCodesFactory]
(ServerService Thread Pool -- 51) SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix,
MS-SQL, MySQL, Oracle, PostgreSQL, Sybase, Hana]
2022-04-26 13:52:42,220+02 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool
-- 51) MSC000001: Failed to start service
jboss.deployment.subunit."engine.ear"."bll.jar".component.Backend.START:
org.jboss.msc.service.StartException in service
jboss.deployment.subunit."engine.ear"."bll.jar".component.Backend.START:
java.lang.IllegalStateException: WFLYEE0042: Failed to construct component instance
2022-04-26 13:52:42,231+02 ERROR [org.jboss.as.controller.management-operation]
(Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address:
([("deployment" => "engine.ear")]) - failure description:
{"WFLYCTL0080: Failed services" =>
{"jboss.deployment.subunit.\"engine.ear\".\"bll.jar\".component.Backend.START"
=> "java.lang.IllegalStateException: WFLYEE0042: Failed to construct component
instance
2022-04-26 13:52:42,240+02 INFO [org.jboss.as.server] (ServerService Thread Pool -- 25)
WFLYSRV0010: Deployed "ovirt-web-ui.war" (runtime-name :
"ovirt-web-ui.war")
2022-04-26 13:52:42,241+02 INFO [org.jboss.as.server] (ServerService Thread Pool -- 25)
WFLYSRV0010: Deployed "apidoc.war" (runtime-name : "apidoc.war")
2022-04-26 13:52:42,241+02 INFO [org.jboss.as.server] (ServerService Thread Pool -- 25)
WFLYSRV0010: Deployed "restapi.war" (runtime-name : "restapi.war")
2022-04-26 13:52:42,241+02 INFO [org.jboss.as.server] (ServerService Thread Pool -- 25)
WFLYSRV0010: Deployed "engine.ear" (runtime-name : "engine.ear")
2022-04-26 13:52:42,254+02 INFO [org.jboss.as.controller] (Controller Boot Thread)
WFLYCTL0183: Service status report
2022-04-26 13:52:42,564+02 INFO [org.jboss.as.server] (Controller Boot Thread)
WFLYSRV0212: Resuming server
2022-04-26 13:52:42,568+02 ERROR [org.jboss.as] (Controller Boot Thread) WFLYSRV0026:
WildFly Full 23.0.2.Final (WildFly Core 15.0.1.Final) started (with errors) in 24096ms -
Started 1668 of 1888 services (6 services failed or missing dependencies, 393 services are
lazy, passive or on-demand)
2022-04-26 13:52:42,574+02 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http
management interface listening on http://127.0.0.1:8706/management
2022-04-26 13:52:42,575+02 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051:
Admin console listening on http://127.0.0.1:8706