Hi, I recently upgraded my testlab to oVirt 4.2-pre and have been having some issues. One of which is fencing with Dell Drac 8 remote access cards. The configuration I'm using works fine on my other (4.1.6) cluster... Error: Health check on Host host.example.com indicates that future attempts to Start this host using Power-Management are expected to fail. Fence agent setup for the host appears to be working though: Which logs would be helpful in debugging this issue ? Regards, Brett
On Mon, Oct 9, 2017 at 2:40 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi,
I recently upgraded my testlab to oVirt 4.2-pre and have been having some issues.
One of which is fencing with Dell Drac 8 remote access cards. The configuration I'm using works fine on my other (4.1.6) cluster...
Error: Health check on Host host.example.com indicates that future attempts to Start this host using Power-Management are expected to fail.
Hi, Could you please share engine.log around above message? We are executing get power management status call to check, so if this get PM status failed, the above error is raised. Thanks Martin
Fence agent setup for the host appears to be working though:
Which logs would be helpful in debugging this issue ?
Regards, Brett
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Martin, Sure is there a way to trigger the check (I cleared the alerts), or do I just need to have a poke around the engine log next time it happens? Regards, Brett On 9 October 2017 at 14:26, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 2:40 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi,
I recently upgraded my testlab to oVirt 4.2-pre and have been having some issues.
One of which is fencing with Dell Drac 8 remote access cards. The configuration I'm using works fine on my other (4.1.6) cluster...
Error: Health check on Host host.example.com indicates that future attempts to Start this host using Power-Management are expected to fail.
Hi,
Could you please share engine.log around above message? We are executing get power management status call to check, so if this get PM status failed, the above error is raised.
Thanks
Martin
Fence agent setup for the host appears to be working though:
Which logs would be helpful in debugging this issue ?
Regards, Brett
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hope this is enough of the log, no idea why it's saying invalid fence agents: 2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] HostName = ov_host.example.com 2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Command 'ConnectStoragePoolVDSCommand(HostName = ov_host.example.com, ConnectStoragePoolVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', vdsId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147', masterVersion='2'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Could not connect host 'ov_host.example.com' to pool 'GHDC', as the master domain is in inactive/unknown status - not failing the operation 2017-10-09 16:01:01,565+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [594458d4] START, SetMOMPolicyParametersVDSCommand(HostName = ov_host.example.com, MomPolicyVDSParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'}), log id: 43532a7a 2017-10-09 16:01:01,595+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] START, SpmStopVDSCommand(HostName = ov_host.example.com, SpmStopVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147'}), log id: 3861bb24 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command HSMGetAllTasksStatusesVDS failed: Not SPM: () 2017-10-09 16:01:01,602+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] HostName = ov_host.example.com 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = ov_host.example.com, VdsIdVDSCommandParametersBase:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2017-10-09 16:01:01,653+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_DETECTED(13), Status of host ov_host.example.com was set to Up. 2017-10-09 16:01:01,656+01 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host ov_host.example.com.Invalid fence agents defined for host '192.168.1.12'. 2017-10-09 16:01:01,659+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host ov_host.example.com power management. On 9 October 2017 at 14:41, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi Martin,
Sure is there a way to trigger the check (I cleared the alerts), or do I just need to have a poke around the engine log next time it happens?
Regards, Brett
On 9 October 2017 at 14:26, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 2:40 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi,
I recently upgraded my testlab to oVirt 4.2-pre and have been having some issues.
One of which is fencing with Dell Drac 8 remote access cards. The configuration I'm using works fine on my other (4.1.6) cluster...
Error: Health check on Host host.example.com indicates that future attempts to Start this host using Power-Management are expected to fail.
Hi,
Could you please share engine.log around above message? We are executing get power management status call to check, so if this get PM status failed, the above error is raised.
Thanks
Martin
Fence agent setup for the host appears to be working though:
Which logs would be helpful in debugging this issue ?
Regards, Brett
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Mon, Oct 9, 2017 at 6:00 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hope this is enough of the log, no idea why it's saying invalid fence agents:
2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE- ManagedThreadFactory-engine-Thread-184) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850- 8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE- ManagedThreadFactory-engine-Thread-184) [] HostName = ov_host.example.com 2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core. vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE- ManagedThreadFactory-engine-Thread-184) [] Command 'ConnectStoragePoolVDSCommand(HostName = ov_host.example.com, ConnectStoragePoolVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', vdsId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147', masterVersion='2'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850- 8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Could not connect host 'ov_host.example.com' to pool 'GHDC', as the master domain is in inactive/unknown status - not failing the operation
This seems like severe storage issue , if your master storage domain is inactive/unknown, then your VMs cannot run. You would need to check VDSM logs to find the real cause of the issue. Are you sure there are no storage or network issues on your setup?
2017-10-09 16:01:01,565+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [594458d4] START, SetMOMPolicyParametersVDSCommand(HostName = ov_host.example.com, MomPolicyVDSParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'}), log id: 43532a7a 2017-10-09 16:01:01,595+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.EE- ManagedThreadFactory-engine-Thread-210) [6eeda756] START, SpmStopVDSCommand(HostName = ov_host.example.com, SpmStopVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147'}), log id: 3861bb24 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE- ManagedThreadFactory-engine-Thread-210) [6eeda756] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command HSMGetAllTasksStatusesVDS failed: Not SPM: () 2017-10-09 16:01:01,602+01 INFO [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] HostName = ov_host.example.com 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core. vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = ov_host.example.com, VdsIdVDSCommandParametersBase:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2017-10-09 16:01:01,653+01 INFO [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_DETECTED(13), Status of host ov_host.example.com was set to Up. 2017-10-09 16:01:01,656+01 WARN [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host ov_host.example.com.Invalid fence agents defined for host '192.168.1.12'. 2017-10-09 16:01:01,659+01 ERROR [org.ovirt.engine.core.dal. dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host ov_host.example.com power management.
We are trying to get power management status of the host, when activating the host. In that case there was an error trying when getting power management status, more details should be in VDSM logs of the host that acted as fence proxy.
On 9 October 2017 at 14:41, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi Martin,
Sure is there a way to trigger the check (I cleared the alerts), or do I just need to have a poke around the engine log next time it happens?
Yes, please take a look at engine-config option PMHealthCheckIntervalInSec: engine-config -l | grep PMHealthCheckIntervalInSec After updating this value you need to restart ovirt-engine service.
Regards, Brett
On 9 October 2017 at 14:26, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 2:40 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi,
I recently upgraded my testlab to oVirt 4.2-pre and have been having some issues.
One of which is fencing with Dell Drac 8 remote access cards. The configuration I'm using works fine on my other (4.1.6) cluster...
Error: Health check on Host host.example.com indicates that future attempts to Start this host using Power-Management are expected to fail.
Hi,
Could you please share engine.log around above message? We are executing get power management status call to check, so if this get PM status failed, the above error is raised.
Thanks
Martin
Fence agent setup for the host appears to be working though:
Which logs would be helpful in debugging this issue ?
Regards, Brett
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Martin, Storage isn't an issue, that was me trying to force fix another problem (removing 'dead' hosted engine hosts) This just popped up in engine.log: 2017-10-10 06:53:09,022+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Exception: java.lang.IllegalArgumentException: VdsFenceOptionMapping has no value for version: general at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:65) [dal.jar:] at org.ovirt.engine.core.common.config.Config.getValue(Config.java:28) [common.jar:] at org.ovirt.engine.core.utils.pm.FenceConfigHelper.getFenceConfigurationValue(FenceConfigHelper.java:96) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.cacheFenceAgentsOptionMapping(VdsFenceOptions.java:78) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.initCache(VdsFenceOptions.java:236) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:63) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:47) [utils.jar:] at org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery.executeQueryCommand(GetAgentFenceOptionsQuery.java:18) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor.execute(DefaultBackendQueryExecutor.java:14) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:570) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:539) [bll.jar:] at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor150.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runQuery(GenericApiGWTServiceImpl.java:90) at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleQueries(GenericApiGWTServiceImpl.java:121) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.service(GenericApiGWTServiceImpl.java:78) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) [branding.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:66) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_144] 2017-10-10 06:53:09,032+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Query 'GetAgentFenceOptionsQuery' failed: VdsFenceOptionMapping has no value for version: general 2017-10-10 06:53:09,032+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Exception: java.lang.IllegalArgumentException: VdsFenceOptionMapping has no value for version: general at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:65) [dal.jar:] at org.ovirt.engine.core.common.config.Config.getValue(Config.java:28) [common.jar:] at org.ovirt.engine.core.utils.pm.FenceConfigHelper.getFenceConfigurationValue(FenceConfigHelper.java:96) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.cacheFenceAgentsOptionMapping(VdsFenceOptions.java:78) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.initCache(VdsFenceOptions.java:236) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:63) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:47) [utils.jar:] at org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery.executeQueryCommand(GetAgentFenceOptionsQuery.java:18) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor.execute(DefaultBackendQueryExecutor.java:14) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:570) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:539) [bll.jar:] at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor150.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runQuery(GenericApiGWTServiceImpl.java:90) at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleQueries(GenericApiGWTServiceImpl.java:121) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.service(GenericApiGWTServiceImpl.java:78) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) [branding.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:66) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_144] On 9 October 2017 at 19:21, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 6:00 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hope this is enough of the log, no idea why it's saying invalid fence agents:
2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] HostName = ov_host.example.com 2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.vdsbrok er.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Command 'ConnectStoragePoolVDSCommand(HostName = ov_host.example.com, ConnectStoragePoolVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', vdsId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147', masterVersion='2'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Could not connect host 'ov_host.example.com' to pool 'GHDC', as the master domain is in inactive/unknown status - not failing the operation
This seems like severe storage issue , if your master storage domain is inactive/unknown, then your VMs cannot run. You would need to check VDSM logs to find the real cause of the issue. Are you sure there are no storage or network issues on your setup?
2017-10-09 16:01:01,565+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [594458d4] START, SetMOMPolicyParametersVDSCommand(HostName = ov_host.example.com, MomPolicyVDSParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'}), log id: 43532a7a 2017-10-09 16:01:01,595+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] START, SpmStopVDSCommand(HostName = ov_host.example.com, SpmStopVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147'}), log id: 3861bb24 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command HSMGetAllTasksStatusesVDS failed: Not SPM: () 2017-10-09 16:01:01,602+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] HostName = ov_host.example.com 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.vdsbrok er.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = ov_host.example.com, VdsIdVDSCommandParametersBase:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2017-10-09 16:01:01,653+01 INFO [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_DETECTED(13), Status of host ov_host.example.com was set to Up. 2017-10-09 16:01:01,656+01 WARN [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host ov_host.example.com.Invalid fence agents defined for host '192.168.1.12'. 2017-10-09 16:01:01,659+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host ov_host.example.com power management.
We are trying to get power management status of the host, when activating the host. In that case there was an error trying when getting power management status, more details should be in VDSM logs of the host that acted as fence proxy.
On 9 October 2017 at 14:41, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi Martin,
Sure is there a way to trigger the check (I cleared the alerts), or do I just need to have a poke around the engine log next time it happens?
Yes, please take a look at engine-config option PMHealthCheckIntervalInSec:
engine-config -l | grep
PMHealthCheckIntervalInSec
After updating this value you need to restart ovirt-engine service.
Regards, Brett
On 9 October 2017 at 14:26, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 2:40 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi,
I recently upgraded my testlab to oVirt 4.2-pre and have been having some issues.
One of which is fencing with Dell Drac 8 remote access cards. The configuration I'm using works fine on my other (4.1.6) cluster...
Error: Health check on Host host.example.com indicates that future attempts to Start this host using Power-Management are expected to fail.
Hi,
Could you please share engine.log around above message? We are executing get power management status call to check, so if this get PM status failed, the above error is raised.
Thanks
Martin
Fence agent setup for the host appears to be working though:
Which logs would be helpful in debugging this issue ?
Regards, Brett
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi, according to error and source codes you are using either oVirt 4.2 Alpha or some nightly build, right? The error has already been fixed, so you need to wait for oVirt 4.2 Beta or upgrade to some recent nighlty build. Anyway please be aware that we cannot yet recommend using 4.2 Alpha/nightly in production, so use at your won risk. Regards Martin On Tue, Oct 10, 2017 at 7:56 AM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi Martin,
Storage isn't an issue, that was me trying to force fix another problem (removing 'dead' hosted engine hosts)
This just popped up in engine.log:
2017-10-10 06:53:09,022+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Exception: java.lang.IllegalArgumentException: VdsFenceOptionMapping has no value for version: general at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:65) [dal.jar:] at org.ovirt.engine.core.common.config.Config.getValue(Config.java:28) [common.jar:] at org.ovirt.engine.core.utils.pm.FenceConfigHelper. getFenceConfigurationValue(FenceConfigHelper.java:96) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions. cacheFenceAgentsOptionMapping(VdsFenceOptions.java:78) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.initCache(VdsFenceOptions.java:236) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:63) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:47) [utils.jar:] at org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery. executeQueryCommand(GetAgentFenceOptionsQuery.java:18) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor. execute(DefaultBackendQueryExecutor.java:14) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:570) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:539) [bll.jar:] at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceMethodIntercep tor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation. proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocatio nContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.weld.interceptor.proxy.WeldInvocationContext. interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext. proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final. jar:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor. LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$ SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext. proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final. jar:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocatio nContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport .delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. delegateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. doMethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. processInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1. processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation. proceed(InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors. CorrelationIdTrackerInterceptor.aroundInvoke( CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor150.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceLifecycleMetho dInterceptor.processInvocation(ManagedReferenceLifecycleMetho dInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics. ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor. processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation( InitialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation( ChainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors. ComponentDispatcherInterceptor.processInvocation( ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.singleton. SingletonComponentInstanceAssociationInterceptor.processInvocation( SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation. proceed(InterceptorContext.java:509) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivat ionInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInter ceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors. CurrentInvocationContextInterceptor.processInvocation( CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.CR1.jar: 11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor. processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor. processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor. processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors. ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor. processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor. processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor. processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext.run( InterceptorContext.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecked( WildFlySecurityManager.java:609) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation( AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation( ChainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke( ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation( ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke( ProxyInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl. runQuery(GenericApiGWTServiceImpl.java:90) at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl. runMultipleQueries(GenericApiGWTServiceImpl.java:121) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC. java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall( RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall( RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost( RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost( AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl. service(GenericApiGWTServiceImpl.java:78) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest( ServletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter. doFilter(GwtCachingFilter.java:132) at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at org.ovirt.engine.core.branding.BrandingFilter. doFilter(BrandingFilter.java:73) [branding.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:66) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest( FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler. handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler. handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security. SecurityContextAssociationHandler.handleRequest( SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.servlet.handlers.security. SSLInformationAssociationHandler.handleRequest( SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security. ServletAuthenticationCallHandler.handleRequest( ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandle r.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler .handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security. ServletConfidentialityConstraintHandler.handleRequest( ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security. ServletSecurityConstraintHandler.handleRequest( ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandle r.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security. CachedAuthenticatedSessionHandler.handleRequest( CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler. handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssocia tionHandler.handleRequest(AbstractSecurityContextAssocia tionHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler. handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler. handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$ 100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call( ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call( ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSet upAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call( ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security. SecurityContextThreadSetupAction.lambda$create$0( SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment. UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment. UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment. UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest( ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$ 000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest( ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors. java:326) at io.undertow.server.HttpServerExchange$1.run( HttpServerExchange.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_144]
2017-10-10 06:53:09,032+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Query 'GetAgentFenceOptionsQuery' failed: VdsFenceOptionMapping has no value for version: general 2017-10-10 06:53:09,032+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Exception: java.lang.IllegalArgumentException: VdsFenceOptionMapping has no value for version: general at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:65) [dal.jar:] at org.ovirt.engine.core.common.config.Config.getValue(Config.java:28) [common.jar:] at org.ovirt.engine.core.utils.pm.FenceConfigHelper. getFenceConfigurationValue(FenceConfigHelper.java:96) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions. cacheFenceAgentsOptionMapping(VdsFenceOptions.java:78) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.initCache(VdsFenceOptions.java:236) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:63) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:47) [utils.jar:] at org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery. executeQueryCommand(GetAgentFenceOptionsQuery.java:18) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor. execute(DefaultBackendQueryExecutor.java:14) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:570) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:539) [bll.jar:] at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceMethodIntercep tor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation. proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocatio nContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.weld.interceptor.proxy.WeldInvocationContext. interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext. proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final. jar:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor. LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$ SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext. proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final. jar:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocatio nContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport .delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. delegateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. doMethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. processInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1. processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation. proceed(InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors. CorrelationIdTrackerInterceptor.aroundInvoke( CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor150.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceLifecycleMetho dInterceptor.processInvocation(ManagedReferenceLifecycleMetho dInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics. ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor. processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation( InitialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation( ChainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors. ComponentDispatcherInterceptor.processInvocation( ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.singleton. SingletonComponentInstanceAssociationInterceptor.processInvocation( SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation. proceed(InterceptorContext.java:509) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivat ionInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInter ceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors. CurrentInvocationContextInterceptor.processInvocation( CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.CR1.jar: 11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor. processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor. processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor. processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors. ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor. processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor. processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor. processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext.run( InterceptorContext.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecked( WildFlySecurityManager.java:609) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation( AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed( InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation( ChainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke( ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation( ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke( ProxyInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl. runQuery(GenericApiGWTServiceImpl.java:90) at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl. runMultipleQueries(GenericApiGWTServiceImpl.java:121) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC. java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall( RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall( RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost( RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost( AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl. service(GenericApiGWTServiceImpl.java:78) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest( ServletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter. doFilter(GwtCachingFilter.java:132) at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at org.ovirt.engine.core.branding.BrandingFilter. doFilter(BrandingFilter.java:73) [branding.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:66) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter( ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl. doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest( FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler. handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler. handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security. SecurityContextAssociationHandler.handleRequest( SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.servlet.handlers.security. SSLInformationAssociationHandler.handleRequest( SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security. ServletAuthenticationCallHandler.handleRequest( ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandle r.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler .handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security. ServletConfidentialityConstraintHandler.handleRequest( ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security. ServletSecurityConstraintHandler.handleRequest( ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandle r.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security. CachedAuthenticatedSessionHandler.handleRequest( CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler. handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssocia tionHandler.handleRequest(AbstractSecurityContextAssocia tionHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler. handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest( PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler. handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$ 100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call( ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call( ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSet upAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call( ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security. SecurityContextThreadSetupAction.lambda$create$0( SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment. UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment. UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment. UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest( ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$ 000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest( ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors. java:326) at io.undertow.server.HttpServerExchange$1.run( HttpServerExchange.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_144]
On 9 October 2017 at 19:21, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 6:00 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hope this is enough of the log, no idea why it's saying invalid fence agents:
2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] HostName = ov_host.example.com 2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.vdsbrok er.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Command 'ConnectStoragePoolVDSCommand(HostName = ov_host.example.com, ConnectStoragePoolVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', vdsId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147', masterVersion='2'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Could not connect host 'ov_host.example.com' to pool 'GHDC', as the master domain is in inactive/unknown status - not failing the operation
This seems like severe storage issue , if your master storage domain is inactive/unknown, then your VMs cannot run. You would need to check VDSM logs to find the real cause of the issue. Are you sure there are no storage or network issues on your setup?
2017-10-09 16:01:01,565+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [594458d4] START, SetMOMPolicyParametersVDSCommand(HostName = ov_host.example.com, MomPolicyVDSParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'}), log id: 43532a7a 2017-10-09 16:01:01,595+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] START, SpmStopVDSCommand(HostName = ov_host.example.com, SpmStopVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147'}), log id: 3861bb24 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command HSMGetAllTasksStatusesVDS failed: Not SPM: () 2017-10-09 16:01:01,602+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] HostName = ov_host.example.com 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.vdsbrok er.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = ov_host.example.com, VdsIdVDSCommandParametersBase:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2017-10-09 16:01:01,653+01 INFO [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_DETECTED(13), Status of host ov_host.example.com was set to Up. 2017-10-09 16:01:01,656+01 WARN [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host ov_host.example.com.Invalid fence agents defined for host '192.168.1.12'. 2017-10-09 16:01:01,659+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host ov_host.example.com power management.
We are trying to get power management status of the host, when activating the host. In that case there was an error trying when getting power management status, more details should be in VDSM logs of the host that acted as fence proxy.
On 9 October 2017 at 14:41, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi Martin,
Sure is there a way to trigger the check (I cleared the alerts), or do I just need to have a poke around the engine log next time it happens?
Yes, please take a look at engine-config option PMHealthCheckIntervalInSec:
engine-config -l | grep
PMHealthCheckIntervalInSec
After updating this value you need to restart ovirt-engine service.
Regards, Brett
On 9 October 2017 at 14:26, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 2:40 PM, Maton, Brett <matonb@ltresources.co.uk
wrote:
Hi,
I recently upgraded my testlab to oVirt 4.2-pre and have been having some issues.
One of which is fencing with Dell Drac 8 remote access cards. The configuration I'm using works fine on my other (4.1.6) cluster...
Error: Health check on Host host.example.com indicates that future attempts to Start this host using Power-Management are expected to fail.
Hi,
Could you please share engine.log around above message? We are executing get power management status call to check, so if this get PM status failed, the above error is raised.
Thanks
Martin
Fence agent setup for the host appears to be working though:
Which logs would be helpful in debugging this issue ?
Regards, Brett
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Ok no problem, This is a testlab setup, I'll give the nightly a go. Thanks, Brett On 10 October 2017 at 07:48, Martin Perina <mperina@redhat.com> wrote:
Hi,
according to error and source codes you are using either oVirt 4.2 Alpha or some nightly build, right? The error has already been fixed, so you need to wait for oVirt 4.2 Beta or upgrade to some recent nighlty build.
Anyway please be aware that we cannot yet recommend using 4.2 Alpha/nightly in production, so use at your won risk.
Regards
Martin
On Tue, Oct 10, 2017 at 7:56 AM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi Martin,
Storage isn't an issue, that was me trying to force fix another problem (removing 'dead' hosted engine hosts)
This just popped up in engine.log:
2017-10-10 06:53:09,022+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Exception: java.lang.IllegalArgumentException: VdsFenceOptionMapping has no value for version: general at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils. getValue(DBConfigUtils.java:65) [dal.jar:] at org.ovirt.engine.core.common.config.Config.getValue(Config.java:28) [common.jar:] at org.ovirt.engine.core.utils.pm.FenceConfigHelper.getFenceCon figurationValue(FenceConfigHelper.java:96) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.cacheFenceAge ntsOptionMapping(VdsFenceOptions.java:78) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.initCache(VdsFenceOptions.java:236) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:63) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:47) [utils.jar:] at org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery.executeQ ueryCommand(GetAgentFenceOptionsQuery.java:18) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecut or.execute(DefaultBackendQueryExecutor.java:14) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:570) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:539) [bll.jar:] at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor. processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed( InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext .proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.inter ceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proce ed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.ja r:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingIntercept or.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocatio n$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invok eNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.ja r:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proce ed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.ja r:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext .proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSuppor t.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.del egateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doM ethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.pro cessInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactor y$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed( InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerI nterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor150.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInt erceptor.processInvocation(ManagedReferenceLifecycleMethodIn terceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeI nterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.proc essInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation(In itialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(Ch ainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.ComponentDispatcherIn terceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.singleton.SingletonComponentInst anceAssociationInterceptor.processInvocation(SingletonCompon entInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.CR1.jar:1 1.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed( InterceptorContext.java:509) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationIntercep tor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor. processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationCo ntextInterceptor.processInvocation(CurrentInvoc ationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterc eptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor.proces sInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitIntercep tor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptor Factory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor. processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor.proces sInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor.processIn vocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext.run(InterceptorConte xt.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecke d(WildFlySecurityManager.java:609) at org.jboss.invocation.AccessCheckingInterceptor.processInvoca tion(AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(Ch ainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke(ViewServic e.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocatio n(ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(Prox yInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTService Impl.runQuery(GenericApiGWTServiceImpl.java:90) at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTService Impl.runMultipleQueries(GenericApiGWTServiceImpl.java:121) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse( RPC.java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processC all(RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processC all(RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processP ost(RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet. doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTService Impl.service(GenericApiGWTServiceImpl.java:78) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest(Se rvletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doF ilter(GwtCachingFilter.java:132) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) [branding.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:66) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(Fil terHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHan dler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handl eRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssoc iationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociat ionHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationC allHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandle r.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler .handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentiality ConstraintHandler.handleRequest(ServletConfident ialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstra intHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandle r.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSes sionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.ha ndleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssocia tionHandler.handleRequest(AbstractSecurityContextAssociation Handler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHa ndler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFir stRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$10 0(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(Se rvletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(Se rvletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAct ion$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1. call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThrea dSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeployment InfoService$UndertowThreadSetupAction.lambda$create$0(Undert owDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeployment InfoService$UndertowThreadSetupAction.lambda$create$0(Undert owDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeployment InfoService$UndertowThreadSetupAction.lambda$create$0(Undert owDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchR equest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$00 0(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleR equest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors. java:326) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchan ge.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_144]
2017-10-10 06:53:09,032+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Query 'GetAgentFenceOptionsQuery' failed: VdsFenceOptionMapping has no value for version: general 2017-10-10 06:53:09,032+01 ERROR [org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery] (default task-62) [3bfc7ef1-bede-40b4-8a97-a8cd1cf2a0f6] Exception: java.lang.IllegalArgumentException: VdsFenceOptionMapping has no value for version: general at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils. getValue(DBConfigUtils.java:65) [dal.jar:] at org.ovirt.engine.core.common.config.Config.getValue(Config.java:28) [common.jar:] at org.ovirt.engine.core.utils.pm.FenceConfigHelper.getFenceCon figurationValue(FenceConfigHelper.java:96) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.cacheFenceAge ntsOptionMapping(VdsFenceOptions.java:78) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.initCache(VdsFenceOptions.java:236) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:63) [utils.jar:] at org.ovirt.engine.core.utils.pm.VdsFenceOptions.<init>(VdsFenceOptions.java:47) [utils.jar:] at org.ovirt.engine.core.bll.GetAgentFenceOptionsQuery.executeQ ueryCommand(GetAgentFenceOptionsQuery.java:18) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecut or.execute(DefaultBackendQueryExecutor.java:14) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:570) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:539) [bll.jar:] at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor. processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed( InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext .proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.inter ceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proce ed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.ja r:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingIntercept or.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocatio n$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invok eNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.ja r:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proce ed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.ja r:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext .proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSuppor t.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.del egateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doM ethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.pro cessInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactor y$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed( InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerI nterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor150.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInt erceptor.processInvocation(ManagedReferenceLifecycleMethodIn terceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeI nterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.proc essInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation(In itialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(Ch ainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.ComponentDispatcherIn terceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.singleton.SingletonComponentInst anceAssociationInterceptor.processInvocation(SingletonCompon entInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.CR1.jar:1 1.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed( InterceptorContext.java:509) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationIntercep tor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor. processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationCo ntextInterceptor.processInvocation(CurrentInvoc ationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterc eptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor.proces sInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitIntercep tor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptor Factory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor. processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.CR1.jar:11.0.0.CR1] at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor.proces sInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor.processIn vocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.InterceptorContext.run(InterceptorConte xt.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecke d(WildFlySecurityManager.java:609) at org.jboss.invocation.AccessCheckingInterceptor.processInvoca tion(AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed(InterceptorC ontext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(Ch ainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke(ViewServic e.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocatio n(ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(Prox yInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runQuery(Unknown Source) [common.jar:] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTService Impl.runQuery(GenericApiGWTServiceImpl.java:90) at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTService Impl.runMultipleQueries(GenericApiGWTServiceImpl.java:121) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) [:1.8.0_144] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_144] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_144] at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse( RPC.java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processC all(RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processC all(RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processP ost(RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet. doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTService Impl.service(GenericApiGWTServiceImpl.java:78) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest(Se rvletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doF ilter(GwtCachingFilter.java:132) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) [branding.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:66) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilte r.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.d oFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(Fil terHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHan dler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handl eRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssoc iationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociat ionHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationC allHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandle r.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler .handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentiality ConstraintHandler.handleRequest(ServletConfident ialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstra intHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandle r.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSes sionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.ha ndleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssocia tionHandler.handleRequest(AbstractSecurityContextAssociation Handler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHa ndler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(P redicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFir stRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$10 0(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(Se rvletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(Se rvletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAct ion$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1. call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThrea dSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeployment InfoService$UndertowThreadSetupAction.lambda$create$0(Undert owDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeployment InfoService$UndertowThreadSetupAction.lambda$create$0(Undert owDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeployment InfoService$UndertowThreadSetupAction.lambda$create$0(Undert owDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchR equest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$00 0(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleR equest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors. java:326) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchan ge.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_144] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_144] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_144]
On 9 October 2017 at 19:21, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 6:00 PM, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hope this is enough of the log, no idea why it's saying invalid fence agents:
2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] HostName = ov_host.example.com 2017-10-09 16:01:01,535+01 ERROR [org.ovirt.engine.core.vdsbrok er.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Command 'ConnectStoragePoolVDSCommand(HostName = ov_host.example.com, ConnectStoragePoolVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', vdsId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147', masterVersion='2'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=47d12516-a41c-41a7-9da4-320f08fda147, msdUUID=a28ebb26-98be-4850-8e88-0ac2c3d6037a' 2017-10-09 16:01:01,535+01 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-184) [] Could not connect host 'ov_host.example.com' to pool 'GHDC', as the master domain is in inactive/unknown status - not failing the operation
This seems like severe storage issue , if your master storage domain is inactive/unknown, then your VMs cannot run. You would need to check VDSM logs to find the real cause of the issue. Are you sure there are no storage or network issues on your setup?
2017-10-09 16:01:01,565+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [594458d4] START, SetMOMPolicyParametersVDSCommand(HostName = ov_host.example.com, MomPolicyVDSParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'}), log id: 43532a7a 2017-10-09 16:01:01,595+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] START, SpmStopVDSCommand(HostName = ov_host.example.com, SpmStopVDSCommandParameters:{hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74', storagePoolId='47d12516-a41c-41a7-9da4-320f08fda147'}), log id: 3861bb24 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov_host.example.com command HSMGetAllTasksStatusesVDS failed: Not SPM: () 2017-10-09 16:01:01,602+01 INFO [org.ovirt.engine.core.vdsbrok er.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] HostName = ov_host.example.com 2017-10-09 16:01:01,602+01 ERROR [org.ovirt.engine.core.vdsbrok er.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (org.ovirt.thread.EE-ManagedThreadFactory-engine-Thread-210) [6eeda756] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = ov_host.example.com, VdsIdVDSCommandParametersBase: {hostId='9390df92-5c79-4a25-9e94-1ca2d4aedc74'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () 2017-10-09 16:01:01,653+01 INFO [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_DETECTED(13), Status of host ov_host.example.com was set to Up. 2017-10-09 16:01:01,656+01 WARN [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host ov_host.example.com.Invalid fence agents defined for host '192.168.1.12'. 2017-10-09 16:01:01,659+01 ERROR [org.ovirt.engine.core.dal.dbb roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [2e03c9cf] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host ov_host.example.com power management.
We are trying to get power management status of the host, when activating the host. In that case there was an error trying when getting power management status, more details should be in VDSM logs of the host that acted as fence proxy.
On 9 October 2017 at 14:41, Maton, Brett <matonb@ltresources.co.uk> wrote:
Hi Martin,
Sure is there a way to trigger the check (I cleared the alerts), or do I just need to have a poke around the engine log next time it happens?
Yes, please take a look at engine-config option PMHealthCheckIntervalInSec:
engine-config -l | grep
PMHealthCheckIntervalInSec
After updating this value you need to restart ovirt-engine service.
Regards, Brett
On 9 October 2017 at 14:26, Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 9, 2017 at 2:40 PM, Maton, Brett < matonb@ltresources.co.uk> wrote:
> Hi, > > I recently upgraded my testlab to oVirt 4.2-pre and have been > having some issues. > > One of which is fencing with Dell Drac 8 remote access cards. > The configuration I'm using works fine on my other (4.1.6) > cluster... > > Error: > Health check on Host host.example.com indicates that future > attempts to Start this host using Power-Management are expected to fail. >
Hi,
Could you please share engine.log around above message? We are executing get power management status call to check, so if this get PM status failed, the above error is raised.
Thanks
Martin
> > > Fence agent setup for the host appears to be working though: > > > > > Which logs would be helpful in debugging this issue ? > > Regards, > Brett > > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > >
participants (2)
-
Martin Perina -
Maton, Brett