<div dir="ltr"><div>Hi,<br><br></div><div>I have two-node self hosted engine cluster with iscsi shared storage. Every ovirt-engine restart produces the following error in /var/log/ovirt-engine/engine.log. After that engine works without any problems.<br></div><div><br></div><div>How to fix it?<br></div><div><br>2015-12-18 11:33:00,053 ERROR [org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand] (org.ovirt.thread.pool-8-thread-11) [674068d8] Command &#39;org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand&#39; failed: CallableStatementCallback; SQL [{call deletelun(?)}]; ERROR: update or delete on table &quot;luns&quot; violates foreign key constraint &quot;disk_lun_to_lun_fk&quot; on table &quot;disk_lun_map&quot;<br>  Detail: Key (lun_id)=(36001405821da830526f48d0a961767c8) is still referenced from table &quot;disk_lun_map&quot;.<br>  Where: SQL statement &quot;DELETE FROM LUNs<br>   WHERE LUN_id = v_LUN_id&quot;<br>PL/pgSQL function deletelun(character varying) line 10 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: update or delete on table &quot;luns&quot; violates foreign key constraint &quot;disk_lun_to_lun_fk&quot; on table &quot;disk_lun_map&quot;<br>  Detail: Key (lun_id)=(36001405821da830526f48d0a961767c8) is still referenced from table &quot;disk_lun_map&quot;.<br>  Where: SQL statement &quot;DELETE FROM LUNs<br>   WHERE LUN_id = v_LUN_id&quot;<br>PL/pgSQL function deletelun(character varying) line 10 at SQL statement<br>2015-12-18 11:33:00,053 ERROR [org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand] (org.ovirt.thread.pool-8-thread-11) [674068d8] Exception: org.springframework.dao.DataIntegrityViolationException: CallableStatementCallback; SQL [{call deletelun(?)}]; ERROR: update or delete on table &quot;luns&quot; violates foreign key constraint &quot;disk_lun_to_lun_fk&quot; on table &quot;disk_lun_map&quot;<br>  Detail: Key (lun_id)=(36001405821da830526f48d0a961767c8) is still referenced from table &quot;disk_lun_map&quot;.<br>  Where: SQL statement &quot;DELETE FROM LUNs<br>   WHERE LUN_id = v_LUN_id&quot;<br>PL/pgSQL function deletelun(character varying) line 10 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: update or delete on table &quot;luns&quot; violates foreign key constraint &quot;disk_lun_to_lun_fk&quot; on table &quot;disk_lun_map&quot;<br>  Detail: Key (lun_id)=(36001405821da830526f48d0a961767c8) is still referenced from table &quot;disk_lun_map&quot;.<br>  Where: SQL statement &quot;DELETE FROM LUNs<br>   WHERE LUN_id = v_LUN_id&quot;<br>PL/pgSQL function deletelun(character varying) line 10 at SQL statement<br>        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:245) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:]<br>        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:142) [dal.jar:]<br>        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:80) [dal.jar:]<br>        at org.ovirt.engine.core.dao.LunDaoImpl.remove(LunDaoImpl.java:127) [dal.jar:]<br>        at org.ovirt.engine.core.dao.LunDaoImpl.remove(LunDaoImpl.java:18) [dal.jar:]<br>        at org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand.refreshLunsInfo(SyncLunsInfoForBlockStorageDomainCommand.java:94) [bll.jar:]<br>        at org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand$1.runInTransaction(SyncLunsInfoForBlockStorageDomainCommand.java:51) [bll.jar:]<br>        at org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand$1.runInTransaction(SyncLunsInfoForBlockStorageDomainCommand.java:48) [bll.jar:]<br>        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210) [utils.jar:]<br>        at org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand.executeCommand(SyncLunsInfoForBlockStorageDomainCommand.java:48) [bll.jar:]<br>        at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1215) [bll.jar:]<br>        at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1359) [bll.jar:]<br>        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1983) [bll.jar:]<br>        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:]<br>        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:]<br>        at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1396) [bll.jar:]<br>        at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:378) [bll.jar:]<br>        at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:475) [bll.jar:]<br>        at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:457) [bll.jar:]<br>        at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:404) [bll.jar:]<br>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_65]<br>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_65]<br>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_65]<br>        at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_65]<br>        at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)<br>        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407)<br>        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)<br>        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407)<br>        at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:55) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]<br>        at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)<br>        at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:260) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:399) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:243) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:95) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final]<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:326)<br>        at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:448)<br>        at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:326)<br>        at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)<br>        at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185)<br>        at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182)<br>        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)<br>        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)<br>        at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)<br>        at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view2.runInternalAction(Unknown Source) [bll.jar:]<br>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_65]<br>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_65]<br>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_65]<br>        at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_65]<br>        at org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:414) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]<br>        at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:127) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]<br>        at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]<br>        at org.jboss.weld.bean.proxy.InjectionPointPropagatingEnterpriseTargetBeanInstance.invoke(InjectionPointPropagatingEnterpriseTargetBeanInstance.java:65) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]<br>        at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]<br>        at org.ovirt.engine.core.bll.BackendCommandObjectsHandler$BackendInternal$BackendLocal$2049259618$Proxy$_$$_Weld$EnterpriseProxy$.runInternalAction(Unknown Source) [bll.jar:]<br>        at org.ovirt.engine.core.bll.VdsEventListener$3.run(VdsEventListener.java:255) [bll.jar:]<br>        at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:92) [utils.jar:]<br>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_65]<br>        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_65]<br>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_65]<br>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_65]<br>        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_65]<br>Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table &quot;luns&quot; violates foreign key constraint &quot;disk_lun_to_lun_fk&quot; on table &quot;disk_lun_map&quot;<br>  Detail: Key (lun_id)=(36001405821da830526f48d0a961767c8) is still referenced from table &quot;disk_lun_map&quot;.<br>  Where: SQL statement &quot;DELETE FROM LUNs<br>   WHERE LUN_id = v_LUN_id&quot;<br>PL/pgSQL function deletelun(character varying) line 10 at SQL statement<br>        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103)<br>        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)<br>        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)<br>        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)<br>        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)<br>        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:381)<br>        at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:301)<br>        at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404)<br>        at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE]<br>        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE]<br>        ... 106 more<br><br><br>Aleksey<br></div></div>