We recently upgraded from postgres 9.5.4 to 9.5.6 and then to 9.5.7. We restarted ovirt-engine between 9.5.4 and it worked as expected. After we updated to 9.5.7 we have started to get postgres errors in engine.log and a 404 error in the browser when we try to start ovirt-engine.

Updates were done on client and server to keep them in sync.

The error is complaining about a missing function, but we can verify that the function exists in the database. We've also restored engine and ovirt_engine_history from a backup at 8am this morning when it was working and before we upgraded to 9.5.7 with no apparent help and run engine-setup afterwards, still no luck.

2017-05-23 15:07:48,418 INFO [org.ovirt.engine.core.dal.dbbroker.DbFacade] (ServerService Thread Pool -- 51) [] Initializing the DbFacade
2017-05-23 15:07:48,480 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 51) [] Start initializing Backend
2017-05-23 15:07:48,502 ERROR [org.ovirt.engine.core.bll.network.macpool.MacPoolPerDc] (ServerService Thread Pool -- 46) [] Error initializing: PreparedStatementCallback; bad SQL grammar [select * from getallmacpoolrangesbymacpoolid()]; nested exception is org.postgresql.util.PSQL
Exception: ERROR: function getallmacpoolrangesbymacpoolid() does not exist
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Position: 16
2017-05-23 15:07:48,517 ERROR [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 51) [] Error during initialization: org.jboss.weld.exceptions.WeldException: WELD-000049: Unable to invoke public void org.ovirt.engine.core.bll.hostedengine.PreviousHostedEngineHost.cre
ate() on org.ovirt.engine.core.bll.hostedengine.PreviousHostedEngineHost@3a35f461
       at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:100) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:81) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:126) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:162) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.bean.ContextualInstanceStrategy$ApplicationScopedContextualInstanceStrategy.get(ContextualInstanceStrategy.java:141) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:742) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.bean.builtin.InstanceImpl.getBeanInstance(InstanceImpl.java:189) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.weld.bean.builtin.InstanceImpl.get(InstanceImpl.java:100) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.ovirt.engine.core.bll.Backend.loadService(Backend.java:301) [bll.jar:]
       at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:212) [bll.jar:]
       at org.ovirt.engine.core.bll.Backend.create(Backend.java:159) [bll.jar:]
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_131]
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_131]
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_131]
       at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_131]
       at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96)
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
       at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:117) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:103) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)
       at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final]
       at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:53) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.as.ee.component.AroundConstructInterceptorFactory$1.processInvocation(AroundConstructInterceptorFactory.java:28)
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.as.weld.injection.WeldInterceptorInjectionInterceptor.processInvocation(WeldInterceptorInjectionInterceptor.java:56) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.as.ee.component.ComponentInstantiatorInterceptor.processInvocation(ComponentInstantiatorInterceptor.java:74)
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.as.weld.ejb.Jsr299BindingsCreateInterceptor.processInvocation(Jsr299BindingsCreateInterceptor.java:100) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
       at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)
       at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:263) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]
       at org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:70) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final]