<div dir="ltr">Thank you. Hope we will have the fix soon.</div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Nov 20, 2015 at 11:45 AM, Martin Perina <span dir="ltr">&lt;<a href="mailto:mperina@redhat.com" target="_blank">mperina@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
database connections in aaa-jdbc extension are not closed properly,<br>
I&#39;ve created following bug [1] to track this issue.<br>
<br>
Thanks<br>
<br>
Martin Perina<br>
<br>
[1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1283936" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1283936</a><br>
<span class="im HOEnZb"><br>
----- Original Message -----<br>
&gt; From: &quot;Maksim Naumov&quot; &lt;<a href="mailto:maksim.naumov@hitmeister.de">maksim.naumov@hitmeister.de</a>&gt;<br>
&gt; To: &quot;Martin Perina&quot; &lt;<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>&gt;<br>
&gt; Cc: &quot;users&quot; &lt;<a href="mailto:users@ovirt.org">users@ovirt.org</a>&gt;<br>
</span><span class="im HOEnZb">&gt; Sent: Thursday, November 19, 2015 3:52:57 PM<br>
&gt; Subject: Re: [ovirt-users] [3.6] API - Internal Server Error<br>
&gt;<br>
</span><div class="HOEnZb"><div class="h5">&gt; Here is the new log entries:<br>
&gt;<br>
&gt;<br>
&gt; 2015-11-19 15:51:05,453 ERROR<br>
&gt; [org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension] (default<br>
&gt; task-285) [] Unexpected Exception invoking:<br>
&gt; AAA_AUTHN_AUTHENTICATE_CREDENTIALS[d9605c75-6b43-4b00-b32c-06bdfa80244c]<br>
&gt; 2015-11-19 15:51:05,453 DEBUG<br>
&gt; [org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension] (default<br>
&gt; task-285) [] Exception:: org.postgresql.util.PSQLException: FATAL:<br>
&gt; remaining connection slots are reserved for non-replication superuser<br>
&gt; connections<br>
&gt; at<br>
&gt; org.postgresql.core.v3.ConnectionFactoryImpl.readStartupMessages(ConnectionFactoryImpl.java:471)<br>
&gt; [postgresql.jar:]<br>
&gt; at<br>
&gt; org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:112)<br>
&gt; [postgresql.jar:]<br>
&gt; at<br>
&gt; org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)<br>
&gt; [postgresql.jar:]<br>
&gt; at<br>
&gt; org.postgresql.jdbc2.AbstractJdbc2Connection.&lt;init&gt;(AbstractJdbc2Connection.java:125)<br>
&gt; [postgresql.jar:]<br>
&gt; at<br>
&gt; org.postgresql.jdbc3.AbstractJdbc3Connection.&lt;init&gt;(AbstractJdbc3Connection.java:30)<br>
&gt; [postgresql.jar:]<br>
&gt; at<br>
&gt; org.postgresql.jdbc3g.AbstractJdbc3gConnection.&lt;init&gt;(AbstractJdbc3gConnection.java:22)<br>
&gt; [postgresql.jar:]<br>
&gt; at<br>
&gt; org.postgresql.jdbc4.AbstractJdbc4Connection.&lt;init&gt;(AbstractJdbc4Connection.java:32)<br>
&gt; [postgresql.jar:]<br>
&gt; at org.postgresql.jdbc4.Jdbc4Connection.&lt;init&gt;(Jdbc4Connection.java:24)<br>
&gt; [postgresql.jar:]<br>
&gt; at org.postgresql.Driver.makeConnection(Driver.java:393) [postgresql.jar:]<br>
&gt; at org.postgresql.Driver.connect(Driver.java:267) [postgresql.jar:]<br>
&gt; at java.sql.DriverManager.getConnection(DriverManager.java:571)<br>
&gt; [rt.jar:1.7.0_91]<br>
&gt; at java.sql.DriverManager.getConnection(DriverManager.java:215)<br>
&gt; [rt.jar:1.7.0_91]<br>
&gt; at<br>
&gt; org.ovirt.engine.extension.aaa.jdbc.core.datasource.DataSourceProvider$1.invoke(DataSourceProvider.java:86)<br>
&gt; [ovirt-engine-extension-aaa-jdbc.jar:]<br>
&gt; at com.sun.proxy.$Proxy162.getConnection(Unknown Source)<br>
&gt; at<br>
&gt; org.ovirt.engine.extension.aaa.jdbc.core.datasource.SchemaAwareDataSource.getConnection(SchemaAwareDataSource.java:40)<br>
&gt; [ovirt-engine-extension-aaa-jdbc.jar:]<br>
&gt; at<br>
&gt; org.ovirt.engine.extension.aaa.jdbc.core.datasource.Sql$Query.asResults(Sql.java:298)<br>
&gt; [ovirt-engine-extension-aaa-jdbc.jar:]<br>
&gt; at<br>
&gt; org.ovirt.engine.extension.aaa.jdbc.core.datasource.Sql$Query.asResults(Sql.java:285)<br>
&gt; [ovirt-engine-extension-aaa-jdbc.jar:]<br>
&gt; at org.ovirt.engine.extension.aaa.jdbc.core.Schema.get(Schema.java:743)<br>
&gt; [ovirt-engine-extension-aaa-jdbc.jar:]<br>
&gt; at org.ovirt.engine.extension.aaa.jdbc.core.Tasks.execute(Tasks.java:49)<br>
&gt; [ovirt-engine-extension-aaa-jdbc.jar:]<br>
&gt; at<br>
&gt; org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension.invoke(AuthnExtension.java:50)<br>
&gt; [ovirt-engine-extension-aaa-jdbc.jar:]<br>
&gt; at<br>
&gt; org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:49)<br>
&gt; at<br>
&gt; org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:73)<br>
&gt; at<br>
&gt; org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:109)<br>
&gt; at<br>
&gt; org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.handleCredentials(BasicAuthenticationFilter.java:134)<br>
&gt; at<br>
&gt; org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:84)<br>
&gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; at<br>
&gt; org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:77)<br>
&gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; at<br>
&gt; org.ovirt.engine.core.aaa.filters.EngineSessionTokenAuthenticationFilter.doFilter(EngineSessionTokenAuthenticationFilter.java:31)<br>
&gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; at<br>
&gt; org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35)<br>
&gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; at<br>
&gt; org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111)<br>
&gt; at<br>
&gt; org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102)<br>
&gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; at<br>
&gt; org.ovirt.engine.api.common.security.CORSSupportFilter.doFilter(CORSSupportFilter.java:183)<br>
&gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)<br>
&gt; at<br>
&gt; org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)<br>
&gt; at<br>
&gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)<br>
&gt; at<br>
&gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; at<br>
&gt; io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51)<br>
&gt; at<br>
&gt; io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56)<br>
&gt; at<br>
&gt; io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70)<br>
&gt; at<br>
&gt; io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)<br>
&gt; at<br>
&gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; at<br>
&gt; org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)<br>
&gt; at<br>
&gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; at<br>
&gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:248)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:77)<br>
&gt; at<br>
&gt; io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:167)<br>
&gt; at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)<br>
&gt; at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:766)<br>
&gt; at<br>
&gt; java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)<br>
&gt; [rt.jar:1.7.0_91]<br>
&gt; at<br>
&gt; java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<br>
&gt; [rt.jar:1.7.0_91]<br>
&gt; at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_91]<br>
&gt;<br>
&gt; On Wed, Nov 18, 2015 at 7:51 PM, Martin Perina &lt;<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>&gt; wrote:<br>
&gt;<br>
&gt; &gt; Hi,<br>
&gt; &gt;<br>
&gt; &gt; I&#39;m sorry, but I&#39;m not able to find out the cause. Could you please<br>
&gt; &gt; enable debug log for aaa-jdbc extension using following steps and<br>
&gt; &gt; send me logs again?<br>
&gt; &gt;<br>
&gt; &gt; 1. Edit file<br>
&gt; &gt; /usr/share/ovirt-engine/share/ovirt-engine/services/ovirt-engine/<br>
&gt; &gt; <a href="http://ovirt-engine.xml.in" rel="noreferrer" target="_blank">ovirt-engine.xml.in</a><br>
&gt; &gt;<br>
&gt; &gt; 2. Locate line that contains following comment:<br>
&gt; &gt;<br>
&gt; &gt;      &lt;!-- Loggers for the engine: --&gt;<br>
&gt; &gt;<br>
&gt; &gt;    and append following lines:<br>
&gt; &gt;<br>
&gt; &gt;      &lt;logger category=&quot;org.ovirt.engine.extension.aaa.jdbc&quot;&gt;<br>
&gt; &gt;         &lt;level name=&quot;DEBUG&quot;/&gt;<br>
&gt; &gt;       &lt;/logger&gt;<br>
&gt; &gt;<br>
&gt; &gt; 3. Restart ovirt-engine service:<br>
&gt; &gt;<br>
&gt; &gt;    EL7/Fedora22:<br>
&gt; &gt;      systemctl restart ovirt-engine<br>
&gt; &gt;<br>
&gt; &gt;    EL6:<br>
&gt; &gt;      service ovirt-engine restart<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Thanks<br>
&gt; &gt;<br>
&gt; &gt; Martin Perina<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; ----- Original Message -----<br>
&gt; &gt; &gt; From: &quot;Maksim Naumov&quot; &lt;<a href="mailto:maksim.naumov@hitmeister.de">maksim.naumov@hitmeister.de</a>&gt;<br>
&gt; &gt; &gt; To: &quot;Martin Perina&quot; &lt;<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>&gt;<br>
&gt; &gt; &gt; Cc: &quot;users&quot; &lt;<a href="mailto:users@ovirt.org">users@ovirt.org</a>&gt;<br>
&gt; &gt; &gt; Sent: Wednesday, November 18, 2015 6:15:35 PM<br>
&gt; &gt; &gt; Subject: Re: [ovirt-users] [3.6] API - Internal Server Error<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Here is my versions:<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Name        : ovirt-engine<br>
&gt; &gt; &gt; Version     : 3.6.0.3<br>
&gt; &gt; &gt; Release     : 1.el7.centos<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Name        : ovirt-engine-extension-aaa-jdbc<br>
&gt; &gt; &gt; Version     : 1.0.1<br>
&gt; &gt; &gt; Release     : 1.el7<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; About the log:<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; engile.log contains only one line related to this error, you can see it<br>
&gt; &gt; in<br>
&gt; &gt; &gt; the first message. But server.log has a bit more.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Currently, I do the following:<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; I am running the script which is asking for API every 10 sec for a list<br>
&gt; &gt; of<br>
&gt; &gt; &gt; current running VMs.<br>
&gt; &gt; &gt; After some hours (1-2 hours) the script reporting that there are no VMs.<br>
&gt; &gt; &gt; I went to the API endpoint by myself and everything that I was able to<br>
&gt; &gt; see<br>
&gt; &gt; &gt; is Internal error.<br>
&gt; &gt; &gt; I stopped the script and in a couple of minutes API worked again.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Thank you<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; SERVER.LOG<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; 2015-11-18 18:13:40,833 ERROR [io.undertow.request] (default task-291)<br>
&gt; &gt; &gt; UT005023: Exception handling request to /api/vms: Class: class<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.extensions.mgr.ExtensionInvokeCommandFailedException<br>
&gt; &gt; &gt; Input:<br>
&gt; &gt; &gt; {Extkey[name=AAA_AUTHN_CREDENTIALS;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=AAA_AUTHN_CREDENTIALS[03b96485-4bb5-4592-8167-810a5c909706];]=***,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_INVOKE_CONTEXT;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.api.extensions.ExtMap;uuid=EXTENSION_INVOKE_CONTEXT[886d2ebb-312a-49ae-9cc3-e1f849834b7d];]={Extkey[name=EXTENSION_INTERFACE_VERSION_MAX;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.Integer;uuid=EXTENSION_INTERFACE_VERSION_MAX[f4cff49f-2717-4901-8ee9-df362446e3e7];]=0,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_LICENSE;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_LICENSE[8a61ad65-054c-4e31-9c6d-1ca4d60a4c18];]=ASL<br>
&gt; &gt; &gt; 2.0, Extkey[name=EXTENSION_NOTES;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_NOTES[2da5ad7e-185a-4584-aaff-97f66978e4ea];]=Display<br>
&gt; &gt; &gt; name: &quot;ovirt-engine-extension-aaa-jdbc&quot;,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_HOME_URL;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_HOME_URL[4ad7a2f4-f969-42d4-b399-72d192e18304];]=<br>
&gt; &gt; &gt; <a href="http://www.ovirt.org" rel="noreferrer" target="_blank">http://www.ovirt.org</a>, Extkey[name=EXTENSION_LOCALE;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_LOCALE[0780b112-0ce0-404a-b85e-8765d778bb29];]=en_US,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_NAME;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_NAME[651381d3-f54f-4547-bf28-b0b01a103184];]=&quot;ovirt-engine-extension-aaa-jdbc&quot;.authn,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_INTERFACE_VERSION_MIN;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.Integer;uuid=EXTENSION_INTERFACE_VERSION_MIN[2b84fc91-305b-497b-a1d7-d961b9d2ce0b];]=0,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_CONFIGURATION;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.util.Properties;uuid=EXTENSION_CONFIGURATION[2d48ab72-f0a1-4312-b4ae-5068a226b0fc];]=***,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_AUTHOR;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_AUTHOR[ef242f7a-2dad-4bc5-9aad-e07018b7fbcc];]=The<br>
&gt; &gt; &gt; oVirt Project, Extkey[name=EXTENSION_INSTANCE_NAME;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_INSTANCE_NAME[65c67ff6-aeca-4bd5-a245-8674327f011b];]=internal-authn,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_BUILD_INTERFACE_VERSION;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.Integer;uuid=EXTENSION_BUILD_INTERFACE_VERSION[cb479e5a-4b23-46f8-aed3-56a4747a8ab7];]=0,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_CONFIGURATION_SENSITIVE_KEYS;type=interface<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.util.Collection;uuid=EXTENSION_CONFIGURATION_SENSITIVE_KEYS[a456efa1-73ff-4204-9f9b-ebff01e35263];]=[],<br>
&gt; &gt; &gt; Extkey[name=AAA_AUTHN_CAPABILITIES;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.Long;uuid=AAA_AUTHN_CAPABILITIES[9d16bee3-10fd-46f2-83f9-3d3c54cf258d];]=44,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_GLOBAL_CONTEXT;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.api.extensions.ExtMap;uuid=EXTENSION_GLOBAL_CONTEXT[9799e72f-7af6-4cf1-bf08-297bc8903676];]=*skip*,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_VERSION;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_VERSION[fe35f6a8-8239-4bdb-ab1a-af9f779ce68c];]=&quot;1.0.1&quot;,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_MANAGER_TRACE_LOG;type=interface<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.slf4j.Logger;uuid=EXTENSION_MANAGER_TRACE_LOG[863db666-3ea7-4751-9695-918a3197ad83];]=org.slf4j.impl.Slf4jLogger(org.ovirt.engine.core.extensions.mgr.ExtensionsManager.trace.&quot;ovirt-engine-extension-aaa-jdbc&quot;.authn.internal-authn),<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_PROVIDES;type=interface<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.util.Collection;uuid=EXTENSION_PROVIDES[8cf373a6-65b5-4594-b828-0e275087de91];]=[org.ovirt.engine.api.extensions.aaa.Authn],<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_CONFIGURATION_FILE;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=EXTENSION_CONFIGURATION_FILE[4fb0ffd3-983c-4f3f-98ff-9660bd67af6a];]=/etc/ovirt-engine/extensions.d/internal-authn.properties},<br>
&gt; &gt; &gt; Extkey[name=AAA_AUTHN_USER;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.String;uuid=AAA_AUTHN_USER[1ceaba26-1bdc-4663-a3c6-5d926f9dd8f0];]=admin,<br>
&gt; &gt; &gt; Extkey[name=EXTENSION_INVOKE_COMMAND;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.api.extensions.ExtUUID;uuid=EXTENSION_INVOKE_COMMAND[485778ab-bede-4f1a-b823-77b262a2f28d];]=AAA_AUTHN_AUTHENTICATE_CREDENTIALS[d9605c75-6b43-4b00-b32c-06bdfa80244c]}<br>
&gt; &gt; &gt; Output:<br>
&gt; &gt; &gt; {Extkey[name=EXTENSION_INVOKE_RESULT;type=class<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.lang.Integer;uuid=EXTENSION_INVOKE_RESULT[0909d91d-8bde-40fb-b6c0-099c772ddd4e];]=2}<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:91)<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.extensions.mgr.ExtensionProxy.invoke(ExtensionProxy.java:109)<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.handleCredentials(BasicAuthenticationFilter.java:134)<br>
&gt; &gt; &gt; [aaa.jar:]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:84)<br>
&gt; &gt; &gt; [aaa.jar:]<br>
&gt; &gt; &gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:77)<br>
&gt; &gt; &gt; [aaa.jar:]<br>
&gt; &gt; &gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.aaa.filters.EngineSessionTokenAuthenticationFilter.doFilter(EngineSessionTokenAuthenticationFilter.java:31)<br>
&gt; &gt; &gt; [aaa.jar:]<br>
&gt; &gt; &gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35)<br>
&gt; &gt; &gt; [aaa.jar:]<br>
&gt; &gt; &gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111)<br>
&gt; &gt; &gt; [interface-common-jaxrs.jar:]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102)<br>
&gt; &gt; &gt; [interface-common-jaxrs.jar:]<br>
&gt; &gt; &gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.ovirt.engine.api.common.security.CORSSupportFilter.doFilter(CORSSupportFilter.java:183)<br>
&gt; &gt; &gt; [interface-common-jaxrs.jar:]<br>
&gt; &gt; &gt; at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:51)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:248)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:77)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:167)<br>
&gt; &gt; &gt; [undertow-servlet.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:766)<br>
&gt; &gt; &gt; [undertow-core.jar:1.1.5.Final]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)<br>
&gt; &gt; &gt; [rt.jar:1.7.0_91]<br>
&gt; &gt; &gt; at<br>
&gt; &gt; &gt;<br>
&gt; &gt; java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<br>
&gt; &gt; &gt; [rt.jar:1.7.0_91]<br>
&gt; &gt; &gt; at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_91]<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; On Wed, Nov 18, 2015 at 6:05 PM, Martin Perina &lt;<a href="mailto:mperina@redhat.com">mperina@redhat.com</a>&gt;<br>
&gt; &gt; wrote:<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; Hi,<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; could you please take a look what versions do you have installed?<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;   rpm -qi ovirt-engine<br>
&gt; &gt; &gt; &gt;   rpm -qi ovirt-engine-extension-aaa-jdbc<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; Could you please also send your server.log and engine.log?<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; Thanks<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; Martin Perina<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; ----- Original Message -----<br>
&gt; &gt; &gt; &gt; &gt; From: &quot;Maksim Naumov&quot; &lt;<a href="mailto:maksim.naumov@hitmeister.de">maksim.naumov@hitmeister.de</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; To: &quot;users&quot; &lt;<a href="mailto:users@ovirt.org">users@ovirt.org</a>&gt;<br>
&gt; &gt; &gt; &gt; &gt; Sent: Wednesday, November 18, 2015 5:56:53 PM<br>
&gt; &gt; &gt; &gt; &gt; Subject: [ovirt-users] [3.6] API - Internal Server Error<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; After some time, API stopped work.<br>
&gt; &gt; &gt; &gt; &gt; When I try to run any request I see only &quot;Internal Server Error&quot; and<br>
&gt; &gt; the<br>
&gt; &gt; &gt; &gt; log<br>
&gt; &gt; &gt; &gt; &gt; like this:<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; 2015-11-18 17:53:05,634 ERROR<br>
&gt; &gt; &gt; &gt; &gt; [org.ovirt.engine.extension.aaa.jdbc.binding.api.AuthnExtension]<br>
&gt; &gt; (default<br>
&gt; &gt; &gt; &gt; &gt; task-314) [] Unexpected Exception invoking:<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; AAA_AUTHN_AUTHENTICATE_CREDENTIALS[d9605c75-6b43-4b00-b32c-06bdfa80244c]<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; How can fix it?<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; --<br>
&gt; &gt; &gt; &gt; &gt; Maksim Naumov<br>
&gt; &gt; &gt; &gt; &gt; Hitmeister GmbH<br>
&gt; &gt; &gt; &gt; &gt; Softwareentwickler<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; Habsburgerring 2<br>
&gt; &gt; &gt; &gt; &gt; 50674 Köln<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; E: <a href="mailto:maksim.naumov@hitmeister.de">maksim.naumov@hitmeister.de</a><br>
&gt; &gt; &gt; &gt; &gt; <a href="http://www.hitmeister.de" rel="noreferrer" target="_blank">www.hitmeister.de</a><br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; HRB 59046, Amtsgericht Köln<br>
&gt; &gt; &gt; &gt; &gt; Geschäftsführer: Dr. Gerald Schönbucher<br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt; &gt; _______________________________________________<br>
&gt; &gt; &gt; &gt; &gt; Users mailing list<br>
&gt; &gt; &gt; &gt; &gt; <a href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
&gt; &gt; &gt; &gt; &gt; <a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
&gt; &gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; --<br>
&gt; &gt; &gt; Maksim Naumov<br>
&gt; &gt; &gt; Hitmeister GmbH<br>
&gt; &gt; &gt; Softwareentwickler<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Habsburgerring 2<br>
&gt; &gt; &gt; 50674 Köln<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; E: <a href="mailto:maksim.naumov@hitmeister.de">maksim.naumov@hitmeister.de</a><br>
&gt; &gt; &gt; <a href="http://www.hitmeister.de" rel="noreferrer" target="_blank">www.hitmeister.de</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; HRB 59046, Amtsgericht Köln<br>
&gt; &gt; &gt; Geschäftsführer: Dr. Gerald Schönbucher<br>
&gt; &gt; &gt;<br>
&gt; &gt;<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; --<br>
&gt; Maksim Naumov<br>
&gt; Hitmeister GmbH<br>
&gt; Softwareentwickler<br>
&gt;<br>
&gt; Habsburgerring 2<br>
&gt; 50674 Köln<br>
&gt;<br>
&gt; E: <a href="mailto:maksim.naumov@hitmeister.de">maksim.naumov@hitmeister.de</a><br>
&gt; <a href="http://www.hitmeister.de" rel="noreferrer" target="_blank">www.hitmeister.de</a><br>
&gt;<br>
&gt; HRB 59046, Amtsgericht Köln<br>
&gt; Geschäftsführer: Dr. Gerald Schönbucher<br>
&gt;<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr">Maksim Naumov<div>Hitmeister GmbH </div><div>Softwareentwickler</div><div><br></div><div>Habsburgerring 2</div><div>50674 Köln</div><div><br></div><div>E: <a href="mailto:maksim.naumov@hitmeister.de" target="_blank">maksim.naumov@hitmeister.de</a></div><div><a href="http://www.hitmeister.de" target="_blank">www.hitmeister.de</a> </div><div><br></div><div>HRB 59046, Amtsgericht Köln</div><div>Geschäftsführer: Dr. Gerald Schönbucher<br></div></div></div></div></div>
</div>