[Users] Ovirt-Engine stop working
Juan Hernandez
jhernand at redhat.com
Mon Oct 29 09:50:52 UTC 2012
On 10/28/2012 06:13 PM, EricD wrote:
> Hello Guys,
>
> Hope you can help on this one.
>
> I have a fresh install of ovirt-engine 3.1, it was working fine since
> the installation 4 days ago, now the Ovirt-Engine is experiencing issue
> while I tried to shutdown a VM on the management interfaces. Since then
> the is not going further after I try to login with the right credentials.
>
> I try to reboot several times
We have seen some problems with some very long running operations, but
mainly using REST API, not with the GWT user interface. Can you check
what is the content of the /etc/httpd.conf.d/ovirt-engine.conf file? You
should have something like this:
ProxyPass / ajp://localhost:8009/
This doesn't specify a timeout for the connection between the web server
and the application server. The default is 60 seconds, so some long
running operations could fail to complete. Try to add an explicit timeout:
ProxyPass / ajp://localhost:8009/ timeout=3600
That will increase the timeout to one hour.
Restart the web server (service httpd restart) and try again.
Please let us know what is the result.
>
> here the log that I have
>
> engine.log :
> 2012-10-28 08:55:00,023 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Autorecovering 0 storage domains
> 2012-10-28 08:55:00,023 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Checking autorecoverable storage domains done
> 2012-10-28 08:55:14,330 INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> (QuartzScheduler_Worker-23) [23d9d0ed] Domain
> 8da021a8-d5d7-4dd0-bdaf-d9f317757705 recovered from problem. vds:
> hypervisor02-b.in.miromedia.ca <http://hypervisor02-b.in.miromedia.ca>
> 2012-10-28 08:55:14,332 INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> (QuartzScheduler_Worker-23) [23d9d0ed] Domain
> 8da021a8-d5d7-4dd0-bdaf-d9f317757705 has recovered from problem. No
> active host in the DC is reporting it as problematic, so clearing the
> domain recovery timer.
> 2012-10-28 09:00:00,001 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Checking autorecoverable hosts
> 2012-10-28 09:00:00,004 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Autorecovering 0 hosts
> 2012-10-28 09:00:00,004 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Checking autorecoverable hosts done
> 2012-10-28 09:00:00,005 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Checking autorecoverable storage domains
> 2012-10-28 09:00:00,007 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Autorecovering 0 storage domains
> 2012-10-28 09:00:00,008 INFO
> [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (QuartzScheduler_Worker-90) Checking autorecoverable storage domains done
>
> server.log :
> 2012-10-28 08:06:32,260 ERROR
> [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/webadmin].[org.ovirt.engine.ui.frontend.WebAdmin
> GenericApiGWTService]] (ajp--0.0.0.0-8009-13) Servlet.service() for
> servlet org.ovirt.engine.ui.frontend.WebAdmin GenericApiGWTService threw
> exception: java.lang.RuntimeException: Unable to report failure
> at
> com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doUnexpectedFailure(AbstractRemoteServiceServlet.java:107)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:67)
> [gwt-servlet-2.3.0.jar:]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
> [jboss-servlet-3.0-api.jar:1.0.1.Final]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
> [jboss-servlet-3.0-api.jar:1.0.1.Final]
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
> [jboss-web.jar:]
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
> [jboss-web.jar:]
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
> [jboss-web.jar:]
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
> [jboss-web.jar:]
> at
> org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
> [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
> [jboss-web.jar:]
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> [jboss-web.jar:]
> at
> org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
> [jboss-web.jar:]
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> [jboss-web.jar:]
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
> [jboss-web.jar:]
> at
> org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505)
> [jboss-web.jar:]
> at
> org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445)
> [jboss-web.jar:]
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
> [jboss-web.jar:]
> at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
> Caused by: java.lang.RuntimeException: Unable to continue
> at
> com.google.gwt.rpc.client.ast.RpcCommandVisitor.halt(RpcCommandVisitor.java:337)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.WebModePayloadSink$PayloadVisitor.commit(WebModePayloadSink.java:520)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.WebModePayloadSink$PayloadVisitor.commit(WebModePayloadSink.java:493)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.WebModePayloadSink$PayloadVisitor.visit(WebModePayloadSink.java:407)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.client.ast.ReturnCommand.traverse(ReturnCommand.java:44)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.client.ast.RpcCommandVisitor.doAccept(RpcCommandVisitor.java:320)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.client.ast.RpcCommandVisitor.accept(RpcCommandVisitor.java:42)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.WebModePayloadSink.accept(WebModePayloadSink.java:883)
> [gwt-servlet-2.3.0.jar:]
> at com.google.gwt.rpc.server.RPC.streamResponse(RPC.java:472)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:198)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:161)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:222)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
> [gwt-servlet-2.3.0.jar:]
> ... 16 more
> Caused by: com.google.gwt.user.client.rpc.SerializationException: Could
> not send data
> at
> com.google.gwt.rpc.server.WebModePayloadSink.send(WebModePayloadSink.java:923)
> [gwt-servlet-2.3.0.jar:]
> at
> com.google.gwt.rpc.server.WebModePayloadSink$PayloadVisitor.commit(WebModePayloadSink.java:518)
> [gwt-servlet-2.3.0.jar:]
> ... 27 more
> Caused by: ClientAbortException: java.net.SocketException: Broken pipe
> at
> org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:403)
> [jboss-web.jar:]
> at
> org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:449)
> [jboss-web.jar:]
> at
> org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
> [jboss-web.jar:]
> at
> org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:426)
> [jboss-web.jar:]
> at
> org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:415)
> [jboss-web.jar:]
> at
> org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
> [jboss-web.jar:]
> at
> com.google.gwt.rpc.server.WebModePayloadSink.send(WebModePayloadSink.java:921)
> [gwt-servlet-2.3.0.jar:]
> ... 28 more
> Caused by: java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> [rt.jar:1.7.0_09-icedtea]
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
> [rt.jar:1.7.0_09-icedtea]
> at
> java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> [rt.jar:1.7.0_09-icedtea]
> at
> org.apache.coyote.ajp.AjpProcessor$SocketOutputBuffer.doWrite(AjpProcessor.java:1337)
> [jboss-web.jar:]
> at org.apache.coyote.Response.doWrite(Response.java:594)
> [jboss-web.jar:]
> at
> org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:398)
> [jboss-web.jar:]
> ... 34 more
>
>
> httpd error_log :
> [Sun Oct 28 09:08:06 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:09:26 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:09:31 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:09:46 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:10:46 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:10:51 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:11:11 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:11:27 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:12:07 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
> [Sun Oct 28 09:12:11 2012] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
>
> iptables rules :
> Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
> pkts bytes target prot opt in out source
> destination
> 45397 17M RH-Firewall-1-INPUT all -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
>
> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
> pkts bytes target prot opt in out source
> destination
> 0 0 RH-Firewall-1-INPUT all -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
>
> Chain OUTPUT (policy ACCEPT 48291 packets, 15M bytes)
> pkts bytes target prot opt in out source
> destination
>
> Chain RH-Firewall-1-INPUT (2 references)
> pkts bytes target prot opt in out source
> destination
> 24226 8281K ACCEPT all -- lo * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> 0 0 ACCEPT icmp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> icmptype 255
> 20372 9116K ACCEPT all -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state RELATED,ESTABLISHED
> 1 60 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:22
> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:80
> 103 6180 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:443
> 0 0 ACCEPT udp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW udp dpt:111
> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:111
> 0 0 ACCEPT udp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW udp dpt:892
> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:892
> 0 0 ACCEPT udp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW udp dpt:875
> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:875
> 0 0 ACCEPT udp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW udp dpt:662
> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:662
> 1 60 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:2049
> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW tcp dpt:32803
> 0 0 ACCEPT udp -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> state NEW udp dpt:32769
> 694 76288 REJECT all -- * * 0.0.0.0/0
> <http://0.0.0.0/0> 0.0.0.0/0 <http://0.0.0.0/0>
> reject-with icmp-host-prohibited
>
>
> LISTEN PORT :
> tcp 0 0 0.0.0.0:4447 <http://0.0.0.0:4447>
> 0.0.0.0:* LISTEN 663/-server
> tcp 0 0 0.0.0.0:2049 <http://0.0.0.0:2049>
> 0.0.0.0:* LISTEN -
> tcp 0 0 0.0.0.0:32803 <http://0.0.0.0:32803>
> 0.0.0.0:* LISTEN -
> tcp 0 0 127.0.0.1:9990 <http://127.0.0.1:9990>
> 0.0.0.0:* LISTEN 663/-server
> tcp 0 0 0.0.0.0:8009 <http://0.0.0.0:8009>
> 0.0.0.0:* LISTEN 663/-server
> tcp 0 0 127.0.0.1:9999 <http://127.0.0.1:9999>
> 0.0.0.0:* LISTEN 663/-server
> tcp 0 0 0.0.0.0:111 <http://0.0.0.0:111>
> 0.0.0.0:* LISTEN 589/rpcbind
> tcp 0 0 0.0.0.0:8080 <http://0.0.0.0:8080>
> 0.0.0.0:* LISTEN 663/-server
> tcp 0 0 0.0.0.0:20048 <http://0.0.0.0:20048>
> 0.0.0.0:* LISTEN 641/rpc.mountd
> tcp 0 0 0.0.0.0:22 <http://0.0.0.0:22>
> 0.0.0.0:* LISTEN 600/sshd
> tcp 0 0 127.0.0.1:5432 <http://127.0.0.1:5432>
> 0.0.0.0:* LISTEN 620/postgres
> tcp 0 0 0.0.0.0:53144 <http://0.0.0.0:53144>
> 0.0.0.0:* LISTEN 619/rpc.statd
> tcp 0 0 127.0.0.1:25 <http://127.0.0.1:25>
> 0.0.0.0:* LISTEN 890/sendmail: accep
> tcp 0 0 0.0.0.0:8443 <http://0.0.0.0:8443>
> 0.0.0.0:* LISTEN 663/-server
> tcp6 0 0 :::2049 :::*
> LISTEN -
> tcp6 0 0 :::32803 :::*
> LISTEN -
> tcp6 0 0 :::111 :::*
> LISTEN 589/rpcbind
> tcp6 0 0 :::80 :::*
> LISTEN 645/httpd
> tcp6 0 0 :::20048 :::*
> LISTEN 641/rpc.mountd
> tcp6 0 0 :::22 :::*
> LISTEN 600/sshd
> tcp6 0 0 ::1:5432 :::*
> LISTEN 620/postgres
> tcp6 0 0 :::443 :::*
> LISTEN 645/httpd
> tcp6 0 0 :::44413 :::*
> LISTEN 619/rpc.statd
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
--
Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3ºD, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
More information about the Users
mailing list