Re: [ovirt-users] Moving a Hosted Engine from Fedora 20 to CentOS 7

On Sat, Nov 7, 2015 at 2:38 AM, John Florian <jflorian(a)doubledog.org&gt; wrote: > On 10/29/2015 05:49 AM, Roy Golan wrote: > > > > On Thu, Oct 29, 2015 at 11:39 AM, Roy Golan <rgolan(a)redhat.com&gt; wrote: >> >> >> On Wed, Oct 28, 2015 at 10:23 PM, John Florian <jflorian(a)doubledog.org&gt; >> wrote: >>> Can somebody please point me to documentation or describe how I should >>> proceed with this task? I see lots of pages for moving from a physical >>> engine to a VM and vice-versa but am having no luck finding how to go >>> about building a new HE to obsolete my original. >>> >> using ovirt-hosted-engine-setup you can choose a setup without using the >> appliance. So you can scratch install your VM > > > BTW the ovirt-engine-appliacnce we build [1] is Centos based. Seems like > perfect candidate. > > #install the appliance > yum install ovirt-engine-appliance > > #and then run the setup > ovirt-hosted-engine-setup > > choose "disk" in this stage > > Please specify the device to boot the VM from (cdrom, disk, pxe) [cdrom]: > disk > > it will suggest the downloaded appliance automatically > > See this wiki for reference > http://www.ovirt.org/Features/HEApplianceFlow#Testing > > > > I'm afraid I'm lost here. Here's a map of my setup: > > oVirt 3.5.5 hosted engine is named enceladus-f20 (on Fedora 20) > I have one oVirt 3.5.5 Host named oberon-f20 (also on Fedora 20) > I previously had one other oVirt 3.5.5 Host named ophelia-f20 > > I took opehlia down, installed CentOS 7 on it and attempted a "hosted-engine > --deploy". I can't remember if that was 3.5.5 or 3.6, but I could not add > it to my cluster. From what I could gather enceladus-f20 provided an > emulation type of pc1.0 while ophelia-c7 didn't seem to have anything that > matched, the closest being just "pc". That looked hopelessly complicated to > resolve so I thought I'd try again, but this time putting F22 on the ophelia > and doing a 3.6 HE deploy saying yes to the redeploy prompt. This time I > was met with: > > Checking for oVirt-Engine status at enceladus-f20.doubledog.org... > > [ INFO ] Engine replied: DB Up!Welcome to Health Status! > > [ ERROR ] Cannot automatically add the host to cluster Default: Cannot add > Host. Connecting to host via SSH has failed, verify that the host is > reachable (IP address, routable address etc.) You may refer to the > engine.log file for further details. > > > > Please check Engine VM configuration. > > On enceladus-f20, I see: > > ==> /var/log/ovirt-engine/engine.log <== > > 2015-11-06 19:17:12,085 INFO > [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp--127.0.0.1-8702-9) > Running command: LoginUserCommand internal: false. > > 2015-11-06 19:17:12,128 INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (ajp--127.0.0.1-8702-9) Correlation ID: null, Call Stack: null, Custom Event > ID: -1, Message: User admin@internal logged in. > > ==> /var/log/ovirt-engine/server.log <== > > 2015-11-06 19:17:13,654 INFO > [org.apache.sshd.client.session.ClientSessionImpl] (pool-18-thread-1) Client > session created > > 2015-11-06 19:17:13,663 INFO > [org.apache.sshd.client.session.ClientSessionImpl] (pool-18-thread-2) Server > version string: SSH-2.0-OpenSSH_6.9 > > 2015-11-06 19:17:13,667 WARN > [org.apache.sshd.client.session.ClientSessionImpl] (pool-18-thread-3) > Exception caught: java.lang.IllegalStateException: Unable to negotiate key > exchange for kex algorithms (client: diffie-hellman-group1-sha1 / server: > curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1) > > at > org.apache.sshd.common.session.AbstractSession.negotiate(AbstractSession.java:1098) > > at > org.apache.sshd.common.session.AbstractSession.doHandleMessage(AbstractSession.java:357) > > at > org.apache.sshd.common.session.AbstractSession.handleMessage(AbstractSession.java:295) > > at > org.apache.sshd.client.session.ClientSessionImpl.handleMessage(ClientSessionImpl.java:266) > > at > org.apache.sshd.common.session.AbstractSession.decode(AbstractSession.java:720) > > at > org.apache.sshd.common.session.AbstractSession.messageReceived(AbstractSession.java:277) > > at > org.apache.sshd.common.AbstractSessionIoHandler.messageReceived(AbstractSessionIoHandler.java:54) > > at > org.apache.sshd.common.io.nio2.Nio2Session$1.completed(Nio2Session.java:188) > > at > org.apache.sshd.common.io.nio2.Nio2Session$1.completed(Nio2Session.java:174) > > at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) > [rt.jar:1.7.0_79] > > at sun.nio.ch.Invoker$2.run(Invoker.java:218) [rt.jar:1.7.0_79] > > at > sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) > [rt.jar:1.7.0_79] > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [rt.jar:1.7.0_79] > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [rt.jar:1.7.0_79] > > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79] > > ==> /var/log/ovirt-engine/engine.log <== > > 2015-11-06 19:17:13,672 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] > (ajp--127.0.0.1-8702-9) [65094082] Failed to establish session with host > hosted_engine_2: java.io.IOException: SSH session closed during connection > &#39;root(a)ophelia.doubledog.org&#39; > > at > org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:309) > [uutils.jar:] > > at > org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) > [bll.jar:] > > at > org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:465) > [bll.jar:] > > at > org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:364) > [bll.jar:] > > at > org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:768) > [bll.jar:] > > at > org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) > [bll.jar:] > > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:435) > [bll.jar:] > > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:416) > [bll.jar:] > > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:374) > [bll.jar:] > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [rt.jar:1.7.0_79] > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > [rt.jar:1.7.0_79] > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.7.0_79] > > at java.lang.reflect.Method.invoke(Method.java:606) > [rt.jar:1.7.0_79] > > at > org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) > [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) > [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) > [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) > [bll.jar:] > > at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) > [:1.7.0_79] > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.7.0_79] > > at java.lang.reflect.Method.invoke(Method.java:606) > [rt.jar:1.7.0_79] > > at > org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) > [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) > [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) > [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) > [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) > [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > > at > org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) > [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > > at > org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runAction(Unknown > Source) [common.jar:] > > at > org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:264) > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:243) > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:120) > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:110) > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:129) > > at > org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:100) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [rt.jar:1.7.0_79] > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > [rt.jar:1.7.0_79] > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.7.0_79] > > at java.lang.reflect.Method.invoke(Method.java:606) > [rt.jar:1.7.0_79] > > at > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:155) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:257) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:222) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:211) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:525) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:502) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:69) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:39) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:74) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:113) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:90) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:73) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.EngineSessionTokenAuthenticationFilter.doFilter(EngineSessionTokenAuthenticationFilter.java:31) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:32) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:110) > > at > org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:101) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > > at > org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505) > > at > org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445) > > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79] > > 2015-11-06 19:17:13,708 WARN [org.ovirt.engine.core.bll.AddVdsCommand] > (ajp--127.0.0.1-8702-9) [65094082] CanDoAction of action AddVds failed for > user admin@internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server > ophelia.doubledog.org,VDS_CANNOT_CONNECT_TO_SERVER > > 2015-11-06 19:17:13,728 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > (ajp--127.0.0.1-8702-9) Operation Failed: [Cannot add Host. Connecting to > host via SSH has failed, verify that the host is reachable (IP address, > routable address etc.) You may refer to the engine.log file for further > details.] > > ==> /var/log/ovirt-engine/server.log <== > > 2015-11-06 19:17:13,729 WARN > [org.jboss.resteasy.core.SynchronousDispatcher] (ajp--127.0.0.1-8702-9) > failed to execute: > org.ovirt.engine.api.restapi.resource.BaseBackendResource$WebFaultException > > at > org.ovirt.engine.api.restapi.resource.BaseBackendResource.handleError(BaseBackendResource.java:255) > [restapi-jaxrs.jar:] > > at > org.ovirt.engine.api.restapi.resource.BaseBackendResource.handleError(BaseBackendResource.java:230) > [restapi-jaxrs.jar:] > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:245) > [restapi-jaxrs.jar:] > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:120) > [restapi-jaxrs.jar:] > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:110) > [restapi-jaxrs.jar:] > > at > org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:129) > [restapi-jaxrs.jar:] > > at > org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:100) > [restapi-jaxrs.jar:] > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [rt.jar:1.7.0_79] > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > [rt.jar:1.7.0_79] > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.7.0_79] > > at java.lang.reflect.Method.invoke(Method.java:606) > [rt.jar:1.7.0_79] > > at > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:155) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:257) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:222) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:211) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:525) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:502) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) > [resteasy-jaxrs-2.3.2.Final.jar:] > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:69) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:39) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:74) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:113) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:90) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:73) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.EngineSessionTokenAuthenticationFilter.doFilter(EngineSessionTokenAuthenticationFilter.java:31) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:32) > [aaa.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:110) > [interface-common-jaxrs.jar:] > > at > org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:101) > [interface-common-jaxrs.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > [jbossweb-7.0.13.Final.jar:] > > at > org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > [jboss-as-web-7.1.1.Final.jar:7.1.1.Final] > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > [jbossweb-7.0.13.Final.jar:] > > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > [jbossweb-7.0.13.Final.jar:] > > at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445) > [jbossweb-7.0.13.Final.jar:] > > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > [jbossweb-7.0.13.Final.jar:] > > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79] > > > > I don't understand the ssh failure. From enceladus-f20, I can ssh as root > into ophelia-f22 and vice versa. I really don't care if it's all Fedora or > CentOS or a mix, my primary objective is to just be on 3.6... somehow. The failure is in the engine vm - it fails to ssh back to the host it's running on. Most probably due to name resolution and not due to network access.