[ovirt-users] Moving a Hosted Engine from Fedora 20 to CentOS 7

John Florian jflorian at doubledog.org
Mon Nov 9 23:25:13 UTC 2015 

On 11/08/2015 04:16 AM, Yedidyah Bar David wrote:
> On Sat, Nov 7, 2015 at 2:38 AM, John Florian <jflorian at doubledog.org> wrote:
>> On 10/29/2015 05:49 AM, Roy Golan wrote:
>>
>>
>>
>> On Thu, Oct 29, 2015 at 11:39 AM, Roy Golan <rgolan at redhat.com> wrote:
>>>
>>>
>>> On Wed, Oct 28, 2015 at 10:23 PM, John Florian <jflorian at doubledog.org>
>>> wrote:
>>>> Can somebody please point me to documentation or describe how I should
>>>> proceed with this task?  I see lots of pages for moving from a physical
>>>> engine to a VM and vice-versa but am having no luck finding how to go
>>>> about building a new HE to obsolete my original.
>>>>
>>> using ovirt-hosted-engine-setup you can choose a setup without using the
>>> appliance. So you can scratch install your VM
>>
>>
>> BTW the ovirt-engine-appliacnce we build [1] is Centos based. Seems like
>> perfect candidate.
>>
>> #install the appliance
>> yum install ovirt-engine-appliance
>>
>> #and then run the setup
>> ovirt-hosted-engine-setup
>>
>> choose "disk" in this stage
>>
>> Please specify the device to boot the VM from (cdrom, disk, pxe) [cdrom]:
>> disk
>>
>> it will suggest the downloaded appliance automatically
>>
>> See this wiki for reference
>> http://www.ovirt.org/Features/HEApplianceFlow#Testing
>>
>>
>>
>> I'm afraid I'm lost here.  Here's a map of my setup:
>>
>> oVirt 3.5.5 hosted engine is named enceladus-f20 (on Fedora 20)
>> I have one oVirt 3.5.5 Host named oberon-f20 (also on Fedora 20)
>> I previously had one other oVirt 3.5.5 Host named ophelia-f20
>>
>> I took opehlia down, installed CentOS 7 on it and attempted a "hosted-engine
>> --deploy".  I can't remember if that was 3.5.5 or 3.6, but I could not add
>> it to my cluster.  From what I could gather enceladus-f20 provided an
>> emulation type of pc1.0 while ophelia-c7 didn't seem to have anything that
>> matched, the closest being just "pc".  That looked hopelessly complicated to
>> resolve so I thought I'd try again, but this time putting F22 on the ophelia
>> and doing a 3.6 HE deploy saying yes to the redeploy prompt.  This time I
>> was met with:
>>
>>           Checking for oVirt-Engine status at enceladus-f20.doubledog.org...
>>
>> [ INFO  ] Engine replied: DB Up!Welcome to Health Status!
>>
>> [ ERROR ] Cannot automatically add the host to cluster Default: Cannot add
>> Host. Connecting to host via SSH has failed, verify that the host is
>> reachable (IP address, routable address etc.) You may refer to the
>> engine.log file for further details.
>>
>>
>>
>>           Please check Engine VM configuration.
>>
>> On enceladus-f20, I see:
>>
>> ==> /var/log/ovirt-engine/engine.log <==
>>
>> 2015-11-06 19:17:12,085 INFO
>> [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp--127.0.0.1-8702-9)
>> Running command: LoginUserCommand internal: false.
>>
>> 2015-11-06 19:17:12,128 INFO
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> (ajp--127.0.0.1-8702-9) Correlation ID: null, Call Stack: null, Custom Event
>> ID: -1, Message: User admin at internal logged in.
>>
>> ==> /var/log/ovirt-engine/server.log <==
>>
>> 2015-11-06 19:17:13,654 INFO
>> [org.apache.sshd.client.session.ClientSessionImpl] (pool-18-thread-1) Client
>> session created
>>
>> 2015-11-06 19:17:13,663 INFO
>> [org.apache.sshd.client.session.ClientSessionImpl] (pool-18-thread-2) Server
>> version string: SSH-2.0-OpenSSH_6.9
>>
>> 2015-11-06 19:17:13,667 WARN
>> [org.apache.sshd.client.session.ClientSessionImpl] (pool-18-thread-3)
>> Exception caught: java.lang.IllegalStateException: Unable to negotiate key
>> exchange for kex algorithms (client: diffie-hellman-group1-sha1 / server:
>> curve25519-sha256 at libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1)
>>
>>         at
>> org.apache.sshd.common.session.AbstractSession.negotiate(AbstractSession.java:1098)
>>
>>         at
>> org.apache.sshd.common.session.AbstractSession.doHandleMessage(AbstractSession.java:357)
>>
>>         at
>> org.apache.sshd.common.session.AbstractSession.handleMessage(AbstractSession.java:295)
>>
>>         at
>> org.apache.sshd.client.session.ClientSessionImpl.handleMessage(ClientSessionImpl.java:266)
>>
>>         at
>> org.apache.sshd.common.session.AbstractSession.decode(AbstractSession.java:720)
>>
>>         at
>> org.apache.sshd.common.session.AbstractSession.messageReceived(AbstractSession.java:277)
>>
>>         at
>> org.apache.sshd.common.AbstractSessionIoHandler.messageReceived(AbstractSessionIoHandler.java:54)
>>
>>         at
>> org.apache.sshd.common.io.nio2.Nio2Session$1.completed(Nio2Session.java:188)
>>
>>         at
>> org.apache.sshd.common.io.nio2.Nio2Session$1.completed(Nio2Session.java:174)
>>
>>         at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126)
>> [rt.jar:1.7.0_79]
>>
>>         at sun.nio.ch.Invoker$2.run(Invoker.java:218) [rt.jar:1.7.0_79]
>>
>>         at
>> sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> [rt.jar:1.7.0_79]
>>
>>         at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]
>>
>> ==> /var/log/ovirt-engine/engine.log <==
>>
>> 2015-11-06 19:17:13,672 ERROR [org.ovirt.engine.core.bll.AddVdsCommand]
>> (ajp--127.0.0.1-8702-9) [65094082] Failed to establish session with host
>> hosted_engine_2: java.io.IOException: SSH session closed during connection
>> 'root at ophelia.doubledog.org'
>>
>>         at
>> org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:309)
>> [uutils.jar:]
>>
>>         at
>> org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59)
>> [bll.jar:]
>>
>>         at
>> org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:465)
>> [bll.jar:]
>>
>>         at
>> org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:364)
>> [bll.jar:]
>>
>>         at
>> org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:768)
>> [bll.jar:]
>>
>>         at
>> org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347)
>> [bll.jar:]
>>
>>         at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:435)
>> [bll.jar:]
>>
>>         at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:416)
>> [bll.jar:]
>>
>>         at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:374)
>> [bll.jar:]
>>
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> [rt.jar:1.7.0_79]
>>
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114)
>> [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125)
>> [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135)
>> [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13)
>> [bll.jar:]
>>
>>         at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
>> [:1.7.0_79]
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> [rt.jar:1.7.0_79]
>>
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82)
>> [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
>> [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211)
>> [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363)
>> [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194)
>> [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
>> [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59)
>> [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>> [jboss-invocation-1.1.1.Final.jar:1.1.1.Final]
>>
>>         at
>> org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72)
>> [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runAction(Unknown
>> Source) [common.jar:]
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:264)
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:243)
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:120)
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:110)
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:129)
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:100)
>>
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> [rt.jar:1.7.0_79]
>>
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:155)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:257)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:222)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:211)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:525)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:502)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>> [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:69)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:39)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:74)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:113)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:90)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:73)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.EngineSessionTokenAuthenticationFilter.doFilter(EngineSessionTokenAuthenticationFilter.java:31)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:32)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:110)
>>
>>         at
>> org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:101)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>>
>>         at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
>>
>>         at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
>>
>>         at
>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489)
>>
>>         at
>> org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
>>
>>         at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
>>
>>         at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>
>>         at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
>>
>>         at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>
>>         at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
>>
>>         at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505)
>>
>>         at
>> org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445)
>>
>>         at
>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
>>
>>         at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]
>>
>> 2015-11-06 19:17:13,708 WARN  [org.ovirt.engine.core.bll.AddVdsCommand]
>> (ajp--127.0.0.1-8702-9) [65094082] CanDoAction of action AddVds failed for
>> user admin at internal. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server
>> ophelia.doubledog.org,VDS_CANNOT_CONNECT_TO_SERVER
>>
>> 2015-11-06 19:17:13,728 ERROR
>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
>> (ajp--127.0.0.1-8702-9) Operation Failed: [Cannot add Host. Connecting to
>> host via SSH has failed, verify that the host is reachable (IP address,
>> routable address etc.) You may refer to the engine.log file for further
>> details.]
>>
>> ==> /var/log/ovirt-engine/server.log <==
>>
>> 2015-11-06 19:17:13,729 WARN
>> [org.jboss.resteasy.core.SynchronousDispatcher] (ajp--127.0.0.1-8702-9)
>> failed to execute:
>> org.ovirt.engine.api.restapi.resource.BaseBackendResource$WebFaultException
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.BaseBackendResource.handleError(BaseBackendResource.java:255)
>> [restapi-jaxrs.jar:]
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.BaseBackendResource.handleError(BaseBackendResource.java:230)
>> [restapi-jaxrs.jar:]
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:245)
>> [restapi-jaxrs.jar:]
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:120)
>> [restapi-jaxrs.jar:]
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:110)
>> [restapi-jaxrs.jar:]
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:129)
>> [restapi-jaxrs.jar:]
>>
>>         at
>> org.ovirt.engine.api.restapi.resource.BackendHostsResource.add(BackendHostsResource.java:100)
>> [restapi-jaxrs.jar:]
>>
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> [rt.jar:1.7.0_79]
>>
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>> [rt.jar:1.7.0_79]
>>
>>         at
>> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:155)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:257)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:222)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:211)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:525)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:502)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at
>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
>> [resteasy-jaxrs-2.3.2.Final.jar:]
>>
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>> [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:69)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:39)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:74)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:113)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:90)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:73)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.EngineSessionTokenAuthenticationFilter.doFilter(EngineSessionTokenAuthenticationFilter.java:31)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:32)
>> [aaa.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:110)
>> [interface-common-jaxrs.jar:]
>>
>>         at
>> org.ovirt.engine.api.common.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:101)
>> [interface-common-jaxrs.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
>> [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
>>
>>         at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at
>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
>> [jbossweb-7.0.13.Final.jar:]
>>
>>         at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]
>>
>>
>>
>> I don't understand the ssh failure.  From enceladus-f20, I can ssh as root
>> into ophelia-f22 and vice versa.  I really don't care if it's all Fedora or
>> CentOS or a mix, my primary objective is to just be on 3.6... somehow.
> The failure is in the engine vm - it fails to ssh back to the host it's running
> on. Most probably due to name resolution and not due to network
> access.

I don't think it has anything to do with name resolution either.  I
believe the telltale clue is this bit...

2015-11-09 18:22:31,738 WARN 
[org.apache.sshd.client.session.ClientSessionImpl] (pool-20-thread-3)
Exception caught: java.lang.IllegalStateException: Unable to negotiate
key exchange for kex algorithms (client: diffie-hellman-group1-sha1 /
server:
curve25519-sha256 at libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1)

As mentioned, I can ssh from my engine to the host just fine.  It
appears that the Java-based ssh client however cannot.

-- 
John Florian

More information about the Users mailing list