<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">What does gluster peer status say on each host?<br class=""><div class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 13px; background-color: rgb(255, 255, 255);" class=""><br class="Apple-interchange-newline">T</span><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 13px; background-color: rgb(255, 255, 255);" class="">om Gamull</span></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 13px;" class="">Red Hat | Solutions Architect</div><div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 13px;" class="">(404) 494-0280 | @magicalyak</div></div>
</div>
<br class=""><div style=""><blockquote type="cite" class=""><div class="">On Oct 26, 2016, at 8:55 PM, Tom Gamull <<a href="mailto:tgamull@redhat.com" class="">tgamull@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=us-ascii" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">This is pointing to network connectivity issues</div><div class=""><br class=""></div><div class="">You can look at logs under /var/logs/ovirt-*</div><div class=""><br class=""></div><div class="">Check your dns and make sure the hosts can connect to each other. You can restart the host, mark it as restarted in the manager and then activate it.</div><div class=""><br class=""></div><div class="">Worst case, you can remove it using hosted-engine commands</div><div class=""><br class=""></div><div class="">Tom</div>
<br class=""><div class=""><blockquote type="cite" class=""><div class="">On Oct 26, 2016, at 8:53 PM, Thing <<a href="mailto:thing.thing@gmail.com" class="">thing.thing@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">from the engine log I see,<br class=""><br class="">========<br class="">2016-10-27 13:45:24,595 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler2) [3877c2ef] START, GetGlusterHostUUIDVDSCommand(HostName = glusterp1, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='260c0a92-2856-4cd6-a784-01ac95fc41d5'}), log id: 51f83f78<br class="">2016-10-27 13:46:15,780 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler3) [] No up server in cluster<br class="">2016-10-27 13:47:12,326 INFO [org.ovirt.engine.core.bll.gluster.GlusterSnapshotSyncJob] (DefaultQuartzScheduler10) [76412dcc] No UP server found in cluster 'Default' for snapshot monitoring<br class="">2016-10-27 13:47:12,329 INFO [org.ovirt.engine.core.bll.gluster.GlusterSnapshotSyncJob] (DefaultQuartzScheduler10) [76412dcc] No UP server found in cluster 'Default' for snapshot configurations monitoring<br class="">2016-10-27 13:47:15,783 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler4) [27554ab4] No up server in cluster<br class="">2016-10-27 13:48:15,785 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler4) [27554ab4] No up server in cluster<br class="">2016-10-27 13:48:24,879 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler2) [3877c2ef] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM glusterp1 command failed: Message timeout which can be caused by communication issues<br class="">2016-10-27 13:48:24,879 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler2) [3877c2ef] Command 'GetGlusterHostUUIDVDSCommand(HostName = glusterp1, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='260c0a92-2856-4cd6-a784-01ac95fc41d5'})' execution failed: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues<br class="">2016-10-27 13:48:24,879 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler2) [3877c2ef] FINISH, GetGlusterHostUUIDVDSCommand, log id: 51f83f78<br class="">2016-10-27 13:48:24,879 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler2) [3877c2ef] Command 'org.ovirt.engine.core.bll.InitVdsOnUpCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues (Failed with error VDS_NETWORK_ERROR and code 5022)<br class="">2016-10-27 13:48:24,887 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler2) [3877c2ef] Correlation ID: 3877c2ef, Call Stack: null, Custom Event ID: -1, Message: Failed to fetch gluster peer list from server glusterp1 on Cluster Default.<br class="">2016-10-27 13:48:24,908 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler2) [6083469d] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 260c0a92-2856-4cd6-a784-01ac95fc41d5 Type: VDS<br class="">2016-10-27 13:48:24,959 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-8-thread-42) [3877c2ef] Server failed to respond, vds_id='260c0a92-2856-4cd6-a784-01ac95fc41d5', vds_name='glusterp1', vm_count=0, spm_status='None', non-responsive_timeout (seconds)=60, error: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues<br class="">2016-10-27 13:48:25,003 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-42) [3877c2ef] Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues<br class=""> at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:188)<br class=""> at org.ovirt.engine.core.vdsbroker.gluster.AbstractGlusterBrokerCommand.proceedProxyReturnValue(AbstractGlusterBrokerCommand.java:122)<br class=""> at org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand.executeVdsBrokerCommand(GetGlusterHostUUIDVDSCommand.java:19)<br class=""> at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:110)<br class=""> at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:73)<br class=""> at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)<br class=""> at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:451)<br class=""> at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33)<br class=""> at org.ovirt.engine.core.bll.CommandBase.runVdsCommand(CommandBase.java:2171)<br class=""> at org.ovirt.engine.core.bll.storage.StorageHandlingCommandBase.runVdsCommand(StorageHandlingCommandBase.java:657)<br class=""> at org.ovirt.engine.core.bll.InitVdsOnUpCommand.initGlusterHost(InitVdsOnUpCommand.java:397)<br class=""> at org.ovirt.engine.core.bll.InitVdsOnUpCommand.executeCommand(InitVdsOnUpCommand.java:149)<br class=""> at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1305)<br class=""> at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1447)<br class=""> at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2075)<br class=""> at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:166)<br class=""> at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:105)<br class=""> at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1490)<br class=""> at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:398)<br class=""> at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:493)<br class=""> at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:475)<br class=""> at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:419)<br class=""> at sun.reflect.GeneratedMethodAccessor225.invoke(Unknown Source)<br class=""> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br class=""> at java.lang.reflect.Method.invoke(Method.java:498)<br class=""> at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)<br class=""> at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70)<br class=""> at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80)<br class=""> at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93)<br class=""> at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437)<br class=""> at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)<br class=""> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)<br class=""> at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:263)<br class=""> at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374)<br class=""> at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:243)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)<br class=""> at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)<br class=""> at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356)<br class=""> at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)<br class=""> at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195)<br class=""> at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)<br class=""> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340)<br class=""> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)<br class=""> at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)<br class=""> at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view3.runInternalAction(Unknown Source)<br class=""> at sun.reflect.GeneratedMethodAccessor226.invoke(Unknown Source)<br class=""> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br class=""> at java.lang.reflect.Method.invoke(Method.java:498)<br class=""> at org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:436)<br class=""> at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:127)<br class=""> at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)<br class=""> at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100)<br class=""> at org.ovirt.engine.core.bll.BackendCommandObjectsHandler$BackendInternal$BackendLocal$2049259618$Proxy$_$$_Weld$EnterpriseProxy$.runInternalAction(Unknown Source)<br class=""> at org.ovirt.engine.core.bll.VdsEventListener.vdsUpEvent(VdsEventListener.java:308)<br class=""> at org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refresh(HostMonitoring.java:93)<br class=""> at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:238)<br class=""> at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)<br class=""> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br class=""> at java.lang.reflect.Method.invoke(Method.java:498)<br class=""> at org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(JobWrapper.java:77)<br class=""> at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:51)<br class=""> at org.quartz.core.JobRunShell.run(JobRunShell.java:213)<br class=""> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<br class=""> at java.util.concurrent.FutureTask.run(FutureTask.java:266)<br class=""> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)<br class=""> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)<br class=""> at java.lang.Thread.run(Thread.java:745)<br class="">, Custom Event ID: -1, Message: Host glusterp1 is non responsive.<br class="">2016-10-27 13:48:25,019 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-29) [3877c2ef] ResourceManager::vdsNotResponding entered for Host '260c0a92-2856-4cd6-a784-01ac95fc41d5', '192.168.1.31'<br class="">2016-10-27 13:48:25,054 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-29) [43df6659] Lock Acquired to object 'EngineLock:{exclusiveLocks='[260c0a92-2856-4cd6-a784-01ac95fc41d5=<VDS_FENCE, POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS>]', sharedLocks='null'}'<br class="">2016-10-27 13:48:25,069 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-29) [43df6659] Running command: VdsNotRespondingTreatmentCommand internal: true. Entities affected : ID: 260c0a92-2856-4cd6-a784-01ac95fc41d5 Type: VDS<br class="">2016-10-27 13:48:25,079 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-29) [43df6659] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 260c0a92-2856-4cd6-a784-01ac95fc41d5 Type: VDS<br class="">2016-10-27 13:48:25,081 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-29) [43df6659] Opening SSH Soft Fencing session on host '192.168.1.31'<br class="">2016-10-27 13:48:25,189 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-29) [43df6659] Executing SSH Soft Fencing command on host '192.168.1.31'<br class="">2016-10-27 13:48:27,914 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to /<a href="http://192.168.1.31/" class="">192.168.1.31</a><br class="">2016-10-27 13:48:29,379 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler7) [] START, GetHardwareInfoVDSCommand(HostName = glusterp1, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='260c0a92-2856-4cd6-a784-01ac95fc41d5', vds='Host[glusterp1,260c0a92-2856-4cd6-a784-01ac95fc41d5]'}), log id: 46d1a470<br class="">2016-10-27 13:48:30,383 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler7) [] FINISH, GetHardwareInfoVDSCommand, log id: 46d1a470<br class="">2016-10-27 13:48:30,449 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler7) [a8ef973] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 260c0a92-2856-4cd6-a784-01ac95fc41d5 Type: VDS<br class="">2016-10-27 13:48:30,464 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler7) [1c434304] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 58098515-0126-0151-02eb-0000000003cb Type: StoragePool<br class="">2016-10-27 13:48:30,469 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterHostUUIDVDSCommand] (DefaultQuartzScheduler7) [1c434304] START, GetGlusterHostUUIDVDSCommand(HostName = glusterp1, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='260c0a92-2856-4cd6-a784-01ac95fc41d5'}), log id: 4ca089d8<br class="">2016-10-27 13:48:37,296 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-29) [43df6659] Lock freed to object 'EngineLock:{exclusiveLocks='[260c0a92-2856-4cd6-a784-01ac95fc41d5=<VDS_FENCE, POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS>]', sharedLocks='null'}'<br class="">2016-10-27 13:48:37,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-29) [43df6659] Correlation ID: 43df6659, Job ID: 091c3b5f-d8e6-4200-b8fb-39d0d8b5b330, Call Stack: null, Custom Event ID: -1, Message: Host glusterp1 is rebooting.<br class="">2016-10-27 13:48:37,336 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-29) [43df6659] Trying to release exclusive lock which does not exist, lock key: '260c0a92-2856-4cd6-a784-01ac95fc41d5VDS_FENCE'<br class="">2016-10-27 13:48:37,336 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-29) [43df6659] Lock freed to object 'EngineLock:{exclusiveLocks='[260c0a92-2856-4cd6-a784-01ac95fc41d5=<VDS_FENCE, POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS>]', sharedLocks='null'}'<br class="">2016-10-27 13:49:15,787 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler8) [beed7f3] No up server in cluster<br class="">2016-10-27 13:50:15,790 INFO [org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService] (DefaultQuartzScheduler1) [2d374249] No up server in cluster<br class="">=======<br class=""><br class=""><br class=""></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On 27 October 2016 at 13:45, Thing <span dir="ltr" class=""><<a href="mailto:thing.thing@gmail.com" target="_blank" class="">thing.thing@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class=""><div class=""><div class="">While trying to figure out how to deploy storage I put 1 host into maintenance mode, trying to re-activate it and its failed.<br class=""></div><br class=""></div>It seems to be stuck as neither in activated nor maintenance, so how would I go about fixing this?<br class=""><br class="">So what log(s) would this be written to? <br class=""></div>
</blockquote></div><br class=""></div>
_______________________________________________<br class="">Users mailing list<br class=""><a href="mailto:Users@ovirt.org" class="">Users@ovirt.org</a><br class=""><a href="http://lists.ovirt.org/mailman/listinfo/users" class="">http://lists.ovirt.org/mailman/listinfo/users</a><br class=""></div></blockquote></div><br class=""></div></div></blockquote></div><br class=""></body></html>