[ovirt-users] No hosted engine SD after upgrading to 3.6.1 RC4 from 3.6.0

Hello, The Hosted Engine import did not work upon upgrading from 3.6.0 to 3.6.1 RC4. I had upgraded in an attempt to import the Hosted Engine storage domain and VM. Here’s my setup: * 2x Dell R630s with SAS HBAs * 1x Dell Compellent SAS SAN * Each R630 is connected to the SAS SAN via the SAS HBAs. * Storage domains are attached as FC (even though they’re SAS) * Hosted-engine deployed from 3.6.0 using image in yum repo Packages installed on engine: ovirt-engine-setup-plugin-websocket-proxy-3.6.1.3-1.el7.centos.noarch ovirt-engine-restapi-3.6.1.3-1.el7.centos.noarch ovirt-engine-setup-plugin-ovirt-engine-common-3.6.1.3-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch ovirt-engine-webadmin-portal-3.6.1.3-1.el7.centos.noarch ovirt-engine-extension-aaa-ldap-setup-1.1.2-1.el7.centos.noarch ovirt-image-uploader-3.6.0-1.el7.centos.noarch ovirt-engine-extension-aaa-jdbc-1.0.4-1.el7.noarch ovirt-release36-002-2.noarch ovirt-engine-websocket-proxy-3.6.1.3-1.el7.centos.noarch ovirt-engine-wildfly-overlay-8.0.4-1.el7.noarch ovirt-engine-tools-3.6.1.3-1.el7.centos.noarch ovirt-engine-extensions-api-impl-3.6.1.3-1.el7.centos.noarch ovirt-vmconsole-1.0.0-1.el7.centos.noarch ebay-cors-filter-1.0.1-0.1.ovirt.el7.noarch ovirt-guest-agent-common-1.0.11-1.el7.noarch ovirt-engine-wildfly-8.2.1-1.el7.x86_64 ovirt-engine-backend-3.6.1.3-1.el7.centos.noarch ovirt-engine-setup-base-3.6.1.3-1.el7.centos.noarch ovirt-engine-setup-plugin-vmconsole-proxy-helper-3.6.1.3-1.el7.centos.noarch ovirt-engine-vmconsole-proxy-helper-3.6.1.3-1.el7.centos.noarch ovirt-engine-cli-3.6.0.2-1.el7.centos.noarch ovirt-engine-setup-3.6.1.3-1.el7.centos.noarch ovirt-engine-dbscripts-3.6.1.3-1.el7.centos.noarch ovirt-engine-extension-aaa-ldap-1.1.2-1.el7.centos.noarch ovirt-iso-uploader-3.6.0-1.el7.centos.noarch ovirt-engine-3.6.1.3-1.el7.centos.noarch ovirt-vmconsole-proxy-1.0.0-1.el7.centos.noarch ovirt-engine-setup-plugin-ovirt-engine-3.6.1.3-1.el7.centos.noarch ovirt-host-deploy-java-1.4.1-1.el7.centos.noarch ovirt-engine-lib-3.6.1.3-1.el7.centos.noarch ovirt-engine-userportal-3.6.1.3-1.el7.centos.noarch ovirt-host-deploy-1.4.1-1.el7.centos.noarch I found this in engine.log. Please let me know what additional details I can provide. 2015-12-16 05:25:33,149 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-12) [fa1efd1] START, FullListVDSCommand(HostName = , FullListVDSCommandParameters:{runAsync='true', hostId='2fe6c27b-9346-4678-8cd3-c9d367ec447f', vds='Host[,2fe6c27b-9346-4678-8cd3-c9d367ec447f]', vmIds='[347620e8-79f1-4672-851d-2ad6103c0854]'}), log id: 7759b45b 2015-12-16 05:25:34,155 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (DefaultQuartzScheduler_Worker-12) [fa1efd1] FINISH, FullListVDSCommand, return: [{status=Up, nicModel=rtl8139,pv, emulatedMachine=pc, guestDiskMapping={67805d76-66e4-4256-8={name=/dev/vda}, QEMU_DVD-ROM={name=/dev/sr0}}, vmId=347620e8-79f1-4672-851d-2ad6103c0854, pid=6377, devices=[Ljava.lang.Object;@76b3df47, smp=4, vmType=kvm, displayIp=0, display=vnc, displaySecurePort=-1, memSize=8192, displayPort=5900, cpuType=SandyBridge, spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, statusTime=4298688650, vmName=HostedEngine, clientIp=, pauseCode=NOERR}], log id: 7759b45b 2015-12-16 05:25:34,162 INFO [org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] (org.ovirt.thread.pool-8-thread-40) [1b9a02d1] START, GetExistingStorageDomainListQuery(GetExistingStorageDomainListParameters:{refresh='true', filtered='false'}), log id: d197cac 2015-12-16 05:25:34,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (org.ovirt.thread.pool-8-thread-40) [1b9a02d1] START, HSMGetStorageDomainsListVDSCommand(HostName = ov-101, HSMGetStorageDomainsListVDSCommandParameters:{runAsync='true', hostId='2fe6c27b-9346-4678-8cd3-c9d367ec447f', storagePoolId='00000000-0000-0000-0000-000000000000', storageType='null', storageDomainType='Data', path='null'}), log id: 7bc578c4 2015-12-16 05:25:35,422 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (org.ovirt.thread.pool-8-thread-40) [1b9a02d1] FINISH, HSMGetStorageDomainsListVDSCommand, return: [8c1981b2-ac04-433d-80f4-6cfd9023f935, 1fb79d91-b245-4447-91e0-e57671152a8c], log id: 7bc578c4 2015-12-16 05:25:35,431 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (org.ovirt.thread.pool-8-thread-40) [1b9a02d1] START, HSMGetStorageDomainInfoVDSCommand(HostName = ov-101, HSMGetStorageDomainInfoVDSCommandParameters:{runAsync='true', hostId='2fe6c27b-9346-4678-8cd3-c9d367ec447f', storageDomainId='8c1981b2-ac04-433d-80f4-6cfd9023f935'}), log id: f35ad27 2015-12-16 05:25:37,622 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (org.ovirt.thread.pool-8-thread-40) [1b9a02d1] FINISH, HSMGetStorageDomainInfoVDSCommand, return: <StorageDomainStatic:{name='hosted_storage', id='8c1981b2-ac04-433d-80f4-6cfd9023f935'}, null>, log id: f35ad27 2015-12-16 05:25:37,622 INFO [org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] (org.ovirt.thread.pool-8-thread-40) [1b9a02d1] FINISH, GetExistingStorageDomainListQuery, log id: d197cac 2015-12-16 05:25:37,622 INFO [org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-8-thread-40) [] Running command: ImportHostedEngineStorageDomainCommand internal: true. 2015-12-16 05:25:37,635 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (org.ovirt.thread.pool-8-thread-40) [] START, GetDeviceListVDSCommand(HostName = ov-101, GetDeviceListVDSCommandParameters:{runAsync='true', hostId='2fe6c27b-9346-4678-8cd3-c9d367ec447f', storageType='FCP', checkStatus='false', lunIds='null'}), log id: 367b5a8f 2015-12-16 05:25:38,918 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (org.ovirt.thread.pool-8-thread-40) [] FINISH, GetDeviceListVDSCommand, return: [LUNs:{id='36000d310031d3c000000000000000022', physicalVolumeId='9f71iB-EMMF-1Wvi-Ql9X-82v3-iS90-DjNJKn', volumeGroupId='YBYVbS-juhU-049c-JFyL-Yavx-GUoV-CkODEc', serial='SCOMPELNTCompellent_Vol_00031d3c-00000022', lunMapping='2', vendorId='COMPELNT', productId='Compellent Vol', lunConnections='[]', deviceSize='5120', pvSize='5119', vendorName='COMPELNT', pathsDictionary='[sdb=true, sde=true]', pathsCapacity='[sdb=5120, sde=5120]', lunType='FCP', status='Unknown', diskId='null', diskAlias='null', storageDomainId='null', storageDomainName='null'}, LUNs:{id='36000d310031d3c000000000000000021', physicalVolumeId='23ibJS-iHT2-2fmx-rsp4-cf5u-MK2G-KEMgUX', volumeGroupId='', serial='SCOMPELNTCompellent_Vol_00031d3c-00000021', lunMapping='4', vendorId='COMPELNT', productId='Compellent Vol', lunConnections='[]', deviceSize='500', pvSize='500', vendorName='COMPELNT', pathsDictionary='[sdc=true, sdf=true]', pathsCapacity='[sdc=500, sdf=500]', lunType='FCP', status='Unknown', diskId='null', diskAlias='null', storageDomainId='null', storageDomainName='null'}, LUNs:{id='36000d310031d3c00000000000000001e', physicalVolumeId='2lrROZ-Wx2a-0TbH-P2fh-87DA-lVBe-hesaKc', volumeGroupId='colhMa-P92x-BltB-3izj-YiE7-dkcW-6AkTZ2', serial='SCOMPELNTCompellent_Vol_00031d3c-0000001e', lunMapping='1', vendorId='COMPELNT', productId='Compellent Vol', lunConnections='[]', deviceSize='50', pvSize='49', vendorName='COMPELNT', pathsDictionary='[sda=true, sdd=true]', pathsCapacity='[sda=50, sdd=50]', lunType='FCP', status='Unknown', diskId='null', diskAlias='null', storageDomainId='null', storageDomainName='null'}], log id: 367b5a8f 2015-12-16 05:25:38,919 ERROR [org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-8-thread-40) [] Command 'org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand' failed: Index: 0, Size: 0 2015-12-16 05:25:38,919 ERROR [org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-8-thread-40) [] Exception: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:653) [rt.jar:1.8.0_65] at java.util.ArrayList.get(ArrayList.java:429) [rt.jar:1.8.0_65] at org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand.discoverBlockConnectionDetails(ImportHostedEngineStorageDomainCommand.java:133) [bll.jar:] at org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand.executeCommand(ImportHostedEngineStorageDomainCommand.java:97) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1215) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1359) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1983) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:149) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:118) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1396) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:378) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:475) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:457) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:404) [bll.jar:] at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source) [:1.8.0_65] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_65] at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_65] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final] at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final] at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final] at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:55) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) [wildfly-weld-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:260) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:399) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:243) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:95) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [wildfly-ejb3-8.2.1.Final.jar:8.2.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:326) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:448) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:326) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view3.runInternalAction(Unknown Source) [bll.jar:] at org.ovirt.engine.core.bll.HostedEngineImporter.getHEStorageDomain(HostedEngineImporter.java:171) [bll.jar:] at org.ovirt.engine.core.bll.HostedEngineImporter.doImport(HostedEngineImporter.java:80) [bll.jar:] at org.ovirt.engine.core.bll.VdsEventListener$12.run(VdsEventListener.java:591) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:92) [utils.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_65] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_65] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_65] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_65] 2015-12-16 05:25:38,923 ERROR [org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand] (org.ovirt.thread.pool-8-thread-40) [] Transaction rolled-back for command 'org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand'.