[ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 28-02-2019 ] [ 002_bootstrap.resize_and_refresh_storage_domain ]

Hi, The test failed due to not finding master storage domain I am seeing a build of engine that passed so its possible that this is related to the random failures we have been having on other tests but since the change reported is storage related, can you please review the logs to see if anything happened to the master domain while trying to resize? CQ reported this change as root cause: https://gerrit.ovirt.org/#/c/98015/6 - blockSD: Move legacy block size constants to sd build failure: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/ logs: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/arti... errors in log: 2019-02-27 17:35:02,222-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM la go-basic-suite-master-host-1 command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58, msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9' 2019-02-27 17:35:02,222-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker. ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status [code=304, message=Cannot find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58, msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5 a9']]' 2019-02-27 17:35:02,222-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] HostName = lago-basic-suite-master-host-1 2019-02-27 17:35:02,223-05 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command 'ConnectStoragePoolVDSCommand(HostName = lago-basic-suite-master-host-1, ConnectStoragePoolVDSCommandParameters:{hostId='1ac0eda2-0980-40df-b029-b262680696a4', vdsId='1ac0eda2-0980-40df-b029-b262680696a4', storagePoolId='1c79b83e-32d6-43e0-bc61-2955e42c3a58', masterVersion='1'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58, msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9' 2019-02-27 17:35:02,223-05 DEBUG [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Exception: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58, msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9' at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:50) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedConnectProxyReturnValue(ConnectStoragePoolVDSCommand.java:48) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedProxyReturnValue(ConnectStoragePoolVDSCommand.java:36) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.executeVdsBrokerCommand(ConnectStoragePoolVDSCommand.java:32) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65) [vdsbroker.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:396) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown Source) [vdsbroker.jar:] at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source) [:1.8.0_191] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_191] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_191] at org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) [:1.8.0_191] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_191] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_191] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown Source) [vdsbroker.jar:] at org.ovirt.engine.core.bll.storage.pool.RefreshPoolSingleAsyncOperation.execute(RefreshPoolSingleAsyncOperation.java:59) [bll.jar:] at org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:43) [utils.jar:] at org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:32) [utils.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:117) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_191] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_191] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:]