[ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 28-02-2019 ] [ 002_bootstrap.resize_and_refresh_storage_domain ]
4 a.m.
Hi,
The test failed due to not finding master storage domain
I am seeing a build of engine that passed so its possible that this is
related to the random failures we have been having on other tests but since
the change reported is storage related, can you please review the logs to
see if anything happened to the master domain while trying to resize?
CQ reported this change as root cause:
https://gerrit.ovirt.org/#/c/98015/6 - blockSD: Move legacy block size
constants to sd
build failure:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/
logs:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/arti...
errors in log:
2019-02-27 17:35:02,222-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] EVENT_ID:
VDS_BROKER_COMMAND_FAILURE(10,802), VDSM la
go-basic-suite-master-host-1 command ConnectStoragePoolVDS failed: Cannot
find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
2019-02-27 17:35:02,222-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command
'org.ovirt.engine.core.vdsbroker.vdsbroker.
ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn [status=Status
[code=304, message=Cannot find master domain:
u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5
a9']]'
2019-02-27 17:35:02,222-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] HostName =
lago-basic-suite-master-host-1
2019-02-27 17:35:02,223-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command
'ConnectStoragePoolVDSCommand(HostName = lago-basic-suite-master-host-1,
ConnectStoragePoolVDSCommandParameters:{hostId='1ac0eda2-0980-40df-b029-b262680696a4',
vdsId='1ac0eda2-0980-40df-b029-b262680696a4',
storagePoolId='1c79b83e-32d6-43e0-bc61-2955e42c3a58',
masterVersion='1'})'
execution failed: IRSGenericException: IRSErrorException:
IRSNoMasterDomainException: Cannot find master domain:
u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
2019-02-27 17:35:02,223-05 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Exception:
org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException:
IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot
find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:50)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedConnectProxyReturnValue(ConnectStoragePoolVDSCommand.java:48)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedProxyReturnValue(ConnectStoragePoolVDSCommand.java:36)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.executeVdsBrokerCommand(ConnectStoragePoolVDSCommand.java:32)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
[vdsbroker.jar:]
at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31)
[dal.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:396)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
Source) [vdsbroker.jar:]
at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
[:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_191]
at
org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
[common.jar:]
at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
[:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_191]
at
org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
Source) [vdsbroker.jar:]
at
org.ovirt.engine.core.bll.storage.pool.RefreshPoolSingleAsyncOperation.execute(RefreshPoolSingleAsyncOperation.java:59)
[bll.jar:]
at
org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:43)
[utils.jar:]
at
org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:32)
[utils.jar:]
at
org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:117)
[utils.jar:]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[rt.jar:1.8.0_191]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_191]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_191]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_191]
at
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]
5:12 a.m.
New subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [
28-02-2019 ] [ 002_bootstrap.resize_and_refresh_storage_domain ]
On Thu, Feb 28, 2019 at 12:01 PM Dafna Ron <dron(a)redhat.com> wrote:
Hi,
The test failed due to not finding master storage domain
I am seeing a build of engine that passed so its possible that this is
related to the random failures we have been having on other tests but since
the change reported is storage related, can you please review the logs to
see if anything happened to the master domain while trying to resize?
First the patch cannot affect the storage domain, this is just moving of
constant from one module to another, and verified by vdsm tests. If the
patch
was broken we would fail with a NameError in vdsm, accessing non-exisitng
variable.
I think it is a flaky test, since I ran many successful OST with this patch.
These run include the patch:
- https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4173/
- https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4172/
- https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4171/
There may be real issue behind this, but it is not related to the patch.
I think this is related to not deactivating lvs when putting storage domain
in maintenance.
We have this bug for the issue:
https://bugzilla.redhat.com/1163890
The issue is probably random since storage domain monitor is refreshed
every 5 minutes,
so the system correct itself automatically. It probably fails if the test
happened to start at the
wrong time, timing out before the system refreshed the storage.
If you see more of this issue, please file a new bug and we will handle it
when we have time.
In the meantime you can disable the test since it is not reliable.
Nir
CQ reported this change as root cause:
https://gerrit.ovirt.org/#/c/98015/6 - blockSD: Move legacy block size
constants to sd
build failure:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/
logs:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/arti...
errors in log:
2019-02-27 17:35:02,222-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] EVENT_ID:
VDS_BROKER_COMMAND_FAILURE(10,802), VDSM la
go-basic-suite-master-host-1 command ConnectStoragePoolVDS failed: Cannot
find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
2019-02-27 17:35:02,222-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command
'org.ovirt.engine.core.vdsbroker.vdsbroker.
ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn
[status=Status [code=304, message=Cannot find master domain:
u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5
a9']]'
2019-02-27 17:35:02,222-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] HostName =
lago-basic-suite-master-host-1
2019-02-27 17:35:02,223-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command
'ConnectStoragePoolVDSCommand(HostName = lago-basic-suite-master-host-1,
ConnectStoragePoolVDSCommandParameters:{hostId='1ac0eda2-0980-40df-b029-b262680696a4',
vdsId='1ac0eda2-0980-40df-b029-b262680696a4',
storagePoolId='1c79b83e-32d6-43e0-bc61-2955e42c3a58',
masterVersion='1'})'
execution failed: IRSGenericException: IRSErrorException:
IRSNoMasterDomainException: Cannot find master domain:
u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
2019-02-27 17:35:02,223-05 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Exception:
org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException:
IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot
find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
at
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:50)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedConnectProxyReturnValue(ConnectStoragePoolVDSCommand.java:48)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedProxyReturnValue(ConnectStoragePoolVDSCommand.java:36)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.executeVdsBrokerCommand(ConnectStoragePoolVDSCommand.java:32)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
[vdsbroker.jar:]
at
org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31)
[dal.jar:]
at
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:396)
[vdsbroker.jar:]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
Source) [vdsbroker.jar:]
at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
[:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_191]
at
org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
[common.jar:]
at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
[:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_191]
at
org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
[weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
at
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
Source) [vdsbroker.jar:]
at
org.ovirt.engine.core.bll.storage.pool.RefreshPoolSingleAsyncOperation.execute(RefreshPoolSingleAsyncOperation.java:59)
[bll.jar:]
at
org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:43)
[utils.jar:]
at
org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:32)
[utils.jar:]
at
org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:117)
[utils.jar:]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[rt.jar:1.8.0_191]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_191]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_191]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_191]
at
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]
7:21 a.m.
New subject: [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [
28-02-2019 ] [ 002_bootstrap.resize_and_refresh_storage_domain ]
Thanks Nir.
I will skip the test until the bug is fixed.
Thanks,
Dafna
On Thu, Feb 28, 2019 at 11:12 AM Nir Soffer <nsoffer(a)redhat.com> wrote:
On Thu, Feb 28, 2019 at 12:01 PM Dafna Ron <dron(a)redhat.com>
wrote:
> Hi,
>
> The test failed due to not finding master storage domain
> I am seeing a build of engine that passed so its possible that this is
> related to the random failures we have been having on other tests but since
> the change reported is storage related, can you please review the logs to
> see if anything happened to the master domain while trying to resize?
>
First the patch cannot affect the storage domain, this is just moving of
constant from one module to another, and verified by vdsm tests. If the
patch
was broken we would fail with a NameError in vdsm, accessing non-exisitng
variable.
I think it is a flaky test, since I ran many successful OST with this
patch.
These run include the patch:
- https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4173/
- https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4172/
- https://jenkins.ovirt.org/job/ovirt-system-tests_manual/4171/
There may be real issue behind this, but it is not related to the patch.
I think this is related to not deactivating lvs when putting storage
domain in maintenance.
We have this bug for the issue:
https://bugzilla.redhat.com/1163890
The issue is probably random since storage domain monitor is refreshed
every 5 minutes,
so the system correct itself automatically. It probably fails if the test
happened to start at the
wrong time, timing out before the system refreshed the storage.
If you see more of this issue, please file a new bug and we will handle it
when we have time.
In the meantime you can disable the test since it is not reliable.
Nir
> CQ reported this change as root cause:
> https://gerrit.ovirt.org/#/c/98015/6 - blockSD: Move legacy block size
> constants to sd
>
> build failure:
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/
>
> logs:
>
>
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13175/arti...
>
> errors in log:
>
> 2019-02-27 17:35:02,222-05 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] EVENT_ID:
> VDS_BROKER_COMMAND_FAILURE(10,802), VDSM la
> go-basic-suite-master-host-1 command ConnectStoragePoolVDS failed: Cannot
> find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
> msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
> 2019-02-27 17:35:02,222-05 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command
> 'org.ovirt.engine.core.vdsbroker.vdsbroker.
> ConnectStoragePoolVDSCommand' return value 'StatusOnlyReturn
> [status=Status [code=304, message=Cannot find master domain:
> u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
> msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5
> a9']]'
> 2019-02-27 17:35:02,222-05 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] HostName =
> lago-basic-suite-master-host-1
> 2019-02-27 17:35:02,223-05 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Command
> 'ConnectStoragePoolVDSCommand(HostName = lago-basic-suite-master-host-1,
>
ConnectStoragePoolVDSCommandParameters:{hostId='1ac0eda2-0980-40df-b029-b262680696a4',
> vdsId='1ac0eda2-0980-40df-b029-b262680696a4',
> storagePoolId='1c79b83e-32d6-43e0-bc61-2955e42c3a58',
masterVersion='1'})'
> execution failed: IRSGenericException: IRSErrorException:
> IRSNoMasterDomainException: Cannot find master domain:
> u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
> msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
> 2019-02-27 17:35:02,223-05 DEBUG
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-23) [558e4473] Exception:
> org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException:
> IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot
> find master domain: u'spUUID=1c79b83e-32d6-43e0-bc61-2955e42c3a58,
> msdUUID=7a7e1d7b-08aa-49f6-bdf2-4be64260b5a9'
> at
>
org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:50)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedConnectProxyReturnValue(ConnectStoragePoolVDSCommand.java:48)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.proceedProxyReturnValue(ConnectStoragePoolVDSCommand.java:36)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand.executeVdsBrokerCommand(ConnectStoragePoolVDSCommand.java:32)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
> [vdsbroker.jar:]
> at
> org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31)
> [dal.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:396)
> [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
> Source) [vdsbroker.jar:]
> at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
> [:1.8.0_191]
> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [rt.jar:1.8.0_191]
> at java.lang.reflect.Method.invoke(Method.java:498)
> [rt.jar:1.8.0_191]
> at
>
org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
> [common.jar:]
> at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
> [:1.8.0_191]
> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [rt.jar:1.8.0_191]
> at java.lang.reflect.Method.invoke(Method.java:498)
> [rt.jar:1.8.0_191]
> at
>
org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
> [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> at
>
org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
> Source) [vdsbroker.jar:]
> at
>
org.ovirt.engine.core.bll.storage.pool.RefreshPoolSingleAsyncOperation.execute(RefreshPoolSingleAsyncOperation.java:59)
> [bll.jar:]
> at
>
org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:43)
> [utils.jar:]
> at
>
org.ovirt.engine.core.utils.SynchronizeNumberOfAsyncOperations$AsyncOpThread.call(SynchronizeNumberOfAsyncOperations.java:32)
> [utils.jar:]
> at
>
org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:117)
> [utils.jar:]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [rt.jar:1.8.0_191]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [rt.jar:1.8.0_191]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [rt.jar:1.8.0_191]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_191]
> at
>
org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
> [javax.enterprise.concurrent-1.0.jar:]
>
>
2124
days inactive
2124
days old
2 comments
2 participants
participants (2)
-
Dafna Ron -
Nir Soffer