5:52 a.m.
Hi,
We seem to have a problem in the OST tests that is causing failure in
multiple projects
The issue is that setting __name__ on local function is not working
correctly and we are exiting instead of skipping when Glance image is not
available.
We had the same issue on a different test where Milan fixed it:
https://gerrit.ovirt.org/#/c/93167/
but now it seems that we have the same issue on list_glace_images as well.
ERROR:
Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line
78, in wrapper
prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 633, in list_glance_images
raise SkipTest('%s: GLANCE connectivity test failed' %
list_glance_images_4.__name__ )
NameError: global name 'list_glance_images_4' is not defined
5:58 a.m.
Gal posted a fix: https://gerrit.ovirt.org/#/c/93350/
it seems that the change to the test was done a while back and we did not
hit it since Glance was available.
On Fri, Jul 27, 2018 at 11:52 AM, Dafna Ron <dron(a)redhat.com> wrote:
Hi,
We seem to have a problem in the OST tests that is causing failure in
multiple projects
The issue is that setting __name__ on local function is not working
correctly and we are exiting instead of skipping when Glance image is not
available.
We had the same issue on a different test where Milan fixed it:
https://gerrit.ovirt.org/#/c/93167/
but now it seems that we have the same issue on list_glace_images as well.
ERROR:
Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in
wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in
wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in
wrapper
prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 633, in list_glance_images
raise SkipTest('%s: GLANCE connectivity test failed' %
list_glance_images_4.__name__ )
NameError: global name 'list_glance_images_4' is not defined
9:37 a.m.
There are two issues here:
1. OST is exiting with wring error due to the local function not working
which Gal has a patch for
2. there is an actual code regression which we suspects comes from the SDK
We suspect the issue is a new sdk package build yesterday
https://cbs.centos.org/koji/buildinfo?buildID=23581
Here is a link to the fist failure's logs:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8800/artif...
Dominic is currently looking at the issue.
<error>
2018-07-26 13:47:37,745-04 DEBUG
[org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3283d2df] Got:
***L:INFO Yum install: 217/529: libosinfo-1.0.0-1.el7.x86_64
2018-07-26 13:47:37,745-04 DEBUG
[org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3283d2df]
nextEvent: Log INFO Yum install: 217/529: libosinfo-1.0.0-1.el7.x86_64
2018-07-26 13:47:37,754-04 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(default task-2) [b4915769-1cf0-4526-9214-e932d078cf07] method: runAction,
params: [TestProviderConnectivity, ProviderParameters:{comma
ndId='d29721ff-3dd7-4932-b43d-eee819f1afee', user='null',
commandType='Unknown'}], timeElapsed: 33ms
2018-07-26 13:47:37,766-04 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(VdsDeploy) [3283d2df] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing
Host lago-basic-suite-master-host-1. Yum install: 217/529: l
ibosinfo-1.0.0-1.el7.x86_64.
2018-07-26 13:47:37,782-04 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
task-2) [] Operation Failed: WFLYEJB0442: Unexpected Error
2018-07-26 13:47:37,782-04 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
task-2) [] Exception: javax.ejb.EJBException: WFLYEJB0442: Unexpected Error
at
org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:218)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:418)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:148)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at
org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
at
org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
at
org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:619)
at
org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
at
org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
at
org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
at
org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
at
org.ovirt.engine.core.common.interfaces.BackendLocal$$$view2.runAction(Unknown
Source)
at
org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:250)
[restapi-jaxrs.jar:]
at
org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:182)
[restapi-jaxrs.jar:]
at
org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:174)
[restapi-jaxrs.jar:]
at
org.ovirt.engine.api.restapi.resource.AbstractBackendExternalProviderResource.testConnectivity(AbstractBackendExternalProviderResource.java:46)
[restapi-jaxrs.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[rt.jar:1.8.0_181]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[rt.jar:1.8.0_181]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_181]
<error>
On Fri, Jul 27, 2018 at 11:58 AM, Dafna Ron <dron(a)redhat.com> wrote:
Gal posted a fix: https://gerrit.ovirt.org/#/c/93350/
it seems that the change to the test was done a while back and we did not
hit it since Glance was available.
On Fri, Jul 27, 2018 at 11:52 AM, Dafna Ron <dron(a)redhat.com> wrote:
> Hi,
>
> We seem to have a problem in the OST tests that is causing failure in
> multiple projects
>
> The issue is that setting __name__ on local function is not working
> correctly and we are exiting instead of skipping when Glance image is not
> available.
>
> We had the same issue on a different test where Milan fixed it:
> https://gerrit.ovirt.org/#/c/93167/
> but now it seems that we have the same issue on list_glace_images as
> well.
>
>
> ERROR:
>
> Traceback (most recent call last):
> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
> testMethod()
> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in
runTest
> self.test(*self.arg)
> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129,
in wrapped_test
> test()
> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in
wrapper
> return func(get_test_prefix(), *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in
wrapper
> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
> File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 633, in list_glance_images
> raise SkipTest('%s: GLANCE connectivity test failed' %
list_glance_images_4.__name__ )
> NameError: global name 'list_glance_images_4' is not defined
>
>
10:58 a.m.
On Fri, 27 Jul 2018 15:37:23 +0100
Dafna Ron <dron(a)redhat.com> wrote:
There are two issues here:
1. OST is exiting with wring error due to the local function not
working which Gal has a patch for
2. there is an actual code regression which we suspects comes from
the SDK
We suspect the issue is a new sdk package build yesterday
https://cbs.centos.org/koji/buildinfo?buildID=23581
Here is a link to the fist failure's logs:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8800/artif...
Dominic is currently looking at the issue.
One reason was an incompatibility on bytecode level of the
openstack-java-sdk used at compile time and runtime.
Fix posted on https://gerrit.ovirt.org/93352
<error>
2018-07-26 13:47:37,745-04 DEBUG
[org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3283d2df]
Got: ***L:INFO Yum install: 217/529: libosinfo-1.0.0-1.el7.x86_64
2018-07-26 13:47:37,745-04 DEBUG
[org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [3283d2df]
nextEvent: Log INFO Yum install: 217/529: libosinfo-1.0.0-1.el7.x86_64
2018-07-26 13:47:37,754-04 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(default task-2) [b4915769-1cf0-4526-9214-e932d078cf07] method:
runAction, params: [TestProviderConnectivity,
ProviderParameters:{comma
ndId='d29721ff-3dd7-4932-b43d-eee819f1afee', user='null',
commandType='Unknown'}], timeElapsed: 33ms 2018-07-26 13:47:37,766-04
INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(VdsDeploy) [3283d2df] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509),
Installing Host lago-basic-suite-master-host-1. Yum install: 217/529:
l ibosinfo-1.0.0-1.el7.x86_64. 2018-07-26 13:47:37,782-04 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
(default task-2) [] Operation Failed: WFLYEJB0442: Unexpected Error
2018-07-26 13:47:37,782-04 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
(default task-2) [] Exception: javax.ejb.EJBException: WFLYEJB0442:
Unexpected Error at
org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:218)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:418)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:148)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
at
org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
at
org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
[wildfly-ejb3-13.0.0.Final.jar:13.0.0.Final]
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
at
org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:619)
at
org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
at
org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
at
org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
at
org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185)
at
org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
at
org.ovirt.engine.core.common.interfaces.BackendLocal$$$view2.runAction(Unknown
Source)
at
org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:250)
[restapi-jaxrs.jar:]
at
org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:182)
[restapi-jaxrs.jar:]
at
org.ovirt.engine.api.restapi.resource.BackendResource.performAction(BackendResource.java:174)
[restapi-jaxrs.jar:]
at
org.ovirt.engine.api.restapi.resource.AbstractBackendExternalProviderResource.testConnectivity(AbstractBackendExternalProviderResource.java:46)
[restapi-jaxrs.jar:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[rt.jar:1.8.0_181]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[rt.jar:1.8.0_181]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[rt.jar:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498)
[rt.jar:1.8.0_181]
<error>
On Fri, Jul 27, 2018 at 11:58 AM, Dafna Ron <dron(a)redhat.com> wrote:
> Gal posted a fix: https://gerrit.ovirt.org/#/c/93350/
>
> it seems that the change to the test was done a while back and we
> did not hit it since Glance was available.
>
>
>
> On Fri, Jul 27, 2018 at 11:52 AM, Dafna Ron <dron(a)redhat.com> wrote:
>
>> Hi,
>>
>> We seem to have a problem in the OST tests that is causing failure
>> in multiple projects
>>
>> The issue is that setting __name__ on local function is not working
>> correctly and we are exiting instead of skipping when Glance image
>> is not available.
>>
>> We had the same issue on a different test where Milan fixed it:
>> https://gerrit.ovirt.org/#/c/93167/
>> but now it seems that we have the same issue on list_glace_images
>> as well.
>>
>>
>> ERROR:
>>
>> Traceback (most recent call last):
>> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
>> testMethod()
>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197,
>> in runTest self.test(*self.arg)
>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>> line 129, in wrapped_test test()
>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>> line 59, in wrapper return func(get_test_prefix(), *args, **kwargs)
>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>> line 78, in wrapper
>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
>> File
>>
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
>> line 633, in list_glance_images raise SkipTest('%s: GLANCE
>> connectivity test failed' % list_glance_images_4.__name__ )
>> NameError: global name 'list_glance_images_4' is not defined
>>
>>
>
2098
days inactive
2098
days old
3 comments
2 participants
participants (2)
-
Dafna Ron -
Dominik Holler