[JIRA] (OVIRT-1626) Java Excpetion during CQ tester pipeline run

Java Excpetion during CQ tester pipeline run -------------------------------------------- Key: OVIRT-1626 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1626 Project: oVirt - virtualization made easy Issue Type: New Feature Components: oVirt CI Reporter: Barak Korren Assignee: infra Priority: High Labels: change-queue, ost_failures The following 'ovirt-master_change-queue-tester' run failed with a Java exception: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2257/ The exception seen in blue ocean is the following: {code} Stashed 1 file(s) Failed to serialize org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage$Tag#actions for class org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage$Tag {code} In the job console one can see a different exception: {code} java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302) at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:275) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111) at org.jenkinsci.plugins.workflow.cps.CpsStepContext.getThreadGroupSynchronously(CpsStepContext.java:248) at org.jenkinsci.plugins.workflow.cps.CpsStepContext.getThreadSynchronously(CpsStepContext.java:237) at org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:298) at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:61) at org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:258) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:203) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:150) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1283) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1177) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1027) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:19) at Script1.mk_ost_runner(Script1.groovy:300) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor613.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:173) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:162) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:162) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:330) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE {code} The exception in the console seem to indicate that the exception is in line 300 of the pipeline script in the mk_ost_runner funtction. Here is the code of the function for reference: {code} 289 def mk_ost_runner(ovirt_release, suit_type, distro) { 290 return { 291 def suit_dir = "$suit_type-suit-$ovirt_release-$distro" 292 // stash an empty file so we don`t get errors in no artifacts get stashed 293 writeFile file: '__no_artifacts_stashed__', text: '' 294 stash includes: '__no_artifacts_stashed__', name: suit_dir 295 try { 296 node('integ-tests') { 297 run_ost_on_node(ovirt_release, suit_type, distro, suit_dir) 298 } 299 } finally { 300 dir("exported-artifacts/$suit_dir") { 301 unstash suit_dir 302 } 303 } 304 } 305 } {code} It looks like the exception seen in the console is a secondary exception that happens because of the failure to stash some files earlier in the code inside the {{basic-suit}} run branch as indicated in the blue ocean display. Looking further at the log, we can fined the following which aligns with what we see in blue ocean (Lines that belong to other run branches had been removed): {code} 19:40:55 Will run the following OST (master) suits: basic, upgrade-from-release, upgrade-from-prevrelease [Pipeline] [basic-suit] { (Branch: basic-suit) [Pipeline] [basic-suit] writeFile [Pipeline] [basic-suit] stash 19:40:55 [basic-suit] Stashed 1 file(s) [Pipeline] [basic-suit] node [Pipeline] [basic-suit] // dir [Pipeline] [basic-suit] } 19:47:04 [basic-suit] Failed in branch basic-suit {code} From the log above it seems that the actual failure is around line 294 if the function quoted above.