[JIRA] (OVIRT-1067) Java updates without restarting Jenkins agent are probably causing jobs to fail

[Barak Korren (oVirt JIRA)]

     [ https://ovirt-jira.atlassian.net/browse/OVIRT-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Barak Korren updated OVIRT-1067:
--------------------------------
    Epic Link: OVIRT-403

> Java updates without restarting Jenkins agent are probably causing jobs to fail
> -------------------------------------------------------------------------------
>
>                 Key: OVIRT-1067
>                 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1067
>             Project: oVirt - virtualization made easy
>          Issue Type: Bug
>            Reporter: Barak Korren
>            Assignee: infra
>
> A [recently invoked|http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_3.6/4523/console] system teasts job failed with the following error:
> {code}
> java.io.IOException: remote file operation failed: /home/jenkins/workspace/test-repo_ovirt_experimental_3.6 at hudson.remoting.Channel at 432401f8:el7-vm26.phx.ovirt.org: java.io.IOException: Remote call on el7-vm26.phx.ovirt.org failed
> 	at hudson.FilePath.act(FilePath.java:992)
> ...
> Caused by: java.lang.InternalError: java.io.FileNotFoundException: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-2.b15.el7_3.x86_64/jre/lib/resources.jar
> 	at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1003)
> 	at sun.misc.URLClassPath.getResource(URLClassPath.java:212)
> ...
> {code}
> Looking at {{el7-vm26.phx.ovirt.org failed}} where this happened proved that indeed the file '{{/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-2.b15.el7_3.x86_64/jre/lib/resources.jar}}' does not exist on the node. The file '{{/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/lib/resources.jar}}' does exist however.
> Further examination in the logs revealed that the system's java version was recently updated:
> {code}
> Jan 21 20:19:36 el7-vm26 yum[31962]: Updated: 1:java-1.8.0-openjdk-headless-1.8.0.121-0.b13.el7_3.x86_64
> Jan 21 20:19:38 el7-vm26 yum[31962]: Updated: 1:java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64
> Jan 21 20:19:40 el7-vm26 yum[31962]: Updated: 1:java-1.8.0-openjdk-devel-1.8.0.121-0.b13.el7_3.x86_64
> {code}
> I suppose the issue is does to the system Jave and libraries being updated while the Jenkins agent was still running and trying to use the older libraries.
> I suggest we endure that the Jenkins agent is shut down every time package updates are done on the node and restarted afterwards. This could be done by setting the node to "offline" in Jenkins.



--
This message was sent by Atlassian JIRA
(v1000.695.3#100025)