[JIRA] (OVIRT-1551) Add slaves to staging Jenkins instance
by Barak Korren (oVirt JIRA)
Barak Korren created OVIRT-1551:
-----------------------------------
Summary: Add slaves to staging Jenkins instance
Key: OVIRT-1551
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1551
Project: oVirt - virtualization made easy
Issue Type: New Feature
Components: oVirt Infra
Reporter: Barak Korren
Assignee: infra
We currently have just one slave attached to the staging Jenkins instance.
We need some more, at least one of each slave platform we support so that we can:
# Test jobs running in paralllel
# Test jobs running on different slave platforms.
--
This message was sent by Atlassian JIRA
(v1000.1139.0#100055)
7 years, 3 months
Jenkins error: java.io.FileNotFoundException: ... (Too many open files)
by Nir Soffer
Jenkins needs restart?
Building remotely on ovirt-srv21.phx.ovirt.org
<http://jenkins.ovirt.org/computer/ovirt-srv21.phx.ovirt.org> (fc24
phx integ-tests physical) in workspace
/home/jenkins/workspace/ovirt-system-tests_manual
java.io.FileNotFoundException:
/var/lib/data/jenkins/jobs/ovirt-system-tests_manual/builds/856/changelog.xml
(Too many open files)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:119)
at hudson.scm.SCM.checkout(SCM.java:495)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1276)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:560)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:485)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
ERROR: Step ?Post build task? aborted due to exception:
7 years, 3 months
[JIRA] (OVIRT-1550) Configure and monitor nofiles on Jenkins
by Barak Korren (oVirt JIRA)
Barak Korren created OVIRT-1550:
-----------------------------------
Summary: Configure and monitor nofiles on Jenkins
Key: OVIRT-1550
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1550
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: oVirt CI
Reporter: Barak Korren
Assignee: infra
Priority: High
We had an issue today (Jul 25th, 2017) of Jenkins running out of file descriptors.
Things already reached the point where Jenkins was barely functioning so to get things back to working order I've:
* Manually added a line in '{{/etc/security/limits.conf}}' to set 'nofiles' to 16384 for the 'jenkins' user (Previously it was just not configured and seems to have defaulted to 4096).
* Restarted the 'jenkins' service.
We now need to:
* Place this configuration in Puppet/Ansible.
* Add monitoring so we'll know in advance the next time Jenkins goes near to the nofiles limit.
--
This message was sent by Atlassian JIRA
(v1000.1139.0#100055)
7 years, 3 months
[JIRA] (OVIRT-1550) Configure and monitor nofiles on Jenkins
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1550?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1550:
--------------------------------
Epic Link: OVIRT-403
> Configure and monitor nofiles on Jenkins
> ----------------------------------------
>
> Key: OVIRT-1550
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1550
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: oVirt CI
> Reporter: Barak Korren
> Assignee: infra
> Priority: High
> Labels: infra, jenkins
>
> We had an issue today (Jul 25th, 2017) of Jenkins running out of file descriptors.
> Things already reached the point where Jenkins was barely functioning so to get things back to working order I've:
> * Manually added a line in '{{/etc/security/limits.conf}}' to set 'nofiles' to 16384 for the 'jenkins' user (Previously it was just not configured and seems to have defaulted to 4096).
> * Restarted the 'jenkins' service.
> We now need to:
> * Place this configuration in Puppet/Ansible.
> * Add monitoring so we'll know in advance the next time Jenkins goes near to the nofiles limit.
--
This message was sent by Atlassian JIRA
(v1000.1139.0#100055)
7 years, 3 months
Jenkins was restarted
by Barak Korren
Jenkins ran out of file descriptors so we had to restart it to bump the number.
Jobs should be running jobs already but the UI will probably take
30-40 minutes to become available.
Issues seem to have started around 15:00 UTC yesterday (18:00 IST) so
you may need to re-trigger jobs for patches submitted since then.
--
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
7 years, 3 months