[JIRA] (OVIRT-2782) Configure Jenkins to wait less and allocate
more
by Daniel Belenky (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2782?page=com.atlassian.jir... ]
Daniel Belenky updated OVIRT-2782:
----------------------------------
Description:
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting
{code:java}
hudson.slaves.NodeProvisioner.initialDelay
{code}
to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can control it's margins to lower its threshold so that Jenkins will allocate nodes faster and in advance.
The recommended settings by the k8s plugin to spawn a node for every build in the queue:
```
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance.
{code:java}
-Dhudson.slaves.NodeProvisioner.initialDelay=0
-Dhudson.slaves.NodeProvisioner.MARGIN=50
-Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
{code}
was:
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can control it's margins to lower its threshold so that Jenkins will allocate nodes faster and in advance.
The recommended settings by the k8s plugin to spawn a node for every build in the queue:
```
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance.
{code:java}
-Dhudson.slaves.NodeProvisioner.initialDelay=0
-Dhudson.slaves.NodeProvisioner.MARGIN=50
-Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
{code}
> Configure Jenkins to wait less and allocate more
> ------------------------------------------------
>
> Key: OVIRT-2782
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2782
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Reporter: Daniel Belenky
> Assignee: infra
>
> By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting
> {code:java}
> hudson.slaves.NodeProvisioner.initialDelay
> {code}
> to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can control it's margins to lower its threshold so that Jenkins will allocate nodes faster and in advance.
> The recommended settings by the k8s plugin to spawn a node for every build in the queue:
> ```
> By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance.
> {code:java}
> -Dhudson.slaves.NodeProvisioner.initialDelay=0
> -Dhudson.slaves.NodeProvisioner.MARGIN=50
> -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
> {code}
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100107)
5 years, 4 months
[JIRA] (OVIRT-2782) Configure Jenkins to wait less and allocate
more
by Daniel Belenky (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2782?page=com.atlassian.jir... ]
Daniel Belenky updated OVIRT-2782:
----------------------------------
Description:
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can control it's margins to lower its threshold so that Jenkins will allocate nodes faster and in advance.
The recommended settings by the k8s plugin to spawn a node for every build in the queue:
```
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance.
{code:java}
-Dhudson.slaves.NodeProvisioner.initialDelay=0
-Dhudson.slaves.NodeProvisioner.MARGIN=50
-Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
{code}
was:
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can control it's margins to lower its threshold so that Jenkins will allocate nodes faster and in advance.
The recommended settings by the k8s plugin to spawn a node for every build in the queue:
```
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance.
```
-Dhudson.slaves.NodeProvisioner.initialDelay=0
-Dhudson.slaves.NodeProvisioner.MARGIN=50
-Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
```
> Configure Jenkins to wait less and allocate more
> ------------------------------------------------
>
> Key: OVIRT-2782
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2782
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Reporter: Daniel Belenky
> Assignee: infra
>
> By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can control it's margins to lower its threshold so that Jenkins will allocate nodes faster and in advance.
> The recommended settings by the k8s plugin to spawn a node for every build in the queue:
> ```
> By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance.
> {code:java}
> -Dhudson.slaves.NodeProvisioner.initialDelay=0
> -Dhudson.slaves.NodeProvisioner.MARGIN=50
> -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
> {code}
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100107)
5 years, 4 months
[JIRA] (OVIRT-2782) Configure Jenkins to wait less and allocate
more
by Daniel Belenky (oVirt JIRA)
Daniel Belenky created OVIRT-2782:
-------------------------------------
Summary: Configure Jenkins to wait less and allocate more
Key: OVIRT-2782
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2782
Project: oVirt - virtualization made easy
Issue Type: Improvement
Reporter: Daniel Belenky
Assignee: infra
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can control it's margins to lower its threshold so that Jenkins will allocate nodes faster and in advance.
The recommended settings by the k8s plugin to spawn a node for every build in the queue:
```
By default, Jenkins is configured to wait a few seconds before allocating a new node in a hope that a node in use to be freed. We can control this setting by setting `hudson.slaves.NodeProvisioner.initialDelay` to `0`. But, because Jenkins computes the excess workload value (which decides if we need to allocate a new node) using an EMA, we also can lower its threshold so that Jenkins will allocate nodes faster and in advance.
```
-Dhudson.slaves.NodeProvisioner.initialDelay=0
-Dhudson.slaves.NodeProvisioner.MARGIN=50
-Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
```
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100107)
5 years, 4 months
Build failed in Jenkins:
system-sync_mirrors-centos-qemu-ev-release-el7-x86_64 #2354
by jenkins@jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-qemu-ev-release-e...>
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-centos-qemu-ev-release-e...>
No credentials specified
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Pruning obsolete local branches
Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git
> git --version # timeout=10
> git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision cfeb68c000d32e1c9843a2f8650b7b9a65e01a8e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f cfeb68c000d32e1c9843a2f8650b7b9a65e01a8e
Commit message: "Disable use_bootstrap_container for Fedora rawhide"
> git rev-list --no-walk cfeb68c000d32e1c9843a2f8650b7b9a65e01a8e # timeout=10
[system-sync_mirrors-centos-qemu-ev-release-el7-x86_64] $ /bin/bash -xe /tmp/jenkins6854356434980691008.sh
+ jenkins/scripts/mirror_mgr.sh resync_yum_mirror centos-qemu-ev-release-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ MIRRORS_MP_BASE=/var/www/html/repos
+ MIRRORS_HTTP_BASE=http://mirrors.phx.ovirt.org/repos
+ MIRRORS_CACHE=/home/jenkins/mirrors_cache
+ MAX_LOCK_ATTEMPTS=120
+ LOCK_WAIT_INTERVAL=5
+ LOCK_BASE=/home/jenkins
+ OLD_MD_TO_KEEP=100
+ HTTP_SELINUX_TYPE=httpd_sys_content_t
+ HTTP_FILE_MODE=644
+ main resync_yum_mirror centos-qemu-ev-release-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ local command=resync_yum_mirror
+ command_args=("${@:2}")
+ local command_args
+ cmd_resync_yum_mirror centos-qemu-ev-release-el7 x86_64 jenkins/data/mirrors-reposync.conf
+ local repo_name=centos-qemu-ev-release-el7
+ local repo_archs=x86_64
+ local reposync_conf=jenkins/data/mirrors-reposync.conf
+ local sync_needed
+ mkdir -p /home/jenkins/mirrors_cache
+ verify_repo_fs centos-qemu-ev-release-el7 yum
+ local repo_name=centos-qemu-ev-release-el7
+ local repo_type=yum
+ sudo install -o jenkins -d /var/www/html/repos/yum /var/www/html/repos/yum/centos-qemu-ev-release-el7 /var/www/html/repos/yum/centos-qemu-ev-release-el7/base
+ check_yum_sync_needed centos-qemu-ev-release-el7 x86_64 jenkins/data/mirrors-reposync.conf sync_needed
+ local repo_name=centos-qemu-ev-release-el7
+ local repo_archs=x86_64
+ local reposync_conf=jenkins/data/mirrors-reposync.conf
+ local p_sync_needed=sync_needed
+ local reposync_out
+ echo 'Checking if mirror needs a resync'
Checking if mirror needs a resync
+ rm -rf /home/jenkins/mirrors_cache/centos-qemu-ev-release-el7
++ IFS=,
++ echo x86_64
+ for arch in '$(IFS=,; echo $repo_archs)'
++ run_reposync centos-qemu-ev-release-el7 x86_64 jenkins/data/mirrors-reposync.conf --urls --quiet
++ local repo_name=centos-qemu-ev-release-el7
++ local repo_arch=x86_64
++ local reposync_conf=jenkins/data/mirrors-reposync.conf
++ extra_args=("${@:4}")
++ local extra_args
++ reposync --config=jenkins/data/mirrors-reposync.conf --repoid=centos-qemu-ev-release-el7 --arch=x86_64 --cachedir=/home/jenkins/mirrors_cache --download_path=/var/www/html/repos/yum/centos-qemu-ev-release-el7/base --norepopath --newest-only --urls --quiet
Traceback (most recent call last):
File "/usr/bin/reposync", line 373, in <module>
main()
File "/usr/bin/reposync", line 185, in main
my.doRepoSetup()
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup
return self._getRepos(thisrepo, True)
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos
self._repos.doSetup(thisrepo)
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup
self.retrieveAllMD()
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD
dl = repo._async and repo._commonLoadRepoXML(repo)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1477, in _commonLoadRepoXML
if self._latestRepoXML(local):
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1455, in _latestRepoXML
repomd = self.metalink_data.repomd
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 921, in <lambda>
metalink_data = property(fget=lambda self: self._getMetalink(),
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 917, in _getMetalink
self._metalink = metalink.MetaLinkRepoMD(self.metalink_filename)
File "/usr/lib/python2.7/site-packages/yum/metalink.py", line 189, in __init__
raise MetaLinkRepoErrorParseFail, "File %s is not XML" % filename
yum.metalink.MetaLinkRepoErrorParseFail: File /home/jenkins/mirrors_cache/fedora-base-fc29/metalink.xml is not XML
+ reposync_out='Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=fedora-29&arch=x86_64 error was
16: error opening file from https://mirrors.fedoraproject.org/metalink?repo=fedora-29&arch=x86_64, IOError: [Errno 2] No such file or directory: '\''/home/jenkins/mirrors_cache/fedora-base-fc29/metalink.xml.tmp'\'''
Build step 'Execute shell' marked build as failure
5 years, 4 months