[JIRA] (OVIRT-1885) Remove slave/resource management from Jenkins
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1885?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1885:
--------------------------------
Epic Link: OVIRT-400
> Remove slave/resource management from Jenkins
> ---------------------------------------------
>
> Key: OVIRT-1885
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1885
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: Jenkins Master, Jenkins Slaves
> Reporter: Barak Korren
> Assignee: infra
> Labels: scale
>
> The way out infra is configure right now, resource allocation and management is done by Jenkins itself. This means that all resources need to pre-allocated to Jenkins in the form of slaves.
> If we want to have separate Jenkins masters, our current configuration will force us to hard-split the resources between different Jenkins masters.
> If we want to share our pool of resources between multiple masters, it means that resource allocation needs to move out of Jenkins to external tools.
> Potential tools for different types of resources are:
> * Beaker for managing physical hosts
> * oVirt + a Jenkins could provider plugin for managing VMs
> * OpenShift for managing containers (As a replacement for chroots).
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
6 years, 10 months
[JIRA] (OVIRT-1885) Remove slave/resource management from Jenkins
by Barak Korren (oVirt JIRA)
Barak Korren created OVIRT-1885:
-----------------------------------
Summary: Remove slave/resource management from Jenkins
Key: OVIRT-1885
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1885
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Master, Jenkins Slaves
Reporter: Barak Korren
Assignee: infra
The way out infra is configure right now, resource allocation and management is done by Jenkins itself. This means that all resources need to pre-allocated to Jenkins in the form of slaves.
If we want to have separate Jenkins masters, our current configuration will force us to hard-split the resources between different Jenkins masters.
If we want to share our pool of resources between multiple masters, it means that resource allocation needs to move out of Jenkins to external tools.
Potential tools for different types of resources are:
* Beaker for managing physical hosts
* oVirt + a Jenkins could provider plugin for managing VMs
* OpenShift for managing containers (As a replacement for chroots).
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
6 years, 10 months
[JIRA] (OVIRT-1884) Scale up to multipile Jenkins masters
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1884?page=com.atlassian.jir... ]
Barak Korren reassigned OVIRT-1884:
-----------------------------------
Assignee: Barak Korren (was: infra)
> Scale up to multipile Jenkins masters
> -------------------------------------
>
> Key: OVIRT-1884
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1884
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: Jenkins Master
> Reporter: Barak Korren
> Assignee: Barak Korren
> Labels: scale
>
> There are several reasons why we would like to enable scaling up to multiple Jenkins masters:
> * We may reach a point where a single jenkins master will simple not be able to cope with the workload we have - this is especially true as we increase our use of heavy features like pipelines and blue ocean.
> * If we choose to start using OpenShift's embedded Jenkins, the approach there is that every project gets its own Jenkins instance.
> * As we increase out reach and support more projects, some of them would not like to use an oVirt branded Jenkins instance.
> This ticket would track the work we need to do to enable supporting multiple Jenkins master in out infra. Individual pieces of work should be created as separate tickets and be linked as blockers to this one.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
6 years, 10 months
[JIRA] (OVIRT-1884) Scale up to multipile Jenkins masters
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1517996808-26764-286
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Barak Korren created OVIRT-1884:
-----------------------------------
Summary: Scale up to multipile Jenkins masters
Key: OVIRT-1884
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1884
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Master
Reporter: Barak Korren
Assignee: infra
There are several reasons why we would like to enable scaling up to multiple Jenkins masters:
* We may reach a point where a single jenkins master will simple not be able to cope with the workload we have - this is especially true as we increase our use of heavy features like pipelines and blue ocean.
* If we choose to start using OpenShift's embedded Jenkins, the approach there is that every project gets its own Jenkins instance.
* As we increase out reach and support more projects, some of them would not like to use an oVirt branded Jenkins instance.
This ticket would track the work we need to do to enable supporting multiple Jenkins master in out infra. Individual pieces of work should be created as separate tickets and be linked as blockers to this one.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
------------=_1517996808-26764-286
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<h3>Barak Korren created OVIRT-1884:</h3>
<pre> Summary: Scale up to multipile Jenkins masters
Key: OVIRT-1884
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1884
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Master
Reporter: Barak Korren
Assignee: infra</pre>
<p>There are several reasons why we would like to enable scaling up to multiple Jenkins masters:</p>
<ul><li><p>We may reach a point where a single jenkins master will simple not be able to cope with the workload we have – this is especially true as we increase our use of heavy features like pipelines and blue ocean.</p></li>
<li><p>If we choose to start using OpenShift's embedded Jenkins, the approach there is that every project gets its own Jenkins instance.</p></li>
<li><p>As we increase out reach and support more projects, some of them would not like to use an oVirt branded Jenkins instance.</p></li></ul>
<p>This ticket would track the work we need to do to enable supporting multiple Jenkins master in out infra. Individual pieces of work should be created as separate tickets and be linked as blockers to this one.</p>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100079)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BS..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1517996808-26764-286--
6 years, 10 months
[JIRA] (OVIRT-1884) Scale up to multipile Jenkins masters
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1517996808-12609-300
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1884?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1884:
--------------------------------
Epic Link: OVIRT-400
> Scale up to multipile Jenkins masters
> -------------------------------------
>
> Key: OVIRT-1884
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1884
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: Jenkins Master
> Reporter: Barak Korren
> Assignee: infra
> Labels: scale
>
> There are several reasons why we would like to enable scaling up to multiple Jenkins masters:
> * We may reach a point where a single jenkins master will simple not be able to cope with the workload we have - this is especially true as we increase our use of heavy features like pipelines and blue ocean.
> * If we choose to start using OpenShift's embedded Jenkins, the approach there is that every project gets its own Jenkins instance.
> * As we increase out reach and support more projects, some of them would not like to use an oVirt branded Jenkins instance.
> This ticket would track the work we need to do to enable supporting multiple Jenkins master in out infra. Individual pieces of work should be created as separate tickets and be linked as blockers to this one.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
------------=_1517996808-12609-300
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1884?page=com.atlassian.jir... ]</pre>
<h3>Barak Korren updated OVIRT-1884:</h3>
<pre>Epic Link: OVIRT-400</pre>
<blockquote><h3>Scale up to multipile Jenkins masters</h3>
<pre> Key: OVIRT-1884
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1884
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Master
Reporter: Barak Korren
Assignee: infra
Labels: scale</pre>
<p>There are several reasons why we would like to enable scaling up to multiple Jenkins masters:</p>
<ul><li><p>We may reach a point where a single jenkins master will simple not be able to cope with the workload we have – this is especially true as we increase our use of heavy features like pipelines and blue ocean.</p></li>
<li><p>If we choose to start using OpenShift's embedded Jenkins, the approach there is that every project gets its own Jenkins instance.</p></li>
<li><p>As we increase out reach and support more projects, some of them would not like to use an oVirt branded Jenkins instance.</p></li></ul>
<p>This ticket would track the work we need to do to enable supporting multiple Jenkins master in out infra. Individual pieces of work should be created as separate tickets and be linked as blockers to this one.</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100079)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BS..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1517996808-12609-300--
6 years, 10 months
[JIRA] (OVIRT-1883) Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1517995716-25255-216
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1883?page=com.atlassian.jir... ]
Barak Korren reassigned OVIRT-1883:
-----------------------------------
Assignee: Barak Korren (was: infra)
> Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)
> -------------------------------------------------------------------------------
>
> Key: OVIRT-1883
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1883
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: Jenkins Slaves
> Reporter: Barak Korren
> Assignee: Barak Korren
> Labels: beaker
>
> The CI system currently relies on the fact that the Jenkins slaves are stateful, and stores various caches on them.
> If we move to a mechanism where slaves are provisioned dynamically, we need to consider ho to keep having effective caches.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
------------=_1517995716-25255-216
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1883?page=com.atlassian.jir... ]</pre>
<h3>Barak Korren reassigned OVIRT-1883:</h3>
<pre>Assignee: Barak Korren (was: infra)</pre>
<blockquote><h3>Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)</h3>
<pre> Key: OVIRT-1883
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1883
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Slaves
Reporter: Barak Korren
Assignee: Barak Korren
Labels: beaker</pre>
<p>The CI system currently relies on the fact that the Jenkins slaves are stateful, and stores various caches on them. If we move to a mechanism where slaves are provisioned dynamically, we need to consider ho to keep having effective caches.</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100079)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BS..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1517995716-25255-216--
6 years, 10 months
[JIRA] (OVIRT-1883) Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1517995701-11810-157
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1883?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1883:
--------------------------------
Epic Link: OVIRT-400
> Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)
> -------------------------------------------------------------------------------
>
> Key: OVIRT-1883
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1883
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: Jenkins Slaves
> Reporter: Barak Korren
> Assignee: infra
> Labels: beaker
>
> The CI system currently relies on the fact that the Jenkins slaves are stateful, and stores various caches on them.
> If we move to a mechanism where slaves are provisioned dynamically, we need to consider ho to keep having effective caches.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
------------=_1517995701-11810-157
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1883?page=com.atlassian.jir... ]</pre>
<h3>Barak Korren updated OVIRT-1883:</h3>
<pre>Epic Link: OVIRT-400</pre>
<blockquote><h3>Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)</h3>
<pre> Key: OVIRT-1883
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1883
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Slaves
Reporter: Barak Korren
Assignee: infra
Labels: beaker</pre>
<p>The CI system currently relies on the fact that the Jenkins slaves are stateful, and stores various caches on them. If we move to a mechanism where slaves are provisioned dynamically, we need to consider ho to keep having effective caches.</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100079)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BS..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1517995701-11810-157--
6 years, 10 months
[JIRA] (OVIRT-1883) Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1517995701-29869-301
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Barak Korren created OVIRT-1883:
-----------------------------------
Summary: Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)
Key: OVIRT-1883
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1883
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Slaves
Reporter: Barak Korren
Assignee: infra
The CI system currently relies on the fact that the Jenkins slaves are stateful, and stores various caches on them.
If we move to a mechanism where slaves are provisioned dynamically, we need to consider ho to keep having effective caches.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100079)
------------=_1517995701-29869-301
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<h3>Barak Korren created OVIRT-1883:</h3>
<pre> Summary: Figure out how to make caches work with re-provisioned hosts (E.g. from beaker)
Key: OVIRT-1883
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1883
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Jenkins Slaves
Reporter: Barak Korren
Assignee: infra</pre>
<p>The CI system currently relies on the fact that the Jenkins slaves are stateful, and stores various caches on them.</p>
<p>If we move to a mechanism where slaves are provisioned dynamically, we need to consider ho to keep having effective caches.</p>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100079)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BS..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1517995701-29869-301--
6 years, 10 months
