restricting check patch/merged parallel jobs

[Sandro Bonazzola]

On Wed, May 11, 2016 at 12:34 PM, Eyal Edri <eedri at redhat.com> wrote:

> From what I saw, it was mostly ovirt-engine and vdsm jobs pending on the
> queue while other slaves are idle.
> we have over 40 slaves and we're about to add more, so I don't think that
> will be an issue and IMO 3 per job is not enough, especially if you get
> idle slaves.
>
>
+1 on raising then.



> We are thinking on a more dynamic approach of dynamic vm allocation on
> demand, so in the long run we'll have more control over it,
> for now i'm monitoring the queue size and slaves on a regular basis [1],
> so if anything will get blocked too much time we'll act and adjust
> accordingly.
>
>
> [1] http://graphite.phx.ovirt.org/dashboard/db/jenkins-monitoring
>
> On Wed, May 11, 2016 at 1:10 PM, Sandro Bonazzola <sbonazzo at redhat.com>
> wrote:
>
>>
>>
>> On Tue, May 10, 2016 at 1:01 PM, Eyal Edri <eedri at redhat.com> wrote:
>>
>>> Shlomi,
>>> Can you submit a patch to increase the limit to 6 for (i think all jobs
>>> are using the same yaml template) and we'll continue to monitor to queue
>>> and see if there is an improvement in the utilization of slaves?
>>>
>>
>> Issue was that long lasting jobs caused queue to increase too much.
>> Example: a patch set rebased on master and merged will cause triggering
>> of check-merged jobs, upgrade jobs, ...; running 6 instance of each of them
>> will cause all other projects to be queued for a lot of time.
>>
>>
>>
>>>
>>> E.
>>>
>>> On Tue, May 10, 2016 at 1:58 PM, David Caro <dcaro at redhat.com> wrote:
>>>
>>>> On 05/10 13:54, Eyal Edri wrote:
>>>> > Is there any reason we're limiting the amount of check patch & check
>>>> merged
>>>> > jobs to run only 3 in parallel?
>>>> >
>>>>
>>>> We had some mess in the past where enabling parallel runs did not
>>>> really force
>>>> not using the same slave at the same time, I guess we never reenabled
>>>> them.
>>>>
>>>> > Each jobs runs in mock and on its own VM, anything presenting us from
>>>> > removing this limitation so we won't have idle slaves while other
>>>> jobs are
>>>> > in the queue?
>>>> >
>>>> > We can increase it at least to a higher level if we won't one
>>>> specific job
>>>> > to take over all slaves and starve other jobs, but i think
>>>> ovirt-engine
>>>> > jobs are probably the biggest consumer of ci, so the threshold should
>>>> be
>>>> > updated.
>>>>
>>>> +1
>>>>
>>>> >
>>>> > --
>>>> > Eyal Edri
>>>> > Associate Manager
>>>> > RHEV DevOps
>>>> > EMEA ENG Virtualization R&D
>>>> > Red Hat Israel
>>>> >
>>>> > phone: +972-9-7692018
>>>> > irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>
>>>> > _______________________________________________
>>>> > Infra mailing list
>>>> > Infra at ovirt.org
>>>> > http://lists.ovirt.org/mailman/listinfo/infra
>>>>
>>>>
>>>> --
>>>> David Caro
>>>>
>>>> Red Hat S.L.
>>>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>>>
>>>> Tel.: +420 532 294 605
>>>> Email: dcaro at redhat.com
>>>> IRC: dcaro|dcaroest@{freenode|oftc|redhat}
>>>> Web: www.redhat.com
>>>> RHT Global #: 82-62605
>>>>
>>>
>>>
>>>
>>> --
>>> Eyal Edri
>>> Associate Manager
>>> RHEV DevOps
>>> EMEA ENG Virtualization R&D
>>> Red Hat Israel
>>>
>>> phone: +972-9-7692018
>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>
>>> _______________________________________________
>>> Infra mailing list
>>> Infra at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/infra
>>>
>>>
>>
>>
>> --
>> Sandro Bonazzola
>> Better technology. Faster innovation. Powered by community collaboration.
>> See how it works at redhat.com
>>
>
>
>
> --
> Eyal Edri
> Associate Manager
> RHEV DevOps
> EMEA ENG Virtualization R&D
> Red Hat Israel
>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>



-- 
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20160511/afafef4f/attachment.html>