Gerrit load

[David Caro]

On Sun 15 Jun 2014 03:28:45 PM CEST, Eyal Edri wrote:
>
>
> ----- Original Message -----
>> From: "Sandro Bonazzola" <sbonazzo at redhat.com>
>> To: "David Caro" <dcaroest at redhat.com>, "infra" <infra at ovirt.org>
>> Sent: Saturday, June 14, 2014 11:52:15 AM
>> Subject: Re: Gerrit load
>>
>> Il 13/06/2014 23:05, David Caro ha scritto:
>>> Hi!
>>>
>>>
>>> I think I've found the main cause for the latest gerrit load problems, we
>>> must
>>> run gerrit gc periodically to tidy up the repos, I've ran it today on all
>>> the
>>> repos and ovirt-engine passed from ~20 min to clone to ~2 minutes on my
>>> machine.
>>>
>>> I'll set up a cron to run it periodically, let me know if you see any
>>> issues
>>> with it.
>>>
>>> FYI I've also changed some engine jobs to clone first from github and then
>>> fetch
>>> from gerrit, to ease the load and bandwidth on gerrit itself, I've tested
>>> it on
>>> gerrit and non gerrit jobs before setting it up so no problem should come
>>> from
>>> it, but don't get surprised if you see an extra git repo on the jobs
>>> configs.
>>
>>
>> Looks like it's not working as expected:
>>
>> http://jenkins.ovirt.org/job/ovirt-engine_master_create-rpms-quick_gerrit/2603/console
>>
>
> seems to be working now, maybe a temp error?
> we should however monitor if github starts blocking connections due to load.
> and add another mirror we can use to offload (maybe similar to universities mirrors).
>
>> 09:50:12 Retriggered by user sbonazzo for Gerrit:
>> http://gerrit.ovirt.org/28579
>> 09:50:12 Building remotely on os1-rhel6-vm02 (centos6 os1) in workspace
>> /home/jenkins/workspace/ovirt-engine_master_create-rpms-quick_gerrit
>> 09:50:14 Fetching changes from the remote Git repository
>> 09:50:23 Cleaning workspace
>> 09:50:23 Resetting working tree
>> 09:50:24 Fetching upstream changes from
>> https://github.com/oVirt/ovirt-engine.git
>> 09:50:26 Checking out Revision fdf1b82f8cb6d643a7267535c4b2faeabfe94d1c
>> (origin/master)
>> 09:50:27 FATAL: Could not checkout null with start point
>> fdf1b82f8cb6d643a7267535c4b2faeabfe94d1c
>> 09:50:27 hudson.plugins.git.GitException: Could not checkout null with start
>> point fdf1b82f8cb6d643a7267535c4b2faeabfe94d1c
>> 09:50:27 	at
>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$8.execute(CliGitAPIImpl.java:1448)
>> 09:50:27 	at
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
>> 09:50:27 	at
>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
>> 09:50:27 	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
>> 09:50:27 	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
>> 09:50:27 	at hudson.remoting.Request$2.run(Request.java:328)
>> 09:50:27 	at
>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
>> 09:50:27 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> 09:50:27 	at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> 09:50:27 	at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> 09:50:27 	at java.lang.Thread.run(Thread.java:744)
>> 09:50:27 Caused by: hudson.plugins.git.GitException: Command "git checkout -f
>> fdf1b82f8cb6d643a7267535c4b2faeabfe94d1c" returned status code 128:
>> 09:50:27 stdout:
>> 09:50:27 stderr: fatal: unable to write new index file
>>
>>
>>>
>>>
>>> Cheers!
>>>
>>>
>>>
>>> _______________________________________________
>>> Infra mailing list
>>> Infra at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/infra
>>>
>>
>>
>> --
>> Sandro Bonazzola
>> Better technology. Faster innovation. Powered by community collaboration.
>> See how it works at redhat.com
>> _______________________________________________
>> Infra mailing list
>> Infra at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>

With the garbage collecting it seems that it improved a lot, maybe it's 
not necessary to do this right now, but work looking into though.

--
David Caro

Red Hat S.L.
Continuous Integration Engineer - EMEA ENG Virtualization R&D

Email: dcaro at redhat.com
Web: www.redhat.com
RHT Global #: 82-62605

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20140616/63f8ea49/attachment.sig>