Some insight in the gerrit problem

[David Caro]

One of the problems that I've seen is that xinetd is configured to a low
limit of simultaneous connections and it's dropping them when a few
jenkins jobs are running in parallel.

Changing/adding these options in the /etc/xinit.d/git file should be
enough (those are the actual values):
#       Connection per second and time to wait when reached
        cps             = 50 10
#       Max number of parallel instances
        instances       = 50
#       Max number of connection per source
        per_source      = 10

Here is the trace of one of the failures:


Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X1
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X2
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X2
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X2
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X2
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X2
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X3
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X4
Jun  6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X2
Jun  6 10:30:08 gerrit xinetd[1153]: FAIL: git service_limit
from=::ffff:XX.XX.XX.X5
Jun  6 10:30:08 gerrit xinetd[1153]: EXIT: git status=0 pid=340
duration=1(sec)

### The XX.XX.XX.XX are obfuscated ips


Also I've seen that the number of open files is stable and between
reasonable values, that's good :)
And memory and load seem fine too.


I'll keep monitoring it and see if I see something more, but I recommend
raising the values and see if it's enough of there's another underlying
problem.


-- 
David Caro

Red Hat Czech s.r.o.
Continuous Integration Engineer - EMEA ENG Virtualization R&D

Tel.: +420 532 294 605
Email: dcaro at redhat.com
Web: www.cz.redhat.com
Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno, Czech Republic
RHT Global #: 82-62605

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20130606/8df96d3c/attachment.sig>