On 06/06/2013 05:44 PM, David Caro wrote:
One of the problems that I've seen is that xinetd is configured to a low limit of simultaneous connections and it's dropping them when a few jenkins jobs are running in parallel.
Changing/adding these options in the /etc/xinit.d/git file should be enough (those are the actual values): # Connection per second and time to wait when reached cps = 50 10 # Max number of parallel instances instances = 50 # Max number of connection per source per_source = 10
Here is the trace of one of the failures:
Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X1 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X2 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X2 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X2 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X2 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X2 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X3 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X4 Jun 6 10:30:07 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X2 Jun 6 10:30:08 gerrit xinetd[1153]: FAIL: git service_limit from=::ffff:XX.XX.XX.X5 Jun 6 10:30:08 gerrit xinetd[1153]: EXIT: git status=0 pid=340 duration=1(sec)
### The XX.XX.XX.XX are obfuscated ips
Also I've seen that the number of open files is stable and between reasonable values, that's good :) And memory and load seem fine too.
I'll keep monitoring it and see if I see something more, but I recommend raising the values and see if it's enough of there's another underlying problem.
thanks, please go ahead and do the xinit.d change.