[JIRA] (OVIRT-2586) Jenkins terribly slow and unresponsive

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2586?page=com.atlassian.jir... ] Evgheni Dereveanchin commented on OVIRT-2586: --------------------------------------------- I planned to reboot Jenkins tonight but it was busy running pipelines that aren't easy to cancel. As a result, Jenkins completely locked up in the morning with the UI being completely unreachable and backend threads timing out in the background. Had to restart it and it's now coming back up. The monitoring plugin was partially responsive still during the outage and showed the following info: |Java memory used: |15,590 Mb / 16,384 Mb *Usage is near the maximum, you may need to optimize or to reconfigure (-Xmx)| |Nb of http sessions: |8 | |Nb of active threads (current http requests): |33 | |System load |2.78 | |% System CPU |17.17| Almost all memory got exhausted which is likely caused by a memory leak in the SSE-gateway plugin coinciding with a large number of CI jobs appearing in the queue. Adding memory to Java will likely just delay the symptoms as the memory leak is still there (see JENKINS-51057) !usedMemory_year.png|thumbnail! From the yearly memory graph the leak started around May-June this year and intensified in November. To confirm the exact root cause we may need some lower-level troubleshooting of the Java process yet I am not familiar with how that's done. [~mwperina] maybe you can assist with the info that can be gathered to identify the root cause? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100095)