From jira at ovirt-jira.atlassian.net Sun Nov 12 17:00:39 2017 Content-Type: multipart/mixed; boundary="===============4924406774440274924==" MIME-Version: 1.0 From: rgolanbb (oVirt JIRA) To: infra at ovirt.org Subject: [JIRA] (OVIRT-1756) configure jenkins.ovirt.org to G1 garbage collector Date: Sun, 12 Nov 2017 17:00:38 +0000 Message-ID: In-Reply-To: JIRA.33288.1510506037000@Atlassian.JIRA --===============4924406774440274924== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format... ------------=3D_1510506038-15316-115 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 7bit rgolanbb created OVIRT-1756: ------------------------------- Summary: configure jenkins.ovirt.org to G1 garbage collector Key: OVIRT-1756 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1756 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: rgolanbb Assignee: infra I noticed jenkins process cpu consumption going over 100% and stalling the web handlers. The io wait is not a problem nor mem. What I suspect is going on is tons of GC and GC pressure given that 12GB heap and fairly nice amount of users and requests. What we can do is to configure the gc logging to see if that is really GC pauses really and to move to using the G1 garbage collector. See this post from cloudbees on the move to G1 collector https://www.cloudbees.com/blog/joining-big-leagues-tuning-jenkins-gc-respon= siveness-and-stability -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100070) ------------=3D_1510506038-15316-115 Content-Type: text/html; charset=3D"UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit

rgolanbb created OVIRT-1756:

   Summary: configure jenkins.ovirt.org to G1 garbage collector
       Key: OVIRT-1756
       URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1756
   Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
  Reporter: rgolanbb
  Assignee: infra

I noticed jenkins process cpu consumption going over 100% and stalling t= he web handlers. The io wait is not a problem nor mem.

What I suspect is going on is tons of GC and GC pressure given that 12GB= heap and fairly nice amount of users and requests.

What we can do is to configure the gc logging to see if that is really G= C pauses really and to move to using the G1 garbage collector.

See this post from cloudbees on the move to G1 collector https://www.cloudbees.com/blog/joining-big-leagues-tuni= ng-jenkins-gc-responsiveness-and-stability

— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#1000= 70)

3D"" ------------=3D_1510506038-15316-115-- --===============4924406774440274924== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4uLgoKLS0tLS0tLS0t LS0tPV8xNTEwNTA2MDM4LTE1MzE2LTExNQpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJz ZXQ9VVRGLTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdAoKcmdvbGFuYmIgY3JlYXRl ZCBPVklSVC0xNzU2OgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgogICAgICAgICAg ICAgU3VtbWFyeTogY29uZmlndXJlIGplbmtpbnMub3ZpcnQub3JnIHRvIEcxIGdhcmJhZ2UgY29s bGVjdG9yCiAgICAgICAgICAgICAgICAgS2V5OiBPVklSVC0xNzU2CiAgICAgICAgICAgICAgICAg VVJMOiBodHRwczovL292aXJ0LWppcmEuYXRsYXNzaWFuLm5ldC9icm93c2UvT1ZJUlQtMTc1Ngog ICAgICAgICAgICAgUHJvamVjdDogb1ZpcnQgLSB2aXJ0dWFsaXphdGlvbiBtYWRlIGVhc3kKICAg ICAgICAgIElzc3VlIFR5cGU6IEJ5LUVNQUlMCiAgICAgICAgICAgIFJlcG9ydGVyOiByZ29sYW5i YgogICAgICAgICAgICBBc3NpZ25lZTogaW5mcmEKCgpJIG5vdGljZWQgamVua2lucyBwcm9jZXNz IGNwdSBjb25zdW1wdGlvbiBnb2luZyBvdmVyIDEwMCUgYW5kIHN0YWxsaW5nIHRoZQp3ZWIgaGFu ZGxlcnMuIFRoZSBpbyB3YWl0IGlzIG5vdCBhIHByb2JsZW0gbm9yIG1lbS4KCldoYXQgSSBzdXNw ZWN0IGlzIGdvaW5nIG9uIGlzIHRvbnMgb2YgR0MgYW5kIEdDIHByZXNzdXJlIGdpdmVuIHRoYXQg MTJHQgpoZWFwIGFuZCBmYWlybHkgbmljZSBhbW91bnQgb2YgdXNlcnMgYW5kIHJlcXVlc3RzLgoK V2hhdCB3ZSBjYW4gZG8gaXMgdG8gY29uZmlndXJlIHRoZSBnYyBsb2dnaW5nIHRvIHNlZSBpZiB0 aGF0IGlzIHJlYWxseSBHQwpwYXVzZXMgcmVhbGx5IGFuZCB0byBtb3ZlIHRvIHVzaW5nIHRoZSBH MSBnYXJiYWdlIGNvbGxlY3Rvci4KClNlZSB0aGlzIHBvc3QgZnJvbSBjbG91ZGJlZXMgb24gdGhl IG1vdmUgdG8gRzEgY29sbGVjdG9yCmh0dHBzOi8vd3d3LmNsb3VkYmVlcy5jb20vYmxvZy9qb2lu aW5nLWJpZy1sZWFndWVzLXR1bmluZy1qZW5raW5zLWdjLXJlc3BvbnNpdmVuZXNzLWFuZC1zdGFi aWxpdHkKCgoKLS0KVGhpcyBtZXNzYWdlIHdhcyBzZW50IGJ5IEF0bGFzc2lhbiBKaXJhCih2MTAw MS4wLjAtU05BUFNIT1QjMTAwMDcwKQoKLS0tLS0tLS0tLS0tPV8xNTEwNTA2MDM4LTE1MzE2LTEx NQpDb250ZW50LVR5cGU6IHRleHQvaHRtbDsgY2hhcnNldD0iVVRGLTgiCkNvbnRlbnQtRGlzcG9z aXRpb246IGlubGluZQpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA3Yml0Cgo8aHRtbD48Ym9k eT4KPGgzPnJnb2xhbmJiIGNyZWF0ZWQgT1ZJUlQtMTc1Njo8L2gzPgo8cHJlPiAgIFN1bW1hcnk6 IGNvbmZpZ3VyZSBqZW5raW5zLm92aXJ0Lm9yZyB0byBHMSBnYXJiYWdlIGNvbGxlY3RvcgogICAg ICAgS2V5OiBPVklSVC0xNzU2CiAgICAgICBVUkw6IGh0dHBzOi8vb3ZpcnQtamlyYS5hdGxhc3Np YW4ubmV0L2Jyb3dzZS9PVklSVC0xNzU2CiAgIFByb2plY3Q6IG9WaXJ0IC0gdmlydHVhbGl6YXRp b24gbWFkZSBlYXN5Cklzc3VlIFR5cGU6IEJ5LUVNQUlMCiAgUmVwb3J0ZXI6IHJnb2xhbmJiCiAg QXNzaWduZWU6IGluZnJhPC9wcmU+CjxwPkkgbm90aWNlZCBqZW5raW5zIHByb2Nlc3MgY3B1IGNv bnN1bXB0aW9uIGdvaW5nIG92ZXIgMTAwJSBhbmQgc3RhbGxpbmcgdGhlIHdlYiBoYW5kbGVycy4g VGhlIGlvIHdhaXQgaXMgbm90IGEgcHJvYmxlbSBub3IgbWVtLjwvcD4KPHA+V2hhdCBJIHN1c3Bl Y3QgaXMgZ29pbmcgb24gaXMgdG9ucyBvZiBHQyBhbmQgR0MgcHJlc3N1cmUgZ2l2ZW4gdGhhdCAx MkdCIGhlYXAgYW5kIGZhaXJseSBuaWNlIGFtb3VudCBvZiB1c2VycyBhbmQgcmVxdWVzdHMuPC9w Pgo8cD5XaGF0IHdlIGNhbiBkbyBpcyB0byBjb25maWd1cmUgdGhlIGdjIGxvZ2dpbmcgdG8gc2Vl IGlmIHRoYXQgaXMgcmVhbGx5IEdDIHBhdXNlcyByZWFsbHkgYW5kIHRvIG1vdmUgdG8gdXNpbmcg dGhlIEcxIGdhcmJhZ2UgY29sbGVjdG9yLjwvcD4KPHA+U2VlIHRoaXMgcG9zdCBmcm9tIGNsb3Vk YmVlcyBvbiB0aGUgbW92ZSB0byBHMSBjb2xsZWN0b3IgPGEgaHJlZj0iaHR0cHM6Ly93d3cuY2xv dWRiZWVzLmNvbS9ibG9nL2pvaW5pbmctYmlnLWxlYWd1ZXMtdHVuaW5nLWplbmtpbnMtZ2MtcmVz cG9uc2l2ZW5lc3MtYW5kLXN0YWJpbGl0eSI+aHR0cHM6Ly93d3cuY2xvdWRiZWVzLmNvbS9ibG9n L2pvaW5pbmctYmlnLWxlYWd1ZXMtdHVuaW5nLWplbmtpbnMtZ2MtcmVzcG9uc2l2ZW5lc3MtYW5k LXN0YWJpbGl0eTwvYT48L3A+CjxwPiZtZGFzaDsgVGhpcyBtZXNzYWdlIHdhcyBzZW50IGJ5IEF0 bGFzc2lhbiBKaXJhICh2MTAwMS4wLjAtU05BUFNIT1QjMTAwMDcwKTwvcD4KCjxpbWcgc3JjPSJo dHRwczovL3U0MDQzNDAyLmN0LnNlbmRncmlkLm5ldC93Zi9vcGVuP3Vwbj1pNVRNV0dWOTlhbUpi TnhKcFNwMi0yQkNtcFlMeXpZUzNyMVE5MzlUMnNnejJCWjJhNXZzTEFBa0VFc0xNYWZON1NybXQ4 ZnNjUENBQlB2c1FsR1pzd0pzRWJsUmRiVGppTDZmdFktMkYtMkJHV1BmYXE3LTJGaDlNb1lHQnZE Z2QwZGpwcExNNmk1VGNEOUY1MWZaM01iV3V4aUJhNEx6VmlneHVubHA1bHIxUk9lNExhSmdhbmhS UW12Z0RsaVJaTWdhcEN1VlpmcnhRYXFrdFBJemdoazQ4VWhaWmRTSWF3blN5bGV2RklVdlExYTlq SUktM0QiIGFsdD0iIiB3aWR0aD0iMSIgaGVpZ2h0PSIxIiBib3JkZXI9IjAiIHN0eWxlPSJoZWln aHQ6MXB4ICFpbXBvcnRhbnQ7d2lkdGg6MXB4ICFpbXBvcnRhbnQ7Ym9yZGVyLXdpZHRoOjAgIWlt cG9ydGFudDttYXJnaW4tdG9wOjAgIWltcG9ydGFudDttYXJnaW4tYm90dG9tOjAgIWltcG9ydGFu dDttYXJnaW4tcmlnaHQ6MCAhaW1wb3J0YW50O21hcmdpbi1sZWZ0OjAgIWltcG9ydGFudDtwYWRk aW5nLXRvcDowICFpbXBvcnRhbnQ7cGFkZGluZy1ib3R0b206MCAhaW1wb3J0YW50O3BhZGRpbmct cmlnaHQ6MCAhaW1wb3J0YW50O3BhZGRpbmctbGVmdDowICFpbXBvcnRhbnQ7Ii8+CjwvYm9keT48 L2h0bWw+CgotLS0tLS0tLS0tLS09XzE1MTA1MDYwMzgtMTUzMTYtMTE1LS0K --===============4924406774440274924==--