From jira at ovirt-jira.atlassian.net Mon Nov 20 22:37:02 2017 Content-Type: multipart/mixed; boundary="===============4703979609716173097==" MIME-Version: 1.0 From: Evgheni Dereveanchin (oVirt JIRA) To: infra at ovirt.org Subject: [JIRA] (OVIRT-1775) system-sync_mirrors-fedora-updates-fc25-x86_64 job sometimes takes 3+ hours to complete Date: Mon, 20 Nov 2017 22:37:01 +0000 Message-ID: In-Reply-To: JIRA.33308.1511217420000@Atlassian.JIRA --===============4703979609716173097== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format... ------------=3D_1511217421-12837-375 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 7bit Evgheni Dereveanchin created OVIRT-1775: ------------------------------------------- Summary: system-sync_mirrors-fedora-updates-fc25-x86_64 job so= metimes takes 3+ hours to complete Key: OVIRT-1775 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1775 Project: oVirt - virtualization made easy Issue Type: Bug Components: oVirt Infra Reporter: Evgheni Dereveanchin Assignee: infra Priority: Low The fc25 mirror sync job takes hours to complete: http://jenkins.ovirt.org/view/All%20Running%20jobs/job/system-sync_mirrors-= fedora-updates-fc25-x86_64/buildTimeTrend = Builds #801, #813, #822, #825, #831, #839 took more than 3 hours to complet= e. Looking at the mirror server during build #839, reposync and genpkgmetad= ata.py processes were consuming resources and constantly waiting in D state= with I/O Wait being around 70% Opening a ticket to investigate the root cause of this slowdown. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100072) ------------=3D_1511217421-12837-375 Content-Type: text/html; charset=3D"UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit

Evgheni Dereveanchin created OVIRT-1775:

   Summary: system-sync_mirrors-fedora-updates-fc25-x86_64 job sometim=
es takes 3+ hours to complete
       Key: OVIRT-1775
       URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1775
   Project: oVirt - virtualization made easy
Issue Type: Bug
Components: oVirt Infra
  Reporter: Evgheni Dereveanchin
  Assignee: infra
  Priority: Low

The fc25 mirror sync job takes hours to complete: http://jenkins.ovirt.org/view/All%20Running= %20jobs/job/system-sync_mirrors-fedora-updates-fc25-x86_64/buildTimeTrend

Builds #801, #813, #822, #825, #831, #839 took more than 3 hours to comp= lete. Looking at the mirror server during build #839, reposync and genpkgme= tadata.py processes were consuming resources and constantly waiting in D st= ate with I/O Wait being around 70%

Opening a ticket to investigate the root cause of this slowdown.

— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#1000= 72)

3D"" ------------=3D_1511217421-12837-375-- --===============4703979609716173097== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4uLgoKLS0tLS0tLS0t LS0tPV8xNTExMjE3NDIxLTEyODM3LTM3NQpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJz ZXQ9VVRGLTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdAoKRXZnaGVuaSBEZXJldmVh bmNoaW4gY3JlYXRlZCBPVklSVC0xNzc1OgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tCgogICAgICAgICAgICAgU3VtbWFyeTogc3lzdGVtLXN5bmNfbWlycm9ycy1m ZWRvcmEtdXBkYXRlcy1mYzI1LXg4Nl82NCBqb2Igc29tZXRpbWVzIHRha2VzIDMrIGhvdXJzIHRv IGNvbXBsZXRlCiAgICAgICAgICAgICAgICAgS2V5OiBPVklSVC0xNzc1CiAgICAgICAgICAgICAg ICAgVVJMOiBodHRwczovL292aXJ0LWppcmEuYXRsYXNzaWFuLm5ldC9icm93c2UvT1ZJUlQtMTc3 NQogICAgICAgICAgICAgUHJvamVjdDogb1ZpcnQgLSB2aXJ0dWFsaXphdGlvbiBtYWRlIGVhc3kK ICAgICAgICAgIElzc3VlIFR5cGU6IEJ1ZwogICAgICAgICAgQ29tcG9uZW50czogb1ZpcnQgSW5m cmEKICAgICAgICAgICAgUmVwb3J0ZXI6IEV2Z2hlbmkgRGVyZXZlYW5jaGluCiAgICAgICAgICAg IEFzc2lnbmVlOiBpbmZyYQogICAgICAgICAgICBQcmlvcml0eTogTG93CgoKVGhlIGZjMjUgbWly cm9yIHN5bmMgam9iIHRha2VzIGhvdXJzIHRvIGNvbXBsZXRlOgpodHRwOi8vamVua2lucy5vdmly dC5vcmcvdmlldy9BbGwlMjBSdW5uaW5nJTIwam9icy9qb2Ivc3lzdGVtLXN5bmNfbWlycm9ycy1m ZWRvcmEtdXBkYXRlcy1mYzI1LXg4Nl82NC9idWlsZFRpbWVUcmVuZCAKCkJ1aWxkcyAjODAxLCAj ODEzLCAjODIyLCAjODI1LCAjODMxLCAjODM5IHRvb2sgbW9yZSB0aGFuIDMgaG91cnMgdG8gY29t cGxldGUuIExvb2tpbmcgYXQgdGhlIG1pcnJvciBzZXJ2ZXIgZHVyaW5nIGJ1aWxkICM4MzksIHJl cG9zeW5jIGFuZCBnZW5wa2dtZXRhZGF0YS5weSBwcm9jZXNzZXMgd2VyZSBjb25zdW1pbmcgcmVz b3VyY2VzIGFuZCBjb25zdGFudGx5IHdhaXRpbmcgaW4gRCBzdGF0ZSB3aXRoIEkvTyBXYWl0IGJl aW5nIGFyb3VuZCA3MCUKCk9wZW5pbmcgYSB0aWNrZXQgdG8gaW52ZXN0aWdhdGUgdGhlIHJvb3Qg Y2F1c2Ugb2YgdGhpcyBzbG93ZG93bi4KCgoKLS0KVGhpcyBtZXNzYWdlIHdhcyBzZW50IGJ5IEF0 bGFzc2lhbiBKaXJhCih2MTAwMS4wLjAtU05BUFNIT1QjMTAwMDcyKQoKLS0tLS0tLS0tLS0tPV8x NTExMjE3NDIxLTEyODM3LTM3NQpDb250ZW50LVR5cGU6IHRleHQvaHRtbDsgY2hhcnNldD0iVVRG LTgiCkNvbnRlbnQtRGlzcG9zaXRpb246IGlubGluZQpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5n OiA3Yml0Cgo8aHRtbD48Ym9keT4KPGgzPkV2Z2hlbmkgRGVyZXZlYW5jaGluIGNyZWF0ZWQgT1ZJ UlQtMTc3NTo8L2gzPgo8cHJlPiAgIFN1bW1hcnk6IHN5c3RlbS1zeW5jX21pcnJvcnMtZmVkb3Jh LXVwZGF0ZXMtZmMyNS14ODZfNjQgam9iIHNvbWV0aW1lcyB0YWtlcyAzKyBob3VycyB0byBjb21w bGV0ZQogICAgICAgS2V5OiBPVklSVC0xNzc1CiAgICAgICBVUkw6IGh0dHBzOi8vb3ZpcnQtamly YS5hdGxhc3NpYW4ubmV0L2Jyb3dzZS9PVklSVC0xNzc1CiAgIFByb2plY3Q6IG9WaXJ0IC0gdmly dHVhbGl6YXRpb24gbWFkZSBlYXN5Cklzc3VlIFR5cGU6IEJ1ZwpDb21wb25lbnRzOiBvVmlydCBJ bmZyYQogIFJlcG9ydGVyOiBFdmdoZW5pIERlcmV2ZWFuY2hpbgogIEFzc2lnbmVlOiBpbmZyYQog IFByaW9yaXR5OiBMb3c8L3ByZT4KPHA+VGhlIGZjMjUgbWlycm9yIHN5bmMgam9iIHRha2VzIGhv dXJzIHRvIGNvbXBsZXRlOiA8YSBocmVmPSJodHRwOi8vamVua2lucy5vdmlydC5vcmcvdmlldy9B bGwlMjBSdW5uaW5nJTIwam9icy9qb2Ivc3lzdGVtLXN5bmNfbWlycm9ycy1mZWRvcmEtdXBkYXRl cy1mYzI1LXg4Nl82NC9idWlsZFRpbWVUcmVuZCI+aHR0cDovL2plbmtpbnMub3ZpcnQub3JnL3Zp ZXcvQWxsJTIwUnVubmluZyUyMGpvYnMvam9iL3N5c3RlbS1zeW5jX21pcnJvcnMtZmVkb3JhLXVw ZGF0ZXMtZmMyNS14ODZfNjQvYnVpbGRUaW1lVHJlbmQ8L2E+PC9wPgo8cD5CdWlsZHMgIzgwMSwg IzgxMywgIzgyMiwgIzgyNSwgIzgzMSwgIzgzOSB0b29rIG1vcmUgdGhhbiAzIGhvdXJzIHRvIGNv bXBsZXRlLiBMb29raW5nIGF0IHRoZSBtaXJyb3Igc2VydmVyIGR1cmluZyBidWlsZCAjODM5LCBy ZXBvc3luYyBhbmQgZ2VucGtnbWV0YWRhdGEucHkgcHJvY2Vzc2VzIHdlcmUgY29uc3VtaW5nIHJl c291cmNlcyBhbmQgY29uc3RhbnRseSB3YWl0aW5nIGluIEQgc3RhdGUgd2l0aCBJL08gV2FpdCBi ZWluZyBhcm91bmQgNzAlPC9wPgo8cD5PcGVuaW5nIGEgdGlja2V0IHRvIGludmVzdGlnYXRlIHRo ZSByb290IGNhdXNlIG9mIHRoaXMgc2xvd2Rvd24uPC9wPgo8cD4mbWRhc2g7IFRoaXMgbWVzc2Fn ZSB3YXMgc2VudCBieSBBdGxhc3NpYW4gSmlyYSAodjEwMDEuMC4wLVNOQVBTSE9UIzEwMDA3Mik8 L3A+Cgo8aW1nIHNyYz0iaHR0cHM6Ly91NDA0MzQwMi5jdC5zZW5kZ3JpZC5uZXQvd2Yvb3Blbj91 cG49aTVUTVdHVjk5YW1KYk54SnBTcDItMkJDbXBZTHl6WVMzcjFROTM5VDJzZ3ozR3lZYXdYc01z T1FOUWJSbTYtMkY1Ui0yRnE2VmYyTkpIV1lXMTZhc1V5aHJyUzZBQW80c3ctMkI3OUVhZ1gzdHNI ODZtMC0yQkQ0U0M1NmpraGdTWWVGcFpMNkNNamYwb1RtTlB6eVdFTUNsN2lyS2xGLTJCRjJlb3lN V2hPMEV3LTJCSkhFRlRUQVlTdDExZm9vT2xndGZmRkNXc01GUEVIc0hqSUZXRFRsa0pvQklsUmpN NWRucHBXNnRscy0yQjFyaENMWjlEOWctMkJZby0zRCIgYWx0PSIiIHdpZHRoPSIxIiBoZWlnaHQ9 IjEiIGJvcmRlcj0iMCIgc3R5bGU9ImhlaWdodDoxcHggIWltcG9ydGFudDt3aWR0aDoxcHggIWlt cG9ydGFudDtib3JkZXItd2lkdGg6MCAhaW1wb3J0YW50O21hcmdpbi10b3A6MCAhaW1wb3J0YW50 O21hcmdpbi1ib3R0b206MCAhaW1wb3J0YW50O21hcmdpbi1yaWdodDowICFpbXBvcnRhbnQ7bWFy Z2luLWxlZnQ6MCAhaW1wb3J0YW50O3BhZGRpbmctdG9wOjAgIWltcG9ydGFudDtwYWRkaW5nLWJv dHRvbTowICFpbXBvcnRhbnQ7cGFkZGluZy1yaWdodDowICFpbXBvcnRhbnQ7cGFkZGluZy1sZWZ0 OjAgIWltcG9ydGFudDsiLz4KPC9ib2R5PjwvaHRtbD4KCi0tLS0tLS0tLS0tLT1fMTUxMTIxNzQy MS0xMjgzNy0zNzUtLQo= --===============4703979609716173097==--