December 2016 - Infra - oVirt List Archives

[JIRA] (OVIRT-920) Add a standard-CI job and button "build-scratch-artifacts"
by Yedidyah Bar David (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-920?page=com.atlassian.jira.p… ] Yedidyah Bar David commented on OVIRT-920: ------------------------------------------ On Thu, Dec 8, 2016 at 10:13 AM, eyal edri [Administrator] (oVirt JIRA) < Fine with me, but then please add to the page a link somewhere to the list of possible commands. I commented "Rerun-Hooks: all" tens of times so far, and in each and every one of them had to search for the exact syntax - never managed to memorize it. -- Didi > Add a standard-CI job and button "build-scratch-artifacts" > ---------------------------------------------------------- > > Key: OVIRT-920 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-920 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Yedidyah Bar David > Assignee: infra > > Hi, > I think it will be useful to have an option to very easily (such as > pressing a button in gerrit) to do a full build of a project, > similarly to current build-artifacts, but for a pending patch - so it > will not be collected by any publisher, and will keep its artifacts > for a short time (say, 2 days). > This will allow to use these artifacts for manual tests, perhaps > longer than check-patch, in case a developer wants to do them. > -- > Didi -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-920) Add a standard-CI job and button "build-scratch-artifacts"
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-920?page=com.atlassian.jira.p… ] eyal edri [Administrator] commented on OVIRT-920: ------------------------------------------------- We might be able to do it but not with a button, but with a comment, similar to what GitHub has ( for e.g in Lago we do "ci test please" to run a job on PR ). This still needs checking, but it might be possible via adding new 'build-artifacts-from-patch.sh' script added to standard CI which will run the same code as build-artifacts (it can symlink to the build-artifacts if there is no diff between it to normal build-artifacts) and the trigger for it will be done by typing a comment in the patch, something like "jenkins please build". We'll implement it by adding a yaml patch with new template for the gerrit trigger plugin with the 'comment added contains regular expression' option. > Add a standard-CI job and button "build-scratch-artifacts" > ---------------------------------------------------------- > > Key: OVIRT-920 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-920 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Yedidyah Bar David > Assignee: infra > > Hi, > I think it will be useful to have an option to very easily (such as > pressing a button in gerrit) to do a full build of a project, > similarly to current build-artifacts, but for a pending patch - so it > will not be collected by any publisher, and will keep its artifacts > for a short time (say, 2 days). > This will allow to use these artifacts for manual tests, perhaps > longer than check-patch, in case a developer wants to do them. > -- > Didi -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-904) False CI-1 due to unreachable copr-be.cloud.fedoraproject.org
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-904?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-904: ------------------------------------ [~gshereme(a)redhat.com] are you dong that manually, or do you have some automation built around it? I'm not sure oVirt's own infra would be the best place, because Patternfly is not strictly an oVirt project. Perhaps make travis run the build in Copr, and then take the result and push it to GitHub's artifact storage? Another possibility is to use [BinTray|https://bintray.com/] We might eventually mirror this in oVirt infra to make CI resilient to changes there (see OVIRT-575) > False CI-1 due to unreachable copr-be.cloud.fedoraproject.org > ------------------------------------------------------------- > > Key: OVIRT-904 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-904 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: danken > Assignee: infra > > Many patches on > https://gerrit.ovirt.org/#/q/topic:passthroughMigration fail CI due to > failure: repodata/repomd.xml from patternfly: [Errno 256] No more > mirrors to try. > http://copr-be.cloud.fedoraproject.org/results/patternfly/patternfly1/epel-…: > [Errno 14] curl#7 - "Failed connect to > copr-be.cloud.fedoraproject.org:80; No route to host" > http://copr-be.cloud.fedoraproject.org/results/patternfly/patternfly1/epel-… -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-920) Add a standard-CI job and button "build-scratch-artifacts"
by Yedidyah Bar David (oVirt JIRA) 08 Dec '16

08 Dec '16

Yedidyah Bar David created OVIRT-920: ---------------------------------------- Summary: Add a standard-CI job and button "build-scratch-artifacts" Key: OVIRT-920 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-920 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Yedidyah Bar David Assignee: infra Hi, I think it will be useful to have an option to very easily (such as pressing a button in gerrit) to do a full build of a project, similarly to current build-artifacts, but for a pending patch - so it will not be collected by any publisher, and will keep its artifacts for a short time (say, 2 days). This will allow to use these artifacts for manual tests, perhaps longer than check-patch, in case a developer wants to do them. -- Didi -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-919) Fwd: CI slaves extremely slow - overloaded slaves?
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-919?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-919: ------------------------------------ {quote} ... snip ... 17:19:49 1.105877 seconds This test runs in 0.048 seconds on my laptop: ... snip ... Ran 4 tests in 0.189s {quote} [~nsoffer(a)redhat.com] Lets make sure we're comparing apples and apples first, did you run this using "{{mock_runner.sh}}" on your laptop? Perhaps try this in a VM on your laptop? {quote} It seems that we are overloading the CI slaves. We should not use nested kvm for the CI, such vms are much slower then regular vms, and we probably run too many vms per cpu. {quote} Not sure where nested kvm comes into play here, "{{check_patch.sh}}" runs on plain VMs. (As an OT side-note: Are you sure nested VMs should be slower CPU-wise? AFAIK those are just VMs with nested I/O routing... it may be interesting to check this with Lago...) WRT "overloaded slaves" - that can hardly happen, no slave ever runs more then one job at at time. The hypervisors themselves may be loaded, we'll try to gather some more data here: [~ngoldin(a)redhat.com] Do we have hypervisor load graphs in Graphite? [~ederevea] What is out vm/core ratio? Are we running oVirt DWH? maybe it can help us here... {quote} We can disable such tests in the CI, but we do want to know when there is a regression in this code. Before it was fixed, the same test took 9 seconds on my laptop. We need fast machines in the CI for this. {quote} As far as machines go, our upstream hardware is quite beefy these days, so lets not jump to any conclusions. More data is needed, what is this test actually doing? is this pure CPU, or is there so I/O involved here as well? [~rgolan(a)redhat.com] Are there any oVirt features that can help us here? Higher priority VMs? Realtime? Better scheduling? > Fwd: CI slaves extremely slow - overloaded slaves? > -------------------------------------------------- > > Key: OVIRT-919 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-919 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins > Reporter: Barak Korren > Assignee: infra > Labels: hypervisors, performance, slaves, standard-ci > > From: Nir Soffer <nsoffer(a)redhat.com> > Date: 7 December 2016 at 21:33 > Subject: CI slaves extremely slow - overloaded slaves? > To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan > Kenigsberg <danken(a)redhat.com> > Hi all, > In the last weeks we see more and more test failures due to timeouts in the CI. > For example: > 17:19:49 ====================================================================== > 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) > 17:19:49 ---------------------------------------------------------------------- > 17:19:49 Traceback (most recent call last): > 17:19:49 File > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", > line 165, in test_scale > 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" > % elapsed) > 17:19:49 AssertionError: Elapsed time: 1.105877 seconds > 17:19:49 -------------------- >> begin captured stdout << --------------------- > 17:19:49 1.105877 seconds > This test runs in 0.048 seconds on my laptop: > $ ./run_tests_local.sh storage_filesd_test.py -s > nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] > storage_filesd_test.GetAllVolumesTests > test_no_templates OK > test_no_volumes OK > test_scale 0.047932 seconds > OK > test_with_template OK > ---------------------------------------------------------------------- > Ran 4 tests in 0.189s > It seems that we are overloading the CI slaves. We should not use nested kvm > for the CI, such vms are much slower then regular vms, and we probably run > too many vms per cpu. > We can disable such tests in the CI, but we do want to know when there is > a regression in this code. Before it was fixed, the same test took 9 seconds > on my laptop. We need fast machines in the CI for this. > Nir -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-919) Fwd: CI slaves extremely slow - overloaded slaves?
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-919?page=com.atlassian.jira.p… ] Barak Korren updated OVIRT-919: ------------------------------- Labels: hypervisors performance slaves standard-ci (was: ) > Fwd: CI slaves extremely slow - overloaded slaves? > -------------------------------------------------- > > Key: OVIRT-919 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-919 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins > Reporter: Barak Korren > Assignee: infra > Labels: hypervisors, performance, slaves, standard-ci > > From: Nir Soffer <nsoffer(a)redhat.com> > Date: 7 December 2016 at 21:33 > Subject: CI slaves extremely slow - overloaded slaves? > To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan > Kenigsberg <danken(a)redhat.com> > Hi all, > In the last weeks we see more and more test failures due to timeouts in the CI. > For example: > 17:19:49 ====================================================================== > 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) > 17:19:49 ---------------------------------------------------------------------- > 17:19:49 Traceback (most recent call last): > 17:19:49 File > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", > line 165, in test_scale > 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" > % elapsed) > 17:19:49 AssertionError: Elapsed time: 1.105877 seconds > 17:19:49 -------------------- >> begin captured stdout << --------------------- > 17:19:49 1.105877 seconds > This test runs in 0.048 seconds on my laptop: > $ ./run_tests_local.sh storage_filesd_test.py -s > nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] > storage_filesd_test.GetAllVolumesTests > test_no_templates OK > test_no_volumes OK > test_scale 0.047932 seconds > OK > test_with_template OK > ---------------------------------------------------------------------- > Ran 4 tests in 0.189s > It seems that we are overloading the CI slaves. We should not use nested kvm > for the CI, such vms are much slower then regular vms, and we probably run > too many vms per cpu. > We can disable such tests in the CI, but we do want to know when there is > a regression in this code. Before it was fixed, the same test took 9 seconds > on my laptop. We need fast machines in the CI for this. > Nir -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-919) Fwd: CI slaves extremely slow - overloaded slaves?
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-919?page=com.atlassian.jira.p… ] Barak Korren updated OVIRT-919: ------------------------------- Component/s: Jenkins > Fwd: CI slaves extremely slow - overloaded slaves? > -------------------------------------------------- > > Key: OVIRT-919 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-919 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Jenkins > Reporter: Barak Korren > Assignee: infra > > From: Nir Soffer <nsoffer(a)redhat.com> > Date: 7 December 2016 at 21:33 > Subject: CI slaves extremely slow - overloaded slaves? > To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan > Kenigsberg <danken(a)redhat.com> > Hi all, > In the last weeks we see more and more test failures due to timeouts in the CI. > For example: > 17:19:49 ====================================================================== > 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) > 17:19:49 ---------------------------------------------------------------------- > 17:19:49 Traceback (most recent call last): > 17:19:49 File > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", > line 165, in test_scale > 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" > % elapsed) > 17:19:49 AssertionError: Elapsed time: 1.105877 seconds > 17:19:49 -------------------- >> begin captured stdout << --------------------- > 17:19:49 1.105877 seconds > This test runs in 0.048 seconds on my laptop: > $ ./run_tests_local.sh storage_filesd_test.py -s > nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] > storage_filesd_test.GetAllVolumesTests > test_no_templates OK > test_no_volumes OK > test_scale 0.047932 seconds > OK > test_with_template OK > ---------------------------------------------------------------------- > Ran 4 tests in 0.189s > It seems that we are overloading the CI slaves. We should not use nested kvm > for the CI, such vms are much slower then regular vms, and we probably run > too many vms per cpu. > We can disable such tests in the CI, but we do want to know when there is > a regression in this code. Before it was fixed, the same test took 9 seconds > on my laptop. We need fast machines in the CI for this. > Nir -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

CI slaves extremely slow - overloaded slaves?
by Nir Soffer 08 Dec '16

08 Dec '16

Hi all, In the last weeks we see more and more test failures due to timeouts in the CI. For example: 17:19:49 ====================================================================== 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) 17:19:49 ---------------------------------------------------------------------- 17:19:49 Traceback (most recent call last): 17:19:49 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", line 165, in test_scale 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" % elapsed) 17:19:49 AssertionError: Elapsed time: 1.105877 seconds 17:19:49 -------------------- >> begin captured stdout << --------------------- 17:19:49 1.105877 seconds This test runs in 0.048 seconds on my laptop: $ ./run_tests_local.sh storage_filesd_test.py -s nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] storage_filesd_test.GetAllVolumesTests test_no_templates OK test_no_volumes OK test_scale 0.047932 seconds OK test_with_template OK ---------------------------------------------------------------------- Ran 4 tests in 0.189s It seems that we are overloading the CI slaves. We should not use nested kvm for the CI, such vms are much slower then regular vms, and we probably run too many vms per cpu. We can disable such tests in the CI, but we do want to know when there is a regression in this code. Before it was fixed, the same test took 9 seconds on my laptop. We need fast machines in the CI for this. Nir

2 1

[JIRA] (OVIRT-919) Fwd: CI slaves extremely slow - overloaded slaves?
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-919?page=com.atlassian.jira.p… ] Barak Korren updated OVIRT-919: ------------------------------- Description: From: Nir Soffer <nsoffer(a)redhat.com> Date: 7 December 2016 at 21:33 Subject: CI slaves extremely slow - overloaded slaves? To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan Kenigsberg <danken(a)redhat.com> Hi all, In the last weeks we see more and more test failures due to timeouts in the CI. For example: 17:19:49 ====================================================================== 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) 17:19:49 ---------------------------------------------------------------------- 17:19:49 Traceback (most recent call last): 17:19:49 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", line 165, in test_scale 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" % elapsed) 17:19:49 AssertionError: Elapsed time: 1.105877 seconds 17:19:49 -------------------- >> begin captured stdout << --------------------- 17:19:49 1.105877 seconds This test runs in 0.048 seconds on my laptop: $ ./run_tests_local.sh storage_filesd_test.py -s nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] storage_filesd_test.GetAllVolumesTests test_no_templates OK test_no_volumes OK test_scale 0.047932 seconds OK test_with_template OK ---------------------------------------------------------------------- Ran 4 tests in 0.189s It seems that we are overloading the CI slaves. We should not use nested kvm for the CI, such vms are much slower then regular vms, and we probably run too many vms per cpu. We can disable such tests in the CI, but we do want to know when there is a regression in this code. Before it was fixed, the same test took 9 seconds on my laptop. We need fast machines in the CI for this. Nir _______________________________________________ Infra mailing list Infra(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/infra -- Barak Korren bkorren(a)redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ was: More discussion and tracking needed, moving to Jira (please put any further discussion there) ---------- Forwarded message ---------- From: Nir Soffer <nsoffer(a)redhat.com> Date: 7 December 2016 at 21:33 Subject: CI slaves extremely slow - overloaded slaves? To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan Kenigsberg <danken(a)redhat.com> Hi all, In the last weeks we see more and more test failures due to timeouts in the CI. For example: 17:19:49 ====================================================================== 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) 17:19:49 ---------------------------------------------------------------------- 17:19:49 Traceback (most recent call last): 17:19:49 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", line 165, in test_scale 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" % elapsed) 17:19:49 AssertionError: Elapsed time: 1.105877 seconds 17:19:49 -------------------- >> begin captured stdout << --------------------- 17:19:49 1.105877 seconds This test runs in 0.048 seconds on my laptop: $ ./run_tests_local.sh storage_filesd_test.py -s nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] storage_filesd_test.GetAllVolumesTests test_no_templates OK test_no_volumes OK test_scale 0.047932 seconds OK test_with_template OK ---------------------------------------------------------------------- Ran 4 tests in 0.189s It seems that we are overloading the CI slaves. We should not use nested kvm for the CI, such vms are much slower then regular vms, and we probably run too many vms per cpu. We can disable such tests in the CI, but we do want to know when there is a regression in this code. Before it was fixed, the same test took 9 seconds on my laptop. We need fast machines in the CI for this. Nir _______________________________________________ Infra mailing list Infra(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/infra -- Barak Korren bkorren(a)redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ > Fwd: CI slaves extremely slow - overloaded slaves? > -------------------------------------------------- > > Key: OVIRT-919 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-919 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Barak Korren > Assignee: infra > > From: Nir Soffer <nsoffer(a)redhat.com> > Date: 7 December 2016 at 21:33 > Subject: CI slaves extremely slow - overloaded slaves? > To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan > Kenigsberg <danken(a)redhat.com> > Hi all, > In the last weeks we see more and more test failures due to timeouts in the CI. > For example: > 17:19:49 ====================================================================== > 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) > 17:19:49 ---------------------------------------------------------------------- > 17:19:49 Traceback (most recent call last): > 17:19:49 File > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", > line 165, in test_scale > 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" > % elapsed) > 17:19:49 AssertionError: Elapsed time: 1.105877 seconds > 17:19:49 -------------------- >> begin captured stdout << --------------------- > 17:19:49 1.105877 seconds > This test runs in 0.048 seconds on my laptop: > $ ./run_tests_local.sh storage_filesd_test.py -s > nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] > storage_filesd_test.GetAllVolumesTests > test_no_templates OK > test_no_volumes OK > test_scale 0.047932 seconds > OK > test_with_template OK > ---------------------------------------------------------------------- > Ran 4 tests in 0.189s > It seems that we are overloading the CI slaves. We should not use nested kvm > for the CI, such vms are much slower then regular vms, and we probably run > too many vms per cpu. > We can disable such tests in the CI, but we do want to know when there is > a regression in this code. Before it was fixed, the same test took 9 seconds > on my laptop. We need fast machines in the CI for this. > Nir > _______________________________________________ > Infra mailing list > Infra(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > -- > Barak Korren > bkorren(a)redhat.com > RHCE, RHCi, RHV-DevOps Team > https://ifireball.wordpress.com/ -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-919) Fwd: CI slaves extremely slow - overloaded slaves?
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-919?page=com.atlassian.jira.p… ] Barak Korren updated OVIRT-919: ------------------------------- Description: From: Nir Soffer <nsoffer(a)redhat.com> Date: 7 December 2016 at 21:33 Subject: CI slaves extremely slow - overloaded slaves? To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan Kenigsberg <danken(a)redhat.com> Hi all, In the last weeks we see more and more test failures due to timeouts in the CI. For example: 17:19:49 ====================================================================== 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) 17:19:49 ---------------------------------------------------------------------- 17:19:49 Traceback (most recent call last): 17:19:49 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", line 165, in test_scale 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" % elapsed) 17:19:49 AssertionError: Elapsed time: 1.105877 seconds 17:19:49 -------------------- >> begin captured stdout << --------------------- 17:19:49 1.105877 seconds This test runs in 0.048 seconds on my laptop: $ ./run_tests_local.sh storage_filesd_test.py -s nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] storage_filesd_test.GetAllVolumesTests test_no_templates OK test_no_volumes OK test_scale 0.047932 seconds OK test_with_template OK ---------------------------------------------------------------------- Ran 4 tests in 0.189s It seems that we are overloading the CI slaves. We should not use nested kvm for the CI, such vms are much slower then regular vms, and we probably run too many vms per cpu. We can disable such tests in the CI, but we do want to know when there is a regression in this code. Before it was fixed, the same test took 9 seconds on my laptop. We need fast machines in the CI for this. Nir was: From: Nir Soffer <nsoffer(a)redhat.com> Date: 7 December 2016 at 21:33 Subject: CI slaves extremely slow - overloaded slaves? To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan Kenigsberg <danken(a)redhat.com> Hi all, In the last weeks we see more and more test failures due to timeouts in the CI. For example: 17:19:49 ====================================================================== 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) 17:19:49 ---------------------------------------------------------------------- 17:19:49 Traceback (most recent call last): 17:19:49 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", line 165, in test_scale 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" % elapsed) 17:19:49 AssertionError: Elapsed time: 1.105877 seconds 17:19:49 -------------------- >> begin captured stdout << --------------------- 17:19:49 1.105877 seconds This test runs in 0.048 seconds on my laptop: $ ./run_tests_local.sh storage_filesd_test.py -s nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] storage_filesd_test.GetAllVolumesTests test_no_templates OK test_no_volumes OK test_scale 0.047932 seconds OK test_with_template OK ---------------------------------------------------------------------- Ran 4 tests in 0.189s It seems that we are overloading the CI slaves. We should not use nested kvm for the CI, such vms are much slower then regular vms, and we probably run too many vms per cpu. We can disable such tests in the CI, but we do want to know when there is a regression in this code. Before it was fixed, the same test took 9 seconds on my laptop. We need fast machines in the CI for this. Nir _______________________________________________ Infra mailing list Infra(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/infra -- Barak Korren bkorren(a)redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ > Fwd: CI slaves extremely slow - overloaded slaves? > -------------------------------------------------- > > Key: OVIRT-919 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-919 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Barak Korren > Assignee: infra > > From: Nir Soffer <nsoffer(a)redhat.com> > Date: 7 December 2016 at 21:33 > Subject: CI slaves extremely slow - overloaded slaves? > To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan > Kenigsberg <danken(a)redhat.com> > Hi all, > In the last weeks we see more and more test failures due to timeouts in the CI. > For example: > 17:19:49 ====================================================================== > 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) > 17:19:49 ---------------------------------------------------------------------- > 17:19:49 Traceback (most recent call last): > 17:19:49 File > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", > line 165, in test_scale > 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" > % elapsed) > 17:19:49 AssertionError: Elapsed time: 1.105877 seconds > 17:19:49 -------------------- >> begin captured stdout << --------------------- > 17:19:49 1.105877 seconds > This test runs in 0.048 seconds on my laptop: > $ ./run_tests_local.sh storage_filesd_test.py -s > nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] > storage_filesd_test.GetAllVolumesTests > test_no_templates OK > test_no_volumes OK > test_scale 0.047932 seconds > OK > test_with_template OK > ---------------------------------------------------------------------- > Ran 4 tests in 0.189s > It seems that we are overloading the CI slaves. We should not use nested kvm > for the CI, such vms are much slower then regular vms, and we probably run > too many vms per cpu. > We can disable such tests in the CI, but we do want to know when there is > a regression in this code. Before it was fixed, the same test took 9 seconds > on my laptop. We need fast machines in the CI for this. > Nir -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0