December 2016 - Infra - oVirt List Archives

[JIRA] (OVIRT-919) Fwd: CI slaves extremely slow - overloaded slaves?
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

Barak Korren created OVIRT-919: ---------------------------------- Summary: Fwd: CI slaves extremely slow - overloaded slaves? Key: OVIRT-919 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-919 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Barak Korren Assignee: infra More discussion and tracking needed, moving to Jira (please put any further discussion there) ---------- Forwarded message ---------- From: Nir Soffer <nsoffer(a)redhat.com> Date: 7 December 2016 at 21:33 Subject: CI slaves extremely slow - overloaded slaves? To: infra <infra(a)ovirt.org>, Eyal Edri <eedri(a)redhat.com>, Dan Kenigsberg <danken(a)redhat.com> Hi all, In the last weeks we see more and more test failures due to timeouts in the CI. For example: 17:19:49 ====================================================================== 17:19:49 FAIL: test_scale (storage_filesd_test.GetAllVolumesTests) 17:19:49 ---------------------------------------------------------------------- 17:19:49 Traceback (most recent call last): 17:19:49 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/storage_filesd_test.py", line 165, in test_scale 17:19:49 self.assertTrue(elapsed < 1.0, "Elapsed time: %f seconds" % elapsed) 17:19:49 AssertionError: Elapsed time: 1.105877 seconds 17:19:49 -------------------- >> begin captured stdout << --------------------- 17:19:49 1.105877 seconds This test runs in 0.048 seconds on my laptop: $ ./run_tests_local.sh storage_filesd_test.py -s nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] storage_filesd_test.GetAllVolumesTests test_no_templates OK test_no_volumes OK test_scale 0.047932 seconds OK test_with_template OK ---------------------------------------------------------------------- Ran 4 tests in 0.189s It seems that we are overloading the CI slaves. We should not use nested kvm for the CI, such vms are much slower then regular vms, and we probably run too many vms per cpu. We can disable such tests in the CI, but we do want to know when there is a regression in this code. Before it was fixed, the same test took 9 seconds on my laptop. We need fast machines in the CI for this. Nir _______________________________________________ Infra mailing list Infra(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/infra -- Barak Korren bkorren(a)redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/ -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-918) Special characters escaped in Jenkins manual triggering of CI
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-918?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-918: ------------------------------------ That is not a job, its a special screen that is part of the [Gerrit trigger plugin|https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger] We do seem to be a couple releases behind (2.21.1) the latest plugin release (2.23.0). So we may upgrade, but nothing in the plugin changelog implies that this issue was dealt with. We may as well just go and report this on the [Jenkins issue tracker|http://issues.jenkins-ci.org/secure/IssueNavigator.jspa?mode=hide&r…] > Special characters escaped in Jenkins manual triggering of CI > ------------------------------------------------------------- > > Key: OVIRT-918 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-918 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Jenkins > Reporter: Allon Mureinik > Assignee: infra > > If a subject of a change has special characters, they get HTML-encoded when presenting it in the Jenkins manual trigger job. > See screenshot for an example: > http://imgur.com/G1M7vNu -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

oVirt infra daily report - unstable production jobs - 161
by jenkins＠jenkins.phx.ovirt.org 07 Dec '16

07 Dec '16

------=_Part_41_1154478995.1481151609825 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/161//artifact/exported-a… Cheers, Jenkins ------=_Part_41_1154478995.1481151609825 Content-Type: text/html; charset=us-ascii; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 07/12/2016 </th></tr><tr><th colspan=2 class=sub> <font color="blue"><a href="http://jenkins.ovirt.org/">00 Unstable Jobs (Production)</a></font> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/cockpit-ovirt_ovirt-4.1_build-artifacts-fc23-x…">cockpit-ovirt_ovirt-4.1_build-artifacts-fc23-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/fabric-ovirt_master_check-merged-el7-x86_64/">fabric-ovirt_master_check-merged-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-engine-api-explorer_master_build-artifac…">ovirt-engine-api-explorer_master_build-artifacts-fc23-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-guest-agent_master_build-artifacts-el6-x…">ovirt-guest-agent_master_build-artifacts-el6-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-guest-agent_master_build-artifacts-fc23-…">ovirt-guest-agent_master_build-artifacts-fc23-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-hosted-engine-ha_master_build-artifacts-…">ovirt-hosted-engine-ha_master_build-artifacts-fc24-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-live_master_experimental_create-iso-el7-…">ovirt-live_master_experimental_create-iso-el7-x86_64</a> </td><td> <h3>This job generates a nightly iso of ovirt-live using experimental repos</h3> </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6…">ovirt-node-ng_master_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-3.6_build-artifacts-el7-x8…">ovirt-node-ng_ovirt-3.6_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.0_build-artifacts-el7-x8…">ovirt-node-ng_ovirt-4.0_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.1-pre_build-artifacts-el…">ovirt-node-ng_ovirt-4.1-pre_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-master-experimental_build-…">ovirt-node-ng_ovirt-master-experimental_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node_ovirt-3.6_create-iso-el7_merged/">ovirt-node_ovirt-3.6_create-iso-el7_merged</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-optimizer_4.0_build-artifacts-el6-x86_64/">ovirt-optimizer_4.0_build-artifacts-el6-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/">ovirt_3.6_he-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/repos_master_check-closure_fc24_merged/">repos_master_check-closure_fc24_merged</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_4.0_check-merged-fc23-x86_64/">vdsm_4.0_check-merged-fc23-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_41_1154478995.1481151609825--

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_3.6 - Build #4322 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 07 Dec '16

07 Dec '16

------=_Part_37_255403216.1481135748279 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_3.6/4322/, Build Number: 4322, Build Status: FAILURE ------=_Part_37_255403216.1481135748279--

1 1

[JIRA] (OVIRT-918) Special characters escaped in Jenkins manual triggering of CI
by Allon Mureinik (oVirt JIRA) 07 Dec '16

07 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-918?page=com.atlassian.jira.p… ] Allon Mureinik updated OVIRT-918: --------------------------------- Epic Link: OVIRT-400 > Special characters escaped in Jenkins manual triggering of CI > ------------------------------------------------------------- > > Key: OVIRT-918 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-918 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Jenkins > Reporter: Allon Mureinik > Assignee: infra > > If a subject of a change has special characters, they get HTML-encoded when presenting it in the Jenkins manual trigger job. > See screenshot for an example: > http://imgur.com/G1M7vNu -- This message was sent by Atlassian JIRA (v1000.610.2#100023)

1 0

[JIRA] (OVIRT-918) Special characters escaped in Jenkins manual triggering of CI
by Allon Mureinik (oVirt JIRA) 07 Dec '16

07 Dec '16

Allon Mureinik created OVIRT-918: ------------------------------------ Summary: Special characters escaped in Jenkins manual triggering of CI Key: OVIRT-918 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-918 Project: oVirt - virtualization made easy Issue Type: Bug Components: Jenkins Reporter: Allon Mureinik Assignee: infra If a subject of a change has special characters, they get HTML-encoded when presenting it in the Jenkins manual trigger job. See screenshot for an example: http://imgur.com/G1M7vNu -- This message was sent by Atlassian JIRA (v1000.610.2#100023)

1 0

[JIRA] (OVIRT-917) Investigate creating "permalinks" for ML messages
by Barak Korren (oVirt JIRA) 07 Dec '16

07 Dec '16

Barak Korren created OVIRT-917: ---------------------------------- Summary: Investigate creating "permalinks" for ML messages Key: OVIRT-917 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-917 Project: oVirt - virtualization made easy Issue Type: Improvement Components: Mailing lists (Mailman) Reporter: Barak Korren Assignee: infra This is so messages and threads could be linked to from other messages, blogs, etc. -- This message was sent by Atlassian JIRA (v1000.610.2#100023)

1 0

[JIRA] (OVIRT-917) Investigate creating "permalinks" for ML messages
by Barak Korren (oVirt JIRA) 07 Dec '16

07 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-917?page=com.atlassian.jira.p… ] Barak Korren reassigned OVIRT-917: ---------------------------------- Assignee: Marc Dequènes (Duck) (was: infra) > Investigate creating "permalinks" for ML messages > ------------------------------------------------- > > Key: OVIRT-917 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-917 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Mailing lists (Mailman) > Reporter: Barak Korren > Assignee: Marc Dequènes (Duck) > > This is so messages and threads could be linked to from other messages, blogs, etc. -- This message was sent by Atlassian JIRA (v1000.610.2#100023)

1 0

[JIRA] (OVIRT-917) Investigate creating "permalinks" for ML messages
by Barak Korren (oVirt JIRA) 07 Dec '16

07 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-917?page=com.atlassian.jira.p… ] Barak Korren updated OVIRT-917: ------------------------------- Epic Link: OVIRT-403 > Investigate creating "permalinks" for ML messages > ------------------------------------------------- > > Key: OVIRT-917 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-917 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Mailing lists (Mailman) > Reporter: Barak Korren > Assignee: infra > > This is so messages and threads could be linked to from other messages, blogs, etc. -- This message was sent by Atlassian JIRA (v1000.610.2#100023)

1 0

Vdsm tests are 4X times faster on travis
by Nir Soffer 07 Dec '16

07 Dec '16

HI all, Watching vdsm travis builds in the last weeks, it is clear that vdsm tests on travis are about 4X times faster compared with jenkins builds. Here is a typical build: ovirt ci: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/5101/conso… travis ci: https://travis-ci.org/nirs/vdsm/builds/179056079 The build took 4:34 on travis, and 19:34 on ovirt ci. This has a huge impact on vdsm maintainers. Having to wait 20 minutes for each patch means that we must ignore the ci and merge and hope that previous tests without rebasing on master were good enough. The builds are mostly the same, expect: - In travis we don't check if the build system was changed and packages should be built takes 9:18 minutes in ovirt ci. - In travis we don't clean or install anything before the test, we use a container with all the available packages, pulled from dockerhub. takes about 3:52 minutes in ovirt ci - In travis we don't enable coverage. Running the tests with coverage may slow down the tests takes 5:04 minutes in ovirt ci creating the coverage report takes only 15 seconds, not interesting - In travis we don't cleanup anything after the test this takes 34 seconds in ovirt ci The biggest problem is the build system check taking 9:18 minutes. fixing it will cut the build time in half. This is how time is spent in ovirt ci: 1. Starting (1:28) 00:00:00.001 Triggered by Gerrit: https://gerrit.ovirt.org/67338 00:00:00.039 [EnvInject] - Loading node environment variables. 00:00:00.056 Building remotely on fc24-vm15.phx.ovirt.org (phx nested local_disk fc24) in workspace /home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 2. Installing packages (2:24) 00:01:28.338 INFO: installing package(s): autoconf automake gdb git libguestfs-tools-c libselinux-python3 libvirt-python3 m2crypto make mom openvswitch policycoreutils-python PyYAML python-blivet python-coverage python2-decorator python-devel python-inotify python-ioprocess python-mock python-netaddr python-pthreading python-setuptools python-six python-requests python3-decorator python3-netaddr python3-nose python3-six python3-yaml rpm-build sanlock-python sudo yum yum-utils 3. Setup in check-patch.sh (00:04) 00:03:52.838 + export VDSM_AUTOMATION=1 4. Running the actual tests 00:03:56.670 + ./autogen.sh --system --enable-hooks --enable-vhostmd 5. Installing python-debugifo (00:19) 00:04:15.385 Yum-utils package has been deprecated, use dnf instead. 6. Running make check (05:04) 00:04:30.948 + TIMEOUT=600 00:04:30.948 + make check NOSE_WITH_COVERAGE=1 NOSE_COVER_PACKAGE=/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/vdsm,/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/lib ... 00:09:33.981 tests: commands succeeded 00:09:33.981 congratulations :) 7. Creating coverage report (00:15) 00:09:34.017 + coverage html -d /home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/exported-artifacts/htmlcov 8. Finding modified files (09:18) 00:09:49.213 + git diff-tree --no-commit-id --name-only -r HEAD 00:09:49.213 + egrep --quiet 'vdsm.spec.in|Makefile.am' 9. Cleaing up (00:27) 00:19:07.994 Took 915 seconds 00:19:34.973 Finished: SUCCESS

3 13