- Infra - oVirt List Archives

RHEVM CI Jenkins daily report - 14
by jenkins＠jenkins.phx.ovirt.org 15 Jul '16

15 Jul '16

------=_Part_221_1339366229.1468537203606 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/14//artifact/exported-ar… Cheers, Jenkins ------=_Part_221_1339366229.1468537203606 Content-Type: application/octet-stream; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 14/07/2016 </th></tr><tr><th colspan=2 class=sub> <font color="blue"><a href="http://jenkins.ovirt.org/">00 Unstable Jobs (Production)</a></font> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/deploy-to-ovirt_experimental_3.6/">deploy-to-ovirt_experimental_3.6</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.0-pre_build-artifacts-…">ovirt-appliance_ovirt-4.0-pre_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-dwh_3.6_build-artifacts-el6-x86_64/">ovirt-dwh_3.6_build-artifacts-el6-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-dwh_3.6_check-merged-el6-x86_64/">ovirt-dwh_3.6_check-merged-el6-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-engine_master_coverity-analysis/">ovirt-engine_master_coverity-analysis</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-hosted-engine-ha_master_create-rpms-fc24…">ovirt-hosted-engine-ha_master_create-rpms-fc24-x86_64_merged</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-fc22-x86_…">ovirt-node-ng_master_build-artifacts-fc22-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-3.6_build-artifacts-fc22-x…">ovirt-node-ng_ovirt-3.6_build-artifacts-fc22-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.0_build-artifacts-fc22-x…">ovirt-node-ng_ovirt-4.0_build-artifacts-fc22-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-plugin-hosted-engine_ovirt-3.6_chec…">ovirt-node-plugin-hosted-engine_ovirt-3.6_check-merged-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-optimizer_4.0_build-artifacts-fc24-x86_6…">ovirt-optimizer_4.0_build-artifacts-fc24-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-optimizer_master_build-artifacts-fc24-x8…">ovirt-optimizer_master_build-artifacts-fc24-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_master_publish-rpms_nightly/">ovirt_master_publish-rpms_nightly</a> </td><td> Collects RPMs from all oVirt projects (master branches) and publish them to resources.ovirt.org yum repository.<br> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/repos_master_check-closure_merged/">repos_master_check-closure_merged</a> </td><td> No Description </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_3.6/">test-repo_ovirt_experimental_3.6</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_4.0_check-merged-fc24-x86_64/">vdsm_4.0_check-merged-fc24-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc24-x86_64/">vdsm_master_build-artifacts-fc24-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_master_check-merged-fc24-x86_64/">vdsm_master_check-merged-fc24-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_221_1339366229.1468537203606--

1 0

Build failed in Jenkins: ovirt_4.0_he-system-tests #13
by jenkins＠jenkins.phx.ovirt.org 14 Jul '16

14 Jul '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/13/changes> Changes: [Lev Veyde] ovirt-system-tests: Remove ovirt-engine-appliance exclusion from el7 [Sandro Bonazzola] ovirt-optimizer: move from fc22 to fc24 ------------------------------------------ [...truncated 1374 lines...] ## took 3524 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 3322 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=4.0 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson4882168752186255621.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=4.0 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + OVIRT_SUITE=4.0 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/13/artifact/exported…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/13/artifact/exported…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/13/artifact/exported…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson8365424873316215114.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-23-x86_64.fc23.cfg + mock_root=mocker-fedora-23-x86_64.fc23 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.17 starting (python version = 3.4.3)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + mock_root=fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + [[ -n fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 1

Build failed in Jenkins: ovirt_3.6_he-system-tests #150
by jenkins＠jenkins.phx.ovirt.org 14 Jul '16

14 Jul '16

See <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/150/changes> Changes: [Eyal Edri] add hystrix deps to yum repos include list [Eyal Edri] remove missing fedora builds from 3.6 publisher [Eyal Edri] replace f23 with f24 builds ------------------------------------------ [...truncated 151 lines...] @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ ========== Scrubbing chroot mock \ --configdir="<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests"> \ --root="mocker-fedora-23-x86_64.fc23" \ --resultdir="logs/mocker-fedora-23-x86_64.fc23.scrub" \ --scrub=chroot WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.5.1)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Start: scrub ['chroot'] INFO: scrubbing chroot for mocker-fedora-23-x86_64.fc23 Finish: scrub ['chroot'] Finish: run Scrub chroot took 1 seconds ============================ ########################################################## ## Fri Jul 8 11:52:08 UTC 2016 Finished env: fc23:fedora-23-x86_64 ## took 2 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log ##! ========== Initializing chroot mock \ --configdir="<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests"> \ --root="mocker-fedora-23-x86_64.fc23" \ --resultdir="logs/mocker-fedora-23-x86_64.fc23.init" \ --init Init took 1 seconds ============================ ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=3.6 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_3.6_he-system-tests] $ /bin/bash -xe /tmp/hudson4399205298069700868.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=3.6 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> + OVIRT_SUITE=3.6 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/150/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/150/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> ]] POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_3.6_he-system-tests] $ /bin/bash -xe /tmp/hudson2551884655390886920.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-23-x86_64.fc23.cfg + mock_root=mocker-fedora-23-x86_64.fc23 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.5.1)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> + mock_root=fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f + [[ -n fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ grep fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f ++ awk '{print $3}' + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? Archiving artifacts

1 27

[JIRA] (OVIRT-638) Tune Gerrit
by Barak Korren (oVirt JIRA) 14 Jul '16

14 Jul '16

Barak Korren created OVIRT-638: ---------------------------------- Summary: Tune Gerrit Key: OVIRT-638 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-638 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: Barak Korren Assignee: infra We should probably tune Gerrit to take proper advantage of the bigger server we now have. This could be useful: http://ctf-dev-environment-vagrant.s3.amazonaws.com/Gerrit-Performance-Tuni… Few notable details: # Most RAM on the new server is currently taken by the FS cache instead of the Gerrit Java processes. # It seems that the Gerrit can do Git garbage collection on its own, and do so more efficiently then doing so with the Git command line. -- This message was sent by Atlassian JIRA (v1000.148.3#100005)

1 0

Build failed in Jenkins: ovirt_4.0_he-system-tests #1
by jenkins＠jenkins.phx.ovirt.org 14 Jul '16

14 Jul '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/1/> ------------------------------------------ [...truncated 43 lines...] > git config core.sparsecheckout # timeout=10 > git checkout -f e9e722a45fa07e8a466b9e162d78816013885940 First time build. Skipping changelog. > git branch -a # timeout=10 > git rev-parse remotes/origin/master^{commit} # timeout=10 [ovirt_4.0_he-system-tests] $ /bin/bash -e /tmp/hudson6941084399903629586.sh shell-scripts/cleanup_slave.sh ################################################################### # Cleaning up slave # ################################################################### Filesystem Size Used Avail Use% Mounted on devtmpfs 24G 0 24G 0% /dev tmpfs 24G 4.0K 24G 1% /dev/shm tmpfs 24G 884K 24G 1% /run tmpfs 24G 0 24G 0% /sys/fs/cgroup /dev/sda3 908G 9.0G 899G 1% / tmpfs 24G 248K 24G 1% /tmp /dev/sda1 253M 99M 155M 40% /boot tmpfs 4.8G 0 4.8G 0% /run/user/1012 --------------------------------------------------------------- Cleaning up postgres databases /tmp ~/workspace/ovirt_4.0_he-system-tests ~/workspace/ovirt_4.0_he-system-tests done Cleaning up journal logs (if any) Redirecting to /bin/systemctl restart systemd-journald.service Cleaning up /var/tmp done Emptying some common logs /var/log/wtmp Done /home/jenkins/workspace/ovirt-engine_3.6_build-artifacts-fc23-x86_64 Making sure there are no device mappings... Removing the used loop devices... Redirecting to /bin/systemctl restart libvirtd.service --------------------------------------------------------------- Filesystem Size Used Avail Use% Mounted on devtmpfs 24G 0 24G 0% /dev tmpfs 24G 4.0K 24G 1% /dev/shm tmpfs 24G 884K 24G 1% /run tmpfs 24G 0 24G 0% /sys/fs/cgroup /dev/sda3 908G 8.5G 899G 1% / tmpfs 24G 248K 24G 1% /tmp /dev/sda1 253M 99M 155M 40% /boot tmpfs 4.8G 0 4.8G 0% /run/user/1012 ################################################################### # Slave cleanup done # ################################################################### [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson8136113498799623821.sh + echo shell-scripts/system_tests.sh shell-scripts/system_tests.sh + VERSION=4.0 + SUITE_TYPE=he_basic + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + OVIRT_SUITE=he_basic_suite_4.0 + PREFIX=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + chmod g+x <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + cd ovirt-system-tests + ../jenkins/mock_configs/mock_runner.sh --mock-confs-dir ../jenkins/mock_configs --try-proxy --execute-script automation/he_basic_suite_4.0.sh fc23 ########################################################## ########################################################## ## Thu Jul 14 08:26:03 UTC 2016 Running env: fc23:fedora-23-x86_64 ########################################################## ERROR: Script automation/he_basic_suite_4.0.sh does not exist or is not readable. ########################################################## ## Thu Jul 14 08:26:03 UTC 2016 Finished env: fc23:fedora-23-x86_64 ## took 0 seconds ## rc = 1 ########################################################## No log files found, check command output ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=4.0 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson1870730102070286691.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=4.0 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + OVIRT_SUITE=4.0 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/1/artifact/exported-…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/1/artifact/exported-…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson4230101493759804961.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + false + shopt -u nullglob + sudo rm -Rf '/var/cache/mock/*' + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? Archiving artifacts

1 10

[JIRA] (OVIRT-636) Add disks in memory for lago slaves
by Barak Korren (oVirt JIRA) 14 Jul '16

14 Jul '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-636?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-636: ------------------------------------ We can also create the zram drive and do the mounts in the job code... I think its best to let the job create and destroy things because that will ensure things get cleaned. But - we need to ensure we keep cache directories around, so maybe we need both - move caches to RAM on host startup and write them back to disk periodically and on shut-down. And have the job workspaces be put on ram and destroyed after every run. IIRC /var/lib/lago is the local cache directory that is bind-mounted into the Mock chroot, so that need to me moved to RAM on boot and synced back to disk. The rest of the job I/O seems to indeed happen in /var/lib/mock and /home/jenkins so we can consider placing a ram drive there each time the job starts. But /home/jenkins might be tricky because it may be used for other stuff (the slave binary for example), so we should probably just dynamically create a ram-drive for $WORKSPACE at the job level. One way to move things into RAM at boot time is to use '/etc/sysconfig/readonly-root' and '/etc/rwtab', this has the benefit of making the hosts mostly stateless, but may also make changing configuration a little tricky. > Add disks in memory for lago slaves > ------------------------------------ > > Key: OVIRT-636 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-636 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: eyal edri [Administrator] > Assignee: infra > > We need to try moving the Lago slaves in CI to work with memory instead of local disks, > It might speed up the tests significantly. > We have 2 choices: > 1. add code to the job and mount the dir on /dev/shm/ > 2. create zram drive and mount the following dirs on it: > # /var/lib/mock > # /var/lib/lago > # /home/jenkins > Each host should have enough memory to run current tests on them, > however we'll need to make sure to clean that drive after each run. > Lets try it on one of the slaves to see what is the best solution before implementing for all slaves. -- This message was sent by Atlassian JIRA (v1000.148.3#100005)

1 0

Build failed in Jenkins: ovirt_3.6_system-tests #237
by jenkins＠jenkins.phx.ovirt.org 14 Jul '16

14 Jul '16

See <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/237/changes> Changes: [Lev Veyde] Update repos for Hosted-Engine 3.6 test [Eyal Edri] fixing publisher for master post f24 changes [Sandro Bonazzola] publisher: 4.0: moved HE HA to 4.0 branch ------------------------------------------ [...truncated 559 lines...] ## took 2157 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/basic_suite_3.6.sh.log ##! @ Collect artifacts: # ovirt-role metadata entry will be soon deprecated, instead you should use the vm-provider entry in the domain definiton and set it no one of: ovirt-node, ovirt-engine, ovirt-host # [Thread-1] lago_basic_suite_3_6_engine: # [Thread-2] lago_basic_suite_3_6_host1: # [Thread-3] lago_basic_suite_3_6_host0: # [Thread-4] lago_basic_suite_3_6_storage: # [Thread-4] lago_basic_suite_3_6_storage: Success (in 0:00:15) # [Thread-3] lago_basic_suite_3_6_host0: Success (in 0:00:15) # [Thread-2] lago_basic_suite_3_6_host1: Success (in 0:00:15) # [Thread-1] lago_basic_suite_3_6_engine: Success (in 0:00:16) @ Collect artifacts: Success (in 0:00:16) <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests> @@@@ ERROR: Failed running <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/b…> ######################### ======== Cleaning up ----------- Cleaning with lago ----------- Cleaning with lago done ======== Cleanup done Took 1986 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=3.6 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson2586379427978401779.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=3.6 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> + OVIRT_SUITE=3.6 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/237/artifact/exported-a…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/237/artifact/exported-a…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/237/artifact/exported-a…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson5691359662978306568.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/basic_suite_3.6.sh.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/m…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-23-x86_64.fc23.cfg + mock_root=mocker-fedora-23-x86_64.fc23 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/l….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.5.1)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/m…> + mock_root=fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + [[ -n fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 5

[JIRA] (OVIRT-637) F24 slaves are missing autoconf, automake and rpmbuild
by Evgheni Dereveanchin (oVirt JIRA) 14 Jul '16

14 Jul '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-637?page=com.atlassian.jira.p… ] Evgheni Dereveanchin reassigned OVIRT-637: ------------------------------------------ Assignee: Evgheni Dereveanchin (was: infra) > F24 slaves are missing autoconf, automake and rpmbuild > ------------------------------------------------------ > > Key: OVIRT-637 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-637 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > > FC24 slaves seem to be missing autoconf, automake and rpmbuild -- This message was sent by Atlassian JIRA (v1000.148.3#100005)

1 0

[JIRA] (OVIRT-637) F24 slaves are missing autoconf, automake and rpmbuild
by Evgheni Dereveanchin (oVirt JIRA) 14 Jul '16

14 Jul '16

Evgheni Dereveanchin created OVIRT-637: ------------------------------------------ Summary: F24 slaves are missing autoconf, automake and rpmbuild Key: OVIRT-637 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-637 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Evgheni Dereveanchin Assignee: infra FC24 slaves seem to be missing autoconf, automake and rpmbuild -- This message was sent by Atlassian JIRA (v1000.148.3#100005)

1 0

[JIRA] (OVIRT-636) Add disks in memory for lago slaves
by eyal edri [Administrator] (oVirt JIRA) 14 Jul '16

14 Jul '16

eyal edri [Administrator] created OVIRT-636: ----------------------------------------------- Summary: Add disks in memory for lago slaves Key: OVIRT-636 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-636 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: eyal edri [Administrator] Assignee: infra We need to try moving the Lago slaves in CI to work with memory instead of local disks, It might speed up the tests significantly. We have 2 choices: 1. add code to the job and mount the dir on /dev/shm/ 2. create zram drive and mount the following dirs on it: # /var/lib/mock # /var/lib/lago # /home/jenkins Each host should have enough memory to run current tests on them, however we'll need to make sure to clean that drive after each run. Lets try it on one of the slaves to see what is the best solution before implementing for all slaves. -- This message was sent by Atlassian JIRA (v1000.148.3#100005)

1 0