October 2016 - Infra - oVirt List Archives

Jenkins build became unstable: ovirt_4.0_he-system-tests #382
by jenkins＠jenkins.phx.ovirt.org 12 Oct '16

12 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/382/changes>

1 0

Build failed in Jenkins: ovirt_4.0_system-tests #529
by jenkins＠jenkins.phx.ovirt.org 12 Oct '16

12 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/529/changes> Changes: [Gal Ben Haim] Fix check-patch to collect logs on failure ------------------------------------------ [...truncated 702 lines...] ## took 2492 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.basic_suite_4.0.sh/basic_suite_4.0.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/d…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/d…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 2324 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=4.0 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_4.0_system-tests] $ /bin/bash -xe /tmp/hudson5917274643114980000.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=4.0 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> + OVIRT_SUITE=4.0 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/529/artifact/exported-a…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/529/artifact/exported-a…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/529/artifact/exported-a…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_4.0_system-tests] $ /bin/bash -xe /tmp/hudson2916980851551151237.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_4.0.sh/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_4.0.sh/basic_suite_4.0.sh.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/m…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-23-x86_64.fc23.cfg + mock_root=mocker-fedora-23-x86_64.fc23 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/l….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.4.3)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/m…> + mock_root=fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + [[ -n fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 1

[JIRA] (OVIRT-770) intermittent failures when fetching from gerrit
by Evgheni Dereveanchin (oVirt JIRA) 12 Oct '16

12 Oct '16

Evgheni Dereveanchin created OVIRT-770: ------------------------------------------ Summary: intermittent failures when fetching from gerrit Key: OVIRT-770 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-770 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Evgheni Dereveanchin Assignee: infra Jenkins jobs fail sometimes with errors like this: {quote}12:43:28 ERROR: Error fetching remote repo 'origin' 12:43:28 hudson.plugins.git.GitException: Failed to fetch from git://gerrit.ovirt.org/ovirt-node-ng.git{quote} Subsequent runs work fine. This is similar to what happened in OVIRT-619 before we added resources to the gerrit VM. Sample job withs these symptoms: http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.0-pre_build-artifacts-el… -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

[JIRA] (OVIRT-770) intermittent failures when fetching from gerrit
by Evgheni Dereveanchin (oVirt JIRA) 12 Oct '16

12 Oct '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-770?page=com.atlassian.jira.p… ] Evgheni Dereveanchin reassigned OVIRT-770: ------------------------------------------ Assignee: Evgheni Dereveanchin (was: infra) > intermittent failures when fetching from gerrit > ----------------------------------------------- > > Key: OVIRT-770 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-770 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > > Jenkins jobs fail sometimes with errors like this: > {quote}12:43:28 ERROR: Error fetching remote repo 'origin' > 12:43:28 hudson.plugins.git.GitException: Failed to fetch from git://gerrit.ovirt.org/ovirt-node-ng.git{quote} > Subsequent runs work fine. This is similar to what happened in OVIRT-619 before we added resources to the gerrit VM. > Sample job withs these symptoms: > http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.0-pre_build-artifacts-el… -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

[JIRA] (OVIRT-745) gerrit was unreachable for several minutes
by Evgheni Dereveanchin (oVirt JIRA) 12 Oct '16

12 Oct '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-745?page=com.atlassian.jira.p… ] Evgheni Dereveanchin reassigned OVIRT-745: ------------------------------------------ Assignee: Evgheni Dereveanchin (was: infra) > gerrit was unreachable for several minutes > ------------------------------------------ > > Key: OVIRT-745 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-745 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > > gerrit.ovirt.org was unreachable for several minutes around 11:46 - 12:06 today (26.09.2016) > This ticket is to check the reason and see if we can avoid it in the future. -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

Jenkins build is back to stable : ovirt_4.0_he-system-tests #380
by jenkins＠jenkins.phx.ovirt.org 12 Oct '16

12 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/380/changes>

1 0

Build failed in Jenkins: ovirt_master_system-tests #639
by jenkins＠jenkins.phx.ovirt.org 12 Oct '16

12 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_master_system-tests/639/changes> Changes: [Gal Ben Haim] Fix check-patch to collect logs on failure [Sandro Bonazzola] vdsm: add ppc64le publisher ------------------------------------------ [...truncated 715 lines...] ## took 1483 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 1306 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=master SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson4082897219783623081.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=master + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + OVIRT_SUITE=master + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_master_system-tests/639/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_master_system-tests/639/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/639/artifact/exporte…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson3298945270453114636.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/state.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/build.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/root.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-24-x86_64.fc24.cfg + mock_root=mocker-fedora-24-x86_64.fc24 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-24-x86_64.fc24' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-24-x86_64.fc24 --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.5.1)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> + mock_root=fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + [[ -n fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 7

[ACTION REQUIRED] oVirt 4.0.5 RC2 build starting
by Sandro Bonazzola 12 Oct '16

12 Oct '16

Fyi oVirt products maintainers, An oVirt build for an official release is going to start right now. If you're a maintainer for any of the projects included in oVirt distribution and you have changes in your package ready to be released please: - bump version and release to be GA ready - tag your release within git (implies a GitHub Release to be automatically created) - build your packages within jenkins / koji / copr / whatever - verify all bugs on MODIFIED have target release and target milestone set. - add your builds to releng-tools/releases/ovirt-4.0.5_rc2.conf within releng-tools project -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>

1 0

oVirt infra daily report - unstable production jobs - 104
by jenkins＠jenkins.phx.ovirt.org 12 Oct '16

12 Oct '16

------=_Part_488_398449109.1476226807509 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/104//artifact/exported-a… Cheers, Jenkins ------=_Part_488_398449109.1476226807509 Content-Type: application/octet-stream; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 11/10/2016 </th></tr><tr><th colspan=2 class=sub> <font color="blue"><a href="http://jenkins.ovirt.org/">00 Unstable Jobs (Production)</a></font> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el7-x86…">ovirt-appliance_master_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-live_3.6-create-iso/">ovirt-live_3.6-create-iso</a> </td><td> need to move to std-ci or to yamlize </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86_6…">ovirt-node-ng_master_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.0-pre_build-artifacts-el…">ovirt-node-ng_ovirt-4.0-pre_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-release_master_build-artifacts-all-x86_6…">ovirt-release_master_build-artifacts-all-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/">ovirt_4.0_he-system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt_master_system-tests/">ovirt_master_system-tests</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/repos_master_check-closure_fc24_merged/">repos_master_check-closure_fc24_merged</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_4.0_check-merged-fc23-x86_64/">vdsm_4.0_check-merged-fc23-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-ppc64le/">vdsm_master_build-artifacts-el7-ppc64le</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/">vdsm_master_build-artifacts-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/">vdsm_master_check-merged-el7-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_488_398449109.1476226807509--

1 0

Build failed in Jenkins: ovirt_4.0_system-tests #525
by jenkins＠jenkins.phx.ovirt.org 11 Oct '16

11 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/525/changes> Changes: [Gal Ben Haim] Fix check-patch to collect logs on failure ------------------------------------------ [...truncated 703 lines...] ## took 2211 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.basic_suite_4.0.sh/basic_suite_4.0.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/d…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/d…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 2030 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=4.0 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_4.0_system-tests] $ /bin/bash -xe /tmp/hudson2574482535475189795.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=4.0 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> + OVIRT_SUITE=4.0 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/525/artifact/exported-a…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/525/artifact/exported-a…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/525/artifact/exported-a…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_4.0_system-tests] $ /bin/bash -xe /tmp/hudson6836943158705484424.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_4.0.sh/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_4.0.sh/basic_suite_4.0.sh.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/m…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-23-x86_64.fc23.cfg + mock_root=mocker-fedora-23-x86_64.fc23 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/l….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.4.3)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests/m…> + mock_root=fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + [[ -n fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 2