- Infra - oVirt List Archives

patches failing continuous integration after being merged
by Sandro Bonazzola 30 Oct '16

30 Oct '16

Hi, we have more than 300 patches failing continuous integration after being merged[1] Is anybody monitoring what's happening there? Have we broken check-merge tests? Have we auto rebase on merge automation breaking our patches? I would suggest to go over the failures and nail down issues. [1] https://gerrit.ovirt.org/#/q/status:merged+label:Continuous-Integration%253… -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>

3 2

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build # 2885 - null!
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

------=_Part_390_457413394.1477820670026 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/2885/ Build Number: 2885 Build Status: null ------=_Part_390_457413394.1477820670026--

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build # 2884 - null!
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

------=_Part_388_161347878.1477818793545 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/2884/ Build Number: 2884 Build Status: null ------=_Part_388_161347878.1477818793545--

1 0

[JIRA] (OVIRT-792) Jenkins job on fc23-vm05.phx.ovirt.org stuck on '... Removing the used loop devices...'
by Nadav Goldin (oVirt JIRA) 30 Oct '16

30 Oct '16

Nadav Goldin created OVIRT-792: ---------------------------------- Summary: Jenkins job on fc23-vm05.phx.ovirt.org stuck on '... Removing the used loop devices...' Key: OVIRT-792 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-792 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Nadav Goldin Assignee: infra Not sure if this is a regression, maybe added in [1], or a temporary glitch: http://jenkins.ovirt.org/job/lago_master_check-patch-el7-x86_64/336/console http://jenkins.ovirt.org/job/lago_master_check-patch-el7-x86_64/337/console for now going to clean it manually. [1] https://gerrit.ovirt.org/#/c/64511/ -- This message was sent by Atlassian JIRA (v1000.482.3#100017)

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build # 2883 - null!
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

------=_Part_386_173219582.1477816927862 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/2883/ Build Number: 2883 Build Status: null ------=_Part_386_173219582.1477816927862--

1 0

Jenkins build is back to stable : ovirt_4.0_he-system-tests #451
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/451/changes>

1 0

Build failed in Jenkins: ovirt_master_system-tests #706
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_master_system-tests/706/changes> Changes: [Ond��ej Svoboda] Simplify variable replacement in control.sh. ------------------------------------------ [...truncated 768 lines...] ## took 1694 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log entries: logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 1533 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=master SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson1649379955106932121.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=master + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + OVIRT_SUITE=master + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_master_system-tests/706/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_master_system-tests/706/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/706/artifact/exporte…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson305905691934850035.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/state.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/build.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/root.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/root.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-24-x86_64.fc24.cfg + mock_root=mocker-fedora-24-x86_64.fc24 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-24-x86_64.fc24' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-24-x86_64.fc24 --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.17 starting (python version = 3.4.3)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> + mock_root=fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + [[ -n fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 16

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build # 2881 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

------=_Part_379_334810844.1477810316478 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/2881/ Build Number: 2881 Build Status: FAILURE ------=_Part_379_334810844.1477810316478--

1 1

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build # 2880 - null!
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

------=_Part_377_490807304.1477806775219 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/2880/ Build Number: 2880 Build Status: null ------=_Part_377_490807304.1477806775219--

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build # 2879 - null!
by jenkins＠jenkins.phx.ovirt.org 30 Oct '16

30 Oct '16

------=_Part_375_574122964.1477803027080 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/2879/ Build Number: 2879 Build Status: null ------=_Part_375_574122964.1477803027080--

1 0