- Infra - oVirt List Archives

[JIRA] (OVIRT-854) repoman: check if repoman can keep more than 1 version for experimental repo
by eyal edri [Administrator] (oVirt JIRA) 20 Nov '16

20 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-854?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-854: -------------------------------------------- Epic Link: OVIRT-400 > repoman: check if repoman can keep more than 1 version for experimental repo > ---------------------------------------------------------------------------- > > Key: OVIRT-854 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-854 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: eyal edri [Administrator] > Assignee: infra > Priority: High > > Today we see text experimental master jobs fails because it has different vdsm rpms versions in the same repo. > This can happen because VDSM has 2 jobs publishing the "same" noarch rpms, once from x86_64 and once from ppc64le. > Since the job only takes latest rpms, it might take a newer rpm from one arch and older rpms from another, causing add host to fail installing VDSM. > We need to check if its possible to tell repoman to keep more than 1 version (should be enough) so the tests won't wait in this period of time where one arch got a head of another one. -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-854) repoman: check if repoman can keep more than 1 version for experimental repo
by eyal edri [Administrator] (oVirt JIRA) 20 Nov '16

20 Nov '16

eyal edri [Administrator] created OVIRT-854: ----------------------------------------------- Summary: repoman: check if repoman can keep more than 1 version for experimental repo Key: OVIRT-854 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-854 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: eyal edri [Administrator] Assignee: infra Priority: High Today we see text experimental master jobs fails because it has different vdsm rpms versions in the same repo. This can happen because VDSM has 2 jobs publishing the "same" noarch rpms, once from x86_64 and once from ppc64le. Since the job only takes latest rpms, it might take a newer rpm from one arch and older rpms from another, causing add host to fail installing VDSM. We need to check if its possible to tell repoman to keep more than 1 version (should be enough) so the tests won't wait in this period of time where one arch got a head of another one. -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build #3497 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 20 Nov '16

20 Nov '16

------=_Part_696_756287645.1479633620064 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/3497/, Build Number: 3497, Build Status: FAILURE ------=_Part_696_756287645.1479633620064--

1 1

[JIRA] (OVIRT-853) check out blue ocean jenkins project
by eyal edri [Administrator] (oVirt JIRA) 20 Nov '16

20 Nov '16

eyal edri [Administrator] created OVIRT-853: ----------------------------------------------- Summary: check out blue ocean jenkins project Key: OVIRT-853 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-853 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: eyal edri [Administrator] Assignee: infra https://jenkins.io/projects/blueocean/ -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-853) check out blue ocean jenkins project
by eyal edri [Administrator] (oVirt JIRA) 20 Nov '16

20 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-853?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-853: -------------------------------------------- Epic Link: OVIRT-400 > check out blue ocean jenkins project > ------------------------------------ > > Key: OVIRT-853 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-853 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: eyal edri [Administrator] > Assignee: infra > > https://jenkins.io/projects/blueocean/ -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

Build failed in Jenkins: ovirt_4.0_he-system-tests #538
by jenkins＠jenkins.phx.ovirt.org 20 Nov '16

20 Nov '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/538/changes> Changes: [Gal Ben Haim] Changed glance image and fixed glance list test ------------------------------------------ [...truncated 1453 lines...] ## Sat Nov 19 20:11:52 UTC 2016 Finished env: el7:epel-7-x86_64 ## took 2202 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log entries: logs/mocker-epel-7-x86_64.el7.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log ##! + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 + exit Took 2026 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=4.0 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson7413475678765069024.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=4.0 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + OVIRT_SUITE=4.0 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/538/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/538/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/538/artifact/exporte…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then for log in "${logs[@]}" do echo "Copying ${log} to exported-artifacts" mv $log exported-artifacts/ done fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" # Drop all left over libvirt domains for UUID in $(virsh list --all --uuid); do virsh destroy $UUID || : sleep 2 virsh undefine --remove-all-storage --storage vda --snapshots-metadata $UUID || : done [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson698562601973182595.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + for log in '"${logs[@]}"' + echo 'Copying ./ovirt-system-tests/logs to exported-artifacts' Copying ./ovirt-system-tests/logs to exported-artifacts + mv ./ovirt-system-tests/logs exported-artifacts/ + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-epel-7-x86_64.el7.cfg + mock_root=mocker-epel-7-x86_64.el7 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-epel-7-x86_64.el7' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests> --root=mocker-epel-7-x86_64.el7 --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.21 starting (python version = 3.4.3)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + mock_root=epel-7-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + [[ -n epel-7-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep epel-7-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/epel-7-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> ++ virsh list --all --uuid POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 1

[JIRA] (OVIRT-682) Improve CI logging - check if its possible to include mock errors in console output
by Barak Korren (oVirt JIRA) 20 Nov '16

20 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-682?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-682: ------------------------------------ We already have this - > mock runner shows last 20 lines from logs on failure. If the error is not there, the std-ci script's output probably needs to be improved. Nevertheless, maybe we can improve the output a little to make sure those last lines are easier to find. Reading the Jenkins output is not trivial to someone unfamiliar with it because the sat thing you see in it is a clean-up task's output and not the main job's. > Improve CI logging - check if its possible to include mock errors in console output > ----------------------------------------------------------------------------------- > > Key: OVIRT-682 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-682 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: eyal edri [Administrator] > Assignee: eyal edri [Administrator] > > On Fri, Aug 12, 2016 at 8:08 PM, Nir Soffer <nsoffer(a)redhat.com> wrote: > > Hi all, > > > > Lately vdsm builds are failing in the install packages stage with > > this unhelpful error: > > > > 13:59:42 INFO: installing package(s): autoconf automake gdb ... > > 13:59:53 ERROR: Command failed. See logs for output. > > > > I downloaded the logs.tgz file from > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24- > > x86_64/1154/artifact/exported-artifacts/logs.tgz > > > > And indeed in /./vdsm/logs/mocker-fedora-24-x86_64.fc24.install_packages/ > > root.log > > I found found this error: > > > > DEBUG util.py:421: Error: Package: > > python-ioprocess-0.17.0-1.201608111609.gitbd272f2.fc24.noarch > > (ovirt-snapshot) > > DEBUG util.py:421: Requires: ioprocess = > > 0.17.0-1.201608111609.gitbd272f2.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.15.1-1.fc24.x86_64 (fedora) > > DEBUG util.py:421: ioprocess = 0.15.1-1.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.16.1-1.fc24.x86_64 (updates) > > DEBUG util.py:421: ioprocess = 0.16.1-1.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.17.0-1.201607121058.gitbd272f2.fc24.x86_64 > > (ovirt-snapshot) > > DEBUG util.py:421: ioprocess = > > 0.17.0-1.201607121058.gitbd272f2.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.17.0-1.201608111129.gitbd272f2.fc24.x86_64 > > (ovirt-snapshot) > > DEBUG util.py:421: ioprocess = > > 0.17.0-1.201608111129.gitbd272f2.fc24 > > > > So we have 2 issues here: > > > > 1. We need *all* errors in the console > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24- > > x86_64/1154/console > Not sure its possible or easy to do, these errors usually come from mock > and are not relevant to the code running on the console. > I've cc'd infra-support and updated topic to track it and check our options. > Going foward, we might want to install a log collector like logstash and > then searching errors there should be much easier. > > > > > > 2. Someone need to fix the ovirt-snapshot repository - it should have > > the missing ioprocess > > package. > > Maybe the project xml is not correct? > > > > Nir > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_3.6 - Build #3786 - SUCCESS!
by jenkins＠jenkins.phx.ovirt.org 20 Nov '16

20 Nov '16

------=_Part_693_1655644444.1479598417306 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_3.6/3786/, Build Number: 3786, Build Status: SUCCESS ------=_Part_693_1655644444.1479598417306--

1 0

oVirt infra daily report - unstable production jobs - 143
by jenkins＠jenkins.phx.ovirt.org 20 Nov '16

20 Nov '16

Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/143//artifact/exported-a… Cheers, Jenkins

1 0

Build failed in Jenkins: ovirt_3.6_he-system-tests #704
by jenkins＠jenkins.phx.ovirt.org 19 Nov '16

19 Nov '16

See <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/704/changes> Changes: [Gal Ben Haim] Changed glance image and fixed glance list test ------------------------------------------ [...truncated 1460 lines...] ## Sat Nov 19 16:43:46 UTC 2016 Finished env: el7:epel-7-x86_64 ## took 3096 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log entries: logs/mocker-epel-7-x86_64.el7.he_basic_suite_3.6.sh/he_basic_suite_3.6.sh.log ##! + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 + exit Took 2947 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=3.6 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_3.6_he-system-tests] $ /bin/bash -xe /tmp/hudson827508074724700295.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=3.6 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> + OVIRT_SUITE=3.6 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/704/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/704/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/704/artifact/exporte…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then for log in "${logs[@]}" do echo "Copying ${log} to exported-artifacts" mv $log exported-artifacts/ done fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" # Drop all left over libvirt domains for UUID in $(virsh list --all --uuid); do virsh destroy $UUID || : sleep 2 virsh undefine --remove-all-storage --storage vda --snapshots-metadata $UUID || : done [ovirt_3.6_he-system-tests] $ /bin/bash -xe /tmp/hudson1726281692871176161.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + for log in '"${logs[@]}"' + echo 'Copying ./ovirt-system-tests/logs to exported-artifacts' Copying ./ovirt-system-tests/logs to exported-artifacts + mv ./ovirt-system-tests/logs exported-artifacts/ + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-epel-7-x86_64.el7.cfg + mock_root=mocker-epel-7-x86_64.el7 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-epel-7-x86_64.el7' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests> --root=mocker-epel-7-x86_64.el7 --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.21 starting (python version = 3.5.1)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-test…> + mock_root=epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + [[ -n epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> ++ virsh list --all --uuid POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 1