- Infra - oVirt List Archives

[JIRA] (OVIRT-700) Cant use jjb to update jobs
by eyal edri [Administrator] (oVirt JIRA) 18 Aug '16

18 Aug '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-700?page=com.atlassian.jira.p… ] eyal edri [Administrator] commented on OVIRT-700: ------------------------------------------------- [~ederevea] can you check if this is related to the latest upgrade of jenkins? > Cant use jjb to update jobs > --------------------------- > > Key: OVIRT-700 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-700 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: eyal edri [Administrator] > Assignee: infra > > API tocken is ok and still get following error > jenkins-jobs --allow-empty-variables --conf ../../../jenkins_conf/upstream.conf > test . ovirt-node-ng_*_build-artifacts* > Traceback (most recent call last): > File "/usr/bin/jenkins-jobs", line 10, in <module> > sys.exit(main()) > File "/usr/lib/python2.7/site-packages/jenkins_jobs/cmd.py", line 179, in > main > execute(options, config) > File "/usr/lib/python2.7/site-packages/jenkins_jobs/cmd.py", line 351, in > execute > n_workers=1) > File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line > 333, in update_jobs > self.load_files(input_fn) > File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line > 230, in load_files > self.parser = YamlParser(self.global_config, self.plugins_list) > File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line > 226, in plugins_list > self._plugins_list = self.jenkins.get_plugins_info() > File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line > 190, in get_plugins_info > raise e > jenkins.JenkinsException: Error in request. Possibly authentication > failed [403]: Forbidden > INFO:jenkins_jobs.builder:Cache saved > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

Build failed in Jenkins: ovirt_4.0_he-system-tests #157
by jenkins＠jenkins.phx.ovirt.org 18 Aug '16

18 Aug '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/157/changes> Changes: [Martin Polednik] use spice+vnc in basic sanity tests [Sharon Naftaly] Adding el7 to master repo-check-closure jobs ------------------------------------------ [...truncated 1393 lines...] ## took 3533 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 3351 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=4.0 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson7685585842006748333.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=4.0 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + OVIRT_SUITE=4.0 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/157/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/157/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/157/artifact/exporte…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson6168487646776492379.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-23-x86_64.fc23.cfg + mock_root=mocker-fedora-23-x86_64.fc23 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.4.3)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-test…> + mock_root=fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + [[ -n fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 1

[JIRA] (OVIRT-702) add 'isort --check-only' in check-patch.for python projects in addition to pep8 style checking.
by sbonazzo (oVirt JIRA) 18 Aug '16

18 Aug '16

sbonazzo created OVIRT-702: ------------------------------ Summary: add 'isort --check-only' in check-patch.for python projects in addition to pep8 style checking. Key: OVIRT-702 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-702 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: sbonazzo Assignee: infra As proposed some time ago, please add 'isort --check-only' in check-patch.for python projects in addition to pep8 style checking. initial configuration: $ cat ~/.isort.cfg [settings] line_length=79 known_standard_library=configparser,Cheetah.Template force_single_line=True default_section=FIRSTPARTY known_otopi=otopi known_host_deploy=ovirt_host_deploy known_ovirt_engine=ovirt_engine known_ovirt_engine_setup=ovirt_engine_setup known_ovirt_setup_lib=ovirt_setup_lib known_vdsm=vdsm known_ovirt_hosted_engine_setup=ovirt_hosted_engine_setup sections=FUTURE,STDLIB,FIRSTPARTY,OTOPI,VDSM,HOST_DEPLOY, OVIRT_ENGINE,OVIRT_ENGINE_SETUP,OVIRT_SETUP_LIB,OVIRT_ HOSTED_ENGINE_SETUP,THIRDPARTY,LOCALFOLDER lines_between_types=2 actions: 1) update ovirt site on code styling section 2) add the check on ovirt-engine project 3) add the check to other projects that opts in. As a start all those maintained by integration team. -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

[JIRA] (OVIRT-617) NullPointerException when logging in or registering on Jenkins
by eyal edri [Administrator] (oVirt JIRA) 18 Aug '16

18 Aug '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-617?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-617: -------------------------------------------- Resolution: Cannot Reproduce Status: Done (was: Blocked) please reopen if still relevant > NullPointerException when logging in or registering on Jenkins > -------------------------------------------------------------- > > Key: OVIRT-617 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-617 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: infra > Priority: High > > It was reported that it's impossible to log in or register new users on jenkins.ovirt.org > sample page showing the error: > http://jenkins.ovirt.org/securityRealm/createAccount -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

[JIRA] (OVIRT-678) vdsm check-merge jobs doesn't work for el7
by eyal edri [Administrator] (oVirt JIRA) 18 Aug '16

18 Aug '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-678?page=com.atlassian.jira.p… ] eyal edri [Administrator] reassigned OVIRT-678: ----------------------------------------------- Assignee: snaftaly (was: infra) > vdsm check-merge jobs doesn't work for el7 > ------------------------------------------ > > Key: OVIRT-678 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-678 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: snaftaly > > On Wed, Aug 10, 2016 at 3:41 PM, Irit Goihman <igoihman(a)redhat.com> wrote: > > Hi, > > jobs vdsm_master_build-artifacts-el7-ppc64le and vdsm_master_check-merged-el7-x86_64 > > keep failing. They were created recently and should be removed. > > > The ppc64le should be fixed, not removed. > We want ppc64le builds now that we have ppc64le slaves. > The check-merged for el7 is your call, fine for me if you want to remove. > Just weird since el7 is our main target. > > > > Thanks > > > > -- > > Irit Goihman > > Software Engineer > > Red Hat Israel Ltd. > > > > _______________________________________________ > > Infra mailing list > > Infra(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/infra > > > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

[JIRA] (OVIRT-701) fix reverse DNS zone delegation
by Evgheni Dereveanchin (oVirt JIRA) 18 Aug '16

18 Aug '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-701?page=com.atlassian.jira.p… ] Evgheni Dereveanchin reassigned OVIRT-701: ------------------------------------------ Assignee: Evgheni Dereveanchin (was: infra) > fix reverse DNS zone delegation > ------------------------------- > > Key: OVIRT-701 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-701 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Foreman > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > > Reverse DNS delegation seems to be broken in the PHX2 subnet which may cause problems with email. To migrate mailman we need that to work. -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

[JIRA] (OVIRT-701) fix reverse DNS zone delegation
by Evgheni Dereveanchin (oVirt JIRA) 18 Aug '16

18 Aug '16

Evgheni Dereveanchin created OVIRT-701: ------------------------------------------ Summary: fix reverse DNS zone delegation Key: OVIRT-701 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-701 Project: oVirt - virtualization made easy Issue Type: Bug Components: Foreman Reporter: Evgheni Dereveanchin Assignee: infra Reverse DNS delegation seems to be broken in the PHX2 subnet which may cause problems with email. To migrate mailman we need that to work. -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

[JIRA] (OVIRT-700) Cant use jjb to update jobs
by eyal edri [Administrator] (oVirt JIRA) 18 Aug '16

18 Aug '16

eyal edri [Administrator] created OVIRT-700: ----------------------------------------------- Summary: Cant use jjb to update jobs Key: OVIRT-700 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-700 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: eyal edri [Administrator] Assignee: infra API tocken is ok and still get following error jenkins-jobs --allow-empty-variables --conf ../../../jenkins_conf/upstream.conf test . ovirt-node-ng_*_build-artifacts* Traceback (most recent call last): File "/usr/bin/jenkins-jobs", line 10, in <module> sys.exit(main()) File "/usr/lib/python2.7/site-packages/jenkins_jobs/cmd.py", line 179, in main execute(options, config) File "/usr/lib/python2.7/site-packages/jenkins_jobs/cmd.py", line 351, in execute n_workers=1) File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line 333, in update_jobs self.load_files(input_fn) File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line 230, in load_files self.parser = YamlParser(self.global_config, self.plugins_list) File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line 226, in plugins_list self._plugins_list = self.jenkins.get_plugins_info() File "/usr/lib/python2.7/site-packages/jenkins_jobs/builder.py", line 190, in get_plugins_info raise e jenkins.JenkinsException: Error in request. Possibly authentication failed [403]: Forbidden INFO:jenkins_jobs.builder:Cache saved -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

[JIRA] (OVIRT-697) Re: Failed Check-Merged Job
by eyal edri [Administrator] (oVirt JIRA) 18 Aug '16

18 Aug '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-697?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-697: -------------------------------------------- Resolution: Cannot Reproduce Status: Done (was: To Do) We are no control over external repos atm, so retry should be the fastest fix on this errors. However, we are also working on mirroring epel and other repos into our proxy to reduce these failures > Re: Failed Check-Merged Job > --------------------------- > > Key: OVIRT-697 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-697 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: eyal edri [Administrator] > Assignee: infra > > Adding Infra support so we'll have a ticket documenting this as well. > On Wed, Aug 17, 2016 at 5:23 PM, Phillip Bailey <phbailey(a)redhat.com> wrote: > > Hi infra team, > > > > One of my patches [1] failed the check-merged-el6-x86 job [2]. It looks > > like a yum install failed at 12:59:29. > > > > Could someone please take a look and let me know if any action is required > > on my part? > > > > Thanks! > > > > -Phillip Bailey > > > > [1] https://gerrit.ovirt.org/#/c/62165/ > > [2] http://jenkins.ovirt.org/job/ovirt-engine_3.6_check-merged- > > el6-x86_64/101/console > > > > _______________________________________________ > > Infra mailing list > > Infra(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/infra > > > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian JIRA (v1000.253.3#100011)

1 0

Build failed in Jenkins: ovirt_3.6_system-tests #381
by jenkins＠jenkins.phx.ovirt.org 18 Aug '16

18 Aug '16

See <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/381/changes> Changes: [Yaniv Kaul] ovirt-system-tests: activate hot-added disk ------------------------------------------ [...truncated 706 lines...] ## took 1732 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/basic_suite_3.6.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/d…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/d…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 1553 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=3.6 SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson3236269281003386660.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=3.6 + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> + OVIRT_SUITE=3.6 + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/381/artifact/exported-a…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/381/artifact/exported-a…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/e…> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/381/artifact/exported-a…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson6767108514701853852.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/ ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/basic_suite_3.6.sh.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/m…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-23-x86_64.fc23.cfg + mock_root=mocker-fedora-23-x86_64.fc23 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/l….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.5.1)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/m…> + mock_root=fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + [[ -n fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 1