- Infra - oVirt List Archives

Jenkins build became unstable: ovirt_4.0_he-system-tests #355
by jenkins＠jenkins.phx.ovirt.org 06 Oct '16

06 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/355/changes>

1 1

[JIRA] (OVIRT-751) Persistent maven caches on the mock slaves
by Gil Shinar (oVirt JIRA) 06 Oct '16

06 Oct '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-751?page=com.atlassian.jira.p… ] Gil Shinar commented on OVIRT-751: ---------------------------------- The default maven's cache is a .m2 folder under the user's home directory so the mount should be: source: <mock user's home>/.m2 destination: <jenkins or root's home>/.m2 Something like that. > Persistent maven caches on the mock slaves > ------------------------------------------ > > Key: OVIRT-751 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-751 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Anton Marchukov > Assignee: infra > > I think we need to design a way to retain maven caches on the mocked jenkins slaves. Currently it is stored inside the mock and thus maven downloads packages from artifactory server each time. > However there is really no reason for that. Maven artifacts are designed to be immutable so once artifact is in the repo there is no trivial way to change it without creating a new version. In fact it should never be needed and the correct solution for that is to always create a new version. > SNAPSHOOT artifacts are in fact timestamped and each one have different file name. It is just not visible since maven automatically takes the latest one. But it is not related to caching as the new snapshoot will be a new artifact still. > So based on that point there is no reason to purge maven cache each time, but there are reasons why not to purge. Not purging them will reduce the build times of all java jobs and also reduce the network traffic we have. -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

[JIRA] (OVIRT-751) Persistent maven caches on the mock slaves
by Barak Korren (oVirt JIRA) 06 Oct '16

06 Oct '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-751?page=com.atlassian.jira.p… ] Barak Korren edited comment on OVIRT-751 at 10/6/16 3:20 PM: ------------------------------------------------------------- This is something a project can decide to do on its own by simply bind-mounting a known location into the chroot (with the *.mounts file). You can either have a per-job cache by mounting from under $WORKSPACE, or a global per-slave cache by mounting from under /home/jenkins was (Author: bkorren(a)redhat.com) This is something a project can decide to do on its own by simply bind-mounting a known location into the chroot (with the *.mounts) file. You can either have a per-job cache by mounting from under $WORKSPACE, or a global per-slave cache by mounting from under /home/jenkins > Persistent maven caches on the mock slaves > ------------------------------------------ > > Key: OVIRT-751 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-751 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Anton Marchukov > Assignee: infra > > I think we need to design a way to retain maven caches on the mocked jenkins slaves. Currently it is stored inside the mock and thus maven downloads packages from artifactory server each time. > However there is really no reason for that. Maven artifacts are designed to be immutable so once artifact is in the repo there is no trivial way to change it without creating a new version. In fact it should never be needed and the correct solution for that is to always create a new version. > SNAPSHOOT artifacts are in fact timestamped and each one have different file name. It is just not visible since maven automatically takes the latest one. But it is not related to caching as the new snapshoot will be a new artifact still. > So based on that point there is no reason to purge maven cache each time, but there are reasons why not to purge. Not purging them will reduce the build times of all java jobs and also reduce the network traffic we have. -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

[JIRA] (OVIRT-751) Persistent maven caches on the mock slaves
by Barak Korren (oVirt JIRA) 06 Oct '16

06 Oct '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-751?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-751: ------------------------------------ If fact one can just send a patch to engine right now and see of this works and improves anything... > Persistent maven caches on the mock slaves > ------------------------------------------ > > Key: OVIRT-751 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-751 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Anton Marchukov > Assignee: infra > > I think we need to design a way to retain maven caches on the mocked jenkins slaves. Currently it is stored inside the mock and thus maven downloads packages from artifactory server each time. > However there is really no reason for that. Maven artifacts are designed to be immutable so once artifact is in the repo there is no trivial way to change it without creating a new version. In fact it should never be needed and the correct solution for that is to always create a new version. > SNAPSHOOT artifacts are in fact timestamped and each one have different file name. It is just not visible since maven automatically takes the latest one. But it is not related to caching as the new snapshoot will be a new artifact still. > So based on that point there is no reason to purge maven cache each time, but there are reasons why not to purge. Not purging them will reduce the build times of all java jobs and also reduce the network traffic we have. -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

[JIRA] (OVIRT-751) Persistent maven caches on the mock slaves
by Barak Korren (oVirt JIRA) 06 Oct '16

06 Oct '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-751?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-751: ------------------------------------ This is something a project can decide to do on its own by simply bind-mounting a known location into the chroot (with the *.mounts) file. You can either have a per-job cache by mounting from under $WORKSPACE, or a global per-slave cache by mounting from under /home/jenkins > Persistent maven caches on the mock slaves > ------------------------------------------ > > Key: OVIRT-751 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-751 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Reporter: Anton Marchukov > Assignee: infra > > I think we need to design a way to retain maven caches on the mocked jenkins slaves. Currently it is stored inside the mock and thus maven downloads packages from artifactory server each time. > However there is really no reason for that. Maven artifacts are designed to be immutable so once artifact is in the repo there is no trivial way to change it without creating a new version. In fact it should never be needed and the correct solution for that is to always create a new version. > SNAPSHOOT artifacts are in fact timestamped and each one have different file name. It is just not visible since maven automatically takes the latest one. But it is not related to caching as the new snapshoot will be a new artifact still. > So based on that point there is no reason to purge maven cache each time, but there are reasons why not to purge. Not purging them will reduce the build times of all java jobs and also reduce the network traffic we have. -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

Build failed in Jenkins: ovirt_master_system-tests #620
by jenkins＠jenkins.phx.ovirt.org 06 Oct '16

06 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/> ------------------------------------------ [...truncated 5 lines...] > git config remote.origin.url git://gerrit.ovirt.org/ovirt-system-tests.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from git://gerrit.ovirt.org/ovirt-system-tests.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:766) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1022) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1053) at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:129) at hudson.scm.SCM.checkout(SCM.java:485) at hudson.model.AbstractProject.checkout(AbstractProject.java:1269) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1738) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410) Caused by: hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 1: stdout: Removing basic_suite_master/LagoInitFile Removing mocker-fedora-24-x86_64.fc24.cfg stderr: warning: failed to remove deployment-basic_suite_master/default/uuid warning: failed to remove deployment-basic_suite_master/default/id_rsa warning: failed to remove deployment-basic_suite_master/default/id_rsa.pub warning: failed to remove deployment-basic_suite_master/default/initialized warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_host0_root.qcow2 warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_root.qcow2 warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_nfs.raw warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_export.raw warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_iscsi.raw warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_host1_root.qcow2 warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_add_local_repo.sh warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_setup_host_el7.sh warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_setup_storage_unified_el7.sh warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_setup_engine.sh warning: failed to remove deployment-basic_suite_master/current warning: failed to remove exported-artifacts/lago_logs/lago.log at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1640) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1616) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1612) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1254) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1266) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:621) at hudson.plugins.git.GitAPI.clean(GitAPI.java:311) at sun.reflect.GeneratedMethodAccessor818.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:884) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:859) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:818) at hudson.remoting.UserRequest.perform(UserRequest.java:152) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:332) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at ......remote call to ovirt-srv19.phx.ovirt.org(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) at hudson.remoting.UserResponse.retrieve(UserRequest.java:252) at hudson.remoting.Channel.call(Channel.java:781) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:249) at com.sun.proxy.$Proxy58.clean(Unknown Source) at org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:444) at hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:32) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:762) ... 12 more ERROR: null Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=master SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson6461574210440785323.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=master + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + OVIRT_SUITE=master + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/exporte…> mv: cannot move '<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…'> to '<http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/exporte…'>: Permission denied POST BUILD TASK : FAILURE END OF POST BUILD TASK : 0 ESCALATE FAILED POST BUILD TASK TO JOB STATUS Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson4415215435510982708.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n '' ]] + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? Archiving artifacts

1 2

[JIRA] (OVIRT-751) Persistent maven caches on the mock slaves
by Anton Marchukov (oVirt JIRA) 06 Oct '16

06 Oct '16

Anton Marchukov created OVIRT-751: ------------------------------------- Summary: Persistent maven caches on the mock slaves Key: OVIRT-751 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-751 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: Anton Marchukov Assignee: infra I think we need to design a way to retain maven caches on the mocked jenkins slaves. Currently it is stored inside the mock and thus maven downloads packages from artifactory server each time. However there is really no reason for that. Maven artifacts are designed to be immutable so once artifact is in the repo there is no trivial way to change it without creating a new version. In fact it should never be needed and the correct solution for that is to always create a new version. SNAPSHOOT artifacts are in fact timestamped and each one have different file name. It is just not visible since maven automatically takes the latest one. But it is not related to caching as the new snapshoot will be a new artifact still. So based on that point there is no reason to purge maven cache each time, but there are reasons why not to purge. Not purging them will reduce the build times of all java jobs and also reduce the network traffic we have. -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

[JIRA] (OVIRT-750) Issues with mvn central syncrhronization
by Piotr Kliczewski (oVirt JIRA) 06 Oct '16

06 Oct '16

Piotr Kliczewski created OVIRT-750: -------------------------------------- Summary: Issues with mvn central syncrhronization Key: OVIRT-750 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-750 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Piotr Kliczewski Assignee: infra When I release new version of jsonrpc and update the engine to use it the build fails and it is required to manually perform refresh of artifactory cache. -- This message was sent by Atlassian JIRA (v1000.383.2#100014)

1 0

Build failed in Jenkins: ovirt_master_system-tests #591
by jenkins＠jenkins.phx.ovirt.org 06 Oct '16

06 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/changes> Changes: [Yaniv Kaul] Fix Hosted-Engine 3.6 suite. [Gil Shinar] Removed mistaken el6 ------------------------------------------ [...truncated 687 lines...] ## took 2219 seconds ## rc = 1 ########################################################## ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ##! Last 20 log enties: logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log ##! + true + env_cleanup + echo '#########################' ######################### + local res=0 + local uuid + echo '======== Cleaning up' ======== Cleaning up + [[ -e <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + echo '----------- Cleaning with lago' ----------- Cleaning with lago + lago --workdir <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> destroy --yes --all-prefixes + echo '----------- Cleaning with lago done' ----------- Cleaning with lago done + [[ 0 != \0 ]] + echo '======== Cleanup done' ======== Cleanup done + exit 0 Took 2041 seconds =================================== ##! ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ##!######################################################## ########################################################## Build step 'Execute shell' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo 'shell_scripts/system_tests.collect_logs.sh' # # Required jjb vars: # version # VERSION=master SUITE_TYPE= WORKSPACE="$PWD" OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION" TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts" rm -rf "$WORKSPACE/exported-artifacts" mkdir -p "$WORKSPACE/exported-artifacts" if [[ -d "$TESTS_LOGS" ]]; then mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/" fi [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson4965861553390697910.sh + echo shell_scripts/system_tests.collect_logs.sh shell_scripts/system_tests.collect_logs.sh + VERSION=master + SUITE_TYPE= + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + OVIRT_SUITE=master + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> + rm -rf <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/artifact/exporte…> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/artifact/exporte…> + [[ -d <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + mv <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/artifact/exporte…> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Match found for :.* : True Logical operation result is TRUE Running script : #!/bin/bash -xe echo "shell-scripts/mock_cleanup.sh" shopt -s nullglob WORKSPACE="$PWD" # Make clear this is the cleanup, helps reading the jenkins logs cat <<EOC _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### EOC # Archive the logs, we want them anyway logs=( ./*log ./*/logs ) if [[ "$logs" ]]; then tar cvzf exported-artifacts/logs.tgz "${logs[@]}" rm -rf "${logs[@]}" fi # stop any processes running inside the chroot failed=false mock_confs=("$WORKSPACE"/*/mocker*) # Clean current jobs mockroot if any for mock_conf_file in "${mock_confs[@]}"; do [[ "$mock_conf_file" ]] || continue echo "Cleaning up mock $mock_conf" mock_root="${mock_conf_file##*/}" mock_root="${mock_root%.*}" my_mock="/usr/bin/mock" my_mock+=" --configdir=${mock_conf_file%/*}" my_mock+=" --root=${mock_root}" my_mock+=" --resultdir=$WORKSPACE" #TODO: investigate why mock --clean fails to umount certain dirs sometimes, #so we can use it instead of manually doing all this. echo "Killing all mock orphan processes, if any." $my_mock \ --orphanskill \ || { echo "ERROR: Failed to kill orphans on $chroot." failed=true } mock_root="$(\ grep \ -Po "(?<=config_opts\['root'\] = ')[^']*" \ "$mock_conf_file" \ )" || : [[ "$mock_root" ]] || continue mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $chroot. Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true } done done # Clean any leftover chroot from other jobs for mock_root in /var/lib/mock/*; do this_chroot_failed=false mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || : if [[ "$mounts" ]]; then echo "Found mounted dirs inside the chroot $mock_root." \ "Trying to umount." fi for mount in "${mounts[@]}"; do sudo umount --lazy "$mount" \ || { echo "ERROR: Failed to umount $mount." failed=true this_chroot_failed=true } done if ! $this_chroot_failed; then sudo rm -rf "$mock_root" fi done if $failed; then echo "Aborting." exit 1 fi # remove mock system cache, we will setup proxies to do the caching and this # takes lots of space between runs shopt -u nullglob sudo rm -Rf /var/cache/mock/* # restore the permissions in the working dir, as sometimes it leaves files # owned by root and then the 'cleanup workspace' from jenkins job fails to # clean and breaks the jobs sudo chown -R "$USER" "$WORKSPACE" [ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson6122160673590153533.sh + echo shell-scripts/mock_cleanup.sh shell-scripts/mock_cleanup.sh + shopt -s nullglob + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> + cat _______________________________________________________________________ ####################################################################### # # # CLEANUP # # # ####################################################################### + logs=(./*log ./*/logs) + [[ -n ./ovirt-system-tests/logs ]] + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs ./ovirt-system-tests/logs/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/state.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/build.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/root.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/state.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/build.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/root.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/stdout_stderr.log ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/ ./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log + rm -rf ./ovirt-system-tests/logs + failed=false + mock_confs=("$WORKSPACE"/*/mocker*) + for mock_conf_file in '"${mock_confs[@]}"' + [[ -n <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> ]] + echo 'Cleaning up mock ' Cleaning up mock + mock_root=mocker-fedora-24-x86_64.fc24.cfg + mock_root=mocker-fedora-24-x86_64.fc24 + my_mock=/usr/bin/mock + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests'> + my_mock+=' --root=mocker-fedora-24-x86_64.fc24' + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/'> + echo 'Killing all mock orphan processes, if any.' Killing all mock orphan processes, if any. + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-24-x86_64.fc24 --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> --orphanskill WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test….> Using default... INFO: mock.py version 1.2.18 starting (python version = 3.5.1)... Start: init plugins INFO: selinux enabled Finish: init plugins Start: run Finish: run ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-test…> + mock_root=fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + [[ -n fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a ]] + mounts=($(mount | awk '{print $3}' | grep "$mock_root")) ++ mount ++ awk '{print $3}' ++ grep fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + : + [[ -n '' ]] + false + shopt -u nullglob + sudo rm -Rf /var/cache/mock/fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 1 Recording test results Archiving artifacts

1 27

Jenkins build became unstable: ovirt_4.0_he-system-tests #353
by jenkins＠jenkins.phx.ovirt.org 06 Oct '16

06 Oct '16

See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/353/changes>

1 1