Build failed in Jenkins: ovirt_master_system-tests #620
by jenkins＠jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/>
------------------------------------------
[...truncated 5 lines...]
> git config remote.origin.url git://gerrit.ovirt.org/ovirt-system-tests.git # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git://gerrit.ovirt.org/ovirt-system-tests.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:766)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1022)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1053)
at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:129)
at hudson.scm.SCM.checkout(SCM.java:485)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1269)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
at hudson.model.Run.execute(Run.java:1738)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Caused by: hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 1:
stdout: Removing basic_suite_master/LagoInitFile
Removing mocker-fedora-24-x86_64.fc24.cfg
stderr: warning: failed to remove deployment-basic_suite_master/default/uuid
warning: failed to remove deployment-basic_suite_master/default/id_rsa
warning: failed to remove deployment-basic_suite_master/default/id_rsa.pub
warning: failed to remove deployment-basic_suite_master/default/initialized
warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_host0_root.qcow2
warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_root.qcow2
warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_nfs.raw
warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_export.raw
warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_engine_iscsi.raw
warning: failed to remove deployment-basic_suite_master/default/images/lago_basic_suite_master_host1_root.qcow2
warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_add_local_repo.sh
warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_setup_host_el7.sh
warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_setup_storage_unified_el7.sh
warning: failed to remove deployment-basic_suite_master/default/scripts/_home_jenkins_workspace_ovirt_master_system-tests_ovirt-system-tests_basic_suite_master_.._common_deploy-scripts_setup_engine.sh
warning: failed to remove deployment-basic_suite_master/current
warning: failed to remove exported-artifacts/lago_logs/lago.log
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1640)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1616)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1612)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1254)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1266)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:621)
at hudson.plugins.git.GitAPI.clean(GitAPI.java:311)
at sun.reflect.GeneratedMethodAccessor818.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:884)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:859)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:818)
at hudson.remoting.UserRequest.perform(UserRequest.java:152)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:332)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at ......remote call to ovirt-srv19.phx.ovirt.org(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:252)
at hudson.remoting.Channel.call(Channel.java:781)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:249)
at com.sun.proxy.$Proxy58.clean(Unknown Source)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:444)
at hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:32)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:762)
... 12 more
ERROR: null
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo 'shell_scripts/system_tests.collect_logs.sh'
#
# Required jjb vars:
# version
#
VERSION=master
SUITE_TYPE=
WORKSPACE="$PWD"
OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
rm -rf "$WORKSPACE/exported-artifacts"
mkdir -p "$WORKSPACE/exported-artifacts"
if [[ -d "$TESTS_LOGS" ]]; then
mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
fi
[ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson6461574210440785323.sh
+ echo shell_scripts/system_tests.collect_logs.sh
shell_scripts/system_tests.collect_logs.sh
+ VERSION=master
+ SUITE_TYPE=
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/>
+ OVIRT_SUITE=master
+ TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...>
+ rm -rf <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/expor...>
+ mkdir -p <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/expor...>
+ [[ -d <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> ]]
+ mv <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/expor...>
mv: cannot move '<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...'> to '<http://jenkins.ovirt.org/job/ovirt_master_system-tests/620/artifact/expor...'>: Permission denied
POST BUILD TASK : FAILURE
END OF POST BUILD TASK : 0
ESCALATE FAILED POST BUILD TASK TO JOB STATUS
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo "shell-scripts/mock_cleanup.sh"
shopt -s nullglob
WORKSPACE="$PWD"
# Make clear this is the cleanup, helps reading the jenkins logs
cat <<EOC
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
EOC
# Archive the logs, we want them anyway
logs=(
./*log
./*/logs
)
if [[ "$logs" ]]; then
tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
rm -rf "${logs[@]}"
fi
# stop any processes running inside the chroot
failed=false
mock_confs=("$WORKSPACE"/*/mocker*)
# Clean current jobs mockroot if any
for mock_conf_file in "${mock_confs[@]}"; do
[[ "$mock_conf_file" ]] || continue
echo "Cleaning up mock $mock_conf"
mock_root="${mock_conf_file##*/}"
mock_root="${mock_root%.*}"
my_mock="/usr/bin/mock"
my_mock+=" --configdir=${mock_conf_file%/*}"
my_mock+=" --root=${mock_root}"
my_mock+=" --resultdir=$WORKSPACE"
#TODO: investigate why mock --clean fails to umount certain dirs sometimes,
#so we can use it instead of manually doing all this.
echo "Killing all mock orphan processes, if any."
$my_mock \
--orphanskill \
|| {
echo "ERROR: Failed to kill orphans on $chroot."
failed=true
}
mock_root="$(\
grep \
-Po "(?<=config_opts\['root'\] = ')[^']*" \
"$mock_conf_file" \
)" || :
[[ "$mock_root" ]] || continue
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $chroot. Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
}
done
done
# Clean any leftover chroot from other jobs
for mock_root in /var/lib/mock/*; do
this_chroot_failed=false
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $mock_root." \
"Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
this_chroot_failed=true
}
done
if ! $this_chroot_failed; then
sudo rm -rf "$mock_root"
fi
done
if $failed; then
echo "Aborting."
exit 1
fi
# remove mock system cache, we will setup proxies to do the caching and this
# takes lots of space between runs
shopt -u nullglob
sudo rm -Rf /var/cache/mock/*
# restore the permissions in the working dir, as sometimes it leaves files
# owned by root and then the 'cleanup workspace' from jenkins job fails to
# clean and breaks the jobs
sudo chown -R "$USER" "$WORKSPACE"
[ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson4415215435510982708.sh
+ echo shell-scripts/mock_cleanup.sh
shell-scripts/mock_cleanup.sh
+ shopt -s nullglob
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/>
+ cat
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
+ logs=(./*log ./*/logs)
+ [[ -n '' ]]
+ failed=false
+ mock_confs=("$WORKSPACE"/*/mocker*)
+ false
+ shopt -u nullglob
+ sudo rm -Rf /var/cache/mock/fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a
+ sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/>
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 1
Recording test results
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Archiving artifacts
7 years, 6 months
[JIRA] (OVIRT-751) Persistent maven caches on the mock slaves
by Anton Marchukov (oVirt JIRA)
Anton Marchukov created OVIRT-751:
-------------------------------------
Summary: Persistent maven caches on the mock slaves
Key: OVIRT-751
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-751
Project: oVirt - virtualization made easy
Issue Type: Improvement
Reporter: Anton Marchukov
Assignee: infra
I think we need to design a way to retain maven caches on the mocked jenkins slaves. Currently it is stored inside the mock and thus maven downloads packages from artifactory server each time.
However there is really no reason for that. Maven artifacts are designed to be immutable so once artifact is in the repo there is no trivial way to change it without creating a new version. In fact it should never be needed and the correct solution for that is to always create a new version.
SNAPSHOOT artifacts are in fact timestamped and each one have different file name. It is just not visible since maven automatically takes the latest one. But it is not related to caching as the new snapshoot will be a new artifact still.
So based on that point there is no reason to purge maven cache each time, but there are reasons why not to purge. Not purging them will reduce the build times of all java jobs and also reduce the network traffic we have.
--
This message was sent by Atlassian JIRA
(v1000.383.2#100014)
7 years, 6 months
[JIRA] (OVIRT-750) Issues with mvn central syncrhronization
by Piotr Kliczewski (oVirt JIRA)
Piotr Kliczewski created OVIRT-750:
--------------------------------------
Summary: Issues with mvn central syncrhronization
Key: OVIRT-750
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-750
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: Piotr Kliczewski
Assignee: infra
When I release new version of jsonrpc and update the engine to use it the
build fails and it is required to manually perform refresh of artifactory
cache.
--
This message was sent by Atlassian JIRA
(v1000.383.2#100014)
7 years, 6 months
Build failed in Jenkins: ovirt_master_system-tests #591
by jenkins＠jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/changes>
Changes:
[Yaniv Kaul] Fix Hosted-Engine 3.6 suite.
[Gil Shinar] Removed mistaken el6
------------------------------------------
[...truncated 687 lines...]
## took 2219 seconds
## rc = 1
##########################################################
##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
##! Last 20 log enties: logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log
##!
+ true
+ env_cleanup
+ echo '#########################'
#########################
+ local res=0
+ local uuid
+ echo '======== Cleaning up'
======== Cleaning up
+ [[ -e <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> ]]
+ echo '----------- Cleaning with lago'
----------- Cleaning with lago
+ lago --workdir <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> destroy --yes --all-prefixes
+ echo '----------- Cleaning with lago done'
----------- Cleaning with lago done
+ [[ 0 != \0 ]]
+ echo '======== Cleanup done'
======== Cleanup done
+ exit 0
Took 2041 seconds
===================================
##!
##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##!########################################################
##########################################################
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo 'shell_scripts/system_tests.collect_logs.sh'
#
# Required jjb vars:
# version
#
VERSION=master
SUITE_TYPE=
WORKSPACE="$PWD"
OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
rm -rf "$WORKSPACE/exported-artifacts"
mkdir -p "$WORKSPACE/exported-artifacts"
if [[ -d "$TESTS_LOGS" ]]; then
mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
fi
[ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson4965861553390697910.sh
+ echo shell_scripts/system_tests.collect_logs.sh
shell_scripts/system_tests.collect_logs.sh
+ VERSION=master
+ SUITE_TYPE=
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/>
+ OVIRT_SUITE=master
+ TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...>
+ rm -rf <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/artifact/expor...>
+ mkdir -p <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/artifact/expor...>
+ [[ -d <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> ]]
+ mv <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> <http://jenkins.ovirt.org/job/ovirt_master_system-tests/591/artifact/expor...>
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo "shell-scripts/mock_cleanup.sh"
shopt -s nullglob
WORKSPACE="$PWD"
# Make clear this is the cleanup, helps reading the jenkins logs
cat <<EOC
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
EOC
# Archive the logs, we want them anyway
logs=(
./*log
./*/logs
)
if [[ "$logs" ]]; then
tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
rm -rf "${logs[@]}"
fi
# stop any processes running inside the chroot
failed=false
mock_confs=("$WORKSPACE"/*/mocker*)
# Clean current jobs mockroot if any
for mock_conf_file in "${mock_confs[@]}"; do
[[ "$mock_conf_file" ]] || continue
echo "Cleaning up mock $mock_conf"
mock_root="${mock_conf_file##*/}"
mock_root="${mock_root%.*}"
my_mock="/usr/bin/mock"
my_mock+=" --configdir=${mock_conf_file%/*}"
my_mock+=" --root=${mock_root}"
my_mock+=" --resultdir=$WORKSPACE"
#TODO: investigate why mock --clean fails to umount certain dirs sometimes,
#so we can use it instead of manually doing all this.
echo "Killing all mock orphan processes, if any."
$my_mock \
--orphanskill \
|| {
echo "ERROR: Failed to kill orphans on $chroot."
failed=true
}
mock_root="$(\
grep \
-Po "(?<=config_opts\['root'\] = ')[^']*" \
"$mock_conf_file" \
)" || :
[[ "$mock_root" ]] || continue
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $chroot. Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
}
done
done
# Clean any leftover chroot from other jobs
for mock_root in /var/lib/mock/*; do
this_chroot_failed=false
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $mock_root." \
"Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
this_chroot_failed=true
}
done
if ! $this_chroot_failed; then
sudo rm -rf "$mock_root"
fi
done
if $failed; then
echo "Aborting."
exit 1
fi
# remove mock system cache, we will setup proxies to do the caching and this
# takes lots of space between runs
shopt -u nullglob
sudo rm -Rf /var/cache/mock/*
# restore the permissions in the working dir, as sometimes it leaves files
# owned by root and then the 'cleanup workspace' from jenkins job fails to
# clean and breaks the jobs
sudo chown -R "$USER" "$WORKSPACE"
[ovirt_master_system-tests] $ /bin/bash -xe /tmp/hudson6122160673590153533.sh
+ echo shell-scripts/mock_cleanup.sh
shell-scripts/mock_cleanup.sh
+ shopt -s nullglob
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/>
+ cat
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
+ logs=(./*log ./*/logs)
+ [[ -n ./ovirt-system-tests/logs ]]
+ tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs
./ovirt-system-tests/logs/
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/stdout_stderr.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/state.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/build.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.init/root.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/stdout_stderr.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/state.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/build.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.install_packages/root.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.clean_rpmdb/stdout_stderr.log
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/
./ovirt-system-tests/logs/mocker-fedora-24-x86_64.fc24.basic_suite_master.sh/basic_suite_master.sh.log
+ rm -rf ./ovirt-system-tests/logs
+ failed=false
+ mock_confs=("$WORKSPACE"/*/mocker*)
+ for mock_conf_file in '"${mock_confs[@]}"'
+ [[ -n <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...> ]]
+ echo 'Cleaning up mock '
Cleaning up mock
+ mock_root=mocker-fedora-24-x86_64.fc24.cfg
+ mock_root=mocker-fedora-24-x86_64.fc24
+ my_mock=/usr/bin/mock
+ my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests'>
+ my_mock+=' --root=mocker-fedora-24-x86_64.fc24'
+ my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/'>
+ echo 'Killing all mock orphan processes, if any.'
Killing all mock orphan processes, if any.
+ /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-24-x86_64.fc24 --resultdir=<http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/> --orphanskill
WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te....> Using default...
INFO: mock.py version 1.2.18 starting (python version = 3.5.1)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Finish: run
++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/ovirt-system-te...>
+ mock_root=fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a
+ [[ -n fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a ]]
+ mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
++ mount
++ awk '{print $3}'
++ grep fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a
+ :
+ [[ -n '' ]]
+ false
+ shopt -u nullglob
+ sudo rm -Rf /var/cache/mock/fedora-24-x86_64-4ddbe48c8f8b7d8c2c3635b52313f04a
+ sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_master_system-tests/ws/>
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 1
Recording test results
Archiving artifacts
7 years, 6 months
Re: [Ftpcom] Fwd: ** PROBLEM Service Alert: ovirt-mirrorchecker/ftp.snt.utwente.nl/pub/software/ovirt mirror site last sync is WARNING **
by Nadav Goldin
Hi Maarten,
Thanks for the reply. It seems the synchronization is still broken,
from the logs I see an attempted rsync connection at:
> START:Tue Oct 4 02:00:24 UTC 2016
> END:Tue Oct 4 02:23:55 UTC 2016
But looking at the timestamps, all files I sampled still date back to
29/09/2016.
> In other news; we have replacement hardware underway for the mirror, so
> I expect a maintenance window before the end of the year. If it'll
> impact the ovirt mirror in a meaningful way, we'll reach out.
Great, we'll appreciate that.
Nadav.
On Sun, Oct 2, 2016 at 9:34 PM, Maarten Aertsen <maarten(a)snt.utwente.nl> wrote:
> Hi Nadav,
>
> On 2016-10-02 11:22, Nadav Goldin wrote:
>> It looks like ovirt mirror site is not syncing for the past 48 hours
>> or so. From the logs I see you are connecting and running rsync, but
>> the files aren't synchronized.
>
> We're currently experiencing a pretty big spike for one of the other
> projects we're mirroring, so our own monitoring shows similar warnings.
> The rsync is currently running but proceeding very slowly. We expect
> traffic to slow down shortly; please let us know if things do not return
> to normal within one or two days.
>
> In other news; we have replacement hardware underway for the mirror, so
> I expect a maintenance window before the end of the year. If it'll
> impact the ovirt mirror in a meaningful way, we'll reach out.
>
> best regards, Maarten
>
> --
> Maarten Aertsen
> SNT FTPcom
7 years, 6 months
oVirt infra daily report - unstable production jobs - 98
by jenkins＠jenkins.phx.ovirt.org
------=_Part_399_143397503.1475708407063
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Good morning!
Attached is the HTML page with the jenkins status report. You can see it also here:
- http://jenkins.ovirt.org/job/system_jenkins-report/98//artifact/exported-...
Cheers,
Jenkins
------=_Part_399_143397503.1475708407063
Content-Type: application/octet-stream; name=upstream_report.html
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=upstream_report.html
Content-ID: <upstream_report.html>
<!DOCTYPE html><head><style type="text/css">
table.gridtable {
border-collapse: collapse;
table-layout:fixed;
width:1600px;
font-family: monospace;
font-size:13px;
}
.head {
font-size:20px;
font-family: arial;
}
.sub {
font-size:18px;
background-color:#e5e5e5;
font-family: arial;
}
pre {
font-family: monospace;
display: inline;
white-space: pre-wrap;
white-space: -moz-pre-wrap !important;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;
}
</style>
</head>
<body>
<table class="gridtable" border=2>
<tr><th colspan=2 class=head>
RHEVM CI Jenkins Daily Report - 05/10/2016
</th></tr><tr><th colspan=2 class=sub>
<font color="blue"><a href="http://jenkins.ovirt.org/">00 Unstable Jobs (Production)</a></font>
</th></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-el7-x...">ovirt-appliance_master_build-artifacts-el7-x86_64</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt-appliance_ovirt-4.0-snapshot_build-art...">ovirt-appliance_ovirt-4.0-snapshot_build-artifacts-el7-x86_64</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-4.0_el7_mer...">ovirt-engine_master_upgrade-from-4.0_el7_merged</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-el7-x86...">ovirt-node-ng_master_build-artifacts-el7-x86_64</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt-node-ng_ovirt-4.0-snapshot_build-artif...">ovirt-node-ng_ovirt-4.0-snapshot_build-artifacts-el7-x86_64</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt-release_master_build-artifacts-all-x86...">ovirt-release_master_build-artifacts-all-x86_64</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/">ovirt_4.0_he-system-tests</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt_master_system-tests/">ovirt_master_system-tests</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/repos_master_check-closure_fc24_merged/">repos_master_check-closure_fc24_merged</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/vdsm_4.0_check-merged-fc23-x86_64/">vdsm_4.0_check-merged-fc23-x86_64</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
------=_Part_399_143397503.1475708407063--
7 years, 6 months
Build failed in Jenkins: ovirt_4.0_system-tests #501
by jenkins＠jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/501/changes>
Changes:
[Yaniv Kaul] VDSM on master requires an updated initscripts - reposync it.
------------------------------------------
[...truncated 154 lines...]
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
========== Scrubbing chroot
mock \
--configdir="<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests"> \
--root="mocker-fedora-23-x86_64.fc23" \
--resultdir="logs/mocker-fedora-23-x86_64.fc23.scrub" \
--scrub=chroot
WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests....> Using default...
INFO: mock.py version 1.2.18 starting (python version = 3.4.3)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Start: scrub ['chroot']
INFO: scrubbing chroot for mocker-fedora-23-x86_64.fc23
Finish: scrub ['chroot']
Finish: run
Scrub chroot took 0 seconds
============================
##########################################################
## Wed Oct 5 11:50:11 UTC 2016 Finished env: fc23:fedora-23-x86_64
## took 62 seconds
## rc = 1
##########################################################
##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log
##!
========== Initializing chroot
mock \
--configdir="<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests"> \
--root="mocker-fedora-23-x86_64.fc23" \
--resultdir="logs/mocker-fedora-23-x86_64.fc23.init" \
--init
Init took 62 seconds
============================
##!
##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##!########################################################
##########################################################
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo 'shell_scripts/system_tests.collect_logs.sh'
#
# Required jjb vars:
# version
#
VERSION=4.0
SUITE_TYPE=
WORKSPACE="$PWD"
OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
rm -rf "$WORKSPACE/exported-artifacts"
mkdir -p "$WORKSPACE/exported-artifacts"
if [[ -d "$TESTS_LOGS" ]]; then
mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
fi
[ovirt_4.0_system-tests] $ /bin/bash -xe /tmp/hudson5787857819655797102.sh
+ echo shell_scripts/system_tests.collect_logs.sh
shell_scripts/system_tests.collect_logs.sh
+ VERSION=4.0
+ SUITE_TYPE=
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/>
+ OVIRT_SUITE=4.0
+ TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests...>
+ rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/501/artifact/exported...>
+ mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/501/artifact/exported...>
+ [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests...> ]]
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo "shell-scripts/mock_cleanup.sh"
shopt -s nullglob
WORKSPACE="$PWD"
# Make clear this is the cleanup, helps reading the jenkins logs
cat <<EOC
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
EOC
# Archive the logs, we want them anyway
logs=(
./*log
./*/logs
)
if [[ "$logs" ]]; then
tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
rm -rf "${logs[@]}"
fi
# stop any processes running inside the chroot
failed=false
mock_confs=("$WORKSPACE"/*/mocker*)
# Clean current jobs mockroot if any
for mock_conf_file in "${mock_confs[@]}"; do
[[ "$mock_conf_file" ]] || continue
echo "Cleaning up mock $mock_conf"
mock_root="${mock_conf_file##*/}"
mock_root="${mock_root%.*}"
my_mock="/usr/bin/mock"
my_mock+=" --configdir=${mock_conf_file%/*}"
my_mock+=" --root=${mock_root}"
my_mock+=" --resultdir=$WORKSPACE"
#TODO: investigate why mock --clean fails to umount certain dirs sometimes,
#so we can use it instead of manually doing all this.
echo "Killing all mock orphan processes, if any."
$my_mock \
--orphanskill \
|| {
echo "ERROR: Failed to kill orphans on $chroot."
failed=true
}
mock_root="$(\
grep \
-Po "(?<=config_opts\['root'\] = ')[^']*" \
"$mock_conf_file" \
)" || :
[[ "$mock_root" ]] || continue
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $chroot. Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
}
done
done
# Clean any leftover chroot from other jobs
for mock_root in /var/lib/mock/*; do
this_chroot_failed=false
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $mock_root." \
"Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
this_chroot_failed=true
}
done
if ! $this_chroot_failed; then
sudo rm -rf "$mock_root"
fi
done
if $failed; then
echo "Aborting."
exit 1
fi
# remove mock system cache, we will setup proxies to do the caching and this
# takes lots of space between runs
shopt -u nullglob
sudo rm -Rf /var/cache/mock/*
# restore the permissions in the working dir, as sometimes it leaves files
# owned by root and then the 'cleanup workspace' from jenkins job fails to
# clean and breaks the jobs
sudo chown -R "$USER" "$WORKSPACE"
[ovirt_4.0_system-tests] $ /bin/bash -xe /tmp/hudson7549406309925809547.sh
+ echo shell-scripts/mock_cleanup.sh
shell-scripts/mock_cleanup.sh
+ shopt -s nullglob
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/>
+ cat
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
+ logs=(./*log ./*/logs)
+ [[ -n ./ovirt-system-tests/logs ]]
+ tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs
./ovirt-system-tests/logs/
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log
+ rm -rf ./ovirt-system-tests/logs
+ failed=false
+ mock_confs=("$WORKSPACE"/*/mocker*)
+ for mock_conf_file in '"${mock_confs[@]}"'
+ [[ -n <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests...> ]]
+ echo 'Cleaning up mock '
Cleaning up mock
+ mock_root=mocker-fedora-23-x86_64.fc23.cfg
+ mock_root=mocker-fedora-23-x86_64.fc23
+ my_mock=/usr/bin/mock
+ my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests'>
+ my_mock+=' --root=mocker-fedora-23-x86_64.fc23'
+ my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/'>
+ echo 'Killing all mock orphan processes, if any.'
Killing all mock orphan processes, if any.
+ /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/> --orphanskill
WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests....> Using default...
INFO: mock.py version 1.2.18 starting (python version = 3.4.3)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Finish: run
++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/ovirt-system-tests...>
+ mock_root=fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad
+ [[ -n fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]]
+ mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
++ mount
++ grep fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad
++ awk '{print $3}'
+ :
+ [[ -n '' ]]
+ false
+ shopt -u nullglob
+ sudo rm -Rf /var/cache/mock/fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad
+ sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_system-tests/ws/>
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 1
Recording test results
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Archiving artifacts
7 years, 6 months
Build failed in Jenkins: ovirt_3.6_system-tests #580
by jenkins＠jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/580/changes>
Changes:
[Yaniv Kaul] VDSM on master requires an updated initscripts - reposync it.
------------------------------------------
[...truncated 153 lines...]
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
========== Scrubbing chroot
mock \
--configdir="<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests"> \
--root="mocker-fedora-23-x86_64.fc23" \
--resultdir="logs/mocker-fedora-23-x86_64.fc23.scrub" \
--scrub=chroot
WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests....> Using default...
INFO: mock.py version 1.2.17 starting (python version = 3.4.3)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Start: scrub ['chroot']
INFO: scrubbing chroot for mocker-fedora-23-x86_64.fc23
Finish: scrub ['chroot']
Finish: run
Scrub chroot took 0 seconds
============================
##########################################################
## Wed Oct 5 11:50:11 UTC 2016 Finished env: fc23:fedora-23-x86_64
## took 62 seconds
## rc = 1
##########################################################
##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log
##!
========== Initializing chroot
mock \
--configdir="<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests"> \
--root="mocker-fedora-23-x86_64.fc23" \
--resultdir="logs/mocker-fedora-23-x86_64.fc23.init" \
--init
Init took 62 seconds
============================
##!
##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##!########################################################
##########################################################
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo 'shell_scripts/system_tests.collect_logs.sh'
#
# Required jjb vars:
# version
#
VERSION=3.6
SUITE_TYPE=
WORKSPACE="$PWD"
OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
rm -rf "$WORKSPACE/exported-artifacts"
mkdir -p "$WORKSPACE/exported-artifacts"
if [[ -d "$TESTS_LOGS" ]]; then
mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
fi
[ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson2234905793749144291.sh
+ echo shell_scripts/system_tests.collect_logs.sh
shell_scripts/system_tests.collect_logs.sh
+ VERSION=3.6
+ SUITE_TYPE=
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
+ OVIRT_SUITE=3.6
+ TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...>
+ rm -rf <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/580/artifact/exported...>
+ mkdir -p <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/580/artifact/exported...>
+ [[ -d <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> ]]
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo "shell-scripts/mock_cleanup.sh"
shopt -s nullglob
WORKSPACE="$PWD"
# Make clear this is the cleanup, helps reading the jenkins logs
cat <<EOC
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
EOC
# Archive the logs, we want them anyway
logs=(
./*log
./*/logs
)
if [[ "$logs" ]]; then
tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
rm -rf "${logs[@]}"
fi
# stop any processes running inside the chroot
failed=false
mock_confs=("$WORKSPACE"/*/mocker*)
# Clean current jobs mockroot if any
for mock_conf_file in "${mock_confs[@]}"; do
[[ "$mock_conf_file" ]] || continue
echo "Cleaning up mock $mock_conf"
mock_root="${mock_conf_file##*/}"
mock_root="${mock_root%.*}"
my_mock="/usr/bin/mock"
my_mock+=" --configdir=${mock_conf_file%/*}"
my_mock+=" --root=${mock_root}"
my_mock+=" --resultdir=$WORKSPACE"
#TODO: investigate why mock --clean fails to umount certain dirs sometimes,
#so we can use it instead of manually doing all this.
echo "Killing all mock orphan processes, if any."
$my_mock \
--orphanskill \
|| {
echo "ERROR: Failed to kill orphans on $chroot."
failed=true
}
mock_root="$(\
grep \
-Po "(?<=config_opts\['root'\] = ')[^']*" \
"$mock_conf_file" \
)" || :
[[ "$mock_root" ]] || continue
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $chroot. Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
}
done
done
# Clean any leftover chroot from other jobs
for mock_root in /var/lib/mock/*; do
this_chroot_failed=false
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $mock_root." \
"Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
this_chroot_failed=true
}
done
if ! $this_chroot_failed; then
sudo rm -rf "$mock_root"
fi
done
if $failed; then
echo "Aborting."
exit 1
fi
# remove mock system cache, we will setup proxies to do the caching and this
# takes lots of space between runs
shopt -u nullglob
sudo rm -Rf /var/cache/mock/*
# restore the permissions in the working dir, as sometimes it leaves files
# owned by root and then the 'cleanup workspace' from jenkins job fails to
# clean and breaks the jobs
sudo chown -R "$USER" "$WORKSPACE"
[ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson2514147512101533680.sh
+ echo shell-scripts/mock_cleanup.sh
shell-scripts/mock_cleanup.sh
+ shopt -s nullglob
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
+ cat
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
+ logs=(./*log ./*/logs)
+ [[ -n ./ovirt-system-tests/logs ]]
+ tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs
./ovirt-system-tests/logs/
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log
./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log
+ rm -rf ./ovirt-system-tests/logs
+ failed=false
+ mock_confs=("$WORKSPACE"/*/mocker*)
+ for mock_conf_file in '"${mock_confs[@]}"'
+ [[ -n <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> ]]
+ echo 'Cleaning up mock '
Cleaning up mock
+ mock_root=mocker-fedora-23-x86_64.fc23.cfg
+ mock_root=mocker-fedora-23-x86_64.fc23
+ my_mock=/usr/bin/mock
+ my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests'>
+ my_mock+=' --root=mocker-fedora-23-x86_64.fc23'
+ my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/'>
+ echo 'Killing all mock orphan processes, if any.'
Killing all mock orphan processes, if any.
+ /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> --orphanskill
WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests....> Using default...
INFO: mock.py version 1.2.17 starting (python version = 3.4.3)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Finish: run
++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...>
+ mock_root=fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ [[ -n fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c ]]
+ mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
++ mount
++ awk '{print $3}'
++ grep fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ :
+ [[ -n '' ]]
+ false
+ shopt -u nullglob
+ sudo rm -Rf /var/cache/mock/fedora-23-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 1
Recording test results
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Archiving artifacts
7 years, 6 months