Build failed in Jenkins: ovirt_3.6_system-tests #748
by jenkins＠jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/748/changes>
Changes:
[Yaniv Kaul] Explictly use systemctl for services.
[Fabian Deutsch] mock_cleanup: Drop all leftover libvirt domains
[Eyal Edri] whitesource: update project name to oVirt
------------------------------------------
[...truncated 757 lines...]
#########################
+ local res=0
+ local uuid
+ echo '======== Cleaning up'
======== Cleaning up
+ [[ -e <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> ]]
+ echo '----------- Cleaning with lago'
----------- Cleaning with lago
+ lago --workdir <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> destroy --yes --all-prefixes
+ echo '----------- Cleaning with lago done'
----------- Cleaning with lago done
+ [[ 0 != \0 ]]
+ echo '======== Cleanup done'
======== Cleanup done
+ exit 0
+ exit
Took 1545 seconds
===================================
##!
##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##!########################################################
##########################################################
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo 'shell_scripts/system_tests.collect_logs.sh'
#
# Required jjb vars:
# version
#
VERSION=3.6
SUITE_TYPE=
WORKSPACE="$PWD"
OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
rm -rf "$WORKSPACE/exported-artifacts"
mkdir -p "$WORKSPACE/exported-artifacts"
if [[ -d "$TESTS_LOGS" ]]; then
mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
fi
[ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson8286416771805850752.sh
+ echo shell_scripts/system_tests.collect_logs.sh
shell_scripts/system_tests.collect_logs.sh
+ VERSION=3.6
+ SUITE_TYPE=
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
+ OVIRT_SUITE=3.6
+ TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...>
+ rm -rf <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/748/artifact/exported...>
+ mkdir -p <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/748/artifact/exported...>
+ [[ -d <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> ]]
+ mv <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/748/artifact/exported...>
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo "shell-scripts/mock_cleanup.sh"
shopt -s nullglob
WORKSPACE="$PWD"
# Make clear this is the cleanup, helps reading the jenkins logs
cat <<EOC
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
EOC
# Archive the logs, we want them anyway
logs=(
./*log
./*/logs
)
if [[ "$logs" ]]; then
tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
rm -rf "${logs[@]}"
fi
# stop any processes running inside the chroot
failed=false
mock_confs=("$WORKSPACE"/*/mocker*)
# Clean current jobs mockroot if any
for mock_conf_file in "${mock_confs[@]}"; do
[[ "$mock_conf_file" ]] || continue
echo "Cleaning up mock $mock_conf"
mock_root="${mock_conf_file##*/}"
mock_root="${mock_root%.*}"
my_mock="/usr/bin/mock"
my_mock+=" --configdir=${mock_conf_file%/*}"
my_mock+=" --root=${mock_root}"
my_mock+=" --resultdir=$WORKSPACE"
#TODO: investigate why mock --clean fails to umount certain dirs sometimes,
#so we can use it instead of manually doing all this.
echo "Killing all mock orphan processes, if any."
$my_mock \
--orphanskill \
|| {
echo "ERROR: Failed to kill orphans on $chroot."
failed=true
}
mock_root="$(\
grep \
-Po "(?<=config_opts\['root'\] = ')[^']*" \
"$mock_conf_file" \
)" || :
[[ "$mock_root" ]] || continue
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $chroot. Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
}
done
done
# Clean any leftover chroot from other jobs
for mock_root in /var/lib/mock/*; do
this_chroot_failed=false
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $mock_root." \
"Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
this_chroot_failed=true
}
done
if ! $this_chroot_failed; then
sudo rm -rf "$mock_root"
fi
done
if $failed; then
echo "Aborting."
exit 1
fi
# remove mock system cache, we will setup proxies to do the caching and this
# takes lots of space between runs
shopt -u nullglob
sudo rm -Rf /var/cache/mock/*
# restore the permissions in the working dir, as sometimes it leaves files
# owned by root and then the 'cleanup workspace' from jenkins job fails to
# clean and breaks the jobs
sudo chown -R "$USER" "$WORKSPACE"
# Drop all left over libvirt domains
for UUID in $(virsh list --all --uuid); do
virsh destroy $UUID || :
sleep 2
virsh undefine --remove-all-storage --storage vda --snapshots-metadata $UUID || :
done
[ovirt_3.6_system-tests] $ /bin/bash -xe /tmp/hudson2419942323115778781.sh
+ echo shell-scripts/mock_cleanup.sh
shell-scripts/mock_cleanup.sh
+ shopt -s nullglob
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
+ cat
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
+ logs=(./*log ./*/logs)
+ [[ -n ./ovirt-system-tests/logs ]]
+ tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs
./ovirt-system-tests/logs/
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.init/
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.init/stdout_stderr.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.init/state.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.init/build.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.init/root.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.install_packages/
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.install_packages/stdout_stderr.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.install_packages/state.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.install_packages/build.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.install_packages/root.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.clean_rpmdb/
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.clean_rpmdb/stdout_stderr.log
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.basic_suite_3.6.sh/
./ovirt-system-tests/logs/mocker-epel-7-x86_64.el7.basic_suite_3.6.sh/basic_suite_3.6.sh.log
+ rm -rf ./ovirt-system-tests/logs
+ failed=false
+ mock_confs=("$WORKSPACE"/*/mocker*)
+ for mock_conf_file in '"${mock_confs[@]}"'
+ [[ -n <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...> ]]
+ echo 'Cleaning up mock '
Cleaning up mock
+ mock_root=mocker-epel-7-x86_64.el7.cfg
+ mock_root=mocker-epel-7-x86_64.el7
+ my_mock=/usr/bin/mock
+ my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests'>
+ my_mock+=' --root=mocker-epel-7-x86_64.el7'
+ my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/'>
+ echo 'Killing all mock orphan processes, if any.'
Killing all mock orphan processes, if any.
+ /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests> --root=mocker-epel-7-x86_64.el7 --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/> --orphanskill
WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests....> Using default...
INFO: mock.py version 1.2.21 starting (python version = 3.5.2)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Finish: run
++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests...>
+ mock_root=epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ [[ -n epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c ]]
+ mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
++ mount
++ awk '{print $3}'
++ grep epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ :
+ [[ -n '' ]]
+ false
+ shopt -u nullglob
+ sudo rm -Rf /var/cache/mock/epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
++ virsh list --all --uuid
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 1
Recording test results
Archiving artifacts
8 years, 1 month
Fwd: Linode Support Ticket 7156775 - Critical Xen Maintenance / Reboot Schedule
by Karsten Wade
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--g9BeT0r2PKq1TcR1b4AXeiU3UXAxwkWIQ
Content-Type: multipart/mixed; boundary="XoqTt80bHUtfEg8mmbt6CVL3pSVUp5UAI";
protected-headers="v1"
From: Karsten Wade <quaid(a)iquaid.org>
To: infra <infra(a)ovirt.org>
Message-ID: <91f74acb-36c7-f954-2362-73af9d80d626(a)iquaid.org>
Subject: Fwd: Linode Support Ticket 7156775 - Critical Xen Maintenance /
Reboot Schedule
References: <2108364301.128572.1479261262542.JavaMail.cfusion(a)mail2.linode.com>
In-Reply-To: <2108364301.128572.1479261262542.JavaMail.cfusion(a)mail2.linode.com>
--XoqTt80bHUtfEg8mmbt6CVL3pSVUp5UAI
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
The irony here is palpable ...
What's the plan or blocker on migrating off this Linode instance?
-------- Forwarded Message --------
Subject: Linode Support Ticket 7156775 - Critical Xen Maintenance /
Reboot Schedule
Date: Tue, 15 Nov 2016 20:54:22 -0500 (EST)
From: support(a)linode.com
To: quaid(a)iquaid.org
Hello,
Linode has received a Xen Security Advisory (XSA) that requires us to
perform updates to our legacy Xen host servers. We must complete these
updates before November 22nd. For general information, please see our
status page post below:
https://status.linode.com/incidents/6lqznlfwlv1r
Your Linodes have been assigned a maintenance window in which a reboot
will occur. These times are listed within the Linode Manager[1] in the
timezone set in your user's My Profile[2]. Your schedule in UTC timezone
is as follows:
* 2016-11-21 12:00:00 PM UTC - linodeXXXXXXXX
Upgrading to KVM will allow you to avoid this maintenance entirely. You
can use the =E2=80=9CUpgrade to KVM=E2=80=9D link in your Linode=E2=80=99=
s dashboard to move to
KVM. Please note that KVM upgrades are not available in Tokyo at this
time. More KVM upgrading information can be found here:
https://www.linode.com/docs/platform/kvm-reference
We strongly recommend you read through our Reboot guide, which will help
you prepare your Linode for these reboots. After going through that
guide, we recommend you reboot your Linode at a convenient time to
ensure your services come up correctly. The Reboot guide is available her=
e:
https://www.linode.com/docs/uptime/reboot-survival-guide
During the maintenance window, your Linodes will be cleanly shut down
while we perform the updates. Your Linodes will be inaccessible during
this time. A two-hour window is allocated, however the actual downtime
should be much less. After the maintenance, each Linode will be returned
to its last state (running or powered off).
Please let us know if there is anything we can do to assist.
[1] <https://manager.linode.com/linodes>
[2] <https://manager.linode.com/profile>
--Linode
---
https://manager.linode.com/support/ticket/7156775
--XoqTt80bHUtfEg8mmbt6CVL3pSVUp5UAI--
--g9BeT0r2PKq1TcR1b4AXeiU3UXAxwkWIQ
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlgr5TcACgkQ2ZIOBq0ODEFbdgCgiedG8EAYDuwjg7HFPOpWAC0S
5K0An3tyA9/PzAv/HPLb2dgUd/RuHP0P
=rBqw
-----END PGP SIGNATURE-----
--g9BeT0r2PKq1TcR1b4AXeiU3UXAxwkWIQ--
8 years, 1 month
