Another job failure: "Command failed. See logs for output"
Gil Shinar
gshinar at redhat.com
Tue May 17 07:36:13 UTC 2016
Hi,
I have tried to debug this issue yesterday with the help of Barak.
I don't think that the problem is history as I saw another failure like
this yesterday (it happens from time to time). Problem is the slave it
builds on being taken by another build and cleans the workspace. I tried to
look for logs under the workspace and they missing probably because a newer
build is being built on the same slave.
Gil
On Tue, May 17, 2016 at 8:52 AM, Eyal Edri <eedri at redhat.com> wrote:
> Can we increase the history for builds on that job? We should be able to
> debug jobs at least a week back. Artifacts are not needed
> On May 16, 2016 11:42 PM, "David Caro" <dcaro at redhat.com> wrote:
>
>> On 05/15 23:23, Nir Soffer wrote:
>> > Another instance:
>> >
>> > 19:38:15 Start: yum install
>> > 19:38:22 ERROR: Command failed. See logs for output.
>>
>> That means that there was an issue with the yum repos, most common causes
>> are:
>>
>> * The repos were actually down (mirrors fail once a day usually, while
>> syncing
>> the rpms, though the cause is just a guess)
>> * Repoproxy (that we use to cache rpms) was overloaded
>>
>> Both jobs histories are now gone, in order to be able to debug those
>> issues
>> (for the next time), try looking into the mock logs, under the 'logs.tgz'
>> file
>> that's archived in the job, then under the path:
>> /./vdsm/logs/mocker-epel-7-x86_64.el7.init/
>> or
>> /./vdsm/logs/mocker-epel-7-x86_64.el7.install_packages/
>>
>> Usually in the log named 'root.log' (check which one has more size).
>>
>> >
>> >
>> http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/1213/console
>>
>> >
>> > On Sun, May 15, 2016 at 11:21 PM, Nir Soffer <nsoffer at redhat.com>
>> wrote:
>> > >
>> http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/1209/console
>> > >
>> > > 19:30:56 Start: yum install
>> > > 19:31:03 ERROR: Command failed. See logs for output.
>> > >
>> > > Including the to make sure it will not disapper
>> > > ----
>> > >
>> > > 19:30:48 Triggered by Gerrit: https://gerrit.ovirt.org/56550
>> > > 19:30:48 Building remotely on fc23-vm07.phx.ovirt.org (fc23 nested)
>> in
>> > > workspace /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64
>> > > 19:30:48 > git rev-parse --is-inside-work-tree # timeout=10
>> > > 19:30:49 Fetching changes from the remote Git repository
>> > > 19:30:49 > git config remote.origin.url
>> > > git://gerrit.ovirt.org/vdsm.git # timeout=10
>> > > 19:30:49 Cleaning workspace
>> > > 19:30:49 > git rev-parse --verify HEAD # timeout=10
>> > > 19:30:49 Resetting working tree
>> > > 19:30:49 > git reset --hard # timeout=10
>> > > 19:30:49 > git clean -fdx # timeout=10
>> > > 19:30:49 Pruning obsolete local branches
>> > > 19:30:49 Fetching upstream changes from git://
>> gerrit.ovirt.org/vdsm.git
>> > > 19:30:49 > git --version # timeout=10
>> > > 19:30:49 > git -c core.askpass=true fetch --tags --progress
>> > > git://gerrit.ovirt.org/vdsm.git refs/changes/50/56550/6 --prune
>> > > 19:30:52 > git rev-parse
>> > > 0940208483f3a21261eb5d725348e65c23becdc0^{commit} # timeout=10
>> > > 19:30:52 Checking out Revision
>> 0940208483f3a21261eb5d725348e65c23becdc0 (master)
>> > > 19:30:52 > git config core.sparsecheckout # timeout=10
>> > > 19:30:52 > git checkout -f 0940208483f3a21261eb5d725348e65c23becdc0
>> > > 19:30:52 > git rev-parse FETCH_HEAD^{commit} # timeout=10
>> > > 19:30:52 > git rev-list a09c577837b939096f97108fdbbcafe5980d4a0d #
>> timeout=10
>> > > 19:30:52 > git branch -a # timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/master^{commit} # timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.1^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.2^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.3^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.3.0^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.4^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.5^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.5-gluster^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.5.0^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.5.2^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.5.4^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.5.6^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.6^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.6.0^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.6.1^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.6.2^{commit} #
>> timeout=10
>> > > 19:30:52 > git rev-parse remotes/origin/ovirt-3.6.3^{commit} #
>> timeout=10
>> > > 19:30:53 > git rev-parse --is-inside-work-tree # timeout=10
>> > > 19:30:53 Fetching changes from the remote Git repository
>> > > 19:30:53 > git config remote.origin.url
>> > > git://gerrit.ovirt.org/jenkins.git # timeout=10
>> > > 19:30:53 Cleaning workspace
>> > > 19:30:53 > git rev-parse --verify HEAD # timeout=10
>> > > 19:30:53 Resetting working tree
>> > > 19:30:53 > git reset --hard # timeout=10
>> > > 19:30:53 > git clean -fdx # timeout=10
>> > > 19:30:53 Pruning obsolete local branches
>> > > 19:30:53 Fetching upstream changes from git://
>> gerrit.ovirt.org/jenkins.git
>> > > 19:30:53 > git --version # timeout=10
>> > > 19:30:53 > git -c core.askpass=true fetch --tags --progress
>> > > git://gerrit.ovirt.org/jenkins.git
>> +refs/heads/*:refs/remotes/origin/*
>> > > --prune
>> > > 19:30:53 > git rev-parse origin/master^{commit} # timeout=10
>> > > 19:30:53 Checking out Revision
>> > > 68a517c3ccc1fe58a6c52646e3a99f3ab22ddab9 (origin/master)
>> > > 19:30:53 > git config core.sparsecheckout # timeout=10
>> > > 19:30:53 > git checkout -f 68a517c3ccc1fe58a6c52646e3a99f3ab22ddab9
>> > > 19:30:53 > git rev-list 68a517c3ccc1fe58a6c52646e3a99f3ab22ddab9 #
>> timeout=10
>> > > 19:30:53 > git branch -a # timeout=10
>> > > 19:30:53 > git rev-parse remotes/origin/master^{commit} # timeout=10
>> > > 19:30:53 [vdsm_master_check-patch-el7-x86_64] $ /bin/bash -e
>> > > /tmp/hudson8265618520208925180.sh
>> > > 19:30:53 shell-scripts/cleanup_slave.sh
>> > > 19:30:53
>> ###################################################################
>> > > 19:30:53 # Cleaning up slave
>> #
>> > > 19:30:53
>> ###################################################################
>> > > 19:30:53 Filesystem Size Used Avail
>> > > Use% Mounted on
>> > > 19:30:53 devtmpfs 3.9G 0
>> 3.9G 0% /dev
>> > > 19:30:53 tmpfs 3.9G 0 3.9G
>> > > 0% /dev/shm
>> > > 19:30:53 tmpfs 3.9G 2.1M
>> 3.9G 1% /run
>> > > 19:30:53 tmpfs 3.9G 0 3.9G
>> > > 0% /sys/fs/cgroup
>> > > 19:30:53 /dev/mapper/fedora--server_fc23--base-root 35G 6.4G
>> 27G 20% /
>> > > 19:30:53 tmpfs 3.9G 244K
>> 3.9G 1% /tmp
>> > > 19:30:53 /dev/vda1 988M 94M
>> 828M 11% /boot
>> > > 19:30:53 tmpfs 799M 0 799M
>> > > 0% /run/user/1000
>> > > 19:30:53
>> ---------------------------------------------------------------
>> > > 19:30:53 Cleaning up postgres databases
>> > > 19:30:53 Postgres installation not found, skipping
>> > > 19:30:53 Cleaning up journal logs (if any)
>> > > 19:30:53 Redirecting to /bin/systemctl restart
>> systemd-journald.service
>> > > 19:30:54 Cleaning up /var/tmp
>> > > 19:30:54 done
>> > > 19:30:54 Emptying some common logs
>> > > 19:30:54 /var/log/wtmp
>> > > 19:30:54 Done
>> > > 19:30:54 Making sure there are no device mappings...
>> > > 19:30:54 Removing the used loop devices...
>> > > 19:30:54 Redirecting to /bin/systemctl restart libvirtd.service
>> > > 19:30:54
>> ---------------------------------------------------------------
>> > > 19:30:54 Filesystem Size Used Avail
>> > > Use% Mounted on
>> > > 19:30:54 devtmpfs 3.9G 0
>> 3.9G 0% /dev
>> > > 19:30:54 tmpfs 3.9G 0 3.9G
>> > > 0% /dev/shm
>> > > 19:30:54 tmpfs 3.9G 2.1M
>> 3.9G 1% /run
>> > > 19:30:54 tmpfs 3.9G 0 3.9G
>> > > 0% /sys/fs/cgroup
>> > > 19:30:54 /dev/mapper/fedora--server_fc23--base-root 35G 6.4G
>> 27G 20% /
>> > > 19:30:54 tmpfs 3.9G 244K
>> 3.9G 1% /tmp
>> > > 19:30:54 /dev/vda1 988M 94M
>> 828M 11% /boot
>> > > 19:30:54 tmpfs 799M 0 799M
>> > > 0% /run/user/1000
>> > > 19:30:54
>> ###################################################################
>> > > 19:30:54 # Slave cleanup done
>> #
>> > > 19:30:54
>> ###################################################################
>> > > 19:30:54 [vdsm_master_check-patch-el7-x86_64] $ /bin/bash -xe
>> > > /tmp/hudson7354644880288923842.sh
>> > > 19:30:54 + echo shell-scripts/global_setup.sh
>> > > 19:30:54 shell-scripts/global_setup.sh
>> > > 19:30:54 + rm -rf
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/tmp
>> > > 19:30:54 + mkdir -p
>> > > /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/tmp
>> > > 19:30:54 [vdsm_master_check-patch-el7-x86_64] $ /bin/bash -xe
>> > > /tmp/hudson4023353013258685526.sh
>> > > 19:30:54 + echo shell-scripts/mock_setup.sh
>> > > 19:30:54 shell-scripts/mock_setup.sh
>> > > 19:30:54 + shopt -s nullglob
>> > > 19:30:54 + [[ '' == \t\r\u\e ]]
>> > > 19:30:54 + failed=false
>> > > 19:30:54 +
>> mock_dir=/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/mock
>> > > 19:30:54 + chroots=("$WORKSPACE"/mock/*)
>> > > 19:30:54 + false
>> > > 19:30:54 + sudo rm -Rf mock mock-cache exported-artifacts
>> > > 19:30:54 + mkdir -p mock exported-artifacts
>> > > 19:30:54 + chgrp mock mock
>> > > /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/exported-artifacts
>> > > 19:30:54 + chmod g+rws mock
>> > > 19:30:54 + sudo touch
>> > > 19:30:54 + :
>> > > 19:30:54 + sudo yum clean all
>> > > 19:30:54 Yum command has been deprecated, redirecting to '/usr/bin/dnf
>> > > clean all'.
>> > > 19:30:54 See 'man dnf' and 'man yum2dnf' for more information.
>> > > 19:30:54 To transfer transaction metadata from yum to DNF, run:
>> > > 19:30:54 'dnf install python-dnf-plugins-extras-migrate && dnf-2
>> migrate'
>> > > 19:30:54
>> > > 19:30:55 Cleaning repos: updates ci-tools fedora
>> > > 19:30:55 Cleaning up Everything
>> > > 19:30:55 + exit 0
>> > > 19:30:55 [vdsm_master_check-patch-el7-x86_64] $ /bin/bash -e
>> > > /tmp/hudson4405890361943185034.sh
>> > > 19:30:55 shell-scripts/standard_check_patch.sh
>> > > 19:30:55 ##########################################################
>> > > 19:30:55 ##########################################################
>> > > 19:30:55 ## Sun May 15 15:30:55 EDT 2016 Running env:
>> el7:epel-7-x86_64
>> > > 19:30:55 ##########################################################
>> > > 19:30:55
>> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
>> > > 19:30:55 @@ Sun May 15 15:30:55 EDT 2016 Running chroot for script:
>> > > automation/check-patch.sh
>> > > 19:30:55
>> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
>> > > 19:30:55 Using base mock conf
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/jenkins/mock_configs/epel-7-x86_64.cfg
>> > > 19:30:55 Using proxified config
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/jenkins/mock_configs/epel-7-x86_64_proxied.cfg
>> > > 19:30:55 Generating temporary mock conf
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/mocker-epel-7-x86_64.el7
>> > > 19:30:55 Skipping mount points
>> > > 19:30:55 Using temp chroot =
>> epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc
>> > > 19:30:55 Adding repo ovirt-snapshot ->
>> > > http://resources.ovirt.org/pub/ovirt-master-snapshot/rpm/el7
>> > > 19:30:55 Adding repo ovirt-snapshot-static ->
>> > > http://resources.ovirt.org/pub/ovirt-master-snapshot-static/rpm/el7
>> > > 19:30:55 Adding repo centos-glusterfs ->
>> > > http://mirror.centos.org/centos/7/storage/x86_64/gluster-3.7/
>> > > 19:30:55 Adding repo centos-ovirt40-candidate ->
>> > > http://cbs.centos.org/repos/virt7-ovirt-40-candidate/x86_64/os/
>> > > 19:30:55 Adding repo lago ->
>> > > http://resources.ovirt.org/repos/lago/stable/0.0/rpm/el7
>> > > 19:30:55 Adding repo epel ->
>> http://download.fedoraproject.org/pub/epel/7/x86_64
>> > > 19:30:55 ========== Initializing chroot
>> > > 19:30:55 mock \
>> > > 19:30:55
>> > >
>> --configdir="/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm"
>> > > \
>> > > 19:30:55 --root="mocker-epel-7-x86_64.el7" \
>> > > 19:30:55 --resultdir="logs/mocker-epel-7-x86_64.el7.init" \
>> > > 19:30:55 --init
>> > > 19:30:55 WARNING: Could not find required logging config file:
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/logging.ini.
>> > > Using default...
>> > > 19:30:55 INFO: mock.py version 1.2.14 starting (python version =
>> 3.4.3)...
>> > > 19:30:55 Start: init plugins
>> > > 19:30:55 INFO: selinux enabled
>> > > 19:30:55 Finish: init plugins
>> > > 19:30:55 Start: run
>> > > 19:30:55 Start: clean chroot
>> > > 19:30:55 Finish: clean chroot
>> > > 19:30:55 Start: chroot init
>> > > 19:30:55 INFO: calling preinit hooks
>> > > 19:30:55 INFO: enabled root cache
>> > > 19:30:55 INFO: enabled yum cache
>> > > 19:30:55 Start: cleaning yum metadata
>> > > 19:30:55 Finish: cleaning yum metadata
>> > > 19:30:55 INFO: enabled ccache
>> > > 19:30:55 Mock Version: 1.2.14
>> > > 19:30:55 INFO: Mock Version: 1.2.14
>> > > 19:30:56 Start: yum install
>> > > 19:31:03 ERROR: Command failed. See logs for output.
>> > > 19:31:03 # /usr/bin/yum-deprecated --installroot
>> > > /var/lib/mock/epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc/root/
>> > > --releasever 7 install @buildsys-build --setopt=tsflags=nocontexts
>> > > 19:31:03 Init took 8 seconds
>> > > 19:31:03 ============================
>> > > 19:31:03
>> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
>> > > 19:31:03 @@ Sun May 15 15:31:03 EDT 2016 automation/check-patch.sh
>> > > chroot finished
>> > > 19:31:03 @@ took 8 seconds
>> > > 19:31:03 @@ rc = 1
>> > > 19:31:03
>> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
>> > > 19:31:03 ##########################################################
>> > > 19:31:03 ## Sun May 15 15:31:03 EDT 2016 Finished env:
>> el7:epel-7-x86_64
>> > > 19:31:03 ## took 8 seconds
>> > > 19:31:03 ## rc = 1
>> > > 19:31:03 ##########################################################
>> > > 19:31:03 ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
>> > > 19:31:03 ##! Last 20 log enties:
>> > > logs/mocker-epel-7-x86_64.el7.init/stdout_stderr.log
>> > > 19:31:03 ##!
>> > > 19:31:03 ========== Initializing chroot
>> > > 19:31:03 mock \
>> > > 19:31:03
>> > >
>> --configdir="/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm"
>> > > \
>> > > 19:31:03 --root="mocker-epel-7-x86_64.el7" \
>> > > 19:31:03 --resultdir="logs/mocker-epel-7-x86_64.el7.init" \
>> > > 19:31:03 --init
>> > > 19:31:03 Init took 8 seconds
>> > > 19:31:03 ============================
>> > > 19:31:03 ##!
>> > > 19:31:03 ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> > > 19:31:03 ##!########################################################
>> > > 19:31:03 Build step 'Execute shell' marked build as failure
>> > > 19:31:03 Performing Post build task...
>> > > 19:31:03 Match found for :.* : True
>> > > 19:31:03 Logical operation result is TRUE
>> > > 19:31:03 Running script : #!/bin/bash -xe
>> > > 19:31:03 echo "shell-scripts/mock_cleanup.sh"
>> > > 19:31:03
>> > > 19:31:03 shopt -s nullglob
>> > > 19:31:03
>> > > 19:31:03
>> > > 19:31:03 WORKSPACE="$PWD"
>> > > 19:31:03
>> > > 19:31:03 # Make clear this is the cleanup, helps reading the jenkins
>> logs
>> > > 19:31:03 cat <<EOC
>> > > 19:31:03
>> _______________________________________________________________________
>> > > 19:31:03
>> #######################################################################
>> > > 19:31:03 #
>> #
>> > > 19:31:03 # CLEANUP
>> #
>> > > 19:31:03 #
>> #
>> > > 19:31:03
>> #######################################################################
>> > > 19:31:03 EOC
>> > > 19:31:03
>> > > 19:31:03
>> > > 19:31:03 # Archive the logs, we want them anyway
>> > > 19:31:03 logs=(
>> > > 19:31:03 ./*log
>> > > 19:31:03 ./*/logs
>> > > 19:31:03 )
>> > > 19:31:03 if [[ "$logs" ]]; then
>> > > 19:31:03 tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
>> > > 19:31:03 rm -rf "${logs[@]}"
>> > > 19:31:03 fi
>> > > 19:31:03
>> > > 19:31:03 # stop any processes running inside the chroot
>> > > 19:31:03 failed=false
>> > > 19:31:03 mock_confs=("$WORKSPACE"/*/mocker*)
>> > > 19:31:03 # Clean current jobs mockroot if any
>> > > 19:31:03 for mock_conf_file in "${mock_confs[@]}"; do
>> > > 19:31:03 [[ "$mock_conf_file" ]] || continue
>> > > 19:31:03 echo "Cleaning up mock $mock_conf"
>> > > 19:31:03 mock_root="${mock_conf_file##*/}"
>> > > 19:31:03 mock_root="${mock_root%.*}"
>> > > 19:31:03 my_mock="/usr/bin/mock"
>> > > 19:31:03 my_mock+=" --configdir=${mock_conf_file%/*}"
>> > > 19:31:03 my_mock+=" --root=${mock_root}"
>> > > 19:31:03 my_mock+=" --resultdir=$WORKSPACE"
>> > > 19:31:03
>> > > 19:31:03 #TODO: investigate why mock --clean fails to umount
>> > > certain dirs sometimes,
>> > > 19:31:03 #so we can use it instead of manually doing all this.
>> > > 19:31:03 echo "Killing all mock orphan processes, if any."
>> > > 19:31:03 $my_mock \
>> > > 19:31:03 --orphanskill \
>> > > 19:31:03 || {
>> > > 19:31:03 echo "ERROR: Failed to kill orphans on $chroot."
>> > > 19:31:03 failed=true
>> > > 19:31:03 }
>> > > 19:31:03
>> > > 19:31:03 mock_root="$(\
>> > > 19:31:03 grep \
>> > > 19:31:03 -Po "(?<=config_opts\['root'\] = ')[^']*" \
>> > > 19:31:03 "$mock_conf_file" \
>> > > 19:31:03 )" || :
>> > > 19:31:03 [[ "$mock_root" ]] || continue
>> > > 19:31:03 mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
>> || :
>> > > 19:31:03 if [[ "$mounts" ]]; then
>> > > 19:31:03 echo "Found mounted dirs inside the chroot $chroot.
>> > > Trying to umount."
>> > > 19:31:03 fi
>> > > 19:31:03 for mount in "${mounts[@]}"; do
>> > > 19:31:03 sudo umount "$mount" \
>> > > 19:31:03 || {
>> > > 19:31:03 echo "ERROR: Failed to umount $mount."
>> > > 19:31:03 failed=true
>> > > 19:31:03 }
>> > > 19:31:03 done
>> > > 19:31:03 done
>> > > 19:31:03
>> > > 19:31:03 # Clean any leftover chroot from other jobs
>> > > 19:31:03 for mock_root in /var/lib/mock/*; do
>> > > 19:31:03 this_chroot_failed=false
>> > > 19:31:03 mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
>> || :
>> > > 19:31:03 if [[ "$mounts" ]]; then
>> > > 19:31:03 echo "Found mounted dirs inside the chroot
>> $mock_root." \
>> > > 19:31:03 "Trying to umount."
>> > > 19:31:03 fi
>> > > 19:31:03 for mount in "${mounts[@]}"; do
>> > > 19:31:03 sudo umount "$mount" \
>> > > 19:31:03 || {
>> > > 19:31:03 echo "ERROR: Failed to umount $mount."
>> > > 19:31:03 failed=true
>> > > 19:31:03 this_chroot_failed=true
>> > > 19:31:03 }
>> > > 19:31:03 done
>> > > 19:31:03 if ! $this_chroot_failed; then
>> > > 19:31:03 sudo rm -rf "$mock_root"
>> > > 19:31:03 fi
>> > > 19:31:03 done
>> > > 19:31:03
>> > > 19:31:03 if $failed; then
>> > > 19:31:03 echo "Aborting."
>> > > 19:31:03 exit 1
>> > > 19:31:03 fi
>> > > 19:31:03
>> > > 19:31:03 # remove mock system cache, we will setup proxies to do the
>> > > caching and this
>> > > 19:31:03 # takes lots of space between runs
>> > > 19:31:03 shopt -u nullglob
>> > > 19:31:03 sudo rm -Rf /var/cache/mock/*
>> > > 19:31:03
>> > > 19:31:03 # restore the permissions in the working dir, as sometimes it
>> > > leaves files
>> > > 19:31:03 # owned by root and then the 'cleanup workspace' from jenkins
>> > > job fails to
>> > > 19:31:03 # clean and breaks the jobs
>> > > 19:31:03 sudo chown -R "$USER" "$WORKSPACE"
>> > > 19:31:03
>> > > 19:31:03 [vdsm_master_check-patch-el7-x86_64] $ /bin/bash -xe
>> > > /tmp/hudson3681642653853563663.sh
>> > > 19:31:03 + echo shell-scripts/mock_cleanup.sh
>> > > 19:31:03 shell-scripts/mock_cleanup.sh
>> > > 19:31:03 + shopt -s nullglob
>> > > 19:31:03 +
>> WORKSPACE=/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64
>> > > 19:31:03 + cat
>> > > 19:31:03
>> _______________________________________________________________________
>> > > 19:31:03
>> #######################################################################
>> > > 19:31:03 #
>> #
>> > > 19:31:03 # CLEANUP
>> #
>> > > 19:31:03 #
>> #
>> > > 19:31:03
>> #######################################################################
>> > > 19:31:03 + logs=(./*log ./*/logs)
>> > > 19:31:03 + [[ -n ./vdsm/logs ]]
>> > > 19:31:03 + tar cvzf exported-artifacts/logs.tgz ./vdsm/logs
>> > > 19:31:03 ./vdsm/logs/
>> > > 19:31:03 ./vdsm/logs/mocker-epel-7-x86_64.el7.init/
>> > > 19:31:03 ./vdsm/logs/mocker-epel-7-x86_64.el7.init/root.log
>> > > 19:31:03 ./vdsm/logs/mocker-epel-7-x86_64.el7.init/state.log
>> > > 19:31:03 ./vdsm/logs/mocker-epel-7-x86_64.el7.init/stdout_stderr.log
>> > > 19:31:03 ./vdsm/logs/mocker-epel-7-x86_64.el7.init/build.log
>> > > 19:31:03 + rm -rf ./vdsm/logs
>> > > 19:31:03 + failed=false
>> > > 19:31:03 + mock_confs=("$WORKSPACE"/*/mocker*)
>> > > 19:31:03 + for mock_conf_file in '"${mock_confs[@]}"'
>> > > 19:31:03 + [[ -n
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/mocker-epel-7-x86_64.el7.cfg
>> > > ]]
>> > > 19:31:03 + echo 'Cleaning up mock '
>> > > 19:31:03 Cleaning up mock
>> > > 19:31:03 + mock_root=mocker-epel-7-x86_64.el7.cfg
>> > > 19:31:03 + mock_root=mocker-epel-7-x86_64.el7
>> > > 19:31:03 + my_mock=/usr/bin/mock
>> > > 19:31:03 + my_mock+='
>> > >
>> --configdir=/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm'
>> > > 19:31:03 + my_mock+=' --root=mocker-epel-7-x86_64.el7'
>> > > 19:31:03 + my_mock+='
>> > >
>> --resultdir=/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64'
>> > > 19:31:03 + echo 'Killing all mock orphan processes, if any.'
>> > > 19:31:03 Killing all mock orphan processes, if any.
>> > > 19:31:03 + /usr/bin/mock
>> > >
>> --configdir=/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm
>> > > --root=mocker-epel-7-x86_64.el7
>> > > --resultdir=/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64
>> > > --orphanskill
>> > > 19:31:04 WARNING: Could not find required logging config file:
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/logging.ini.
>> > > Using default...
>> > > 19:31:04 INFO: mock.py version 1.2.14 starting (python version =
>> 3.4.3)...
>> > > 19:31:04 Start: init plugins
>> > > 19:31:04 INFO: selinux enabled
>> > > 19:31:04 Finish: init plugins
>> > > 19:31:04 Start: run
>> > > 19:31:04 Finish: run
>> > > 19:31:04 ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*'
>> > >
>> /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/mocker-epel-7-x86_64.el7.cfg
>> > > 19:31:04 + mock_root=epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc
>> > > 19:31:04 + [[ -n epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc ]]
>> > > 19:31:04 + mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
>> > > 19:31:04 ++ mount
>> > > 19:31:04 ++ grep epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc
>> > > 19:31:04 ++ awk '{print $3}'
>> > > 19:31:04 + :
>> > > 19:31:04 + [[ -n '' ]]
>> > > 19:31:04 + for mock_root in '/var/lib/mock/*'
>> > > 19:31:04 + this_chroot_failed=false
>> > > 19:31:04 + mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
>> > > 19:31:04 ++ mount
>> > > 19:31:04 ++ grep
>> /var/lib/mock/epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc
>> > > 19:31:04 ++ awk '{print $3}'
>> > > 19:31:04 + :
>> > > 19:31:04 + [[ -n '' ]]
>> > > 19:31:04 + false
>> > > 19:31:04 + sudo rm -rf
>> > > /var/lib/mock/epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc
>> > > 19:31:04 + false
>> > > 19:31:04 + shopt -u nullglob
>> > > 19:31:04 + sudo rm -Rf
>> > > /var/cache/mock/epel-7-x86_64-46ef12ce4362729a0f4c411e00edd8fc
>> > > 19:31:04 + sudo chown -R jenkins
>> > > /home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64
>> > > 19:31:04 POST BUILD TASK : SUCCESS
>> > > 19:31:04 END OF POST BUILD TASK : 0
>> > > 19:31:04 Archiving artifacts
>> > > 19:31:04 Build step 'Groovy Postbuild' marked build as failure
>> > > 19:31:04 Started calculate disk usage of build
>> > > 19:31:04 Finished Calculation of disk usage of build in 0 seconds
>> > > 19:31:04 Started calculate disk usage of workspace
>> > > 19:31:04 Finished Calculation of disk usage of workspace in 0 seconds
>> > > 19:31:04 Finished: FAILURE
>> > _______________________________________________
>> > Infra mailing list
>> > Infra at ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/infra
>>
>> --
>> David Caro
>>
>> Red Hat S.L.
>> Continuous Integration Engineer - EMEA ENG Virtualization R&D
>>
>> Tel.: +420 532 294 605
>> Email: dcaro at redhat.com
>> IRC: dcaro|dcaroest@{freenode|oftc|redhat}
>> Web: www.redhat.com
>> RHT Global #: 82-62605
>>
>> _______________________________________________
>> Infra mailing list
>> Infra at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>
>>
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20160517/a6a2954d/attachment.html>
More information about the Infra
mailing list