What would you expect to happen if a system test fail which can be from any project build in the past day?
This is not a place for sending automated error message.
On Tue, Jun 28, 2016 at 7:58 PM, Eyal Edri <eedri@redhat.com> wrote:
> Why not?
> If this is a real regression?
>
> On Jun 28, 2016 7:54 PM, "Nir Soffer" <nsoffer@redhat.com> wrote:
>>
>> I think this was sent by mistake to devel@ovirt.org, please do not
>> send such message to this list.
>>
>> On Tue, Jun 28, 2016 at 7:49 PM, <jenkins@jenkins.phx.ovirt.org> wrote:
>> > See <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/168/changes>
>> >
>> > Changes:
>> >
>> > [Ondra Machacek] Deploy FreeIPA to storage machine
>> >
>> > ------------------------------------------
>> > [...truncated 554 lines...]
>> > ## took 2194 seconds
>> > ## rc = 1
>> > ##########################################################
>> > ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
>> > ##! Last 20 log enties:
>> > logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/basic_suite_3.6.sh.log
>> > ##!
>> > @ Collect artifacts:
>> > # ovirt-role metadata entry will be soon deprecated, instead you
>> > should use the vm-provider entry in the domain definiton and set it no one
>> > of: ovirt-node, ovirt-engine, ovirt-host
>> > # [Thread-1] lago_basic_suite_3_6_engine:
>> > # [Thread-2] lago_basic_suite_3_6_host1:
>> > # [Thread-3] lago_basic_suite_3_6_host0:
>> > # [Thread-4] lago_basic_suite_3_6_storage:
>> > # [Thread-3] lago_basic_suite_3_6_host0: Success (in 0:00:16)
>> > # [Thread-2] lago_basic_suite_3_6_host1: Success (in 0:00:16)
>> > # [Thread-1] lago_basic_suite_3_6_engine: Success (in 0:00:16)
>> > # [Thread-4] lago_basic_suite_3_6_storage: Success (in 0:00:16)
>> > @ Collect artifacts: Success (in 0:00:16)
>> >
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests>
>> > @@@@ ERROR: Failed running
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/basic_suite_3.6/test-scenarios/002_bootstrap.py>
>> > #########################
>> > ======== Cleaning up
>> > ----------- Cleaning with lago
>> > ----------- Cleaning with lago done
>> > ======== Cleanup done
>> > Took 2033 seconds
>> > ===================================
>> > ##!
>> > ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> > ##!########################################################
>> > ##########################################################
>> > Build step 'Execute shell' marked build as failure
>> > Performing Post build task...
>> > Match found for :.* : True
>> > Logical operation result is TRUE
>> > Running script : #!/bin/bash -xe
>> > echo 'shell_scripts/system_tests.collect_logs.sh'
>> >
>> > #
>> > # Required jjb vars:
>> > # version
>> > #
>> > VERSION=3.6
>> > SUITE_TYPE=
>> >
>> > WORKSPACE="$PWD"
>> > OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
>> > TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
>> >
>> > rm -rf "$WORKSPACE/exported-artifacts"
>> > mkdir -p "$WORKSPACE/exported-artifacts"
>> >
>> > if [[ -d "$TESTS_LOGS" ]]; then
>> > mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
>> > fi
>> >
>> > [ovirt_3.6_system-tests] $ /bin/bash -xe
>> > /tmp/hudson3155110657872379488.sh
>> > + echo shell_scripts/system_tests.collect_logs.sh
>> > shell_scripts/system_tests.collect_logs.sh
>> > + VERSION=3.6
>> > + SUITE_TYPE=
>> > + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
>> > + OVIRT_SUITE=3.6
>> > +
>> > TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/exported-artifacts>
>> > + rm -rf
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/168/artifact/exported-artifacts>
>> > + mkdir -p
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/168/artifact/exported-artifacts>
>> > + [[ -d
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/exported-artifacts>
>> > ]]
>> > + mv
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/exported-artifacts/lago_logs>
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/exported-artifacts/nosetests-001_initialize_engine.py.xml>
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/exported-artifacts/nosetests-002_bootstrap.py.xml>
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/exported-artifacts/test_logs>
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/168/artifact/exported-artifacts/>
>> > POST BUILD TASK : SUCCESS
>> > END OF POST BUILD TASK : 0
>> > Match found for :.* : True
>> > Logical operation result is TRUE
>> > Running script : #!/bin/bash -xe
>> > echo "shell-scripts/mock_cleanup.sh"
>> >
>> > shopt -s nullglob
>> >
>> >
>> > WORKSPACE="$PWD"
>> >
>> > # Make clear this is the cleanup, helps reading the jenkins logs
>> > cat <<EOC
>> > _______________________________________________________________________
>> > #######################################################################
>> > # #
>> > # CLEANUP #
>> > # #
>> > #######################################################################
>> > EOC
>> >
>> >
>> > # Archive the logs, we want them anyway
>> > logs=(
>> > ./*log
>> > ./*/logs
>> > )
>> > if [[ "$logs" ]]; then
>> > tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
>> > rm -rf "${logs[@]}"
>> > fi
>> >
>> > # stop any processes running inside the chroot
>> > failed=false
>> > mock_confs=("$WORKSPACE"/*/mocker*)
>> > # Clean current jobs mockroot if any
>> > for mock_conf_file in "${mock_confs[@]}"; do
>> > [[ "$mock_conf_file" ]] || continue
>> > echo "Cleaning up mock $mock_conf"
>> > mock_root="${mock_conf_file##*/}"
>> > mock_root="${mock_root%.*}"
>> > my_mock="/usr/bin/mock"
>> > my_mock+=" --configdir=${mock_conf_file%/*}"
>> > my_mock+=" --root=${mock_root}"
>> > my_mock+=" --resultdir=$WORKSPACE"
>> >
>> > #TODO: investigate why mock --clean fails to umount certain dirs
>> > sometimes,
>> > #so we can use it instead of manually doing all this.
>> > echo "Killing all mock orphan processes, if any."
>> > $my_mock \
>> > --orphanskill \
>> > || {
>> > echo "ERROR: Failed to kill orphans on $chroot."
>> > failed=true
>> > }
>> >
>> > mock_root="$(\
>> > grep \
>> > -Po "(?<=config_opts\['root'\] = ')[^']*" \
>> > "$mock_conf_file" \
>> > )" || :
>> > [[ "$mock_root" ]] || continue
>> > mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
>> > if [[ "$mounts" ]]; then
>> > echo "Found mounted dirs inside the chroot $chroot. Trying to
>> > umount."
>> > fi
>> > for mount in "${mounts[@]}"; do
>> > sudo umount --lazy "$mount" \
>> > || {
>> > echo "ERROR: Failed to umount $mount."
>> > failed=true
>> > }
>> > done
>> > done
>> >
>> > # Clean any leftover chroot from other jobs
>> > for mock_root in /var/lib/mock/*; do
>> > this_chroot_failed=false
>> > mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
>> > if [[ "$mounts" ]]; then
>> > echo "Found mounted dirs inside the chroot $mock_root." \
>> > "Trying to umount."
>> > fi
>> > for mount in "${mounts[@]}"; do
>> > sudo umount --lazy "$mount" \
>> > || {
>> > echo "ERROR: Failed to umount $mount."
>> > failed=true
>> > this_chroot_failed=true
>> > }
>> > done
>> > if ! $this_chroot_failed; then
>> > sudo rm -rf "$mock_root"
>> > fi
>> > done
>> >
>> > if $failed; then
>> > echo "Aborting."
>> > exit 1
>> > fi
>> >
>> > # remove mock system cache, we will setup proxies to do the caching and
>> > this
>> > # takes lots of space between runs
>> > shopt -u nullglob
>> > sudo rm -Rf /var/cache/mock/*
>> >
>> > # restore the permissions in the working dir, as sometimes it leaves
>> > files
>> > # owned by root and then the 'cleanup workspace' from jenkins job fails
>> > to
>> > # clean and breaks the jobs
>> > sudo chown -R "$USER" "$WORKSPACE"
>> >
>> > [ovirt_3.6_system-tests] $ /bin/bash -xe
>> > /tmp/hudson1974597178250713835.sh
>> > + echo shell-scripts/mock_cleanup.sh
>> > shell-scripts/mock_cleanup.sh
>> > + shopt -s nullglob
>> > + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
>> > + cat
>> > _______________________________________________________________________
>> > #######################################################################
>> > # #
>> > # CLEANUP #
>> > # #
>> > #######################################################################
>> > + logs=(./*log ./*/logs)
>> > + [[ -n ./ovirt-system-tests/logs ]]
>> > + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs
>> > ./ovirt-system-tests/logs/
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.basic_suite_3.6.sh/basic_suite_3.6.sh.log
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/
>> >
>> > ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log
>> > + rm -rf ./ovirt-system-tests/logs
>> > + failed=false
>> > + mock_confs=("$WORKSPACE"/*/mocker*)
>> > + for mock_conf_file in '"${mock_confs[@]}"'
>> > + [[ -n
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg>
>> > ]]
>> > + echo 'Cleaning up mock '
>> > Cleaning up mock
>> > + mock_root=mocker-fedora-23-x86_64.fc23.cfg
>> > + mock_root=mocker-fedora-23-x86_64.fc23
>> > + my_mock=/usr/bin/mock
>> > + my_mock+='
>> > --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests'>
>> > + my_mock+=' --root=mocker-fedora-23-x86_64.fc23'
>> > + my_mock+='
>> > --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/'>
>> > + echo 'Killing all mock orphan processes, if any.'
>> > Killing all mock orphan processes, if any.
>> > + /usr/bin/mock
>> > --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests>
>> > --root=mocker-fedora-23-x86_64.fc23
>> > --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
>> > --orphanskill
>> > WARNING: Could not find required logging config file:
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/logging.ini.>
>> > Using default...
>> > INFO: mock.py version 1.2.17 starting (python version = 3.4.3)...
>> > Start: init plugins
>> > INFO: selinux enabled
>> > Finish: init plugins
>> > Start: run
>> > Finish: run
>> > ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*'
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg>
>> > + mock_root=fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f
>> > + [[ -n fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f ]]
>> > + mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
>> > ++ mount
>> > ++ grep fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f
>> > ++ awk '{print $3}'
>> > + :
>> > + [[ -n '' ]]
>> > + false
>> > + shopt -u nullglob
>> > + sudo rm -Rf
>> > /var/cache/mock/fedora-23-x86_64-ecdfd9c866cbf1a2d41342199fdb011f
>> > + sudo chown -R jenkins
>> > <http://jenkins.ovirt.org/job/ovirt_3.6_system-tests/ws/>
>> > POST BUILD TASK : SUCCESS
>> > END OF POST BUILD TASK : 1
>> > Recording test results
>> > Archiving artifacts
>> > _______________________________________________
>> > Infra mailing list
>> > Infra@ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/infra
>> _______________________________________________
>> Infra mailing list
>> Infra@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>
>>
>