Il 15/Ago/2016 10:29 AM, "Eyal Edri" <eedri@redhat.com> ha scritto:
>
> can we disable the add secondary storage domain test if this is what is failing the test?

Isn't the secondary storage domain the one needed to create other vms? First storage should be the hosted engine domain imported after setup. If second storage domain fails we have a blocker bug here.

> at least until the new sanlock will be released.
>
> GA date is 13/09 and I'm guessing it will take time to reach centos also, so we still have at least one month of this error possibly masking real regressions.
>
> On Mon, Aug 15, 2016 at 11:25 AM, <jenkins@jenkins.phx.ovirt.org> wrote:
>>
>> See <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/changes>
>>
>> Changes:
>>
>> [Gal Ben Haim] Removed the hard coded path of the template repo. The template repo file
>>
>> ------------------------------------------
>> [...truncated 1273 lines...]
>> ##      rc = 1
>> ##########################################################
>> ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
>> ##! Last 20 log enties: logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log
>> ##!
>> + true
>> + env_cleanup
>> + echo '#########################'
>> #########################
>> + local res=0
>> + local uuid
>> + echo '======== Cleaning up'
>> ======== Cleaning up
>> + [[ -e <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/deployment-he_basic_suite_4.0> ]]
>> + echo '----------- Cleaning with lago'
>> ----------- Cleaning with lago
>> + lago --workdir <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/deployment-he_basic_suite_4.0> destroy --yes --all-prefixes
>> + echo '----------- Cleaning with lago done'
>> ----------- Cleaning with lago done
>> + [[ 0 != \0 ]]
>> + echo '======== Cleanup done'
>> ======== Cleanup done
>> + exit 0
>> Took 2834 seconds
>> ===================================
>> ##!
>> ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> ##!########################################################
>> ##########################################################
>> Build step 'Execute shell' marked build as failure
>> Performing Post build task...
>> Match found for :.* : True
>> Logical operation result is TRUE
>> Running script  : #!/bin/bash -xe
>> echo 'shell_scripts/system_tests.collect_logs.sh'
>>
>> #
>> # Required jjb vars:
>> #    version
>> #
>> VERSION=4.0
>> SUITE_TYPE=
>>
>> WORKSPACE="$PWD"
>> OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
>> TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
>>
>> rm -rf "$WORKSPACE/exported-artifacts"
>> mkdir -p "$WORKSPACE/exported-artifacts"
>>
>> if [[ -d "$TESTS_LOGS" ]]; then
>>     mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
>> fi
>>
>> [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson2053675743404291966.sh
>> + echo shell_scripts/system_tests.collect_logs.sh
>> shell_scripts/system_tests.collect_logs.sh
>> + VERSION=4.0
>> + SUITE_TYPE=
>> + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/>
>> + OVIRT_SUITE=4.0
>> + TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts>
>> + rm -rf <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts>
>> + mkdir -p <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts>
>> + [[ -d <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts> ]]
>> + mv <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts/lago_logs> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts/test_logs> <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts/>
>> POST BUILD TASK : SUCCESS
>> END OF POST BUILD TASK : 0
>> Match found for :.* : True
>> Logical operation result is TRUE
>> Running script  : #!/bin/bash -xe
>> echo "shell-scripts/mock_cleanup.sh"
>>
>> shopt -s nullglob
>>
>>
>> WORKSPACE="$PWD"
>>
>> # Make clear this is the cleanup, helps reading the jenkins logs
>> cat <<EOC
>> _______________________________________________________________________
>> #######################################################################
>> #                                                                     #
>> #                               CLEANUP                               #
>> #                                                                     #
>> #######################################################################
>> EOC
>>
>>
>> # Archive the logs, we want them anyway
>> logs=(
>>     ./*log
>>     ./*/logs
>> )
>> if [[ "$logs" ]]; then
>>     tar cvzf exported-artifacts/logs.tgz "${logs[@]}"
>>     rm -rf "${logs[@]}"
>> fi
>>
>> # stop any processes running inside the chroot
>> failed=false
>> mock_confs=("$WORKSPACE"/*/mocker*)
>> # Clean current jobs mockroot if any
>> for mock_conf_file in "${mock_confs[@]}"; do
>>     [[ "$mock_conf_file" ]] || continue
>>     echo "Cleaning up mock $mock_conf"
>>     mock_root="${mock_conf_file##*/}"
>>     mock_root="${mock_root%.*}"
>>     my_mock="/usr/bin/mock"
>>     my_mock+=" --configdir=${mock_conf_file%/*}"
>>     my_mock+=" --root=${mock_root}"
>>     my_mock+=" --resultdir=$WORKSPACE"
>>
>>     #TODO: investigate why mock --clean fails to umount certain dirs sometimes,
>>     #so we can use it instead of manually doing all this.
>>     echo "Killing all mock orphan processes, if any."
>>     $my_mock \
>>         --orphanskill \
>>     || {
>>         echo "ERROR:  Failed to kill orphans on $chroot."
>>         failed=true
>>     }
>>
>>     mock_root="$(\
>>         grep \
>>             -Po "(?<=config_opts\['root'\] = ')[^']*" \
>>             "$mock_conf_file" \
>>     )" || :
>>     [[ "$mock_root" ]] || continue
>>     mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
>>     if [[ "$mounts" ]]; then
>>         echo "Found mounted dirs inside the chroot $chroot. Trying to umount."
>>     fi
>>     for mount in "${mounts[@]}"; do
>>         sudo umount --lazy "$mount" \
>>         || {
>>             echo "ERROR:  Failed to umount $mount."
>>             failed=true
>>         }
>>     done
>> done
>>
>> # Clean any leftover chroot from other jobs
>> for mock_root in /var/lib/mock/*; do
>>     this_chroot_failed=false
>>     mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
>>     if [[ "$mounts" ]]; then
>>         echo "Found mounted dirs inside the chroot $mock_root." \
>>              "Trying to umount."
>>     fi
>>     for mount in "${mounts[@]}"; do
>>         sudo umount --lazy "$mount" \
>>         || {
>>             echo "ERROR:  Failed to umount $mount."
>>             failed=true
>>             this_chroot_failed=true
>>         }
>>     done
>>     if ! $this_chroot_failed; then
>>         sudo rm -rf "$mock_root"
>>     fi
>> done
>>
>> if $failed; then
>>     echo "Aborting."
>>     exit 1
>> fi
>>
>> # remove mock system cache, we will setup proxies to do the caching and this
>> # takes lots of space between runs
>> shopt -u nullglob
>> sudo rm -Rf /var/cache/mock/*
>>
>> # restore the permissions in the working dir, as sometimes it leaves files
>> # owned by root and then the 'cleanup workspace' from jenkins job fails to
>> # clean and breaks the jobs
>> sudo chown -R "$USER" "$WORKSPACE"
>>
>> [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson895710842739065954.sh
>> + echo shell-scripts/mock_cleanup.sh
>> shell-scripts/mock_cleanup.sh
>> + shopt -s nullglob
>> + WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/>
>> + cat
>> _______________________________________________________________________
>> #######################################################################
>> #                                                                     #
>> #                               CLEANUP                               #
>> #                                                                     #
>> #######################################################################
>> + logs=(./*log ./*/logs)
>> + [[ -n ./ovirt-system-tests/logs ]]
>> + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs
>> ./ovirt-system-tests/logs/
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log
>> + rm -rf ./ovirt-system-tests/logs
>> + failed=false
>> + mock_confs=("$WORKSPACE"/*/mocker*)
>> + for mock_conf_file in '"${mock_confs[@]}"'
>> + [[ -n <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg> ]]
>> + echo 'Cleaning up mock '
>> Cleaning up mock
>> + mock_root=mocker-fedora-23-x86_64.fc23.cfg
>> + mock_root=mocker-fedora-23-x86_64.fc23
>> + my_mock=/usr/bin/mock
>> + my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests'>
>> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23'
>> + my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/'>
>> + echo 'Killing all mock orphan processes, if any.'
>> Killing all mock orphan processes, if any.
>> + /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/> --orphanskill
>> WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/logging.ini.> Using default...
>> INFO: mock.py version 1.2.18 starting (python version = 3.5.1)...
>> Start: init plugins
>> INFO: selinux enabled
>> Finish: init plugins
>> Start: run
>> Finish: run
>> ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg>
>> + mock_root=fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad
>> + [[ -n fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]]
>> + mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
>> ++ mount
>> ++ awk '{print $3}'
>> ++ grep fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad
>> + :
>> + [[ -n '' ]]
>> + false
>> + shopt -u nullglob
>> + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad
>> + sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/>
>> POST BUILD TASK : SUCCESS
>> END OF POST BUILD TASK : 1
>> Recording test results
>> ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
>> Archiving artifacts
>> _______________________________________________
>> Infra mailing list
>> Infra@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/infra
>>
>>
>
>
>
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R&D
> Red Hat Israel
>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)