<p dir="ltr"></p>
<p dir="ltr">Il 15/Ago/2016 10:29 AM, "Eyal Edri" <<a href="mailto:eedri@redhat.com">eedri@redhat.com</a>> ha scritto:<br>
><br>
> can we disable the add secondary storage domain test if this is what is failing the test?</p>
<p dir="ltr">Isn't the secondary storage domain the one needed to create other vms? First storage should be the hosted engine domain imported after setup. If second storage domain fails we have a blocker bug here.<br><br></p>
<p dir="ltr">> at least until the new sanlock will be released.<br>
><br>
> GA date is 13/09 and I'm guessing it will take time to reach centos also, so we still have at least one month of this error possibly masking real regressions.<br>
><br>
> On Mon, Aug 15, 2016 at 11:25 AM, <<a href="mailto:jenkins@jenkins.phx.ovirt.org">jenkins@jenkins.phx.ovirt.org</a>> wrote:<br>
>><br>
>> See <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/changes">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/changes</a>><br>
>><br>
>> Changes:<br>
>><br>
>> [Gal Ben Haim] Removed the hard coded path of the template repo. The template repo file<br>
>><br>
>> ------------------------------------------<br>
>> [...truncated 1273 lines...]<br>
>> ## rc = 1<br>
>> ##########################################################<br>
>> ##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv<br>
>> ##! Last 20 log enties: logs/<a href="http://mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log">mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log</a><br>
>> ##!<br>
>> + true<br>
>> + env_cleanup<br>
>> + echo '#########################'<br>
>> #########################<br>
>> + local res=0<br>
>> + local uuid<br>
>> + echo '======== Cleaning up'<br>
>> ======== Cleaning up<br>
>> + [[ -e <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/deployment-he_basic_suite_4.0">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/deployment-he_basic_suite_4.0</a>> ]]<br>
>> + echo '----------- Cleaning with lago'<br>
>> ----------- Cleaning with lago<br>
>> + lago --workdir <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/deployment-he_basic_suite_4.0">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/deployment-he_basic_suite_4.0</a>> destroy --yes --all-prefixes<br>
>> + echo '----------- Cleaning with lago done'<br>
>> ----------- Cleaning with lago done<br>
>> + [[ 0 != \0 ]]<br>
>> + echo '======== Cleanup done'<br>
>> ======== Cleanup done<br>
>> + exit 0<br>
>> Took 2834 seconds<br>
>> ===================================<br>
>> ##!<br>
>> ##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
>> ##!########################################################<br>
>> ##########################################################<br>
>> Build step 'Execute shell' marked build as failure<br>
>> Performing Post build task...<br>
>> Match found for :.* : True<br>
>> Logical operation result is TRUE<br>
>> Running script : #!/bin/bash -xe<br>
>> echo 'shell_scripts/<a href="http://system_tests.collect_logs.sh">system_tests.collect_logs.sh</a>'<br>
>><br>
>> #<br>
>> # Required jjb vars:<br>
>> # version<br>
>> #<br>
>> VERSION=4.0<br>
>> SUITE_TYPE=<br>
>><br>
>> WORKSPACE="$PWD"<br>
>> OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"<br>
>> TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"<br>
>><br>
>> rm -rf "$WORKSPACE/exported-artifacts"<br>
>> mkdir -p "$WORKSPACE/exported-artifacts"<br>
>><br>
>> if [[ -d "$TESTS_LOGS" ]]; then<br>
>> mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"<br>
>> fi<br>
>><br>
>> [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson2053675743404291966.sh<br>
>> + echo shell_scripts/<a href="http://system_tests.collect_logs.sh">system_tests.collect_logs.sh</a><br>
>> shell_scripts/<a href="http://system_tests.collect_logs.sh">system_tests.collect_logs.sh</a><br>
>> + VERSION=4.0<br>
>> + SUITE_TYPE=<br>
>> + WORKSPACE=<<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/</a>><br>
>> + OVIRT_SUITE=4.0<br>
>> + TESTS_LOGS=<<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts</a>><br>
>> + rm -rf <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts</a>><br>
>> + mkdir -p <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts</a>><br>
>> + [[ -d <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts</a>> ]]<br>
>> + mv <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts/lago_logs">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts/lago_logs</a>> <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts/test_logs">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/exported-artifacts/test_logs</a>> <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts/">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/144/artifact/exported-artifacts/</a>><br>
>> POST BUILD TASK : SUCCESS<br>
>> END OF POST BUILD TASK : 0<br>
>> Match found for :.* : True<br>
>> Logical operation result is TRUE<br>
>> Running script : #!/bin/bash -xe<br>
>> echo "shell-scripts/mock_cleanup.sh"<br>
>><br>
>> shopt -s nullglob<br>
>><br>
>><br>
>> WORKSPACE="$PWD"<br>
>><br>
>> # Make clear this is the cleanup, helps reading the jenkins logs<br>
>> cat <<EOC<br>
>> _______________________________________________________________________<br>
>> #######################################################################<br>
>> # #<br>
>> # CLEANUP #<br>
>> # #<br>
>> #######################################################################<br>
>> EOC<br>
>><br>
>><br>
>> # Archive the logs, we want them anyway<br>
>> logs=(<br>
>> ./*log<br>
>> ./*/logs<br>
>> )<br>
>> if [[ "$logs" ]]; then<br>
>> tar cvzf exported-artifacts/logs.tgz "${logs[@]}"<br>
>> rm -rf "${logs[@]}"<br>
>> fi<br>
>><br>
>> # stop any processes running inside the chroot<br>
>> failed=false<br>
>> mock_confs=("$WORKSPACE"/*/mocker*)<br>
>> # Clean current jobs mockroot if any<br>
>> for mock_conf_file in "${mock_confs[@]}"; do<br>
>> [[ "$mock_conf_file" ]] || continue<br>
>> echo "Cleaning up mock $mock_conf"<br>
>> mock_root="${mock_conf_file##*/}"<br>
>> mock_root="${mock_root%.*}"<br>
>> my_mock="/usr/bin/mock"<br>
>> my_mock+=" --configdir=${mock_conf_file%/*}"<br>
>> my_mock+=" --root=${mock_root}"<br>
>> my_mock+=" --resultdir=$WORKSPACE"<br>
>><br>
>> #TODO: investigate why mock --clean fails to umount certain dirs sometimes,<br>
>> #so we can use it instead of manually doing all this.<br>
>> echo "Killing all mock orphan processes, if any."<br>
>> $my_mock \<br>
>> --orphanskill \<br>
>> || {<br>
>> echo "ERROR: Failed to kill orphans on $chroot."<br>
>> failed=true<br>
>> }<br>
>><br>
>> mock_root="$(\<br>
>> grep \<br>
>> -Po "(?<=config_opts\['root'\] = ')[^']*" \<br>
>> "$mock_conf_file" \<br>
>> )" || :<br>
>> [[ "$mock_root" ]] || continue<br>
>> mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :<br>
>> if [[ "$mounts" ]]; then<br>
>> echo "Found mounted dirs inside the chroot $chroot. Trying to umount."<br>
>> fi<br>
>> for mount in "${mounts[@]}"; do<br>
>> sudo umount --lazy "$mount" \<br>
>> || {<br>
>> echo "ERROR: Failed to umount $mount."<br>
>> failed=true<br>
>> }<br>
>> done<br>
>> done<br>
>><br>
>> # Clean any leftover chroot from other jobs<br>
>> for mock_root in /var/lib/mock/*; do<br>
>> this_chroot_failed=false<br>
>> mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :<br>
>> if [[ "$mounts" ]]; then<br>
>> echo "Found mounted dirs inside the chroot $mock_root." \<br>
>> "Trying to umount."<br>
>> fi<br>
>> for mount in "${mounts[@]}"; do<br>
>> sudo umount --lazy "$mount" \<br>
>> || {<br>
>> echo "ERROR: Failed to umount $mount."<br>
>> failed=true<br>
>> this_chroot_failed=true<br>
>> }<br>
>> done<br>
>> if ! $this_chroot_failed; then<br>
>> sudo rm -rf "$mock_root"<br>
>> fi<br>
>> done<br>
>><br>
>> if $failed; then<br>
>> echo "Aborting."<br>
>> exit 1<br>
>> fi<br>
>><br>
>> # remove mock system cache, we will setup proxies to do the caching and this<br>
>> # takes lots of space between runs<br>
>> shopt -u nullglob<br>
>> sudo rm -Rf /var/cache/mock/*<br>
>><br>
>> # restore the permissions in the working dir, as sometimes it leaves files<br>
>> # owned by root and then the 'cleanup workspace' from jenkins job fails to<br>
>> # clean and breaks the jobs<br>
>> sudo chown -R "$USER" "$WORKSPACE"<br>
>><br>
>> [ovirt_4.0_he-system-tests] $ /bin/bash -xe /tmp/hudson895710842739065954.sh<br>
>> + echo shell-scripts/mock_cleanup.sh<br>
>> shell-scripts/mock_cleanup.sh<br>
>> + shopt -s nullglob<br>
>> + WORKSPACE=<<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/</a>><br>
>> + cat<br>
>> _______________________________________________________________________<br>
>> #######################################################################<br>
>> # #<br>
>> # CLEANUP #<br>
>> # #<br>
>> #######################################################################<br>
>> + logs=(./*log ./*/logs)<br>
>> + [[ -n ./ovirt-system-tests/logs ]]<br>
>> + tar cvzf exported-artifacts/logs.tgz ./ovirt-system-tests/logs<br>
>> ./ovirt-system-tests/logs/<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/stdout_stderr.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/state.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/build.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.init/root.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/stdout_stderr.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/state.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/build.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.install_packages/root.log<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/<br>
>> ./ovirt-system-tests/logs/mocker-fedora-23-x86_64.fc23.clean_rpmdb/stdout_stderr.log<br>
>> ./ovirt-system-tests/logs/<a href="http://mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/">mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/</a><br>
>> ./ovirt-system-tests/logs/<a href="http://mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log">mocker-fedora-23-x86_64.fc23.he_basic_suite_4.0.sh/he_basic_suite_4.0.sh.log</a><br>
>> + rm -rf ./ovirt-system-tests/logs<br>
>> + failed=false<br>
>> + mock_confs=("$WORKSPACE"/*/mocker*)<br>
>> + for mock_conf_file in '"${mock_confs[@]}"'<br>
>> + [[ -n <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg</a>> ]]<br>
>> + echo 'Cleaning up mock '<br>
>> Cleaning up mock<br>
>> + mock_root=mocker-fedora-23-x86_64.fc23.cfg<br>
>> + mock_root=mocker-fedora-23-x86_64.fc23<br>
>> + my_mock=/usr/bin/mock<br>
>> + my_mock+=' --configdir=<<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests</a>'><br>
>> + my_mock+=' --root=mocker-fedora-23-x86_64.fc23'<br>
>> + my_mock+=' --resultdir=<<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/</a>'><br>
>> + echo 'Killing all mock orphan processes, if any.'<br>
>> Killing all mock orphan processes, if any.<br>
>> + /usr/bin/mock --configdir=<<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests</a>> --root=mocker-fedora-23-x86_64.fc23 --resultdir=<<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/</a>> --orphanskill<br>
>> WARNING: Could not find required logging config file: <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/logging.ini">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/logging.ini</a>.> Using default...<br>
>> INFO: mock.py version 1.2.18 starting (python version = 3.5.1)...<br>
>> Start: init plugins<br>
>> INFO: selinux enabled<br>
>> Finish: init plugins<br>
>> Start: run<br>
>> Finish: run<br>
>> ++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/ovirt-system-tests/mocker-fedora-23-x86_64.fc23.cfg</a>><br>
>> + mock_root=fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad<br>
>> + [[ -n fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad ]]<br>
>> + mounts=($(mount | awk '{print $3}' | grep "$mock_root"))<br>
>> ++ mount<br>
>> ++ awk '{print $3}'<br>
>> ++ grep fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad<br>
>> + :<br>
>> + [[ -n '' ]]<br>
>> + false<br>
>> + shopt -u nullglob<br>
>> + sudo rm -Rf /var/cache/mock/fedora-23-x86_64-0c362156a2fa4a935ea8b988eb73b2ad<br>
>> + sudo chown -R jenkins <<a href="http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/">http://jenkins.ovirt.org/job/ovirt_4.0_he-system-tests/ws/</a>><br>
>> POST BUILD TASK : SUCCESS<br>
>> END OF POST BUILD TASK : 1<br>
>> Recording test results<br>
>> ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?<br>
>> Archiving artifacts<br>
>> _______________________________________________<br>
>> Infra mailing list<br>
>> <a href="mailto:Infra@ovirt.org">Infra@ovirt.org</a><br>
>> <a href="http://lists.ovirt.org/mailman/listinfo/infra">http://lists.ovirt.org/mailman/listinfo/infra</a><br>
>><br>
>><br>
><br>
><br>
><br>
> -- <br>
> Eyal Edri<br>
> Associate Manager<br>
> RHV DevOps<br>
> EMEA ENG Virtualization R&D<br>
> Red Hat Israel<br>
><br>
> phone: +972-9-7692018<br>
> irc: eedri (on #tlv #rhev-dev #rhev-integ)<br></p>