[OVIRT CI] Tests succeeded, build failed

Hi all, Looking in another unrelated vdsm test failure, I found this: 18:36:52 ########################################################## 18:36:52 ## Tue Dec 13 18:36:52 UTC 2016 Finished env: fc24:fedora-24-x86_64 18:36:52 ## took 417 seconds 18:36:52 ## rc = 0 18:36:52 ########################################################## 18:36:52 ## FINISHED SUCCESSFULLY 18:36:52 ########################################################## 18:36:52 ########################################################## The test succeeded, but during cleanup: 18:36:55 + sudo umount --lazy /var/lib/mock/epel-7-x86_64-95d9ead9d725499a15a9021ba2fe9831-54661/root/proc/filesystems 18:36:55 umount: /var/lib/mock/epel-7-x86_64-95d9ead9d725499a15a9021ba2fe9831-54661/root/proc/filesystems: mountpoint not found 18:36:55 + echo 'ERROR: Failed to umount /var/lib/mock/epel-7-x86_64-95d9ead9d725499a15a9021ba2fe9831-54661/root/proc/filesystems.' 18:36:55 ERROR: Failed to umount /var/lib/mock/epel-7-x86_64-95d9ead9d725499a15a9021ba2fe9831-54661/root/proc/filesystems. 18:36:55 + failed=true 18:36:55 + this_chroot_failed=true 18:36:55 + true 18:36:55 + find /var/cache/mock/ -mindepth 1 -maxdepth 1 -type d -mtime +2 -print0 18:36:55 + xargs -0 -tr sudo rm -rf 18:36:55 ++ virsh list --all --uuid 18:36:55 + true 18:36:55 + echo 'Cleanup script failed, propegating failure to job' 18:36:55 Cleanup script failed, propegating failure to job 18:36:55 + exit 1 18:36:55 POST BUILD TASK : FAILURE 18:36:55 END OF POST BUILD TASK : 0 18:36:55 ESCALATE FAILED POST BUILD TASK TO JOB STATUS 18:36:55 Build step 'Post build task' changed build result to FAILURE 18:36:55 Archiving artifacts 18:37:00 Build step 'Groovy Postbuild' marked build as failure 18:37:00 Started calculate disk usage of build 18:37:00 Finished Calculation of disk usage of build in 0 seconds 18:37:00 Started calculate disk usage of workspace 18:37:01 Finished Calculation of disk usage of workspace in 0 seconds 18:37:01 Finished: FAILURE The build failed because cleanup script failed. I discussed this lot of time with David Caro, trying to convince him that there are 3 possible results for a build: 1. Tests run and passed 2. Tests run and failed 3. System could not run the tests or had another error I know that this adds 33% more work to the CI team, having to handle 3 results instead of two, but we really need this distinction. Barak, do you think we can change the script so setup and cleanup failures are not treated as build failures but build errors? In travis such failure seem to start another build automatically, making developers life much nicer. I kept this build forever so people can inspect it: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/6291/ Nir