[JIRA] (OVIRT-943) Filter out 'latest.under_testing' from repoman '--latest' calculation in 'deploy-to-experiminetal' flow

Filter out 'latest.under_testing' from repoman '--latest' calculation in 'deploy-to-experiminetal' flow ------------------------------------------------------------------------------------------------------- Key: OVIRT-943 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-943 Project: oVirt - virtualization made easy Issue Type: Bug Components: repoman Reporter: Barak Korren Assignee: infra It seems that the repoman '--latest' switch, as it is used in the 'deploy-to-experimatal' job, is not (Or cannot be) configured to ignore the 'latest.under_testing' directory. This can lead to the deploy job failing when other jobs manipulate the 'latest.under_testing' directory. An example for this kind of failure: http://jenkins.ovirt.org/job/deploy-to-ovirt_experimental_master/8392/con... Relevant error messages: {code} 2016-12-18 10:02:09,021::INFO::repoman.common.stores.RPM.add_rpm:248::Adding package /srv/resources/repos/ovirt/experimental/master/latest.under_testing/rpm/el7/SRPMS/vdsm-jsonrpc-java-1.3.5-1.20161214084205.gitfbe3dec.el7.centos.src.rpm to repo Non persistent RPMStore 2016-12-18 10:02:09,022::INFO::repoman.common.stores.RPM.add_rpm:248::Adding package /srv/resources/repos/ovirt/experimental/master/latest.under_testing/rpm/el7/SRPMS/unboundid-ldapsdk-3.2.0-1.el7.src.rpm to repo Non persistent RPMStore Traceback (most recent call last): File "/usr/bin/repoman", line 10, in <module> sys.exit(main()) File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 455, in main exit_code = do_add(args, config, repo) File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 360, in do_add latest_repo.add_source('%s:latest' % root_dir) File "/usr/lib/python2.7/site-packages/repoman/common/repo.py", line 175, in add_source artifact_paths = self.parser.parse(artifact_source) File "/usr/lib/python2.7/site-packages/repoman/common/parser.py", line 96, in parse art_list, File "/usr/lib/python2.7/site-packages/repoman/common/filters/latest.py", line 37, in filter store.add_artifact(artifact) File "/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line 192, in add_artifact self.add_rpm(pkg, **args) File "/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line 213, in add_rpm verify_ssl=self.config.getboolean('verify_ssl'), File "/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/RPM.py", line 114, in __init__ with open(path) as fdno: IOError: [Errno 2] No such file or directory: '/srv/resources/repos/ovirt/experimental/master/latest.under_testing/rpm/el7/SRPMS/vdsm-4.18.999-1188.gitbf3f9e9.el7.centos.src.rpm' 2016-12-18 10:02:09,810::INFO::repoman.common.repo.cleanup:35::Cleaning up temporary dir /srv/resources/repos/ovirt/experimental/.lago_tmp/tmpx7iqIC/tmpQ_3ssv/tmp7HZmrN/tmplYE1KR 2016-12-18 10:02:09,813::INFO::repoman.common.repo.cleanup:35::Cleaning up temporary dir /srv/resources/repos/ovirt/experimental/.lago_tmp/tmpx7iqIC/tmpQ_3ssv/tmp7HZmrN 2016-12-18 10:02:09,814::INFO::repoman.common.repo.cleanup:35::Cleaning up temporary dir /srv/resources/repos/ovirt/experimental/.lago_tmp/tmpx7iqIC/tmpQ_3ssv 2016-12-18 10:02:09,814::INFO::repoman.common.repo.cleanup:35::Cleaning up temporary dir /srv/resources/repos/ovirt/experimental/.lago_tmp/tmpx7iqIC Build step 'Execute shell' marked build as failure {code} This issue can also be mitigated with global exclusive locking as described in OVIRT-899