Marcin Sobczyk <msobczyk@redhat.com> writes:
On 11/4/20 11:29 AM, Yedidyah Bar David wrote:
Perhaps what you want, some day, is for the individual tests to have make-style dependencies? So you'll issue just a single test, and OST will only run the bare minimum for running it. Yeah, I had the same idea. It's not easy to implement it though. 'pytest' has a "tests are independent" design, so we would need to build something on top of that (or try to invent our own test framework, which is a very bad idea). But even with a dependency-resolving solution, there are tests that set something up just to bring it down in a moment (by design), so we'd probably need some kind of "provides" and "tears down" markers. Then you have the fact that some things take a lot of time and we do other stuff in between, while waiting - dependency resolving could force things to happen linearly and the run times could skyrocket... It's a complex subject that requires a serious think-through.
Actually I was once thinking about introducing test dependencies in order to run independent tests in parallel and to speed up OST runs this way. The idea was that OST just waits on something at many places and it could run other tests in the meantime (we do some test interleaving in extreme cases but it's suboptimal and difficult to maintain). When arranging some things manually, I could achieve a significant speedup. But the problem is, of course, how to make an automated dependency management and handle all the possible situations and corner cases. It would be quite a lot of work, I think.