| Thread Previous • Date Previous • Date Next • Thread Next |
The question is whether packages we release should include their test dependencies, even if we do not believe them to be necessary for the packages' operation. For instance, should setup.py for lazr.authentication specify the testing tool wsgi_intercept as an installation requirement, even if it is only a testing tool?
Gavin felt we shouldn't, and I felt we should. Sounds like it's time for a discussion. :-) The following is my argument, ripped from the MP.
There's a tension between wanting to reduce dependencies for your users, and wanting to be confident that your tests actually reflect the performance of your code. The second consideration is sometimes called "test what you fly, fly what you test," and is associated in the States with NASA, our space agency. I have seen well-tested code that directly or indirectly depended on a package that was expected to only be a test dependency, but was actually a run-time dependency. Because the automated tests relied on the test dependencies to run, the only way to catch these kinds of packaging errors was to do manual QA that effectively duplicates the role of your automated tests. I don't think that's a good way to release packages: I want to rely on my automated tests.
Therefore, I think that the "test what you fly, fly what you test" axiom should be followed as a rule, unless there are really compelling reasons not to. A few additional dependencies are generally worth it to me for the additional confidence in my releases.
I do think that there is a subjective point at which the benefit outweighs the cost. I probably start to feel that I can break the rule for library packages when the test dependencies are hard to build or have a count of more than five or six. On the other hand, I pretty much never want to break the rule for deployments of production systems. In other words, if lazr.authentication started getting really bloated because of test dependencies, I would be tempted to break the rule for its release, and separate out the test dependencies; contrariwise, if Launchpad got a lot of test dependencies, I would not be tempted at all to break the rule for its deployment.
Perhaps this is worth a reviewers' meeting discussion, or maybe we can reach consensus on the list.
Gary
| Thread Previous • Date Previous • Date Next • Thread Next |
