is 15 minute smoke tests and 40-ish minute full regression runs, which
in my case is within my pain threshold. In terms of actions, we've paid
the majority of the time price for testing scenarios. The additional
tests I'm adding (such as the SSH code) don't add significant time to
tests, but keep adding value. The same can be said of checking for
events that should be generated on server actions, and other side
effects that take minimal time to run. As we start to move in that
direction, test classes will become more streamlined/optimized. The test
class I added with the initial SSH tests is a good example of this, and
I'd be more than glad to share some other examples of how we can do more
with less. Eventually these types of optimizations can grow into our
coding standards.
What's everyone else's feelings on smoke test pain point times? I
think
getting a sense of that will make any further decisions a bit more
clear.
Regardless of what length of time we say, I still think we need to get
parallel execution working properly (with the existing --processes
option in nosetest's multiprocessing plugin).
Agreed. I think like we talked about at the conference, I think this
discussion and the overall discussion on optimization would probably be
better handled in a Skype meeting where we can all discuss this in real
time. I'm free every day this week save Friday.
