| Thread Previous • Date Previous • Date Next • Thread Next |
On 2011-07-15 22:56, Robert Collins wrote:
We'll learn how to write incremental patches. Meeting the 15s goal on a cold database will be hard though, since we can't easily reproduce results or try out tweaks. It might help if we could break a slave out of replication to time a DB patch against a realistic cache.I should clarify then - what I mean by cold is more of a 'no cheating' - no running it twice on qastaging within a transaction and then taking the best time. Run it once, take the time. This is to get as close as we can approximate to actual-behaviour on prod (though we should expect prod to be consistently faster :)).
That is exactly what I meant. Not knowing either cache's state doesn't give us a better basis for comparison. It just introduces randomness. Which will hinder the learning process and penalize arbitrary patches.
If you have convincing statistics that say our database patches are orders of magnitude more consistent between staging and production than run-of-the-mill queries are, then a cold staging cache is fine. But in that case a warm cache will be just as fine.
Until we have convincing numbers, I would prefer to time against a predictable, warm staging cache and track how the production timings compare: took X seconds on staging, estimated at Y seconds for production, actually took Z seconds; calibrate estimation function if Y and Z straddle or approximate the budget.
Of course you're concerned about letting slow patches through. But won't those overrun staging's smaller caches anyway? A warm cache can't fix that, but it does eliminate flukes. "Cold" timings are sensible to contention with qastaging, pressure from incoming database dumps, unrelated filesystem activity, other engineers' experiments, app server load spikes, locks, the post-restore cache void, what have you. Those things don't make timing any more reliable, just more conservative — and that means more downtime.
Jeroen
| Thread Previous • Date Previous • Date Next • Thread Next |
