launchpad-dev team mailing list archive

[Stuart Bishop <stuart.bishop@xxxxxxxxxxxxx>]

On Tue, Feb 23, 2010 at 9:51 PM, Tom Haddon <tom.haddon@xxxxxxxxxxxxx> wrote:

> How does the replication_lag check function work in this case? I know
> that for web app servers, they revert to the master if the lag gets too
> long, but would the same automatically happen for scripts, or only if we
> set that up explicitly? I'm just a little concerned that developers may

This is used by the application servers database policy to determine
the default store. It is used by DBLoopTuner scripts to block when lag
is too high. We don't override scripts, putting them into a 'master
only' mode if load is too high. We could if we wanted too, but I
suspect it would cause more problems than it would solve. For scripts,
I think the correct behavior would be to block until all the data you
might need has replicated rather than switch to using the master.

> see "it may be lagged a few hours" under some conditions, and think
> "this isn't worth me chancing it - I don't need data that's absolutely
> up to the second up to date, but if there's a chance it's a few hours
> old, it's not worth risking it - I'll just always connect to the
> master".

They shouldn't. When you are rebuilding the karma caches it doesn't
really matter if the database lag is high. Also, well behaved scripts
such as the ones that use DBLoopTuner block when lag gets too high
automatically. There is nothing to stop similar behavior in other
scripts if necessary, or using more complex database policies that
encode arbitrary rules like 'gimme a slave unless it is lagged more
than 5 minutes', or a check at the start of a script to 'block until
all data committed before I was spawned has replicated'. These
scenarios are all pretty trivial to implement (the helper functions
already exist) and I can assist, but for most cases I suspect it is
just better to go with the flow and ignore the issue rather than over
engineer solutions to something that I doubt is actually a problem in
most cases.

-- 
Stuart Bishop <stuart@xxxxxxxxxxxxxxxx>
http://www.stuartbishop.net/