launchpad-dev team mailing list archive

[Stuart Bishop <stuart.bishop@xxxxxxxxxxxxx>]

On Thu, Feb 25, 2010 at 10:28 PM, Bjorn Tillenius <bjorn@xxxxxxxxxxxxx> wrote:
> On Tue, Feb 23, 2010 at 03:43:42PM +0700, stuart.bishop@xxxxxxxxxxxxx wrote:
> > Hi. There were some inquiries recently on how to get scripts using slave databases, in order to reduce load on the master databases. I've taken this opportunity to improve the syntax and write improved documentation. The new documentation is in lib/canonical/launchpad/doc/db-policy.txt and is reproduced below.
> >
> > Storm Stores & Database Policies
> > ================================
> >
> > Launchpad has multiple master and slave databases. Changes to data are
> > made on the master and replicated asynchronously to the slave
> > databases. Slave databases will usually lag a few seconds behind their
> > master. Under high load they may lag a few minutes behind, during
> > maintenance they may lag a few hours behind and if things explode
> > while admins are on holiday they may lag days behind.
> >
> > If know your code needs to change data, or must have the latest posible
> > information, you retrieve objects from the master databases that stores
> > the data for your database class.
> >
> >    >>> from canonical.launchpad.interfaces.lpstorm import IMasterStore
> >    >>> from lp.registry.model.person import Person
> >    >>> import transaction
> >
> >    >>> writable_janitor = IMasterStore(Person).find(
> >    ...     Person, Person.name == 'janitor').one()
>
> All these examples deal with the Person class directly. A lot of scripts
> use getUtility(IFooSet) to get security proxied objects. Can you give an
> example of how that works?

It works identically:

> > > with SlaveDatabasePolicy():
...     ro_person = getUtility(IPersonSet).getByName('foo')

Unless code explicitly pulls things from a particular Store or installs its own database policy, it is pulled from the 'default' Store. You control what this means by installing a database policy.

This is how all our legacy code kept working when we went replicated - the appserver publisher just installed a database policy at the start of the request to set the 'default' store to the slave when sane, and database load got distributed to the slaves. No further code changes necessary to all the old code using the SQLObject compatibility layer.

--
Stuart Bishop <stuart@xxxxxxxxxxxxxxxx>
http://www.stuartbishop.net/

Attachment: signature.asc
Description: OpenPGP digital signature