launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Tue, Jun 14, 2011 at 9:11 PM, Jeroen Vermeulen <jtv@xxxxxxxxxxxxx> wrote:
> On 2011-06-14 08:50, Robert Collins wrote:
>
>> In terms of development process we would land schema changes that are
>> compatible with the python code on devel - just the schema change, no
>> python code changes at all. Then we'd do nodowntime deploys as normal
>> up to and past that revision; when a good time to do the downtime
>> arrives (e.g. we might set a fixed time of the day or week) we'd do
>> the downtime deploy described above. After thats live developers would
>> then land code that uses the new schema / populates new columns etc.
>
> This is something I suggested a few years back in my "weekly devel rollouts
> with safe DB patches" proposal.  Needless to say, apart from having no clue
> about deployment practicalities I'm all for it.  :-)
>
> When I proposed weekly devel rollouts the response was that a rollout took
> too long to do on a weekly basis, but this is no longer true.  By my
> calculation back then, even for a modestly complex schema/code change,
> rolling out a slow succession of safe increments would look like more work
> but would beat a monthly "full" rollout to production rollout.
>
> And that was based on weekly rollouts, which pale in comparison to what we
> can do now!  IMHO the extra work of producing safe increments is in most
> cases preferable over the waste, planning pressure, and frustration inherent
> in synchronizing with the monthly rollout.
>
> Your notion of "a devel branch must pass tests and have either only code
> changes or only schema changes" sounds like a convenient and reasonable
> rule.  It ensures that the branch does not rely on synchronicity between
> schema and python changes.
>
> If I understand the design correctly we'll also need to lockstep deployment
> whenever we go from deploying database patches to deploying python branches
> or vice versa: an app server can't start accessing a new column before all
> databases have it in their schema, and no database can drop a column before
> all app servers have stopped accessing it.  We can't reorder patches freely,
> e.g. do all the database patches in one go and then all the python changes,
> but there's ways of dealing with that.

I think we'd want to constraint it like so:
 - never land a schema change that requires a python code change.
 - never land python code dependent on a non-deployed schema change.
 - with the exception being when something is just genuinely Too Hard
to do with migrations and we decide we have to do a hybrid deploy with
everything getting bounced & deployed to.

That avoids deploys ever getting blocked on schema deployment...
except when we've decided to accept the cost of a hybrid downtime.

OTOH it means we have to really focus on getting the land->schema
deploy reliable and snappy, or folk will still get backed up.

-Rob