launchpad-dev team mailing list archive

[Stuart Bishop <stuart.bishop@xxxxxxxxxxxxx>]

On Fri, Aug 7, 2009 at 3:50 AM, Christian Robottom
Reis<kiko@xxxxxxxxxxxxx> wrote:
> On Wed, Aug 05, 2009 at 01:40:34PM +0200, Danilo Šegan wrote:
>> > I want us to stick with Storm trunk, or else when we need to extend it
>> > or provide bug fixes we will be forking. We should be doing Storm work
>> > upstream rather than locally and backporting later (which is unlikely
>> > to ever happen). Storm isn't a big enough project to support multiple
>> > lines of development.
>>
>> I'd like that too.  But first of all, before we are using Storm trunk,
>> I'd like us to have some time to stabilize on what we have in 2.2.7, and
>> then after 3.0 is out, move to Storm trunk, and keep working with that
>> (i.e. upgrade to the latest trunk whenever we start a new milestone,
>> thus giving us a month to test changes out).
>>
>> 0.14 upgrade was just too close for me to want another 40 revisions of
>> possible breakages.
>
> If there are indeed 40 other revisions of unrelated changes, then the
> only sane choice for 3.0 is what we have right now or 0.14.1 (if that
> will be basically what we have right now). I realize there are other
> fixes in there, but if they aren't affecting us in production, we can
> worry about them in september.

There are not 40 other revisions. A theoretical 0.14.1 -> 0.15 is not
that big a leap.

I thought now was the perfect time to upgrade because we have two
entire cycles on edge. We are currently running what will be one
revision away from 0.15 on edge (some compile fixes for Windows need
to land). I personally would rather run what everyone else will be
running (Landscape, Online services, external contributers) rather
than forking our own release containing the bug fixes and features we
depend on, and trying to support that ourselves as we get burnt by
bugs that have already been fixed in 0.15. Note that we no longer have
anyone on the team familiar with the guts of Storm.

> What we should be discussing as well is how to avoid having a release of
> Storm break so spectacularly on us; should we have a more active upgrade
> policy like we do for bzr, maybe rolling it out on one or two appservers
> at a time to see what the effect is?

What exactly broke so spectacularly? We updated one of our
dependencies (and pulled out a a heap of workarounds). We tested it,
pushed it out to the testing environments, discovered regressions,
rolled back, patched, tested again, rolled it out to production.

Like all our other dependencies, I'd say stick to best practice of
testing Launchpad against release candidates, keeping up to date so we
don't have years worth of changes to deal with at once, fixing bugs
upstream rather than adding workarounds in Launchpad, and encouraging
regular upstream releases. The worst issue with the recent upgrade is
because the latest official Storm release has know bugs which turned
out to be show stoppers yet no patch pushed out for 6 months - that is
just embarrassing for a Canonical backed project. The changes that
needed to be made to our code to support the new Storm was removal of
all the workarounds, the code that relied on the odd behavior of the
workarounds, and the tests that demonstrated how we are relying on the
workarounds.


-- 
Stuart Bishop <stuart@xxxxxxxxxxxxxxxx>
http://www.stuartbishop.net/