launchpad-dev team mailing list archive

[Guilherme Salgado <salgado@xxxxxxxxxxxxx>]

On 08/02/12 16:57, Robert Collins wrote:
> On Thu, Feb 9, 2012 at 12:49 PM, Guilherme Salgado
> <salgado@xxxxxxxxxxxxx> wrote:
>>
>> I think that's going to be challenging. My gut reaction would be to look for
>>
>>  1. Code that's no longer used and can be removed
>>  2. Duplicated code which could be moved to a common base
>>  3. Code that could be written in a simpler way
>>
>> Now, just searching for this kind of stuff takes some time, and there
>> probably aren't many occurrences of 1 left, so I'd probably focus on 2
>> and 3, which certainly abound, although there's no guarantee that I'll
>> have a net negative until after the refactor/rewrite is done. That means
>> I may end up spending a day on something just to realize it'll have a
>> net negative of a dozen lines or so. That is certainly an improvement to
>> Launchpad, but to me (Linaro, in fact) it's mostly wasted effort as it
>> didn't take us significantly closer to our goal.
> 
> There is a 4: extract or delete code that shouldn't be part of LP
> core. We have a -raft- of 'helpers' (e.g. the thing for running
> subprocesses) which are either:
> a) uninterestingly different to other helpers on pypi
> b) interestingly different but embedded in the LP code base so that
> no-one else can use them
> c) interestingly different but not pulling their weight
> 
> These things can be pulled out to separate reusable projects, or just
> removed entirely and migrated away from.
> 
> and 5: remove code that is used but is a net negative. (e.g. polls)
> 
> In the last week I've seen 2 commits doing (1), one doing (2), and
> there is one for (3) about to land. I estimate about 15% of LP is
> covered by (4), and another 10%-15% by 5.
> 
> Here are some ideas that should (possibly each on their own) easily
> offset your project:
>  - remove the massive duplication between modules in the blueprints
> subsystem; last I checked about 30% of the code in there was verbatim
> copies between different contexts.
>  - move most of utilities/* to the lpdevtools project - thats 6K LOC on its own
>  - extract poppy to a separate project (it no longer needs to talk to
> the DB, so has no need to be part of LP)
>  - remove polls (not a trivial thing politically, but I'm sure mrevell
> is able to clear the way)
>  - move the ec2 land/test logic to lpdevtools
>  - delete the representation cache from lazr.restful (there is a bug
> open for this)
>  - delete all uses of memcache except for the one for the blog on the front page
>  - do the duplicate code part of bug 531720
>  - do bug 125637
>  - bug 614275
>  - bug 706129
>  - bug 708736 (should pay for itself via test cleanups)
>  - bug 888010

This one seemed like the most likely to give us a decent net negative
once fixed as it's only a matter of removing some production code and
replacing some test lines with equivalent lines using monkey-patching
helpers. However, the diff[1] actually has a net positive, because the
patch(...) line ended up being too long and had to be broken.

So, unless test lines don't count this experiment hasn't taken us any
closer to our goal of offsetting the maintenance burden introduced by
the work-item changes. I'm still hoping we can get polls removed, and
that would probably be enough to offset our stuff, but in general I
think the LoC metric will make it even harder for occasional
contributors to make changes to Launchpad.

[1] https://code.launchpad.net/~salgado/launchpad/tech-debt/+merge/92390

>  - bug 4449
> 
> (those bugs come from the first page of tech-debt bugs for launchpad
> itself; there may be more if we consider launchpad-project...)
> 
>> But maybe the situation is not as bad (depending on your perspective) as
>> I'm painting and there are plenty of stuff we could do that would give
>> us a clear net negative in the diff. If that's the case, we'd love some
>> suggestions. :)
>>
>>> And yes, removing a custom view that doesn't work in favour of one
>>> that does seems like a great step forward.
>>
>> Is there a process we should follow to remove a page?  I suppose we
>> should at least discuss that in a separate thread?
> 
> Yes; generally we can remove it trivially if the use cases will still
> be met; if they won't we need to look at how much use it gets (e.g.
> via the page performance report) and discuss (usually here) removal.

Ok, I'll have a look at the performance report and start another thread
for this.

-- 
Guilherme Salgado <https://launchpad.net/~salgado>