launchpad-dev team mailing list archive

[Tim Penhey <tim@xxxxxxxxxx>]

Hi All,

During my jet-lagged recovery from the Epic I thought it would be a good idea 
to send out a statement of where we are on all the various moving parts of the 
launchpad code, bazaar integration, codehosting, code imports and various 
other parts under out team's remit.  I'm adding notes on things that are 
either in progress, or things that I'd like to see RSN (Real Soon Now).


Bazaar smart server access (a.k.a. bzr+ssh)
 - I paired with jml at the epic to work on the lp: short name for private 
trunk branches.  This is still work in progress and will most likely land in a 
number of progressively useful branches.  This will work by making 
bzr+ssh://bazaar.launchpad.net/+branch/project actually work and refer to a 
branch.
 - TODO: We still have a staging test machine to test out accessing the branch 
store over NFS in order to evaluate a strategy for scaling the number of 
concurrent connections to Launchpad using the smart server.  No work is being 
done on this at the moment due to other priorities knocking this down the work 
queue.
 - Some smart server processes seem to take a very large amount of RAM, 
occasionally driving the codehosting machine into swap (bug 416990 [other 
memory related bugs: loggerhead bug 518689, bzr-svn bug 536486])

Anonymous branch access (http)
 - Generally working pretty well
 - There is a bug outstanding to make this process optional for developer 
machines (branch rewrite script).
 - TODO: We would like to offer anonymous access using the smart server 
protocol, but IS is a little worried about the memory usage of the processes 
compared to serving the files using apache.  No work is being done on this at 
the moment either by lp-code team members.  It is possible that this could be 
looked at in conjunction with the NFS access as we would need more public 
facing machines to manage the load.

Codehosting filestore
 - We now only need one copy of hosted branches thanks to work my mwhudson 
earlier this year to unify the mirrored and hosted areas.
 - We still have problems when a branch that is being stacked on is renamed by 
the web app (bug 377519).  This is probably best solved with a branch job that 
updates the stacked on location of all branches that were stacked on the 
updated branch.
 - Branch upgrades are now working (mostly).  We have some undiagnosed 
failures, and a serious lack of feedback during the upgrade process (bugs 
523565 and 586467)

Loggerhead (a.k.a. codebrowse, codebounce)
 - More stable than it has been in the past, but can still be brought down by 
a mere request per second causing OOMing (see bug 518689 mentioned above)
 - I'd like to see file browsing and revision histories integrated more into 
the actual launchpad pages, but we need some more stability on loggerhead for 
that to happen.  Also, probably a view of a file that doesn't annotate (bug 
568148)
 - Launchpad's loggerhead is at least 10 revisions behind loggerhead trunk

Bazaar and plugins
 - Launchpad is still using bzr 2.1 and has many failing tests against bzr 
trunk.  We need to get these tests fixed to update launchpad to bzr 2.2.
 - both dulwich and bzr-git are over 8 weeks out of date, and trunk on these 
move quickly, so we haven't made available a number of bug fixes to Launchpad 
code imports
 - bzr-svn and bzr-hg are even further out of date
 - bzr-loom is almost certainly out of date too

Code imports
 - on the whole the import system is running very smoothly
 - the majority of failing git imports now are due to lack of submodule 
support
 - new Subversion imports have been using bzr-svn for some time, and it is 
doing a great job

Code review
 - Over 30k merge proposals have been created through Launchpad
 - The whole code review workflow for getting changes into trunk has been seen 
as a great Launchpad success
 - IN PROGRESS: providing incremental diffs for changes that have happened 
since the last comment
 - IN PROGRESS: breaking up the queue_status into review_status and 
merge_status with the intent of making a more definite semantic split between a 
proposal to merge a branch and the associated code review
 - TODO: get the diff using ajax so librarian issues don't stop the page 
rendering
 - TODO: finish off merge queues
 - TODO: more UI polish on the merge proposal page
 - TODO: a "Merge It" button to merge the proposal server-side.  This is a 
relatively simple job might slightly harder by a bzr bug (bug 375013)
 - TODO: ability to regenerate the diff for when the target branch has moved 
but the source has not

Build recipe to archive (a.k.a daily builds, build branch to archive)
 - lucid builds seem to be working
 - maverick builds are failing (bug 599102)
 - some builds for older series fail with 2a format branches (bug 599102)
 - we think there is a a problem with the daily build scheduling, we are 
investigating
 - staging builder is still not working, abentley is still chasing this up
 - IN PROGRESS: many bug fixes, and updating the build records to use the new 
goodness from soyuz
 - TODO: ui review and AJAX goodness prior to production enablement

Generic Code UI
 - recent bug fixes have reduced the query count on the main branch and merge 
proposal page
 - other pages still need some work on query reduction
 - IN PROGRESS: a redesign of the main branch page
 - TODO: a better branch chooser, some multi-option AJAX popup is planned, but 
awaiting lazr-js work

Email processing
 - processing merge directives are failing currently for two different reasons
   - merge directives into 2a format are failing (couldn't find a bug for this) 
   - process is requesting a mirror when it shouldn't be (bug 608915)

Other bits I'd like to see
 - Converting bug patches to branches and propose them for merging, relatively 
simple using a job
 - wikis (obviously)


No doubt I've missed some bits.  If I have missed anything you think I should 
comment on, please let me know.

Cheers,
Tim

Attachment: signature.asc
Description: This is a digitally signed message part.