launchpad-reviewers team mailing list archive

[Jelmer Vernooij <jelmer@xxxxxxxxx>]

Jelmer Vernooij has proposed merging lp:~jelmer/launchpad/reduce-svn-revision-limit into lp:launchpad.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)
Related bugs:
  Bug #891887 in Launchpad itself: "sourceforge imports fail due to connection hangup"
  https://bugs.launchpad.net/launchpad/+bug/891887

For more details, see:
https://code.launchpad.net/~jelmer/launchpad/reduce-svn-revision-limit/+merge/83697

Reduce the batch size of SVN code imports to 500 revisions.

Whenever importing a single revision that is part of a code import batch fails, the entire batch is cancelled and discarded. Some hosting sites have unreliable SVN servers that cause occasional connection resets. The larger the batch size, the likelier it is to be interrupted by one of these connection resets.

Reducing the batch size from 5000 to 500 should reduce the impact of these connection resets when they occur.

There is some overhead to each batch (fetching the import base branch, and pushing the branch back after the import), so decreasing the batch size means that large branches will slightly some more time. That should be insignificant compared to the overall run time though.

There is more that we can do to improve this situation - such as preserving the revisions that have already been imported when the server hangs up - but I'll leave that for another MP.
-- 
https://code.launchpad.net/~jelmer/launchpad/reduce-svn-revision-limit/+merge/83697
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~jelmer/launchpad/reduce-svn-revision-limit into lp:launchpad.

=== modified file 'lib/canonical/config/schema-lazr.conf'
--- lib/canonical/config/schema-lazr.conf	2011-11-28 08:30:05 +0000
+++ lib/canonical/config/schema-lazr.conf	2011-11-28 21:50:24 +0000
@@ -496,7 +496,7 @@
 git_revisions_import_limit: 30000
 
 # Import only this many revisions from svn (via bzr-svn) at once.
-svn_revisions_import_limit: 5000
+svn_revisions_import_limit: 500
 
 # Import only this many revisions from hg at once.
 hg_revisions_import_limit: 5000