dulwich-users team mailing list archive

[Dave Borowitz <dborowitz@xxxxxxxxxx>]

On Tue, Mar 1, 2011 at 13:48, David Blewett <david@xxxxxxxxxxxxxxxx> wrote:

> On Tue, Mar 1, 2011 at 4:39 PM, Dave Borowitz <dborowitz@xxxxxxxxxx>
> wrote:
> > On Tue, Mar 1, 2011 at 13:27, David Blewett <david@xxxxxxxxxxxxxxxx>
> wrote:
> >> Indeed, that was the problem. I looked around and couldn't find a way
> >> to disable this behavior in Git,
> >
> > Last I checked it was hard-coded over a certain (also hard-coded)
> threshold,
> > unfortuantely.
>
> I found this in the man page for git-config, but I'm not sure it's
> what we're after:
>
>       http.postBuffer
>           Maximum size in bytes of the buffer used by smart HTTP transports
>           when POSTing data to the remote system. For requests larger than
>           this buffer size, HTTP/1.1 and Transfer-Encoding: chunked is used
>           to avoid creating a massive pack file locally. Default is 1 MiB,
>           which is sufficient for most requests.
>
> I couldn't figure out where to set that, or what units it was using etc.


It's a config variable that you can set with git config [--global]
http.postBuffer xxx (in bytes). But that's not what you want. That refers to
the maximum amount of pack data that the client is willing to buffer in
memory (as opposed to writing directly to the wire). For example, if the
client is uploading a 400MB pack of the Linux kernel repo to the server, and
it's generating it on the fly, it doesn't want to have to buffer all that in
memory. But that's never gzipped, since the pack is already highly
compressed.

The gzipping only happens (I think) when an upload-pack request exceeds the
smaller LARGE_PACKET_MAX. An upload-pack request is not pack data, but
rather a list of SHAs that the client has/wants.

For reference, see:
http://git.kernel.org/?p=git/git.git;a=blob;f=remote-curl.c;h=d0fb0a044a85ec6169c4623c7c2ab2143dbb5784;hb=HEAD#l422
(Note that use_gzip is set to 1 by default and is only turned off if the
size is under the threshold.)


>  >> but I couldn't find any such
> >> in a cursory look last night.
> >
> > I haven't even done that :). I would not be opposed to rolling our own,
> both
> > because it doesn't seem very complicated and because Jelmer doesn't like
> > adding required third-party dependencies.
>
> Yeah, I think I could get something to work this week/weekend maybe.
>
> Thanks,
>
> David Blewett
>