dulwich-users team mailing list archive

[David Borowitz <dborowitz@xxxxxxxxxx>]

On Tue, Oct 26, 2010 at 09:39, Jelmer Vernooij <jelmer@xxxxxxxxx> wrote:

> On Mon, 2010-10-25 at 16:36 -0700, David Borowitz wrote:
> > On Mon, Oct 25, 2010 at 16:26, Jelmer Vernooij <jelmer@xxxxxxxxx>
> > wrote:
> >         On Mon, 2010-10-25 at 15:13 -0700, David Borowitz wrote:
> >         > Seeing Scott give an update on libgit2 makes me think we
> >         should
> >         > consider it for C optimizations for dulwich.
> >
> >         > The big pro is that it's existing fast, reusable code for
> >         stuff we
> >         > know dulwich is slow for (commit parsing and pack inflating
> >         come to
> >         > mind). I'm happy to keep hand-rolling C optimizations, but
> >         they'll
> >         > probably boil down to roughly the same code that's in C git
> >         and now in
> >         > libgit2. Seems like wasted effort to me.
> >
> >         > We wouldn't be sacrificing a moderately-optimized
> >         pure-Python
> >         > implementation, either. We wouldn't be using it to implement
> >         any new
> >         > functionality.
> >
> >         > The big con is that it's a third-party dependency, which
> >         we've avoided
> >         > thus far. My counter to that argument is that this could
> >         easily be the
> >         > *only* third-party dependency we'd ever need.
> >
> >         I think that's a reasonable tradeoff. It means somebody who
> >         wants to do
> >         a quick Dulwich project can still just check it out and get it
> >         to run.
> >         Right now having a C compiler is already a dependency for
> >         getting the C
> >         extensions to run (nontrivial on *nix, but it's a fairly high
> >         barrier on
> >         Windows).
> >
> >         > Another open question is whether we dulwich devs should be
> >         involved in
> >         > developing Python libgit2 bindings, which will happen
> >         regardless of
> >         > whether we use libgit2 or not.
> >
> >         GMTA. I started hacking on this a bit to see how feasible it
> >         was when we
> >         had Scott on the big screen.
> >
> >         My initial work is here (separate repository because bzr
> >         doesn't support
> >         colocated branches yet... ):
> >
> >
> http://git.samba.org/jelmer/dulwich-libgit2.git/?p=jelmer/dulwich-libgit2.git;a=summary
>
> > Yeah, this is awesome. We do need to think a bit about where to draw
> > the dulwich/libgit2 distinction. For example, will the
> > C-extension-using ShaFile instances delegate to separate python
> > wrapped libgit2 objects, or will python wrapped libgit2 objects be
> > drop-in replacements for ShaFile?
> >
> > The argument for keeping libgit2 separate and using delegation within
> > dulwich is that some people might want to use libgit2 from python
> > without buying into dulwich's object model and hierarchy. (Or would
> > they?)
> I guess it depends a bit on how close the libgit2 and Dulwich models
> are. If we can avoid extra layer that'd be great, but I can see how it
> would make sense for some folks to use something that is a thinner layer
> on libgit2.


The Ruby wrapper is a thin wrapper. I assume the github folks have built up
some additional functionality around it, but that's not part of
http://github.com/libgit2/rabbit.

Alternatively, we could modify either to be closer to the
> other.
>

In the long term I would like if Dulwich were a Python reimplementation of
the same interface as the libgit2 wrapper. Then the equivalent libgit2
objects/functions could be used as drop-in optimized replacements, the same
way our current C extensions are.

In the short term, I think this might be hard to do. The Dulwich interface
may be close to the libgit2 interface, but if it's not perfect, then we will
definitely need a layer of indirection, even if it's mostly trivial. And I
think it might be hard to gain traction with the libgit2 community (yes, I
know it's only two people :) if we come in saying we need to deviate from
their interface for backwards compatibility reasons.

So here's my proposed plan:
-Reach out to Scott and Vice ASAP offering to help with Python bindings.
-See if any parts of the interface align perfectly.
-Write the compatibility layer.
-Work on migrating Dulwich to the new interface, with the goal of eventually
removing the compatibility layer.


> I've only really looked at the index bits in libgit2 so far and those
> look pretty similar to Dulwich.
>
> Cheers,
>
> Jelmer
>
>