dulwich-users team mailing list archive

[Andi McClure <andi.m.mcclure@xxxxxxxxx>]

Hello, I have been meaning to email this list about this. I have been
seeing a similar problem for the last week.

I have a large (>1GB) repository I work with. I used hg-git several times
to push and pull from it with success. I always push/pull *locally*, i.e.,
from a local git checkout, I don't push/pull to the server using
hg-git/dulwich. Eventually, one day, out of the blue, whenever I tried to
push *or* pull I started to get a stack like:

pulling from /home/REDACTED/REDACTED
** Unknown exception encountered with possibly-broken third-party extension
hggit
** which supports versions 2.3.1 of Mercurial.
** Please disable hggit and try your action again.
** If that fixes the bug please report it to
https://bitbucket.org/durin42/hg-git/issues
** Python 2.6.5 (r265:79063, Sep 26 2013, 18:51:09) [GCC 4.4.3]
** Mercurial Distributed SCM (version 2.8)
** Extensions loaded: hggit, graphlog, strip, mq, pager, color
Traceback (most recent call last):
  File "/usr/local/bin/hg", line 5, in <module>
    pkg_resources.run_script('mercurial==2.8', 'hg')
  File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 461, in
run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 1194, in
run_script
    execfile(script_filename, namespace, namespace)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/EGG-INFO/scripts/hg",
line 38, in <module>
    mercurial.dispatch.run()
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 28, in run
    sys.exit((dispatch(request(sys.argv[1:])) or 0) & 255)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 69, in dispatch
    ret = _runcatch(req)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 133, in _runcatch
    return _dispatch(req)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 806, in _dispatch
    cmdpats, cmdoptions)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 585, in runcommand
    ret = _runcommand(ui, options, cmd, d)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/extensions.py",
line 196, in wrap
    return wrapper(origfn, *args, **kwargs)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/hgext/color.py",
line 419, in colorcmd
    return orig(ui_, opts, cmd, cmdfunc)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/extensions.py",
line 196, in wrap
    return wrapper(origfn, *args, **kwargs)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/hgext/pager.py",
line 138, in pagecmd
    return orig(ui, options, cmd, cmdfunc)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 897, in _runcommand
    return checkargs()
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 868, in checkargs
    return cmdfunc()
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/dispatch.py",
line 803, in <lambda>
    d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/util.py",
line 512, in check
    return func(*args, **kwargs)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/extensions.py",
line 151, in wrap
    util.checksignature(origfn), *args, **kwargs)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/util.py",
line 512, in check
    return func(*args, **kwargs)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/hgext/mq.py",
line 3376, in mqcommand
    return orig(ui, repo, *args, **kwargs)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/util.py",
line 512, in check
    return func(*args, **kwargs)
  File
"/usr/local/lib/python2.6/dist-packages/mercurial-2.8-py2.6-linux-i686.egg/mercurial/commands.py",
line 4585, in pull
    remotebookmarks = other.listkeys('bookmarks')
  File "/home/REDACTED/hg-git/hggit/gitrepo.py", line 63, in listkeys
    refs = handler.fetch_pack(self.path)
  File "/home/REDACTED/hg-git/hggit/git_handler.py", line 1020, in
fetch_pack
    ret = client.fetch_pack(path, determine_wants, graphwalker, f.write,
progress.progress)
  File
"/usr/local/lib/python2.6/dist-packages/dulwich-0.9.4-py2.6-linux-i686.egg/dulwich/client.py",
line 522, in fetch_pack
    refs, server_capabilities = read_pkt_refs(proto)
  File
"/usr/local/lib/python2.6/dist-packages/dulwich-0.9.4-py2.6-linux-i686.egg/dulwich/client.py",
line 147, in read_pkt_refs
    for pkt in proto.read_pkt_seq():
  File
"/usr/local/lib/python2.6/dist-packages/dulwich-0.9.4-py2.6-linux-i686.egg/dulwich/protocol.py",
line 148, in read_pkt_seq
    pkt = self.read_pkt_line()
  File
"/usr/local/lib/python2.6/dist-packages/dulwich-0.9.4-py2.6-linux-i686.egg/dulwich/protocol.py",
line 105, in read_pkt_line
    size = int(sizestr, 16)
ValueError: invalid literal for int() with base 16: 'GIT:'

I then re-cloned a new local hg repository from the local git repository
(an arduous process with hg-git). Once I did this, the newly cloned hg
repository could again read and write from the local git repository.
However, a week later, this broke again with the same problem.

One more thing. Let's say I patch hg-git, such that when the "size =
int(sizestr, 16)" line where the problem always occurs is reached, I first
print "sizestr". It turns out to be equal to the following two lines:
GIT:
 upload-pack /home/REDACTED/REDACTED

(Of course REDACTED/REDACTED was in the original the path to the git
repository I am pulling/pushing.)

This was on Ubuntu Linux BTW. I am using the newest release Dulwich and a
slightly out of date, slightly patched version of hg-git.

What should I do?

"Can you try patching hg-git so it uses LocalGitClient rather than
SubprocessGitClient? "

Does this just mean swapping out a class? Are the APIs/external interfaces
for these classes at all different?


On Wed, Dec 18, 2013 at 8:12 AM, Jelmer Vernooij <jelmer@xxxxxxxxx> wrote:

> On Wed, Dec 18, 2013 at 05:38:56PM +0300, anatoly techtonik wrote:
> > On Wed, Dec 18, 2013 at 3:24 PM, Jelmer Vernooij <jelmer@xxxxxxxxx>
> wrote:
> > > On Wed, Dec 18, 2013 at 02:36:54PM +0300, anatoly techtonik wrote:
> > >> I try to clone my repository from GitHub with hg-git and it fails.
> > >> Dulwich 1.9.4, Hg-Git 0.5.0, Windows.
> > >> Please, CC.
> > >>
> > >> destination directory: recordscreen
> > >> ["git-upload-pack '/techtonik/recordscreen'"]
> > >> ** Unknown exception encountered with possibly-broken third-party
> > >> extension hggit
> > >> ** which supports versions 2.2.3 of Mercurial.
> > >> ** Please disable hggit and try your action again.
> > >> ** If that fixes the bug please report it to
> > >> https://bitbucket.org/durin42/hg-git/issues
> > >> ** Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit
> (Intel)]
> > >> ** Mercurial Distributed SCM (version 2.8)
> > >> ** Extensions loaded: churn, color, convert, evolve, hggit, strip, mq,
> > >> rebase, record
> > >> Traceback (most recent call last):
> > >>   File "hg", line 38, in <module>
> > >>   File "mercurial\dispatch.pyc", line 28, in run
> > >>   File "mercurial\dispatch.pyc", line 69, in dispatch
> > >>   File "mercurial\dispatch.pyc", line 133, in _runcatch
> > >>   File "mercurial\dispatch.pyc", line 806, in _dispatch
> > >>   File "mercurial\dispatch.pyc", line 585, in runcommand
> > >>   File "mercurial\extensions.pyc", line 196, in wrap
> > >>   File "hgext\color.pyc", line 419, in colorcmd
> > >>   File "mercurial\dispatch.pyc", line 897, in _runcommand
> > >>   File "mercurial\dispatch.pyc", line 868, in checkargs
> > >>   File "mercurial\dispatch.pyc", line 803, in <lambda>
> > >>   File "mercurial\util.pyc", line 512, in check
> > >>   File "mercurial\commands.pyc", line 1282, in clone
> > >>   File "mercurial\hg.pyc", line 372, in clone
> > >>   File "mercurial\localrepo.pyc", line 2431, in clone
> > >>   File "C:\hg-git\hggit\hgrepo.py", line 14, in pull
> > >>     return git.fetch(remote.path, heads)
> > >>   File "C:\hg-git\hggit\git_handler.py", line 205, in fetch
> > >>     refs = self.fetch_pack(remote, heads)
> > >>   File "C:\hg-git\hggit\git_handler.py", line 1016, in fetch_pack
> > >>     ret = client.fetch_pack(path, determine_wants, graphwalker,
> > >> f.write, progress.progress)
> > >>   File "dulwich\client.pyc", line 527, in fetch_pack
> > >>   File "dulwich\client.pyc", line 144, in read_pkt_refs
> > >>   File "dulwich\protocol.pyc", line 147, in read_pkt_seq
> > >>   File "dulwich\protocol.pyc", line 104, in read_pkt_line
> > >> ValueError: invalid literal for int() with base 16: 'logi'
> > >
> > > Can you reproduce it with dulwich clone (without hg-git)? Is there an
> easy way
> > > for us to reproduce this?
> > No. I can't reproduce this error with "dulwich clone".
> Can you try patching hg-git so it uses LocalGitClient rather than
> SubprocessGitClient? Does that fix the issue?
>
> > > py dulwich-cli.py clone git+ssh://github.com/jelmer/dulwich.git aaa
> > Traceback (most recent call last):
> >   File "dulwich-cli.py", line 253, in <module>
> >     commands[cmd](sys.argv[2:])
> >   File "dulwich-cli.py", line 174, in cmd_clone
> >     porcelain.clone(source, target, bare=("--bare" in opts))
> >   File "C:\hg-git\dulwich\porcelain.py", line 167, in clone
> >     progress=outstream.write)
> >   File "C:\hg-git\dulwich\client.py", line 212, in fetch
> >     target.get_graph_walker(), f.write, progress)
> >   File "C:\hg-git\dulwich\client.py", line 521, in fetch_pack
> >     proto, can_read = self._connect('upload-pack', path)
> >   File "C:\hg-git\dulwich\client.py", line 881, in _connect
> >     port=self.port, username=self.username)
> >   File "C:\hg-git\dulwich\client.py", line 744, in run_command
> >     stdout=subprocess.PIPE)
> >   File "C:\Python27\lib\subprocess.py", line 709, in __init__
> >     errread, errwrite)
> >   File "C:\Python27\lib\subprocess.py", line 957, in _execute_child
> >     startupinfo)
> > WindowsError: [Error 2] The system cannot find the file specified
> This is almost referring to SSH. Do you have SSH installed and accessible
> from
> the path on Windows?
>
> Cheers,
>
> Jelmer
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dulwich-users
> Post to     : dulwich-users@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dulwich-users
> More help   : https://help.launchpad.net/ListHelp
>