launchpad-dev team mailing list archive

[Jonathan Lange <jml@xxxxxxxxx>]

Hello,

I'm writing a program with launchpadlib that needs to run through a
lot of data. As such, I'm parallelizing it. It only does read
operations against Launchpad, but it turns out that even those read
operations manage to corrupt the cached WADL.

Essentially, after running the importer for a little bit, every worker
starts to raise these errors on login_anonymously:

    http://paste.ubuntu.com/697373/

I used strace to confirm that the command is indeed opening up an
invalid XML file, and that file contains a large chunk of the
Launchpad WADL.

My guess is that at some previous point, two workers are trying to
login_anonymously together, and that somehow corrupts the copy on
disk. Note that the same filename is used regardless of the name of
the app.

Going forward, there seem to be a bunch of bugs and a couple of
options as to how I proceed.

It seems to be a bug that a corrupt entry in the cache causes complete
login failure. I would file it, but I'm a little tired of playing the
"is it launchpadlib, wadllib, lazr.restful or lazr.restfulclient"
game.

It's probably also a bug that launchpadlib allows this corruption in
the first place.

As for me, I'm *probably* going to just restructure my program so I
don't have the workers talking to Launchpad. If someone here can give
me a clear path to fixing these bugs, and I can have a reasonable hope
of a prompt release, then I might try fixing them in the underlying
libraries.

cheers,
jml