pkgme-devs team mailing list archive

[Jonathan Lange <jml@xxxxxxxxx>]

Hello pkgme developers,

This is about lp:pkgme-service, which "weaponizes" pkgme into a web
service. It's used by http://developer.ubuntu.com to package incoming
application submissions.

djpkgme.tasks.BuildPackageTask.build_package is huge, hard to read and
nearly untestable. The tests we do have are a mess of mocks.  It's
starting to bug me, and I've had too much coffee, so here are my
thoughts.

build_package does only one thing that makes testing hard: downloading
files. Currently the mocking for this centres around
devportalbinary.database.download_file, but I think that's a mistake.
build_package is so filesystem-heavy that it to test it properly,
download_file needs to be writing a thing to disk. Looking at the
implementation, the correct point to insert a test double is
'urlopen'. We should make that return a StringIO() in our tests.

However, doing that kind of sucks and is a little fragile. What do you
think about using bzrlib.transport for all of our filesystem and
downloading needs instead? This would probably have to be done for
pkgme-binary and pkgme-service. We might be able to get away with not
using it in pkgme itself. It feels a little crazy and "purist" to me,
but I suspect it might actually help.

build_package does one thing that makes testing slow: actually
generating packages. It calls pkgme.write_packaging, which can be
quite cumbersome. To make this faster, and to avoid mocking, I suggest
that we provide an ultra-fast, trivial backend and force pkgme to use
that & only that for task tests. Perhaps we could even avoid spawning
processes for 'want' and 'all-info'.

We would still then have the problem of generating the source package.
Currently pkgme shells out to debuild. I think we can cross that
bridge when we get to it. Perhaps we could also provide a fast double
of that which meets the simplified interface that we need?

jml