launchpad-dev team mailing list archive
Mailing list archive
RFC: Bug import ideas
I've had a few ideas swimming around in my head for a while to make
bug importing a little/lot less painful. They're probably similar to a
lot of the ideas that anyone who's had to do a bug import has had, but
here they are written down:
* Allow imports via the web UI or API.
For example, upload bug data to the librarian, job comes along,
processes it, and sends email to the uploader and the project
owner. This would be quite easy to do, now that we have experience
with the job system (and that we have a job system).
This is definitely not innovative, but with the two features below
it becomes a complete solution, imo.
* Commit only at the end.
With this we can do dry runs, and not have to faff with cache
(Currently the bug importer commits after each bug, and records
those bugs that imported correctly in a cache file. This is then
handed to subsequent bug import runs so that it doesn't attempt to
import the same bug twice. Fiddly and fragile are two ways to
Additionally, a failure in an import can point to a broader data
issue, also present in the bugs that did get imported but were not
corrupted enough to fail. They cannot now be reverted, so we must
wait for a refresh in staging, or figure out a manual fix in
Only committing at the end would be an important part of allowing
self-service imports to Launchpad users. As in, users could do
trial imports for ever and a day, but can't commit. They would be
responsible for refining the data, and a LOSA or LP dev would only
be needed for support along the way, a final sanity check and a
trigger-pull. In staging, this restriction could be relaxed.
* More up-front validation.
The bug importer currently works with anything that has the correct
tags in there somewhere. We could add another layer of validation
to the input. Not to be difficult, but to catch misunderstandings,
to give good feedback when a field is formatted incorrectly or when
there are extra tags that we don't recognise, all with guidance on
how to make it better.
Having a good reference validator also makes it easier for people
to test any bug export code they come up with.
Even before addressing the first point and allowing anyone at all to
do/try LOSA-less imports, the commit-at-end and better-validation
ideas will help us a lot for a (at a guess) smallish amount of effort.
Any comments? It would be nice to get a story started for this even if
it's going to be a while before we can do any work on it.