launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

I'm going to reboot this thread: I think we got somewhat stuck because
we currently customise our notifications a great deal. The obvious
alternative to designing a system that can do exactly what we do today
is to design something that *can't* - but that is still better than
what we have.

tl;dr:
 - subscription service
 - one notification per event
 - each notification comes in several flavours - small, summary, detailed
 - subscriptions can choose what flavour they want, including
'detailed on the first notification'.
 - we no longer try to forward received mails when sending
notifications. Notifications are not a mailing list.

My thanks go to wgrant who volunteered some of his holiday time to
sanity check me on this - I couldn't let the thread go even though I
was on leave too :).

We have a number of fugly bugs around notifications precisely because
lots of different bits of code take care of doing them, and do so in
interestingly different ways.

So, I think the question needs to be:
 - whats the smallest API and feature set that we *would be willing*
to migrate to, *even if that means things change or even in some
respects regress to earlier/simpler functionality*.

To answer that question, we need to answer - would the gain be worth
the cost/loss? And that needs more detail :)
 - what would we lose
 - what would we gain

The gain is easy I think: a single consistent, reusable, scalable
notification system with event combining, multiple forms of
notification (initially in-app + email, but with twitter/identi.ca etc
ready to roll as soon as the will is there). We also move from writing
code to handle new combinations of notification to writing
subscription rules - its a domain specific language.

What would we lose is harder: that depends on what bits of complexity
we cut out.

There are, I think two key things to excise - they are not truely high
value to our users, no matter that we've invested time to have them.

Firstly, there is the 'reply by email and that email is forwarded like
a mailing list' - its a nifty little feature but as most(citation
needed) interactions happen over the web, forwarding what we receive
verbatim is relatively rarely exercised and not intrinsically
valuable. (Note that this is different to 'bug mails thread well in
mail clients', which merely depends on common subjects (gmail) or
reply/thread ids (thunderbird / evolution) etc).

Secondly, we currently send multiple different notifications for the
same event. Adding a reviewer for instance, or when a package is
accepted. This actually leads to bugs in some circumstances, which we
then need to stamp out (e.g. I recall one where a user is assigned to
a bug they already have a structural subscription to).

An aside: A useful thing to model is 'is this the first notification
to the recipient about the object?'. If it is the first notification,
some subscriptions will be more useful if they show more data (for
mediums where that makes sense - e.g. email). Consider the email
option for bugs -  always include the bug description in emails. Thats
'give me details every time'. A sensible default would be to include
the description in the first email sent to a user about the bug.

I've mentioned previously that I think we need multiple views of an
event - an in-app one, one for twitter etc, one for email in summary,
one for email in detail. If we allow subscriptions to specify what
view they want - e.g.
web/twitter/brief-mail/detailed-mail/detailed-first-only-mail, then we
can handle in a different way the current behaviour - and handle it
better.

For instance, to replace our code for new reviewer on a merge
proposal, we create a subscription for (is a reviewer) with a
subscription type of detailed-first-only-mail. We use the same type
for subscriptions to the source and target branches. When a new
reviewer is added, anyone that is already subscribed to the branch
will get the brief-mail notification, and the new reviewer, if not
*already* subscribed, will get the detailed-mail notification. This
won't be customised as it is today ('you have been requested to review
xxx') but it will have all the functional information to let someone
do the review, without duplicating the whole diff if they are already
subscribed (and thus it fixes a current bug).

A sufficiently smart template system could identify the notification
recipient as an actor in the event and replace phrases; I think that
is something we can live without for a bit, and add in a future round:
to me, the case for a single notification service is extremely
compelling even with these tradeoffs.

So, those are the two things I think we should give up. The former
frees us from knowing *where* an input to the system came from. The
latter frees us from knowing how to combine two notifications about
the same event: one notification == one event, always.

There were some other grey areas I think we can put stronger straw men
behind... the system will capture *what was done*, not *what would
have been done* - that is, if you subscribe to a bug, and look at your
notifications, it won't show a notice for previous bug comments in the
web UI or anywhere else : if you were not subscribed when an event
happened, you were not notified, period. Conversely, if you were
subscribed, the event will be in your history even after you
unsubscribe. I think this is easier to understand and explain to
users. 'Events on services which I am currently interested in' is a
related, but separate problem - it is separate because things like
event combining and 'detailed-first-only-mail' depend on knowing some
history.

That doesn't prevent us from writing a history rewriter to jiggle
things up, or adding in a 'current view' to the subscription system.
Consider though, that some subscriptions are complex to do
efficiently, and we don't want to be in the business of reevaluating
complex criteria in page renders: if we want a 'all events I am
currently interested in' view, it implies rewriting deep history
(years old in some cases) when someone e.g. unsubscribes from Ubuntu
bugs. So regardless of what we may eventually choose as a product/UI
feature, the core implementation has to start out with the assumption
of small incremental operations, not massive history rewrites.

And SOA ids as covered in the other thread will be opaque strings,
possibly bytes (if we choose urls) or unicode, or constrained unicode.

Oh, and failed notifications need to tell LP that the address (be that
twitter, irc, email etc) cannot be communicated with; even if we do
nothing with this initially, its clearly *correct* to do that, and
means that other consumers can handle bounce processing better from
day one.

I haven't revisited all the nitty gritty discussion about topics,
object and event tags etc yet, but I think we can simplify it back
down a bit with these axioms.

-Rob