launchpad-dev team mailing list archive

[Stuart Bishop <stuart.bishop@xxxxxxxxxxxxx>]

On Wed, Feb 22, 2012 at 12:58 AM, Matthew Revell
<matthew.revell@xxxxxxxxxxxxx> wrote:
> On 15 February 2012 11:36, Jonathan Lange <jml@xxxxxxxxx> wrote:
>> On Tue, Feb 14, 2012 at 8:44 PM, Robert Collins
>> <robertc@xxxxxxxxxxxxxxxxx> wrote:
>>> .... include blueprints and bugs in the same bucket for karma
>>> normalisation. (On the basis that blueprints aren't distinctly more
>>> valuable than bug reports to a project).
>>>
>>> I realise this ignores the bigger questions around karma, but it seems
>>> like a cheapish thing to me.
>>
>> Maybe drop rebalancing altogether?
>
> What was the intention behind rebalancing? To encourage more activity
> in the lesser used parts of LP? To show that even a relatively small
> amount of activity has a larger impact in a lesser used part of LP?

Karma was always to encourage use of Launchpad. Strangely enough, it
does motivate quite a lot of people. Or at least it did... I haven't
been paying attention to karma the last year or three.

When talking about balancing, you need to be a bit more specific.
Balancing is absolutely crucial - if I create an event worth one
million points it rather obviously overshadows all the 1 and 2 point
events. But with different sections of Launchpad being developed by
different teams, balance was only relative within those sections.
People using one section of Launchpad dominated the karma ladders, not
because they put in more effort but because the arbitrary numbers
assigned to the section of lp they used were higher, or they got them
more frequently. So we added automatic rebalancing across realms,
because we didn't know enough to do it ourselves. People using
translations where on a more even footing to people working on bugs.
It still sucked, but it was much better. More people where motivated
to 'play Launchpad' once we fixed the top karma ladders to a) have
representatives using different parts of Launchpad and b) penalize
people paid to work on Launchpad (Canonical employees get their Karma
halved IIRC). It became a game more people wanted to play because more
people could actually participate and have a hope of recognition.

> What does rebalancing give us? Seemingly unfair karma scores.

Automatic rebalancing makes the karma scores fairer, because different
regions of Launchpad have never had their scores manually balanced. It
is unfair if 1 minutes effort working on bugs is worth the same as 20
minutes of working on translations.

Now we have a better idea of what Launchpad looks like and how people
use it, we can go through and rebalance the scores ourselves and turn
off the automatic rebalancing. This will turn off the wobble (except
when we manually tweak the scores again), and should be even fairer.
mpt's old suggestion was for scores to be effort based - say 60 points
for 1 minutes worth of effort. We should by now be able to actually
score these events like that.

> I wonder if karma's time has passed. We know that some people value it
> but perhaps it's one of those things that doesn't justify its
> maintenance costs. Are people *really* motivated by karma? If so,
> karma is such a poor reflection of value provided that I imagine karma
> only serves to disappoint most people who find it genuinely
> motivating.

Its weird, but yeah - it actually motivates some people. I am
interested in jono's Ubuntu achievements work though - perhaps that
sort of model is better? He is keep for a less gameable system, but in
practice I don't think we have seen anyone bother to game lp karma -
perhaps it is so obviously gameable that nobody has bothered as it
just makes them look like a tosser.

One way to see how many people who currently care is to twiddle the
system and listen to the uproar. In the past, it has been surprisingly
loud, and probably why nobody has gone ahead and twiddled the
implementation despite some tentative agreements on direction in the
past :)

There are other ideas to make it better too. Displaying with a
logarithmic scale was one that had traction, in order to close the
gaps between the leaders and make catching up actually seem
achievable. Even trashing displaying the score, and instead displaying
your position (Ranked 267834th of 402876! Or 'Top 10%'. Or whatever).
But these are orthogonal to the underlying score and automatic
rebalancing.

> If we did kill karma, I'd lobby to keep the list of karma earning
> actions at ~/+karma; the most useful aspect of karma, to my mind.
>
> If the karma score went away, but we kept the activity list, would
> anyone miss it?

I've always said if we want an activity list, we should implement an
activity list. Karma events are a rather poor implementation of an
activity log.

-- 
Stuart Bishop <stuart.bishop@xxxxxxxxxxxxx>