launchpad-dev team mailing list archive

[Deryck Hodge <deryck.hodge@xxxxxxxxxxxxx>]

Hi, Graham.

On Fri, Jan 15, 2010 at 5:05 AM, Graham Binns <graham@xxxxxxxxxxxxx> wrote:
>
> Hi folks,
>
> We're working on getting bug heat calculations working before the Bugs
> team sprint next week. We were originally going to do this on a daily
> basis, with the script iterating over all bugs as part of garbo-daily,
> but that takes too long (based on a test on staging it would take ~30
> hours to update all the bugs we currently have, and that will obviously
> increase as time goes by).
>
> So, I've come up with a few ideas about how we could do it, and I'd like
> to elicit your opinions on which might be the best one.
>
>  Idea 1:
>  - Add a heat_last_updated field to the Bug table.
>  - When the bug is touched (in a way that generates heat, see [1]),
>   reset heat_last_updated to None.
>  - As part of garbo-hourly, have a script that will check for all Bugs
>   where heat_last_updated is None and update their heat scores
>   accordingly.
>
>  Idea 2:
>  - Use the Jobs system for heat calculations.
>  - When a bug is touched, queue a job to calculate its heat.
>
>  Idea 3:
>  - Use an Event to update bug heat whenever a bug is touched.
>
> Note, in each of these cases we would still have, as part of
> garbo-daily, a script that will check all bugs whose heat hasn't been
> updated for an arbitrary period of time and update their heat
> accordingly.
>
> My own thinking is:
>
>  - Idea 1 is the least complex to implement (at least with my current
>   knowledge set).
>  - Idea 3 is a bad idea (because we have to hit the
>   subscribers-from-duplicates when calculating bug heat and that causes
>   timeouts)
>  - Idea 2 is the most scalable.
>
> So, have at it Launchpadders! I await your wisdom with bated breath.
>
>  [1] https://dev.launchpad.net/Bugs/BugHeat#Algorithm

These are not necessarily mutually-exclusive options.  We could us the
jobs system to keep the heat number updated when bugs are touched, and
we could use a heat_last_updated field to decay heat on bugs that
haven't been touched in the last N days.  Yes, this would add
complexity, but I fear scaling issues with #1 by itself (the day we
have 100,000 bugs updated) and I fear losing the decaying option
without some way to know how long since a bug has been touched.

Maybe the jobs system can bake in the updated time as well?
Admittedly, my jobs system knowledge is very low.

Also, like Tom, I don't think we should set the date field to none if
we use that option.  We can search against a date for bugs that
haven't been updated recently.

Cheers,
deryck



-- 
Deryck Hodge
https://launchpad.net/~deryck
http://www.devurandom.org/