launchpad-dev team mailing list archive

[Tom Berger <tom.berger@xxxxxxxxxxxxx>]

2010/1/15 Graham Binns <graham@xxxxxxxxxxxxx>:
>
> Hi folks,
>
> We're working on getting bug heat calculations working before the Bugs
> team sprint next week. We were originally going to do this on a daily
> basis, with the script iterating over all bugs as part of garbo-daily,
> but that takes too long (based on a test on staging it would take ~30
> hours to update all the bugs we currently have, and that will obviously
> increase as time goes by).
>
> So, I've come up with a few ideas about how we could do it, and I'd like
> to elicit your opinions on which might be the best one.
>
>  Idea 1:
>  - Add a heat_last_updated field to the Bug table.
>  - When the bug is touched (in a way that generates heat, see [1]),
>   reset heat_last_updated to None.
>  - As part of garbo-hourly, have a script that will check for all Bugs
>   where heat_last_updated is None and update their heat scores
>   accordingly.
>
>  Idea 2:
>  - Use the Jobs system for heat calculations.
>  - When a bug is touched, queue a job to calculate its heat.
>
>  Idea 3:
>  - Use an Event to update bug heat whenever a bug is touched.
>
> Note, in each of these cases we would still have, as part of
> garbo-daily, a script that will check all bugs whose heat hasn't been
> updated for an arbitrary period of time and update their heat
> accordingly.
>
> My own thinking is:
>
>  - Idea 1 is the least complex to implement (at least with my current
>   knowledge set).
>  - Idea 3 is a bad idea (because we have to hit the
>   subscribers-from-duplicates when calculating bug heat and that causes
>   timeouts)
>  - Idea 2 is the most scalable.
>
> So, have at it Launchpadders! I await your wisdom with bated breath.

I also think #1 is probably the best. We will have to monitor it
closely to make sure that hotness doesn't become an echo from the past
as the system grows in scale. You don't need to set heat_last_updated
to None, b.t.w. You simply have to select bugs where heat_last_updated
is older than date_last_updated, ordered by date_last_update.