mahara-contributors team mailing list archive

[Aaron Wells <778254@xxxxxxxxxxxxxxxxxx>]

So here's why this is complicated.

Mahara has a separation between "Activities" and "Notifications". The
Activities are the things that cause notifications to be sent out --
posting to a forum, posting to a wall, sending a user message, these all
create an Activity record in the "activity_queue" table.

Then, the activity_process_queue() method is called by the cron. It
pulls all the existing records from activity_queue, and for each one it
deletes it (to avoid duplicate messages going out) and then calls
"handle_activity()" which then instantiates the right AcitivtyType
class, and calls its ->notify_users() method.

So the problem is that, because we delete the record from
activity_queue, everything has to be processed in one go. And in the
case of the mahara.org news forum, the 36,000 records to process is too
many.

The proposed fix is:

1. Add a "last_processed_user" column to "activity_queue".
2. In the cron, only process users whose usr.id > last_processed_user
3. After processing each user, update the activity_queue.last_processed_user value to that user's id.
4. After processing a certain number of users (1000 is probably good), quit processing users
5. If all the users get processed, then delete the activity_queue record.

It appears that we'll need to make some changes to
activity_process_queue(), handle_activity()... and several methods in
the ActivityType class, including the ActivityType->__construct() (which
is where the list of users to notify is generated),
activity_get_users(), and/or ActivityType->notify_users().

Hm... so, that it is a little tricky. None of the standard ActivityType
subclasses override the constructor or the notify_users() method, so...
that's good. It may be worth rethinking how we do all this ActivityType
stuff in order to make it easier to split it into chunks.

-- 
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it!
https://bugs.launchpad.net/bugs/778254

Title:
  Split multiple user activity notifications into chunks

Status in Mahara ePortfolio:
  Confirmed

Bug description:
  See bug #716812.

  When there are lots of users that need to be notified about a single
  activity, they need to be chunked and processed over several cron runs
  to avoid running out of memory.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mahara/+bug/778254/+subscriptions