mahara-contributors team mailing list archive

[Richard Mansfield <716812@xxxxxxxxxxxxxxxxxx>]

** Changed in: mahara
   Importance: Medium => Low

-- 
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
https://bugs.launchpad.net/bugs/716812

Title:
  Forum email notifications fail for large subscriber lists

Status in Mahara ePortfolio:
  In Progress

Bug description:
  When there are lots of subscribers to a forum or topic, some posts are
  not being sent out by email because the cron script runs out of
  memory.  This has become a problem on mahara.org where most users on
  the site are subscribed to the news forum.

  The cron script sends email out to some subscribers before failing.
  Something in dml, adodb, or the postgres drivers eats memory on every
  query, so scripts that perform a lot of queries will hit the php
  memory limit eventually, even when the list of subscribers to be
  notified fits in memory okay.

  As a stopgap I'm going to make the following changes to reduce peak
  memory usage a bit:

  - When retrieving the list of subscribers, don't pre-generate email
  content for the whole subscriber list; instead leave generation until
  the point at which a single user is notified.

  - Remove each user object from the list as it's notified.

  - Make email_user skip the mail disabled query whenever the
  maildisabled property is set in the user object.  In the
  activity_get_users query, pull the maildisabled preference from every
  user at the same time we get the list of users.

  - Remove an unused artefact_internal_profile_email query from
  email_user.

  - Only do the mailssent update in email_user when bounces_handle is
  configured.

  Even if we do all that, and fix the memory leak, we'll soon get to the
  stage where the subscriber list won't fit in memory, so we'll need to
  change the way it all works, possibly pull subscribers out in chunks
  that fit in memory, and push individual notifications for each of them
  back on to the activity queue to be handled by a later cron run.