mahara-contributors team mailing list archive

[Robert Lyon <robertl@xxxxxxxxxxxxxxx>]

Public bug reported:

Currently if we want to upgrade some of our tables like view, artefact,
view_atefact they can take up lots of time/memory if elasticsearch is
being used as that has triggers that add info to the
search_elasticsearch_queue table as well.

And with elasticsearch queue triggers it checks the queue for the item
about to be added and only add it if not already there. So if the number
of rows to update is 100,000+ each row add/check is slightly slower than
the last.


Instead we should just drop the triggers when we do the change and add them back in after and to make this easier we should have a 
   drop_elasticsearch_trigger($table);
and
   create_elasticsearch_trigger($table);
pair that we can wrap around specific updates that will do the check to see if elasticsearch is being used and do the correct dropping/adding of triggers

** Affects: mahara
     Importance: High
     Assignee: Robert Lyon (robertl-9)
         Status: In Progress

** Affects: mahara/17.04
     Importance: High
         Status: In Progress

** Affects: mahara/17.10
     Importance: High
     Assignee: Robert Lyon (robertl-9)
         Status: In Progress

** Affects: mahara/18.04
     Importance: High
         Status: In Progress

** Also affects: mahara/17.10
   Importance: High
     Assignee: Robert Lyon (robertl-9)
       Status: In Progress

** Also affects: mahara/17.04
   Importance: Undecided
       Status: New

** Also affects: mahara/18.04
   Importance: Undecided
       Status: New

** Changed in: mahara/18.04
   Importance: Undecided => High

** Changed in: mahara/17.04
   Importance: Undecided => High

** Changed in: mahara/17.10
    Milestone: 18.04.0 => 17.10rc2

** Changed in: mahara/18.04
    Milestone: None => 18.04.0

** Changed in: mahara/17.04
    Milestone: None => 17.04.4

** Changed in: mahara/18.04
       Status: New => In Progress

** Changed in: mahara/17.04
       Status: New => In Progress

-- 
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it!
https://bugs.launchpad.net/bugs/1720912

Title:
  Better way to disable elasticsearch triggers during upgrade

Status in Mahara:
  In Progress
Status in Mahara 17.04 series:
  In Progress
Status in Mahara 17.10 series:
  In Progress
Status in Mahara 18.04 series:
  In Progress

Bug description:
  Currently if we want to upgrade some of our tables like view,
  artefact, view_atefact they can take up lots of time/memory if
  elasticsearch is being used as that has triggers that add info to the
  search_elasticsearch_queue table as well.

  And with elasticsearch queue triggers it checks the queue for the item
  about to be added and only add it if not already there. So if the
  number of rows to update is 100,000+ each row add/check is slightly
  slower than the last.

  
  Instead we should just drop the triggers when we do the change and add them back in after and to make this easier we should have a 
     drop_elasticsearch_trigger($table);
  and
     create_elasticsearch_trigger($table);
  pair that we can wrap around specific updates that will do the check to see if elasticsearch is being used and do the correct dropping/adding of triggers

To manage notifications about this bug go to:
https://bugs.launchpad.net/mahara/+bug/1720912/+subscriptions