mahara-contributors team mailing list archive

[Robert Lyon <robertl@xxxxxxxxxxxxxxx>]

I notice that on my local machine Mahara 17.04+ saves

João Jiménez Māori

in the title fields as 'João Jiménez Māori'
and in the description fields as 'João Jiménez Māori'

But on cluster machines in 16.10 it saves

in the title fields as 'Jo<C3><A3>o Jim<C3><A9>nez M<C4><81>ori'
in the description fields as 'Jo&atilde;o Jim&eacute;nez M<C4><81>ori'

If I do a

 SELECT 'João Jiménez Māori' AS test;

The both show the special chars correctly

But if I do

 UPDATE view SET description = '<p>João Jiménez Māori</p>' where id =
10;

My local shows it like local above but cluster shows it like cluster
above.

So the cluster setup for postgres must be different in the way it
handles special utf8 characters

** Changed in: mahara
    Milestone: 17.10.0 => 18.04.0

-- 
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it!
https://bugs.launchpad.net/bugs/1487274

Title:
  Elasticsearch choking on non-ASCII characters

Status in Mahara:
  Confirmed

Bug description:
  In 15.10 I've added code to "quarantine" records that Elasticsearch
  won't index. That is, if Elasticsearch errors out while processing a
  batch of records, then I re-try each record individually. And if it
  errors out while processing one of those individual records, I mark
  the record as quarantined, and keep it in the
  search_elasticsearch_queue table.

  I've backported that to one of our large 15.04 sites, and since then
  I've taken a look at the data in the records that have caused
  Elasticsearch to choke. They all contain non-ASCII characters, i.e.
  Unicode characters. These can be as simple as "e with an accent over
  it", all the way up to exotic ones like emoji and the Unicode snowman.

  I was not able to replicate this when testing on my local machine, but
  it is certainly in place on our production servers, and bugs such as
  Bug 1408577 make me think it's probably also present on some other
  servers as well.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mahara/+bug/1487274/+subscriptions