c2c-oerpscenario team mailing list archive

["Olivier Dony \(OpenERP\)" <610883@xxxxxxxxxxxxxxxxxx>]

Basically, using strftime("%B") or any of the local-dependent values (%a,%b,%c,etc.) is unsafe because there are platforms where the default locale will not be using Unicode charset. This can be clearly seen in the bug description, where the locale is 'fr_FR.cp1252' i.e. french with Windows-1252 charset, Microsoft's variation of Latin1. 
And indeed, the 0xe97672 bytes mentioned in the error represent "évr" in Windows-1252. If you need to convince yourself:
  >>> "\xe9\x76\x72".decode('latin1') == u"évr"
  True

With a UTF8 locale this behaves correctly, even with accents:
  >>> import datetime
  >>> locale.setlocale(locale.LC_ALL,'')
  'fr_BE.utf8'
  >>> datetime.datetime(2000,2,1).strftime('%B')
  'f\xc3\xa9vrier'  # this is UTF-8 for 'février'

I'm afraid we can't rely on the default locale for translations, as it
can fail but will also be wrong in many cases, and we can't force the
locale to 'en_US' because locales are system-dependent and it may not be
available.

A cleaner solution would be to have a map of english month names in tools.misc, with a simple lookup function that all modules can depend on, and replace all references to strftime('%B') etc. by a lookup in this map, followed by a call to _( ) to properly translate it to the user language.
However this requires changing many modules.

We can already mitigate this issue by making sure the terms to be
translated in ir.translation.get_source() are converted to valid UTF-8
first, to catch the DataError cases. The translation may still not work,
because there will be no translation for non-english month names, but at
least the user will see the month/day names in the locale of the server,
better than nothing.


** Changed in: openobject-server
       Status: Confirmed => In Progress

** Changed in: openobject-server
    Milestone: 6.0 => 6.0-rc2

-- 
You received this bug notification because you are a member of C2C
OERPScenario, which is subscribed to the OpenERP Project Group.
https://bugs.launchpad.net/bugs/610883

Title:
  v6 employee/employees timesheet Dataerror

Status in OpenObject Server:
  In Progress

Bug description:
  menus :
human resources>reporting>timesheet>employee timesheet
human resources>reporting>timesheet>employees timesheet

same error for this 2 submenus :

Environment Information : 
System : Windows-32bit-SP2
OS Name : nt
Operating System Release : 
Operating System Version : 32bit
Operating System Architecture : 32bit
Operating System Locale : fr_FR.cp1252
Python Version : 2.5.4
OpenERP-Client Version : 6.0dev
Last revision No. & ID :Bazaar Package not Found !Traceback (most recent call last):
  File "netsvc.pyo", line 401, in dispatch
  File "service\web_services.pyo", line 584, in dispatch
  File "osv\osv.pyo", line 57, in wrapper
  File "osv\osv.pyo", line 133, in execute
  File "osv\osv.pyo", line 123, in execute_cr
  File "d:\OpenERP AllInOne\Server\addons\base_calendar\base_calendar.py", line 1755, in get
  File "d:\OpenERP AllInOne\Server\addons\base\ir\ir_values.py", line 221, in get
  File "d:\OpenERP AllInOne\Server\addons\base\ir\ir_values.py", line 221, in <lambda>
  File "d:\OpenERP AllInOne\Server\addons\base\ir\ir_values.py", line 207, in _result_get
  File "d:\OpenERP AllInOne\Server\addons\hr\hr_department.py", line 102, in read
  File "osv\orm.pyo", line 2805, in read
  File "osv\orm.pyo", line 2941, in _read_flat
  File "osv\fields.pyo", line 695, in get
  File "d:\OpenERP AllInOne\Server\addons\base\ir\ir_actions.py", line 172, in _search_view
  File "osv\orm.pyo", line 1136, in fields_get
  File "tools\misc.pyo", line 802, in cached_result
  File "d:\OpenERP AllInOne\Server\addons\base\ir\ir_translation.py", line 136, in _get_source
  File "sql_db.pyo", line 74, in wrapper
  File "sql_db.pyo", line 119, in execute
DataError: invalid byte sequence for encoding "UTF8": 0xe97672
HINT:  This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".