openupgrade-drivers team mailing list archive

[Manuel Vázquez Acosta <mva.led@xxxxxxxxx>]

Hi all,

First, let me apologize myself for the long email.  Is both a log of my
progress and proposals of change.


I'm testing the upgrade with Stefan's 8.0-crm branch (I have a DB with
CRM in use).

I've seen a couple of problems related to the base_calendar -> calendar
migration.

The first error occurs cause some of my recurrent events have
timezone-unaware UNTIL specs, ie:
FREQ=MONTHLY;INTERVAL=1;UNTIL=20151231T235959;BYDAY=2WE

When the post migration script of `calendar` calls the
`get_recurrent_date_by_event` method and then `rruleset` fails while
iterating over the set, cause the startdate is timezone-aware::

  File
"/home/manu/src/merchise/pgi/openupgrade/addons/calendar/migrations/8.0.1.0/post-migration.py",
line 207, in migrate
    cr, SUPERUSER_ID, pool, ['calendar.event'])
  File
"/home/manu/src/merchise/pgi/openupgrade/openerp/openupgrade/openupgrade_80.py",
line 76, in set_message_last_post
    cr, uid, [], context={'active_test': False})
  File "/home/manu/src/merchise/pgi/openupgrade/openerp/api.py", line
237, in wrapper
    return old_api(self, *args, **kwargs)
  File
"/home/manu/src/merchise/pgi/openupgrade/addons/calendar/calendar.py",
line 1450, in search
    res = self.get_recurrent_ids(cr, uid, res, args, order=order,
context=context)
  File "/home/manu/src/merchise/pgi/openupgrade/openerp/api.py", line
237, in wrapper
    return old_api(self, *args, **kwargs)
  File
"/home/manu/src/merchise/pgi/openupgrade/addons/calendar/calendar.py",
line 1156, in get_recurrent_ids
    rdates = self.get_recurrent_date_by_event(cr, uid, ev, context=context)
  File "/home/manu/src/merchise/pgi/openupgrade/openerp/api.py", line
237, in wrapper
    return old_api(self, *args, **kwargs)
  File
"/home/manu/src/merchise/pgi/openupgrade/addons/calendar/calendar.py",
line 702, in get_recurrent_date_by_event
    return [d.astimezone(pytz.UTC) for d in rset1]
  File
"/home/manu/.buildout/eggs/python_dateutil-2.3-py2.7.egg/dateutil/rrule.py",
line 998, in _iter
    self._genitem(rlist, gen)
  File
"/home/manu/.buildout/eggs/python_dateutil-2.3-py2.7.egg/dateutil/rrule.py",
line 936, in __init__
    self.dt = advance_iterator(gen)
  File
"/home/manu/.buildout/eggs/python_dateutil-2.3-py2.7.egg/dateutil/rrule.py",
line 611, in _iter
    if until and res > until:
TypeError: can't compare offset-naive and offset-aware datetimes


Probably this is an error in one of our addons that didn't set the until
right.  I will solve it by introducing a pre migration that rewrites all
UNTIL rrules ensuring they are tz-aware in UTC (that would be the right
thing to do most of the time, isn't it?).

For the moment I patched calendar.py to do exactly that and I moved on.

The 2nd error now happens in openupgrade_80.set_message_last_post::

  File
"/home/manu/src/merchise/pgi/openupgrade/openerp/openupgrade/openupgrade_80.py",
line 84, in set_message_last_post
    (datetime.strptime(value, DATETIME_FMT), res_id))
  File "/home/manu/src/merchise/pgi/openupgrade/openerp/sql_db.py", line
158, in wrapper
    return f(self, *args, **kwargs)
  File "/home/manu/src/merchise/pgi/openupgrade/openerp/sql_db.py", line
234, in execute
    res = self._obj.execute(query, params)
DataError: invalid input syntax for integer: "3759-20140212193001"
LINE 1: ...ost = '2013-12-27T22:09:24'::timestamp WHERE id = '3759-2014...


This is because calendar's search method returns the "virtual ids"
instead of the real ids.   Has anyone had the same issues?

What I did was to simply replace the search call in
set_message_last_post with::

cr.execute('SELECT id FROM {table}'.format(table=model_pool._table))
obj_ids = [row[0] for row in cr.fetchall()]

This change got me beyond I was getting so far without introducing new
errors in other modules' migration scripts.

After that, the game stopper was a::

   File
"/home/manu/src/merchise/pgi/openupgrade/addons/hr_holidays/migrations/8.0.1.5/pre-migration.py",
line 30, in migrate
    openupgrade.get_legacy_name('crm_meeting_id'),
  File "/home/manu/src/merchise/pgi/openupgrade/openerp/sql_db.py", line
158, in wrapper
    return f(self, *args, **kwargs)
  File "/home/manu/src/merchise/pgi/openupgrade/openerp/sql_db.py", line
234, in execute
    res = self._obj.execute(query, params)
IntegrityError: insert or update on table "hr_holidays" violates foreign
key constraint "hr_holidays_meeting_id_fkey"
DETAIL:  Key (meeting_id)=(3990) is not present in table "crm_meeting".

But then the hr_holidays modules is not reported as Done yet (in the
OpenUpgrade page is still blank).  Nevertheless the
openupgrade_analysis_work.txt file seems to be Done (both in 8.0-crm and
8.0).  Any ideas?


Best regards,
Manuel.