dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Hi

I started importing around 700,000 events 2 days ago (about 5 mill
individual values) - and the import is still running, 48 hours later.
Import is slowing - it seemed to be importing around 7-8 values per second,
now down to 2-3 per second. It looks like that import might take around 200
hours - which makes no sense from a usability point of view. Importing
OrgUnits used to be equally slow, but a revamp recently change a 10,000
OrgUnit import&update from hours to something like 5 seconds.

It seems obvious that event imports have been designed to import values one
by one, which is equivalent to asking people to fetch water using a leaking
tea-spoon.

I've searched JIRA but cannot find any issue related to this, thus this
post: Are there any plans underway to make import of Event data usable for
larger record sets?

Regards
Calle

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************