← Back to team overview

dhis2-devs team mailing list archive

Re: Import of Events is extremely slow - any plans to improve it?

 

Hi Calle,

Thanks for your question.
Just curious (I don't have answer). As we are planning to do this soon.

Are you doing this through UI or API?
If API, which endpoint enables this?

Thanks

Ime



On Jun 28, 2017 19:15, "Calle Hedberg" <calle.hedberg@xxxxxxxxx> wrote:

> Hi
>
> I started importing around 700,000 events 2 days ago (about 5 mill
> individual values) - and the import is still running, 48 hours later.
> Import is slowing - it seemed to be importing around 7-8 values per second,
> now down to 2-3 per second. It looks like that import might take around 200
> hours - which makes no sense from a usability point of view. Importing
> OrgUnits used to be equally slow, but a revamp recently change a 10,000
> OrgUnit import&update from hours to something like 5 seconds.
>
> It seems obvious that event imports have been designed to import values
> one by one, which is equivalent to asking people to fetch water using a
> leaking tea-spoon.
>
> I've searched JIRA but cannot find any issue related to this, thus this
> post: Are there any plans underway to make import of Event data usable for
> larger record sets?
>
> Regards
> Calle
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19119
>
> Email: calle.hedberg@xxxxxxxxx
>
> Skype: calle_hedberg
>
> *******************************************
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>

Follow ups

References