dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Ime,

I was using the UI - have not looked at the API endpoint.

For now I will use sql scripts as I've done before - I'm just trying to
find out if anybody is working on improving what is (now) an import
function that in reality only caters for small data sets. There is no
question about the core developers ability to speed this up - I saw that
with the import of OU meta-data - but not sure if anybody has done anything.

If no answers from core developers, the I will write a JIRA issue for it.

Regards
Calle

On 29 June 2017 at 02:25, Ime Asangansi <imeasangansi@xxxxxxxxx> wrote:

> Hi Calle,
>
> Thanks for your question.
> Just curious (I don't have answer). As we are planning to do this soon.
>
> Are you doing this through UI or API?
> If API, which endpoint enables this?
>
> Thanks
>
> Ime
>
>
>
> On Jun 28, 2017 19:15, "Calle Hedberg" <calle.hedberg@xxxxxxxxx> wrote:
>
>> Hi
>>
>> I started importing around 700,000 events 2 days ago (about 5 mill
>> individual values) - and the import is still running, 48 hours later.
>> Import is slowing - it seemed to be importing around 7-8 values per second,
>> now down to 2-3 per second. It looks like that import might take around 200
>> hours - which makes no sense from a usability point of view. Importing
>> OrgUnits used to be equally slow, but a revamp recently change a 10,000
>> OrgUnit import&update from hours to something like 5 seconds.
>>
>> It seems obvious that event imports have been designed to import values
>> one by one, which is equivalent to asking people to fetch water using a
>> leaking tea-spoon.
>>
>> I've searched JIRA but cannot find any issue related to this, thus this
>> post: Are there any plans underway to make import of Event data usable for
>> larger record sets?
>>
>> Regards
>> Calle
>>
>> *******************************************
>>
>> Calle Hedberg
>>
>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>
>> Tel/fax (home): +27-21-685-6472 <+27%2021%20685%206472>
>>
>> Cell: +27-82-853-5352 <+27%2082%20853%205352>
>>
>> Iridium SatPhone: +8816-315-19119 <+881%206%20315%2019119>
>>
>> Email: calle.hedberg@xxxxxxxxx
>>
>> Skype: calle_hedberg
>>
>> *******************************************
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>


-- 

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************