← Back to team overview

dhis2-devs team mailing list archive

Re: Event-data CSV import issues

 

Also I think the performance was enhanced recently.
On 11 Mar 2015 23:09, "Pierre Dane" <pierre@xxxxxxxxx> wrote:

> hi Calle
>
> FYI I have some database scripts that I used to import events into our
> program - 300k records in 10 mins or so. if you need to bulk insert events
> let me know and I can share. p
>
>
>
> On 11 Mar 2015, at 23:49, Calle Hedberg <calle.hedberg@xxxxxxxxx> wrote:
>
> Hi
>
> There seems to be at least two bugs related to the import of event-data in
> CSV format introduced in 2.17.
>
> Firstly, a dry run importing around 29,000 events (~250,000 trackervalue
> records) into 2.18 (version as of 09 Mar 2015) did not seem to work - no
> feedback within reasonable time. NOTE that the import with dry run set to
> NO worked fine, but it was quite slow - so there is a possibility the dry
> run actually works BUT since it gives no indication of activity for a LOONG
> time it is perceived as not working. If that is the case, it is important
> to add a counter or similar to the user messages (e.g. "1000 events
> imported" .... "2000 events imported" etc).
>
> Secondly, DHIS2 has a confusing set of standards for uppercase and
> lowercase keywords. When you export some event data to CSV, it will export
> the content of "ProvidedElsewhere" as it is stored: e.g. "FALSE" as an
> uppercase word. If you try to import the same file, it is rejected and the
> tomcat log indicates that only "false" and "true" are accepted (lowercase).
>
> Thirdly, from a more "cosmetic" perspective, I find it annoying that a
> tomcat log entry is produced for EVERY event imported - it would be better
> if only "abnormal" issues are logged otherwise any abnormalities or
> warnings drowns in the "info about every event record" clutter.
>
> Regards
> Calle
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19274
>
> Email: calle.hedberg@xxxxxxxxx
>
> Skype: calle_hedberg
>
> *******************************************
>
>  _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>

Follow ups

References