dhis2-users team mailing list archive

[Edward Ari Bichetero <ebichete@xxxxxxxxx>]

The import file contains 46800 records (individual CSV lines).


- Edward -

________________________________
From: Lars Helge Øverland <larshelge@xxxxxxxxx>
To: Edward Ari Bichetero <ebichete@xxxxxxxxx> 
Cc: "dhis2-users@xxxxxxxxxxxxxxxxxxx" <dhis2-users@xxxxxxxxxxxxxxxxxxx> 
Sent: Friday, May 4, 2012 6:07 PM
Subject: Re: [Dhis2-users] DHIS version 2.8 released





On Fri, May 4, 2012 at 4:37 PM, Edward Ari Bichetero <ebichete@xxxxxxxxx> wrote:

Congratulation on the new release, it is nice to see the new features especially the CSV import functionality.
>However ...
>
>
>I've just had a go at importing a small set of historical data (Monthly outpatient attendance by district) going back six years (2005-2011). This is just two data elements (Outpatient attendance, Outpatient reattendance) with four combo categories (Male, Female, Below 5 yrs old, 5 yrs and older) for each of our 112 districts. The CSV data file is about 4.6 megabytes in size.
>
>I gave up on watching the import process after an hour. At that point it had been using 90% of our test servers memory (4GB) and burning 100% of one cpu/core almost the entire time. This is just one of the smallest datasets we would be looking to import. It appears that the CSV import in it's current state is not able to cope with reasonably large data. Or am I getting this wrong ? Do you have any ideas/workarounds ?
>
>

Hi, approximately how many records are in your import file?

Lars