← Back to team overview

dhis2-users team mailing list archive

Re: Benchmarking the speed of Importing Data into DHIS2.14

 

Sounds to me like something is hogging your system resources. How much
memory have you allocated to both Tomcat and Postgres? How many rows of
data do you have in your datavalue table? Which database are you using?
What do the logs say? What is you current CPU usage before you start the
import?




On Fri, Apr 4, 2014 at 10:52 AM, Moses Marimo <marimo@xxxxxxxxx> wrote:

> Thanks Jason. I have tried both options. XML, and CSV. Same result. I have
> also tried running data integrity check and it took forever. Never
> completed the check. Could there be a connection?
> On Apr 4, 2014 10:13 AM, "Jason Pickering" <jason.p.pickering@xxxxxxxxx>
> wrote:
>
>> Hi Moses,
>> No, it is not normal. We regularly import data around this size, and it
>> only takes a few minutes, on a much smaller virtual machine than what you
>> mention.
>>
>> What type of importer are you using? XML or CSV?
>>
>> Regards,
>> Jason
>>
>>
>>
>> On Fri, Apr 4, 2014 at 9:41 AM, Moses Marimo <marimo@xxxxxxxxx> wrote:
>>
>>> Dear Colleagues,
>>>
>>> I am trying to benchmark the time it takes for my users to import data
>>> into DHIS2.14, Build 13793. I know this import time also depends on the
>>> resources available on my server, BUT a little comparison would not hurt.
>>> My server has a Xeon Processor Quad-Core and the RAM is 12GB.
>>>
>>> On average, compressed import files of 764KB (uncompressing this file
>>> gives about 15MB) in size take as much as 2-3 hours to import. Is this
>>> normal? What are the experience of others out there with the time it takes
>>> to import files. 3hrs just appears too much and I may be missing something.
>>>
>>> Regards,
>>>
>>>
>>> Moses
>>>
>>> --
>>> ICT Warehouse (Pvt) Ltd - "The Stockist of *e*-nnovation"
>>> 7 Chelmsford Belgravia, Harare
>>> Tel: +263-8644-102349
>>> Cell: +263-772-268353
>>> Website: *www.ict.co.zw <http://www.ict.co.zw>*
>>>
>>>
>>> Innovative solutions in Web Design and E-Commerce, Software Engineering,
>>> Mobile and Bulk SMS Solutions, Graphic Design and Printing, General ICT
>>> Solutions and Consultancy.
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>

Follow ups

References