dhis2-users team mailing list archive

[Moses Marimo <marimo@xxxxxxxxx>]

Ok. Thanks Bob. One other thing. Jason asked me the number of records in my
datavalue table. I just checked, its now over 3.6 million records. Is there
a size that would start to cause performance issues and if yes, what is
that size.



On Fri, Apr 4, 2014 at 2:07 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> Hi Moses
>
> 2G seems like a lot for the OS, but agree that over allocating (and the
> subsequent swapping and thrashing) would certainly be harmful.  Probably 1G
> for the OS would be sufficient.
>
> Bob
>
>
> On 4 April 2014 13:47, Moses Marimo <marimo@xxxxxxxxx> wrote:
>
>> Thanks Jason. Problem Solved. "How much memory have you allocated to both
>> Tomcat and Postgres" Well, this jogged my memory. Tomcat+Postgres memory
>> was more than available RAM. So I guess one of the two was boxing out the
>> other. I reduced the size of both memories so that there was at least 2GB
>> Free for the OS and it did the trick.
>>
>> Regards,
>>
>> Moses
>>
>>
>> On Fri, Apr 4, 2014 at 10:58 AM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> Sounds to me like something is hogging your system resources. How much
>>> memory have you allocated to both Tomcat and Postgres? How many rows of
>>> data do you have in your datavalue table? Which database are you using?
>>> What do the logs say? What is you current CPU usage before you start the
>>> import?
>>>
>>>
>>>
>>>
>>> On Fri, Apr 4, 2014 at 10:52 AM, Moses Marimo <marimo@xxxxxxxxx> wrote:
>>>
>>>> Thanks Jason. I have tried both options. XML, and CSV. Same result. I
>>>> have also tried running data integrity check and it took forever. Never
>>>> completed the check. Could there be a connection?
>>>>  On Apr 4, 2014 10:13 AM, "Jason Pickering" <
>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>
>>>>> Hi Moses,
>>>>> No, it is not normal. We regularly import data around this size, and
>>>>> it only takes a few minutes, on a much smaller virtual machine than what
>>>>> you mention.
>>>>>
>>>>> What type of importer are you using? XML or CSV?
>>>>>
>>>>> Regards,
>>>>> Jason
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Apr 4, 2014 at 9:41 AM, Moses Marimo <marimo@xxxxxxxxx> wrote:
>>>>>
>>>>>> Dear Colleagues,
>>>>>>
>>>>>> I am trying to benchmark the time it takes for my users to import
>>>>>> data into DHIS2.14, Build 13793. I know this import time also depends on
>>>>>> the resources available on my server, BUT a little comparison would not
>>>>>> hurt. My server has a Xeon Processor Quad-Core and the RAM is 12GB.
>>>>>>
>>>>>> On average, compressed import files of 764KB (uncompressing this file
>>>>>> gives about 15MB) in size take as much as 2-3 hours to import. Is this
>>>>>> normal? What are the experience of others out there with the time it takes
>>>>>> to import files. 3hrs just appears too much and I may be missing something.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>
>>>>>> Moses
>>>>>>
>>>>>> --
>>>>>> ICT Warehouse (Pvt) Ltd - "The Stockist of *e*-nnovation"
>>>>>> 7 Chelmsford Belgravia, Harare
>>>>>> Tel: +263-8644-102349
>>>>>> Cell: +263-772-268353
>>>>>> Website: *www.ict.co.zw <http://www.ict.co.zw>*
>>>>>>
>>>>>>
>>>>>> Innovative solutions in Web Design and E-Commerce, Software
>>>>>> Engineering, Mobile and Bulk SMS Solutions, Graphic Design and Printing,
>>>>>> General ICT Solutions and Consultancy.
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>
>>>
>>
>>
>> --
>> ICT Warehouse (Pvt) Ltd - "The Stockist of *e*-nnovation"
>> 7 Chelmsford Belgravia, Harare
>> Tel: +263-8644-102349
>> Cell: +263-772-268353
>> Website: *www.ict.co.zw <http://www.ict.co.zw>*
>>
>>
>> Innovative solutions in Web Design and E-Commerce, Software Engineering,
>> Mobile and Bulk SMS Solutions, Graphic Design and Printing, General ICT
>> Solutions and Consultancy.
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-users
>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-users
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>


-- 
ICT Warehouse (Pvt) Ltd - "The Stockist of *e*-nnovation"
7 Chelmsford Belgravia, Harare
Tel: +263-8644-102349
Cell: +263-772-268353
Website: *www.ict.co.zw <http://www.ict.co.zw>*


Innovative solutions in Web Design and E-Commerce, Software Engineering,
Mobile and Bulk SMS Solutions, Graphic Design and Printing, General ICT
Solutions and Consultancy.