dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Hi,

I have found that there is a limitation in file size when importing data
into our SERVER-based instances, while I have found no equivalent
limitation when importing large data files (e.g. XML format) into an
equivalent localhost instance. A few key aspects:

1. Both the server and localhost are running the latest version of 2.20
2. Both are running java 8 64 bits and tomcat 8.026 or 8.029
3. Localhost tomcat has 4GB (min) and 8GB (max) allocated
4. The server instance (running under Ubuntu Linux) has ~5.3GB RAM, but
increasing/decreasing RAM has no effect on the issue.

The problem is related to the upload process.

Example:
When importing a 75MB data file with around 8 mill data records (XML,
zipped) on localhost, the initial upload step is almost instantaneous (2-3
seconds) and then the actual import starts (takes about 10 minutes overall).

When importing the same file to the equivalent instance on the server, it
takes around 30 seconds to reach 2% upload and then the upload re-starts at
0% - this goes on ad infinitum.

Smaller files - e.g. 10-20MB - will maybe import 15-20%, then reset to 0%
and start over.

It seems to me that the problem is related to the DHIS2 web server
configuration, it do not allow sufficient time for the upload to happen.

Any indications of how to fix this would be appreciated. While dumping the
server instance into localhost, import the data, and then upload/restore
the instance does work, it is a pain in the b....

Regards from a sunny Cape Town
Calle

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************