dhis2-devs team mailing list archive

[Lars Helge Øverland <larshelge@xxxxxxxxx>]

Hi Calle,

I think this depends on the web server configuration. One can configure max
file size for uploads in both the proxy (nginx, apache) and servlet
container (tomcat
<http://stackoverflow.com/questions/2947683/httprequest-maximum-allowable-size-in-tomcat>
).

On nginx
<http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size>
the directive is:

client_max_body_size 200M;


regards,

Lars

On Thu, Dec 3, 2015 at 2:44 PM, Calle Hedberg <calle.hedberg@xxxxxxxxx>
wrote:

> Hi,
>
> I have found that there is a limitation in file size when importing data
> into our SERVER-based instances, while I have found no equivalent
> limitation when importing large data files (e.g. XML format) into an
> equivalent localhost instance. A few key aspects:
>
> 1. Both the server and localhost are running the latest version of 2.20
> 2. Both are running java 8 64 bits and tomcat 8.026 or 8.029
> 3. Localhost tomcat has 4GB (min) and 8GB (max) allocated
> 4. The server instance (running under Ubuntu Linux) has ~5.3GB RAM, but
> increasing/decreasing RAM has no effect on the issue.
>
> The problem is related to the upload process.
>
> Example:
> When importing a 75MB data file with around 8 mill data records (XML,
> zipped) on localhost, the initial upload step is almost instantaneous (2-3
> seconds) and then the actual import starts (takes about 10 minutes overall).
>
> When importing the same file to the equivalent instance on the server, it
> takes around 30 seconds to reach 2% upload and then the upload re-starts at
> 0% - this goes on ad infinitum.
>
> Smaller files - e.g. 10-20MB - will maybe import 15-20%, then reset to 0%
> and start over.
>
> It seems to me that the problem is related to the DHIS2 web server
> configuration, it do not allow sufficient time for the upload to happen.
>
> Any indications of how to fix this would be appreciated. While dumping the
> server instance into localhost, import the data, and then upload/restore
> the instance does work, it is a pain in the b....
>
> Regards from a sunny Cape Town
> Calle
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19119
>
> Email: calle.hedberg@xxxxxxxxx
>
> Skype: calle_hedberg
>
> *******************************************
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Lars Helge Øverland
Lead developer, DHIS 2
University of Oslo
Skype: larshelgeoverland
http://www.dhis2.org <https://www.dhis2.org>