dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Hi

We are now standardising on 200M

Regards
Calle

On 3 December 2015 at 17:38, Alan Ivey <aivey@xxxxxxxxxxxxxx> wrote:

> Also, it's worth noting that the default for "client_max_body_size" is
> only 1 MB:
> http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size .
> It will need to be increased on most deployments of DHIS2.
>
> On Thu, Dec 3, 2015 at 9:09 AM, Lars Helge Øverland <larshelge@xxxxxxxxx>
> wrote:
>
>> If you are indeed using nginx, the "client_max_body_size" directive is
>> part of the installation docs example, can be increased as appropriate:
>>
>> https://www.dhis2.org/doc/snapshot/en/implementer/html/ch08s04.html#d5e575
>>
>>
>>
>> Lars
>>
>>
>>
>>
>> On Thu, Dec 3, 2015 at 3:02 PM, Lars Helge Øverland <larshelge@xxxxxxxxx>
>> wrote:
>>
>>> Hi Calle,
>>>
>>> I think this depends on the web server configuration. One can configure
>>> max file size for uploads in both the proxy (nginx, apache) and servlet
>>> container (tomcat
>>> <http://stackoverflow.com/questions/2947683/httprequest-maximum-allowable-size-in-tomcat>
>>> ).
>>>
>>> On nginx
>>> <http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size>
>>> the directive is:
>>>
>>> client_max_body_size 200M;
>>>
>>>
>>> regards,
>>>
>>> Lars
>>>
>>> On Thu, Dec 3, 2015 at 2:44 PM, Calle Hedberg <calle.hedberg@xxxxxxxxx>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have found that there is a limitation in file size when importing
>>>> data into our SERVER-based instances, while I have found no equivalent
>>>> limitation when importing large data files (e.g. XML format) into an
>>>> equivalent localhost instance. A few key aspects:
>>>>
>>>> 1. Both the server and localhost are running the latest version of 2.20
>>>> 2. Both are running java 8 64 bits and tomcat 8.026 or 8.029
>>>> 3. Localhost tomcat has 4GB (min) and 8GB (max) allocated
>>>> 4. The server instance (running under Ubuntu Linux) has ~5.3GB RAM, but
>>>> increasing/decreasing RAM has no effect on the issue.
>>>>
>>>> The problem is related to the upload process.
>>>>
>>>> Example:
>>>> When importing a 75MB data file with around 8 mill data records (XML,
>>>> zipped) on localhost, the initial upload step is almost instantaneous (2-3
>>>> seconds) and then the actual import starts (takes about 10 minutes overall).
>>>>
>>>> When importing the same file to the equivalent instance on the server,
>>>> it takes around 30 seconds to reach 2% upload and then the upload re-starts
>>>> at 0% - this goes on ad infinitum.
>>>>
>>>> Smaller files - e.g. 10-20MB - will maybe import 15-20%, then reset to
>>>> 0% and start over.
>>>>
>>>> It seems to me that the problem is related to the DHIS2 web server
>>>> configuration, it do not allow sufficient time for the upload to happen.
>>>>
>>>> Any indications of how to fix this would be appreciated. While dumping
>>>> the server instance into localhost, import the data, and then
>>>> upload/restore the instance does work, it is a pain in the b....
>>>>
>>>> Regards from a sunny Cape Town
>>>> Calle
>>>>
>>>> *******************************************
>>>>
>>>> Calle Hedberg
>>>>
>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>
>>>> Tel/fax (home): +27-21-685-6472
>>>>
>>>> Cell: +27-82-853-5352
>>>>
>>>> Iridium SatPhone: +8816-315-19119
>>>>
>>>> Email: calle.hedberg@xxxxxxxxx
>>>>
>>>> Skype: calle_hedberg
>>>>
>>>> *******************************************
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>> Lars Helge Øverland
>>> Lead developer, DHIS 2
>>> University of Oslo
>>> Skype: larshelgeoverland
>>> http://www.dhis2.org <https://www.dhis2.org>
>>>
>>>
>>
>>
>> --
>> Lars Helge Øverland
>> Lead developer, DHIS 2
>> University of Oslo
>> Skype: larshelgeoverland
>> http://www.dhis2.org <https://www.dhis2.org>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>


-- 

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************