dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

Yes you are right :-(

I was thinking of apache reverse proxy with mod_deflate.  Nginx won't
do this out of the box, but there is a module
(http://nginx.org/en/docs/http/ngx_http_gunzip_module.html) which is
not built as standard which probably would.

Meanwhile looks like Morten has fixed anyway.

On 28 July 2016 at 14:07, Victor Garcia <vgarciabnz@xxxxxxxxx> wrote:
> I tried Bob's suggestion, but no success... I will wait until next built of
> 2.21 and let you know the results.
>
> Thanks you all
>
> On 28 July 2016 at 14:48, Morten Olav Hansen <morten@xxxxxxxxx> wrote:
>>
>> Wait about 1 hour, and try again with latest 2.21
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 28, 2016 at 2:33 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
>> wrote:
>>>
>>> Hi Victor
>>>
>>> I had a quick look at source code and I am not sure that we do support
>>> zip encoded payloads for datavalueset import at all :-(
>>>
>>> The way that it works auto-magically through the web ui is that we
>>> actually peek into the stream header to deduce whether the stream is
>>> zipped or gzipped.
>>>
>>> Have you tried using gzip to compress your file instead of zip?  Then
>>> it should (I think) be possible to have nginx handle the decompression
>>> for you.  ie.  try this:
>>>
>>> curl -H "Content-Type: application/json" -H "Content-Encoding: gzip"
>>> -u admin:district -d @datavalues.json.gz
>>> "https://play.dhis2.org/demo/api/dataValueSets.json"; -X POST
>>>
>>> On 28 July 2016 at 13:03, Victor Garcia <vgarciabnz@xxxxxxxxx> wrote:
>>> > This is the cURL command I try:
>>> >
>>> > curl -H "Content-Type: application/json" -u admin:district -d
>>> > @datavalues.zip
>>> > "https://play.dhis2.org/demo/api/dataValueSets.json.zip"; -X
>>> > POST
>>> >
>>> > The POST against DEMO server gives the same response that before. I
>>> > tried
>>> > with ".json", ".zip" and ".json.zip" extensions, but no success. The
>>> > "datavalues.zip" only contains a valid .json file (sending this .json
>>> > file
>>> > uncompressed, the import works).
>>> >
>>> > If you want me to try something else, just tell me.
>>> >
>>> >
>>> > On 28 July 2016 at 13:11, Halvdan Hoem Grelland <halvdan@xxxxxxxxx>
>>> > wrote:
>>> >>
>>> >> Ah, my bad. I misread 'data' as 'meta data'.
>>> >>
>>> >> What is the exact request (including headers) you are using? Do you
>>> >> have a
>>> >> reproducible cURL we could try?
>>> >>
>>> >> Also, what is the format of the zipped data file? (The error you are
>>> >> seeing is the importer encountering a ZIP file header byte whilst
>>> >> expecting
>>> >> valid JSON).
>>> >>
>>> >> On Thu, Jul 28, 2016 at 12:10 PM, Victor Garcia <vgarciabnz@xxxxxxxxx>
>>> >> wrote:
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>> Just to clarify, I have tried using extensions, but it does not seem
>>> >>> to
>>> >>> work for "/api/dataValueSets" endpoint.
>>> >>>
>>> >>>
>>> >>> On 28 July 2016 at 12:04, Morten Olav Hansen <morten@xxxxxxxxx>
>>> >>> wrote:
>>> >>>>
>>> >>>> Hm, yeah.. seems our data endpoint don't wrap it using
>>> >>>> StreamUtils.wrapAndCheckCompressionFormat(), so that means it must
>>> >>>> be
>>> >>>> manually applied.. should be an easy fix though
>>> >>>>
>>> >>>> --
>>> >>>> Morten Olav Hansen
>>> >>>> Senior Engineer, DHIS 2
>>> >>>> University of Oslo
>>> >>>> http://www.dhis2.org
>>> >>>>
>>> >>>> On Thu, Jul 28, 2016 at 12:00 PM, Halvdan Hoem Grelland
>>> >>>> <halvdan@xxxxxxxxx> wrote:
>>> >>>>>
>>> >>>>> Quickly glancing over the source code for this stuff: it seems to
>>> >>>>> discern formats using extensions. For example, one would use a POST
>>> >>>>> to
>>> >>>>> api/metaData.json.zip or api/metaData.xml.gz.
>>> >>>>>
>>> >>>>> Could you give that a try, Victor?
>>> >>>>>
>>> >>>>> On Thu, Jul 28, 2016 at 11:23 AM, Morten Olav Hansen
>>> >>>>> <morten@xxxxxxxxx>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> Hi
>>> >>>>>>
>>> >>>>>> Did you try and simply send the data as a zipped file? I think we
>>> >>>>>> support both gzip and zip, you don't need to do anything else than
>>> >>>>>> zip it
>>> >>>>>> down before.
>>> >>>>>>
>>> >>>>>> I'm not 100% sure it will work, but this is how our metadata
>>> >>>>>> endpoints
>>> >>>>>> work.
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Morten Olav Hansen
>>> >>>>>> Senior Engineer, DHIS 2
>>> >>>>>> University of Oslo
>>> >>>>>> http://www.dhis2.org
>>> >>>>>>
>>> >>>>>> On Thu, Jul 28, 2016 at 11:12 AM, Victor Garcia
>>> >>>>>> <vgarciabnz@xxxxxxxxx>
>>> >>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Hi all,
>>> >>>>>>>
>>> >>>>>>> searching in the documentation I couldn't find a way to import a
>>> >>>>>>> data
>>> >>>>>>> file in zip format through the API. I know it is possible to do
>>> >>>>>>> it using the
>>> >>>>>>> user interface (import/export module), but we need to use the
>>> >>>>>>> API. Is it
>>> >>>>>>> supported?
>>> >>>>>>>
>>> >>>>>>> This problem arises in contexts with a poor connection, where
>>> >>>>>>> large
>>> >>>>>>> data exchanges are a pain.
>>> >>>>>>>
>>> >>>>>>> Thank you,
>>> >>>>>>>
>>> >>>>>>> Víctor
>>> >>>>>>>
>>> >>>>>>> _______________________________________________
>>> >>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> >>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> >>>>>>> More help   : https://help.launchpad.net/ListHelp
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> _______________________________________________
>>> >>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> >>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> >>>>>> More help   : https://help.launchpad.net/ListHelp
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> Halvdan Hoem Grelland
>>> >>>>> Software developer, DHIS 2
>>> >>>>> University of Oslo
>>> >>>>> http://www.dhis2.org
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Halvdan Hoem Grelland
>>> >> Software developer, DHIS 2
>>> >> University of Oslo
>>> >> http://www.dhis2.org
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Mailing list: https://launchpad.net/~dhis2-devs
>>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>>> > More help   : https://help.launchpad.net/ListHelp
>>> >
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>