dhis2-devs team mailing list archive
-
dhis2-devs team
-
Mailing list archive
-
Message #46145
Re: Data import through API in zip format
New 2.21 should be ready now for testing
--
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org
On Thu, Jul 28, 2016 at 3:07 PM, Victor Garcia <vgarciabnz@xxxxxxxxx> wrote:
> I tried Bob's suggestion, but no success... I will wait until next built
> of 2.21 and let you know the results.
>
> Thanks you all
>
> On 28 July 2016 at 14:48, Morten Olav Hansen <morten@xxxxxxxxx> wrote:
>
>> Wait about 1 hour, and try again with latest 2.21
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Jul 28, 2016 at 2:33 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
>> wrote:
>>
>>> Hi Victor
>>>
>>> I had a quick look at source code and I am not sure that we do support
>>> zip encoded payloads for datavalueset import at all :-(
>>>
>>> The way that it works auto-magically through the web ui is that we
>>> actually peek into the stream header to deduce whether the stream is
>>> zipped or gzipped.
>>>
>>> Have you tried using gzip to compress your file instead of zip? Then
>>> it should (I think) be possible to have nginx handle the decompression
>>> for you. ie. try this:
>>>
>>> curl -H "Content-Type: application/json" -H "Content-Encoding: gzip"
>>> -u admin:district -d @datavalues.json.gz
>>> "https://play.dhis2.org/demo/api/dataValueSets.json" -X POST
>>>
>>> On 28 July 2016 at 13:03, Victor Garcia <vgarciabnz@xxxxxxxxx> wrote:
>>> > This is the cURL command I try:
>>> >
>>> > curl -H "Content-Type: application/json" -u admin:district -d
>>> > @datavalues.zip "
>>> https://play.dhis2.org/demo/api/dataValueSets.json.zip" -X
>>> > POST
>>> >
>>> > The POST against DEMO server gives the same response that before. I
>>> tried
>>> > with ".json", ".zip" and ".json.zip" extensions, but no success. The
>>> > "datavalues.zip" only contains a valid .json file (sending this .json
>>> file
>>> > uncompressed, the import works).
>>> >
>>> > If you want me to try something else, just tell me.
>>> >
>>> >
>>> > On 28 July 2016 at 13:11, Halvdan Hoem Grelland <halvdan@xxxxxxxxx>
>>> wrote:
>>> >>
>>> >> Ah, my bad. I misread 'data' as 'meta data'.
>>> >>
>>> >> What is the exact request (including headers) you are using? Do you
>>> have a
>>> >> reproducible cURL we could try?
>>> >>
>>> >> Also, what is the format of the zipped data file? (The error you are
>>> >> seeing is the importer encountering a ZIP file header byte whilst
>>> expecting
>>> >> valid JSON).
>>> >>
>>> >> On Thu, Jul 28, 2016 at 12:10 PM, Victor Garcia <vgarciabnz@xxxxxxxxx
>>> >
>>> >> wrote:
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>> Just to clarify, I have tried using extensions, but it does not seem
>>> to
>>> >>> work for "/api/dataValueSets" endpoint.
>>> >>>
>>> >>>
>>> >>> On 28 July 2016 at 12:04, Morten Olav Hansen <morten@xxxxxxxxx>
>>> wrote:
>>> >>>>
>>> >>>> Hm, yeah.. seems our data endpoint don't wrap it using
>>> >>>> StreamUtils.wrapAndCheckCompressionFormat(), so that means it must
>>> be
>>> >>>> manually applied.. should be an easy fix though
>>> >>>>
>>> >>>> --
>>> >>>> Morten Olav Hansen
>>> >>>> Senior Engineer, DHIS 2
>>> >>>> University of Oslo
>>> >>>> http://www.dhis2.org
>>> >>>>
>>> >>>> On Thu, Jul 28, 2016 at 12:00 PM, Halvdan Hoem Grelland
>>> >>>> <halvdan@xxxxxxxxx> wrote:
>>> >>>>>
>>> >>>>> Quickly glancing over the source code for this stuff: it seems to
>>> >>>>> discern formats using extensions. For example, one would use a
>>> POST to
>>> >>>>> api/metaData.json.zip or api/metaData.xml.gz.
>>> >>>>>
>>> >>>>> Could you give that a try, Victor?
>>> >>>>>
>>> >>>>> On Thu, Jul 28, 2016 at 11:23 AM, Morten Olav Hansen <
>>> morten@xxxxxxxxx>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> Hi
>>> >>>>>>
>>> >>>>>> Did you try and simply send the data as a zipped file? I think we
>>> >>>>>> support both gzip and zip, you don't need to do anything else
>>> than zip it
>>> >>>>>> down before.
>>> >>>>>>
>>> >>>>>> I'm not 100% sure it will work, but this is how our metadata
>>> endpoints
>>> >>>>>> work.
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Morten Olav Hansen
>>> >>>>>> Senior Engineer, DHIS 2
>>> >>>>>> University of Oslo
>>> >>>>>> http://www.dhis2.org
>>> >>>>>>
>>> >>>>>> On Thu, Jul 28, 2016 at 11:12 AM, Victor Garcia <
>>> vgarciabnz@xxxxxxxxx>
>>> >>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Hi all,
>>> >>>>>>>
>>> >>>>>>> searching in the documentation I couldn't find a way to import a
>>> data
>>> >>>>>>> file in zip format through the API. I know it is possible to do
>>> it using the
>>> >>>>>>> user interface (import/export module), but we need to use the
>>> API. Is it
>>> >>>>>>> supported?
>>> >>>>>>>
>>> >>>>>>> This problem arises in contexts with a poor connection, where
>>> large
>>> >>>>>>> data exchanges are a pain.
>>> >>>>>>>
>>> >>>>>>> Thank you,
>>> >>>>>>>
>>> >>>>>>> Víctor
>>> >>>>>>>
>>> >>>>>>> _______________________________________________
>>> >>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> >>>>>>> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> >>>>>>> More help : https://help.launchpad.net/ListHelp
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> _______________________________________________
>>> >>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> >>>>>> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> >>>>>> More help : https://help.launchpad.net/ListHelp
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> Halvdan Hoem Grelland
>>> >>>>> Software developer, DHIS 2
>>> >>>>> University of Oslo
>>> >>>>> http://www.dhis2.org
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Halvdan Hoem Grelland
>>> >> Software developer, DHIS 2
>>> >> University of Oslo
>>> >> http://www.dhis2.org
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Mailing list: https://launchpad.net/~dhis2-devs
>>> > Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>>> > More help : https://help.launchpad.net/ListHelp
>>> >
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help : https://help.launchpad.net/ListHelp
>>>
>>
>>
>
References
-
Data import through API in zip format
From: Victor Garcia, 2016-07-28
-
Re: Data import through API in zip format
From: Morten Olav Hansen, 2016-07-28
-
Re: Data import through API in zip format
From: Halvdan Hoem Grelland, 2016-07-28
-
Re: Data import through API in zip format
From: Morten Olav Hansen, 2016-07-28
-
Re: Data import through API in zip format
From: Victor Garcia, 2016-07-28
-
Re: Data import through API in zip format
From: Halvdan Hoem Grelland, 2016-07-28
-
Re: Data import through API in zip format
From: Victor Garcia, 2016-07-28
-
Re: Data import through API in zip format
From: Bob Jolliffe, 2016-07-28
-
Re: Data import through API in zip format
From: Morten Olav Hansen, 2016-07-28
-
Re: Data import through API in zip format
From: Victor Garcia, 2016-07-28