← Back to team overview

dhis2-devs team mailing list archive

Re: Data import through API in zip format

 

I tried Bob's suggestion, but no success... I will wait until next built of
2.21 and let you know the results.

Thanks you all

On 28 July 2016 at 14:48, Morten Olav Hansen <morten@xxxxxxxxx> wrote:

> Wait about 1 hour, and try again with latest 2.21
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 28, 2016 at 2:33 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
> wrote:
>
>> Hi Victor
>>
>> I had a quick look at source code and I am not sure that we do support
>> zip encoded payloads for datavalueset import at all :-(
>>
>> The way that it works auto-magically through the web ui is that we
>> actually peek into the stream header to deduce whether the stream is
>> zipped or gzipped.
>>
>> Have you tried using gzip to compress your file instead of zip?  Then
>> it should (I think) be possible to have nginx handle the decompression
>> for you.  ie.  try this:
>>
>> curl -H "Content-Type: application/json" -H "Content-Encoding: gzip"
>> -u admin:district -d @datavalues.json.gz
>> "https://play.dhis2.org/demo/api/dataValueSets.json"; -X POST
>>
>> On 28 July 2016 at 13:03, Victor Garcia <vgarciabnz@xxxxxxxxx> wrote:
>> > This is the cURL command I try:
>> >
>> > curl -H "Content-Type: application/json" -u admin:district -d
>> > @datavalues.zip "https://play.dhis2.org/demo/api/dataValueSets.json.zip";
>> -X
>> > POST
>> >
>> > The POST against DEMO server gives the same response that before. I
>> tried
>> > with ".json", ".zip" and ".json.zip" extensions, but no success. The
>> > "datavalues.zip" only contains a valid .json file (sending this .json
>> file
>> > uncompressed, the import works).
>> >
>> > If you want me to try something else, just tell me.
>> >
>> >
>> > On 28 July 2016 at 13:11, Halvdan Hoem Grelland <halvdan@xxxxxxxxx>
>> wrote:
>> >>
>> >> Ah, my bad. I misread 'data' as 'meta data'.
>> >>
>> >> What is the exact request (including headers) you are using? Do you
>> have a
>> >> reproducible cURL we could try?
>> >>
>> >> Also, what is the format of the zipped data file? (The error you are
>> >> seeing is the importer encountering a ZIP file header byte whilst
>> expecting
>> >> valid JSON).
>> >>
>> >> On Thu, Jul 28, 2016 at 12:10 PM, Victor Garcia <vgarciabnz@xxxxxxxxx>
>> >> wrote:
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Just to clarify, I have tried using extensions, but it does not seem
>> to
>> >>> work for "/api/dataValueSets" endpoint.
>> >>>
>> >>>
>> >>> On 28 July 2016 at 12:04, Morten Olav Hansen <morten@xxxxxxxxx>
>> wrote:
>> >>>>
>> >>>> Hm, yeah.. seems our data endpoint don't wrap it using
>> >>>> StreamUtils.wrapAndCheckCompressionFormat(), so that means it must be
>> >>>> manually applied.. should be an easy fix though
>> >>>>
>> >>>> --
>> >>>> Morten Olav Hansen
>> >>>> Senior Engineer, DHIS 2
>> >>>> University of Oslo
>> >>>> http://www.dhis2.org
>> >>>>
>> >>>> On Thu, Jul 28, 2016 at 12:00 PM, Halvdan Hoem Grelland
>> >>>> <halvdan@xxxxxxxxx> wrote:
>> >>>>>
>> >>>>> Quickly glancing over the source code for this stuff: it seems to
>> >>>>> discern formats using extensions. For example, one would use a POST
>> to
>> >>>>> api/metaData.json.zip or api/metaData.xml.gz.
>> >>>>>
>> >>>>> Could you give that a try, Victor?
>> >>>>>
>> >>>>> On Thu, Jul 28, 2016 at 11:23 AM, Morten Olav Hansen <
>> morten@xxxxxxxxx>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Hi
>> >>>>>>
>> >>>>>> Did you try and simply send the data as a zipped file? I think we
>> >>>>>> support both gzip and zip, you don't need to do anything else than
>> zip it
>> >>>>>> down before.
>> >>>>>>
>> >>>>>> I'm not 100% sure it will work, but this is how our metadata
>> endpoints
>> >>>>>> work.
>> >>>>>>
>> >>>>>> --
>> >>>>>> Morten Olav Hansen
>> >>>>>> Senior Engineer, DHIS 2
>> >>>>>> University of Oslo
>> >>>>>> http://www.dhis2.org
>> >>>>>>
>> >>>>>> On Thu, Jul 28, 2016 at 11:12 AM, Victor Garcia <
>> vgarciabnz@xxxxxxxxx>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Hi all,
>> >>>>>>>
>> >>>>>>> searching in the documentation I couldn't find a way to import a
>> data
>> >>>>>>> file in zip format through the API. I know it is possible to do
>> it using the
>> >>>>>>> user interface (import/export module), but we need to use the
>> API. Is it
>> >>>>>>> supported?
>> >>>>>>>
>> >>>>>>> This problem arises in contexts with a poor connection, where
>> large
>> >>>>>>> data exchanges are a pain.
>> >>>>>>>
>> >>>>>>> Thank you,
>> >>>>>>>
>> >>>>>>> Víctor
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>> >>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> >>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> >>>>>>> More help   : https://help.launchpad.net/ListHelp
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>> >>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> >>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> >>>>>> More help   : https://help.launchpad.net/ListHelp
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Halvdan Hoem Grelland
>> >>>>> Software developer, DHIS 2
>> >>>>> University of Oslo
>> >>>>> http://www.dhis2.org
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Halvdan Hoem Grelland
>> >> Software developer, DHIS 2
>> >> University of Oslo
>> >> http://www.dhis2.org
>> >>
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~dhis2-devs
>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>

Follow ups

References