← Back to team overview

dhis2-devs team mailing list archive

Re: Data import through API in zip format

 

Wait about 1 hour, and try again with latest 2.21

-- 
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Jul 28, 2016 at 2:33 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> Hi Victor
>
> I had a quick look at source code and I am not sure that we do support
> zip encoded payloads for datavalueset import at all :-(
>
> The way that it works auto-magically through the web ui is that we
> actually peek into the stream header to deduce whether the stream is
> zipped or gzipped.
>
> Have you tried using gzip to compress your file instead of zip?  Then
> it should (I think) be possible to have nginx handle the decompression
> for you.  ie.  try this:
>
> curl -H "Content-Type: application/json" -H "Content-Encoding: gzip"
> -u admin:district -d @datavalues.json.gz
> "https://play.dhis2.org/demo/api/dataValueSets.json"; -X POST
>
> On 28 July 2016 at 13:03, Victor Garcia <vgarciabnz@xxxxxxxxx> wrote:
> > This is the cURL command I try:
> >
> > curl -H "Content-Type: application/json" -u admin:district -d
> > @datavalues.zip "https://play.dhis2.org/demo/api/dataValueSets.json.zip";
> -X
> > POST
> >
> > The POST against DEMO server gives the same response that before. I tried
> > with ".json", ".zip" and ".json.zip" extensions, but no success. The
> > "datavalues.zip" only contains a valid .json file (sending this .json
> file
> > uncompressed, the import works).
> >
> > If you want me to try something else, just tell me.
> >
> >
> > On 28 July 2016 at 13:11, Halvdan Hoem Grelland <halvdan@xxxxxxxxx>
> wrote:
> >>
> >> Ah, my bad. I misread 'data' as 'meta data'.
> >>
> >> What is the exact request (including headers) you are using? Do you
> have a
> >> reproducible cURL we could try?
> >>
> >> Also, what is the format of the zipped data file? (The error you are
> >> seeing is the importer encountering a ZIP file header byte whilst
> expecting
> >> valid JSON).
> >>
> >> On Thu, Jul 28, 2016 at 12:10 PM, Victor Garcia <vgarciabnz@xxxxxxxxx>
> >> wrote:
> >>>
> >>> Thanks,
> >>>
> >>> Just to clarify, I have tried using extensions, but it does not seem to
> >>> work for "/api/dataValueSets" endpoint.
> >>>
> >>>
> >>> On 28 July 2016 at 12:04, Morten Olav Hansen <morten@xxxxxxxxx> wrote:
> >>>>
> >>>> Hm, yeah.. seems our data endpoint don't wrap it using
> >>>> StreamUtils.wrapAndCheckCompressionFormat(), so that means it must be
> >>>> manually applied.. should be an easy fix though
> >>>>
> >>>> --
> >>>> Morten Olav Hansen
> >>>> Senior Engineer, DHIS 2
> >>>> University of Oslo
> >>>> http://www.dhis2.org
> >>>>
> >>>> On Thu, Jul 28, 2016 at 12:00 PM, Halvdan Hoem Grelland
> >>>> <halvdan@xxxxxxxxx> wrote:
> >>>>>
> >>>>> Quickly glancing over the source code for this stuff: it seems to
> >>>>> discern formats using extensions. For example, one would use a POST
> to
> >>>>> api/metaData.json.zip or api/metaData.xml.gz.
> >>>>>
> >>>>> Could you give that a try, Victor?
> >>>>>
> >>>>> On Thu, Jul 28, 2016 at 11:23 AM, Morten Olav Hansen <
> morten@xxxxxxxxx>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi
> >>>>>>
> >>>>>> Did you try and simply send the data as a zipped file? I think we
> >>>>>> support both gzip and zip, you don't need to do anything else than
> zip it
> >>>>>> down before.
> >>>>>>
> >>>>>> I'm not 100% sure it will work, but this is how our metadata
> endpoints
> >>>>>> work.
> >>>>>>
> >>>>>> --
> >>>>>> Morten Olav Hansen
> >>>>>> Senior Engineer, DHIS 2
> >>>>>> University of Oslo
> >>>>>> http://www.dhis2.org
> >>>>>>
> >>>>>> On Thu, Jul 28, 2016 at 11:12 AM, Victor Garcia <
> vgarciabnz@xxxxxxxxx>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> searching in the documentation I couldn't find a way to import a
> data
> >>>>>>> file in zip format through the API. I know it is possible to do it
> using the
> >>>>>>> user interface (import/export module), but we need to use the API.
> Is it
> >>>>>>> supported?
> >>>>>>>
> >>>>>>> This problem arises in contexts with a poor connection, where large
> >>>>>>> data exchanges are a pain.
> >>>>>>>
> >>>>>>> Thank you,
> >>>>>>>
> >>>>>>> Víctor
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
> >>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Mailing list: https://launchpad.net/~dhis2-devs
> >>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Halvdan Hoem Grelland
> >>>>> Software developer, DHIS 2
> >>>>> University of Oslo
> >>>>> http://www.dhis2.org
> >>>>>
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> Halvdan Hoem Grelland
> >> Software developer, DHIS 2
> >> University of Oslo
> >> http://www.dhis2.org
> >>
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>

Follow ups

References