← Back to team overview

dhis2-devs team mailing list archive

Re: async metadata import in 2.27

 

Hi Vlad

Yes, this is on the list of todo ;) hopefully should have time this week
(or it will have to wait for next week)

-- 
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Tue, Aug 22, 2017 at 12:53 PM, Vladimer Shioshvili <vshioshvili@xxxxxxxxx
> wrote:

> Hi Morten,
>
> I did create a JIRA ticket (https://jira.dhis2.org/browse/DHIS2-2023) and
> have not heard back, i really don't think it's working as
> advertised/intended, unless I am the one doing something wrong with my
> calls:
>
>
> 1) I get the OU hierarchy from demo site:
>
> curl -u 'admin:district' -X GET 'https://play.dhis2.org/demo/
> api/26/metadata.xml?organisationUnits=true' > play_ou.xml
>
>
> 2) i use the same file to upload:
>
> curl -u 'admin:district' -H 'Content-Type:application/xml' -X POST '
> https://play.dhis2.org/demo/api/26/metadata?atomicMode=NONE&async=true'
> --data-binary @play_ou.xml -v
>
> it takes about 45 seconds to come back with this:
>
>
> *   Trying 52.30.174.183...
>
> * Connected to play.dhis2.org (52.30.174.183) port 443 (#0)
>
> * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_RC4_128_SHA
>
> * Server certificate: play.dhis2.org
>
> * Server certificate: RapidSSL SHA256 CA - G3
>
> * Server certificate: GeoTrust Global CA
>
> * Server auth using Basic with user 'admin'
>
> > POST /demo/api/26/metadata?atomicMode=NONE&async=true HTTP/1.1
>
> > Host: play.dhis2.org
>
> > Authorization: Basic YWRtaW46ZGlzdHJpY3Q=
>
> > User-Agent: curl/7.43.0
>
> > Accept: */*
>
> > Content-Type:application/xml
>
> > Content-Length: 1568899
>
> > Expect: 100-continue
>
> >
>
> < HTTP/1.1 100 Continue
>
> * We are completely uploaded and fine
>
> < HTTP/1.1 204 No Content
>
> < Server: nginx/1.4.6 (Ubuntu)
>
> < Date: Tue, 22 Aug 2017 10:46:17 GMT
>
> < Connection: keep-alive
>
> < X-XSS-Protection: 1; mode=block
>
> < X-Frame-Options: SAMEORIGIN
>
> < X-Content-Type-Options: nosniff
>
> < Set-Cookie: JSESSIONID=3A6F3850230C09C81BDB67310398FA68; Path=/demo/;
> HttpOnly
>
> <
>
> * Connection #0 to host play.dhis2.org left intact
>
>
> After I get the response back, when i do the status: curl -u
> 'admin:district' -X GET https://play.dhis2.org/demo/
> api/26/system/tasks/METADATA_IMPORT
>
> I get this back:
>
> [{"uid":"lqOCBbHOTew","level":"INFO","category":"METADATA_
> IMPORT","time":"2017-08-22T10:46:17.866","message":"(admin) Import:Done
> took 43.66 seconds","completed":true},{"uid":"y2dBsPZrwMX","level":"
> INFO","category":"METADATA_IMPORT","time":"2017-08-22T10:45:34.428","message":"(admin)
> Updating 1332 object(s) of type OrganisationUnit","completed":
> false},{"uid":"GuGaKcJAToU","level":"INFO","category":"
> METADATA_IMPORT","time":"2017-08-22T10:45:34.209","message":"(admin)
> Import:Start","completed":false}]
>
>
> It takes about 43 seconds to import the file - POST does not come back
> until import is actually done. Documentation has this:
>
>
> async false, true Asynchronous import, returns immediately with a
> *Location* header pointing to the location of the *importReport*.
> And there is no location returned with the 204 (which is not the right
> HTTP code anyway).
>
> I am also concerned about the time - when doing this with some PEPFAR
> countries, hierarchy with 14 thousand OUs took 8 hours to import.
>
> Thanks,
> Vlad
>
> On Sun, Aug 6, 2017 at 1:44 PM Morten Olav Hansen <morten@xxxxxxxxx>
> wrote:
>
>> Hi
>>
>> It should work like that. Would you mind writing a JIRA issue?
>> jira.dhis2.org
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Aug 3, 2017 at 12:03 PM, Vladimer Shioshvili <
>> vshioshvili@xxxxxxxxx> wrote:
>>
>>> Hi everyone,
>>>
>>> I am trying to import xml metadata (org units) into 2.27, and async=true
>>> query parameter does not seem to make a difference - and it still goes in
>>> synchronously. It is in the documentation for 2.27 (and 2.26 and 2.25 -
>>> https://docs.dhis2.org/2.26/en/developer/html/dhis2_
>>> developer_manual_full.html#webapi_metadata_import), and I am using api
>>> version 26. Is it something i am doing wrong?
>>>
>>> This is the curl command i am using:
>>>
>>> curl -X POST 'https://domain/api/26/metadata?atomicMode=NONE&async=true'
>>> --data-binary @file.xml -H 'Content-Type:application/xml'
>>>
>>>
>>> Thanks!
>>>
>>> Vlad
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>

References