| Thread Previous • Date Previous • Date Next • Thread Next |
Thanks for getting back to me so quickly Morten! I've answered inline below.
On Thu, Sep 15, 2016 at 10:19 PM, Morten Olav Hansen <morten@xxxxxxxxx>
wrote:
1. When submitting an identical post containing
{"programTrackedEntityAttributes" [...], "program" [...]}, where there
is an error in the Program being posted, the call to /api/metadata responds
with success for "programTrackedEntityAttributes" with the correct
"importCount" counts, but the call to /api/23/metadata responds for
"programTrackedEntityAttributes" with "stats" counts containing all
zeros, and no "objectReports". It is like the first item being posted is
completely ignored if the second item posted has errors. Is this the
intended behaviour? And should there be "objectReports" regardless? Please
see attached examples.
Yes, what kind of error are you getting? a reference error, or a validation
>>> error? in the old importer, it would always try to just import what it
>>> could.. which was not always ideal (missing references etc was just
>>> ignored). If it's just reference error, and you want to ignore them..
>>> please try with `atomicMode=NONE`, regarding the object report..
>>>
>>
>> We need to write our response parser to be able to parse any kind of
>> error that may happen. Our program ingests CSV files from the user and we
>> attempt to do as much error checking in advance as possible, but there may
>> be other errors that happen when posting to the API such as validation
>> errors, reference errors, or even 500 errors if there is an error within
>> DHIS 2 itself. Given that we need to cater to every possible scenario, I'm
>> not sure if we should be using "atomicMode=NONE" or not. Also, should there
>> be "objectReports" regardless?
>>
>
> I'm not 100% sure what is the best approach here, I think maybe we want to
> add some new parameters to control the output a bit. If you are posting 10
> new data elements, it is very useful to get the object report back as they
> will contain the new UID of the object, on the flipside.. if you are
> importing 50000 objects.. the payload sent back might just too much in most
> cases, and not very useful (in that case, probably you are more interested
> in errors)
>
1. Adding new parameters to control the output is a great idea, is this
something that you plan to implement as part of the blueprint you mention
below?
2. I'm also wondering if it would make more sense to mark the ignored
objects as "ignored" in the stats, instead of having zeros for all the
stats? I've re-attached our output, and you can see that I posted
4 programTrackedEntityAttributes, and the stats show 0 total / 0 ignored
which is not entirely accurate. I would have expected 4 total / 4 ignored.
Thoughts?
>
>
>>
>>>> 1. There is no longer a "lastImported" field. Is this now
>>>> deprecated?
>>>>
>>>> There should now be a id exported in the object report.. I think there
>>> might be some inconsistencies there though.. I will go through it, in
>>> general this field was mostly used for our /api/type/id endpoints.. were
>>> you also using it for /api/metadata? (it's not very useful unless you are
>>> only sending 1 single object)
>>>
>>
>> Agreed that using "lastImported" for /api/metadata is not very useful
>> unless sending one single object. However, we are using our generic
>> response parser for all DHIS 2 API endpoints and are trying to avoid the
>> need for special case logic dependent upon which specific endpoint was
>> called. Additionally, we do still use this field for many of the API calls
>> where we do send one single object. Is the plan to deprecate the
>> "lastImported" field across the API, or only in the /api/metadata call, so
>> we can adjust our parser accordingly?
>>
>
> The lastImported field has been removed all over, what is happening now,
> is that if you POST a new data element to /api/dataElements, you will get a
> single object report back.. and the uid is contained within there
>
Thanks.
>
>>>> 1. The /api/23/metadata response with "errorReports" within
>>>> "objectReports" appears to have replaced the /api/metadata
>>>> "importConflicts". In /api/metadata, the "importConflicts" "object" field
>>>> contains the actual name of the object that has the error (eg: the Program
>>>> short name in our case). However, in /api/23/metadata there is only the
>>>> index which makes it more work to debug. Is it possible to also include the
>>>> name of the object in the error report like the prior API version does?
>>>>
>>>> Hm, right. I understand that it's useful, but that can easily add up to
>>> a lot of data returned. Maybe we should add some kind of "debug" mode where
>>> more output is provided.
>>>
>>
>> Although I understand that this might add more data to the payload, it
>> would decrease the amount of data that we need to pass around in our app,
>> and also potentially decrease the number of API calls we need to do. Not
>> having this information in the response now means we need to pass around
>> our original post data (which can be quite large, and which we aren't
>> currently doing) and do a lookup by ID to send any kind of useful
>> information back to the user. Additionally, we may have even posted these
>> objects by ID only, which means we will now need to do another API call to
>> look up the object to get useful information to send back to the user. Keep
>> in mind that in our app, the user doesn't actually even have a knowledge of
>> IDs - we are doing those lookups for them by name or code.
>> Would it be possible to add a flag to the api call where we can request
>> the name or shortname or code to be returned in addition to just the ID?
>> Perhaps similar to the metadata field filter? Where we could say supply
>> importConflictsFields=:identifiable?
>>
>
> Yes, let me see what I can do for 225 at least (I might be able to
> backport to 224 if it doesn't disrupt things too much)
>
> https://blueprints.launchpad.net/dhis2/+spec/importer-parameter-to-adjust-
> importreport-output
>
Thanks Morten, that's great!
>
>
>
>>
>>
>>>
>>> --
>>> Morten
>>>
>>>
>>>> Thanks,
>>>>
>>>> Lorill
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>>
>> --
>> Lorill Crees
>> Project Leader / Senior Developer
>> 2Paths Solutions Ltd. <http://www.2paths.com>
>>
>> lcrees@xxxxxxxxxx
>> skype: lorill2paths
>> (604) 689-4123 x 15
>>
>
>
--
Lorill Crees
Project Leader / Senior Developer
2Paths Solutions Ltd. <http://www.2paths.com>
lcrees@xxxxxxxxxx
skype: lorill2paths
(604) 689-4123 x 15
Attachment:
metadata23PostResponse.json
Description: application/json
Attachment:
metadataDefaultPostResponse.json
Description: application/json
Attachment:
metadataPostBody.json
Description: application/json
| Thread Previous • Date Previous • Date Next • Thread Next |
