dhis2-devs team mailing list archive

[Lorill Crees <lcrees@xxxxxxxxxx>]

Hi Morten,

I'm just following up on the two items below (inline).

Thanks!

Lorill


On Fri, Sep 16, 2016 at 8:24 AM, Lorill Crees <lcrees@xxxxxxxxxx> wrote:

> Thanks for getting back to me so quickly Morten! I've answered inline
> below.
>
> On Thu, Sep 15, 2016 at 10:19 PM, Morten Olav Hansen <morten@xxxxxxxxx>
> wrote:
>
>    1. When submitting an identical post containing
>    {"programTrackedEntityAttributes" [...], "program" [...]}, where there
>    is an error in the Program being posted, the call to /api/metadata responds
>    with success for "programTrackedEntityAttributes" with the correct
>    "importCount" counts, but the call to /api/23/metadata responds for
>    "programTrackedEntityAttributes" with "stats" counts containing all
>    zeros, and no "objectReports". It is like the first item being posted is
>    completely ignored if the second item posted has errors. Is this the
>    intended behaviour? And should there be "objectReports" regardless? Please
>    see attached examples.
>
> Yes, what kind of error are you getting? a reference error, or a
>>>> validation error? in the old importer, it would always try to just import
>>>> what it could.. which was not always ideal (missing references etc was just
>>>> ignored). If it's just reference error, and you want to ignore them..
>>>> please try with `atomicMode=NONE`, regarding the object report..
>>>>
>>>
>>> We need to write our response parser to be able to parse any kind of
>>> error that may happen. Our program ingests CSV files from the user and we
>>> attempt to do as much error checking in advance as possible, but there may
>>> be other errors that happen when posting to the API such as validation
>>> errors, reference errors, or even 500 errors if there is an error within
>>> DHIS 2 itself. Given that we need to cater to every possible scenario, I'm
>>> not sure if we should be using "atomicMode=NONE" or not. Also, should there
>>> be "objectReports" regardless?
>>>
>>
>> I'm not 100% sure what is the best approach here, I think maybe we want
>> to add some new parameters to control the output a bit. If you are posting
>> 10 new data elements, it is very useful to get the object report back as
>> they will contain the new UID of the object, on the flipside.. if you are
>> importing 50000 objects.. the payload sent back might just too much in most
>> cases, and not very useful (in that case, probably you are more interested
>> in errors)
>>
>
>
1. Adding new parameters to control the output is a great idea, is this
something that you plan to implement as part of the blueprint you mention
below?
2. I'm also wondering if it would make more sense to mark the ignored
objects as "ignored" in the stats, instead of having zeros for all the
stats? I've re-attached our output, and you can see that I posted 4
programTrackedEntityAttributes, and the stats show 0 total / 0 ignored
which is not entirely accurate. I would have expected 4 total / 4 ignored.
Thoughts?

>
>>
>
>>
>>>>>    1. The /api/23/metadata response with "errorReports" within
>>>>>    "objectReports" appears to have replaced the /api/metadata
>>>>>    "importConflicts". In /api/metadata, the "importConflicts" "object" field
>>>>>    contains the actual name of the object that has the error (eg: the Program
>>>>>    short name in our case). However, in /api/23/metadata there is only the
>>>>>    index which makes it more work to debug. Is it possible to also include the
>>>>>    name of the object in the error report like the prior API version does?
>>>>>
>>>>> Hm, right. I understand that it's useful, but that can easily add up
>>>> to a lot of data returned. Maybe we should add some kind of "debug" mode
>>>> where more output is provided.
>>>>
>>>
>>> Although I understand that this might add more data to the payload, it
>>> would decrease the amount of data that we need to pass around in our app,
>>> and also potentially decrease the number of API calls we need to do. Not
>>> having this information in the response now means we need to pass around
>>> our original post data (which can be quite large, and which we aren't
>>> currently doing) and do a lookup by ID to send any kind of useful
>>> information back to the user. Additionally, we may have even posted these
>>> objects by ID only, which means we will now need to do another API call to
>>> look up the object to get useful information to send back to the user. Keep
>>> in mind that in our app, the user doesn't actually even have a knowledge of
>>> IDs - we are doing those lookups for them by name or code.
>>> Would it be possible to add a flag to the api call where we can request
>>> the name or shortname or code to be returned in addition to just the ID?
>>> Perhaps similar to the metadata field filter?  Where we could say supply
>>> importConflictsFields=:identifiable?
>>>
>>
>> Yes, let me see what I can do for 225 at least (I might be able to
>> backport to 224 if it doesn't disrupt things too much)
>>
>> https://blueprints.launchpad.net/dhis2/+spec/importer-parame
>> ter-to-adjust-importreport-output
>>
>
> Thanks Morten, that's great!
>
>>
>>
I see this blueprint was completed in 2.25 - great! Was this backported to
2.24? Is there documentation for this?

-- 
Lorill Crees
Project Leader / Senior Developer
2Paths Solutions Ltd. <http://www.2paths.com>

lcrees@xxxxxxxxxx
skype: lorill2paths
(604) 689-4123 x 15