dhis2-devs team mailing list archive

[Lars Helge Øverland <larshelge@xxxxxxxxx>]

Hi Calle,

yes we will look into it.

The meta-data importer does match against existing objects, first on uid,
then code, then name (if unique).

This problem is specific to category option combos, since option combos are
generated based on category options/combos and not created explicitly. What
could happen is that you have identical categories on another system, but
the generated option combos get different uids. We need to make a special
case for the the category option combos and compare equality on category
options + combo and not just uid/code.

The same goes for org unit levels. What could happen is that you have
create a level on another system with a matching numeric level but another
domain. That way you can end up with two org unit levels with the same
numeric level when importing. We need to make a special case during import
to check not just uid/code/name but also level.

regards,

Lars








On Mon, Feb 23, 2015 at 12:38 PM, Calle Hedberg <calle.hedberg@xxxxxxxxx>
wrote:

> Lars,
>
> We had a nearly identical issue with one of our database on Friday -
> importing one new orgunit from another databaseinstance resulted in
> duplicates in the orgunitlevel table, which in turn caused analytics to
> fail.
>
> The key design issue (I would not call it a bug, even if it appears like
> that) here is the data and meta-data import functionality in DHIS2 does not
> assist users to avoid such duplication by
> - identify potential duplicates by checking e.g. name or level or UID
> - display the potential duplicates to the user to enable her/him to modify
> or drop the conflicting records.
> This type of import checks have been in DHIS 1.4 for a long time.
>
> This design issue does not only affect the orgunitlevel table - it affects
> much more profoundly the whole categoryoptioncombo and attributeoptioncombo
> system, where it is very easy to end up with multiple versions of in
> reality the same categoryoptioncombo name (but each with it's own internal
> id and uid). That in turn causes havoc with e.g. indicators, because the
> indicator engine seemingly randomly picks the first of those duplicates it
> find. This issue has required many days of cleaning up in the back end for
> at least two country systems I've worked on recently (Namibia and Lesotho).
>
> Any chance we could have a round-table discussion on how to re-design the
> import processes to handle this better?
>
> Regards
> Calle
>
> On 23 February 2015 at 11:01, Lars Helge Øverland <larshelge@xxxxxxxxx>
> wrote:
>
>> Hi Pierre,
>>
>> check the organisationunitlevel table and if the level column are
>> continuous and has no duplicates.
>>
>> select * from orgunitlevel order by level;
>>
>> regards,
>>
>> Lars
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19274
>
> Email: calle.hedberg@xxxxxxxxx
>
> Skype: calle_hedberg
>
> *******************************************
>
>