dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Sorry, one more thing:

A more fundamental challenge relates to the whole cat/att-optioncombo
DESIGN for any particular database - DHIS2 enables anything with very
little guidance, and the net result can be very messy configurations with
multiple permutations of what is basically the same thing, like
(age,gender,NEW) and (gender,NEW,age) and (NEW,age, gender) combos for data
broken down on gender, age, and NEW/REPEAT (just to use a very
practical/actual example from Lesotho).

What that reflect is mainly lack of guidance materials (materials,
training) and lack of checks when creating new combos.

Will get back to it, for sure....

Regards
calle

On 23 February 2015 at 14:52, Calle Hedberg <calle.hedberg@xxxxxxxxx> wrote:

> In short, YES.
>
> There is clearly still a need for a more sophisticated type of import,
> though - where the user can SEE what is identified for import and if
> required modify or drop specific things....
>
> Regards
> Calle
>
> On 23 February 2015 at 14:31, Lars Helge Øverland <larshelge@xxxxxxxxx>
> wrote:
>
>>
>> Hi Calle,
>>
>> yes we will look into it.
>>
>> The meta-data importer does match against existing objects, first on uid,
>> then code, then name (if unique).
>>
>> This problem is specific to category option combos, since option combos
>> are generated based on category options/combos and not created explicitly.
>> What could happen is that you have identical categories on another system,
>> but the generated option combos get different uids. We need to make a
>> special case for the the category option combos and compare equality on
>> category options + combo and not just uid/code.
>>
>> The same goes for org unit levels. What could happen is that you have
>> create a level on another system with a matching numeric level but another
>> domain. That way you can end up with two org unit levels with the same
>> numeric level when importing. We need to make a special case during import
>> to check not just uid/code/name but also level.
>>
>> regards,
>>
>> Lars
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Feb 23, 2015 at 12:38 PM, Calle Hedberg <calle.hedberg@xxxxxxxxx>
>> wrote:
>>
>>> Lars,
>>>
>>> We had a nearly identical issue with one of our database on Friday -
>>> importing one new orgunit from another databaseinstance resulted in
>>> duplicates in the orgunitlevel table, which in turn caused analytics to
>>> fail.
>>>
>>> The key design issue (I would not call it a bug, even if it appears like
>>> that) here is the data and meta-data import functionality in DHIS2 does not
>>> assist users to avoid such duplication by
>>> - identify potential duplicates by checking e.g. name or level or UID
>>> - display the potential duplicates to the user to enable her/him to
>>> modify or drop the conflicting records.
>>> This type of import checks have been in DHIS 1.4 for a long time.
>>>
>>> This design issue does not only affect the orgunitlevel table - it
>>> affects much more profoundly the whole categoryoptioncombo and
>>> attributeoptioncombo system, where it is very easy to end up with multiple
>>> versions of in reality the same categoryoptioncombo name (but each with
>>> it's own internal id and uid). That in turn causes havoc with e.g.
>>> indicators, because the indicator engine seemingly randomly picks the first
>>> of those duplicates it find. This issue has required many days of cleaning
>>> up in the back end for at least two country systems I've worked on recently
>>> (Namibia and Lesotho).
>>>
>>> Any chance we could have a round-table discussion on how to re-design
>>> the import processes to handle this better?
>>>
>>> Regards
>>> Calle
>>>
>>> On 23 February 2015 at 11:01, Lars Helge Øverland <larshelge@xxxxxxxxx>
>>> wrote:
>>>
>>>> Hi Pierre,
>>>>
>>>> check the organisationunitlevel table and if the level column are
>>>> continuous and has no duplicates.
>>>>
>>>> select * from orgunitlevel order by level;
>>>>
>>>> regards,
>>>>
>>>> Lars
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> *******************************************
>>>
>>> Calle Hedberg
>>>
>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>
>>> Tel/fax (home): +27-21-685-6472
>>>
>>> Cell: +27-82-853-5352
>>>
>>> Iridium SatPhone: +8816-315-19274
>>>
>>> Email: calle.hedberg@xxxxxxxxx
>>>
>>> Skype: calle_hedberg
>>>
>>> *******************************************
>>>
>>>
>>
>
>
> --
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19274
>
> Email: calle.hedberg@xxxxxxxxx
>
> Skype: calle_hedberg
>
> *******************************************
>
>


-- 

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19274

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************