dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Morten,

I was talking about meta-data, and I'm not suggesting a roll-back feature -
I'm suggesting a more step-wise import process where the user can see and
control what is being imported/updated:
- what meta-data appear as new (it might not actually be new meta-data, but
rather just appear as new due to minor discrepancies in UIDs, names or
codes)
- what meta-data appear as updates (some might be wanted, some not).

I'm frequently addressing these challenges by comparing external meta-data
to DHIS2 meta-data using either PostgreSQL views or ODBC connections via MS
Access, but that's really cumbersome. Still better than importing what
turns out to be duplicates etc, and which later has to be cleaned up
manually.

A more step-wise process will also have major advantages over the current
"import summary" in the sense that you can see exactly WHAT fields are
updated (and presumably also filter out updates where e.g. the LastUpdated
value is older).

Regards
Calle

On 2 February 2016 at 10:29, Morten Olav Hansen <mortenoh@xxxxxxxxx> wrote:

> Hi Calle
>
> Yeah, I think its obvious from the quick emails we are getting that we
> should not remove it. I would be great to have a proper rollback feature,
> but right now this won't be that easy to add.
>
> (I am talking about metadata only though, you seem to also mention
> aggregate/event data? those importers are not scheduled to be rewritten,
> and will behave as today)
>
> --
> Morten
>
> On Tue, Feb 2, 2016 at 3:26 PM, Calle Hedberg <calle.hedberg@xxxxxxxxx>
> wrote:
>
>> Morten,
>>
>> Dropping the dry run is OK if, and I stress if, the import process itself
>> becomes more transparent and step-wise. Basically this means
>> 1.
>> Display new records coming in, with the ability to remove all or some
>> from import
>> 2.
>> Display updated records coming in and the equivalent existing records,
>> with the ability to remove all or some from import/updating.
>>
>> The fundamental problem with DHIS2 imports are that they are "black
>> boxes", you cannot see what's happening until after it's done. Without even
>> the dry run, your only "Safety net" is to back up the whole database just
>> before importing - which is tricky to do in a production environment when
>> manager A imports data while user B,C,D, and E are capturing data directly.
>>
>> Regards
>> Calle
>>
>> On 2 February 2016 at 10:15, Morten Olav Hansen <mortenoh@xxxxxxxxx>
>> wrote:
>>
>>> Hi everyone
>>>
>>> I'm currently in the process of re-writing our DXF2 metadata importer,
>>> and was wondering if anyone is actually using the dryRun option? its adds
>>> quite a bit of complexity to the code, so it will be added only to the new
>>> importer if people are actually using it for imports
>>>
>>> --
>>> Morten
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>>
>> *******************************************
>>
>> Calle Hedberg
>>
>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>
>> Tel/fax (home): +27-21-685-6472
>>
>> Cell: +27-82-853-5352
>>
>> Iridium SatPhone: +8816-315-19119
>>
>> Email: calle.hedberg@xxxxxxxxx
>>
>> Skype: calle_hedberg
>>
>> *******************************************
>>
>>
>


-- 

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************