dhis2-devs team mailing list archive

[Uwe Wahser <uwe@xxxxxxxxx>]

Dear Julhas,

we are doing something like that (on the aggregate level) in a datawarehouse setting  without 0-values, where we have to update whole datasets every night. 

Basically we download all data with kettle, compare the differences to the incoming data, and then push only changed values back to DHIS2. Those values (category/attribute combinations) which are no longer needed, we push as delete requests.

That sounds very complicated (and might need a whole lot of memory on the etl side depending on the number of values you are dealing with) but has the advantage that you reduce the number of records to be pushed into DHIS2 to those that have really changed (and you keep the audit-logs as Lars suggested)

Hope that helps,

Uwe

> Hannan Khan <hannank@xxxxxxxxx> hat am 27. Februar 2017 um 18:56 geschrieben:
> 
> 
> We are not allowing '0' in the data until and unless it is essential. In
> your case '0' will be you significant amount; so we cant allow that.
> 
> On Mon, Feb 27, 2017 at 9:25 PM, Julhas Sujan <julhaspustcse@xxxxxxxxx>
> wrote:
> 
> > Hi Lars,
> >
> > Thank you very much. Yes this is possible to set zero for each field. I
> > will follow your recommendation and share with you the update.
> >
> >
> > *Best Regards*
> >
> > *Julhas *
> >
> > On Mon, Feb 27, 2017 at 8:59 PM, Lars Helge Øverland <lars@xxxxxxxxx>
> > wrote:
> >
> >> Hi Julhas,
> >>
> >> that's an interesting problem. We could certainly implement support for
> >> deletion per org unit/period.
> >>
> >> I would however recommend another approach: For each aggregate report you
> >> send from eTBM, make sure you send a value, including 0, for* each field*
> >> in the aggregate TB form. This will make your information more explicit,
> >> ie. you communicate that for all data element + category option combos
> >> exactly how many cases you have. The benefit then is that if cases are
> >> deleted in the upstream system (eTBM), it will be correctly reflected in
> >> the aggregate form in DHIS 2.
> >>
> >> This will be semantically more correct than simply dropping all data
> >> values first. I.e. when you look at the audit logs for each data element,
> >> you will see that a value changed from a positive value, then potentially
> >> to a 0 value. This will look better compared to audit logs showing that all
> >> values were deleted every day. This will improve auditability and data
> >> validation.
> >>
> >> Of course this might lead to the aggregate export payload becoming a bit
> >> bigger, but I think that can be justified. You can also make sure that the
> >> TB data elements are set to "zero not significant", which will delete or
> >> not store 0 values on the DHIS 2 side.
> >>
> >> Let me know if this sounds reasonable.
> >>
> >> best,
> >>
> >> Lars
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Feb 27, 2017 at 12:43 PM, Julhas Sujan <julhaspustcse@xxxxxxxxx>
> >> wrote:
> >>
> >>> Dear All,
> >>>
> >>> Good evening!
> >>>
> >>> We are working on DHIS2 TB-10,11 and 12 forms (for Bangladesh) to
> >>> integrate with external eTB manager application
> >>> <http://etbmanagerbd.org/>. The TB forms have developed in DHIS2 for
> >>> collecting TB -10,11 and 12 data from Upazila level in Bangladesh but we
> >>> have a local level application(eTBM) that is using to collect individual
> >>> case record and generate aggregated TB-10,11 and 12 reports. Now  our goal
> >>> is to integrate eTMB with DHIS2 TB forms.
> >>>
> >>> Already we have tested the integration that is working fine. But we are
> >>> facing a problem with deleting existing records of these forms. I have a
> >>> question on it:
> >>>
> >>> *Is there any way of requesting DHIS2 to clear or delete the whole TB 10
> >>> form for a specific orgunit and period before sending the external data?*
> >>>
> >>>
> >>> Thank you very much for your co-operations.
> >>>
> >>>
> >>> *If you have any questions or concerns, don’t hesitate to let me know.*
> >>>
> >>> *Best Regards*
> >>> *Julhas Sujan*
> >>> Senior Java Programmer
> >>> Management Science for Health(MSH)
> >>> Dhaka, Bangladesh
> >>> Cell: +8801989 442856
> >>> Skype: julhas08
> >>> ​
> >>> www.msh.org http://siapsprogram.org/wherewework/bangladesh/
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~dhis2-users
> >>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~dhis2-users
> >>> More help   : https://help.launchpad.net/ListHelp
> >>>
> >>>
> >>
> >>
> >> --
> >> Lars Helge Øverland
> >> Lead developer, DHIS 2
> >> University of Oslo
> >> Skype: larshelgeoverland
> >> lars@xxxxxxxxx
> >> http://www.dhis2.org <https://www.dhis2.org/>
> >>
> >>
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-users
> > Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-users
> > More help   : https://help.launchpad.net/ListHelp
> >
> >
> 
> 
> -- 
> Muhammad Abdul Hannan Khan
> DHIS2 Country coordinator & Secretary
> HISP Bangladesh
> 
> T +880-2- 8816459, 8816412 ext 118
> F +88 02 8813 875
> M+88 01819 239 241
> M+88 01534 312 066
> E hannank@xxxxxxxxx
> S hannan.khan.dhaka
> B hannan-tech.blogspot.com
> L https://bd.linkedin.com/in/hannankhan
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-users
> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help   : https://help.launchpad.net/ListHelp