dhis2-devs team mailing list archive

[Uwe Wahser <uwe@xxxxxxxxx>]

Hi Pramil & Lars,

actually I think that Pramil is adressing a very valid problem. In my current
understanding of DHIS2 I wouldn't be able to propose any better solution than
using Lars' analysis api and an external tool (php, r, kettle etc) to read the
results and fill them into the datasets.

However, the use-case he describes is the very typical data warehousing
scenario, where you have different layers of granularity and maturity of data
(e.g. staging area, operational data stores, data marts), especially in an
infrastructure where you are mainly extracting data from source systems. This
would require DHIS2 to be able to push data internally (from tracker to
datasets, but also within tracker or from dataset to dataset), while at the same
time being able to transform these data (e.g. as it was possible with the
aggregation queries). 

This has multiple semantic advantages as you can model your data from raw-data
to architected data marts. But also technical aspects as you don't have to
rebuild your datasets from the source systems, whenever you change their
definition, but can rebuild them from the granular level instead. Leave alone
the performance issues I'd normally expect when having to include huge numbers
of granular data from tracker into DHIS2 analytics. (btw. do you have any
reference data on the that?) Of course this would call for nightly builds, but
that's normal data warehousing business.

I am aware of the discussion last year (e.g.
https://lists.launchpad.net/dhis2-users/msg09609.html) but would like to
encourage a higher priority of the replacement, if there is still the intention
of positioning DHIS2 as a tool for data warehousing.

I also remember slightly that Knut tried to trigger off a discussion on this
(maybe around 2015-12?), but can't find the thread on that - are there any DEV
internal news on the status of that discussion and the future plans of
aggregation within DHIS2?

Best regards, 

Uwe

---
> Pramil Liyanage <pramilcl@xxxxxxxxx> hat am 9. Mai 2016 um 09:00 geschrieben:
> 
> 
> Any help on this?
> 
> Regards
> Pramil
> 
> On Sat, May 7, 2016 at 8:53 PM, Pramil Liyanage <pramilcl@xxxxxxxxx> wrote:
> 
> > Thanks Lars. Let me summarize my issue. We have been using aggregated
> > reporting for TB for a few years and recently introduced individual records
> > and aggregated using query builder to the same aggregated dataset. This was
> > customized to the exact national paper report and report was generated from
> > dataset report for printing.
> >
> > Now since agg. queries are not there we will not be able to get data from
> > tracker into the same data element. If we shift to a new report using
> > program indicators we will not be able to use the previous aggregate data
> > in future analysis and comparison.
> >
> > To make the matter worse, there was a lot of disaggregation of data
> > elements using categories, where we wrote queries for each one.
> >
> > We are stuck at a dead end at 2.22! Please suggest the best solution.
> >
> > Your suggestion to have program indicators in the custom dataset is a good
> > idea for future. But my issue is get them into existing elements if I
> > upgrade to 2.23, and not a few, about 700 in all reports.
> >
> > Regards
> > Pramil
> >
> >
> >
> >
> >
> > On Fri, May 6, 2016 at 6:55 PM, Pamod Amarakoon <pamodm@xxxxxxxxx> wrote:
> >
> >> Dear Lars,
> >>
> >> Making aggregated event data available on the fly in data set report
> >> would certainly be a useful tool especially when maintaining monitoring and
> >> evaluation forms for tracker activities.
> >>
> >> Thank you
> >>
> >> On Fri, May 6, 2016 at 6:35 PM, Lars Helge Øverland <lars@xxxxxxxxx>
> >> wrote:
> >>
> >>> Hi Pramil,
> >>>
> >>> you can retrieve aggregated event data from the analytics api.
> >>>
> >>> http://dhis2.github.io/dhis2-docs/master/en/developer/html/ch01s32.html
> >>>
> >>>
> >>> Excerpt from the docs:
> >>>
> >>> To query for program data elements (of tracker domain type) you can get
> >>> those by specifying the program for each data element using the
> >>> <program-id>.<dataelement-id> syntax:
> >>>
> >>> /api/analytics.json?dimension=dx:eBAyeGv0exc.qrur9Dvnyt5;eBAyeGv0exc.GieVkTxp4HH
> >>> &dimension=pe:LAST_12_MONTHS&filter=ou:ImspTQPwCqd
> >>>
> >>> One work-around could be to write a bit of Javascript and include that
> >>> in the custom form to retrieve and display event based aggregated data
> >>> inside the data entry form, if that is what you are looking to do.
> >>>
> >>>
> >>> I have been pondering for a while on adding a feature where you could
> >>> specify tracker-type data elements in custom data entry forms, and have
> >>> the
> >>> system automatically display aggregated event data, both in forms and in
> >>> data set report. Let me know if that could work.
> >>>
> >>>
> >>> regards,
> >>>
> >>> Lars
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, May 5, 2016 at 11:03 AM, Pramil Liyanage <pramilcl@xxxxxxxxx>
> >>> wrote:
> >>>
> >>>> Hi everyone,
> >>>>
> >>>> Since the aggregation queries are not there anymore is it possible to
> >>>> get program indicator value into an existing Aggregate Dataset through
> >>>> data
> >>>> entry screen design?
> >>>>
> >>>> Or any other alternative workaround?
> >>>>
> >>>> Regards
> >>>> Pramil
> >>>>
> >>>> _______________________________________________
> >>>> Mailing list: https://launchpad.net/~dhis2-users
> >>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> >>>> Unsubscribe : https://launchpad.net/~dhis2-users
> >>>> More help   : https://help.launchpad.net/ListHelp
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Lars Helge Øverland
> >>> Lead developer, DHIS 2
> >>> University of Oslo
> >>> Skype: larshelgeoverland
> >>> lars@xxxxxxxxx
> >>> http://www.dhis2.org <https://www.dhis2.org/>
> >>>
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~dhis2-devs
> >>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>> More help   : https://help.launchpad.net/ListHelp
> >>>
> >>>
> >>
> >>
> >> --
> >> Regards,
> >> Pamod Amarakoon
> >>
> >> Confidentiality Notice: the information contained in this email and any
> >> attachments may be legally privileged and confidential. If you are not an
> >> intended recipient, you are hereby notified that any dissemination,
> >> distribution, or copying of this e-mail is strictly prohibited. If you have
> >> received this e-mail in error, please notify the sender and permanently
> >> delete the e-mail and any attachments immediately. You should not retain,
> >> copy or use this e-mail or any attachments for any purpose, nor disclose
> >> all or any part of the contents to any other person.
> >>
> >>
> >>
> >>
> >
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-users
> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help   : https://help.launchpad.net/ListHelp