dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

Hi Ola, Kim

On 20 May 2010 10:21, Ola Hodne Titlestad <olatitle@xxxxxxxxx> wrote:
> Hi Kim Anh,
>
> I am not sure I fully understand your problem.
> Maybe you could show us a few examples of data sets, data elements (English
> translations) and different period types to illustrate your problem?
> Thanks.
>
> Note that saving values with different period types for the same data
> element will easily become messy and end in duplicate and overlapping data
> that the system will not mange well.
> So this constraint is important in keeping any DHIS database consistent and
> should not be violated. In fact there should be some validation of this in
> data entry and data import as well, and not just in the manually run
> integrity checks.

I think Kim's point is that there is no such constraint.  DataValues
with the same Dataelement can indeed be of different period types and
I'm assuming this was by design.  We'd have to trawl through some data
to see how often this is actually the case.

So when importing datavalues from "somewhere" those datavalues will
have a period and periodtype, but in order to capture that in dhis
they would first have to be members of a dataset.  And note that is
for the sole purpose of attaching a periodtype.  So I can understand
the reluctance.  I've had similar headaches with datasets and sdmx
which I haven't quite solved either.

To make matters slightly worse, in order to find the period type of a
data value you have to infer the dataset first by looking at the
dataelement and the orgunit in combination.  So for a datavalue:

dataset = f(orgunit, dataelement); and
PeriodType = g(dataset);

So its a slightly untidy arrangement which I guess has evolved over time.

Possible resolutions:
Once we have a form object (which I believe is coming) much of the
rationale for the dataset (collecting dataelements which match
collection instruments) might fall away.  In which case we'd have a
few possibilities:
(i)  make the period type a first-class property of the dataelement -
might or might not be possible (see data trawling above); or
(ii) make the period type an attribute of the datavalue (which makes
the datavalue storage bigger); or
(iii) make the period type an implicit category of the categorycombo.

Note that (ii) offers the maximum flexibility and (i) and (iii) are
logically very similar.  I think I would go for (i) as the best
compromise.

In the short term what Ola suggests is probably the best approach.
Even though its not a hard constraint, it is worth trying to treat
dataelements as if they have a fixed period type - and creating
multiple dataelements for multiple periodtypes.

Regards
Bob

>
> In your case it sounds like you need multiple data elements with different
> period types. The use calculated data elements or indicators, or some custom
> aggregated reports to combine the data with different period types for data
> analysis.
>
> The best solution is the non-technical one, to enforce the same data
> collection frequency for all orgunits, and that way achieve a more
> consistent data warehouse. I understand that this might be difficult to
> achieve though.
>
> Note that the reporting frequency (data export up the hierarchy or report
> generation) does not have to be the same as the data entry frequency. The
> important point is to make sure data is collected for the same period type
> across units at the lowest level.
>
> Ola
> ---------------
>
>
>
> On 20 May 2010 11:02, Kim-Anh Vo <catakim@xxxxxxxxx> wrote:
>>
>> Hi all,
>>
>> The subject I have given out: "Dataset vs. import/export vs. dael-dataset
>> integrity checking" appears when I am working on building an
>> integrated-health database.
>>
>> The points for the above objects:
>> 1. Dataset (or discussed as dataelementSet): is a list of dataelements for
>> entry, import/export, etc.
>> 2. Import/Export datavalue (or maybe called dataelementValue) is based on
>> dataset, it means that dataelement value is only exported/imported by being
>> included in at least one dataset.
>> 3. Dataset-integrity checking to find the violate: Data elements assigned
>> to data sets with different period types
>>
>> See the above under a data warehouse (or just a central data repository)
>> situation: all daels, datavalues, etc. from local databases would be
>> imported into that sys.
>>
>> Back to the objects vs. DHIS2, there are some issues:
>>
>> iss1) Data (dataelementvalue) imported from many sources from diff local
>> places (nations, provinces, dis., etc.) SO a unified dataset with a unified
>> period for all is unlikely possible. As a result of this," dael-dataset
>> integrity checking" criterion is hard to be fulfilled
>>
>> iss2) Data import/export all the time goes with datasets so everytime
>> importing/exporting datavalues, dataset is an attached element for the tasks
>> with the additionally important info: period, ou, etc. of the dataelements.
>> So, whether the repository (data warehouse) needs that local dataset or not
>> they (reluctantly-imported-dataset) do exist!
>> And of course, as a central data house/repository: import/export is a
>> basic function, so will we find another criterion for checking-out the
>> dael-dataset-integrity for a data warehouse/repository?
>>
>> Any ideas?
>>
>> --
>> --
>> Best regards,
>> Kim-Anh Vo
>>
>> +84.906612246
>> kavo@xxxxxxxxxx
>> Coordinator of HISP(hisp.info) in Vietnam
>> Master of Information Systems
>> at the University of Oslo
>> ------------------------------------
>> join facebook at www.facebook.com join LinkedIn at www.linkedin.com
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>