dhis2-devs team mailing list archive

[Ola Hodne Titlestad <olatitle@xxxxxxxxx>]

2009/9/29 Jason Pickering <jason.p.pickering@xxxxxxxxx>

> > Yes there is a slight inconsistency here. Still, I think this is as good
> as
> > we get it. Remember that we (in most cases, except for "average"
> > dataelements) don't store 0s in the raw datavalue table. This means that
> the
> > 0s you would get in the agg.datavalue table would represent blanks
> anyway.
>
> Right but this is why I keep hammering this issue. Zeros can be
> significant, and need to be reported. Take "Number of days cold chain
> fridge has been down over the past month". For this case, zeroes are
> important  and significant.
>
>
I would define the data element "Number of days cold chain
fridge has been down over the past month" as an "average" data element so
the "0s" would be stored anyway.
When aggregating I assume you would like to know the average number of days
the fridge was not working, not the total number of days for all clinics in
a district, or all months in a year (for one facility)?



> > This is also a pragmatic issue. When saving 0s in the datavalue table and
> > exporting 0s to the agg.datavalue table we experienced for a typical case
> > that the tables would contain 70-80 % of 0s, which slowed down queries a
> > lot.
>
> Which brings me back to the difference between a zero and a blank. I
> think that users should be able to specify, on a per data element
> basis, whether aggregated values should be written or not. I see
> several cases.
>
> 1) All blanks in a record set. This would be represented as a
> user-specified value, and would be an aggregate of all blanks in a
> recordset. It should be able to be represented as a blank, a value
> (including zero), or omitted entirely from the final recordset in
> agg.datavalue and agg.indicatorvalue tables depending on user settings
> per data element.
> 2) A combination of blanks and zeros  in a recordset. The aggregate
> would be a zero in all cases, and should be able to be represented as
> a blank, a zero, or omitted entirely from the final recordset
> depending on user settings per data element.
> 3) All zeros in a record set. The aggregate would be a zero in all
> cases, and should be able to be represented as a blank, a zero, or
> omitted entirely from the final recordset depending on user settings
> per data element.
> 4) A combination of non-zeros and blanks. This should be represented
> as the aggregate of non-zero values which is the current behaviour.
> 5) A combination of non-zeros and zeros (but no blanks). This should
> be represented as the aggregate of all values which is the current
> behaviour. .
> 6) A combination of zeros, blanks and non-zeros. This would be
> represented as an aggregate of non-zero values which is the current
> behaviour. .
> 7) A combination of non-zeros resulting in a zero (A combination of
> positive and negative numbers). Not really sure about this one.
>
> There might be other cases ,but these are the ones that come to mind.
>
> This is my wish list for how zeros and blanks should be handled. You
> say  this is as good as we will get. This may be the case, but then we
> need to very clearly document that there is no distinction between
> these cases, and that there is no distinction between zeros and blanks
> in a recordset and any aggregate zeros should be considered as the
> possible result of an aggregation of all blank value.  I think this
> may be OK, but then we need to document, document, document this, as
> it is very important to understand to people looking at the data.
>
> Regards,
> Jason
>