dhis2-devs team mailing list archive

[Jason Pickering <jason.p.pickering@xxxxxxxxx>]

> Yes there is a slight inconsistency here. Still, I think this is as good as
> we get it. Remember that we (in most cases, except for "average"
> dataelements) don't store 0s in the raw datavalue table. This means that the
> 0s you would get in the agg.datavalue table would represent blanks anyway.

Right but this is why I keep hammering this issue. Zeros can be
significant, and need to be reported. Take "Number of days cold chain
fridge has been down over the past month". For this case, zeroes are
important  and significant.

> This is also a pragmatic issue. When saving 0s in the datavalue table and
> exporting 0s to the agg.datavalue table we experienced for a typical case
> that the tables would contain 70-80 % of 0s, which slowed down queries a
> lot.

Which brings me back to the difference between a zero and a blank. I
think that users should be able to specify, on a per data element
basis, whether aggregated values should be written or not. I see
several cases.

1) All blanks in a record set. This would be represented as a
user-specified value, and would be an aggregate of all blanks in a
recordset. It should be able to be represented as a blank, a value
(including zero), or omitted entirely from the final recordset in
agg.datavalue and agg.indicatorvalue tables depending on user settings
per data element.
2) A combination of blanks and zeros  in a recordset. The aggregate
would be a zero in all cases, and should be able to be represented as
a blank, a zero, or omitted entirely from the final recordset
depending on user settings per data element.
3) All zeros in a record set. The aggregate would be a zero in all
cases, and should be able to be represented as a blank, a zero, or
omitted entirely from the final recordset depending on user settings
per data element.
4) A combination of non-zeros and blanks. This should be represented
as the aggregate of non-zero values which is the current behaviour.
5) A combination of non-zeros and zeros (but no blanks). This should
be represented as the aggregate of all values which is the current
behaviour. .
6) A combination of zeros, blanks and non-zeros. This would be
represented as an aggregate of non-zero values which is the current
behaviour. .
7) A combination of non-zeros resulting in a zero (A combination of
positive and negative numbers). Not really sure about this one.

There might be other cases ,but these are the ones that come to mind.

This is my wish list for how zeros and blanks should be handled. You
say  this is as good as we will get. This may be the case, but then we
need to very clearly document that there is no distinction between
these cases, and that there is no distinction between zeros and blanks
in a recordset and any aggregate zeros should be considered as the
possible result of an aggregation of all blank value.  I think this
may be OK, but then we need to document, document, document this, as
it is very important to understand to people looking at the data.

Regards,
Jason