dhis2-devs team mailing list archive

[Jason Pickering <jason.p.pickering@xxxxxxxxx>]

Hi Dapo.,

This is just a design issue with DHIS2. There needs to be some better
statistical methods employed to generate the min max, or a means to specify
that the minimum should be zero for all values. There is no linkage between
the value type and the method which the mix-max uses to generate its
values. In our case, we use a combination of R and SQL to calculate
"better" values on a regular basis. A bit of  a hack, but the current
methods are simply not working for us either.

Regards,
Jason





On Wed, Nov 6, 2013 at 12:23 AM, Adebusoye Anifalaje <busoye@xxxxxxxx>wrote:

> Hi all,
>
> To introduce a variation on this theme of generating min-max values: I
> expected that if you set data element number type to Zero of Positive
> Integer this should alter the distribution (in this case skew to the right
> as Jason put it) and avoid the negative minimum values. From the tests I
> have done, this is not happening.
>
> The negative mim values are a real pain, anyone with a solution yet?
>
> Regards,
>
> Busoye
>
> On 31 May 2013, at 14:53, Gulam Khan <gm-khan@xxxxxxxxxxxxxxxxxxxx> wrote:
>
> Hi Dapo,
> If you were to generate the min-max values automatically, insert them into
> the database, and an authorized user was to change them, then they would be
> overwritten.
>
> You can automatically generate these values through the data
> administration module by dataset and orgunit in Data
> Administration->Min-Max Value Generation . The problem is you often end up
> with a lot of dubious values, such as negative numbers. This is a result of
> the Gaussian distribution model which is applied (without taking any
> skewness into effect). I have attached a plot from a real DHIS2 database,
> which show sthat the distribution of many in DHIS2 are skewed left or right
> (more often to the right) . Use of an unbounded, normal distribution for
> calculation of the min-max is therefore not really appropriate. The
> standard deviation approach is also highly sensitive to outliers, which
> also creates problems for the outlier analysis if this approach is used.
> Anyway, I digress. Just wanted to highlight that the in-built functions for
> calculation of the min-max are rather primitive, and may not always lead to
> good results.
>
> I am not so sure about the ability to restrict the time periods for which
> the mix-max is generated. I think you would end up with far too few data
> points to make a meaningful standard deviation. You are really looking for
> the extremes with the min-max, and restricting to such a a narrow time
> interval would probably not give you these extremes.
>
> Regards,
> Jason
>
>
>
> On Thu, Sep 19, 2013 at 11:47 PM, Dapo Adejumo <dapo_adejumo@xxxxxxxxx>wrote:
>
>> Hi,
>>
>> It will be great if the generation of min-max values can be automated
>> based on a global setting ( ability to select last 3,6 or 12 months data) .
>> I am curious what happens to these “hard coded” min/max values when the
>> database is set to generate min/max values from the frontend – I will
>> assume they are discarded since they reside in the same table? Or which
>> takes precedence ?
>>
>> Thanks.
>>
>>
>>
>> ................................................
>>
>>
>>
>> *Regards,*
>>
>>
>>
>> *Dapo Adejumo*
>>
>> *+234803363677*
>>
>> *skype : dapojorge*
>>
>>
>>
>> *From:* Dhis2-devs [mailto:dhis2-devs-bounces+dapo_adejumo=
>> yahoo.com@xxxxxxxxxxxxxxxxxxx] *On Behalf Of *Jason Pickering
>> *Sent:* Thursday, September 19, 2013 6:28 PM
>> *To:* Juma Lungo
>> *Cc:* dhis2-devs
>>
>> *Subject:* Re: [Dhis2-devs] Setting MinMax Values
>>
>>
>>
>> I think I agree with Lars as well. Our use case was we wanted to restrict
>> values between 0 and 31, and not do it with a validation rule, to provide
>> visual clues during data entry.
>>
>>
>>
>> As an aside, one may also want to calculate these values and save them to
>> the database. The current algorithm uses a standard deviation approach,
>> which is problematic, in that in many cases the data which is typically
>> observed does not follow a normal Gaussian distribution. Other statistical
>> models often provide better estimates of the mix/max, but currently, cannot
>> be calculated automatically by the system. If these values are calculated
>> externally in some statistical package, they can be inserted back into the
>> system (or even automated).
>>
>>
>>
>> Regards,
>>
>> Jason
>>
>>
>>
>>
>>
>> On Thu, Sep 19, 2013 at 6:25 PM, Juma Lungo <jlungo@xxxxxxxxx> wrote:
>>
>> I agree with Lars
>>
>>
>>
>> Lungo
>>
>>
>> ------------------------------
>>
>> *From:* Lars Helge Øverland <larshelge@xxxxxxxxx>
>> *To:* Jason Pickering <jason.p.pickering@xxxxxxxxx>
>> *Cc:* dhis2-devs <dhis2-devs@xxxxxxxxxxxxxxxxxxx>
>> *Sent:* Thursday, September 19, 2013 2:43 PM
>> *Subject:* Re: [Dhis2-devs] Setting MinMax Values
>>
>>
>>
>> Hi Conrad,
>>
>>
>>
>> the min-max data element is per org unit and data element like you say,
>> so it applies to all current and future periods by definition. I don't know
>> your use-case... but in general one might want to think twice before
>> setting the same max value for all facilities, since they usually have very
>> different catchment population and hence natural variations in numbers.
>>
>>
>>
>> Lars
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to    : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help  : https://help.launchpad.net/ListHelp
>>
>>
>>
>
> <skewness.png>_______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>