← Back to team overview

dhis2-devs team mailing list archive

Re: Zero or Positive Integer - Variation on negative Min values theme?

 

Hi,

the issue with negative values occurring as min-values for data elements of
type positive / zero-or-positive number, and positive values occurring as
max-values for data elements of type negative number have now been fixed in
trunk and 2.13.

regards,

Lars



On Wed, Nov 6, 2013 at 12:00 AM, Jason Pickering <
jason.p.pickering@xxxxxxxxx> wrote:

> Hi Dapo.,
>
> This is just a design issue with DHIS2. There needs to be some better
> statistical methods employed to generate the min max, or a means to specify
> that the minimum should be zero for all values. There is no linkage between
> the value type and the method which the mix-max uses to generate its
> values. In our case, we use a combination of R and SQL to calculate
> "better" values on a regular basis. A bit of  a hack, but the current
> methods are simply not working for us either.
>
> Regards,
> Jason
>
>
>
>
>
> On Wed, Nov 6, 2013 at 12:23 AM, Adebusoye Anifalaje <busoye@xxxxxxxx>wrote:
>
>> Hi all,
>>
>> To introduce a variation on this theme of generating min-max values: I
>> expected that if you set data element number type to Zero of Positive
>> Integer this should alter the distribution (in this case skew to the right
>> as Jason put it) and avoid the negative minimum values. From the tests I
>> have done, this is not happening.
>>
>> The negative mim values are a real pain, anyone with a solution yet?
>>
>> Regards,
>>
>> Busoye
>>
>> On 31 May 2013, at 14:53, Gulam Khan <gm-khan@xxxxxxxxxxxxxxxxxxxx>
>> wrote:
>>
>> Hi Dapo,
>> If you were to generate the min-max values automatically, insert them
>> into the database, and an authorized user was to change them, then they
>> would be overwritten.
>>
>> You can automatically generate these values through the data
>> administration module by dataset and orgunit in Data
>> Administration->Min-Max Value Generation . The problem is you often end up
>> with a lot of dubious values, such as negative numbers. This is a result of
>> the Gaussian distribution model which is applied (without taking any
>> skewness into effect). I have attached a plot from a real DHIS2 database,
>> which show sthat the distribution of many in DHIS2 are skewed left or right
>> (more often to the right) . Use of an unbounded, normal distribution for
>> calculation of the min-max is therefore not really appropriate. The
>> standard deviation approach is also highly sensitive to outliers, which
>> also creates problems for the outlier analysis if this approach is used.
>> Anyway, I digress. Just wanted to highlight that the in-built functions for
>> calculation of the min-max are rather primitive, and may not always lead to
>> good results.
>>
>> I am not so sure about the ability to restrict the time periods for which
>> the mix-max is generated. I think you would end up with far too few data
>> points to make a meaningful standard deviation. You are really looking for
>> the extremes with the min-max, and restricting to such a a narrow time
>> interval would probably not give you these extremes.
>>
>> Regards,
>> Jason
>>
>>
>>
>> On Thu, Sep 19, 2013 at 11:47 PM, Dapo Adejumo <dapo_adejumo@xxxxxxxxx>wrote:
>>
>>> Hi,
>>>
>>> It will be great if the generation of min-max values can be automated
>>> based on a global setting ( ability to select last 3,6 or 12 months data) .
>>> I am curious what happens to these “hard coded” min/max values when the
>>> database is set to generate min/max values from the frontend – I will
>>> assume they are discarded since they reside in the same table? Or which
>>> takes precedence ?
>>>
>>> Thanks.
>>>
>>>
>>>
>>> ................................................
>>>
>>>
>>>
>>> *Regards,*
>>>
>>>
>>>
>>> *Dapo Adejumo*
>>>
>>> *+234803363677*
>>>
>>> *skype : dapojorge*
>>>
>>>
>>>
>>> *From:* Dhis2-devs [mailto:dhis2-devs-bounces+dapo_adejumo=
>>> yahoo.com@xxxxxxxxxxxxxxxxxxx] *On Behalf Of *Jason Pickering
>>> *Sent:* Thursday, September 19, 2013 6:28 PM
>>> *To:* Juma Lungo
>>> *Cc:* dhis2-devs
>>>
>>> *Subject:* Re: [Dhis2-devs] Setting MinMax Values
>>>
>>>
>>>
>>> I think I agree with Lars as well. Our use case was we wanted to
>>> restrict values between 0 and 31, and not do it with a validation rule, to
>>> provide visual clues during data entry.
>>>
>>>
>>>
>>> As an aside, one may also want to calculate these values and save them
>>> to the database. The current algorithm uses a standard deviation approach,
>>> which is problematic, in that in many cases the data which is typically
>>> observed does not follow a normal Gaussian distribution. Other statistical
>>> models often provide better estimates of the mix/max, but currently, cannot
>>> be calculated automatically by the system. If these values are calculated
>>> externally in some statistical package, they can be inserted back into the
>>> system (or even automated).
>>>
>>>
>>>
>>> Regards,
>>>
>>> Jason
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Sep 19, 2013 at 6:25 PM, Juma Lungo <jlungo@xxxxxxxxx> wrote:
>>>
>>> I agree with Lars
>>>
>>>
>>>
>>> Lungo
>>>
>>>
>>> ------------------------------
>>>
>>> *From:* Lars Helge Øverland <larshelge@xxxxxxxxx>
>>> *To:* Jason Pickering <jason.p.pickering@xxxxxxxxx>
>>> *Cc:* dhis2-devs <dhis2-devs@xxxxxxxxxxxxxxxxxxx>
>>> *Sent:* Thursday, September 19, 2013 2:43 PM
>>> *Subject:* Re: [Dhis2-devs] Setting MinMax Values
>>>
>>>
>>>
>>> Hi Conrad,
>>>
>>>
>>>
>>> the min-max data element is per org unit and data element like you say,
>>> so it applies to all current and future periods by definition. I don't know
>>> your use-case... but in general one might want to think twice before
>>> setting the same max value for all facilities, since they usually have very
>>> different catchment population and hence natural variations in numbers.
>>>
>>>
>>>
>>> Lars
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to    : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help  : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>
>> <skewness.png>_______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>

Follow ups

References