dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Hi,

No generic model will fit ALL types of data elements, obviously - but it
needs to fit the most typical data elements where you want to USE the
min-max to capture e.g. reporting or capturing mistakes. If using a stdev
model, you reset any negative values (which is impossible) to zero.

Using the historic min and max in general makes the min-max useless for
such pattern recognition, because any outbreak or campaign or disruption in
a service for whatever reason will permanently affect the min/max so that
they no longer represent the "normal" variation of a data element.

But it seems I was wrong about this being implemented in DHIS2, then.
Jason's comment that "in the past, we have pulled the data values into
statistical software, calculated the min/max according to a given model,
and then injected them back via SQL" clearly indicates what is required. We
can argue over the model or models to be used, but some kind of model
depicting normal distribution of values over time is clearly required. you
cannot expect average users to be able to pull data values into STATA or
SAS or SPSS, calculate min/max usign a given model, and then re-insert the
values using SQL.

I'll write a blue-print for this - either as part of the core or as an app
- when I get my head above the water :-)

Regards
Calle

On 20 April 2015 at 15:26, Jason Pickering <jason.p.pickering@xxxxxxxxx>
wrote:

> I think that would work, but as far as I know you cannot import these via
> the API, but it is pretty trivial do it directly into the database.
>
> You might be able to script it however by calling to something like
>
>
> https://apps.dhis2.org/demo/dhis-web-dataentry/saveMinMaxLimits.action?organisationUnitId=ImspTQPwCqd&dataElementId=BOSZApCrBni&categoryOptionComboId=TkDhg29x18A&minLimit=10&maxLimit=20
>
> Maybe the devs can suggest a better way, but in the past, we have pulled
> the data values into statistical software, calculated the min/max according
> to a given model, and then injected them back via SQL.
>
> Regards,
> Jason
>
>
> On Mon, Apr 20, 2015 at 3:22 PM Rodolfo Melia <rmelia@xxxxxxxxxxxx> wrote:
>
>> my test also shows that only the min/max across all values of an org unit
>> is use for setting mix/max. As this is unlikely to change any soon, I just
>> want to understand if I can set the min/max values based on my own logic,
>> and then inject it via the API.
>>
>> *Rodolfo Meliá*
>> *Principal  |  *rmelia@xxxxxxxxxxxx
>> Skype: rod.melia  |  +44 777 576 4090  |  +1 708 872 7636
>> www.knowming.com
>>
>> On Mon, Apr 20, 2015 at 2:14 PM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> HI,
>>>
>>> I think as Prosper says, it is simply the overall max and min, and this
>>> can be set by the user, or calculated externally.
>>>
>>> Calle, use of standard deviation is problematic for several reasons
>>> however, mostly because it makes an assumption that the data is actually
>>> normally distributed, which is not really always the case. This may be
>>> appropriate for some data elements, but in many cases, it is not an
>>> appropriate statistical assumption, and results which we have seen after
>>> often zero inflated or follow something more like a logistical distribution
>>> (as opposed to a normal distribution).  So applying something like  a
>>> standard deviation may (and does in the case of DHIS2)  result in many
>>> negative min values. So, although I think the method of DHIS 1.4 may be
>>> somewhat better, it still is not really always appropriate, as the
>>> assumption of a normal distribution is simply not always warranted.
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>>
>>> On Mon, Apr 20, 2015 at 3:02 PM Calle Hedberg <calle.hedberg@xxxxxxxxx>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Sorry, but if the automatic min-max calculation is simply retrieving
>>>> the historical minimum and maximum values, then that makes little sense and
>>>> would have no real value.
>>>>
>>>> I don't have time to verify it right now, but my assumption has always
>>>> been that DHIS2 is using a method similar to the one in DHIS 1.4:
>>>>
>>>> 1.
>>>> The user specify a period to be used for the min/max analysis -
>>>> typically 12-18 months (longer is better in a stable health establishment
>>>> environment, but a shorter period might be optimal in areas where patient
>>>> numbers are changing rapidly)
>>>>
>>>> 2.
>>>> The average and standard deviation is calculated for each
>>>> OrganisationUnit and DataElement combination
>>>>
>>>> 3.
>>>> The min is set to the average minus stdev x constant, max is set to
>>>> average plus stdev x constant. Typically constants are 1.5 - 2.0  (There is
>>>> an "Data analysis std dev factor" specified under General Settings -
>>>> by default set to 2.0. System might be using that).
>>>>
>>>> Best regards
>>>> Calle
>>>>
>>>>
>>>> On 20 April 2015 at 14:49, Rodolfo Melia <rmelia@xxxxxxxxxxxx> wrote:
>>>>
>>>>> Thanks Prosper - that makes sense.
>>>>> I guess that the only way to set a different rule (e.g., Max should be
>>>>> 150% than previous period) the max will need to be set via an app.
>>>>>
>>>>> Are Max/ Min of a Data Element/ Org Unit exposed in the API ?
>>>>>
>>>>> R
>>>>>
>>>>> *Rodolfo Meliá*
>>>>> *Principal  |  *rmelia@xxxxxxxxxxxx
>>>>> Skype: rod.melia  |  +44 777 576 4090  |  +1 708 872 7636
>>>>> www.knowming.com
>>>>>
>>>>> On Mon, Apr 20, 2015 at 1:29 PM, Prosper BT <ptb3000@xxxxxxxxx> wrote:
>>>>>
>>>>>> Hi Rodolfo,
>>>>>>
>>>>>> Used its before and according to the results we got, it takes the
>>>>>> ever max and min entered in the selected period for a given dataset.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> On Mon, Apr 20, 2015 at 3:24 PM, Rodolfo Melia <rmelia@xxxxxxxxxxxx>
>>>>>> wrote:
>>>>>>
>>>>>>> I'm trying to understand how does DHIS generates the Min/ Max values
>>>>>>> for a given Data Element/ Period. The documentation
>>>>>>> <https://www.dhis2.org/doc/snapshot/en/user/html/ch12s03.html> only
>>>>>>> makes reference to the fact that Max/Min can be set manually or
>>>>>>> automatically calculated. When automatically calculated: does anyone knows
>>>>>>> what is the logic used for setting the values? Twice the average of
>>>>>>> previous periods?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Prosper Behumbiize, MPH
>>>>>> Phone:        +256 414 320076
>>>>>> Cell:             +256 772 139037
>>>>>>                      +256 752 751776
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *******************************************
>>>>
>>>> Calle Hedberg
>>>>
>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>
>>>> Tel/fax (home): +27-21-685-6472
>>>>
>>>> Cell: +27-82-853-5352
>>>>
>>>> Iridium SatPhone: +8816-315-19274
>>>>
>>>> Email: calle.hedberg@xxxxxxxxx
>>>>
>>>> Skype: calle_hedberg
>>>>
>>>> *******************************************
>>>>
>>>>  _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>
>>


-- 

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19274

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************