← Back to team overview

dhis2-devs team mailing list archive

Re: [Bug 1612375] [NEW] Program indicators: error in percentage calculations when using d2:condition (inconsistent results at different levels of aggregation)

 

Hi there Sam,
thank you for the detailed report.

The division is done with integers when both the inputs are integers. So the result of the divison is rounded down from f.ex. 0,66 to 0 before the multiplication with 100 happens. You found a workaround yourself, another option would be to do the multiplication before the division:
> ·         sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',1,0)) * 100 / V{event_count}


P.S. To get the given example to work on the latest 2.24 and 2.25 snapshot with custom aggregation it was also necessary to specify aggregation operator for the event count variable:
> ·         sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',1,0)) * 100 / count(V{event_count})


Best regards,
Markus

> 25. aug. 2016 kl. 15.13 skrev Sam Johnson <samuel.johnson@xxxxxxxxxx>:
> 
> Hi all,
>  
> Just following up on this bug report again – is there any chance I could get it looked at/prioritized?
>  
> I’ve dug a little deeper, and it looks like this bug could be as simple as an error in variable handling.
>  
> Specifically, although in the example I originally gave, the following DOESN’T work:
> ·         sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',1,0)) / V{event_count} * 100
> the following DOES seem to work:
> ·         sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',100,0)) / V{event_count}
>  
> Is it possible that when the aggregation step of the indicator calculation is undertaken, any decimal results are at some point forced to an integer, generating incorrect values wherever the result is <1?
>  
> Cheers,
>  
> Sam.
>  
>  
> From: Dhis2-devs <dhis2-devs-bounces+samuel.johnson=qebo.co.uk@xxxxxxxxxxxxxxxxxxx> on behalf of Samuel Scott <samuel.johnson@xxxxxxxxxx>
> Date: Tuesday, 23 August 2016 at 15:08
> To: DHIS2 Developers <dhis2-devs@xxxxxxxxxxxxxxxxxxx>, Lars Helge Øverland <lars@xxxxxxxxx>
> Subject: Re: [Dhis2-devs] [Bug 1612375] [NEW] Program indicators: error in percentage calculations when using d2:condition (inconsistent results at different levels of aggregation)
>  
> Hi all,
>  
> I reported a bug in the web program indicator calculations a couple of weeks ago, and included in the report everything that’s needed to quickly replicate the bug, but I don’t think anyone has had a chance to look at this report yet.
>  
> Have I followed the correct process for reporting this?  Is it at all possible to get an idea of when a fix for this might be available?  (It’s quite a serious bug, as ‘valid’ expressions are returning incorrect/inconsistent results at different levels of aggregation.)
>  
> Many thanks,
>  
> Sam.
>  
>  
> From: Dhis2-devs <dhis2-devs-bounces+samuel.johnson=qebo.co.uk@xxxxxxxxxxxxxxxxxxx> on behalf of Samuel Scott <samuel.johnson@xxxxxxxxxx>
> Reply-To: Bug 1612375 <1612375@xxxxxxxxxxxxxxxxxx>
> Date: Thursday, 11 August 2016 at 19:21
> To: DHIS2 Developers <dhis2-devs@xxxxxxxxxxxxxxxxxxx>
> Subject: [Dhis2-devs] [Bug 1612375] [NEW] Program indicators: error in percentage calculations when using d2:condition (inconsistent results at different levels of aggregation)
>  
> Public bug reported:
>  
> Bug found in 2.22, still present in 2.24 online demo (2.24 rev.23557)
> Online demo accessed using Chrome 52.0.2743.116.
>  
> There appears to be a bug in the program indicator calculations when:
> a sum() custom aggregation is used twice, to enable the calculation of a percentage AND
> an element of the percentage calculation includes d2:condition.
>  
> Note that either of these situations works properly on its own - ie the
> numerator and denominator work perfectly when separated into two
> different indicators - it is only when combined that the bug occurs.
>  
> TO REPRODUCE THIS BUG
>  
> In the online demo instance, using the Antenatal Care Visit program:
>  
> 1. Create a program indicator called TestNumerator, with aggregation type 'custom' and expression:
> sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',1,0))
>  
> 2. Create a program indicator called TestDenominator, with aggregation type 'custom' and expression:
> V{event_count}
>  
> 3. Create a program indicator called TestPercentage, with aggregation type 'custom' and expression, which combines these two expressions as a percentage:
> sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',1,0)) / V{event_count} * 100
>  
> 4. Create a pivot table based on the Antenatal program, add all three indicators to the pivot, and run it:
> Numerator and denominator calculate correctly, but percentage returns 2/3 = 0 (instead of 66.7)
>  
> 5. Now change the Period from Last 12 months to Last 52 weeks, and re-run:
> All indicators now calculate correctly, with percentage correctly showing 0/1 = 0 and 2/2 = 100.
>  
>  
> Note that this is not a bug in the event_count variable, as the same bug consistently appears even when V{event_count} is replaced by sum(1) in the indicator expressions.
>  
> I have a project whose viability depends on being able to correctly
> calculate these percentages at higher levels of aggregation (regions
> etc), so if confirmed as a bug, I would be hugely grateful if it were
> possible to fix this in the current release, rather than in 2.25.
>  
> Many thanks, Sam.
>  
> ** Affects: dhis2
>      Importance: Undecided
>          Status: New
>  
> -- 
> You received this bug notification because you are a member of DHIS 2
> developers, which is subscribed to DHIS.
> https://bugs.launchpad.net/bugs/1612375 <https://bugs.launchpad.net/bugs/1612375>
>  
> Title:
>   Program indicators: error in percentage calculations when using
>   d2:condition (inconsistent results at different levels of aggregation)
>  
> Status in DHIS:
>   New
>  
> Bug description:
>   Bug found in 2.22, still present in 2.24 online demo (2.24 rev.23557)
>   Online demo accessed using Chrome 52.0.2743.116.
>  
>   There appears to be a bug in the program indicator calculations when:
>   a sum() custom aggregation is used twice, to enable the calculation of a percentage AND
>   an element of the percentage calculation includes d2:condition.
>  
>   Note that either of these situations works properly on its own - ie
>   the numerator and denominator work perfectly when separated into two
>   different indicators - it is only when combined that the bug occurs.
>  
>   TO REPRODUCE THIS BUG
>  
>   In the online demo instance, using the Antenatal Care Visit program:
>  
>   1. Create a program indicator called TestNumerator, with aggregation type 'custom' and expression:
>   sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',1,0))
>  
>   2. Create a program indicator called TestDenominator, with aggregation type 'custom' and expression:
>   V{event_count}
>  
>   3. Create a program indicator called TestPercentage, with aggregation type 'custom' and expression, which combines these two expressions as a percentage:
>   sum(d2:condition('#{dBwrot7S420.vANAXwtLwcT}>=10',1,0)) / V{event_count} * 100
>  
>   4. Create a pivot table based on the Antenatal program, add all three indicators to the pivot, and run it:
>   Numerator and denominator calculate correctly, but percentage returns 2/3 = 0 (instead of 66.7)
>  
>   5. Now change the Period from Last 12 months to Last 52 weeks, and re-run:
>   All indicators now calculate correctly, with percentage correctly showing 0/1 = 0 and 2/2 = 100.
>  
>   
>   Note that this is not a bug in the event_count variable, as the same bug consistently appears even when V{event_count} is replaced by sum(1) in the indicator expressions.
>  
>   I have a project whose viability depends on being able to correctly
>   calculate these percentages at higher levels of aggregation (regions
>   etc), so if confirmed as a bug, I would be hugely grateful if it were
>   possible to fix this in the current release, rather than in 2.25.
>  
>   Many thanks, Sam.
>  
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/dhis2/+bug/1612375/+subscriptions <https://bugs.launchpad.net/dhis2/+bug/1612375/+subscriptions>
>  
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs <https://launchpad.net/~dhis2-devs>
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx <mailto:dhis2-devs@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~dhis2-devs <https://launchpad.net/~dhis2-devs>
> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
>  
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp


Follow ups

References