dhis2-devs team mailing list archive

[David Siang Fong Oh <doh@xxxxxxxxxxxxxxxx>]

Hi team,

We've noticed a performance issue when exporting metadata through the API.
The issue appears to have been introduced in revision 20839
<http://bazaar.launchpad.net/~dhis2-devs-core/dhis2/2.21/revision/20839> on
the 2.21 branch (revision 20854
<http://bazaar.launchpad.net/~dhis2-devs-core/dhis2/trunk/revision/20854> on
trunk) where local caching of systemSettings in the DefaultCalendarService
was removed in favour of leveraging the Guava caching in the
DefaultSystemSettingManager.

Performance impact when downloading full metadata export through API on a
development machine (16GB RAM, 2.6 GHz quad-core CPU):

   - Before (revision 20838 on 2.21 branch): ~11 seconds for a ~20MB file
   - After (revision 20839 on 2.21 branch): ~180 seconds for a ~20MB file

The caching performance issue seems to manifest itself when mapping the
period dates for charts and reportTables which requires the system calendar
setting (stacktrace below). Excluding charts and reportTables from the
metadata export resolves the issue (download takes ~10 seconds for a ~13 MB
file).

Have you experienced any similar issues with the Guava cache implemented in
the DefaultSystemSettingManager (or anywhere else)? I wanted to check
before investigating any further.

Cheers,

-doh

Stacktrace when mapping period dates for charts:

*at
org.hisp.dhis.setting.DefaultSystemSettingManager.getSystemSetting(DefaultSystemSettingManager.java:157)*
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy58.getSystemSetting(Unknown Source)
*at
org.hisp.dhis.calendar.DefaultCalendarService.getSystemCalendar(DefaultCalendarService.java:102)*
*at org.hisp.dhis.period.PeriodType.getCalendar(PeriodType.java:101)*
*at
org.hisp.dhis.period.PeriodType.createLocalDateUnitInstance(PeriodType.java:364)*
*at org.hisp.dhis.period.PeriodType.getIsoDate(PeriodType.java:501)*
*at org.hisp.dhis.period.Period.getIsoDate(Period.java:158)*
at
org.hisp.dhis.common.adapter.JacksonPeriodSerializer.serialize(JacksonPeriodSerializer.java:48)
at
org.hisp.dhis.common.adapter.JacksonPeriodSerializer.serialize(JacksonPeriodSerializer.java:42)