dhis2-devs team mailing list archive

[Alex Tumwesigye <atumwesigye@xxxxxxxxx>]

I am very grateful to the DHIS2 team.Finally database/table partitioning
have been introduced in the 2.11. That's one  feature I have longed for.
Surely, I am so excited, "A year ago has yielded fruits".

Bravo.




On Mon, Mar 25, 2013 at 5:27 PM, Paulo Grácio <paulogracio@xxxxxxxxx> wrote:

> Thanks. I missed that email, sorry for  the spam.
>
> Regards,
> Paulo Grácio
>
> On 25 March 2013 15:32, David Muturi <dnmuturi@xxxxxxxxx> wrote:
> > Hi Paulo,
> > Here it is...
> > this update is intended for server administrators looking to upgrade
> > to DHIS 2.11.
> >
> > In version 2.11, DHIS is performing on-the-fly data aggregation as
> > part of the new analytics API. This is in contrast to the datamart API
> > where data is been pre-aggregated for a set of dimensions. The
> > analytics tables contain indexed columns for each data dimension
> > together with a column for the "raw" data values. The tables are
> > partitioned for each year of data. Partition means a part of a table
> > which has been split horizontally on rows - you will find tables like
> > analytics_2011, analytics_2012, analytics_2013 in the database. This
> > has some consequences for server configuration:
> >
> >
> > 1) It is strongly recommended to set up a cache server to reduce the
> > pressure on your web servers. First, caching must be enabled in DHIS
> > from "System general settings" by setting "Cache strategy" to "Cache
> > until 6 am tomorrow". The simplest way to set up a server-side cache
> > if you already have a reverse proxy like nginx and apache running is
> > to configure it directly in the proxy. Enabling caching for nginx is
> > documented here:
> >
> > http://dhis2.org/doc/snapshot/en/implementer/html/ch08s02.html#d5e580
> >
> > Apache caching can be done through the mod_cache module:
> > http://httpd.apache.org/docs/2.2/mod/mod_cache.html
> >
> > Otherwise, dedicated caches like varnish
> > (https://www.varnish-cache.org/) and squid
> > (http://www.squid-cache.org/) could be set up in front of your web
> > servers.
> >
> >
> >
> > 2) Run latest version of PostgreSQL which is 9.2. It has some nice
> > performance improvements over earlier versions.
> >
> >
> >
> > 3) Make sure a lot of memory is available to the database /
> > PostgreSQL. If the analytics tables for the last 2-3 years can fit in
> > the memory cache then on-the-fly aggregation will perform much better.
> > You can see the size of the analytics tables and indexes using this
> > SQL on PostgreSQL:
> >
> > select pg_size_pretty( pg_total_relation_size( 'analytics_2013' ) );
> >
> > The more memory you have the better. Documentation has been updated,
> > have a look at the values for shared_buffers and effective_cache_size
> > settings and update your configurations adjusted for your available
> > memory:
> >
> > http://dhis2.org/doc/snapshot/en/implementer/html/ch08.html#d5e413
> >
> >
> >
> > 4) Make sure you have enough disk space as t
> > he size of the database will increase with the analytics tables. Also,
> > when doing database backups it's not required to include the analytics
> > nor the datamart tables. An efficient command for creating backups
> > which omits these tables and compresses the output in one go (replace
> > db-name and db-user) is:
> >
> > pg_dump db-name -U db-user -T aggregated* -T analytics* -T
> > completeness* | gzip > dhis-pgdump.gz
> >
> >
> > 5) Optional. Use a fast hard disk type/drive. It will help performance
> > both on analytics table generation and read queries if you have a fast
> > disk type, ideally SSD.
> >
> >
> > rgds
> > David
> >
> > 2013/3/25 Paulo Grácio <paulogracio@xxxxxxxxx>:
> >> Congratulations for the achivement!
> >>
> >> Is there any information about how to upgrade from version 2.10 to 2.11?
> >>
> >> Kind regards,
> >> Paulo Grácio
> >>
> >> On 22 March 2013 19:05, Lars Helge Øverland <larshelge@xxxxxxxxx>
> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> DHIS version 2.11 is out and we have some great features for you:
> >>>
> >>>
> >>> - Web-based pivot table: Allows you to create pivot tables with data
> >>> aggregated along all data dimensions, including data
> element/organisation
> >>> unit group sets. Data dimensions can be arranged freely on columns,
> rows and
> >>> as filters. You can include totals and subtotals on all axis and
> control the
> >>> table density and font size under options. Your pivots can be saved as
> >>> favorites and downloaded to your local computer. This design goal is to
> >>> allow users to create their own reports with the data they need
> without any
> >>> involvement of super-users.
> >>>
> >>> A pivot table with immunization indicators for last 4 quarters by
> facility
> >>> ownership, type and location can be made like this (log in with
> >>> admin/district):
> >>>
> >>>
> http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=XWXTmFUffng
> >>>
> >>> A pivot table with number of diagnosis by facility ownership and type
> with
> >>> totals and subtotals looks like this (notice that diagnosis are data
> element
> >>> groups):
> >>>
> >>>
> http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=vlnZeGhgw1T
> >>>
> >>> A pivot table for vaccines, broken down on categories, age group and
> point
> >>> of service, last 12 months:
> >>>
> >>>
> http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=z31v4RMNqnu
> >>>
> >>> A pivot table with reporting rates, major forms for public facilities
> by
> >>> location and type, last 12 months:
> >>>
> >>>
> http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=VHewKMjyLnR
> >>>
> >>> We encourage you to play around - select dimension items for the left
> menu
> >>> and arrange dimensions from "Layout" in the top menu. Documentation
> can be
> >>> found here:
> >>>
> >>> http://dhis2.org/doc/snapshot/en/user/html/ch15.html
> >>>
> >>>
> >>>
> >>> - Data analytics Web-API resource: A flexible API for retrieving
> aggregated,
> >>> analytical data for all data dimensions using a simple query syntax.
> The
> >>> pivot table module is built on top of this API. It allows developers of
> >>> third-party apps and web portals to connect to DHIS 2 and query for
> any type
> >>> of aggregated data, where any number of dimensions and filters can be
> >>> specified. Data can be retrieved in json, jsonp, xml, csv, html and xls
> >>> (Excel) format and responses can be constrained with criteria like
> greater
> >>> than, equals and so on.
> >>>
> >>> A query for retrieving data for certain data elements, periods and
> >>> organisation units might look like this:
> >>>
> >>>
> http://apps.dhis2.org/demo/api/analytics?dimension=dx:fbfJHSPpUQD;cYeuwXTCPkU&dimension=pe:2012Q1;2012Q2&dimension=ou:O6uvpzGd5pu;lc3eMKXaEfw
> >>>
> >>> Documentation can be found here:
> >>>
> >>> http://dhis2.org/doc/snapshot/en/user/html/ch25s18.html
> >>>
> >>>
> >>>
> >>> - Sharing: Several objects including chart, pivot table, map,
> indicator and
> >>> data set can now be shared and have fine-grained access control. An
> object
> >>> can be made private to yourself, shared with any number of user groups
> or be
> >>> made available for all. You can grant edit or view-only rights to other
> >>> people. You can create user groups directly from the dashboard. Who
> should
> >>> be allowed to create public objects can be controlled through user
> roles.
> >>>
> >>> This functionality is useful for example in a scenario where entities
> like
> >>> health programs and implementing partners need to create indicators and
> >>> reports which should be available only for members of their
> organisation. In
> >>> this case, an organisation can create a user group for their staff,
> then
> >>> develop charts and maps which could be made available for that
> organisation
> >>> only. General reports made by the ministry could still be shared with
> all
> >>> and made publicly available.
> >>>
> >>> Another use-case is that of an international NGO with operations in
> multiple
> >>> countries. Here, user groups could be set up for each country and
> >>> indicators, resources, charts and reports which are country-specific
> could
> >>> be made available for staff working in that country only, without
> appearing
> >>> in lists for staff working in other countries. Indicators and reports
> which
> >>> are applicable for all counties could be made accessible for all. The
> >>> sharing solution also opens up for great possibilities when combined
> with
> >>> the self-registration of user account feature.
> >>>
> >>> Documentation here:
> >>>
> >>> http://dhis2.org/doc/snapshot/en/user/html/ch07.html
> >>>
> >>>
> >>>
> >>> - DXF 2 meta-data exchange: You can now import and export a complete
> set of
> >>> objects in the aggregate part of the system, including data sets,
> custom
> >>> data entry forms, maps, charts and option sets. Import/export of
> meta-data
> >>> objects now uses the new DXF 2 format. Import/export can also be done
> >>> through the meta-data Web API resource. This greatly improves the
> ability
> >>> for meta-data exchange between DHIS 2 instances and third-party
> systems.
> >>>
> >>> Documentation here:
> >>>
> >>> http://dhis2.org/doc/snapshot/en/user/html/ch19.html
> >>>
> >>>
> >>> - HTML-based standard reports: You can now develop standard reports in
> pure
> >>> HTML and Javascript. This requires a little bit of development skills
> but
> >>> allows for creating pixel-perfect reports tailored to most needs. It is
> >>> especially flexible when used in combination with the analytics Web
> API,
> >>> where aggregated data can be pulled out of the system and positioned
> in the
> >>> report according to your needs.
> >>>
> >>> An example of such a report can be viewed here:
> >>>
> >>>
> http://apps.dhis2.org/demo/dhis-web-reporting/generateHtmlReport.action?uid=fqERdm6UtkI&ou=ImspTQPwCqd
> >>>
> >>> Documentation here:
> >>>
> >>> http://dhis2.org/doc/snapshot/en/user/html/ch14s04.html#d5e2332
> >>>
> >>>
> >>> - Multi-organisation unit data entry: You can now do data entry for
> multiple
> >>> organisation unit in the same form. This is useful for instance in
> scenarios
> >>> where you have section forms with few data elements and an
> organisational
> >>> hierarchy with hospitals with multiple wards below, as it allows you
> to do
> >>> fast data entry for all wards at the same time. To enable it, go the
> system
> >>> general settings and enable "multi-organisation unit forms", then go
> to data
> >>> entry and select the organisation unit above the ones you want to
> enter data
> >>> for.
> >>>
> >>> Documentation here:
> >>>
> >>> http://dhis2.org/doc/snapshot/en/user/html/ch09s02.html#d5e1122
> >>>
> >>>
> >>> - Tracker ad-hoc aggregate reports: You can now do on-the-fly
> aggregation of
> >>> tracker data in tabular reports. Define aggregation queries with
> filters or
> >>> group statements and view the results instantly. Create reports like
> top 10
> >>> diagnoses from inpatient admissions, save as favorite and run the
> report
> >>> again whenever you like. You can check it out here:
> >>>
> >>> http://apps.dhis2.org/demo/dhis-web-caseentry/app/index.html
> >>>
> >>>
> >>> - Tracker program statistics reports: Tracker now comes with better
> out of
> >>> the box statistics to program managers. Monitor new enrollments,
> programs
> >>> completed, and drop-outs. Also analyse in detail from each stage how
> many
> >>> visits are done, completed and overdue. Try it out on the demo here:
> >>>
> >>>
> http://apps.dhis2.org/demo/dhis-web-caseentry/statisticalProgramReportSelect.action
> >>>
> >>>
> >>> - Tracker general improvements: The tracker module is rapidly being
> improved
> >>> and the user experience is improved in person dashboard, case-based
> tabular
> >>> reports, person aggregation and program maintenance.
> >>>
> >>> The tracker user guide can be found here:
> >>>
> >>> http://dhis2.org/doc/snapshot/en/user/html/ch24s02.html#d5e5474
> >>>
> >>>
> >>> - Facility registry Web API resource: DHIS 2 now implements the API
> >>> specification made by the FRED project. This means that DHIS 2 can be
> >>> considered a feasible facility registry. More info here:
> >>>
> >>> http://facilityregistry.org
> >>>
> >>>
> >>> - Model changes: Indicator formulas, validation rule formulas and
> custom
> >>> data entry forms are now using UIDs (stable identifiers) which makes it
> >>> possible to transfer these easily between systems. Category options
> can now
> >>> be re-used within many categories which improves analysis and naming
> >>> conventions.
> >>>
> >>>
> >>> - Miscellaneous: Report table supports weekly periods. Data visualizer
> has
> >>> an improved favorite user interface, making it consistent with pivot
> table
> >>> and GIS. Touch-friendly navigation for help and profile items (click
> on help
> >>> or profile in top menu).
> >>>
> >>> http://apps.dhis2.org/demo/dhis-web-commons-about/functions.action
> >>>
> >>>
> >>> - The scheduling functionality has been improved. As a consequence you
> must
> >>> re-set your scheduled jobs after upgrading in data administration ->
> >>> scheduling.
> >>>
> >>>
> >>> WAR file, Live package, sample data and source code can be downloaded
> from
> >>> here:
> >>>
> >>> http://dhis2.org/downloads
> >>>
> >>>
> >>> Documentation and Javadocs can be found here:
> >>>
> >>> http://dhis2.org/documentation
> >>>
> >>>
> >>> Launchpad source code page:
> >>>
> >>> https://launchpad.net/dhis2/+milestone/2.11
> >>>
> >>>
> >>> Launchpad blueprint page:
> >>>
> >>> https://launchpad.net/dhis2/+milestone/2.11
> >>>
> >>>
> >>>
> >>> best regards,
> >>>
> >>> the DHIS 2 development team
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~dhis2-users
> >>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~dhis2-users
> >>> More help   : https://help.launchpad.net/ListHelp
> >>>
> >>
> >>
> >>
> >> --
> >> Paulo Grácio
> >>
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~dhis2-users
> >> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~dhis2-users
> >> More help   : https://help.launchpad.net/ListHelp
>



-- 
Alex Tumwesigye
Teaching Assistant,
Department of Electrical and Computer Engineering
College of Engineering, Design, Art and Technology, Makerere University

IT Consultant - BarefootPower Uganda Ltd, SmartSolar, Kenya

IT Specialist (Servers, Networks and Security) & Solar Consultant

+256 774149 775

"I don't want to be anything other than what I have been - one tree hill "