dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

Hi

I have been helping out in Rwanda with their dhis2 setup(s) and I want to
share a couple of issues to see whether some of these have been experienced
by others and whether we can put our collective heads together on the
trickier ones.

The setting is relatively complex in that they have a number of dhis2
instances.  The busiest one is the HMIS which is essentially the national
routine system which is collecting data from each facility (2466
dataelements in total).  There are a few other systems which collect
special data - such as the PBF system - also using dhis2.

Then they have a central datawarehouse/dashboard DHIS2 instance which
accumulates some data from each of these others.  In general it is a
restricted subset.  So for example there is a dataelement group in HMIS
which defines 268 dataelements to be exported to datawarehouse each month.
In addition it collects data from some external (non-dhis) systems.

So there are a number of challenges, some which are solved and some which
remain unsolved.  The first is to synchronize orgunits between them all
which we have had largely working for some time (if a bit clumsily).

The bigger one is keeping the dataelement+categoryXXX structures compatible
between them all.  I have written some simple scripts to extract/filter all
the relevant structural metadata for a dataelement group (its dataelements
and related categoryXXXX's).  So this set can be exported from one system
(eg HMIS) and imported into another (eg Datawarehouse).  This solves the
problem of routinely exporting datavalues from a dataelementgroup in HMIS
and reliably pushing to the datawarehouse without weird conflicts.  But ....

When you have another system (eg PBF) doing the same thing, it gets
hairier.  One problem we see is that the categoryoption lists on different
systems can have common elements (eg "Male") but they are independently
defined on both HMIS and PBF.  So if I export PBF metadata to datawarehouse
I will end up with two "Male" categoryOptions with different uids.  It
seems clear that within this small universe of dhis2 systems they will all
have to harmonize these things.  So it seems there are two options ..
(i) to abandon the project of distributed systems and to bring everything
together on one uber-instance with associated usergroups and access
controls, or
(ii) to setup a metadata repository instance from which all systems derive
their metadata and disable editing on the client systems.

(i) is tempting but I think it escapes the problem rather than solves it.
There are valid reasons to maintain these systems separately and there will
always be similar cases.  So we will probably do (ii).  If dhis2 cannot
interoperate with itself it augurs badly for its interoperability in
general.

Has anybody done something similar?  Large scale rationalization of
metadata will involve also the massaging of the datavalue tables to keep
everything true as the catoptcombos get reorganized.  Given that some of
the systems have been in full operation for over 3 years now there are the
usual messes which have accumulated over time amongst the categoryXXX
structures which will need to be addressed anyway to arrive at a pristine
state.  Maybe there are some nice scripts for this?

Things like sharing settings on categoryoptions provide a bit of a puzzle,
but also possibly something useful.  Presumably we can set them to be
completely readonly on the metadata client systems.

Bob