dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

Hi devs

I started the mydatamart project nearly 2 years ago and its reached a
certain point in its life where some people are using it and its
limitations are becoming clear.  The Liberia workshop highlighted a
particular concern related to the painfully slow metadata import, becoming
effectively unusable for large numbers of orgunits.  I am planning to spend
this week (only) on addressing some longstanding issues.

In the short term there are a few options we have discussed.

1. to restrict the piece of the orgunit tree which is downloaded as
metadata on the server side to user's subtree plus its peers and further
restricted to a certain level. So first question regarding the level: if we
want to leave the client exactly the way it is at the moment, can we
enforce a maximum level on the server side (eg 3 deep, or 4 deep)?  I
wonder how many existing mydatamart users will find this a breaking change.
 Or should it be parameterized, with defaults so existing mydatamart
clients continue to work as before.

2.  the current method of getting the metadata into the sqlite database is
via a (quite complicated) xslt transform which converts the dxf1 metadata
into sql .. generating the resource tables as it goes.  This could be made
quicker by (i) implementing a straight C/C++ parser and (ii) pushing the
orgunit metadata directly into the database and generating the
orgunitstructure after the fact.  Unfortunately I don't want to expend
extra effort on dxf1 so anything new like this would be much better done on
the newer dhis2 api.

Again I am considering doing this as a simple standalone tool to start with
while we plot the larger migration.  So users can carry on using mydatamart
as before, but run a little prog on the side to replace the painful priming
of the database metadata.

3.  The main mydatamart program is written in Tcl which is kind of cool
(though a bit of a mess) but I am also painfully aware that there is
probably me and 3 others in the world proficient in using Tcl and that
hasn't changed markedly in our project over the past 2 years.  So unless I
am to be sole maintainer of this program for the rest of my life, we have a
few choices:
(i) reimplement mydatamart in java, maybe as a swing app.  Java doesn't
have native access to sqlite but can probably reach it through the odbc
bridge.
(ii) reimplement mydatamart in QT.  A bit of effort to get the opensource
version working nicely on windows with MS Visual C++ compiler, but then a
nice slick environment for developers who might find QT more accessible
than Tcl (and it has strong Norwegian antecedents :-) ).
(iii) refactor the horrible tcl code into nicer tcl code using something
like snit or incrTcl and encourage new learners

Can we get some ideas of developer sentiment on the above?  Personally I
don't really like java desktop apps but it maybe the most pragmatic.  Qt
would probably be the nicest but, well its not java and might be harder to
get support from other programmers in our java oriented community.  But
otherwise it is really good, polished and well documented and quite easy to
get into.  (iii) is tempting but might just be me digging an even deeper
hole for myself :-)

Please give opinions.  The choice of 3 will have some impact on how I
approach 2.
Bob