dhis2-devs team mailing list archive

[Knut Staring <knutst@xxxxxxxxx>]

Thanks, Jason - many useful points - it may well be that import
through GML should be the expanded as the main mode for dealing with
"external" data. I would like to summarize what I am getting at in
terms of parameters.

The whole left hand pane in the OHM is devoted to allowing the user to
set the parameters of thematic layers, in terms of selecting the data
to be displayed. These parameters are then used to define queries to
the DHIS2 webservice API. I think it could be of great value if they
could additionally specify queries to other services (as alternative
modes). Geoserver is one obvious candidate, but I agree that there is
a lot of value in abstracting from it to standards such as WMS and WFS
( GML, GeoJSON). However, in the very long term, someone could even
wish to implement the new ArcGIS Server 10 API (most of the ESRI
client is based on the open source Dojo javascript framework).

The way I understand the current client works is that the parameter
dropdowns are populated via the webservice API. The shapes are
represented as GeoJSON, which can come from the DHIS db, a file, or
WFS (e.g. Geoserver), and the data values come from the DHIS2 API, and
is then merged with the GeoJSON on the client. It seems to me like it
could be possible to have the data values already being merged with
GeoJSON coming from a WFS server (while continuing to populate the
metadata/parameters as now).

Finally, let me revise my GIS wishlist, prioritized according to what
I see as most urgent at the top:

1) Points displayed with different sizes or graphical symbols
2) Showing a lot of data for a point when clicking on it
3) Handling unique (text) value data (typically from surveys)
4) Support for multidimensional data elements and indicators
5) Alternative datasources for thematic layers
6) Data import from surveys etc. with automatic assignment of points
7) Time slider
8) Access control

Blueprints need to be created for some of these. Any missing/more
important issues?

Knut

On Tue, Nov 2, 2010 at 10:01 AM, Jason Pickering
<jason.p.pickering@xxxxxxxxx> wrote:
> Hi there. Not entirely sure what the problem with using MapFish and
> Geoserver are actually. There are plenty of examples of how the two
> can be used together. After all MapFish is simply a client, Geoserver
> simply a server capable of producing formats that MapFish can render.
>
> I think the meat of Knuts suggestion is simply that OHM should be able
> to 1) consume and 2) integrate data sources from other sources. I see
> that the first is essentially already done. It could be a bit more
> simple to add WMS layers (have not checked this for a while, perhaps
> it is easier). Integration of external data is a much tricker subject,
> but with the work that has been done with importing XML streams/SDMX,
> I am not really sure why it could not be done. After all, Geoserver is
> fully capable of exporting data as GML, which is just XML. If a
> seperate transform (XSL-T) needed to be develop to transform GML to
> SDMX/DXF, this would seem to be feasible.
>
> For me the major use case for integration is the importation of of
> data from external parties. The major use case I can think of is the
> importation of data from a data provider that may use "complex" data
> sources, such as remote sensing data, to produce risk maps, or
> predictions of rainfall, which are often linked to malaria incidence
> for instance ( I am thinking about NASA, IRI, and other meterological
> institutes). For instance, a data provider could provide a predicted
> level of malaria incidence for each district in a country, publish
> this data through Geoserver or other WFS server, and DHIS/OHM could
> then consume this data in the form of GML. The production of this type
> of data would usually well beyond the capabilities of most clients of
> DHIS2 to produce, but they could certainly use the data if 1) it was
> published in a standards based format and 2) DHIS2 could make sense of
> it. I am not sure we should focus so much on the integration with
> Geoserver, but more with the ability to import GML as an XML stream,
> which Geoserver or other WFS sources (such as ArcGIS server,
> Mapserver, etc) are really good at producing.
>
>
>
> As for the issue with PostGIS, again,I think at some point, people
> will start to develop their own queries using PostGIS. With the query
> functionality of DHIS2, it should be possible to develop queries like
> "Give me all health centers with a utilization rate lower than 50% who
> are within 10 km of a facility that has a utilization rate greater
> than 100% for the year 2010.". This type of query would help to
> reallocate resources from clinics with high load, to those with lower
> loads. However, I suspect that these types of queries are going to be
> very ad-hoc, very specific to certain situations. The major advantage
> that I have encountered using PostGIS is the ability to create spatial
> views, which are treated by Geoserver as a seperate layer. No need to
> maintain multiple layers, simple write a view, and you have the layer
> that you want. Now of course, having some nice user-friendly query
> interface to do this type of stuff, would be nice. But as we all know,
> the current query interface is only sufficient for all but the most
> simple of data base queries. Once people need other views of the data,
> they have to develop them themselves (e.g. the PivotSource queries).
>
> I agree with all of Knut's points, but it always comes down to
> resources and priorities to implement what other clients have already
> done (e.g. a Time slider in Google Earth, ESRI's web framework).
>
> Regards,
> jason
>
>
> On Mon, Nov 1, 2010 at 6:05 PM, Jan Henrik Øverland
> <janhenrik.overland@xxxxxxxxx> wrote:
>> There are some good suggestions here, like the general idea of making OHM
>> more susceptible to other data sources. However, relying as much on
>> Geoserver as mentioned here basically means starting from scratch in our
>> case as OHM is based on MapFish. Also, I think using PostGIS violates the
>> DHIS principle of database independency.
>> On Sat, Oct 30, 2010 at 20:55, Knut Staring <knutst@xxxxxxxxx> wrote:
>>>
>>> Forwarding to the developer list
>>>
>>> On Thu, Oct 28, 2010 at 3:47 PM, Jan Henrik Øverland
>>> <janhenrik.overland@xxxxxxxxx> wrote:
>>> >> From: Knut Staring <knutst@xxxxxxxxx>
>>> >> To summarize what I currently see as the next major GIS improvements:
>>> >>
>>> >> 1) Points displayed with different sizes or graphical symbols, not
>>> >> just colored circles
>>> >>
>>> >> 2) Showing a lot of data for a point when clicking on it (typical use
>>> >> case is showing a kind of profile for a health facility, for example
>>> >> all values for a special indicator group as a first go).
>>> >>
>>> >> 3) Geoserver as alternative datasource for thematic layers -
>>> >> substantial expansion of available functionality
>>> >>
>>> >> 4) Support for multidimensional data elements and indicators
>>> >>
>>> >> 5) Access control like rest of DHIS. Not everyone should be able to
>>> >> set administrative settings
>>> >>
>>> >> 6) Reenabling the export to Excel and PDF (why did they disappear?)
>>> >>
>>> >> 7) Data import from surveys etc. - with optional automatic assignment
>>> >> of points based on coordinates
>>>
>>> Adding a point 8): We currently represent time as just another
>>> dropdown box. It would be really nice to have a time slider like e.g.
>>> Instant Atlas, ArcGIS or this (flash) example :
>>> http://labs.slate.com/articles/slate-job-map-unemployment-rate/
>>>
>>> In general in DHIS2, we may need to improve our handling of time
>>> series outputs, perhaps even for input. Point 2) can be related to
>>> time series. The user should also be able to relatively easily
>>> calculate *differences*, e.g. between one month and the next, or even
>>> from March 2009 to March 2010. Such differences are interesting to
>>> map, and can show which areas seems to be improving for a particular
>>> indicator, as in this example:
>>> http://drop.io/inup25f/asset/unemp-change-png
>>>
>>> > 1, 2 and 5) Covered by blueprints for 2.0.6 and 2.0.7.
>>>
>>> Good. We may need to update them as we progress.
>>>
>>> > 3) Geoserver is already an alternative datasource for the thematic
>>> > layers in
>>> > OHM. When it comes to expansion of functionality it would be nice if
>>> > someone
>>> > that knows Geoserver well could summarize what we could take advantage
>>> > of,
>>> > whether it requires special database add-ons like PostGIS etc.
>>>
>>> Right, Geoserver can act as a Web Feautre Service which outputs
>>> GeoJSON, so it is possible to upload shapefiles to Geoserver instead
>>> of importing them (using GML) to the orgunit table. However, I was
>>> thinking of adding a supplementary mode, which would allow for
>>> connection to external data (e.g. in an Excel file or an RDBMS).
>>>
>>> I see two possibilities for such an alternative mode:
>>>
>>> 1) The user's dropdown selections would be communicated to Geoserver,
>>> and Geoserver provides the styling, rather than the styling happening
>>> in the client, like in the previous OH functional prototype. However,
>>> I suppose it would mean that the dropdowns would also have to be
>>> populated from the external datasource? Needs more thought.
>>>
>>> 2) The GeoJSON file that is generated by Geoserver (by linking to
>>> DHIS2 or other data sources) can contain all the relevant data, and
>>> the dropdown box selections will then only be filtering (if there are
>>> more than one indicator) and styling in the client.
>>>
>>> Using PostGIS is in many ways the best option, especially since we are
>>> recomending Postgres now. I think we could require it for some
>>> advanced functionality. However, Geoserver will work nicely with
>>> shapefiles.
>>>
>>> I also think we have to consider how to deal with 4), as I can imagine
>>> people will want to see a map Males vs a map of Females, for example.
>>>
>>> > 6) PDF was removed because of the presence of image export. With the PDF
>>> > functionality followed a lot of code, dependecies, libraries, a print
>>> > servlet and a widget in the already crowded left side of the viewport.
>>> > The
>>> > image export solution is a lot neater and, in my opinion, covers the
>>> > necessary functionality.
>>> > Two months back there were some problems with the image exportation.
>>> > Both
>>> > image and Excel export were temporarily removed (the Excel sheet
>>> > contains an
>>> > image). When the problem was fixed there was a consensus that the Excel
>>> > sheet is currently not very valuable. We may put it back in if you want.
>>>
>>> Ok, thanks for the explanations - I think we can keep it as it is,
>>> especially if the removal also reduced the size of the OHM
>>> considerably, since it is by far the heaviest module.
>>>
>>> I liked the combination of having the data along with the map as
>>> export, but I am not sure if this is a real use case (people may
>>> prefer a full export or a pivot table). If it is, we could just export
>>> the data as CSV.
>>>
>>> Point 7, on importing external data is in a way an alternative to the
>>> Geoserver mode I discuss above. I think both options may need to be
>>> developed, since the particular use case would favor one or the other.
>>>
>>> Knut
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
>
> --
> Jason P. Pickering
> email: jason.p.pickering@xxxxxxxxx
> tel:+260968395190
>



-- 
Cheers,
Knut Staring