dhis2-devs team mailing list archive

[Knut Staring <knutst@xxxxxxxxx>]

On Tue, Mar 31, 2015 at 1:16 PM, Jason Pickering <
jason.p.pickering@xxxxxxxxx> wrote:

> Hi Knut,
>
> To me it makes little sense to be quite honest. QGIS and other tools can
> do all of this, remarkably well.
>

Unfortunately, I have found this to NOT be the case when it comes to the
simplification/generalisation that Mapshaper does so nicely. I have yet to
find similar smooth functionality with either QGIS, OpenJump or PostGIS.
Probably ArcGIS has it, but most people don't have access to that. And
since this step is crucial, the idea was to leverage Mapshaper in its
Javascript reincarnation.


> Yes, it is maybe not the smoothest workflow, but on the other hand, does
> it really make sense to invest time and money into developing a custom tool
> for transformation, manipulation and export of geographical data?
>

Perhaps not, but this is an area that is pretty alien to most people, even
techies. But you are right that what usually happens is that someone like
you, Jan or I do it for them.


> What about another example... a tool to import Excel sheets, instead of
> CSV. Metadata import via CSV is supported, but why should we not direct
> import of Excel files? I am sure an app could be developed for this, but
> since Excel can do this so trivially, it would seem to really make little
> sense to do so.
>

Saving a sheet as CSV is trivial. As you probably knows better than anyone,
there are a lot of non-trivial tasks surrounding import of metadata and
legacy data. My hunch is that though these processes are quite resistant to
full autmation without some very serious AI built in, projects like Tableau
tell me that quite a lot can be done to ease the overall bootstrapping
process. But I do agree that guidance on best practices is probably going
to be even more useful.

This is my feeling as well with the geographical data. There are so many
> other things which we do not cover, such as the linkage between an existing
> hierarchy and a shape file, resolution of broken topology, cleansing of
> donuts holes which often result from generalization, etc. Will these be
> covered as well?
>

Yes, the app does actually aim to cover the linkage, and Mapshaper does a
much better job at the generalization (I've never felt a need to do any
post-cleaning).


> Also, in my experience, we do always  derive the hierarchy itself from the
> shape file (although in certain cases this may happen). How would the
> proposed tool address all of these use cases, and would it not be better to
> create detailed guidance on how to do these sorts of tasks with more
> general tools instead? After all, there are only so many countries.
>

That is a good point - there are no more than 200+ countries, so we could
actually prepare a set of DHIS2 OU import files for all countries a la
http://gadm.org/, maybe in collaboration with them. The same argument of
course goes for any other tool we would develop.


> I do not want to put a damper on the development of this app, as I think
> it could be useful to some, but feel it will be a lot of trouble to keep it
> up and maintained as the platform itself changes.
>

I would expect the hierarchy to be fairly stable, but of course it could be
built out to handle historical changes, and also extend the GIS module to
be more flexible, in which case the API might undergo changes also.

My two cents.
>

Thanks for your input!

Knut

>
> Best regards,
> Jason
>
>
> On Tue, Mar 31, 2015 at 1:02 PM, Knut Staring <knutst@xxxxxxxxx> wrote:
>
>> From an architectural view of DHIS2 as a platform, it is quite
>> interesting to see suggestions coming up about third party solutions such
>> as QGIS when there is already an attempt well underway to build on our own
>> app framework to enable this functionality within DHIS2. Especially when
>> this goes to the heart of configuring the platform, namely bootstrapping
>> the OU hierarchy, which forms the backbone of an implementation.
>>
>> There are several pieces to the puzzle: Reprojection,
>> Simplification/Generalisation, Linking to existing OU hierarchy. To me it
>> seems natural to want to pull this away from a series of command line steps
>> and external tools, but it is interesting to hear arguments in other
>> directions, also for Sushil's thesis. Feedback from the community is
>> important. Of course, we used to have such an external module with the
>> DataMart, but that was on the output side (not configuration)
>>
>> Knut
>>
>> On Tue, Mar 31, 2015 at 12:34 PM, Knut Staring <knutst@xxxxxxxxx> wrote:
>>
>>> My point is that mapshaper already reads the shapefile, and Sushil's app
>>> built around it already exists.
>>>
>>> You are right that some shapefiles are really heavy (thus the need for
>>> mapshaper).
>>>
>>> On Tue, Mar 31, 2015 at 11:50 AM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
>>> wrote:
>>>
>>>> That is a handy enough library for dealing with projection all right.
>>>> But doesn't really help in reading the shapefile, which *might* be a
>>>> bit of a heavy lift for an app dealing with dbf and shp files.  I'm
>>>> not convinced this would be a really good use of developer time.
>>>>
>>>> QGis already does all the grunt work of reading in the shapefile and
>>>> dealing with the various oddnesses.  Getting it to dump its output as
>>>> dhis orgunits seems like a relatively straightforward proposition.
>>>> But you can't assume everyone would want to use qgis.
>>>>
>>>> Maybe another approach would be to create a simple less geeky ui on
>>>> top of ogr2ogr.  dunno really.  I'm not seeing a silver bullet here.
>>>>
>>>> On 31 March 2015 at 10:12, Knut Staring <knutst@xxxxxxxxx> wrote:
>>>> > Much better use of developer time would be to include
>>>> http://proj4js.org/ in
>>>> > Sushil's app, so that everything needed comes in one sweet package.
>>>> >
>>>> >
>>>> > On Tue, Mar 31, 2015 at 11:10 AM, Knut Staring <knutst@xxxxxxxxx>
>>>> wrote:
>>>> >>
>>>> >> shp2gml is taken care of by ogr2ogr, as described in our manual.
>>>> >>
>>>> >> Yes, someone could write a QGis plugin, but why should they, since
>>>> we are
>>>> >> now working on a web app to do this. Granted, it does not (yet)
>>>> include
>>>> >> reprojection, but it does include the other crucial step of
>>>> generalisation
>>>> >> using http://www.mapshaper.org/.
>>>> >>
>>>> >> On Tue, Mar 31, 2015 at 9:53 AM, Bob Jolliffe <bobjolliffe@xxxxxxxxx
>>>> >
>>>> >> wrote:
>>>> >>>
>>>> >>> Of course shp2gml tools already exist - they could be described as a
>>>> >>> bit geeky using the command line.  Though there is need for some of
>>>> >>> those geeky options for dealing with projection systems and the
>>>> like.
>>>> >>>
>>>> >>> I think someone with some python know how could write a neat little
>>>> >>> QGis plugin to export to dhis2 format which *might* make it easier
>>>> for
>>>> >>> users.
>>>> >>>
>>>> >>> On 30 March 2015 at 18:59, Lars Helge Øverland <larshelge@xxxxxxxxx
>>>> >
>>>> >>> wrote:
>>>> >>> > Shape file import sounds like an ideal task for an external
>>>> developer,
>>>> >>> > as it
>>>> >>> > could transform the shapefile content and pipe it into the GML
>>>> importer
>>>> >>> > and
>>>> >>> > hence does not require lots of DHIS 2 knowledge.
>>>> >>> >
>>>> >>> > Lars
>>>> >>> >
>>>> >>> >
>>>> >>> > _______________________________________________
>>>> >>> > Mailing list: https://launchpad.net/~dhis2-devs
>>>> >>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> >>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> >>> > More help   : https://help.launchpad.net/ListHelp
>>>> >>> >
>>>> >>>
>>>> >>> _______________________________________________
>>>> >>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> >>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> >>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> >>> More help   : https://help.launchpad.net/ListHelp
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Knut Staring
>>>> >> Dept. of Informatics, University of Oslo
>>>> >> Norway: +4791880522
>>>> >> Skype: knutstar
>>>> >> http://dhis2.org
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Knut Staring
>>>> > Dept. of Informatics, University of Oslo
>>>> > Norway: +4791880522
>>>> > Skype: knutstar
>>>> > http://dhis2.org
>>>>
>>>
>>>
>>>
>>> --
>>> Knut Staring
>>> Dept. of Informatics, University of Oslo
>>> Norway: +4791880522
>>> Skype: knutstar
>>> http://dhis2.org
>>>
>>
>>
>>
>> --
>> Knut Staring
>> Dept. of Informatics, University of Oslo
>> Norway: +4791880522
>> Skype: knutstar
>> http://dhis2.org
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Jason P. Pickering
> email: jason.p.pickering@xxxxxxxxx
> tel:+46764147049
>



-- 
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
http://dhis2.org