← Back to team overview

dhis2-devs team mailing list archive

Re: Some musings on architecture (Was: Possible issue with GML import)

 

On Tue, Mar 31, 2015 at 1:23 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> Knut, the nub of your argument seems to be to to implement shapefile
> import as an augmentation to dhis - something which I also saw Calle
> and others suggest in earlier thread.
>
> Its not necessarily a bad idea,


It is not just an idea, it has already been done, cf. Sushil's post.


> but my point is that I don't think
> this would be easy to do as an "app". I might of course be wrong about
> this.  But I can't see the processing of shapefiles (which are
> misleadingly called files) being easily done in javascript in the
> browser.


Mapshaper does exactly that: http://www.mapshaper.org/


> So I think you are going to have some additional java code
> in the core - not necessarily a lot, but some.
>

The only challenge will be size. I think the limit of the old, Flash
implementation was 80 MB. No such limit is displayed for the new one:
http://www.mapshaper.org/offline.html


> Someone like Jan might know better.
>
> Regarding a high level architecture view, I suppose the tension here
> is between simple tools which do what they do well but are not always
> easy to put together when you don't know what you are doing (the unix
> philosophy, and ogr approach) vs limitless growth to the web
> application and pumping-up-the-jam :-)  There are forces which drive
> both rationales.
>

Certainly. But in this case, we already have the Unix approach available,
and it's well documented, but still quite daunting for some users. What
Sushil is building is the "point-and-click" version, but better than that,
because we also leverage the DHIS Web API to link with existing OU (e.g
with alternative spellings)

 Knut

>
> On 31 March 2015 at 12:02, Knut Staring <knutst@xxxxxxxxx> wrote:
> > From an architectural view of DHIS2 as a platform, it is quite
> interesting
> > to see suggestions coming up about third party solutions such as QGIS
> when
> > there is already an attempt well underway to build on our own app
> framework
> > to enable this functionality within DHIS2. Especially when this goes to
> the
> > heart of configuring the platform, namely bootstrapping the OU hierarchy,
> > which forms the backbone of an implementation.
> >
> > There are several pieces to the puzzle: Reprojection,
> > Simplification/Generalisation, Linking to existing OU hierarchy. To me it
> > seems natural to want to pull this away from a series of command line
> steps
> > and external tools, but it is interesting to hear arguments in other
> > directions, also for Sushil's thesis. Feedback from the community is
> > important. Of course, we used to have such an external module with the
> > DataMart, but that was on the output side (not configuration)
> >
> > Knut
> >
> > On Tue, Mar 31, 2015 at 12:34 PM, Knut Staring <knutst@xxxxxxxxx> wrote:
> >>
> >> My point is that mapshaper already reads the shapefile, and Sushil's app
> >> built around it already exists.
> >>
> >> You are right that some shapefiles are really heavy (thus the need for
> >> mapshaper).
> >>
> >> On Tue, Mar 31, 2015 at 11:50 AM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
> >> wrote:
> >>>
> >>> That is a handy enough library for dealing with projection all right.
> >>> But doesn't really help in reading the shapefile, which *might* be a
> >>> bit of a heavy lift for an app dealing with dbf and shp files.  I'm
> >>> not convinced this would be a really good use of developer time.
> >>>
> >>> QGis already does all the grunt work of reading in the shapefile and
> >>> dealing with the various oddnesses.  Getting it to dump its output as
> >>> dhis orgunits seems like a relatively straightforward proposition.
> >>> But you can't assume everyone would want to use qgis.
> >>>
> >>> Maybe another approach would be to create a simple less geeky ui on
> >>> top of ogr2ogr.  dunno really.  I'm not seeing a silver bullet here.
> >>>
> >>> On 31 March 2015 at 10:12, Knut Staring <knutst@xxxxxxxxx> wrote:
> >>> > Much better use of developer time would be to include
> >>> > http://proj4js.org/ in
> >>> > Sushil's app, so that everything needed comes in one sweet package.
> >>> >
> >>> >
> >>> > On Tue, Mar 31, 2015 at 11:10 AM, Knut Staring <knutst@xxxxxxxxx>
> >>> > wrote:
> >>> >>
> >>> >> shp2gml is taken care of by ogr2ogr, as described in our manual.
> >>> >>
> >>> >> Yes, someone could write a QGis plugin, but why should they, since
> we
> >>> >> are
> >>> >> now working on a web app to do this. Granted, it does not (yet)
> >>> >> include
> >>> >> reprojection, but it does include the other crucial step of
> >>> >> generalisation
> >>> >> using http://www.mapshaper.org/.
> >>> >>
> >>> >> On Tue, Mar 31, 2015 at 9:53 AM, Bob Jolliffe <
> bobjolliffe@xxxxxxxxx>
> >>> >> wrote:
> >>> >>>
> >>> >>> Of course shp2gml tools already exist - they could be described as
> a
> >>> >>> bit geeky using the command line.  Though there is need for some of
> >>> >>> those geeky options for dealing with projection systems and the
> like.
> >>> >>>
> >>> >>> I think someone with some python know how could write a neat little
> >>> >>> QGis plugin to export to dhis2 format which *might* make it easier
> >>> >>> for
> >>> >>> users.
> >>> >>>
> >>> >>> On 30 March 2015 at 18:59, Lars Helge Øverland <
> larshelge@xxxxxxxxx>
> >>> >>> wrote:
> >>> >>> > Shape file import sounds like an ideal task for an external
> >>> >>> > developer,
> >>> >>> > as it
> >>> >>> > could transform the shapefile content and pipe it into the GML
> >>> >>> > importer
> >>> >>> > and
> >>> >>> > hence does not require lots of DHIS 2 knowledge.
> >>> >>> >
> >>> >>> > Lars
> >>> >>> >
> >>> >>> >
> >>> >>> > _______________________________________________
> >>> >>> > Mailing list: https://launchpad.net/~dhis2-devs
> >>> >>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>> >>> > Unsubscribe : https://launchpad.net/~dhis2-devs
> >>> >>> > More help   : https://help.launchpad.net/ListHelp
> >>> >>> >
> >>> >>>
> >>> >>> _______________________________________________
> >>> >>> Mailing list: https://launchpad.net/~dhis2-devs
> >>> >>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>> >>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>> >>> More help   : https://help.launchpad.net/ListHelp
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Knut Staring
> >>> >> Dept. of Informatics, University of Oslo
> >>> >> Norway: +4791880522
> >>> >> Skype: knutstar
> >>> >> http://dhis2.org
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Knut Staring
> >>> > Dept. of Informatics, University of Oslo
> >>> > Norway: +4791880522
> >>> > Skype: knutstar
> >>> > http://dhis2.org
> >>
> >>
> >>
> >>
> >> --
> >> Knut Staring
> >> Dept. of Informatics, University of Oslo
> >> Norway: +4791880522
> >> Skype: knutstar
> >> http://dhis2.org
> >
> >
> >
> >
> > --
> > Knut Staring
> > Dept. of Informatics, University of Oslo
> > Norway: +4791880522
> > Skype: knutstar
> > http://dhis2.org
>



-- 
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
http://dhis2.org

References