dhis2-devs team mailing list archive

[Jason Pickering <jason.p.pickering@xxxxxxxxx>]

Hi Johan and Bob,


Johan, you are indeed correct that the generalization process may
remove the "cartographic intricacies", but this is very likely because
the generalization is performed either on geographical data where
there is no topological relationships between objects, or the
generalization process does not respect the topology when it is
performed.

It would be possible to generalize a given set of polygons without
affecting their intrinsic topological relationships, but much more
care needs to be exercised when the generalization is performed. This
generalization could take place by removing unnecessary points
(simplification) and/or by reducing the precision of the data.

Ultimately the point in doing this is to decrease the "bulk" of the
data that is presented to the client. I can imagine that a data set of
100 points with 15 decimals would behave more or less the same as a
dataset of 1000 points with 6 decimals (just guessing here). My point
is that there is a certain payload associated with each dataset.
Typically, server side processing in the form of processing the GIS
layer to an image would be employed. However, since we are using
vector data on the client side, the data should  be preprocessed in
order to preserve these cartographic details that are important, as
automated simplification routines normally do not handle this. The
result being that the payload of the layer has been decreased to a
point that is "acceptable" to users. I am sceptical about whether this
step will be possible to automate at all for reasonably complex
polgyon layers (i.e districts) that DHIS typically deals with.

I want to come back to the use of  of DHIS as a repository. At this
point, IMHO, DHIS seems to be not appropriate for a health facility
repository. There is no way to adjust the metadata of a given
organizational unit object easily. I suppose we could use things like
orgunit groups to provide some type of metadata, but for instance, we
may want each orgunit to have a property such as "Address", "Fax" or
"Elevation". Additionally  the  proposed clipping of precision further
complicates matters in this regard. Ultimatley, we want a quick
responsive map for users as the first priority, and we should set our
sites on this.


In summary, I think that the current approach that we have, namely a
recommended workflow of how to preprocess a given set of data should
not be supplanted by the system itself truncating precision of
coordinates. There are many different generalization algorithms, each
with their pros and cons. Additionally, the generalization is highly
dependent on the scale of the map, and ultimately the pixel size of
the users screen, implying that different datasets may need to be
generalized in different ways depending on their scale. A gory detail
of how this done by Geoserver (using Geotools) is here
http://docs.geoserver.org/stable/en/user/tutorials/feature-pregeneralized/feature-pregeneralized_tutorial.html.
  We certainly do not need to recreate GeoTools or Geoserver, as they
are very good already at what they do. I would say instead that we
should consider leveraging these tools instead, and letting them
decide how to generalize or not generalize features, depending on the
scale of the map that is requested by the users. I guess I am
expressing some fundamental gripe, that we should not baby users too
much. If people want to have 15 decimals, well let them. They may have
reasons for this. It obviously does not make much sense, any more than
using 50,000 points to represent a simple polygon that could be
represented with four vertices. In both cases, the GIS guys need to do
their work and understand what type of data is required by the client.
Providing clear recommendations for a workflow  coupled with
guidelines on what a "reasonable" payload to the browser would be,
e.g. 30kb versus 30MB for a given layer, would be the best way to go I
think.

Regards,
Jason





On Mon, Jul 26, 2010 at 2:11 PM, Lemarchand, Johan <lemarchandjo@xxxxxxx> wrote:
> Hi all,
>
> The number of decimals is not really the issue. If you use 6 decimals, it is already enough for the type of GIS application we are interested in. The use of 15 decimals will not change a lot the precision of your map and it is not really necessary.
> 0 decimal places = approx. 112 km (70 miles) (Precision depending on the latitude)
> 3 decimal places = approx 111 m (365 feet)
> 6 decimal places = < 0.3 m (< 1 foot)
>
> The maps used by the system are not that accurate anyway to be more precise than 6 decimal places because there are not very large scale maps (1:1 000 or 1:500). There are medium scale maps 1:50 000 or 1:100 000 or small scale maps.
>
> The issue is more the cartographic generalization and the fact that it is not preserving all intricate geographical or other cartographic details. It is necessary to run the generalization process in order to use the GeoJSON format, but it removes a lot of data and simplifies it as well. As a significant amount of data is lost in the process, the output files are not relevant regarding purpose and scale and the simplified GeoJSON files can't really be used in a GIS.
>
> Johan
>
>
>
> -----Original Message-----
> From: dhis2-devs-bounces+lemarchandjo=who.int@xxxxxxxxxxxxxxxxxxx [mailto:dhis2-devs-bounces+lemarchandjo=who.int@xxxxxxxxxxxxxxxxxxx] On Behalf Of Knut Staring
> Sent: 26 July 2010 10:17
> To: Bob Jolliffe
> Cc: dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Dhis2-devs] Info on GIS development
>
> On Mon, Jul 26, 2010 at 9:38 AM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
>> Hi Jason
>>
>> On 26 July 2010 04:49, Jason Pickering <jason.p.pickering@xxxxxxxxx> wrote:
>>> Hi Knut,
>>>
>>>> It may be that we want to use DHIS as both a repository with full
>>>> precision (though not ridiculously artifical ones like 15 decimal
>>>> lat/lon) and have a faster way of renderin. But for a repo, I think
>>>> something like PostGIS is in order. Or we could just store things as
>>>> GML...
>>>
>>> Well, this is really the issue. If DHIS is going to be a repository,
>>> any self-respecting GIS geek would not use it if the application
>>> clipped precision. Although a few meters is not significant in terms
>>> of rendering a map, it may cause havoc on certain datasets,
>>> particularly if there are topological relationships between different
>>> layers. If a facility is related topologically to a road network, and
>>> the point is shifted a few meters, this may result in disturbance of
>>> the topology between these layers, rendering DHIS useless as a
>>> repository. ogr2ogr is perfectly OK as long as we are not dealing with
>>> these types of layers, but as soon as we start to think about
>>> relationships to other layers, we need to be very careful about how
>>> the data is preprocessed.
>>
>> Would you suggest then that the best place to clip precision would be
>> when the data is retrieved from the database for the specific view/map
>> rendering, rather than prior to it being stored?
>>
>> This would render the current convenience of storing as a geojson
>> string redundant as we would need to process the string on checkout
>> anyway.
>>
>> Can anyone say what the precision is on the shapefiles prior to
>> ogr2ogr conversion  ie. are we introducing a new level of precision
>> here or is that 15 digit precision the precision of the source
>> shapefiles?
>
> Quoting myself:
>
> "Here is a comparison of what I get in GeoJSON vs GML (converting from the same
> shapefile):
> GeoJSON: 38.415412, 1.750212
> GML:        38.415411724082148,1.750212388592194"
>
> Both using ogr2ogr. So 6 vs 15 decimals.
>
> Knut
>
>> Bob
>>
>>>
>>>
>>>> We should be very conscient of not pushing the new, very simple
>>>> solution too far, for more complex functionality we should rather
>>>> employ Geoserver and PostGIS - and I still think this is the best
>>>> solution for a national repository. Our new way of storing orgunit
>>>> boundaries is a very small subset of such a full blown GIS solution,
>>>> but has the advantage of being simple, lightweight and portable.
>>>
>>> Agreed on both points, namely that the solution is lightweight and
>>> aimed at thematic mapping but other solutions would be more
>>> appropriate for use as a repository of GIS data.
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>>
>>> --
>>> Jason P. Pickering
>>> email: jason.p.pickering@xxxxxxxxx
>>> tel:+17069260025
>>>
>>
>
>
>
> --
> Cheers,
> Knut Staring
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>



-- 
Jason P. Pickering
email: jason.p.pickering@xxxxxxxxx
tel:+17069260025