dolfin team mailing list archive

[Anders Logg <logg@xxxxxxxxx>]

On Wed, Aug 24, 2011 at 09:14:55AM -0400, Garth N. Wells wrote:
>
>
> On 24/08/11 08:55, Anders Logg wrote:
> > On Wed, Aug 24, 2011 at 08:43:40AM -0400, Garth N. Wells wrote:
> >>
> >>
> >> On 24/08/11 08:07, Anders Logg wrote:
> >>> On Wed, Aug 24, 2011 at 07:22:53AM -0400, Garth N. Wells wrote:
> >>>>
> >>>>
> >>>> On 24/08/11 03:50, Anders Logg wrote:
> >>>>> What is the plan for XMLLocalMeshData (using the DOM interface) vs
> >>>>> XMLLocalMeshDataDistributed (using the SAX interface)?
> >>>>>
> >>>>
> >>>> Both for the time being.
> >>>>
> >>>>> Reading boundary indicators is currently failing with
> >>>>>
> >>>>> RuntimeError: *** Error: Inconsistent state in XML reader: 6.
> >>>>>
> >>>>> Should this be fixed in XMLLocalMeshDataDistributed or is the plan to
> >>>>> replace it with XMLLocalMeshData?
> >>>>>
> >>>>
> >>>> In XMLLocalMeshDataDistributed.
> >>>
> >>> Could you elaborate? The functionality for reading and distributing
> >>> boundary markers (in parallel) is currently broken and we want to fix
> >>> it. But we need to know more about the design. I don't want to fix
> >>> something if you decide to break it 5 min later.
> >>>
> >>
> >> It never 'properly' worked in parallel. There were some messy ad hoc
> >> changes made on top of functions that were planned for overhaul. I made
> >> clear before this that parallel functionality was being sorted out (it's
> >> not just in io, but also partitioning, etc), so the fact that it's not
> >> working now should not be a surprise.
> >
> > It came as a surprise since there was a unit test for it and the unit
> > test was removed. But nevermind, the important thing now is to get it
> > working again.
> >
> >> There is a lot of missing functionality is parallel, so patience is
> >> required to get things done properly.
> >>
> >>> Should we continue to use libxml2? Why not use the DOM parsing all the
> >>> way?
> >>>
> >>
> >> Because I'm inclined to keep SAX parsing for meshes since meshes are the
> >> most likely to be created externally, and need to be scalable for
> >> reading. Other objects (e.g., vectors) are likely to be created and
> >> written by DOLFIN, so will eventually use parallel HDF5 for scalable
> >> parallel io.
> >
> > The reason I once chose to implement the XML parsing using SAX, and
> > why Ola decided to use SAX in his rewrite 2 years back, is exactly
> > that: scalability and efficiency. I don't see why it should be
> > different for meshes than any other objects. Other objects can also be
> > large. It seems messy to use both.
> >
>
> XML (be it with SAX ad DOM parsers) is not a scalable or efficient
> solution. The scalable and efficient solution in binary + MPI. This will
> appear when time permits.

Sure, but I would claim SAX scales better. Wouldn't it be better to
just use one of DOM or SAX? Either we use SAX all the way if it gives
better performance than DOM, or we use DOM all the way as a solution
for medium sized problems and complement with HDF5 for large scale
problems. Having DOM + SAX + HDF5 seems messy.

> >>> What is the difference between XMLLocalMeshData and
> >>> XMLLocalMeshDataDistributed etc.
> >>>
> >>
> >> Initially I planned to use DOM for all, but as outlined above decided
> >> after some testing to retain SAX for meshes (but update to SAX2, since
> >> the libxml2 SAX parser is deprecated and has memory leaks). Hence,
> >> XMLLocalMeshData uses DOM and XMLLocalMeshDataDistributed uses SAX. So
> >> far I've kept the DOM version since it's easy to code and could be
> >> useful when reading non-distributed meshes on each process (which may
> >> differ on different processes).
> >
> > I don't understand the difference between XMLLocalMeshData and
> > XMLLocalMeshDataDistributed. Is XMLLocalMeshDataDistributed doing now
> > what XMLLocalMeshData did before?
> >
>
> Yes, but updated to SAX2 (which was very painful).
>
> The 'new' XMLLocalMeshData is a DOM version. It could be removed.

Or kept if we will add HDF5 anyway as a more scalable solution.

--
Anders