dolfin team mailing list archive
-
dolfin team
-
Mailing list archive
-
Message #24274
Re: Status of parallel I/O
On 24 August 2011 15:21, Garth N. Wells <gnw20@xxxxxxxxx> wrote:
>
>
> On 24/08/11 08:55, Kent-Andre Mardal wrote:
> >
> >
> > On 24 August 2011 14:43, Garth N. Wells <gnw20@xxxxxxxxx
> > <mailto:gnw20@xxxxxxxxx>> wrote:
> >
> >
> >
> > On 24/08/11 08:07, Anders Logg wrote:
> > > On Wed, Aug 24, 2011 at 07:22:53AM -0400, Garth N. Wells wrote:
> > >>
> > >>
> > >> On 24/08/11 03:50, Anders Logg wrote:
> > >>> What is the plan for XMLLocalMeshData (using the DOM interface)
> vs
> > >>> XMLLocalMeshDataDistributed (using the SAX interface)?
> > >>>
> > >>
> > >> Both for the time being.
> > >>
> > >>> Reading boundary indicators is currently failing with
> > >>>
> > >>> RuntimeError: *** Error: Inconsistent state in XML reader: 6.
> > >>>
> > >>> Should this be fixed in XMLLocalMeshDataDistributed or is the
> > plan to
> > >>> replace it with XMLLocalMeshData?
> > >>>
> > >>
> > >> In XMLLocalMeshDataDistributed.
> > >
> > > Could you elaborate? The functionality for reading and distributing
> > > boundary markers (in parallel) is currently broken and we want to
> fix
> > > it. But we need to know more about the design. I don't want to fix
> > > something if you decide to break it 5 min later.
> > >
> >
> > It never 'properly' worked in parallel. There were some messy ad hoc
> > changes made on top of functions that were planned for overhaul. I
> made
> > clear before this that parallel functionality was being sorted out
> (it's
> > not just in io, but also partitioning, etc), so the fact that it's
> not
> > working now should not be a surprise.
> >
> > There is a lot of missing functionality is parallel, so patience is
> > required to get things done properly.
> >
> >
> > It worked! And there was a unit test there that tested that it worked.
> > However, you use the term 'properly' here, probably to indicate
> something.
> >
>
> I could have written that it 'ran' ;).
>
> It also wasn't a unit test - it was just one of the regression tests
> placed in the unit test directory.
>
I placed it there because the unit tests were ran in parallel. And it was
the parallel stuff that
I wanted to test. But it might not be a unit test.
>
> > What is the way forward here other than patience?
>
> It won't be long - hopefully in the next month.
>
> > It seems to me that
> > the only thing that is actually broken is the parallel io.
> >
>
> If it was just io, that would be easy. The more challenging part is to
> address the global-local numbering in the Mesh class, and distributing
> this data around. The old code that 'ran' had ad hoc lines added in the
> partitioning code. The partitioning function was one huge function that
> failed for some cases (like graphs with no vertices on a process). It's
> been straightened out and broken up to a degree, and bugs fixes made,
> but this involved removing the ad hoc lines.
>
> Garth
>
>
If the io is working I, I think I can make the other parts work, although
probably not in
the 'proper' sense :)
Kent
> >
> >
> >
> >
> >
> > > Should we continue to use libxml2? Why not use the DOM parsing all
> the
> > > way?
> > >
> >
> > Because I'm inclined to keep SAX parsing for meshes since meshes are
> the
> > most likely to be created externally, and need to be scalable for
> > reading. Other objects (e.g., vectors) are likely to be created and
> > written by DOLFIN, so will eventually use parallel HDF5 for scalable
> > parallel io.
> >
> > > What is the difference between XMLLocalMeshData and
> > > XMLLocalMeshDataDistributed etc.
> > >
> >
> > Initially I planned to use DOM for all, but as outlined above decided
> > after some testing to retain SAX for meshes (but update to SAX2,
> since
> > the libxml2 SAX parser is deprecated and has memory leaks). Hence,
> > XMLLocalMeshData uses DOM and XMLLocalMeshDataDistributed uses SAX.
> So
> > far I've kept the DOM version since it's easy to code and could be
> > useful when reading non-distributed meshes on each process (which may
> > differ on different processes).
> >
> > Garth
> >
> > > --
> > > Anders
> > >
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dolfin
> > Post to : dolfin@xxxxxxxxxxxxxxxxxxx
> > <mailto:dolfin@xxxxxxxxxxxxxxxxxxx>
> > Unsubscribe : https://launchpad.net/~dolfin
> > More help : https://help.launchpad.net/ListHelp
> >
> >
>
>
References