← Back to team overview

dolfin team mailing list archive

Re: Status of parallel I/O

 

On Thu, Aug 25, 2011 at 05:57:51PM -0400, Garth N. Wells wrote:
>
>
> On 25/08/11 16:53, Anders Logg wrote:
> > On Thu, Aug 25, 2011 at 09:59:44AM -0400, Garth N. Wells wrote:
> >
> >>>>> How about using DOM everywherme and reserve the use of SAX for an
> >>>>> XML->HDF5 converter?
> >>>>>
> >>>>
> >>>> That could be OK, but if we have the to implement a SAX parser it's
> >>>> probably easiest to have it in DOLFIN anyway. I don't see the advantage
> >>>> over having the SAX parser with the io code.
> >>>
> >>> I agree we should keep it in DOLFIN, but if the only thing it needs to
> >>> do is extract data and spit out HDF5, I imagine it can be simpler than
> >>> the current parser since it doesn't need to be parallel. (?)
> >>>
> >>
> >> To make things clearer, I've just renamed the LocalMeshData parsers to
> >>
> >>   XMLLocalMeshDOM (was XMLLocalMeshData)
> >>
> >> and
> >>
> >>   XMLLocalMeshSAX (was XMLLocalMeshDataDistributed)
> >
> > That's good.
> >
> >> When XMLLocalMeshSAX is complete, it may be desirable to remove
> >> XMLLocalMeshDOM.
> >
> > Either way is fine for me, as long as we decide which one to use. I
> > initially wanted to use SAX (as before) but the DOM looks easier and
> > may be enough if we plan to use HDF5 for large-scale problems anyway.
> > Or is it the case that DOM is a limitation even for medium sized
> > problems?
> >
>
> It works for 'medium' (very arbitrary) size problems.
>
> >> I don't know what you mean by parallel - the XMLLocalMeshSAX works in
> >> the same way as the old parser (each process reading a chunk). I don't
> >> see how it can be made simpler by reading a XML file and then converting
> >> to HDF5. The steps that are there now will all still be required to read
> >> the XML mesh before writing a HDF file.
> >
> > I don't know HDF, but I imagine one could write one single file and
> > HDF will handle parallel parsing of that file later. Then the
> > conversion script we write does not need to do anything parallel, just
> > read line by line and convert from one format to another.
> >
>
> It may not be possible to do it line-by-line (I don't know, but I
> wouldn't want to bank on it). Even if line-by-line is technically
> possible, it could turn out to be terribly slow. We should support that
> a mesh can be read into memory (distributed), and written to HDF5.
>
> Since we'll have support for writing HDF5 meshes, if we can read a large
> XML mesh then we can re-use the HDF5 output code to make the conversion.
>
> I've removed the DOM-based LocalMeshData parser - there is no point to
> it since we can just read the mesh on one process using XMLMesh and use
> it to construct a dolfin::XMLLocalMeshData object.

ok, looks good.

How should we store boundary indicators? I'm not sure whether it needs
to be stored as part of ParallelData. Is it really "parallel data"?
ParallelData will for sure need to be used to compute it (convert
somehow from the input) but it seems it can then be stored
locally. Each facet just needs to know its indicator value.

The input is a list of triples:

  (indicator, facet_cell, facet_number)

This indicates that local facet number `facet_number` of the cell
`facet_cell` should have the indicator value (sub domain number)
`indicator`.

This is currently store as three MeshFunctions and is converted to
a MeshFunction over facets ("exterior_facet_domains") upon reading in
the mesh (in serial).

--
Anders


Follow ups

References