← Back to team overview

dolfin team mailing list archive

Re: Status of parallel I/O

 

On Thu, Aug 25, 2011 at 09:59:44AM -0400, Garth N. Wells wrote:

> >>> How about using DOM everywherme and reserve the use of SAX for an
> >>> XML->HDF5 converter?
> >>>
> >>
> >> That could be OK, but if we have the to implement a SAX parser it's
> >> probably easiest to have it in DOLFIN anyway. I don't see the advantage
> >> over having the SAX parser with the io code.
> >
> > I agree we should keep it in DOLFIN, but if the only thing it needs to
> > do is extract data and spit out HDF5, I imagine it can be simpler than
> > the current parser since it doesn't need to be parallel. (?)
> >
>
> To make things clearer, I've just renamed the LocalMeshData parsers to
>
>   XMLLocalMeshDOM (was XMLLocalMeshData)
>
> and
>
>   XMLLocalMeshSAX (was XMLLocalMeshDataDistributed)

That's good.

> When XMLLocalMeshSAX is complete, it may be desirable to remove
> XMLLocalMeshDOM.

Either way is fine for me, as long as we decide which one to use. I
initially wanted to use SAX (as before) but the DOM looks easier and
may be enough if we plan to use HDF5 for large-scale problems anyway.
Or is it the case that DOM is a limitation even for medium sized
problems?

> I don't know what you mean by parallel - the XMLLocalMeshSAX works in
> the same way as the old parser (each process reading a chunk). I don't
> see how it can be made simpler by reading a XML file and then converting
> to HDF5. The steps that are there now will all still be required to read
> the XML mesh before writing a HDF file.

I don't know HDF, but I imagine one could write one single file and
HDF will handle parallel parsing of that file later. Then the
conversion script we write does not need to do anything parallel, just
read line by line and convert from one format to another.

So we would not need to worry about the splitting of data into chunks,
skipping lines etc in the SAX-based XML --> HDF converter. And so we
would have a minimal amount of simpe SAX-based code.

--
Anders


Follow ups

References