dolfin team mailing list archive

[Anders Logg <logg@xxxxxxxxx>]

On Wed, Jun 23, 2010 at 11:54:28AM +0200, Johannes Ring wrote:
> On Wed, Jun 16, 2010 at 11:17 AM, Anders Logg <logg@xxxxxxxxx> wrote:
> > On Mon, Jun 07, 2010 at 01:24:02PM +0200, Anders Logg wrote:
> >> On Wed, May 05, 2010 at 10:32:07AM +0200, Anders Logg wrote:
> >> > On Wed, May 05, 2010 at 09:11:21AM +0100, Garth N. Wells wrote:
> >> > >
> >> > >
> >> > > On 03/05/10 14:04, Anders Logg wrote:
> >> > > >I've been going through all benchmarks and made changes to get
> >> > > >everything to compile and run properly. I've also made adjustments to
> >> > > >some parameters to get reasonable running times.
> >> > > >
> >> > > >I'm currently working on the parallel speedup benchmark (fem/speedup)
> >> > > >but then I think we should be in pretty good shape (for now).
> >> > > >
> >> > > >It would be good for everyone that has any interest in the benchmark
> >> > > >suite to take a look now at all the benchmarks and see if something
> >> > > >can be improved. As mentioned before, Johannes will backport at least
> >> > > >some of the benchmarks to earlier versions and then it becomes very
> >> > > >important that we keep the benchmarks fixed. We can't artibrarily go
> >> > > >in and change say a mesh size or form since that would break the
> >> > > >history.
> >> > > >
> >> > > >So speak now or forever hold your peace.
> >> > > >
> >> > >
> >> > > I won't be able to look at the benchmarks for a while, but I would
> >> > > like to have benchmarks to cover some areas where I know DOLFIN
> >> > > could do with some performance improvements. Off the top of my head
> >> > > two areas are:
> >> > >
> >> > > - Assembly of forms with many coefficients (calls to eval(..) are slow)
> >> >
> >> > It is easy to add more test cases to bench/fem/assembly/cpp/. Ideally,
> >> > they should be added before recording historical data. Both individual
> >> > and total running time is recorded so only the total time would be
> >> > messed up if we add a new test case, but if we want historical timings
> >> > for handling of coefficients (which might be important since that is
> >> > something we have been changing around) we should add it now.
> >> >
> >> > Do you have any particular case(s) in mind? It's very easy to add so
> >> > we could do it quickly.
> >> >
> >> > > - Repeated solution of linear systems (reuse of preconditioners,
> >> > > reuse of symbolic factorisation, etc)
> >> >
> >> > Those are new test cases so they can just be added when we feel like
> >> > it (but we might not get historical data).
> >>
> >> Does anyone have any more test cases to add? If not, it might be a
> >> good time to ask Johannes to look at the tests and try to backport
> >> them to earlier version.
> >>
> >> I am semi-happy with our current set of benchmarks. If anyone wants to
> >> add something, please do it now (or announce that you plan to add
> >> something).
> >>
> >> Here's the current list:
> >>
> >>   http://www.fenics.org/bench/
> >>
> >> All benchmarks are available in dolfin/bench.
> >
> > ok, so let's consider the current set of benchmarks "good enough" for now.
> >
> > So I'd say we freeze the current set of benchmarks (more can be added
> > later but the current set should not be touched).
> >
> > I will ask Johannes to look at backporting the benchmarks to earlier
> > DOLFIN versions to run on the buildbot.
>
> Most of the benchmarks are now backported for all DOLFIN releases back
> to 0.7.3. The result can be seen at
>
>    http://www.fenics.org/bench/
>
> Johannes

Very nice and very useful!

It looks like we have had quite a few regressions since 0.7.3:

https://blueprints.launchpad.net/dolfin/+spec/benchmark-regressions

I've asked Johannes to investigate the reasons for the regressions
(and perhaps find some improvements). But everyone is welcome to
help out.

--
Anders

Attachment: signature.asc
Description: Digital signature