← Back to team overview

yade-mpi team mailing list archive

Yade-MPI update


Hi all,
After another cycle of improvement (ending with [1]), important changes are: - the size of the messages is decreased by extracting only the necessary data (pos/vel) - some critical calls to send/receive that were previously done with mpi4py are now replaced by calls to c++ functions which contain equivalent OpenMPI instructions (*much* faster). - Yade now has to be compiled with mpicxx: first "export CXX='mpicxx g++' " then cmake as usual - scripts can import the new module "mpy" to use mpirun(). No mpi-specific lines need to exist in simulation scripts beyhond that. - previous examples are removed and replaced by testMPI_2D.py, which can be ran with or without mpiexec and writes timings in a file.
To run this script please check its preamble.

Encouraging results are obtained on the basis of testMPI_2D.py (arguably a mpi-friendly problem), using 4 cores on an ordinary desktop (6x Xeon E5-1650 @3.2GHz). The report can be found in examples/mpi/collect.dat (and summarized in [2]).
- MPI faster than single-thread as soon as the number of bodies exceeds 1000, peak peformance for ~3k particles per core;
- MPI outperforms OpenMP for all problem sizes larger than 1000;
- for 270k particles the MPI run is approaching the best theoretical performance, which is parallelTime = monolithicTime / no. of subdomains (since one thread is always reserved by master process, -n4 means 3 subdomains).

Below is Cundall's number (particle*steps per second) for various problem sizes (plotted from examples/mpi/collect.dat)

Known bug: the total force on the floor after 2000 iterations varies by 0.5% of the total between mpi and non-mpi run. Not small enough to be ok given that the rebound of a regular array on a floor should be rather deterministic. It will have to be investigated.

Best regards


[1] https://github.com/bchareyre/yade-mpi/commit/956551fcb379becdfeedf5b4b13b46bbb1052929
[2] http://perso.3sr-grenoble.fr/users/bchareyre/pubs/CundallNumber.png

Bruno Chareyre
Associate Professor
ENSE³ - Grenoble INP
Lab. 3SR
BP 53
38041 Grenoble cedex 9
Tél : +33 4 56 52 86 21
Fax : +33 4 76 82 70 43

Email too brief?
Here's why! http://emailcharter.org