yade-mpi team mailing list archive

[Bruno Chareyre <bruno.chareyre@xxxxxxxxxxxxxxx>]

Hi all,
After another cycle of improvement (ending with [1]), important changes 
are:
- the size of the messages is decreased by extracting only the necessary 
data (pos/vel)
- some critical calls to send/receive that were previously done with 
mpi4py are now replaced by calls to c++ functions which contain 
equivalent OpenMPI instructions (*much* faster).
- Yade now has to be compiled with mpicxx: first "export CXX='mpicxx 
g++' " then cmake as usual
- scripts can import the new module "mpy" to use mpirun(). No 
mpi-specific lines need to exist in simulation scripts beyhond that.
- previous examples are removed and replaced by testMPI_2D.py, which can 
be ran with or without mpiexec and writes timings in a file.
To run this script please check its preamble.

Encouraging results are obtained on the basis of testMPI_2D.py (arguably 
a mpi-friendly problem), using 4 cores on an ordinary desktop (6x Xeon 
E5-1650 @3.2GHz). The report can be found in examples/mpi/collect.dat 
(and summarized in [2]).
Namely:
- MPI faster than single-thread as soon as the number of bodies exceeds 
1000, peak peformance for ~3k particles per core;
- MPI outperforms OpenMP for all problem sizes larger than 1000;
- for 270k particles the MPI run is approaching the best theoretical 
performance, which is parallelTime = monolithicTime / no. of subdomains 
(since one thread is always reserved by master process, -n4 means 3 
subdomains).

Below is Cundall's number (particle*steps per second) for various 
problem sizes (plotted from examples/mpi/collect.dat)

Known bug: the total force on the floor after 2000 iterations varies by 
0.5% of the total between mpi and non-mpi run. Not small enough to be ok 
given that the rebound of a regular array on a floor should be rather 
deterministic. It will have to be investigated.

Best regards

Bruno


[1] 
https://github.com/bchareyre/yade-mpi/commit/956551fcb379becdfeedf5b4b13b46bbb1052929
[2] http://perso.3sr-grenoble.fr/users/bchareyre/pubs/CundallNumber.png

--
_______________
Bruno Chareyre
Associate Professor
ENSE³ - Grenoble INP
Lab. 3SR
BP 53
38041 Grenoble cedex 9
Tél : +33 4 56 52 86 21
Fax : +33 4 76 82 70 43
________________

Email too brief?
Here's why! http://emailcharter.org