yade-dev team mailing list archive

[Anton Gladky <gladky.anton@xxxxxxxxx>]

Alexander has really found a source of problems.
OpenBlas is really a problem here [1]. It was fixed in Debian package
on 02 Mar 2013:
  * Disable processor affinity. (Closes: #684338)

It means, that Debian Wheezy as well as all Ubuntus 12.10 and older
are affected by the bug. We can disable the linkage of openblas
for these affected distros.

[1] https://bugs.debian.org/684338
[2] http://metadata.ftp-master.debian.org/changelogs/main/o/openblas/unstable_changelog

Anton


2014-02-28 11:47 GMT+01:00 Eulitz, Alexander
<Alexander.Eulitz@xxxxxxxxxxxxxxxx>:
> yes, and it can be assumed that some modules that are imported in yade could be the reason fort he problem:[1]
>
> "After some more googling I found the answer here.
> It turns out that certain Python modules (numpy, scipy, tables, pandas, skimage...) mess with core affinity on import. As far as I can tell, this problem seems to be specifically caused by them linking against multithreaded OpenBLAS libraries.
> A workaround is to reset the task affinity using
> os.system("taskset -p 0xff %d" % os.getpid())
> With this line pasted in after the module imports, my example now runs on all cores.
> There are also two ways to disable the CPU affinity-resetting behaviour of OpenBLAS itself. At run-time you can use the environment variable OPENBLAS_MAIN_FREE (or GOTOBLAS_MAIN_FREE), for example
> OPENBLAS_MAIN_FREE=1 python myscript.py
> Or alternatively, if you're compiling OpenBLAS from source you can permanently disable it at build-time by editing the Makefile.rule to contain the line
> NO_AFFINITY=1
> "
>
> Are we compiling OpenBLAS from source?
> But even if we are, why does the cpu affinity problem appear even for a built from late 2013 which definitely worked well concerning multicore operation?
> Maybe it was an update oft he openblas modul that was done by apt-get update?
>
>
> [1] http://stackoverflow.com/questions/15639779/what-determines-whether-different-python-processes-are-assigned-to-the-same-or-d?lq=1 answer#25