yade-dev team mailing list archive

[Bruno Chareyre <bruno.chareyre@xxxxxxxxxxxxxxx>]

Hi there,

I'm planning to build new docker images in yade's gitlab for production, 
and possibly for development (see second part of this message, some 
background comes first).  This is open to suggestions.

* Background:

I recently started playing with "Singularity" images since I found our 
HPC department made it available on the clusters. There was also a user 
mentioning that on launchpad recently. From end-user POV, singularity 
images work like docker images, but a very practical difference is that 
it is allowed on our (and others') HPC. Docker isn't, for security reason.

It made running yade so easy. The above command worked immediately, and 
should work just the same on every system with singularity installed:
/ssh myHPC//
//singularity exec 
docker://registry.gitlab.com/bchareyre/docker-yade:ubuntu20.04-daily 
yadedaily --check/

or equivalently: /
///export YADE=//'singularity exec 
docker://registry.gitlab.com/bchareyre/docker-yade:ubuntu20.04-daily 
yadedaily'
$YADE --check
$YADE myScript.py
$ etc.
//

Key points:
1- singularity accepts docker images in input.
2- the above command is using some custom docker with yadedaily 
pre-installed (which then needs to be downloadable from somewhere where 
docker is permitted)
3- it is compatible with MPI(!). The host system's MPI is able to 
communicate with the image system's MPI in a scenario like this, as if 
it was just yade running natively on the host:
/mpirun -np 500 $YADE someParrallelStuff.py
/4- a condition for this MPI magic to work is that the mpi library is in 
the same version for the host and for the executed image
5- performance: no measurable difference compared to a yade compiled on 
the host (be it running -j1, -jN or mpiexec).

For the moment the custom dockers are built in [1] 
<https://gitlab.com/bchareyre/docker-yade>.
I'm also building a Singularity images with [2] 
<https://gitlab.com/bchareyre/yade-singularity/-/blob/master/.gitlab-ci.yml> 
but I didn't really use it since I can build it from docker directly on 
the cluster (building the singularity image is implicit in /singularity 
exec docker://.../). Building on-site may not be allowed everywhere, 
though, and in that case [2] could be useful.

* What can be done:

I will move [1,2] or something similar to gitlab/yade-dev and advertise 
it in the install page. Also build more versions for people to use them. 
More versions because of the MPI point above (4): depending on the host 
system someone may want OMPI v1 (unbuntu16), or v2 (ubuntu18), etc.

For production images it would make sense to just use canonical 
debian/ubuntu with yade and/or yadedaily preinstalled. But, it is not 
exactly what I did for the moment. Instead I used docker images from our 
registry. Which implies the images have yade, and also what it needs to 
compile yade (I didn't test compilation yet but it should work).

I was thinking of splitting that into two types of images; minimal 
images for production and "dev" images with all compilation 
pre-requisites. Then I realized that the best "dev" image would be - by 
far - one reflecting the state of the system at the end of our current 
pipeline, i.e. one with a full /build folder and possibly ccache info 
(if not too large).

If such dev images were pushed to yade registry then anyone could grab 
latest build and recompile incrementally. It could save a lot of 
(compilation) time for us when trying to debug something on multiple 
distros.

And what about that?: compiling with a ubuntu20 docker image on a 
ubuntu20 host should make it possible to use the pipeline's ccache while 
still running yade on the native system (provided that the install path 
is in the host filesystem).

Maybe pushing to registry could be done directly as part of current 
pipeline, not sure yet. I am still thinking about some aspects but I 
think you get the general idea. Suggestions and advices are welcome. :)

Chers

Bruno

[1] https://gitlab.com/bchareyre/docker-yade

[2] 
https://gitlab.com/bchareyre/yade-singularity/-/blob/master/.gitlab-ci.yml

/
/

/
/

/
/

/
/