yade-mpi team mailing list archive

[François <francois.kneib@xxxxxxxxx>]

I'll give it a try this afternoon.

Le jeu. 13 juin 2019 à 13:08, Bruno Chareyre <bruno.chareyre@xxxxxxxxxxxxxxx>
a écrit :

> That's right, ipython engines give a viable alternative and it would
> probably enable more interesting features (for e.g. timing).
> Ok to try this.
> And yes, normal mpi execution should remain (that's what we have already,
> it just needs to not break it).
> Bruno
>
> On Thu, 13 Jun 2019 at 12:03, François <francois.kneib@xxxxxxxxx> wrote:
>
>> Hi,
>>
>> I dug this problem this morning, and it seems that the 2nd option is
>> ipython parallel as it avoids hard-drive writes/reads. Ipython parallel
>> allows communications to workers (aka engines) from a basic single-thread
>> ipython (aka communicator). The communicator can ask the workers to execute
>> a python script, a function, or even execute commands within the engines.
>> The engines can load mpi4py and fully use MPI. The idea would be to ask the
>> engines to load yade then execute the user script (as done in the current
>> yade-mpi).
>>
>> François
>>
>> Le jeu. 13 juin 2019 à 11:14, Bruno Chareyre <
>> bruno.chareyre@xxxxxxxxxxxxxxx> a écrit :
>>
>>> Hi,
>>> After speaking with François and turning the problems in various ways I
>>> feel that the idea of full interactivity combined with mpi execution is
>>> chimeric.
>>>
>>> The only realistic solution I come up with, would use the HD to
>>> communicate the scene to a mpi system call. O.mpirun() run would do
>>> something like this if called outside mpi context (i.e. interactively):
>>>
>>> O.save("tmp.yade")
>>> import subprocess
>>> subprocess.run(["mpiexec -n N yade --mpi -niter=1000 userscript.py"])
>>> O.load("tmp.yade")
>>>
>>> The "--mpi" option would trigger:
>>> O.load("tmp.yade") #if master
>>> O.mpirun() #the real, current, mpi behavior
>>> O.save("tmp.yade") #by master, after a mergeScene()
>>>
>>> It probably requires that the userscript encapsulates function
>>> definitions, that's a small constraint:
>>>
>>> def main():
>>>     def function():
>>>         ....
>>>
>>> if __name__ == "__main__":
>>>   main()
>>>
>>>
>>> Let me know if you see other alternatives.
>>>
>>> Bruno
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> _______________
>>> Bruno Chareyre
>>> Associate Professor
>>> ENSE³ - Grenoble INP
>>> Lab. 3SR
>>> BP 53
>>> 38041 Grenoble cedex 9
>>> Tél : +33 4 56 52 86 21
>>> ________________
>>>
>>> Email too brief?
>>> Here's why: email charter
>>> <https://marcuselliott.co.uk/wp-content/uploads/2017/04/emailCharter.jpg>
>>>
>> --
>> Mailing list: https://launchpad.net/~yade-mpi
>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~yade-mpi
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
> --
> --
> _______________
> Bruno Chareyre
> Associate Professor
> ENSE³ - Grenoble INP
> Lab. 3SR
> BP 53
> 38041 Grenoble cedex 9
> Tél : +33 4 56 52 86 21
> ________________
>
> Email too brief?
> Here's why: email charter
> <https://marcuselliott.co.uk/wp-content/uploads/2017/04/emailCharter.jpg>
> --
> Mailing list: https://launchpad.net/~yade-mpi
> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~yade-mpi
> More help   : https://help.launchpad.net/ListHelp
>