yade-mpi team mailing list archive

[Deepak Kn <deepak.kn1990@xxxxxxxxx>]

Yes that was the case with the hackthon too. There is a small difference*
in the forces between the mpi version (with merge) and the serial version,
>From what I understood (from previous discussions), that this is due to the
lack of information on the interactions during the merge operation.
*20000 bodies serial : force value = -35759751.3195
*20000 bodies parallel : mpf : force value = -37733943.6400

On Tue, Apr 30, 2019 at 3:01 PM François <francois.kneib@xxxxxxxxx> wrote:

> In the case without merge, the results are exactly the same in comparison
>> to mpy.py (test2D_MPI.py, no merge.) . We need to add a a list of things to
>> be done (and which are already done ) in the gitlab issue page. I will
>> start his.
>>
> Yes, this is also what I have seen but I can't remember: was it already
> like that at the end of the hackathon ?
>
> Nice list !
> I will work on the second point: exchanging bodies with MPIBodyContainer.
> Even though at this time we don't care about the "when" question.
>
> Le mar. 30 avr. 2019 à 14:57, Deepak Kn <deepak.kn1990@xxxxxxxxx> a
> écrit :
>
>> Hello,
>>
>> So I have added a 'to do list', at gitlab based on my understanding of
>> yade-mpi and the previous discussions : (
>> https://gitlab.com/yade-dev/trunk/issues/68). What do you think?
>>
>> On Tue, Apr 30, 2019 at 2:07 PM Deepak Kn <deepak.kn1990@xxxxxxxxx>
>> wrote:
>>
>>> Hello,
>>>
>>> *Did you try a simulation with merge/split operations ?*
>>>
>>> I am working on the merge operation right now : The issue is once we
>>> send all the bodies to the master proc, the interactions of these bodies
>>> have to be set in to the master's interaction container, I'm looking at
>>> this now. So I have not tried any simulations with the merge/split
>>> operations.
>>>
>>> *are you able to get the right force on the bottom wall ?*
>>>
>>> In the case without merge, the results are exactly the same in
>>> comparison to mpy.py (test2D_MPI.py, no merge.) . We need to add a a list
>>> of things to be done (and which are already done ) in the gitlab issue
>>> page. I will start his.
>>>
>>>
>>> On Tue, Apr 30, 2019 at 1:42 PM François <francois.kneib@xxxxxxxxx>
>>> wrote:
>>>
>>>> Thanks deepak,
>>>>
>>>> I see that, in addition to merge(), you have moved most of the python
>>>> splitScene() into c++ Subdomain::findInterSections(). That's great, not the
>>>> easiest part ! So is that part of the code working ? Did you try a
>>>> simulation with merge/split operations ?
>>>> If I'm right we know communicate bodies/ids only, so the last (but not
>>>> least) question: are you able to get the right force on the bottom wall ?
>>>>
>>>> Le lun. 29 avr. 2019 à 15:21, Deepak Kn <deepak.kn1990@xxxxxxxxx> a
>>>> écrit :
>>>>
>>>>> Hello,
>>>>>
>>>>> Yes, checkout from the branch : mpi_dpk. : there's an example in
>>>>> examples/mpi/testnomerge.py
>>>>> also, there's another py module in py : py/yadempi.py (this is more or
>>>>> less similar to mpy.py but we send bodies instead of states).
>>>>> If you want to see the sizes of messages  : (size as in number of
>>>>> elems in the buffer)
>>>>> add this line  in Subdomain.cpp , function void Subdomain::
>>>>> recvContainerString after line 429 : std::cout << "sizes " <<
>>>>> recvdStringSizes[i] << std::endl;
>>>>> let me know if you have any troubles, also if you find some part of
>>>>> the code inefficient (or could be made better), go ahead and fix it!
>>>>>
>>>>> Thanks and regards,
>>>>> deepak
>>>>>
>>>>> On Mon, Apr 29, 2019 at 2:52 PM François <francois.kneib@xxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> Deepak, could you please provide an example script to test this
>>>>>> feature ?
>>>>>>
>>>>>> Le lun. 29 avr. 2019 à 13:39, François <francois.kneib@xxxxxxxxx> a
>>>>>> écrit :
>>>>>>
>>>>>>> Hi, and nice work Deepak.
>>>>>>> I would add that we can't compare directly the number of elements
>>>>>>> that are transmitted, but rather the total volume, as size of char is 8bits
>>>>>>> and size of double is 64bits by default on modern cpus.
>>>>>>> As a result, we should rather compare 4180*8=33440 with 215456,
>>>>>>> which still represents a factor of 10.
>>>>>>>
>>>>>>> Le lun. 29 avr. 2019 à 13:16, Bruno Chareyre <
>>>>>>> bruno.chareyre@xxxxxxxxxxxxxxx> a écrit :
>>>>>>>
>>>>>>>> Hi Deepak,
>>>>>>>> Clearly there is no reason to compare speed of sending pos+vel vs.
>>>>>>>> speed of sending the entire body, since these communications are not used
>>>>>>>> at the same stages.
>>>>>>>> Sending serialized bodies is an alternative to sending the whole
>>>>>>>> scene, and I guess (?) sending 5% of the bodies in a scene will take less
>>>>>>>> time than sending the scene.
>>>>>>>> The communication which occures at each iteration will always be
>>>>>>>> only pos+vel.
>>>>>>>>
>>>>>>>> @Janek, when we break a scene in multiple pieces we want to send
>>>>>>>> everything because we don't want to worry about what needs to be sent in
>>>>>>>> each other particular case, and actually if an attribute is there then we
>>>>>>>> can safely assume that there is a reason for it. When it's already broken
>>>>>>>> in pieces we update (communicate) only positions and velocity because
>>>>>>>> otherwise it's too expensive, as Deepak found, but it's not foolproof. For
>>>>>>>> instance differential growing of particles in each subdomain would break
>>>>>>>> this concept since radius is not communicated at every iteration.
>>>>>>>>
>>>>>>>> Bruno
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, 28 Apr 2019 at 19:58, Deepak Kn <deepak.kn1990@xxxxxxxxx>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello All,
>>>>>>>>>
>>>>>>>>> I have finished implementing the functions to serialize and
>>>>>>>>> deserialize the bodies and the necessary communications to exchange the
>>>>>>>>> serialized bodies. The serialized messages (binary) are several times
>>>>>>>>> larger than the usual (state) messages in mpy/mpf.py (vel,pos, ori, id,
>>>>>>>>> bounds) etc. And as a result the simulations are far slower ! Here's how it
>>>>>>>>> scales : for exchanging 220 bodies, the serialized messages were around
>>>>>>>>> 215456 elements (chars), and in the previous versions, the message sizes
>>>>>>>>> were 4180 (doubles). (testMPI2D no merge, on 3 procs, 20000 bodies.)
>>>>>>>>>
>>>>>>>>> Most of the functions are implemented in Subdomain.cpp
>>>>>>>>> Subdomain.hpp and MPIBodyContainer.hpp, (branch : mpi_dpk on gitlab) I
>>>>>>>>> followed the same logic of the previous version (mpy/mpf), non blocking
>>>>>>>>> sends and blocking recvs (with probes) are used for the communications
>>>>>>>>> between the workers, and blocking communication for sending the forces to
>>>>>>>>> the master (the same as in mpy/mpf).
>>>>>>>>>
>>>>>>>>> I think this can be improved, we can serialize the states,bounds
>>>>>>>>> and send these instead of the bodies? what do you think? (all the functions
>>>>>>>>> do serialize, deserialize and communicate are ready. We just need a
>>>>>>>>> container to hold only the required info.)
>>>>>>>>> --
>>>>>>>>> Mailing list: https://launchpad.net/~yade-mpi
>>>>>>>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>>>>>>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> --
>>>>>>>> _______________
>>>>>>>> Bruno Chareyre
>>>>>>>> Associate Professor
>>>>>>>> ENSE³ - Grenoble INP
>>>>>>>> Lab. 3SR
>>>>>>>> BP 53
>>>>>>>> 38041 Grenoble cedex 9
>>>>>>>> Tél : +33 4 56 52 86 21
>>>>>>>> ________________
>>>>>>>>
>>>>>>>> Email too brief?
>>>>>>>> Here's why: email charter
>>>>>>>> <https://marcuselliott.co.uk/wp-content/uploads/2017/04/emailCharter.jpg>
>>>>>>>> --
>>>>>>>> Mailing list: https://launchpad.net/~yade-mpi
>>>>>>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>
>>>>>>> --
>>>>>> Mailing list: https://launchpad.net/~yade-mpi
>>>>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>> --
>>>> Mailing list: https://launchpad.net/~yade-mpi
>>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>> --
> Mailing list: https://launchpad.net/~yade-mpi
> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~yade-mpi
> More help   : https://help.launchpad.net/ListHelp
>