← Back to team overview

yade-mpi team mailing list archive

Re: Yade-MPI sending/receiving serialized bodies

 

>
> In the case without merge, the results are exactly the same in comparison
> to mpy.py (test2D_MPI.py, no merge.) . We need to add a a list of things to
> be done (and which are already done ) in the gitlab issue page. I will
> start his.
>
Yes, this is also what I have seen but I can't remember: was it already
like that at the end of the hackathon ?

Nice list !
I will work on the second point: exchanging bodies with MPIBodyContainer.
Even though at this time we don't care about the "when" question.

Le mar. 30 avr. 2019 à 14:57, Deepak Kn <deepak.kn1990@xxxxxxxxx> a écrit :

> Hello,
>
> So I have added a 'to do list', at gitlab based on my understanding of
> yade-mpi and the previous discussions : (
> https://gitlab.com/yade-dev/trunk/issues/68). What do you think?
>
> On Tue, Apr 30, 2019 at 2:07 PM Deepak Kn <deepak.kn1990@xxxxxxxxx> wrote:
>
>> Hello,
>>
>> *Did you try a simulation with merge/split operations ?*
>>
>> I am working on the merge operation right now : The issue is once we send
>> all the bodies to the master proc, the interactions of these bodies have to
>> be set in to the master's interaction container, I'm looking at this now.
>> So I have not tried any simulations with the merge/split operations.
>>
>> *are you able to get the right force on the bottom wall ?*
>>
>> In the case without merge, the results are exactly the same in comparison
>> to mpy.py (test2D_MPI.py, no merge.) . We need to add a a list of things to
>> be done (and which are already done ) in the gitlab issue page. I will
>> start his.
>>
>>
>> On Tue, Apr 30, 2019 at 1:42 PM François <francois.kneib@xxxxxxxxx>
>> wrote:
>>
>>> Thanks deepak,
>>>
>>> I see that, in addition to merge(), you have moved most of the python
>>> splitScene() into c++ Subdomain::findInterSections(). That's great, not the
>>> easiest part ! So is that part of the code working ? Did you try a
>>> simulation with merge/split operations ?
>>> If I'm right we know communicate bodies/ids only, so the last (but not
>>> least) question: are you able to get the right force on the bottom wall ?
>>>
>>> Le lun. 29 avr. 2019 à 15:21, Deepak Kn <deepak.kn1990@xxxxxxxxx> a
>>> écrit :
>>>
>>>> Hello,
>>>>
>>>> Yes, checkout from the branch : mpi_dpk. : there's an example in
>>>> examples/mpi/testnomerge.py
>>>> also, there's another py module in py : py/yadempi.py (this is more or
>>>> less similar to mpy.py but we send bodies instead of states).
>>>> If you want to see the sizes of messages  : (size as in number of elems
>>>> in the buffer)
>>>> add this line  in Subdomain.cpp , function void Subdomain::
>>>> recvContainerString after line 429 : std::cout << "sizes " <<
>>>> recvdStringSizes[i] << std::endl;
>>>> let me know if you have any troubles, also if you find some part of the
>>>> code inefficient (or could be made better), go ahead and fix it!
>>>>
>>>> Thanks and regards,
>>>> deepak
>>>>
>>>> On Mon, Apr 29, 2019 at 2:52 PM François <francois.kneib@xxxxxxxxx>
>>>> wrote:
>>>>
>>>>> Deepak, could you please provide an example script to test this
>>>>> feature ?
>>>>>
>>>>> Le lun. 29 avr. 2019 à 13:39, François <francois.kneib@xxxxxxxxx> a
>>>>> écrit :
>>>>>
>>>>>> Hi, and nice work Deepak.
>>>>>> I would add that we can't compare directly the number of elements
>>>>>> that are transmitted, but rather the total volume, as size of char is 8bits
>>>>>> and size of double is 64bits by default on modern cpus.
>>>>>> As a result, we should rather compare 4180*8=33440 with 215456, which
>>>>>> still represents a factor of 10.
>>>>>>
>>>>>> Le lun. 29 avr. 2019 à 13:16, Bruno Chareyre <
>>>>>> bruno.chareyre@xxxxxxxxxxxxxxx> a écrit :
>>>>>>
>>>>>>> Hi Deepak,
>>>>>>> Clearly there is no reason to compare speed of sending pos+vel vs.
>>>>>>> speed of sending the entire body, since these communications are not used
>>>>>>> at the same stages.
>>>>>>> Sending serialized bodies is an alternative to sending the whole
>>>>>>> scene, and I guess (?) sending 5% of the bodies in a scene will take less
>>>>>>> time than sending the scene.
>>>>>>> The communication which occures at each iteration will always be
>>>>>>> only pos+vel.
>>>>>>>
>>>>>>> @Janek, when we break a scene in multiple pieces we want to send
>>>>>>> everything because we don't want to worry about what needs to be sent in
>>>>>>> each other particular case, and actually if an attribute is there then we
>>>>>>> can safely assume that there is a reason for it. When it's already broken
>>>>>>> in pieces we update (communicate) only positions and velocity because
>>>>>>> otherwise it's too expensive, as Deepak found, but it's not foolproof. For
>>>>>>> instance differential growing of particles in each subdomain would break
>>>>>>> this concept since radius is not communicated at every iteration.
>>>>>>>
>>>>>>> Bruno
>>>>>>>
>>>>>>>
>>>>>>> On Sun, 28 Apr 2019 at 19:58, Deepak Kn <deepak.kn1990@xxxxxxxxx>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello All,
>>>>>>>>
>>>>>>>> I have finished implementing the functions to serialize and
>>>>>>>> deserialize the bodies and the necessary communications to exchange the
>>>>>>>> serialized bodies. The serialized messages (binary) are several times
>>>>>>>> larger than the usual (state) messages in mpy/mpf.py (vel,pos, ori, id,
>>>>>>>> bounds) etc. And as a result the simulations are far slower ! Here's how it
>>>>>>>> scales : for exchanging 220 bodies, the serialized messages were around
>>>>>>>> 215456 elements (chars), and in the previous versions, the message sizes
>>>>>>>> were 4180 (doubles). (testMPI2D no merge, on 3 procs, 20000 bodies.)
>>>>>>>>
>>>>>>>> Most of the functions are implemented in Subdomain.cpp
>>>>>>>> Subdomain.hpp and MPIBodyContainer.hpp, (branch : mpi_dpk on gitlab) I
>>>>>>>> followed the same logic of the previous version (mpy/mpf), non blocking
>>>>>>>> sends and blocking recvs (with probes) are used for the communications
>>>>>>>> between the workers, and blocking communication for sending the forces to
>>>>>>>> the master (the same as in mpy/mpf).
>>>>>>>>
>>>>>>>> I think this can be improved, we can serialize the states,bounds
>>>>>>>> and send these instead of the bodies? what do you think? (all the functions
>>>>>>>> do serialize, deserialize and communicate are ready. We just need a
>>>>>>>> container to hold only the required info.)
>>>>>>>> --
>>>>>>>> Mailing list: https://launchpad.net/~yade-mpi
>>>>>>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> --
>>>>>>> _______________
>>>>>>> Bruno Chareyre
>>>>>>> Associate Professor
>>>>>>> ENSE³ - Grenoble INP
>>>>>>> Lab. 3SR
>>>>>>> BP 53
>>>>>>> 38041 Grenoble cedex 9
>>>>>>> Tél : +33 4 56 52 86 21
>>>>>>> ________________
>>>>>>>
>>>>>>> Email too brief?
>>>>>>> Here's why: email charter
>>>>>>> <https://marcuselliott.co.uk/wp-content/uploads/2017/04/emailCharter.jpg>
>>>>>>> --
>>>>>>> Mailing list: https://launchpad.net/~yade-mpi
>>>>>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>> --
>>>>> Mailing list: https://launchpad.net/~yade-mpi
>>>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>> --
>>> Mailing list: https://launchpad.net/~yade-mpi
>>> Post to     : yade-mpi@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~yade-mpi
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>

Follow ups

References