openerp-expert-framework team mailing list archive

[Nicolas Bessi <nicolas.bessi@xxxxxxxxxxxxxx>]

Hello, 

We should have a good improvement of performance in netrpc protocol by using protocol version 2 of pickle protocol.
http://docs.python.org/library/pickle.html#data-stream-format 

It provide of gain of around 30% in term of data volume and pickle is automatically aware of protocol version so no need to change client behavior.

We can also gain some message exchanges by optimizing the reception of message by aggregating the response and use a single |socket.send| loop.
The previous code call 3 |socket.send|:
- 8 bytes for the size of the message
- 1 byte for the exception flag
- then the actual message

What did you think of patch below for version 6.1?

Thanks to Florent for the patch.


Regards

Nicolas


=== modified file 'openerp/tiny_socket.py'
--- openerp/tiny_socket.py	2011-06-23 09:03:57 +0000
+++ openerp/tiny_socket.py	2011-10-11 11:08:42 +0000
@@ -22,12 +22,15 @@
import socket
import cPickle
import cStringIO
-import marshal

import netsvc

#.apidoc title: Net-RPC classes

+# Pickle protocol version 2 is optimized compared to default (version 0)
+PICKLE_PROTOCOL = 2
+
+
class Myexception(Exception):
    """
    custom exception object store
@@ -63,20 +66,19 @@
        netsvc.close_socket(self.sock)

    def mysend(self, msg, exception=False, traceback=None):
-        msg = cPickle.dumps([msg,traceback])
-        self.sock.sendall('%8d%s%s' % (len(msg), exception and "1" or "0", msg))
+        msg = cPickle.dumps([msg, traceback], PICKLE_PROTOCOL)
+        self.sock.sendall('%8d%d%s' % (len(msg), bool(exception), msg))

    def myreceive(self):
        buf=''
-        while len(buf) < 8:
-            chunk = self.sock.recv(8 - len(buf))
+        while len(buf) < 9:
+            chunk = self.sock.recv(9 - len(buf))
            if not chunk:
                raise socket.timeout
            buf += chunk
-        size = int(buf)
-        buf = self.sock.recv(1)
-        if buf != "0":
-            exception = buf
+        size = int(buf[:8])
+        if buf[8] != "0":
+            exception = buf[8]
        else:
            exception = False
        msg = ''







--------------------------------------------------------------------
Nicolas Bessi 
Senior ERP consultant 
Business Solution technical director

Camptocamp SA
PSE A
CH-1015 Lausanne
http://www.openerp.camptocamp.com

Phone: +41 21 619 10 26 
Office: +41 21 619 10 10 
Fax : +41 21 619 10 00 
--------------------------------------------------------------------