openerp-expert-framework team mailing list archive

[Raphaël Valyi <rvalyi@xxxxxxxxx>]

Cloves, I saw your STOMP stuff of course, I was actually thinking Dukai made
something new over it.

When you say you have overwritten the create and write methods, I think this
can be made compatible with the SARTRE rule engine by Smile that will only
fire if a set of conditions is matches.  I mean, it's technically possible,
now I'm not sure how useful that would be.

Now, overriding create and write made me wonder a lot. Indeed, if you look
at the audittrails module, while it have obvious implementation limitations
(doesn't catch workflow changes for instance), it's not done this way, it's
done intercepting methods at the osv layer.
I'm not sure that's the reason, but I thought that it seems to happen very
often some complex methods, especially when overriden by modules call the
write methods several times upon the same object instance... Meaning you are
likely to fire way too many messages to your MQ...

I wonder if the audittrails method is the right approach, or if your
approach could say accumulate the write params in a hash each time write is
written and fire the message to the MQ only at the end of the action (same
moment as the commit if you like).

Did you guys had that issue and work around it?


Now, I like that approach to be streamline OpenERP to OpenERP communication
because even if Posgres rocks, SQL systems are intrinsically unable to scale
past to a certain size (that's why GMail, Facebook, Twitter... barely use
SQL; it's easier for those who are not ERP of course ). For instance I would
never recommend using the same OpenERP for two holdings of the same
trans-national company located on two different continents: the network
latency would totally kill the speed and no real time multi-master SQL
replication at this scale would be efficient either, not to speak about the
insupportable coupling the two companies would have functionally and
operationally.

Let me recall that with our TermniatOOOR Kettle ETL plugin
http://github.com/rvalyi/terminatooor (easily leverage the full OpenERP API
right into one of the two leading open source ETL), we are also making such
things much easier, though not limited to OpenERP (for instance OSCommerce
or bank integration using TerminatOOOR has been very easy). Also notice that
OOOR is able to work in multi-instances mode as explained here:
http://wiki.github.com/rvalyi/ooor/howto-connect-ooor-to-multiple-openerp-instance-easy-data-migration

Hope you can answer my question and that my pointers might help some (better
doc coming soon).
NB: tomorrow, OOOR-1.2.7 will ship with an API making it straightforward to
find resources by their ir_model_data ids or create absolute ids along with
the resources.

Raphaël Valyi
http://www.akretion.com




On Mon, Feb 22, 2010 at 10:02 PM, Cloves Almeida <cjalmeida@xxxxxxxxx>wrote:

>  For the details of the implementation, we used the STOMP protocol to
> communicate with the ActiveMQ server. I embedded the stomppy client into the
> module for easier distribution - it's pure Python and very light.  ActiveMQ
> do the STOMP-JMS translation, mnkieaning any application that can understand
> JMS and XML/YAML can produce/receive OpenERP messages.
>
> Why not XML-RPC? Because is both synchronized and point-to-point. I could
> build an async, multi-point layer over XML-RPC, but then the end result
> would look like a crappy, non standards compliant MQ broker :).
>
> To trigger the sending of messages, I overloaded the "write" and "create"
> methods. It would call helper methods to serialize and send the message to a
> preconfigured "queue" in a local MQ. There's no need for timed triggers, the
> "de facto" sending of messages through the network is MQ's responsibility -
> there are a number of methods for reliable MQ-to-MQ bridge. To receive the
> message, one would subscribe to the queue on a MQ server (the same or
> bridged, local or not) and register listener methods that would be called on
> method arrival. Again, no need for timers, since this is done as soons as
> the message is available to server - message persistence and delivery is the
> MQ's responsibility. A simple message header "if" condition or XPath query
> serve as a "filter".
>
> Exception handling, enterprise quality monitoring and administration, and a
> powerful routing engine (Apache Camel) are also features of ActiveMQ - even
> if I didn't use them.
>
> I started to develop the module for integrating into our existing POS
> system, but the system was so bad we decided to replace it entirely. It
> meant I had to pause developing the module to work on installing the new POS
> system. But in a few months I'll roll out an OpenERP instance for our
> accounting dept. and I'll definitely need to integrate it with a number of
> existing systems and I intend to use it.
>
>
>
>  -
> On 22-02-2010 19:33, Raphaël Valyi wrote:
>
> Hello,
>
>  maturing OpenERP to use asynch message bus is interesting indeed,
> especially for multi-company and/or larger companies which will be used to
> J2EE ESB's.
> I didn't look into details about what you and Cloves did, but I let you
> know that Tiny did a parallel effort with the Twisted Python asyn messaging
> system. Here is the work of Stephane Wirtel:
> https://code.launchpad.net/~stephane-openerp/openobject-server/twisted<https://code.launchpad.net/%7Estephane-openerp/openobject-server/twisted>
> In any case, I hope you can join efforts rather than double work on those
> issues.
>
>  I think this is also a place where a total or partial (eg
> in redundancy with a CPython instance) usage of OpenERP on Jython would
> shine as the Java platform has plenty of valuable and mature ESB systems
> generally built upon the JMS protocol.
>
>  Finally, may be something that could be used as a rule engine to trigger
> an anych message emission could be the "SARTRE" rule module by Smile in
> extra addons now (currently use to trigger alerts and standard server
> actions). Then message emission would be a regular server action triggered
> by that rule engine upon given conditions. Not sure, but something to
> investigate may be. Would also be nice to have the reverse: a rule engine
> that know how to analyse the message queue en trigger a given server action
> if the message matches a given condition, may be similar to what the Smile
> SARTRE module does. Again not sure.
>
>  All right, good luck with this one...
>
>  Raphaël Valyi
> http://www.akretion.com
>
>
> 2010/2/22 Dukai Gábor <gdukai@xxxxxxxxx>
>
>> Hello!
>> There are tools already present that could be used to integrate OpenERPs
>> (And
>> other systems, too. It's based on xml.) and it would work on slow,
>> unreliable
>> connections.
>>
>> IMHO to develop something like this would take two workdays:
>> -Confirming a PO creates an SO at the supplier's side.
>> -It matches products with bar code and partners with VAT number for
>> example.
>> Or it uses a unique identifier for products and in the first run maps
>> those
>> identifiers to bar codes, and if didn't succeed, uses
>> product.supplierinfo.
>> -Confirming the SO at the supplier's side validates the PO at the
>> customer's
>> side.
>> -Any number of connections can be used and they are relatively easy to set
>> up
>> for an administrator.
>> -It stores the messages if you or the other parties aren't online and
>> automatically sends them as soon as possible.
>>
>> This is all based on Cloves Almeida's mbi module that is the tool to turn
>> any
>> object into xml/read from xml and send/receive it using stomp (we use it
>> with
>> an ActiveMQ broker).
>> I've created a module mbi_openerp that adds easy setup routing, easy setup
>> CRUD operations synchronization between OpenERPs.
>>
>> My original intent was to provide even more: send workflow events and send
>> the
>> call of methods, too. That way I would have synchronized basic OpenERP
>> instances with only the stock module installed (used at warehouses) with
>> the
>> company's main db. Although this approach didn't prove to be reliable,
>> stripping out the workflow and method sending code would result in a small
>> base
>> module suitable for B2B integration that I mentioned above.
>>
>>
>> Regards,
>> Gábor Dukai
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openerp-expert-framework<https://launchpad.net/%7Eopenerp-expert-framework>
>> Post to     : openerp-expert-framework@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openerp-expert-framework<https://launchpad.net/%7Eopenerp-expert-framework>
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-expert-framework <https://launchpad.net/%7Eopenerp-expert-framework>
> Post to     : openerp-expert-framework@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openerp-expert-framework <https://launchpad.net/%7Eopenerp-expert-framework>
> More help   : https://help.launchpad.net/ListHelp
>
>
>