← Back to team overview

openerp-expert-framework team mailing list archive

Re: wide tables: res.partner

 

Hello Raphael,

@Christeas: I agree with the problem but not the solution

I found a more decent solution in Tryton.

Example:

Party (Eq of Partner):
http://hg.tryton.org/hgwebdir.cgi/1.6/modules/party/file/d6f20ea5ba0d/party.py#l24
<http://hg.tryton.org/hgwebdir.cgi/1.6/modules/party/file/d6f20ea5ba0d/party.py#l24>Company
:
http://hg.tryton.org/hgwebdir.cgi/1.6/modules/company/file/1c828b433356/company.py#l11
<http://hg.tryton.org/hgwebdir.cgi/1.6/modules/company/file/1c828b433356/company.py#l11>Employee
:
http://hg.tryton.org/hgwebdir.cgi/1.6/modules/company/file/1c828b433356/company.py#l67

The two tables company and employee _inherits from party.party.
Which means no duplication of data and in addition, no extending one table
like Christeas pointed out. I find this a more clean solution.

Open ERP official modules also rarely use the ORM features (assuming
_inherits on Open Object works). An example is the subscription module. It's
so stupidly written.

Take a look at:
http://bazaar.launchpad.net/~openerp/openobject-addons/5.0/annotate/head:/subscription/subscription.py#L95

The set_process method creates an ir.cron and fills 12 fields which are the
same fields replicated on subscription.subscription table. The ideal
implementation should have been an _inherits on ir.cron. Such coding
practices itself would save the performance issues outlined here and makes
the code clean. (This is from draft implementation of subscriptions for
tryton)

Now to the EAV module:

We did try out EAV and we found the performance to be extremely low and
presentation on the ERP frontend becomes a nightmare. This can only be made
possible if the core modules like partner are refractored. Again referring
to Tryton, the party has an EAV model for contact mechanisms. Refer
http://hg.tryton.org/hgwebdir.cgi/1.6/modules/party/file/d6f20ea5ba0d/contact_mechanism.py#l24
 and
http://hg.tryton.org/hgwebdir.cgi/1.6/modules/party/file/d6f20ea5ba0d/party.py#l49

I suggest that openobject implementation of _inherits should be refreshed to
work and then modules refractored for this.

Thanks,

2010/7/9 Raphaël Valyi <rvalyi@xxxxxxxxx>

> Hello,
>
> I suggested already to create such an "EAV" pattern (attribute tables)
> indeed for OpenObject:
> https://blueprints.launchpad.net/openobject-server/+spec/base-eav-module
> However, it's very important not abusing it either. In the Magento
> ecommerce, the guys went full EAV pattern at first cause it was the simplest
> for them: once they implemented their EAV crap, it was easier for them to
> use it always for any attribute instead of having a decent DDL migration
> engine such as OpenERP. Result: even sorting products by price in Magento
> hit the EAV bloated tables...
> Anyone, used to Magento would know that this is the SQL pattern (along with
> the use of PHP) that make Magento dog slow for anything that can't be
> cached:
> http://inchoo.net/ecommerce/magento/escape-from-eav-the-magento-way/
>
> So, yes, I'm for an EAV pattern in OpenObject, as long as it's used only to
> store some sparse attributes with very occasional search operations upon
> them (but yes this let room for many attributes indeed).
>
> I also recall one general principle: OpenERP doesn't have sufficient
> technology/consistency/functional scope to attract large companies massively
> yet. As it is, it really targets 10 - 50 employees SMB's (until it get clean
> + complete enough eventually). Those small SMB's don't really have large
> perf impact of using OpenERP. Perf trouble lies elsewhere eventually: too
> many HTTP requests (like the like of search_read in the GTK client etc...),
> too many clicks (cause of the lack of genericness of some widgets), widgets
> like many2many that were not paginated in v5, slow Python execution (top
> typically shows at least 40% of CPU usage in the Python part, some smart
> combo like Java/Mirah/Jython/JRuby would have allow to externalize some of
> the Python slowness to C fast languages that are still portable
> and reasonably powerful (I believe that Scala or Duby/Mirah are just as
> powerful as Python while featuring C speed and retaining a runtime
> compatibility with a Python impl (Jython) )). Some of the Python overhead
> would also be removed if we were using a smartest ORM (SQLAlchemy?) that
> would allow to abstract more complex SQL requests in Python rather than
> multiplying the DB calls and Python overhead as it is done currently all
> over the place in the modules (search/search/browse/relation sequences).
>
> Finally, I have some doubts: Sharoon, didn't you start some EAV pattern
> already?
>
> Raphaël Valyi
> Founder and ERP Consultant
> +55 21 3010 9965
> http://www.akretion.com
>
>
>
>
> On Fri, Jul 9, 2010 at 12:35 PM, P. Christeas <xrg@xxxxxxxxx> wrote:
>
>> In a recent commit, I just noticed another module that inherits
>> "res.partner"
>> and adds 3-4 columns.
>> Counted the columns in one db, and found 50.
>>
>> I think we're doing something wrong here.
>>
>> Many modules need to store information per partner. They just add extra
>> fields
>> in the table. They also add one boolean field "is_member", "is_employee",
>> "is_clown" etc.
>> The same, of course, may be true for other tables, like product, for
>> example.
>>
>> I suggest, generally, that we avoid this pattern in setting up our schema.
>> My
>> feeling is that we waste resources (although pg's toast does compression)
>> in
>> storing NULLs for all these columns.
>>
>> At least, these booleans could be somehow covered by the partner
>> categories.
>> Then, other explicit data per partner, could be put in an "attributes"
>> table.
>>
>> wdyt?
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openerp-expert-framework
>> Post to     : openerp-expert-framework@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openerp-expert-framework
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-expert-framework
> Post to     : openerp-expert-framework@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openerp-expert-framework
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Sharoon Thomas
Business Analyst & Open Source ERP Consultant
CEO @ http://openlabs.co.in

References