openerp-expert-framework team mailing list archive

[sebastien beau <sebastien.beau@xxxxxxxxxxxx>]

Hello all I share my script for extracting module from extra addons. It's
just a prototype, but it's work and can help you (if you want to improve
it, you're welcome).

I already use it for extrating module like all dependency of
magentoerpconnect, fiscal_position, kettle_connector.....

First of all you need to download an empty extra-addons branch here : bzr
push lp:~sebastien.beau/+junk/empty-extra

This branch is simply the extra-addons branch where I remove all of the
commit

Then copy/paste the file bzr-super-replay in the folder /usr/local/bin and
make it executable


After that you just have to go in your branch (the empty branch downloaded)
 where you want to replay all of the commit and type:
        bzr-super-replay path revno --module module1 module2 module3

with path the path for the extra-addons
and revno the revno when you want to start to find the commit for your
module
module1 module2... the module you want to extract

you can also type :


bzr-super-replay --help
to see the help message

The help message is :


    Command
        bzr-super-replay path revno --module module1 module2 module3

    mandatory args
        path => path to the branch where the commit come from
        revno => all commit will be replayed from this revision number

    option
        --module module1 module2 module3, -m module1 module2 module3
                Only the commit for this modules will be replayed
        --help
            Show this message
        --auto
            Try to play all commit automagically ;)
        --hide-merge
            This option will hide all commit of merge
        --hide-translation
            This option will hide launchpad translation

This script is not a magic script! (and I cod eit quickly) So sometime you
will need to finish the work by yourself.

Basically the script will search for all of the commit for the module
selected and try to replay it.

Some advice :
- do not replay the translation commit but copy paste teh translation at
the end of the process and commit it.
- when a commit impact various module and some of this module will be not
in your branch, you will have some conflict (becauseyou try to replay a
commit on product_variant_multi and the module is not in your branch). For
solving it there si not magic trick you have to re-commit the change
manually and don't forget the --author for keeping the history.
- Each time a commit failed to be replayed, take the time to understand
why, maybe a previous commit is missing and replaying the missing will
solve your problem


Maybe the best option will to create a bzr plugin, but right now I am very
busy so I just share this script that can avoid some headhack. (or give
some)
If you really have trouble to extract module, you can ask me for helping
you. But in 2 day I go to holliday and I will be not available until 18
nov, so it will be at my return ;)

Hope it can help someone.




2012/10/31 Joël Grand-Guillaume <joel.grandguillaume@xxxxxxxxxxxxxx>

> Hello again,
>
>
> In the same topic, it'll be great to have a common naming convention for
> all those branches what do you think ? I would suggest to have one branch
> per OpenERP version that will be easily recognizable, with the following
> naming:
>
> Name-Of-Project/OpenERP Version
>
> Like :
>
> lp:c2c-financial-addons/7.0
> lp:c2c-financial-addons/6.1
>
> Are you ok with this ?
>
> Regards,
>
>
> Joël
>
>
> Le 31 oct. 2012 à 13:35, Joël Grand-Guillaume <
> joel.grandguillaume@xxxxxxxxxxxxxx> a écrit :
>
> Dear community,
>
>
> We at Camptocamp, strongly approve this new way of organizing the branch.
> Grouping the modules by related topic will help a lot. I completely agree
> for this approach and to join the effort, we'll create new branches as well
> on the same principle.
>
> First, we'll start with our financial module, trying to split it into more
> topic oriented branch. Create team for each one.
>
> Thanks for the initiative, you can count on us !
>
> Regards,
>
>
> Joël
>
>
> Le 30 oct. 2012 à 20:01, Raphael Valyi <rvalyi@xxxxxxxxx> a écrit :
>
> So completing my yesterday's email,
>
> I should have mentioned that Akretion already pull out of the extra-addons
> the Magentoerpconnect and other common architecture e-commerce addons in
> new branches, for instances:
>
> https://code.launchpad.net/~extra-addons-commiter/e-commerce-addons/oerp6.1-cleanning
>
> https://code.launchpad.net/~extra-addons-commiter/openobject-extension/oerp6.1-cleanning
>
> https://code.launchpad.net/~extra-addons-commiter/product-extra-addons/oerp6.1-cleanning
>
>
> Also, as I said, we would now like to pull out the commonly used financial
> addons from the extra-addons.
> We are trying to help ourselves for Brazil, but also thinking to Spanish
> users who share a few axtra-addons with us (many other localizations do but
> these are the 2 localizations I know that depends on these modules), so
> here is a proposal of branch extraction:
>
> https://docs.google.com/spreadsheet/pub?key=0AlrjP6ETn3tJdDRIS3NsTkJ4YmR6eUlZeXJKOHp0N3c&output=html
>
> As for now, if there is no opposition to this I would extract the first "account-payment-extra"
> branch only. That would d the job for us in Brazil and hopefully help
> Spanish users too at least.
> That branch would hence extract together the following modules:
>
>    - account_payment_extension
>    - purchase_payment
>    - sale_payment
>    - pxgo_bank_statement_analytic
>    - pxgo_cash_statement
>    - pxgo_bank_statement_running_balance
>    - paydays
>    - nan_account_bank_statement
>
>
> As described in this superficial analysis, Spanish users may also want to
> extract the account-chart-extra branch, the account-sequence-extra branch
> and may be others.
>
> I asked their opinion about the prominent Spanish localization
> contributors, but if you want to comment on the spreadsheet, please ask me
> to give you write access.
>
> Extraction script and extraction instructions will follow hopefully.
>
> Any comment?
>
> --
> Raphaël Valyi
> Founder and consultant
> http://twitter.com/rvalyi <http://twitter.com/#!/rvalyi>
> +55 21 2516 2954
> www.akretion.com
>
>
>
>
>
> On Mon, Oct 29, 2012 at 2:13 PM, Raphael Valyi <rvalyi@xxxxxxxxx> wrote:
>
>> Dear OpenERP experts,
>>
>> Some of you might not have already think about it, but those who have all
>> came to the same conclusion I will re-explain here:
>>
>> *The problem*
>>
>> The system of the single extra-addons branch should now be avoided
>> because it doesn't scale in term of code and community management:
>>
>>    1. enabling or restricting commit rights doesn't scale: you may want
>>    to allow commit rights to anybody of your team for you little non critical
>>    extra-addons but the other users may want to be sure the extra financial
>>    addons they are using in production are rock solid and not screwed
>>    overnight by some beginner.
>>    2. extra-addons receive commits for a huge number of addons. Hence
>>    it's really challenging to track the code evolution of the critical
>>    extra-addons you use
>>    3. because there are so many extra-addons, the branch is huge! Often
>>    you just need 2 or 3 extra addons and you are still forced to download
>>    hundreds of Mb of code. This is specially discouraging contributions in all
>>    the countries where Internet isn't so fast...
>>    4. because there has been no quality management of those modules, the
>>    extra-addons branch is full of poor quality non reusable, non migrated
>>    modules. Probably 75% of the extra-addons modules are broken and not
>>    reusable outside from the initial company or even POC they where developed
>>    for. Worse than that, at the beginning, say back in 2008, Tiny and Axelor
>>    were putting almost all their POC modules with not enough quality to
>>    qualify as addons in the extra-addons branch, resulting in garbage modules
>>    lagging the extra-addons quality until today. Following this "example", few
>>    companies refrained themselves from pushing their low quality modules in
>>    the extra-addons during those years.
>>    5. often you need to improve some extra-addons for several reasons,
>>    mostly:
>>       1. adapt it further to the current stable OpenERP version your are
>>       using,
>>       2. add new features for the company you are working for if you
>>       think it makes sense to have that feature inside the module
>>       3. fix a bug in a non backward compatible way
>>       4. refactor the code to make the module more compatible with some
>>       other modules or localization
>>
>> In all those cases, the proper way is to create a new "feature branch"
>> and eventually merge it back into the "stable" branch. But because things
>> aren't so mature yet, often changes are not backward compatibles and it's
>> better to stay in a new branch people should explicitly pick, while keeping
>> the "stable" branch for really stable things people can update without
>> special care (ideally). If a branch has many modules like the extra-addons,
>> then it creates many parallel versions of the same modules which is a real
>> hassle to maintain (addons_path won't do the job etc...)
>>
>>
>> *The solution*
>>
>> Some of us know that for some time already, so instead of bloating the
>> extra-addons with new modules, we have been creating new modules in new
>> branches since one or two years.
>>
>> But now, we should go further, we should move the extra-addons we use
>> often out from the extra-addons into smaller manageable branches.
>>
>> Because ERP means big money involved, OpenERP contributor copyrights are
>> often abused while copyrights tend to be an essential pillar of how open
>> source works. So it's essential that the bzr history of the extracted
>> module don't get lost and that real authors can be traced back (if that get
>> lost, it will increase the risk somebody comes and change the license
>> claiming they are the only author).
>> At Akretion we developed some semi-manual script to do that and replay
>> the module commits in the new branch. I'll pass it soon in this list.
>>
>> Also what's the solution: one module per branch or several modules in a
>> branch?
>>
>> Well eco-systems that use well working module managements system tend to
>> use the one module per branch approach as one can easily rebuild at anytime
>> all the dependency tree with the proper module version that have been
>> tested to work together.
>>
>> But given the OpenERP approach of not dealing with module version
>> dependencies
>> http://openerp-server.readthedocs.org/en/latest/module-versioning.html,
>> it will not be easy to spot the compatible modules versions (that
>> information is not even stored!). Imagine if you used 20 extra-addons, you
>> would now need to do bzr pull in 20 different branches and ensure manually
>> you are taking the proper branches and updating to compatible revisions...
>> So given this situation, at Akretion we think it can still make sense to
>> group some related modules inside the same branch until possible better
>> module management appears and possible further branch split.
>>
>>
>> *Grouping modules together*
>>
>> Then comes the grouping problem: what modules go together into the same
>> branch? Well, unfortunately there won't be some clear rule, it has to be a
>> tradeoff. One could think just put everything that depends on sale
>> together, everything that depend on stock together. Unfortunately this
>> isn't that simple: there are modules having multiple core dependencies and
>> many border line cases (where would the stock_rma module fit: stock or
>> CRM?, where would the account_fiscal_position_rule module fit: account or
>> sale?)
>>
>> So let's start discussing those groupings if you like. I'm taking several
>> factors into account:
>>
>>    - dependencies
>>    - functional domain
>>    - authors
>>    - avoid to leave just one required module alone in a new branch to
>>    checkout if possible
>>    - usage in localizations...
>>
>>
>> *Example*
>>
>> Sebastien Beau (Akretion) already extracted the fiscal rule modules
>> together in a new project/branch:
>> https://code.launchpad.net/openerp-fiscal-rules
>> It contains those modules who where extra-addons originally:
>>
>>    - account_fiscal_position_rule
>>    - account_fiscal_position_rule_purchase
>>    - account_fiscal_position_rule_sale
>>    - account_fiscal_position_rule_stock
>>    - account_product_fiscal_classification
>>
>>
>> We developed those above modules for the Brazilian localization. But in
>> fact they are potentially useful for managing the fiscal positions in all
>> countries which are federations (possibly even the USA) and are useful for
>> international fiscal operations. Recently we started using them more for
>> ecommerce projects in Europe and Canada for instance to select the right
>> tax accounts even if the ecommerce front-end (like Magento) already pass
>> the proper VAT ratio to OpenERP.
>>
>> We should continue that migration process further and ideally stop
>> depending on the extra-addons for OpenERP v7 projects.
>>
>> Also, by removing the dependencies on that fat extra-addons repository,
>> we also make automated Continuous Integration testing easier. At Akretion
>> we are experimenting with Github mirrors and Travis-CI triggers to run
>> automated test at every commit. It's not that we don't like the Runbot, but
>> we can run the same tests for free on any branch and easily add support for
>> next testing suites like OERPScenario test suites freely. Also, OpenERP SA
>> re-affirmed they won't care about module dependencies, but this doesn't
>> match what we do at Akretion (who here already tried to install the proper
>> Magentoerpconnect dependencies?), so we will need to be able to specify the
>> proper branch dependencies for all the testing and have no faith the Runbot
>> can do that when the version dependencies will be missing from the module
>> format. Well, instead free open source tools like Travis-CI provision a new
>> Virtualbox machine at every run and do a fresh repository checkout, so by
>> depending on lighter repositories, we also make all that easier to happen.
>> This is part of the road to a better quality of those OpenERP modules.
>>
>>
>> I would appreciate your view on that question and will post again soon to
>> submit a new concrete module extraction proposal for the financial
>> extra-addons used both by the Spanish and Brazilian localization at least.
>>
>>
>> Best regards,
>>
>>
>> --
>> Raphaël Valyi
>> Founder and consultant
>> http://twitter.com/rvalyi <http://twitter.com/#!/rvalyi>
>> +55 21 2516 2954
>> www.akretion.com
>>
>>
>>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-expert-framework
> Post to     : openerp-expert-framework@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openerp-expert-framework
> More help   : https://help.launchpad.net/ListHelp
>
>
> --
>
> *Joël Grand-Guillaume** *
> *Division Manager*
> *Business Solutions*
> *
> *
> *Camptocamp SA*
> PSE A, CH-1015 Lausanne
> http://openerp.camptocamp.com/
>
> Phone: +41 21 619 10 28
> Office: +41 21 619 10 10
> Fax: +41 21 619 10 00
> Email: joel.grandguillaume@xxxxxxxxxxxxxx
>
>
> --
>
>
> *Joël Grand-Guillaume** *
> *Division Manager*
> *Business Solutions*
> *
> *
> *Camptocamp SA*
> PSE A, CH-1015 Lausanne
>
> http://openerp.camptocamp.com/
>
>
> Phone: +41 21 619 10 28
> Office: +41 21 619 10 10
> Fax: +41 21 619 10 00
> Email: joel.grandguillaume@xxxxxxxxxxxxxx
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-expert-framework
> Post to     : openerp-expert-framework@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openerp-expert-framework
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 

BEAU Sébastien
MagentoERPconnect Project Manager at Akretion
twitter : seb_beau
skype : sebastien_beau
www.akretion.com
http://launchpad.net/magentoerpconnect

Attachment: bzr-super-replay
Description: Binary data