openerp-community team mailing list archive

[Raphael Valyi <rvalyi@xxxxxxxxx>]

On Fri, Nov 8, 2013 at 10:11 AM, Eric Caudal <eric.caudal@xxxxxxxxxxxxxx>wrote:

>  For my understanding, what is the reason today to have one branch with
> several modules?
>

Suppose you do an ERP project with Magentoerpconnect.
you will likely have say 30 additional non core modules.
That could certainly be 30 branches of 1 modules of course.

But today, as package management has never been taken seriously, you would
have to handle manually these 30 branches and prey for their version and
branches to be compatible together. And I guarantee that they wouldn't as
nobody would test the same versions together.

Instead, by grouping modules, instead of 30 branches, you have only say 4
branches to manage. Notice that in the past, most of users not managing to
install Magentoerpconnect or localizations would fail to download the right
branches together...

This is also for historical reasons: we had the extra-addons repo which
grew from 10 to over 200 modules till v6.1.
The OCA projects and branches were already a split of that unmanageable
bundle, so a step in the right direction.
see https://lists.launchpad.net/openerp-expert-framework/msg00948.html


But in any case, before telling everybody to add their modules inside new
unmanageable branches that will end up like the extra-addons, we should
take a breath and remember that this is currently a workaround until we
have some decent package management. At least people managing their project
cleanly on say Github shouldn't be invited to grow that future chaos. On
the contrary, core concern modules, lost Launchpad branches, modules inside
non-sense per integrator branches, are very welcome to enter the OCA peer
review and branches even.

Notice that Tryton already fixed that some 3 years ago may be, so it
shouldn't be impossible if OpenERP raises millions. Hey but we have apps!
------>[]


I tried to make a step in that direction last year by trying to make the
module dependencies format more specific (not just a name) last year:
https://code.launchpad.net/~akretion-team/openobject-server/trunk-extra-depdencies-info/+merge/114172
but that was blocked as you can read.
Strangely the guy who had to block it - Vo Minh Thu - is now out from
OpenERP SA and released assertive.io a few days after stepping out. As I
said, Assertive is certainly a big step in the right direction but it
doesn't fix the problem of occasional delta updates that successful OpenERP
integrators do to stay in business.

And again, the shameful thing is that we are all talking here about solved
software engineering problems. I'm not that used about non OpenERP python
projects, but I guaranty you that with Ruby, you get your dependencies in
your Gemfile, just type a little "bundle install", get pinned recursive
dependencies in your Gemfile.lock that you can share or tag for Continuous
Integration. Bundle install will grab the dozens of modules from wherever
they are (folder, git...), the right versions (recursively), the right
branches (recursively), the right delta updates without downloading the
world again.
It just works and people are productive with it.
And it could work with OpenERP too...


Regards.

-- 
Raphaël Valyi
Founder and consultant
http://twitter.com/rvalyi <http://twitter.com/#!/rvalyi>
+55 21 2516 2954
www.akretion.com












> is it historical or is there a technical/practical reason?
>
>  Eric Caudal*CEO*
> --*Elico Corporation, Shanghai branch
> OpenERP Premium Certified Training Partner *
> Cell: + 86 186 2136 1670
> Office: + 86 21 6211 8017/27/37
> Skype: elico.corperic.caudal@elico-corp.comhttp://www.elico-corp.com
>
> [image: Elico Corp]
> On 11/08/2013 07:47 PM, Michel Meyer wrote:
>
> Hi Community,
>
> It seems that the module organization is a sensitive subject, but it's
> obvious that CVS organization is fundamental for good processes.
>
> If I can afford, personally i think 1 module = 1 repo should be the rule
> (in a perfect world :)).
>
> I had similar issues in the past on large plugin based projects and it's
> always a pain, specially if your repo is getting bigger (like
> openobject-addons):
> - it's go against the module pattern
> - heavy, get all modules when you need only some
> - make review process harder (you can't test only one module without being
> sure there's no side effect from others)
> - it's hard (impossible) to manage version of each module
> - merging can be hard/messy
> - 1 revision may affect multiple not related modules (up to the dev to
> avoid it...) and can produce confusion in logs
>
> But I may understand also constraints that pushed the community to create
> mixed modules repositories, and established processes can not be changed
> quickly...
>
>
>
> Anyhow, I'm trying to be compliant with community rules, so pratically,
> I'm fighting with git-bzr, bzr fast-import, git fast-export, since 2
> days... :)
>
> What I'm trying to do is to keep my module alone on github, but merging it
> on a lp branch based on openobject-addons (for example).
>
> I would like to have it bi-directional, I mean be able to get any changes
> to my module (and only my module) from the lp branch back to my github
> branch, and also be able to merge my changes from git to lp... and I'm
> trying to not loose revision history in the sync process...
>
>
> Not sure it's clear, but i don't think there's any script/tool already
> available allowing to sync a part of a lp branch to/from a git repo.
>
>
> I will continue to work on a script to do that, but if anybody has a
> solution, please, share it :)
>
>
> Michel
>
>
> On 11/07/2013 04:09 PM, Joël Grand-Guillaume wrote:
>
> Forgot this link about the review process for you Michel :
> https://doc.openerp.com/contribute/05_developing_modules/#community-review
>
>
> On Thu, Nov 7, 2013 at 10:06 AM, Joël Grand-Guillaume <
> joel.grandguillaume@xxxxxxxxxxxxxx> wrote:
>
>>       Dear Community,
>>
>>
>>  First, I want to thank you Michel for trying to respect the community
>> process and tools. I know it's a bit of work, but it's mandatory to
>> maintain quality. I hope you had all your answer (making a fork and go
>> through Merge Proposal when requesting an update, so the review can occur,
>> even for your first one).
>>
>>  Secondly, I'm completely in-line with Perdo's opinion about what goal do
>> we pursue. It took us (and me) lots of time to setup all those LP
>> repository, team, branches and all that stuff to make the OpenERP community
>> a reality.
>>
>>  Now we finally reach something.. Thanks to all of you ! It starts to
>> work and attract the interest of people and I'm really happy on that.
>>
>> But, even if I know LP and Bazaar sucks on some points, it works. It's
>> organized, reviews are made and the process and tools are clear enough to
>> enough people do the job.
>>
>>  For that reason, I'm strongly in favor of keeping our work there for
>> now. We barely have enough resources to make all the needed reviews and I
>> think moving to GitHub will not help on that but rather introduce quite
>> lots of overhead till we reach what we have now.
>>
>>  My suggestion is :
>>
>>   * Keep LP and Bazaar for now and at least, let's say a year or two
>>   * Keep improving the reviews process, docs and so on so new comers can
>> join the effort
>>   * Starts to run the OCA (we're working on it this month)
>>   * When we starts having a comfortable situation, let's move to GitHub
>> or anywhere else you want, I actually don't really care about the tools. I
>> care about building this community and make it stronger.
>>
>>  I know Raphaël, you angry with Bazaar and Github is better suited. I
>> know you find the packing system a pain in the ass and I agree. But let's
>> make with what we have till we're strong enough to change that !
>>
>>  Regards,
>>
>>
>>  Joël
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Nov 6, 2013 at 5:32 PM, Pedro Manuel Baeza Romero <
>> pedro.baeza@xxxxxxxxx> wrote:
>>
>>>  Hi, community,
>>>
>>>  Sorry in advance for the long message.
>>>
>>>  I think we have to talk about two different topics that are under the
>>> OCA umbrella: OCB branches and community repositories.
>>>
>>>  For *OCB branches*, there are not too much discussion: it's mandatory
>>> that we have them on Launchpad, because there are too many dependencies on
>>> LP that cannot be avoided (see this Olivier Dony answer<https://lists.launchpad.net/openerp-community/msg03629.html>about this question on other thread) and we must go here together with
>>> OpenERP S. A.
>>>
>>> About *community repositories*, the question that Raphael has exposed
>>> is critical: the future and maintanability of them. Having lived a part of
>>> the evolution of them, this is what I think:
>>>
>>>    - In some way, we are trying to compensate the lack of a global
>>>    quality OpenERP repository. OpenERP Apps doesn't count, because the
>>>    categorization in it is not very accurate (it depends on the module
>>>    manifest) and there is a lot of "dummy" modules that makes a few (usability
>>>    module, for example).
>>>     - The only way we have for now is to group relevant modules in a
>>>    repository to make it in somehow available to the groups of interest.
>>>    - We have used Launchpad to continue with the same tools as the
>>>    mother project.
>>>    - These tools have conditioned the organization / reviewing process.
>>>    - Although we are admitting a lot of new modules, it follows some
>>>    criteria (for now based on the sense of the reviewers) to avoid only
>>>    usability modules (we have talked about this on other thread also).
>>>    - Reviewers can use their expertise to teach newcomers on good
>>>    practises, and these newcomers will be soon the next teachers (this is for
>>>    example my case). This musn't been interpreted as a dictatorial regime, but
>>>    some work rules to assure quality. If these practises are not correct, we
>>>    can change or refine them.
>>>    - We don't have the obligation to maintain across the versions all
>>>    the modules that are included in some momment on the community
>>>    repositories, but to assure that when something arrives to them (a new
>>>    module or an adaptation to a new version of an old one), it has enough
>>>    quality to enter.
>>>    - These reviewings can be exhausting, but we are attracting each day
>>>    more contributors that helps with this process.
>>>    - Nobody have the obligation to publish their modules on community
>>>    repositories. They can use their own repositories, even hosting on others
>>>    CVS.
>>>    - My hope is to set these repositories as the reference where
>>>    someone goes when he wants to contribute.
>>>
>>> So, said that, to keep this initiative healthy, this is IMHO what we
>>> have to do:
>>>
>>>    - Set clear rules for the reviewing process. This is already done
>>>    and will be merged on official documentation.
>>>    - Set a path to contributors to become reviewers.
>>>    - Reward contributors with some visibility on OCA mediums (web and
>>>    so on), to encourage new contributions.
>>>    - Set rules for admission of modules. This is not easy, but it can
>>>    be done with a few exceptions.
>>>
>>> Now, about *Launchpad and GitHub*, I feel the same as Raphael in the
>>> questions he told (performance, acknowledgment, subprojects, etc), and
>>> maybe we can switch community repositories to GitHub to avoid questions
>>> like the one that has started this debate, but this must be assured:
>>>
>>>    - We have a way to integrate translations for easy contributions.
>>>    I'm already working on having Launchpad translations on community
>>>    repositories and I see this as mandatory.
>>>    - We must assure atribution and permissions.
>>>    - We cannot lose contribution history.
>>>    - Reconstruct documentation with the new contribution path.
>>>    - We must have anyway synched Launchpad repository/repositories to
>>>    enable modules on apps.openerp.com.
>>>
>>> As you see, this a lot of work to do and questions to solved, but
>>> Raphael, if you're willing to work on this, I can help you and propose to
>>> the community a concrete proposal to make the switch.
>>>
>>> Regards.
>>>
>>>
>>>
>>>
>>> 2013/11/6 Raphael Valyi <rvalyi@xxxxxxxxx>
>>>
>>>>   On Wed, Nov 6, 2013 at 10:48 AM, Joël Grand-Guillaume <
>>>> joel.grandguillaume@xxxxxxxxxxxxxx> wrote:
>>>>
>>>>>  Dear all,
>>>>>
>>>>>
>>>>>  First thanks you Michel to take the time for that ! Now, I'm a bit
>>>>> concerned because we didn't really face this before.
>>>>>
>>>>>  The
>>>>> https://code.launchpad.net/~openerp-community/openobject-addons/7.0-booking-chart
>>>>>  is more a module that should be included in one of our project, and
>>>>> I don't which one, any idea ?
>>>>>
>>>>>  Then the branch of the
>>>>> https://code.launchpad.net/~openerp-community/web-addons/7.0-web-unleashed
>>>>> is on the right place, but it contains a branch replicated from
>>>>> github... That prevent any merge to be done in the "official" community
>>>>> branch here : lp:web-addons<https://code.launchpad.net/%7Ewebaddons-core-editors/web-addons/7.0>
>>>>>
>>>>>  That's a bit sad.. Any suggestion here ? How to handle GitHub
>>>>> branches ?
>>>>>
>>>>
>>>>
>>>>  Hello Joël and others,
>>>>
>>>>  if a branch is Github driven and replicated on Launchpad, merging a
>>>> Launchpad MP isn't that hard: in your git branch just import the branch
>>>> using the native git ber remote helper
>>>> http://felipec.wordpress.com/2012/11/13/git-remote-hg-bzr-2/
>>>>  merge inside your git using usual git merge command, push to Github
>>>> (then auto-replicated to Launchpad) and voila! Still Github submitted
>>>> merges are easier to deal with.
>>>>
>>>>  Of course, now if a module should integrate a branch driven on
>>>> Launchpad, then probably it's better to let it be developed on Launchpad.
>>>>
>>>>  Now a bit more about this: "should a module integrate some existing
>>>> Launchpad branch?"
>>>> In any case, people should remember that we currently group modules
>>>> together just because OpenERP don't have any decent package manager that
>>>> deal with version constraints and multiple repo location. And as you read
>>>> this, please OpenERP SA don't try to reinvent a package manager (apps?),
>>>> that's something hard, OpenERP should standardize instead of wasting
>>>> resource on these solved problems.
>>>> And again, it's not like if I am only "talking":
>>>>
>>>> https://code.launchpad.net/~akretion-team/openobject-server/trunk-extra-depdencies-info/+merge/114172<https://code.launchpad.net/%7Eakretion-team/openobject-server/trunk-extra-depdencies-info/+merge/114172>
>>>>
>>>>  But in any case, grouping modules in the same branch is rather a
>>>> temporary artifact and a bad design than anything else.
>>>>
>>>>  Instead, once it becomes decent to install pinned versions via
>>>> package manager and test with standard Continuous Integration tooling,
>>>> **we should tend to 1 module = 1 branch**
>>>> That's how all the modular dynamic ecosystems work. Of course, we could
>>>> still have a few exceptions. If you take the Ruby ecosystem (extremely
>>>> modular and works well), you see that usually 1 module = 1 Github repo. But
>>>> as for the Rails repo, it's 5 modules altogether in the same repo (but only
>>>> 5, while a typical Rails project will depend on 50+ modules). It's an
>>>> exception and it still works. I know it less, but I think it's the same for
>>>> Pypi, right?
>>>>
>>>>  If we keep grouping modules blindly, that will not scale in term of
>>>> complexity: soon there will be tons of merge for modules that are unknown
>>>> to most of a project team, that will be slow/frustrating or badly reviewed.
>>>> We will have branches of these god branches incompatible between them and
>>>> branch cross dependencies nightmares.
>>>>
>>>>  With that in mind, I think it would be an error to think that only a
>>>> few people should determine a single forge for all the OpenERP eco-system,
>>>> specially if that forge sucks (I didn't tell Launchpad sucks, you are
>>>> interpreting here ;-)
>>>>
>>>>  I think it really make sense to use a single forge for the core, but
>>>> as for the whole eco-system, we would try to impose bad tooling to good
>>>> willing people and that will not work.
>>>>
>>>>  It's cool also to have OpenERP SA and OCA doing some centralization
>>>> work trying to develop and review core modules, but in any case, it will
>>>> never scale to imagine that just a dozen of people will review the entire
>>>> eco-system at every-commit. I think that instead there will be lot's of
>>>> projects, lot's of team, ideally good practices in many places and possibly
>>>> some centralization but only for the core things. If if try to centralize
>>>> and decide for everything, then we will have a sub-optimal core (because
>>>> work is diluted with non core things) and a frustrated community that see
>>>> its freedom to innovate restricted.
>>>>
>>>>  Now
>>>> <my personal opinion about Launchpad and bzr>
>>>> it does the job, but hell it doesn't do it well. Here it's very common
>>>> that bzr branch --stack addons takes over 20 minutes (1h30 without stacked)
>>>> vs 8 minutes for the same Github shallow clone. Want to compare
>>>> purchase_requisition in 7.0 against trunk? in git it's a breeze, in bzr
>>>> slow as hell. If you have a bzr branch that you want to do pull after some
>>>> 2 months (branch for some customer?), bzr pull can easily take more than 1
>>>> hour... (and we tried everything, local bzr mirrors etc...)
>>>>  Also I don't see much activity on bzr repo to fix this. It isn't
>>>> really cloud ready and it's like both bzr and Launchpad are loosing the
>>>> battle here.
>>>> So I'm perfectly okay to contrib on Launchpad. But when I do so, I
>>>> always think it's temporary and that one day anyway Launchpad will probably
>>>> announce that it's closing and we will need to put all these little team
>>>> and karma in the trash and move all that to some git (or hg) forge.
>>>> Other things I don't like about LP: it really badly sells a project:
>>>> navigation is awkward to newcomers, documentation isn't integrated, project
>>>> dynamism doesn't drive to adhesion. Also, we aren't as rich as SAP or
>>>> Microsoft, right? So what we need? We need contributors hell! but again
>>>> Launchpad badly reward contributors vs Github that does it so well that a
>>>> Github profile starts to be what a recruiter will look at to evaluate a job
>>>> candidate (compare this to hackable karma...)
>>>> Anyway this is just
>>>> </my personal opinion about Launchpad and bzr>
>>>>  (now I said it)
>>>>
>>>>
>>>>  So as a conclusion:
>>>> I say no to a dogma that we should group modules inside the same
>>>> branches. That's only a temporary work-around while some believe or make
>>>> believe things like apps is more important than pip modules.
>>>>
>>>>  In the meanwhile, it's not because that there are several branches
>>>> that there cannot be projects hub and their teams: a common team could take
>>>> care of several projects, on several branches and possibly forges.
>>>>
>>>>
>>>>  Finally, I think we should investigate the great work of Vo Minh Thu
>>>> regarding OpenERP module packaging:
>>>> http://assertive.io
>>>>
>>>>  My issue still is: if we have a new addons version at every commit,
>>>> then for any delta update pip update of the addons it will probably
>>>> re-download 400+ Mo of all the fresh addons vs a few ko if you were doing a
>>>> bzr or git pull. Again, not exactly cloud ready...
>>>> Given the very relative stability of OpenERP, I only have seen success
>>>> of people doing these delta updates on their projects to fix bugs as they
>>>> encounter and fix them. Today I don't see this working with assertive yet.
>>>> I may just be dreaming, but instead I was thinking about something
>>>> like: for all modules we have git-subtree (no working bzr equivalent!) cron
>>>> that put every module in a single branch: the module version get
>>>> incremented only if this particular module receives a commit. Possibly, a
>>>> translation commit is the z of the x.y.z semver. Then doing pip update
>>>> would re-download the module only if it received commits, and possibly only
>>>> if these commits aren't just translation.
>>>>
>>>>
>>>>  My two 2 cts. Thanks to all for these input about that essential
>>>> topic.
>>>>
>>>>
>>>>  --
>>>>  Raphaël Valyi
>>>> Founder and consultant
>>>> http://twitter.com/rvalyi <http://twitter.com/#%21/rvalyi>
>>>>  +55 21 2516 2954 <%2B55%2021%202516%202954>
>>>> www.akretion.com
>>>>
>>>>
>>>>
>>>>
>>>>  --
>>>> Mailing list: https://launchpad.net/~openerp-community-reviewer
>>>> Post to     : openerp-community-reviewer@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~openerp-community-reviewer
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>>
>>  --
>>
>>
>>   *camptocamp*
>> INNOVATIVE SOLUTIONS
>> BY OPEN SOURCE EXPERTS
>>
>>  *Joël Grand-Guillaume*
>> Division Manager
>> Business Solutions
>>
>>  +41 21 619 10 28
>> www.camptocamp.com
>>
>>
>
>
> --
>
>
>   *camptocamp*
> INNOVATIVE SOLUTIONS
> BY OPEN SOURCE EXPERTS
>
>  *Joël Grand-Guillaume*
> Division Manager
> Business Solutions
>
>  +41 21 619 10 28
> www.camptocamp.com
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-community
> Post to     : openerp-community@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openerp-community
> More help   : https://help.launchpad.net/ListHelp
>
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-community
> Post to     : openerp-community@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openerp-community
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-community
> Post to     : openerp-community@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openerp-community
> More help   : https://help.launchpad.net/ListHelp
>
>