coapp-developers team mailing list archive

[Mateusz Łoskot <mateusz@xxxxxxxxxx>]

2011/12/20 Vincent Povirk <madewokherd@xxxxxxxxx>:
> I think we should make it a policy to never convert other projects'
> revision control to git, if they're not already git, because the extra
> software it requires packagers to learn, install, and use does not
> convey any extra benefits.

This point is either clear to me nor I agree with the arguments.
But, let's attack it from different angle:

1) Simple library X written in C hosted on SourceForge.net
2) Source code of X is hosted in Subversion (tags, branches, etc.)
3) I want to package X with CoApp (package for stable release 1.0.0,
package development version too)
4) I want to maintain those packages in future (e.g. update package
for 1.0.0 to 1.0.1 once it is released, add new release 2.0.0, etc.)

How do I do it?

> (And I REALLY don't want to have to install
> bzr or hg or perforce on my machine to update a package, EVER.)

You don't have to if you don't want to.
The option I described is only valid in case there is software
hosted in Subversion which one wants to package for CoApp.

> git-svn (which is something completely different from git-svn-mirror)
> might get a pass because it's not an extra thing to install and I
> already mostly know how to use it, but dammit it's still an extra
> thing to learn and it still doesn't help us.

git-svn-mirror *is* git-svn. It is just a thin wrapper on top of git-svn
which simplifies the process of mirroring.

> The way I see it, we need source control to help us do the following things:
>  * Publish the code we're using and the changes we've made.

How do you get the source code?
Where do you host it?
How you maintain it and bring changes in the upstream into your copy
of source code?

>  * Separate the changes we've made from the pristine, upstream version.

I have included this requirement in the proposed workflow.

>  * Trace each one of our changes to an individual author, and
> preferably learn something about why each change was made.

I'm not sure what it means.

>  * Update our version of the code with new code from upstream, while
> still keeping our changes intact.

Yup. It is merging changes from upstream.

>  * Maintain multiple stable branches for projects that have multiple
> stable revisions that we need to care about. Most importantly, our set
> of changes will be very similar for each branch, but not identical,
> and our source control should help us keep it straight.

Indeed. That's why we need to host complete sources of upstream in
revision-controlled system,
in GitHub, with tags and branches from upstream.

Just look how Subversion workflow works.
1) trunk -> upstream, development
2) I want to release 1.0, feature freeze, make branches/1.0
3) I release, then I tag form branches/1.0 as tags/1.0.0
4) I have fix for 1.0, then I merge fix into trunk/ and into branches/1.0
5) I make bugfix release from branches/1.0 so I create tags/1.0.1

If we have mirror of this SVN repo in GitHub, then we just release new
package updated
from tags/1.0.0 to tags/1.0.1. We don't have to maintain
releases/version of upstream on our own.

>  * Backport important fixes that have not yet made it to an upstream
> release, or that are not present on an older stable branch that we
> still care about.

Yes, indeed. We generate patch from our repo at GitHub, preferably
for such fixes we want to contribute back to upstrea, we maintain them
as separate branches in Git:
coapp-<NAME OF FIX or FIXES SET>
So, it is not confused with mirrored branches.

> In order to do everything consistently and with the same tools, we're
> using git for everything. This requires us to get code from projects
> not using git into git somehow.

What is the somehow?
How you bring the code from SVN, Mercurial, whatever into Git?
How you think to synchronise your copy in Git from changes in the
upstreams in foreign repos?

> There are two possible approaches to
> doing this. One is to import the project's entire history, in whatever
> original form it takes, using some automated tool.

That's what SVN to Git mirroring is for.

> I've already explained why I don't like this approach.

IMHO, such reasons sound a bit artificial to me.
But, if there is a simpler method to achieve proper
mirror-maintain-update-backport
workflow, I'd love to learn about it (see further below).

> I think we should instead
> import snapshots of the project into git manually as needed (in most
> cases, I would advocate for doing this every release, but every
> project is different and some may not have releases or may not have
> them often enough).

Release snapshots in SVN (and many more RCS's) are called tags.
We can pull them directly from RCS or we can pull from .tar.gz/.zip packages.
It is may be technically easier to import the latter, I agree.
But, for long-term maintenance, generating patches, I think it is
easier to maintain mirror in Git with complete history upstream.

Again, I'd love to learn I'm wrong (see further below).

> That approach looks like this:
>  * Create a new Git repository with a branch named 'upstream' (or
> something like upstream-2.6 if we need to track multiple upstream
> branches).
>  * Download a tarball of a release of this project, dump the files in
> the repository (without the project-version prefix they usually add),
> and commit them.
>  * Create a master branch based on the upstream branch, and do the
> packaging work.
>  * Create a repository in the coapp-packages organization containing
> both the master and the upstream branch. You can have mirrors in your
> own github account if that is how you like to use github. I won't
> judge.

If this workflow is sufficient to what we need, it makes sense.

Do we consider having -devel packages from development version of codes?
I think it is important to be able to create CoApp packages for mylib-devel.

> To update:
>  * Checkout the upstream branch (if you prefer to keep a separate
> repository for every branch, you're doing it wrong, but git will let
> you), remove all the files, add the new files, and commit.
>  * Checkout the master branch.
>  * Merge the new upstream branch into the master branch.
>  * Test to make sure you didn't completely break things.
>  * Push it to coapp-packages or make a pull request or whatever it is
> you kids with your githubs are doing these days.

(Here comes the place I referred from "see further below").

AFAIU, you are proposing Vendor Branches, aren't you?
The concept is popular and described in SVN book
http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.advanced.vendorbr
The Vendor Branches do not require us to use any other RCS tools than Git.

Concept of Vendor Branches is also present in Git world.
http://happygiraffe.net/blog/2008/02/07/vendor-branches-in-git/

Basically, there are two common approaches to maintain thrid-party sources:
1) git svn
2) vendor branches made from release packages of third-party sources

They are similar but slightly different: 1) preserves complete history
and may make it
easier to generate patches and contribute changes back to upstream.
Thus, this is my preferred appraoch.

However, I understand CoApp requirements are slightly different, so we
don't need
complete history to maintain packages.
Then I agree indeed, vendor branches are simpler, saner approach for us.

My general point is, let's not reinvent the wheel for new workfllow.
The problem we are trying to solve is an old one, well-solved
and developers are familiar with solutions. It is like with design patterns.
If CoApp uses common approach, it will be easier for contributors to adapt.

> As for the idea of keeping one's private repository separate as a
> "testing" stage, this is entirely separate from the question of
> importing changes from a foreign RCS. I think that if there's a need
> for this, it should be done with separate branches in the same
> repository. That is what branches are for. Private repositories are
> for changes you have made that are not yet approved to go in the
> official repository.

Yes, I agree.

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net