maria-developers team mailing list archive
Mailing list archive
Tools/suggestions for MySQL 5.6 merge
I have been doing a bit of preparation/planning for the upcoming merge of
I think the 5.6 merge (and 5.5 merge) is very important for MariaDB. There is
much stuff in there that we really need to make available to our users (and
probably some better left out). If we fail to keep the merges going, it will
leave the community in a very sorry situation; I do not see anyone else
stepping up to this task.
My idea is that we could merge MySQL 5.6 piece-by-piece. Several other
developers mentioned this idea at the Athens meeting:
- It makes it much easier to selectively merge or not merge stuff. For
example there are probably large chunks of optimiser changes that we do not
want to merge (or maybe merge only the testcases for). On the other hand we
probably want all InnoDB stuff.
- I am hopeful it makes the merge much more managable when done in smaller
pieces. Large reorganisations of code (header refactoring, copyright
changes) should be easier when not intermixed with other changes. Merges
from MySQL 5.5 -> MySQL 5.6 are probably also easier to deal with when done
in isolation (I assume we can get conflicts for these, though we already
have the equivalent changes in MariaDB 5.5).
- Easier to share the task between developers. One developer can do a single
piece in a few days, then pass the baton when he needs to switch to other
tasks (or just gets tired). Complex merges of large feature pushes can be
null-merged to allow main merge to progress, and in parallel another
developer can work on reworking the patch into the MariaDB code. For some
changes I think we will even be able to do the actual merge in parallel.
- Easier to plan and understand how far the merge is progressing. Eg. if we
merge M weeks worth of 5.6 pushes every N days, that allows to estimate
when we will catch up with Oracle development
- Easier debugging of test failures; by keeping the trees green between
pieces merged, new test failures will be known to be caused by relatively
smaller sets of changes, simplifying debugging.
I have done some work to facilitate this kind of piecewise merge. As it turns
out, it is not trivial to determine suitable pieces to merge. Oracle
developers use no discipline for the direction they do merges in, so the
sequence of pushes into lp:mysql-server ("trunk") is not immediately
apparent. So I took the time to reconstruct the list of pushes from reading
through the history; this was not overly hard to do.
I next wrote a script that scans the history based on this list of pushes, and
further annotates each push with some useful information:
- Size of diff for the patch
- Number of changesets merged (if a merge)
- Mark if this is a merge from MySQL 5.5
- One-line short comment for the push, either hand-crafted for selected
items, or else extracted from the commit message.
- Revision ID (including developer info)
It should be fairly simple to scan this list from the bottom, collecting
chunks of related or manageable pushes and merging them as one piece. Eg. A
sequence of small pushes could be merged as one, or a sequence of pushes by
one developer, or a single large feature push.
Here is an example extract of such an annotated list:
0 2* tor.didriksen@xxxxxxxxxx-20111101065343-ha3d79h1ayk9j7x9
[5.5] Null merge from 5.1/5.5
Minor test adjustment
0 2* tor.didriksen@xxxxxxxxxx-20111031091354-8p6s03s1zxerrp8p
[5.5] Null merge Bug#12406055 from 5.1/5.5
5380 15 jorgen.loland@xxxxxxxxxx-20111031081122-dnufl9r4li0i3ni6
[opt] Larger merge from optimiser tree
134 2* andrei.elkin@xxxxxxxxxx-20111028212219-cdcuh37i70k9o4ms
[5.5] Merge Bug#56299 (?) from 5.5
451 2 andrei.elkin@xxxxxxxxxx-20111028130851-mz4p2mg28y14wb7k
merge from trunk to local tree.
[opt] Bug#32254 post-merge fix (optimiser)
Bug#11747423 post-merge fix
"Default value for sort_buffer_size changed from 2M to 256K"
The first column is diff size, the second is number of changesets merged (or
empty for non-merge changesets). A "*" marks pushes that are only merges from
I pushed the script to lp:mariadb-tools, in /mysql-5.6-merge-stuff/. It also
includes the full output in 5.6-annotated.txt (it takes 15-20 minutes for bzr
to chew through the history on a fast multi-core machine).
I hope this will be useful for the merge. Comments/suggestions/questions very