maria-developers team mailing list archive

[Paul McCullagh <paul.mccullagh@xxxxxxxxxxxxx>]

Hi Kristian,

On May 11, 2009, at 4:24 PM, Kristian Nielsen wrote:

> Stewart, thanks for providing this procedure!
>
> Nice to know that this method with renaming to common paths and  
> merging can
> work.
>
> Paul McCullagh <paul.mccullagh@xxxxxxxxxxxxx> writes:
>
> > So to correct this I will have to redo the integration of PBXT is  
> > into
> > MariaDB.
>
> Yes, I think we should re-do the merge with this tracking of history  
> and
> merging.
>
> I spent some time today playing with this technique, as I would like  
> to be
> really sure that we have a good base for working with the PBXT code  
> in the
> future, and also I would like to use the same technique for merging  
> XtraDB.
>
> So I created a MariaDB tree with the PBXT merge re-done using  
> Stewart's
> technique, taking some care to get everything done correctly to  
> smooth the
> future work.
>
> If you want to use it as-is, I write the exact commands I used below  
> if you
> want to do something similar for Drizzle. If you want to re-do it  
> yourself to
> tweak it or control things fully, that is perfectly fine with me as
> well. Whatever works best for you.

Wow, excellent! Thanks, you have saved me the work :)

> Just let me know if you want me to push this tree to MariaDB, or  
> prepare
> another tree for me to merge.
>
> I put my merged tree here:
>
>    lp:~knielsen/maria/mariadb-5.1-pbxt-merge-new

I just built it and tested:

bzr merge lp:pbxt

And all looks good :)

> Note that this tree is identical source-wise with the existing  
> lp:maria tree,
> just the history now includes all of the PBXT changesets so history  
> and merges
> can be tracked as per Stewarts method. So this does not include  
> newest PBXT
> yet, just PBXT up to revision 608
> (revid:paul.mccullagh@xxxxxxxxxxxxx-20090402202852-wa2fbcmrdy7gda2f).

OK. Great.

> I did a quick test merge of newest PBXT just to check things, and it  
> seems to
> work well. Most changes automerged. For deleted/renamed/newly added  
> files, a
> conflict is flagged that must be resolved manually. Changes to files  
> in PBXT
> that are omitted from the MariaDB source (COPYING, README, etc) also  
> are
> flagged as a conflict that needs to be resolved. This is to be  
> expected.

Yup. OK.

> I did see some wierd stuff in some testing. Eg. the `bzr diff` of my  
> merge
> changeset is not empty as expected, rather it shows one diff  
> removing a file
> and another identical (but reverse) diff adding that file back. I  
> also saw a
> strange bzr error message in one case of testing merge. I guess we are
> (ab)using some uncommon corners of bzr here, but hopefully any  
> problems/bug in
> bzr can be fixed.
>
> One think I wonder is what to do with the PBXT test suite in
> mysql-test/suite/pbxt/. In my merge I merged it into place with `bzr  
> mv` so
> that merge tracking and history is kept.

So I am not entirely sure how I should handle changes in the future.  
Maybe you have a suggestion.

> However, I don't know how well this
> will work for .result files, as I guess they are going to differ  
> between
> MariaDB, Drizzle, and MySQL?

Yes, this is a problem. And some of the tests have also be changed for  
MariaDB.

> Not 100% sure how we will deal with this. Could
> be that .result files cannot be meaningfully merged automatically.  
> In any
> case, I think this is a usable starting point, and we can refine  
> the .result
> handling later as we understand better how to do it. Just let me  
> know if you
> think otherwise.

Yes, I think we should just leave it as is for the moment, until we  
have a better idea.

My original idea with the PBXT test suite is that it will act as a  
basic verification for the engine within MariaDB. The main testing of  
PBXT will probably continue as a plug-in for MySQL.

In the PBXT tree, and can do "make test", and PBXT is automatically  
tested using the MySQL server that it was compiled with.

So I was thinking of PBXT test suite for MariaDB as a branch of the  
PBXT tests that would never be merged again. Instead any bugs that  
turn up in MariaDB with PBXT would be added to this test suite. And,  
the test suite will be continually updated for any changes in MariaDB  
itself.

This is similar to how things are in Drizzle, where PBXT is also  
tested against a different set of tests. Here I was also not planning  
to merge tests between Drizzle and the PBXT tree.

One good thing about the Drizzle tests though, is that the same tests  
run with both InnoDB and PBXT. Would be great to add something like  
that to MariaDB.

> Here are the exact commands I did to make the merge:
>
>  bzr branch -r'revid:paul.mccullagh@xxxxxxxxxxxxx-20090402202852-wa2fbcmrdy7gda2f 
> ' pbxt tmp-pbxt-rename
>  cd tmp-pbxt-rename/
>  bzr co
>  bzr mkdir storage
>  bzr mkdir storage/pbxt
>  bzr mv `ls|grep -v storage` storage/pbxt/
>  bzr commit -m"Move all PBXT files into storage/pbxt/ in preparation  
> for merging into MariaDB."
>  cd ..
>  bzr branch -r'revid:paul.mccullagh@xxxxxxxxxxxxx-20090407105746-tolv5dita1d3eavm 
> ' mariadb-5.1 tmp-mariadb-pbxt-merge
>  cd tmp-mariadb-pbxt-merge/
>  bzr co
>  bzr merge ../tmp-pbxt-rename -r0..-1
>  bzr mv storage storage.new
>  bzr mv storage.moved storage
>  bzr rm storage/pbxt
>  bzr rm mysql-test/suite/pbxt/t
>  bzr rm mysql-test/suite/pbxt/r
>  bzr mv storage.new/pbxt/test/mysql-test/t mysql-test/suite/pbxt/t
>  bzr mv storage.new/pbxt/test/mysql-test/r mysql-test/suite/pbxt/r
>  bzr mv storage.new/pbxt storage/pbxt
>  bzr rm --force storage.new
>  # bzr rm + manual patch. Applying same changes done manually in  
> previous PBXT merge.
>  bzr resolved storage
>  bzr ci -m "Re-merge PBXT into MariaDB to get proper history and  
> merge tracking."
>  cd ..
>  bzr branch mariadb-5.1 mariadb-5.1-pbxt-merge-new
>  cd mariadb-5.1-pbxt-merge-new
>  bzr co
>  bzr merge ../tmp-mariadb-pbxt-merge
>  bzr ci -m "Pull new/improved PBXT merge into latest MariaDB trunk."
>
> So I used the revisions of MariaDB and PBXT that were also used to  
> do the
> original merge. And I manually applied the same changes in the merge  
> that you
> apparently did in your original merge of PBXT into MariaDB, mainly  
> deleting
> some files like COPYING, configure.in, removing a number of non-pbxt  
> test
> cases, and fixing some .result files.
>
> So I hope this will be a good base for future merging, with this it  
> should be
> possible for either PBXT or MariaDB people to pull latest from PBXT  
> into
> MariaDB without either getting in the way of the other. And it  
> should be quite
> useful to us to have PBXT history available if debugging etc.

Yes. I agree. This works well (thanks Stewart).

Please go ahead and merge to the MariaDB trunk.

Thanks!

Paul



> > On May 11, 2009, at 7:17 AM, Stewart Smith wrote:
>
> > > With BZR you can merge two trees without a common ancestor. This is
> > > useful for things like storage engines, file systems, so when  
> > > they're
> > > merged, you can bring the whole revision history in.
> > >
> > > This is how I did it for PBXT and Drizzle:
> > >
> > > bzr branch lp:drizzle drizzle-pbxt
> > > bzr branch lp:pbxt pbxt
> > > bzr branch pbxt pbxt-rename
> > > pushd pbxt-rename
> > > mkdir storage/pbxt
> >
> > Is this correct, or should it be bzr mkdir storage/pbxt?
>
> I used `bzr mkdir`, got an error from bzr otherwise.
>
> > > bzr merge ../pbxt-rename -r1..-1
> > > (fix up a conflict on storage/ or something... pretty easy, this is
> > > just
> > > done so that auto merges happen on file ops in main pbxt)
>
> I used -r0..-1 (not sure if it makes a difference?). And above log  
> shows how I
> resolved the conflicts.
>
> > > bzr commit
> > >
> > > and then, when mainline pbxt is updated
> > > pushd drizzle-pbxt
> > > bzr merge lp:pbxt
>
> - Kristian.



--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com