← Back to team overview

maria-developers team mailing list archive

Re: MariaDB + PBXT

 

Hi Kristian,

On May 11, 2009, at 4:24 PM, Kristian Nielsen wrote:

Stewart, thanks for providing this procedure!

Nice to know that this method with renaming to common paths and merging can
work.

Paul McCullagh <paul.mccullagh@xxxxxxxxxxxxx> writes:

So to correct this I will have to redo the integration of PBXT is into
MariaDB.

Yes, I think we should re-do the merge with this tracking of history and
merging.

I spent some time today playing with this technique, as I would like to be really sure that we have a good base for working with the PBXT code in the future, and also I would like to use the same technique for merging XtraDB.

So I created a MariaDB tree with the PBXT merge re-done using Stewart's technique, taking some care to get everything done correctly to smooth the
future work.

If you want to use it as-is, I write the exact commands I used below if you want to do something similar for Drizzle. If you want to re-do it yourself to
tweak it or control things fully, that is perfectly fine with me as
well. Whatever works best for you.

Wow, excellent! Thanks, you have saved me the work :)

Just let me know if you want me to push this tree to MariaDB, or prepare
another tree for me to merge.

I put my merged tree here:

   lp:~knielsen/maria/mariadb-5.1-pbxt-merge-new

I just built it and tested:

bzr merge lp:pbxt

And all looks good :)

Note that this tree is identical source-wise with the existing lp:maria tree, just the history now includes all of the PBXT changesets so history and merges can be tracked as per Stewarts method. So this does not include newest PBXT
yet, just PBXT up to revision 608
(revid:paul.mccullagh@xxxxxxxxxxxxx-20090402202852-wa2fbcmrdy7gda2f).

OK. Great.

I did a quick test merge of newest PBXT just to check things, and it seems to work well. Most changes automerged. For deleted/renamed/newly added files, a conflict is flagged that must be resolved manually. Changes to files in PBXT that are omitted from the MariaDB source (COPYING, README, etc) also are flagged as a conflict that needs to be resolved. This is to be expected.

Yup. OK.

I did see some wierd stuff in some testing. Eg. the `bzr diff` of my merge changeset is not empty as expected, rather it shows one diff removing a file and another identical (but reverse) diff adding that file back. I also saw a
strange bzr error message in one case of testing merge. I guess we are
(ab)using some uncommon corners of bzr here, but hopefully any problems/bug in
bzr can be fixed.

One think I wonder is what to do with the PBXT test suite in
mysql-test/suite/pbxt/. In my merge I merged it into place with `bzr mv` so
that merge tracking and history is kept.

So I am not entirely sure how I should handle changes in the future. Maybe you have a suggestion.

However, I don't know how well this
will work for .result files, as I guess they are going to differ between
MariaDB, Drizzle, and MySQL?

Yes, this is a problem. And some of the tests have also be changed for MariaDB.

Not 100% sure how we will deal with this. Could
be that .result files cannot be meaningfully merged automatically. In any case, I think this is a usable starting point, and we can refine the .result handling later as we understand better how to do it. Just let me know if you
think otherwise.

Yes, I think we should just leave it as is for the moment, until we have a better idea.

My original idea with the PBXT test suite is that it will act as a basic verification for the engine within MariaDB. The main testing of PBXT will probably continue as a plug-in for MySQL.

In the PBXT tree, and can do "make test", and PBXT is automatically tested using the MySQL server that it was compiled with.

So I was thinking of PBXT test suite for MariaDB as a branch of the PBXT tests that would never be merged again. Instead any bugs that turn up in MariaDB with PBXT would be added to this test suite. And, the test suite will be continually updated for any changes in MariaDB itself.

This is similar to how things are in Drizzle, where PBXT is also tested against a different set of tests. Here I was also not planning to merge tests between Drizzle and the PBXT tree.

One good thing about the Drizzle tests though, is that the same tests run with both InnoDB and PBXT. Would be great to add something like that to MariaDB.

Here are the exact commands I did to make the merge:

bzr branch -r'revid:paul.mccullagh@xxxxxxxxxxxxx-20090402202852-wa2fbcmrdy7gda2f ' pbxt tmp-pbxt-rename
 cd tmp-pbxt-rename/
 bzr co
 bzr mkdir storage
 bzr mkdir storage/pbxt
 bzr mv `ls|grep -v storage` storage/pbxt/
bzr commit -m"Move all PBXT files into storage/pbxt/ in preparation for merging into MariaDB."
 cd ..
bzr branch -r'revid:paul.mccullagh@xxxxxxxxxxxxx-20090407105746-tolv5dita1d3eavm ' mariadb-5.1 tmp-mariadb-pbxt-merge
 cd tmp-mariadb-pbxt-merge/
 bzr co
 bzr merge ../tmp-pbxt-rename -r0..-1
 bzr mv storage storage.new
 bzr mv storage.moved storage
 bzr rm storage/pbxt
 bzr rm mysql-test/suite/pbxt/t
 bzr rm mysql-test/suite/pbxt/r
 bzr mv storage.new/pbxt/test/mysql-test/t mysql-test/suite/pbxt/t
 bzr mv storage.new/pbxt/test/mysql-test/r mysql-test/suite/pbxt/r
 bzr mv storage.new/pbxt storage/pbxt
 bzr rm --force storage.new
# bzr rm + manual patch. Applying same changes done manually in previous PBXT merge.
 bzr resolved storage
bzr ci -m "Re-merge PBXT into MariaDB to get proper history and merge tracking."
 cd ..
 bzr branch mariadb-5.1 mariadb-5.1-pbxt-merge-new
 cd mariadb-5.1-pbxt-merge-new
 bzr co
 bzr merge ../tmp-mariadb-pbxt-merge
 bzr ci -m "Pull new/improved PBXT merge into latest MariaDB trunk."

So I used the revisions of MariaDB and PBXT that were also used to do the original merge. And I manually applied the same changes in the merge that you apparently did in your original merge of PBXT into MariaDB, mainly deleting some files like COPYING, configure.in, removing a number of non-pbxt test
cases, and fixing some .result files.

So I hope this will be a good base for future merging, with this it should be possible for either PBXT or MariaDB people to pull latest from PBXT into MariaDB without either getting in the way of the other. And it should be quite
useful to us to have PBXT history available if debugging etc.

Yes. I agree. This works well (thanks Stewart).

Please go ahead and merge to the MariaDB trunk.

Thanks!

Paul





On May 11, 2009, at 7:17 AM, Stewart Smith wrote:

With BZR you can merge two trees without a common ancestor. This is
useful for things like storage engines, file systems, so when they're
merged, you can bring the whole revision history in.

This is how I did it for PBXT and Drizzle:

bzr branch lp:drizzle drizzle-pbxt
bzr branch lp:pbxt pbxt
bzr branch pbxt pbxt-rename
pushd pbxt-rename
mkdir storage/pbxt

Is this correct, or should it be bzr mkdir storage/pbxt?

I used `bzr mkdir`, got an error from bzr otherwise.

bzr merge ../pbxt-rename -r1..-1
(fix up a conflict on storage/ or something... pretty easy, this is
just
done so that auto merges happen on file ops in main pbxt)

I used -r0..-1 (not sure if it makes a difference?). And above log shows how I
resolved the conflicts.

bzr commit

and then, when mainline pbxt is updated
pushd drizzle-pbxt
bzr merge lp:pbxt

- Kristian.



--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com






References