maria-developers team mailing list archive

[Henrik Ingo <henrik.ingo@xxxxxxxxxxxxx>]

Kristian, Serg, Colin

Thanks for showing an interest in my work with the MariaDB buildsystem
and bakery scripts. During Open DB camp I finally managed to finish
the last pending task and published a blog post about it. So in this
mail I'll summarize once more what was done and highlight how it may
be relevant to MariaDB, so you can pick enhancements that you find
useful.

This blog post has links to the series published in February:
http://openlife.cc/blogs/2011/march/overview-and-archeological-exploration-mepsql-bakery
It links to posts 1-7.

And this is the new one:
http://openlife.cc/blogs/2011/may/easy-way-manage-virtualcloud-images-outside-userdata-and-runurl-scripts
let's call it number 8.

So the high level of what I've done is this:

5) http://openlife.cc/blogs/2011/february/mariadb-and-mepsql-buildbot-setups

Instead of the standard BuildSlave class, use my own subclass of
EC2LatentBuildSlave. This allows migrating the buildbot configuration
to EC2 cloud.

Relevance to MariaDB: If you don't want to use a cloud service (faster
turnaround times due to more servers in parallel, but costs more than
what you do now) not much.

However, there is a small value in the refactoring effort itself. Now
you have the runvm stuff hardwired into the test cases. The approach
used with EC2LatentBuildSlave abstracts the handling of virtual
instances into python code, and the test cases are identical whether
you have the classic physical slaves or virtual/cloud slaves. They are
cleaner to read and can be moved between classic slave and virtual
slave without modifications. Unfortunately, to do this, you'd need to
both a) write a subclass of AbstractLatentBuildSlave that does what
runvm does for you today, and b) clean up the runvm parts from your
builds/tests, since I only did so for a small part and we have
probably diverged since then anyway.


As part of (5), I also produced an apt repository, not just standalone
deb packages. But you have implemented that separately too. I advised
you to use something else than ~ for the separator in filename.


6) http://openlife.cc/blogs/2011/february/dealing-cambrian-explosion-12-how-parameterize-package-name-source-and-binary-ta
7) http://openlife.cc/blogs/2011/february/dealing-cambrian-explosion-22-parameterizing-package-name-deb-files

Parameterize the package name (mysql, mariadb, percona-server,
mepsql...) in the scripts that build packages (with sed and other
shell magic).
See http://bazaar.launchpad.net/~mepsql-committers/mepsql-bakery/trunk/view/head:/bakery/mysql-version

Relevance to MariaDB: Nothing as such, however, if you adopted these
changes, it would enable you to share the packaging scripts and even
buildbot setup with Percona. Possibly also with Linux distributions
that are packaging multiple MySQL forks now, although their problem is
not quite the same. (you produce a fork for many distros, they package
many forks for one distro)

Personally, I like this enhancement both because it centralizes all
the variables that end up in package names and version strings, and
because it would enable all forks (including, theoretically, Oracle
MySQL) to share the same packaging script. This is not at all possible
with how things are packaged traditionally, where you assume the
package name to be constant and it is used as a hardwired value around
all scripts and filenames.

I didn't yet do this for RPMs and don't know when, if ever, I will.


8) http://openlife.cc/blogs/2011/may/easy-way-manage-virtualcloud-images-outside-userdata-and-runurl-scripts

Instead of configuring a vm image and saving it, only use generic
images and run external script on boot.

Relevance to MariaDB: high. I know you complained about this in your
otherwise perfectly automated build system. Even in my small project I
experienced huge gains during development thanks to this approach. You
can use this technique even with your current runvm setup. (You just
need to edit your images one last time to enable fetching of the
remote configuration script.)


That's all. I hope some of this is seen useful.

henrik


-- 
henrik.ingo@xxxxxxxxxxxxx
+358-40-8211286 skype: henrik.ingo irc: hingo
www.openlife.cc

My LinkedIn profile: http://www.linkedin.com/profile/view?id=9522559