holland-coredev team mailing list archive

[Andrew Garner <muzazzi@xxxxxxxxx>]

I just pushed out holland 1.1.0a1 (an alpha release, with the 1.1
series codenamed amsterdam) which includes several niceties on the
path to a new holland major release.  This includes most of the items
from the holland 1.1 plans here:

http://hollandbackup.org/Holland1-1

With the git repo pushed here:

https://github.com/holland-backup/holland/tree/amsterdam

Developer documentation is forthcoming but basic user documentation on
most of the new core features can be found here:

http://docs.hollandbackup.org/amsterdam/

Some of the new features include:

* New config parser which handles multi-line values, %include (so
config files can pull in other confg files) and generally better error
reporting through the configspec mechanism as well as a more flexible
configspec mechanism going forward.

* Significant refactoring of holland-core with a working test suite
(with nightly runs) and pylint (static code analysis) reports.  Right
now there's about 80% coverage for holland-core and a lot of low
hanging fruit remaining.  See the jenkins project here:

http://jenkins.hollandbackup.org/job/holland-amsterdam/

* A general hook mechanism built on the signal/slot model of django's
dispatch library.  This allows both simple shell command hooks or
somewhat richer python hooks.  For instance, there is an
implementation in holland-common to rewrite [mysql:client] sections to
connect to a mysql-mmm read-only server.  In general there are
mechanisms now to fiddle with both a backup config file and the backup
directory prior to (or after) handing off to a backup plugin.

* Refactored output plugin support.  This is basically the 1.0
compression support refactored to be built on to a pluggable 'stream'
api that allows non-compression plugins (e.g. encryption, various
cloud storage apis, etc.);  currently there are just compression
implementations but the stream plugin system is fairly generic.

* The holland cli frontend has been simplified and split out from the
holland-core api.  holland-core is now just an API and holland-cli
provides the traditional command line front-end.  It should be
relatively straightforward to provide additional interfaces to holland
now without jumping through hoops.   Almost all of the ugly bootstrap
code in 1.0 has been removed.

* Space estimation is now a generic pre-backup plugin and supports
multiple methods now.  The default is 'plugin' which is the 1.0
behavior - asking the backup plugin itself to estimate how big a
backup might be.  A directory (e.g. estimation-method =
dir:/var/lib/mysql), a constant (e.g. estimation-method = const:4G) or
a shell command that returns du compatible output (e.g. cmd:du -sh
/var/lib/mysql | cut -f1) can be used now.   This makes estimation
much more flexible and can avoid inefficencies of some of the plugin
estimation methods as needed.

* The mysqldump backup plugin has been simplified and cleaned up
significantly.  Notably it now supports a 'parallelism=N' (default
N=1) option to when using file-per-database to dump multiple databases
in parallel.

* Logging has been reworking a bit and, while still a work in
progress, should be somewhat cleaner than what we have in 1.0.

This release is intended to be 100% compatible with the holland 1.0
config format.   There are still a several edge cases to track down I
suspect, but upgrading from 1.0 -> 1.1 has worked in my environments.
:)

This is dubbed alpha1 and has quite a few bugs lurking still, but I
think the improvements are rather nice and open a lot of
possibilities.  Some interesting additional plugins should be
forthcoming in the next few alpha releases.

~Andy