maria-discuss team mailing list archive

[Rich Prohaska <prohaska7@xxxxxxxxx>]

Hello All,

Sometimes, TokuDB gets left in a state where it thinks that parts of a
table exist (as observed in TokuDB's file map), but not all of the
underlying files (the TokuDB data files and/or the frm file) exist.  This
probably occurs due to the interaction between the non-transactional DDL
that MySQL/MariaDB uses to create or drop tables, and the transactional
TokuDB engine underneath.  There is some bug there.

So, what can one do to remove broken TokuDB file maps?  There is an
internal fractal tree called the tokudb.directory that names high level
data names to an underlying file that the data is stored in. The
tokudb_file_map information schema dumps out the contents of this tree.
The key to this tree is the high level data name (called a dname).  In
TokuDB, this data name consists of the database name, the table name, and
the index name.  There is also a tree for the table (called status) that
contains table wide metadata.

I added the tokudb_delete_dnames_from_directory tool to the
https://github.com/prohaska7/mariadb-server/tree/fix-tokudb-filemap
branch.  This standalone tool can be used to delete dnames from the
tokudb.directory.   Standalone means that MariaDB must have been previously
shutdown cleanly (which puts the tokudb recovery logs in a nice clean state
so that the tool does not have to run crash recovery).

The input to the tool is the MariaDB data directory with nice and clean
tokudb recovery logs along with a sequence of dname's.  The tool
transactionally deletes rows from the tokudb.directory fractal tree, which
updates the recovery logs.

Building the tool requires a MariaDB build environment a simple compile
sequence something like this:

gcc -g -o tokudb_delete_dnames_from_directory
tokudb_delete_dnames_from_directory.cc -I$TFT/buildheader -L$TFT/src
-ltokufractaltree

Running the tool looks something like this:

Get the dnames of interest from the tokudb_file_map.  In this case, they
are ./test/s-main and ./test/s-status.

shutdown MariaDB

./tokudb_delete_dnames_from_directory
~/projects/mariadb-server-install/data ./test/s-main ./test/s-status