← Back to team overview

drizzle-discuss team mailing list archive

Re: File Deletion issues for Storage Engines


On Dec 10, 2009, at 9:41 PM, Toru Maesaka wrote:

G'morning Drizzlers,

I stumbled across a minor issue regarding deleting files generated by
storage engines that aren't in drizzle. Specifically speaking, files
that are to be deleted on "DROP DATABASE x" with
'mysql_rm_known_files()' in drizzled/db.cc

The problem is that the deletable extensions are predefined like so in db.cc:

const char *del_exts[]= {".dfe", ".blk", ".arz", ".BAK", ".TMD",".opt", NULL};

and say, if the file(s) that I wanted drizzle to delete has a ".bzd"
extension, which happens to be BlitzDB's datafile extension. It gets
caught into this particular case in the deletion loop (I don't know
who commented that function body but it's true...):

if (find_type(extension, &deletable_extentions,1+2) <= 0)
   ass ass ass.

   strange checking for magic extensions that are then deleted if
   not reg_ext (i.e. .frm).

and (previously) we'd err out on drop database if files not matching
   engine ha_known_exts() or deletable_extensions were present.

presumably this was to avoid deleting other user data... except if that
   data happened to be in files ending in .BAK, .opt or .TMD. *fun*

Once we exit the deletion loop (without files being deleted because
the loop is continued), drizzzled then attempts to execute rmdir. This
fails because files still exist in the directory. Meaning, "DROP
DATABASE x" will always fail.

Storage Engines do not have this problem if it doesn't declare
HTON_HAS_DATA_DICTIONARY. This is because drizzle automatically
generates a .dfe file. This allows drizzled to surpass that awkward
condition and make it into this condition:

if (db && !my_strcasecmp(files_charset_info,
                            extension, ".dfe"))

which will create a list of files to delete in the function body. It
then gets to here:

 if (tot_list)
   if (rm_table_part2(session, tot_list))
     goto err;

which will get rid of the files for us through

One solution that I can think straight off my head is to aggregate
extension types that the storage engines have declared in
StorageEngine::bas_ext(). So instead of hardcoding *del_exts[], we
could make it dynamic.

Another solution is something I talked to Stewart about earlier today.
There's a hander::drop_database() interface in drizzle already so we
could rework that to be doDropDatabase() and call that instead of
mysql_rm_known_files() on "DROP DATABASE x". That is, let the storage
engine take care of deleting files that it had generated inside a
particular directory.


Makes absolute sense! The engine created the files, let the engine delete them.

After all engines have had a chance to delete the files, rmdir should work. If not, an error can be generated.

Then there is no need for del_exts[] at all.

Would be great if I can get feedbacks on this matter :)

Toru Maesaka <dev@xxxxxxxxx>

Mailing list: https://launchpad.net/~drizzle-discuss
Post to     : drizzle-discuss@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help   : https://help.launchpad.net/ListHelp

Paul McCullagh
PrimeBase Technologies

Follow ups