← Back to team overview

drizzle-discuss team mailing list archive

Re: Toru Maesaka: End of Year Progress on BlitzDB


On Fri, Jan 1, 2010 at 8:03 AM, Toru Maesaka <dev@xxxxxxxxx> wrote:
> G'day Mark,
>> Is the persistent hash table in Tokyo Cabinet crash safe? I ask because of
>> text in http://1978th.net/tokyocabinet/spex-en.html. I did not find a
>> definition of 'broken' nor mention of commands to fix a broken database.
> The honest answer to this is, "NO, it's not crash safe". If one is
> lucky, she would be able to repair the table(s) using REPAIR TABLE but
> this is not guaranteed (best-effort only). IIRC, TC has an internal
> flag that represents whether the table was left open. The repair
> function will try it's best to repair the database file in that state
> (though I haven't implemented this yet). So, BlitzDB is not much safer
> than MyISAM.
> My answer to this question is the same as Tokyo Tyrant's philosophy.
> Reduce unit durability in return for higher throughput. Users are
> recommended to replicate to gain HA. This isn't a good answer for
> those that doesn't have the luxury of running multiple physical
> servers but this is my policy at the moment...
> HOWEVER, I'm always open to ideas and patches to make this engine
> better. So if you have any cool ideas (or know of anyone), I would
> love to talk about it :)

What is done for repair? I cannot find references to 'repair' in Tokyo
Cabinet and I am not sure where to search in the BlitzDB code.

The reason that I asked about this is because someone I know used TC
and loved the performance. Some time later they realized it wasn't
crash safe. I think any persistence layer should write NOT CRASH SAFE
on the project overview page with a link to the semantics. But TC
isn't alone in making it easy to miss this potential problem --
MyISAM, MySQL slave replication state and MongoDB do the same.

Follow ups