← Back to team overview

drizzle-discuss team mailing list archive

Re: Toru Maesaka: End of Year Progress on BlitzDB

 

G'day Mark,

> Is the persistent hash table in Tokyo Cabinet crash safe? I ask because of
> text in http://1978th.net/tokyocabinet/spex-en.html. I did not find a
> definition of 'broken' nor mention of commands to fix a broken database.

The honest answer to this is, "NO, it's not crash safe". If one is
lucky, she would be able to repair the table(s) using REPAIR TABLE but
this is not guaranteed (best-effort only). IIRC, TC has an internal
flag that represents whether the table was left open. The repair
function will try it's best to repair the database file in that state
(though I haven't implemented this yet). So, BlitzDB is not much safer
than MyISAM.

My answer to this question is the same as Tokyo Tyrant's philosophy.
Reduce unit durability in return for higher throughput. Users are
recommended to replicate to gain HA. This isn't a good answer for
those that doesn't have the luxury of running multiple physical
servers but this is my policy at the moment...

HOWEVER, I'm always open to ideas and patches to make this engine
better. So if you have any cool ideas (or know of anyone), I would
love to talk about it :)

Cheers,
Toru

On Thu, Dec 31, 2009 at 11:54 PM, MARK CALLAGHAN <mdcallag@xxxxxxxxx> wrote:
>
>
> On Thu, Dec 24, 2009 at 3:00 AM, Planet Drizzle <emailer@xxxxxxxxxxxxxxxxx>
> wrote:
>>
>> Toru Maesaka: End of Year Progress on BlitzDB
>>
>> My open source friends might have noticed that I’ve been working quite a
>> bit on BlitzDB lately. To tell the truth, I had a hidden goal to get
>> Version-1 done by Christmas. Unfortunately it’s not quite there yet.
>> However, looking at the brightside I got a lot done in the past few weeks so
>> allow me to “journal” it in this blog post.
>>
>> Agony of Knowing
>>
>> The more I understood Drizzle’s storage mechanism and Tokyo Cabinet’s
>> internals, the more I disliked what I previously had. This led me to
>> spending quite a bit of time rewriting BlitzDB’s codebase. I was using
>> pthread’s rwlock for concurrency control but I decided to design and write
>> BlitzDB’s own lock mechanism to get the best out of TC (in terms of
>> concurrency). I also rewrote the entire table scan code which is something
>> you’d hope won’t be executed that often (people should use indexes!) but
>> needless to say, it’s still a very important component of a relational
>> storage engine so I’ve put in a lot of effort there.
>
> Is the persistent hash table in Tokyo Cabinet crash safe? I ask because of
> text in http://1978th.net/tokyocabinet/spex-en.html. I did not find a
> definition of 'broken' nor mention of commands to fix a broken database.
>
>>>>
>  Before operations to store or retrieve records, it is necessary to open a
> database file and connect the hash database object to it. The function
> `tchdbopen' is used to open a database file and the function `tchdbclose' is
> used to close the database file. To avoid data missing or corruption, it is
> important to close every database file when it is no longer in use. It is
> forbidden for multible database objects in a process to open the same
> database at the same time.
>>>>
>
>>>>
> bool tchdbclose(TCHDB *hdb);`hdb' specifies the hash database object. If
> successful, the return value is true, else, it is false.Update of a database
> is assured to be written when the database is closed. If a writer opens a
> database but does not close it appropriately, the database will be
> broken.>>>
>
> --
> Mark Callaghan
> mdcallag@xxxxxxxxx
>



-- 
Toru Maesaka <dev@xxxxxxxxx>



Follow ups

References