← Back to team overview

drizzle-discuss team mailing list archive

Replication from drizzle to a key-value store, request for comments

 

After working with the txn log for a couple of weeks i decided to have a go
at replicating from drizzle to a key-value store, and i picked memcached as
a first. This mail is a request for comments on the very early version of
the api. The aim is to enable users to easily add a few "business"-rules
(like, take this column and make it into a key, and take these columns and
create this object).

Simple use case, you have a table with a very frequently read column called
"test" which you want to read from memcached instead:

Annotate a class like this (annotation names kind of borrowed from JPA, but
not entirely, might be confusing?):

@Entity(schema = "unittests", table = "test1")
public class ExampleRepl implements Serializable {

    @Id
    @Column("id")
    private String identifier;

    @Column("test")
    private String test;
 ...
}

deploy it to the slave somehow (still not decided how to do this) and do a
master insert like this: INSERT INTO unittests.test1 (id, test, something)
VALUES (1, "yeah", "blahblah"), (2, "asdf", "blahblah2");

This will automatically build two ExampleRepl objects with fields:
identifier = 1 and 2, and test = "yeah" and "asdf" and pin them to memcached
(or any key-value store) with keys "1" and "2". This is implemented as a
slave in my little RabbitReplication (http://launchpad.net/rabbitreplication)
project, so the memcached communication and annotation processing etc does
not have to be done on same machine as the master.

Comments? Should I approach this some other way? Is the use case for this
something that could be useful for anyone?

My TODO is:
* Support deletes and updates
* Make serialization pluggable (json/xml etc)
* Support combined keys
* ... lots more


/Marcus

Follow ups