maria-developers team mailing list archive
Mailing list archive
[GSoC] proposal draft
Hi Colin Charles, maria-developers (cc'd),
I'm looking to do my GSoC project on porting the InnoDB-memcached plugin to
MariaDB, which you will be the mentor for. I asked maria-developers about
it earlier, but haven't had any responses, so I've written a draft proposal
(below) to elicit some responses. I would appreciate your technical
corrections and suggestions.
The MySQL+memcached combination is popular for high-speed, high-throughput
websites that also need durable, long-term storage. By storing data mostly
on disk, MySQL provides the ACID compliance and longevity of a traditional
RDMS; by caching commonly accessed data solely on memory, memcached
provides fast lookup for most transactions.
In general, memcached can be used in two different ways: first, where the
program tries to retrieve a cached copy from a separate memcached process
before resorting to accessing the database; and second, where memcached is
integrated into MySQL, invisibly to the client program. This is possible
because MySQL versions 5.6 and up come with a daemon plugin that connects
the InnoDB storage engine to memcached, bypassing the SQL layer and
replicating data behind the scenes.
This plugin is not in MariaDB, as it forked from MySQL before 5.6. But it
would be very useful if it was. The plugin has a couple advantages over the
separate process; for example, the plugin can automatically serialize
binary data to store it as text in memcached, making it much less
error-prone than manual efforts.
As a GSoC student, I would port the InnoDB-memcached plugin from MySQL
5.6/5.7 to MariaDB 10's InnoDB and XtraDB engines. The original code is
GPL-licensed, so my job would be to use the existing code but modify it to
match the variety of codebase changes that have happened since the MariaDB
fork. For example, upon compiling the MySQL plugin code with MariaDB
instead, I got a compile error about a missing "binlog.h" in the handler
API code for the InnoDB half of the plugin. Apparently, the binary logging
code is different for MySQL and MariaDB, and my job would be to convert
those references over. Additionally, I will write an XtraDB version of the
plugin that, for example, scales well with many memcached nodes.
This project would provide an alternative to MDEV-4674, where the writer
requests a feature to connect MariaDB to memcached using TCP/UDP, and to
MDEV-4675, where the writer requests a feature to detect and address low
memory in the memcached server; instead, for example, the plugin could take
parameters on a strategy to allocate memcached memory.
MariaDB InnoDB+memcached plugin
Documentation similar to MySQL's "Getting Started" guides on using the