maria-developers team mailing list archive

[Piotr Jurkiewicz <piotr.jerzy.jurkiewicz@xxxxxxxxx>]

Hi all,

as the pencils down date has passed, I am posting a report on what I 
have during the whole project.

Repository of my project is available here:

https://github.com/piotrjurkiewicz/mariadb-server/commits/10.1-memcached

Some stats:

33 commits before the midterm
60 commits after the midterm
93 commits total

What have been done in details:
===============================

BEFORE MIDTERM:
---------------

1. Code of InnoDB Memcached plugin has been imported from MySQL into the 
MariaDB tree.

2. There were some differences between MariaDB and MySQL, which blocked 
the plugin from out-of-the-box compilation and correct operation. They 
have been resolved.

3. Test suite for the plugin has been imported from MySQL. Compatibility 
changes were introduced into these test cases which required them. The 
test suite was modified to assign memcached port number dynamically from 
mtr worker's pool, what allows to run the same test in parallel for many 
combinations.

The plugin fully passes the test suite, both for 'innodb_plugin' and 
'xtradb' combinations.

4. Some enhancements and bug fixes have been introduced into the plugin:

- detection of memcached daemon initialization has been fixed -- this 
eliminated unnecessary wait (15 seconds) on plugin shutdown and memory 
leak (in cases when plugin failed to initialize properly before)

- memcached arithmetic commands (incr/decr) handling has been improved 
-- now numeric columns values are handled in a consistent and 
predictable way

- incorrect handling of unsigned not null integers columns has been fixed

AFTER MIDTERM:
--------------

5. Memcached daemon code has been updated to the latest version from 
upstream memcached/engine-pu branch.

6. Plugin architecture has been changed to "one table per socket", as 
originally proposed in MDEV-4674 bug report. This allows usage of well 
known security solutions (firewalls, unix file permissions) to restrict 
access to different tables for different users.

Now user can specify which tables should be accessible on which sockets. 
Memcached daemon will listen on all these sockets and execute commands 
on appropriate table, depending on which socket client connected.

Supported protocols are TCP, UDP and UNIX stream sockets. Sockets are 
configured in 'daemon_memcached/containers' table:

INSERT INTO containers VALUES ("tcp://*:11211", "db0", "table0",
                               "key_col", "val_col", "c3", "c4", "c5",
                               "PRIMARY", "|");

INSERT INTO containers VALUES ("udp://localhost:11222", "db0", "table1",
                               "key_col", "val_col", "c3", "c4", "c5",
                               "PRIMARY", "|");

INSERT INTO containers VALUES ("unix:/tmp/mc.sock", "db1", "table0",
                               "key_col", "val_col", "c3", "c4", "c5",
                               "PRIMARY", "|");

7. Plugin has been decoupled from InnoDB.

Now plugin is split into two parts:

  - generic 'daemon_memcached.so'
  - InnoDB specific 'daemon_memcached_engine_ib.so'

The first library is a MariaDB plugin, which should be installed with 
command:

  install plugin daemon_memcached soname "daemon_memcached.so";

This library is generic. It provides Memcached protocol handling and it 
loads the second library, which is API-specific engine for accessing the 
database. Which engine library should be loaded can be specified in 
'daemon_memcached_engine_lib_name' MariaDB server option.

Now, the only existing engine library is daemon_memcached_engine_ib.so, 
which utilizes InnoDB API to access the data. It can operate on InnoDB 
and XtraDB tables.

8. Some enhancements and bug fixes have been introduced into the plugin:

- CMake options and #cmakedefines associated with the plugin have been 
cleaned up

- embedded default Memcached engine has been removed

- duplicated code in Memcached source has been removed

- fixed MySQL-originating bug with flush_all of non default table -- 
command flush_all always flushed the default table, not the current one

- removed support for in-connection table switching -- after 
introduction of one table/port architecture it was redundant

- plugin shutdown time has been improved -- from constant time of 2 
seconds to average time of 0.5 second

- logging of plugin has been refactored -- now plugin send log messages 
to MariaDB log instead of stderr

- fixed Memcached-originating bug with get_thread_stats() -- bug 
resulted in out of bound array accesses and writes, what leaded to SEGFAULT

- fixed MySQL-originating crash during 'flush_all' memcached command bug 
-- crash happened when table was heavily loaded with DML operations and 
because of that ib_cb_cursor_set_lock() failed to obtain a lock on the 
table, returning DB_LOCK_WAIT_TIMEOUT error

- fixed MySQL-originating multiple memcached_sync_count for the same 
transaction bug -- the memcached_sync_count was incremented multiple 
times for the same transaction and after the end of transaction, it was 
not reset to zero, what blocked TRUNCATE command on any InnoDB table, on 
which Memcached had operated during program lifetime, even after 
uninstalling the plugin

- support for numeric values has been finished -- now BIGINT overflows 
in the same way as smaller types, test to check over/under-flowing of 
all integer types has been added

For details of all changes please see commit messages associated with 
them: 
https://github.com/piotrjurkiewicz/mariadb-server/commits/10.1-memcached

-----------------------------------------------------------------------

Moreover, I planned to implement a new interface inside the Memcached 
plugin, which will use Handler API for database access. This would make 
possible to use the plugin to access databases based on storage engines 
other than InnoDB/XtraDB.

However, during testing I discovered many serious bugs, either in 
Memcached or InnoDB API engine, which originated from upstream Memcached 
or MySQL and resulted in crashes or deadlocks. I decided that first I 
should fix them before implementing new functionalities. I lost a lot of 
time on investigating these bugs and I was unable to finish the 
implementation of Handler API engine.

Nevertheless, I think the whole plugin is ready to be merged into 
MariaDB. All user-facing changes (like one port/table architecture) have 
been completed. Eventual later introduction of Handler API engine will 
not be associated with any user-facing compatibility changes. It will 
just lift the requirement that all databases accessed via the Memcached 
plugin must be InnoDB/XtraDB.

Now I am going to write a separate email to coordinate the merge of my 
code into MariaDB repository.

Piotr