maria-developers team mailing list archive

[worklog-noreply@xxxxxxxxxxxx]

-----------------------------------------------------------------------
                              WORKLOG TASK
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TASK...........: New replication APIs
CREATION DATE..: Mon, 15 Mar 2010, 13:55
SUPERVISOR.....: Knielsen
IMPLEMENTOR....: Knielsen
COPIES TO......: Sergei
CATEGORY.......: Server-Sprint
TASK ID........: 107 (http://askmonty.org/worklog/?tid=107)
VERSION........: Server-9.x
STATUS.........: Un-Assigned
PRIORITY.......: 60
WORKED HOURS...: 36
ESTIMATE.......: 0 (hours remain)
ORIG. ESTIMATE.: 0

PROGRESS NOTES:

-=-=(Knielsen - Wed, 24 Mar 2010, 10:39)=-=-
Design discussions

Worked 11 hours and estimate 0 hours remain (original estimate increased by 11 hours).

-=-=(Knielsen - Mon, 15 Mar 2010, 14:28)=-=-
Research into the problem, and discussions on phone/mailing list

Worked 25 hours and estimate 0 hours remain (original estimate increased by 25 hours).

-=-=(Guest - Mon, 15 Mar 2010, 14:18)=-=-
High-Level Specification modified.
--- /tmp/wklog.107.old.9086     2010-03-15 14:18:18.000000000 +0000
+++ /tmp/wklog.107.new.9086     2010-03-15 14:18:18.000000000 +0000
@@ -1 +1,43 @@
+Current ideas/status after discussions on the mailing list:
+
+ - Implement a set of plugin APIs and use them to move all of the existing
+   MySQL replication into a (set of) plugins.
+
+ - Design the APIs so that they can support full MySQL replication, but also
+   so that they do not hardcode assumptions about how this replication
+   implementation is done, and so that they will be suitable for other types of
+   replication (Tungsten, Galera, parallel replication, ...).
+
+ - APIs need to include the concept of a global transaction ID. Need to
+   determine the extent to which the semantics of such ID will be defined
+   by the API, and to which extend it will be defined by the plugin
+   implementations.
+
+ - APIs should properly support reliable crash-recovery with decent
+   performance (eg. not require multiple mandatory fsync()s per commit, and
+   not make group commit impossible).
+
+ - Would be nice if the API provided facilities for implementing good
+   consistency checking support (mainly checking master tables against slave
+   tables is hard here I think, but also applying wrong binlog data and
+   individual event checksums).
+
+
+Steps to make this more concrete:
+
+ - Investigate the current MySQL replication, and list all of the places where
+   a plugin implementation will need to connect/hook into the MySQL server.
+    * handler::{write,update,delete}_row()
+    * Statement execution
+    * Transaction start/commit
+    * Table open
+    * Query safe/not/safe for statement based replication
+    * Statement-based logging details (user variables, random seed, etc.)
+    * ...
+
+ - Use this list to make an initial sketch of the set of APIs we need.
+
+ - Use the list to determine the feasibility of this project and the level of
+   detail in the API needed to support a full replication implementation as a
+   plugin.
 

-=-=(Serg - Mon, 15 Mar 2010, 14:13)=-=-
Observers changed: Sergei



DESCRIPTION:

This is a top-level task for the project of designing a new set of replication
APIs for MariaDB.

This task is for the initial discussion of what to do and where to focus.

The project is started in this email thread:

    https://lists.launchpad.net/maria-developers/msg01998.html


HIGH-LEVEL SPECIFICATION:



Current ideas/status after discussions on the mailing list:

 - Implement a set of plugin APIs and use them to move all of the existing
   MySQL replication into a (set of) plugins.

 - Design the APIs so that they can support full MySQL replication, but also
   so that they do not hardcode assumptions about how this replication
   implementation is done, and so that they will be suitable for other types of
   replication (Tungsten, Galera, parallel replication, ...).

 - APIs need to include the concept of a global transaction ID. Need to
   determine the extent to which the semantics of such ID will be defined
   by the API, and to which extend it will be defined by the plugin
   implementations.

 - APIs should properly support reliable crash-recovery with decent
   performance (eg. not require multiple mandatory fsync()s per commit, and
   not make group commit impossible).

 - Would be nice if the API provided facilities for implementing good
   consistency checking support (mainly checking master tables against slave
   tables is hard here I think, but also applying wrong binlog data and
   individual event checksums).


Steps to make this more concrete:

 - Investigate the current MySQL replication, and list all of the places where
   a plugin implementation will need to connect/hook into the MySQL server.
    * handler::{write,update,delete}_row()
    * Statement execution
    * Transaction start/commit
    * Table open
    * Query safe/not/safe for statement based replication
    * Statement-based logging details (user variables, random seed, etc.)
    * ...

 - Use this list to make an initial sketch of the set of APIs we need.

 - Use the list to determine the feasibility of this project and the level of
   detail in the API needed to support a full replication implementation as a
   plugin.


ESTIMATED WORK TIME

ESTIMATED COMPLETION DATE
-----------------------------------------------------------------------
WorkLog (v3.5.9)