maria-developers team mailing list archive

[worklog-noreply@xxxxxxxxxxxx]

-----------------------------------------------------------------------
                              WORKLOG TASK
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TASK...........: New replication APIs
CREATION DATE..: Mon, 15 Mar 2010, 13:55
SUPERVISOR.....: Knielsen
IMPLEMENTOR....: Knielsen
COPIES TO......: Sergei
CATEGORY.......: Server-Sprint
TASK ID........: 107 (http://askmonty.org/worklog/?tid=107)
VERSION........: Server-9.x
STATUS.........: In-Progress
PRIORITY.......: 60
WORKED HOURS...: 69
ESTIMATE.......: 0 (hours remain)
ORIG. ESTIMATE.: 0

PROGRESS NOTES:

-=-=(Sergei - Tue, 29 Jun 2010, 13:50)=-=-
Status updated.
--- /tmp/wklog.107.old.31164    2010-06-29 13:50:15.000000000 +0000
+++ /tmp/wklog.107.new.31164    2010-06-29 13:50:15.000000000 +0000
@@ -1 +1 @@
-Un-Assigned
+In-Progress

-=-=(Knielsen - Thu, 24 Jun 2010, 12:04)=-=-
Dependency created: 107 now depends on 120

-=-=(Knielsen - Mon, 21 Jun 2010, 08:36)=-=-
Research and design thoughts.

-=-=(Knielsen - Mon, 07 Jun 2010, 12:11)=-=-
High Level Description modified.
--- /tmp/wklog.107.old.31097    2010-06-07 12:11:57.000000000 +0000
+++ /tmp/wklog.107.new.31097    2010-06-07 12:11:57.000000000 +0000
@@ -7,3 +7,6 @@
 
     https://lists.launchpad.net/maria-developers/msg01998.html
 
+Wiki page for the project:
+
+    http://askmonty.org/wiki/ReplicationProject

-=-=(Knielsen - Mon, 29 Mar 2010, 07:33)=-=-
Research and design discussions: Galera, 2pc/XA, group commit, multi-engine transactions.

-=-=(Knielsen - Wed, 24 Mar 2010, 10:39)=-=-
Design discussions

-=-=(Knielsen - Mon, 15 Mar 2010, 14:28)=-=-
Research into the problem, and discussions on phone/mailing list

-=-=(Guest - Mon, 15 Mar 2010, 14:18)=-=-
High-Level Specification modified.
--- /tmp/wklog.107.old.9086     2010-03-15 14:18:18.000000000 +0000
+++ /tmp/wklog.107.new.9086     2010-03-15 14:18:18.000000000 +0000
@@ -1 +1,43 @@
+Current ideas/status after discussions on the mailing list:
+
+ - Implement a set of plugin APIs and use them to move all of the existing
+   MySQL replication into a (set of) plugins.
+
+ - Design the APIs so that they can support full MySQL replication, but also
+   so that they do not hardcode assumptions about how this replication
+   implementation is done, and so that they will be suitable for other types of
+   replication (Tungsten, Galera, parallel replication, ...).
+
+ - APIs need to include the concept of a global transaction ID. Need to
+   determine the extent to which the semantics of such ID will be defined
+   by the API, and to which extend it will be defined by the plugin
+   implementations.
+
+ - APIs should properly support reliable crash-recovery with decent
+   performance (eg. not require multiple mandatory fsync()s per commit, and
+   not make group commit impossible).
+
+ - Would be nice if the API provided facilities for implementing good
+   consistency checking support (mainly checking master tables against slave
+   tables is hard here I think, but also applying wrong binlog data and
+   individual event checksums).
+
+
+Steps to make this more concrete:
+
+ - Investigate the current MySQL replication, and list all of the places where
+   a plugin implementation will need to connect/hook into the MySQL server.
+    * handler::{write,update,delete}_row()
+    * Statement execution
+    * Transaction start/commit
+    * Table open
+    * Query safe/not/safe for statement based replication
+    * Statement-based logging details (user variables, random seed, etc.)
+    * ...
+
+ - Use this list to make an initial sketch of the set of APIs we need.
+
+ - Use the list to determine the feasibility of this project and the level of
+   detail in the API needed to support a full replication implementation as a
+   plugin.
 

-=-=(Sergei - Mon, 15 Mar 2010, 14:13)=-=-
Observers changed: Sergei



DESCRIPTION:

This is a top-level task for the project of designing a new set of replication
APIs for MariaDB.

This task is for the initial discussion of what to do and where to focus.

The project is started in this email thread:

    https://lists.launchpad.net/maria-developers/msg01998.html

Wiki page for the project:

    http://askmonty.org/wiki/ReplicationProject


HIGH-LEVEL SPECIFICATION:



Current ideas/status after discussions on the mailing list:

 - Implement a set of plugin APIs and use them to move all of the existing
   MySQL replication into a (set of) plugins.

 - Design the APIs so that they can support full MySQL replication, but also
   so that they do not hardcode assumptions about how this replication
   implementation is done, and so that they will be suitable for other types of
   replication (Tungsten, Galera, parallel replication, ...).

 - APIs need to include the concept of a global transaction ID. Need to
   determine the extent to which the semantics of such ID will be defined
   by the API, and to which extend it will be defined by the plugin
   implementations.

 - APIs should properly support reliable crash-recovery with decent
   performance (eg. not require multiple mandatory fsync()s per commit, and
   not make group commit impossible).

 - Would be nice if the API provided facilities for implementing good
   consistency checking support (mainly checking master tables against slave
   tables is hard here I think, but also applying wrong binlog data and
   individual event checksums).


Steps to make this more concrete:

 - Investigate the current MySQL replication, and list all of the places where
   a plugin implementation will need to connect/hook into the MySQL server.
    * handler::{write,update,delete}_row()
    * Statement execution
    * Transaction start/commit
    * Table open
    * Query safe/not/safe for statement based replication
    * Statement-based logging details (user variables, random seed, etc.)
    * ...

 - Use this list to make an initial sketch of the set of APIs we need.

 - Use the list to determine the feasibility of this project and the level of
   detail in the API needed to support a full replication implementation as a
   plugin.


ESTIMATED WORK TIME

ESTIMATED COMPLETION DATE
-----------------------------------------------------------------------
WorkLog (v4.0.0)