← Back to team overview

maria-developers team mailing list archive

Re: Ideas for improving MariaDB/MySQL replication


Hi Kristian,

I agree with Alex's response, and I'll pick the hopefully all the remaining questions to answer here.

Quoting Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>:
So the basic for such an interface would be the ability to install hooks to be
called with row data for every handler::write_row(), handler::update_row(),
and handler::delete_row() invocation, just like the current row-based
binlogging does. And similar for SQL statement execution like statement-based
logging does now. That should be clear enough.

Then comes the need to hook into transaction start and commit and opening
tables.  At this point, more of the internals of the MySQL server start to
appear, and some careful thought will be needed to get an interface that
exposes enough that plugins can do what they need, without exposing too much
internal details of how MySQL query execution is implemented.

Yes, that's a good plan.

(But note that this is two different issues regarding "internal
implementations". One is how the *query execution* is implemented. The other
is how the *plugins* are implemented. If I understood you correctly, the
interface used for semisync in MySQL fails on the latter point).

One example of how a lot of details from query execution pop up is with regard
to the mixed-mode binlogging. This is where queries are logged as statements
when this is safe, and as row events when this is not safe (nondeterministic
queries). The concept of "mixed mode binlogging" certainly seems like
something that should be an implementation detail of the plugin, not part of
the interface. On the other hand, determining whether a query is safe for
statement-based logging is highly complex, and exposing enough of the server
for the plugin to be able to determine this by itself may be too much. (Maybe
just expose an is_safe_for_statement() function to plugins could be enough).

Mixed mode replication (or binlog format, as called in MySQL code), is something I would leave completely for the DBMS to decide about. The replication plugin should offer calls for SQL and ROW level replication, and DBMS just decides which one to call in each case. Note that replication plugin has it next to impossible to judge if passed SQL statement is valid to be replicated directly or if ROW event should be used (this decision would require parsing in replicator...).

In general, it is better if replicator does not need to look inside replication events at all. (ok, there are requirements for SQL level filtering heterogeneous replication, query rewriting etc..., which may be valid use cases as well)

Another example of hairy details is all the extra information that can go with
an SQL statement into the binary log. Things like current timestamp, random
seed, user-set @variables, etc. To support a statement-based replication
plugin, we probably have to expose all of this on the interface in a clean

SQL level replication requires that session context will be maintained, passed for the replicator and enforced in the applying side. This will be a bit complicated to implement, but is something that cannot be avoided. Support for session context management must in replication API, but the context can be presented as opaque object and replicator does not need to know about the details.

http://www.codership.com  seppo.jaakola@xxxxxxxxxxxxx
tel: +358 40 510 5938 skype: seppo_jaakola