maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

Jan Lindström <jplindst@xxxxxxxxxxx> writes:

> some background, lets assume we have two galera clusters that are
> geographically distributed (lets say for example that cluster A is on Europe
> and cluster B on USA). Inside a cluster all nodes are set up so that server_id
> and domain_id are the same and we will use strict GTID mode (for a example on
> cluster A server_id = 1 and domain_id =1 and in cluster B server_id = 2 and
> domain_id = 2). In cluster we naturally use Galera replication. Additionally,
> we have asynchronous replication (MySQL replication) between these clusters.

> 1) We would like to have some special domain_id (i.e. GTID) for transactions
> that are executed only on certain cluster e.g. on cluster A. This would mean
> that there is some dynamic option to enable "local" transaction execution mode,
> where transactions are executed on cluster A but not on cluster B. I know that
> this kind of feature does not exists but is it even possible ? Basically, this
> would mean for MySQL replication point of view a fact that if GTID is formed
> like this special GTID, they are not replicated to slave, but they are written
> to binlog.

It sounds like what you want is some option on the slave that would cause it
to ignore all GTIDs with some specific domain id, similar to how it is
currently possible to filter on server id. That seems a reasonable request;
such option does not exist currently though.

Another option is to use the @@skip_replication variable. Set this for the
sessions doing the "local" transactions. Then set
replicate_events_marked_for_skip to either FILTER_ON_SLAVE or
FILTER_ON_MASTER, as you like. This is specifically designed to allow to log
certain transactions to the binlog but not replicate them, so it sounds like
it is what you asked for.

> 2) There will be some challenge in maintaining the sequence number for GTID
> correctly: for global GTID purposes, we can just count commits, and skip every
> local transaction commit, but, we must also initialize the sequence for every
> node joining the cluster. In MySQL-5.6 this happens by copying binlog files
> (only header part needed) to joining node.  I wonder if same strategy works for
> MGC-10, is MariaDB GTID is stored in binlog header ?

Yes, the GTID is stored in the binlog header.
It is also stored in a file master-bin.state. This file is read at server
start to initialise the GTID sequence numbers. However, if the file is not
found for any reason, the information is extracted from the last binlog file
(the master-bin.state is just a cache).

However, you need the whole binlog file (if master-bin.state is missing), not
just the header. I am sure it must be the same for MySQL-5.6 - you need to
scan to the end of the binlog file, as there may be extra GTIDs logged in the
file that were not there when the header was written. Of course, just after
server start, the most recent binlog file is empty, so just the header works.

It sounds like you would want to just copy over the master-bin.state file.

However, this whole approach sounds wrong to me. The current GTID state is a
cluster-wide property, not a local property, I do not understand why you want
to maintain it locally.

As I understand it, Galera decides on commit order. The certification process
decides for parallel transactions T1 and T2 which one will be committed before
the other. The GTID sequence has to match the commit order. So the GTIDs need
to be generated by Galera, at the point in the code where the commit order is
decided. This Galera code then also needs to do the binlog writing, so that
binlog order matches GTID order and commit order. With Galera handling GTID
number allocations, I do not understand why you would need to worry about
binlog headers.

But maybe I have misunderstood, I do not know Galera in details. Maybe you
could explain how Galera ensures matching commit order, binlog order, and GTID
order? Or alternatively, explain how things work without such ensurance?

> 3) Independent table (in data dictionary) or file for replication filters. I
> can change the replication filters dynamically, but if the server crash and i
> don?t report to my.cnf it?s lost. Therefore, there is a need to store
> replication filters persistently and these should be loaded automatically when
> server is restarted.

Currently we do not have this, replication filters are configuration options,
and those are stored in the config file, not in dynamic tables or files. The
server_id filters are set by CHANGE MASTER though, IIRC, so they are stored in
master.info / relay_log.info.

> 4) Dynamic binlog. Currently options are read-only.

This we do not have. There has been talk of allowing it under global read
lock; this might work, but it would require some work to carefully check all
of the code for possible issues. There are a lot of places in the code where
expensive synchronisation/locking is avoided based on the knowledge that the
binlog can not be enabled or disabled on-the-fly.

Hope this helps,

 - Kristian.