maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

Alex Yurchenko <alexey.yurchenko@xxxxxxxxxxxxx> writes:

> Since both are configured manually, and domain ID can simply default
> to 0 in simple setups, I'd imagine that the possibility of having
> server ID configured incorrectly (just missing to configure it) is way
> more probable than having domain ID incorrect. Actually, being an

Are you sure? Traditional replication has always tried to make sure that
server_id is configured to be unique. You cannot even start a slave if
server_id is not set. And if the master and slave has same server_id, you get
this error when starting the slave:

130508 13:43:04 [ERROR] Slave I/O: Fatal error: The slave I/O thread stops because master and slave have equal MySQL server ids; these ids must be different for replication to work (or the --replicate-same-server-id option must be used on slave but this does not always make sense; please check the manual before using it). Internal MariaDB error code: 1593

> arbitrary node group ID what is "incorrect" here? ANY value just makes
> the node a member of the corresponding domain, so ANY domain ID value
> is legal. Whereas server ID can certainly be incorrect (a duplicate
> within the domain).

Here is how I suggest to think about it:

Server_id is assumed to uniquely identify any server, there is some code to
help ensure this. Each server keeps a local counter which is guaranteed to
increment for each new transaction originating on that server (it can
increment with more than 1, but never decrement). This ensures uniqueness of
the GTID.

The GTID code has the ability to locate any GTID in the binlogs on a server
(or detect that the GTID is missing). The slave remembers the last GTID it
replicated. This allows a slave to connect to a new master and locate the
point in the binlog on the new master that corresponds to where it stopped on
the old master.

All of this works without any regard to replication domain ID. So in this
sense, there is no incorrect configuration of domain ID possible, as you say.

But now, on the next level, we consider what happens if the order of
transactions in the binlog on the new master differs from that on the old
master. If the order is A,B on the old but B,A on the new, and the slave was
at point A on the old, then starting at A on the new will loose a transaction
B (similarly we can duplicate a transaction).

Different binlog order can happen for two reasons. Either by mistake (or
sloppyness), or deliberately, due to transactions happening independently in
parallel (such as multi-source replication). And we want to deal with both
cases.

To handle the mistake, I plan to implement a strict mode. Whenever we
originate a new transaction on a server and write it to the binlog, we ensure
it has a higher sequence number than any previous one. Then, if on replication
we receive a transaction from a master where the sequence number is not bigger
than the last thing logged in the slave's binlog, we have detected an
out-of-order situation, and stop with an error.

To handle sloppyness, users can disable strict mode. If they then get
out-of-order binlog (because of not obeying read-only slave or something),
things will still work (because on the basic level we have the ability to
locate any GTID, even if sequence numbers are not strictly increasing).
However they need to be careful if they change master around the out-of-order
points, as per the A,B / B,A example above. This is no different from
old-style replication (and it is required that non-strict be default, to not
break upgrade of existing setups).

The purpose of domain IDs is to handle the deliberate case. The
user/DBA/application needs to explicitly declare that two transactions are
intended to be independent, generated in parallel in arbitrary order. This
declaration is done by giving independent transactions distinct domain IDs, and
using same domain ID for transactions that were intended to be ordered.

The slave will then remember one binlog position for every domain ID. This
avoids the A,B / B,A problem or related issues, as long as order is in fact
preserved within one replication domain (and with strict mode, we can detect
mistakes). So the issue of "correct" configuration for domain_is is that any
two transactions that can end up in different order in binlogs on different
servers, should have different domain ID.

A further advantage (for later) is that, knowing that GTIDs 0-1-10 and 1-2-20
are independent (different domain), a slave server is free to replicate then
in parallel and commit in any order. This will later be used to implement
robust, out-of-order parallel slave.

So this is how domain ID should be thought of: it is a declaration, by the
user, about which transactions are independent, and which are considered
strictly ordered with one another.

> So suppose we have nodes N0, N1 and N2 with IDs 0-0, 0-1, 0-2 
> respectively.
>
> Initially N1 and N2 both replicate from N0 and have identical DB
> contents.
>
> At 0-0-10 N2 goes to maintenance.
>
> After 0-0-100 someone executes local transaction on N1 and it gets
> logged as 0-1-101. Right?

Yes.

> So if now N0 executes another transaction, what will be its GTID?
> a) on N0 - 0-0-101?

Yes.

> b) on N1 - 0-0-102 or 0-0-101?

0-0-101. (GTIDs are always preserved by replication.)

>>   The server ID is set to the server ID of the server where the
>> event group is first logged into the binlog. The sequence number is
>> increased on a server for
> every event group logged.
> So it is actually another question, sequence number is not set on the
> master server but always computed locally?)

No, this is bad wording on my part. When a new transaction is first created by
a user, it gets a new, increased sequence number. But as it replicates between
servers, the sequence number is kept.

> Or, does N1 detect a problem at this point? If yes, how exactly? How
> server ID is involved there?

Once strict mode is implemented, it will detect the problem that N1's binlog
has 0-1-101. And now replication wants to log another GTID 0-0-101, with a
sequence number that is not increasing. With strict mode on, this will be an
error (replication will be stopped).

I think server ID is not really involved here. It would also be an error to
try to log 0-0-101 if the binlog already contained 0-0-101. Server ID is
mostly used to be able to locate arbitrary GTID in the binlog, even if
sequence numbers happen to be not always increasing.

> Now if we can get past this point without an error at N1, and start N2
> to replicate from N1, I take it will receive and commit 0-1-101. But

Yes.

> will it ever record it somewhere or its state will be simply 0-0-XXX?

Just after applying 0-1-101, the state will be just that, 0-1-101. But then as
we proceed with 0-0-101 and further 0-0-XXX, the state will simply be 0-0-XXX.

> If at 0-0-110 we failover N0 to replicate from N2, will it receive
> 0-1-101?

No. Because 0-1-101 is before 0-0-110 in the binlog on N2.

 - Kristian.