maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

Robert Hodges <robert.hodges@xxxxxxxxxxxxxx> writes:

> In that case it gets complex for outsiders to figure out how to restart if
> there's a failure. Let's say my transaction fails after T1 commits but
> before T1 commits. Then on restart I have to regenerate T2 and rerun it.
>  That could be hard if T2 contains statements that came before T1 in the
> original, serialized log.

Agree, there are different approaches possible.

The facility in MariaDB discussed here is precisely to apply in parallel a
serialized log. We apply the transaction in parallel, but the commits happen
in the original serial order. This makes the parallel apply transparent to
applications, assuming MVCC.

Each transaction inserts in a table its own monotonic transaction number. So
after a restart, it is easy to find the point at which to resume - just find
the highest number in the table.

One limitation of this approach is that with N worker threads, we can never
execute more than N transactions ahead of the slowest transaction in the log,
because every thread needs to wait for the prior commit.

> Without a "group" transaction another approach is to keep separate restart
> points for each worker thread.  It works well as long as you can
> deterministically assign transactions to threads and keep those same
> threads from getting too far apart. The logic to do this is non-trivial, so
> it's not an approach for everybody.

MariaDB also supports this approach. Here, the user explicitly assigns each
transaction to a replication domain, and restart point is kept per-domain (not
per-transaction). Different domains commit independently and out-of-order with
respect to one another. Within one domain, commit order is strictly
serialised.

I like the approach with replication domains. Not only does it avoid the need
for non-trivial logic to assign transactions to different threads and restart
points. It also ensures that those restart points make sense to the user/DBA.

 - Kristian.