maria-developers team mailing list archive

[worklog-noreply@xxxxxxxxxxxx]

-----------------------------------------------------------------------
                              WORKLOG TASK
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TASK...........: Group commits on slave to reduce fsync() load
CREATION DATE..: Fri, 18 Mar 2011, 14:25
SUPERVISOR.....: 
IMPLEMENTOR....: 
COPIES TO......: 
CATEGORY.......: Server-RawIdeaBin
TASK ID........: 185 (http://askmonty.org/worklog/?tid=185)
VERSION........: Server-9.x
STATUS.........: Un-Assigned
PRIORITY.......: 60
WORKED HOURS...: 0
ESTIMATE.......: 0 (hours remain)
ORIG. ESTIMATE.: 0

PROGRESS NOTES:



DESCRIPTION:

MWL#116 introduces group commit for the binary log; this can reduce the number
of fsync() calls needed when running with sync_binlog=1.

But this does not currently help on the slave. The reason is that the slave
SQL thread is single-threaded, so there will never be any opportunity for
group commit.

However, an alternative way to reduce fsync() on the slave when sync_binlog=1
is to combine multiple transactions from the binlog into one transaction on
the slave.

If we have N transactions T_1, ..., T_N in the binlog, we can combine them
simply into one transaction on the slave like this:

 - Change COMMIT in T_1, ..., T_(N-1) into SAVEPOINT T_i

 - Change any ROLLBACK into ROLLBACK to T_i

This will reduce the number of fsync() calls, since it results in fewer
COMMITs.

The downside is that the latency between a transaction being visible on the
master and on the slave will increase.

There should probably be some option to limit how much transactions may be
delayed on the slave.

It would also be useful to have some kind of timestamps in the binlog to be
able to estimate the duration of a transaction on the slave before starting to
apply it; this would allow to avoid combining a later long transaction with a
previous short transaction, which could cause undue delay for the short
transaction to become visible on the slave.


ESTIMATED WORK TIME

ESTIMATED COMPLETION DATE
-----------------------------------------------------------------------
WorkLog (v4.0.0)