drizzle-discuss team mailing list archive
Mailing list archive
Parallel replication slaves?
I have done some experimentations with parallel applying with the Galera
replication engine and to my surprise I could not get much performance gain.
In Galera, the applying happens as follows:
0. Galera is a merge over MySQL 5.1.* and transactions to be applied
contain RBR events
1. a predefined number (1-n) of applier threads will be started at startup
2. applying queue manager assigns tasks (incoming trxs) for the appliers.
For each new task, QM looks for the modified row IDs to see if there
is a conflict with running tasks. Parallelism is allowed only for
3. appliers synchronize before commit, so that commit order is preserved
Benchmarking did not show any significant benefit of having parallel applying.
Reason is that RBR applying is so fast compared to processing
of SQL queries that applying queue did not build up. It turned out that
several applier threads were useful only when transaction length was ~200 SQL
I have blogged something more about this experimentation here:
http://www.codership.com/content/parallel-applying. But, it should be
read just the title...
Note that just by looking at the row IDs in the RBR, it is not guaranteed that
the actual processing of these transactions would not conflict. SE can require
more "resources" than what is visible in the RBR events (we called this
asymmetric lock granularity issue). However, if appliers do conflict,
it is safe to abort the later one and replay immediately. This will
performance if aborting is too frequent.
Despite of the bad results, I have not completely lost my hope for parallel
applying. This was just the first shot, I need to retry with newest code,
different SQL profiles and better HW. We allocate R&D tasks for June sprint
today, I try to fit "parallel applying revisited" in the task list.