maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

I discussed with Monty, and we came up with some more suggested changes for
the options used to configure the optimistic parallel replication feature
(MDEV-6676, https://mariadb.atlassian.net/browse/MDEV-6676).

The biggest change is to split up the --slave-parallel-mode into multiple
options. I think that is reasonable, probably that option was doing too many
things at once.

Instead, we could have the following three options:

--slave-parallel-mode=all_transactions | follow_master_commits |
                      only_commits | none

    "all_transactions" is what was called "transactional" before. The slave
    will try to apply all transactional DML in parallel; in case of conflicts
    it will roll back the later transaction and retry it.

    "follow_master_commits" is the 10.0 functionality, apply in parallel
    transactions that group-committed together on the master (the default).

    "only_commits" was suggested to me by a user testing parallel
    replication. It does not attempt to apply transactions in parallel, but
    still runs the commit steps in parallel, making slave group commit
    possible and thus saving on fsyncs if durability settings are on.

    "none" means the parallel replication code is not used (same as
    --slave-parallel-threads=0, but now configurable per multimaster
    connection). (This corresponds to empty value in old
    --slave-parallel-mode).

--slave-parallel-domains=on|off     (default on)

    "This replaces the "domain" option of old --slave-parallel-mode. When
    enabled, parallel replication will apply in parallel transactions whose
    GTID has different domain ids (GTID mode only).

--slave-parallel-wait-if-conflict-on-master=on|off  (default on)

    When enabled, if a transaction had to do a row lock wait on the master, it
    will not be applied in parallel with any earlier transaction on the slave
    (idea is that such transaction is likely to get a conflict on the slave,
    causing a needless retry). (This was the "waiting" option to old
    --slave-parallel-mode).

These options will also be usable per multi-source master connection, like
--master1.slave-parallel-mode=all_transactions. The options will be possible
to change dynamically also (with SET GLOBAL), though the associated slave
threads must be stopped while changing.

Also, Monty suggested to rename @@replicate_allow_parallel to

    @@SESSION.replicate_expect_conflicts=0|1   (default 0)

When this option is enabled on the master when a transaction is committed,
that transaction will not be applied in parallel with earlier transactions
(when --slave-parallel-mode=all_transactions). This can be used to reduce
retries on the slave, if an application is about to do a transaction that is
likely to cause a conflict and retry on a slave if applied in parallel with
earlier transactions.


Let me know if there are any comments to these or suggestions for changes. It
is best to get these as right as possible before release (seems the intention
is to include optimistic parallel replication in 10.1), since it is the
user-visible part of the feature.

With these option names, the normal way to use optimistic parallel replication
would be these two options in my.cnf:

   slave_parallel_mode=all_transactions
   slave_parallel_threads=20  (or whatever)

This seems reasonably, I think. None of the other options would need be
considered except in more special cases.

 - Kristian.