maria-developers team mailing list archive

[Jonas Oreland <jonaso@xxxxxxxxxx>]

Hi again Kristian,

A slightly off topic question that struck me last night: won't all parallel
transactions conflict when updating the slave_gtid_pos table ?
or is there something i missed...

/Jonas

On Thu, Dec 4, 2014 at 2:49 PM, Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>
wrote:

> I discussed with Monty, and we came up with some more suggested changes for
> the options used to configure the optimistic parallel replication feature
> (MDEV-6676, https://mariadb.atlassian.net/browse/MDEV-6676).
>
> The biggest change is to split up the --slave-parallel-mode into multiple
> options. I think that is reasonable, probably that option was doing too
> many
> things at once.
>
> Instead, we could have the following three options:
>
> --slave-parallel-mode=all_transactions | follow_master_commits |
>                       only_commits | none
>
>     "all_transactions" is what was called "transactional" before. The slave
>     will try to apply all transactional DML in parallel; in case of
> conflicts
>     it will roll back the later transaction and retry it.
>
>     "follow_master_commits" is the 10.0 functionality, apply in parallel
>     transactions that group-committed together on the master (the default).
>
>     "only_commits" was suggested to me by a user testing parallel
>     replication. It does not attempt to apply transactions in parallel, but
>     still runs the commit steps in parallel, making slave group commit
>     possible and thus saving on fsyncs if durability settings are on.
>
>     "none" means the parallel replication code is not used (same as
>     --slave-parallel-threads=0, but now configurable per multimaster
>     connection). (This corresponds to empty value in old
>     --slave-parallel-mode).
>
> --slave-parallel-domains=on|off     (default on)
>
>     "This replaces the "domain" option of old --slave-parallel-mode. When
>     enabled, parallel replication will apply in parallel transactions whose
>     GTID has different domain ids (GTID mode only).
>
> --slave-parallel-wait-if-conflict-on-master=on|off  (default on)
>
>     When enabled, if a transaction had to do a row lock wait on the
> master, it
>     will not be applied in parallel with any earlier transaction on the
> slave
>     (idea is that such transaction is likely to get a conflict on the
> slave,
>     causing a needless retry). (This was the "waiting" option to old
>     --slave-parallel-mode).
>
> These options will also be usable per multi-source master connection, like
> --master1.slave-parallel-mode=all_transactions. The options will be
> possible
> to change dynamically also (with SET GLOBAL), though the associated slave
> threads must be stopped while changing.
>
> Also, Monty suggested to rename @@replicate_allow_parallel to
>
>     @@SESSION.replicate_expect_conflicts=0|1   (default 0)
>
> When this option is enabled on the master when a transaction is committed,
> that transaction will not be applied in parallel with earlier transactions
> (when --slave-parallel-mode=all_transactions). This can be used to reduce
> retries on the slave, if an application is about to do a transaction that
> is
> likely to cause a conflict and retry on a slave if applied in parallel with
> earlier transactions.
>
>
> Let me know if there are any comments to these or suggestions for changes.
> It
> is best to get these as right as possible before release (seems the
> intention
> is to include optimistic parallel replication in 10.1), since it is the
> user-visible part of the feature.
>
> With these option names, the normal way to use optimistic parallel
> replication
> would be these two options in my.cnf:
>
>    slave_parallel_mode=all_transactions
>    slave_parallel_threads=20  (or whatever)
>
> This seems reasonably, I think. None of the other options would need be
> considered except in more special cases.
>
>  - Kristian.
>