maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

I have now most of the implementation of MDEV-12179 done. I wanted to
present the way the feature now looks, and point to the code, in case there
are any further comments on the design or implementation before it is
finalised.

To recap, the idea is to improve performance when using multiple
transactional storage engines (eg. InnoDB/TokuDB/MyRocks) on the same
replication slave, but not in the same transactions. By allowing more than
one mysql.gtid_slave_pos* tables (one for each used engine), costly
cross-engine transactions can be avoided.

The feature is enabled by a system variable:

  --gtid-pos-auto-engines=innodb,myrocks

This way, if the replication slave sees an innodb or myrocks transaction, it
will create a new table mysql.gtid_slave_pos_innodb or
mysql.gtid_slave_pos_myrocks in which to record the GTID position.

For monitoring the feature, I added new status variables:

Transactions_multi_engine

  Number of transactions that changed data in multiple (transactional)
  storage engines.

Rpl_transactions_multi_engine

  Number of replicated transactions that involved changes in multiple
  (transactional) storage engines, before considering the update of the
  mysql.gtid_slave_posXXX table.

Transactions_gtid_foreign_engine

  Number of replicated transactions where the update of the
  mysql.gtid_slave_posXXX table had to choose a storage engine that did not
  otherwise participate in the transaction.

The current code is here:

  https://github.com/knielsen/server/tree/mdev12179

Following previous discussions, the default of --gtid-pos-auto-engines is
currently empty (no automatic creation of tables by default). If desired, we
could later enable auto-creation of a select set of storage engines (eg
"innodb,myisam").

The auto-creation happens asynchroneously, in the background. Auto-creation
should be a very rare event, and I spent some effort to minimise the
overhead of this auto-creation on the normal processing of replicated
transactions. On server start, as well as on START SLAVE, the available
mysql.gtid_slave_pos* tables are auto-discovered and read as needed.

Originally, I had the idea to supply a store procedure
mysql.gtid_pos_add_engine() which would allow the DBA to explicitly create a
table for a specific engine. But it seems such stored procedure is not much
needed when --gtid-pos-auto-engines is available, and we do not have a prior
history of providing such standard stored procedures. It could be easily
added later, if desired.

 - Kristian.