maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

Hi Serg,

Earlier this year we discussed MDEV-6429, which is to implement a proper
storage engine API for a feature used in parallel replication. This is where
InnoDB reports to the replication layer whenever transaction T1 needs to wait
for T2; parallel replication uses this to detect and resolve deadlocks caused
by enforced commit order.

In 10.0, we decided to not do a "proper" API, instead just modify InnoDB to
call into the server layer directly. In 10.0, the need for this API/feature is
somewhat InnoDB-specific, it is not known whether any of the other
transactional storage engines would need to do anything similar for parallel
replication to work.

If we implement MDEV-6676, however, then the need for the API becomes more
critical. Because MDEV-6676 absolutely only works if we have a mechanism to
detect conflicts that cause a deadlock with commit order, as was implemented
for InnoDB. If we were to attempt to enable speculative parallel replication
for eg. Tokutek transactions, without implementing the necessary check in
Tokutek, then the slave would quickly end up hanging.

Thinking more on this, I suppose what we really need is what you requested
also during the original discussion: A true optional feature for storage
engines to implement this additional check. And a way for the server layer to
see if a storage engine has the implementation. If it does, then we can try to
run its transactions in parallel as per MDEV-6676. If not, we will not try,
and things will still work correctly (just without the parallelisation
optimisation).

Maybe for the first version, the best thing to do is to just hard-code a check
for the InnoDB storage engine. This will allow us to get real-world testing to
see if the new speculative parallel replication feature is interesting to any
users. Then if it is, and there turns out to be interest from other storage
engine venders (such as Tokutek) to also support speculative parallel
replication, we can work with them to implement a proper API.

This approach would also allow to consider the needs of a different storage
engine when designing the API, which makes it much more likely that the API
will be of general use and not too tied to InnoDB internals.

What do you think?

 - Kristian.