maria-developers team mailing list archive

[Zardosht Kasheff <zardosht@xxxxxxxxx>]

Hello Kristian,

Thanks for the feedback.

For me, the biggest design goal is to have a way for slaves to be
crash safe without requiring the storage engine to fsync every
transaction it processes. This would be a huge performance win.

In a world where slave replication is single threaded, I solution (2),
where the information is stored in a slave_relay_log_info table, for
the performance reasons you mention. Not requiring fsyncs on
transactional commit seems like a big deal, not just for InnoDB, but
any transactional storage engine.

As far as crash safety goes, is DDL the only issue? If so, engines can
implement the handlerton's discover API to make itself crash safe.
Should there be a crash, a little tweaking will be needed to get the
relay log info to the right position, but it should be doable.

Inlining other points

On Tue, Oct 23, 2012 at 7:49 AM, Kristian Nielsen
<knielsen@xxxxxxxxxxxxxxx> wrote:
> Thinking more about this, I cannot quite make up my mind of the best approach.
> I would like to get some input.
>
> On the one hand, when binlog is enabled on a slave, it really is the best
> solution for storing replication state and crash safety.
>
> On the other hand, running a slave with innodb_flush_log_at_trx_commit=0 and
> no binlog will perform much faster and makes a lot of sense for some setups
> (after all if we crash we can just replicate again any transactions lost).

For performance, this sounds like what users that want to scale will
want, so I am partial towards this solution.

>
> Hm.
>
> It seems to get the best of both worlds, we need a dual approach.
>
> When binlog is enabled, we use it to store and crash-recover slave state.
>
> When binlog is disabled, every transaction inserts a row in some table
> containing the global transaction ID. After a crash we then obtain the last
> applied transaction with SELECT MAX(...) from that table. And we need to
> periodically delete old rows.

To better analyze this, can you please give some details on how a
slave would apply a binary log in parallel? If it is just what MySQL
does, and applies it per database, then perhaps a row in the relay log
table per database would be sufficient. I think whatever solution
makes sense would depend on the implementation of parallel
replication.

>
> Both methods work with any (XA-capable) storage engine. However, the table
> approach will have a large overhead when the engine used for the table is
> different from the engine used in transactions.

True, but that's the way it is. It is (hopefully) well known that
keeping data across multiple engines consistent incurs performance
hits with XA. So, users will hopefully have their data predominately
in one engine. If not, they suffer performance in this scenario.
Nothing anyone can do about it.

>
> Thoughts?
>
>  - Kristian.
>
> Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx> writes:
>
>> Zardosht Kasheff <zardosht@xxxxxxxxx> writes:
>>
>>> Thank you for the reply. I want to make sure I understand this
>>> correctly. Is the planned design for crash safe replication to be that
>>> the slave must have the binary log enabled, so that slaves may use the
>>> GTID and XA to ensure that the slave and master are in sync? If the
>>> binary log is not enabled, then the slaves are not crash safe. Is my
>>> understanding correct?
>>
>> I know of three different features that touch on "crash safe slave":
>>
>> 1. MDEV-26. If/when this is implemented, the plan is that the state is made
>> crash-safe using the binary log, as you said.
>>
>> 2. The MySQL feature, where we store in a transactional table the info that is
>> now in relay-log.info.
>>
>> 3. The XtraDB hack, where InnoDB overwrites the relay-log.info at startup from
>> internal data salvaged from the InnoDB redo log.