← Back to team overview

maria-developers team mailing list archive

Re: MariaDB GTID first impression

 

Kristian,
If performance is an issue that prevents writing to the gtid table, perhaps a INFORMATIOn_SCHEMA table would do the trick? (Having the state of replication in tables has been my pet peeve since 2006, and none of the MySQL implementations seems to share my view)

Cheers

Giuseppe



On Tuesday, May 7, 2013 at 12:52 , Kristian Nielsen wrote:

> Giuseppe Maxia <g.maxia@xxxxxxxxx (mailto:g.maxia@xxxxxxxxx)> writes:
> 
> > Thanks for the docs. It's a very good start. Compared to MySQL 5.6, GTID in MariaDB has some advantages: it is enabled by default and it is easy to read.
> > 
> > There are a few items, though, that should need some more thought:
> 
> > 1) rpl_slave_state is probably ill-named. In this architecture, it is only
> 
> Yeah, I agree. Let's find a better name.
> 
> What about: mysql.rpl_gtid_state?
> 
> Or maybe: mysql.gtid_pos. This matches the similar @@GLOBAL.gtid_pos
> variable. Though the table is not a simple position, it has a more complex
> structure.
> 
> Opinions?
> 
> > written by the slaves, and thus the name matches, but I wonder if the master
> > should write this table (or a similar one) to keep track of its progress in
> > 
> 
> 
> Right. Actually, I've carefully avoided writing the table on the master, to
> avoid introducing extra overhead.
> 
> > the binary log. Rationale: all the info related to replication should be in
> > tables and easy to query.
> > 
> 
> 
> That's one valid viewpoint. Unfortunately this is not really supported in the
> design. I mean, you can query the table if you want, but there is not
> guaranteed to be an easy way to map the results to the actual internal
> state. Similarly, updating the table is not guaranteed to be reflected in the
> internal in-memory state. This is all to reduce overhead.
> 
> > 2) rpl_slave_state does not have enough metadata. What is missing here, and
> > would be extremely useful, is the corresponding binlog+position for each
> > GTID, a timestamp of when the transaction was saved to the binary log, and
> > the timestamp of when it was applied. (The host name of the origin would
> > 
> 
> 
> Right. I had this thought myself that I should maybe put more stuff in
> there, it is good to get your suggestion, based on more experience with actual
> usage.
> 
> I think most of this info is already available. Though timestamps might be
> worth it to log into Gtid with better than whole second precision? And not
> sure about the host name (but we do have server_id, of course).
> 
> I will think about this. At least, this should be easy to add later, but it
> would be good to have as much as possible in the first release.
> 
> > also be useful). Also very important (when you will implement parallel
> > replication) is a shard identifier.
> > 
> 
> 
> Actually, the domain_id is the shard identifier. This is a core part of the
> desing, though actual implementation will be for later.
> 
> > 3) Multiple domains do not integrate well with GTID. In multi-source
> > replication, we use "connection names", but for GTID we use "domain
> > numbers". They do not play well together : there is no correspondence
> > between the gtid_domain_id and the connection name, and a "SHOW ALL SLAVES
> > STATUS" lists all the GTID for every connection. For example:
> > 
> 
> 
> Right. I understand your point of course, it would be natural to expect domain
> id to be equivalent somehow to multi-source connection name.
> 
> Unfortunately, that is not how things work. There is no simple translation
> between multi-source connection name and domain id.
> 
> For example, consider a topology like A->C, B->C, C->E, D->E. Here, lets say A
> has domain id 1, B has 2, and D has 3. There is no single domain_id for the
> connection from E to C.
> 
> Or a ring topology: A->B->C->A. Here, there will be three domains, but there
> is only a single connection.
> 
> And later, we will do parallel replication, where a single master will use
> multiple different domain_ids (basically one for each shard I suppose).
> 
> On the other hand, normally a given domain_id should only be received on a
> single of multi-source connections. It sounds like a good idea to be able to
> specify a list of valid domains for each multi-source connection. Then the
> code could also check for violations and stop replication with an error if
> such violation happens. I will think about adding this.
> 
> > Connection_name: Deer
> > Master_Host: 127.0.0.1
> > Master_Port: 8206
> > Master_Log_File: mysql-bin.000001
> > Read_Master_Log_Pos: 1144
> > Relay_Master_Log_File: mysql-bin.000001
> > Slave_IO_Running: Yes
> > Slave_SQL_Running: Yes
> > Exec_Master_Log_Pos: 1144
> > Gtid_Pos: 100-101-21,400-104-23,200-102-8,300-103-12,500-105-20
> > 
> > Which connection is running which gtid number? Not easy to tell, as the correspondence between domain and connections is only in the DBA's head.
> 
> Well, there is a single global Gtid_pos, it is shared by all connections.
> 
> I do not see how this can be different? For example, Suppose we have A->C,
> B->C, C->D. If C goes away, we will want to switch to A->D, B->D. But the
> Gtid_pos should not change while doing this switch.
> 
> Hm, I will admit I did not before think much about a tighter connection
> between GTID position and individual slave connections. I have just considered
> the GTID state as associated with the data actually in the tables, not with
> any specific configuration ...
> 
> > 4) gtid_pos is text that includes all sequences from all domains. The docs say that I can modify it if needed, but in a case like the one above, modifying this string can be very error prone. For example, if server_ids are assigned using inet_aton(@IP), you could have a gtid_pos like this 
> > '1001-170062692-230000,1002-170062593-12893445,1003-170062700-672009,1003-170062699-700002'
> > Instead of editing a string, and risk breaking replication in all domains, I would expect the availability of commands like:
> > CHANGE GTID for domain 'Wolf' set gtid_pos=1900
> > 
> 
> 
> Right, I agree. On the other hand, if one is doing scripting to e.g. set up a
> new slave automatically, it is much easier to be able to pass around a single
> string of Gtid position, rather than having to contruct complicated SQL
> expressions to add some, change others, and remove any extra.
> 
> So that is why I chose the simple approach with a single string for the first
> version.
> 
> Your comments made me realise that much of the current design is based towards
> how I prefer to use databases - where "everything in the dba's/developer's
> head" (or better the source revision control system) is a good thing. But
> there is also a need to support a more database-centric approach, where the
> only source of configuration and metadata is the current state of the
> DB. Hopefully more of this other approach can be added and extended as we go
> along.
> 
> > That's it for now. Hope there is still room for improvements.
> 
> Yes. Thanks a lot Giuseppe for taking the time to review this and provide your
> opinions.
> 
> - Kristian. 


Follow ups

References