← Back to team overview

maria-developers team mailing list archive

Re: parallel replication monitoring

 

Hi Kristian,

Thanks for the explanation. Let me elaborate a bit about the questions:
* how many workers are used: I know how many workers I have *enabled* by @@slave_parallel_threads. But how many are really used? I see in mysql.gtid_slave_pos that the number of rows grows and shrinks while the workers are replicating, but this does not give me any useful information on the effectiveness of the operations. Just to give you an idea, in Tungsten replication, I can see for every thread how many requests were processed. This can help me fine tuning the process, and add or remove threads to reach optimal performance.
* which database they are running is important even if the parallel replication is not split by schema. As a DBA, I need to know at a glance where the action is occurring. SHOW PROCESSLIST gives me that information, but it doesn't tell me the GTID, which is the info that gives me the pulse of the replication progress. What bothers me is that I can get either the GTID list or where the action is happening. Having both in one place should not be difficult (a I_S or P_S view, for example). I see in MariaDB implementation some of the lack of integration that I have seen in MySQL 5.6.

I hope to see better integration and more tools in the future. As it is now, parallel replication seems to be a very powerful engine with brake and steering wheel but with a blackened out dashboard. Makes for some adventurous driving!

Cheers

Giuseppe

On November 15, 2013 at 08:49:29 , Kristian Nielsen (knielsen@xxxxxxxxxxxxxxx) wrote:

Giuseppe Maxia <g.maxia@xxxxxxxxx> writes:  

> I am looking for:  
> * how many workers are used;  

The number of workers created is determined by the value of  
@@slave_parallel_threads. That many worker threads are in the replication  
worker thread pool, shared among all multi-source master connections.  

Workers are scheduled by the SQL threads dynamically, depending on what  
parallelism is available in the events being read from the relay logs at any  
given point in time. Workers for which there is currently no work available  
will appear as "Waiting for work from SQL thread" in SHOW PROCESSLIST.  

> * what query they are running.  

I think this should appear in SHOW PROCESSLIST, same as for the SQL thread in  
non-parallel replication.  

> * in which schema they are working;  

That is not really meaningful? A worker thread can be working in multiple  
schemas at the same time (multi-table update...).  

The parallel replication feature works like this:  

- On the master, binlog events are marked for whether they can be replicated  
in parallel. Two events can be replication in parallel if 1) their GTID  
events have the same group commit ID (cid=XXX in mysqlbinlog output), or 2)  
if their GTID has different replication domain (first number in the  
GTID=D-S-N tripple), and the slave is operating in GTID mode.  

- On the slave, the SQL thread reads events from the relay log, and  
dynamically schedules the same or a new worker to execute the events so  
read, depending on whether parallelism is possible or not. If the pool of  
threads is exhausted, the SQL thread waits for a worker thread to become  
idle.  

So unlike MySQL multi-threaded slave, worker threads in MariaDB have no  
identity. Any worker thread can potentially be scheduled to work on any event  
from any master connection.  

> What can I use to see the status of parallel replication?  

Nowhere, really :-(  

The parallel replication feature has been basically rushed ruthlessly to  
release, with no time taken for any polish like this or other similar  
reasonable concerns...  

I did briefly think of what could be made available, but I did not come up  
with any immediate ideas. The scheduling of parallel replication work really  
is very dynamic in nature, just like any other application load. Maybe the  
performance schema or something like that could be used, but I am not familiar  
with how it works. Suggestions definitely welcome.  

Giuseppe, btw, thanks for once again taking the time to look and comment on my  
work!  

- Kristian.  

Follow ups

References