maria-developers team mailing list archive

[sujatha <sujatha.sivakumar@xxxxxxxxxxx>]

Hello Sergei,

On 22/03/21 5:42 pm, Sergei Golubchik wrote:
> Hi, Sujatha!
>
> On Mar 22, sujatha wrote:
> > On 21/03/21 8:21 pm, Sergei Golubchik wrote:
> > > Hi, Sujatha!
> > >
> > > Could you split this patch, please?
> > Sure.
> > > 1. Just add replication_applier_status_by_worker table. With
> > >      CHANNEL_NAME column and (perhaps, not sure if it applies) with
> > >      WORKER_ID column. Without extra columns and backup.
> > Reason for not including CHANNEL_NAME and WORKER_ID is, multi-source
> > based parallel replication is bit different in MySQL.
> >
> > Please find the following information.
> >
> > ==> https://dev.mysql.com/doc/refman/5.7/en/replication-channels.html
> >
> >           A multi-source replica can also be set up as a multi-threaded
> > replica, by setting the slave_parallel_workers system variable to a
> > value greater than 0. When you do this on a multi-source replica, each
> > channel on the replica         has the specified number of applier
> > threads, plus a coordinator thread to manage them. You cannot configure
> > the number of applier threads for individual channels.
> >
> > In case of MySQL, worker threads are dedicated to particular channel.
> >
> > In MariaDB "The pool of replication worker threads is shared among all
> > multi-source master connections, and among all replication domains that
> > can replicate in parallel using out-of-order".
> >
> > Because of this I didn't include CHANNEL_NAME. In MariaDB Slave_worker
> > thread's don't have WORKER_ID they only have 'thread_id'.
> Okay, thanks
>
> May be it'd still make sense to keep CHANNEL_NAME, for the connection
> name the worker is currently working for? Even if the pool is shared, at
> any given moment in time the worker can apply a transaction only from
> some specific connection, right?


Sure. The 'Connection_Name' can be printed.

> > > 3. Add backup.
> > >
> > > With these three commits you'll have exactly the same diff as now, just
> > > split (and with CHANNEL_NAME column).
> > >
> > > But really, I wonder whether this backup functionality is needed at all?
> > > In MySQL there is persistent information about these workers, it doesn't
> > > go away when they're stopped, it's stored persistently in a table,
> > > indexed by worker_id. If we don't have anything persistent like that and
> > > all workers completely disappear into oblivion, then, may be,
> > > replication_applier_status_by_worker should not show anything when they
> > > aren't running?
> > MySQL doesn't persist the worker information in a table. When workers
> > are stopped due to an error or STOP SLAVE,  worker information is
> > copied(backup) and it is retained till next START SLAVE. Please find
> > following snippets.
> Well, there is mysql.slave_worker_info, it's persistent.


Yes, you are right `mysql.slave_worker_info` persists data, for server 
internal usage purpose.

The data which gets printed through 
'performance_schema.replication_applier_status_worker' is

not persisted. For example PFS tables can provide more details for the 
user either on stop/error though backup.

mysql> select * from mysql.slave_worker_info;
+----+--------------------------+---------------+-------------------+----------------+---------------------------+--------------------------+----------------------------+---------------------------+------------------+-----------------------+------------------------------------------------------------------+--------------+
| Id | Relay_log_name           | Relay_log_pos | Master_log_name   | 
Master_log_pos | Checkpoint_relay_log_name | Checkpoint_relay_log_pos | 
Checkpoint_master_log_name | Checkpoint_master_log_pos | 
Checkpoint_seqno | Checkpoint_group_size | 
Checkpoint_group_bitmap                                          | 
Channel_name |
+----+--------------------------+---------------+-------------------+----------------+---------------------------+--------------------------+----------------------------+---------------------------+------------------+-----------------------+------------------------------------------------------------------+--------------+
|  1 |                          |             0 |                   
|              0 | |                        0 | 
|                         0 |                0 |                    64 | 
|              |
|  2 |                          |             0 |                   
|              0 | |                        0 | 
|                         0 |                0 |                    64 | 
|              |
|  3 |                          |             0 |                   
|              0 | |                        0 | 
|                         0 |                0 |                    64 | 
|              |
|  4 | ./slave-relay-bin.000002 |           810 | master-bin.000001 
|            595 | ./slave-relay-bin.000002 |                      369 | 
master-bin.000001 |                       154 |                1 
|                    64 |  |              |
+----+--------------------------+---------------+-------------------+----------------+---------------------------+--------------------------+----------------------------+---------------------------+------------------+-----------------------+------------------------------------------------------------------+--------------+
4 rows in set (0.00 sec)

mysql> select * from 
performance_schema.replication_applier_status_by_worker;
+--------------+-----------+-----------+---------------+-----------------------+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+
| CHANNEL_NAME | WORKER_ID | THREAD_ID | SERVICE_STATE | 
LAST_SEEN_TRANSACTION | LAST_ERROR_NUMBER | LAST_ERROR_MESSAGE | 
LAST_ERROR_TIMESTAMP |
+--------------+-----------+-----------+---------------+-----------------------+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+
|              |         1 |      NULL | OFF |                       
|                 0 | | 0000-00-00 00:00:00  |
|              |         2 |      NULL | OFF |                       
|                 0 | | 0000-00-00 00:00:00  |
|              |         3 |      NULL | OFF |                       
|                 0 | | 0000-00-00 00:00:00  |
|              |         4 |      NULL | OFF           | 
ANONYMOUS             |              1062 | Worker 4 failed executing 
transaction 'ANONYMOUS' at master log master-bin.000001, end_log_pos 
817; Could not execute Write_rows event on table test.t1; Duplicate 
entry '30' for key 'PRIMARY', Error_code: 1062; handler error 
HA_ERR_FOUND_DUPP_KEY; the event's master log master-bin.000001, 
end_log_pos 817 | 2021-03-22 15:44:00  |
+--------------+-----------+-----------+---------------+-----------------------+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+
4 rows in set (0.00 sec)


Thank you

S.Sujatha

> Regards,
> Sergei
> VP of MariaDB Server Engineering
> and security@xxxxxxxxxxx