maria-developers team mailing list archive
-
maria-developers team
-
Mailing list archive
-
Message #07514
Re: comments to the plugin.h
Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx> writes:
> So then, the plan seems to be:
>
> 1. I remove the new calls from include/plugin.h, instead place them somewhere
> not part of a public API (maybe just "extern" declarations inside
> InnoDB/XtraDB, and whatever is needed to make it work correctly for
> ha_innodb.dll on Windows).
>
> 2. I try to remove the kill-in-background, instead do it directly in the
> thread doing thd_report_wait_for() (I think that should be possible).
>
> 3. I apply the other review comments that you sent in another mail.
>
> 4. I file a Jira task for 10.1 about a general solution, with a good API and
> other ideas collected so far.
I have now done this. I have included the patch below, which is on top of the
patch you reviewed earlier. You can also extract a complete diff as you wish
from the tree lp:~maria-captains/maria/10.0-knielsen, where I have pushed the
latest changes.
- Kristian.
revno: 4178
committer: Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>
branch nick: tmp-10.0
timestamp: Tue 2014-07-08 12:54:47 +0200
message:
MDEV-5262, MDEV-5914, MDEV-5941, MDEV-6020: Deadlocks during parallel replication causing replication to fail.
After-review changes.
For this patch in 10.0, we do not introduce a new public storage engine API,
we just fix the InnoDB/XtraDB issues. In 10.1, we will make a better public
API that can be used for all storage engines (MDEV-6429).
Eliminate the background thread that did deadlock kills asynchroneously.
Instead, we ensure that the InnoDB/XtraDB code can handle doing the kill from
inside the deadlock detection code (when thd_report_wait_for() needs to kill a
later thread to resolve a deadlock).
(We preserve the part of the original patch that introduces dedicated mutex
and condition for the slave init thread, to remove the abuse of
LOCK_thread_count for start/stop synchronisation of the slave init thread).
=== modified file 'include/mysql/plugin.h'
--- include/mysql/plugin.h 2014-06-10 08:13:15 +0000
+++ include/mysql/plugin.h 2014-07-08 10:54:47 +0000
@@ -622,6 +622,7 @@ void **thd_ha_data(const MYSQL_THD thd,
void thd_storage_lock_wait(MYSQL_THD thd, long long value);
int thd_tx_isolation(const MYSQL_THD thd);
int thd_tx_is_read_only(const MYSQL_THD thd);
+int thd_rpl_is_parallel(const MYSQL_THD thd);
/**
Create a temporary file.
@@ -729,80 +730,6 @@ void thd_set_ha_data(MYSQL_THD thd, cons
*/
void thd_wakeup_subsequent_commits(MYSQL_THD thd, int wakeup_error);
-/*
- Used by a storage engine to report that one transaction THD is about to
- go to wait for a transactional lock held by another transactions OTHER_THD.
-
- This is used for parallel replication, where transactions are required to
- commit in the same order on the slave as they did on the master. If the
- transactions on the slave can encounter lock conflicts on the slave that did
- not exist on the master, this can cause deadlocks.
-
- The storage engine can report such conflicting locks using this call. This
- will allow parallel replication to detect such conflicts and resolve the
- deadlock (by killing the second transaction to release the locks that the
- first is waiting for, and then later re-try the second killed transaction).
-
- The storage engine should not report false positives. That is, it should not
- report any lock waits that do not actually require one transaction to wait
- for the other. Nor should it report waits for locks that will be released
- before the commit of the other transactions.
-*/
-void thd_report_wait_for(const MYSQL_THD thd, MYSQL_THD other_thd);
-
-/*
- This function can optionally be called to check if thd_report_wait_for()
- needs to be called for waits done by a given transaction.
-
- If this function returns false for a given thd, there is no need to do any
- calls to thd_report_wait_for() on that thd.
-
- This call is optional; it is safe to call thd_report_wait_for() in any case.
- This call can be used to save some redundant calls to thd_report_wait_for()
- if desired. (This is unlikely to matter much unless there are _lots_ of
- waits to report, as the overhead of thd_report_wait_for() is small).
-*/
-int thd_need_wait_for(const MYSQL_THD thd);
-
-/*
- This function can be called by storage engines to check if the commit order
- of two transactions has already been decided by the upper layer. This
- happens in parallel replication, where the commit order is forced to be the
- same on the slave as it was originally on the master.
-
- If this function returns false, it means that such commit order will be
- enforced. This allows the storage engine to optionally omit gap lock waitss
- or similar measures that would otherwise be needed to ensure that
- transactions would be serialised in a way that would cause a commit order
- that is correct for binlogging for statement-based replication.
-
- If this function returns true, normal locking should be done as required by
- the binlogging and transaction isolation level in effect.
-*/
-int thd_need_ordering_with(const MYSQL_THD thd, const MYSQL_THD other_thd);
-
-/*
- If the storage engine detects a deadlock, and needs to choose a victim
- transaction to roll back, it can call this function to ask the upper
- server layer for which of two possible transactions is prefered to be
- aborted and rolled back.
-
- In parallel replication, if two transactions are running in parallel and
- one is fixed to commit before the other, then the one that commits later
- will be prefered as the victim - chosing the early transaction as a victim
- will not resolve the deadlock anyway, as the later transaction still needs
- to wait for the earlier to commit.
-
- Otherwise, a transaction that uses only transactional tables, and can thus
- be safely rolled back, will be prefered as a deadlock victim over a
- transaction that also modified non-transactional (eg. MyISAM) tables.
-
- The return value is -1 if the first transaction is prefered as a deadlock
- victim, 1 if the second transaction is prefered, or 0 for no preference (in
- which case the storage engine can make the choice as it prefers).
-*/
-int thd_deadlock_victim_preference(const MYSQL_THD thd1, const MYSQL_THD thd2);
-
#ifdef __cplusplus
}
#endif
=== modified file 'include/mysql/plugin_audit.h.pp'
--- include/mysql/plugin_audit.h.pp 2014-06-10 08:13:15 +0000
+++ include/mysql/plugin_audit.h.pp 2014-07-08 10:54:47 +0000
@@ -303,6 +303,7 @@ void **thd_ha_data(const void* thd, cons
void thd_storage_lock_wait(void* thd, long long value);
int thd_tx_isolation(const void* thd);
int thd_tx_is_read_only(const void* thd);
+int thd_rpl_is_parallel(const void* thd);
int mysql_tmpfile(const char *prefix);
unsigned long thd_get_thread_id(const void* thd);
void thd_get_xid(const void* thd, MYSQL_XID *xid);
@@ -313,10 +314,6 @@ void *thd_get_ha_data(const void* thd, c
void thd_set_ha_data(void* thd, const struct handlerton *hton,
const void *ha_data);
void thd_wakeup_subsequent_commits(void* thd, int wakeup_error);
-void thd_report_wait_for(const void* thd, void *other_thd);
-int thd_need_wait_for(const void* thd);
-int thd_need_ordering_with(const void* thd, const void* other_thd);
-int thd_deadlock_victim_preference(const void* thd1, const void* thd2);
struct mysql_event_general
{
unsigned int event_subclass;
=== modified file 'include/mysql/plugin_auth.h.pp'
--- include/mysql/plugin_auth.h.pp 2014-06-10 08:13:15 +0000
+++ include/mysql/plugin_auth.h.pp 2014-07-08 10:54:47 +0000
@@ -303,6 +303,7 @@ void **thd_ha_data(const void* thd, cons
void thd_storage_lock_wait(void* thd, long long value);
int thd_tx_isolation(const void* thd);
int thd_tx_is_read_only(const void* thd);
+int thd_rpl_is_parallel(const void* thd);
int mysql_tmpfile(const char *prefix);
unsigned long thd_get_thread_id(const void* thd);
void thd_get_xid(const void* thd, MYSQL_XID *xid);
@@ -313,10 +314,6 @@ void *thd_get_ha_data(const void* thd, c
void thd_set_ha_data(void* thd, const struct handlerton *hton,
const void *ha_data);
void thd_wakeup_subsequent_commits(void* thd, int wakeup_error);
-void thd_report_wait_for(const void* thd, void *other_thd);
-int thd_need_wait_for(const void* thd);
-int thd_need_ordering_with(const void* thd, const void* other_thd);
-int thd_deadlock_victim_preference(const void* thd1, const void* thd2);
#include <mysql/plugin_auth_common.h>
typedef struct st_plugin_vio_info
{
=== modified file 'include/mysql/plugin_ftparser.h.pp'
--- include/mysql/plugin_ftparser.h.pp 2014-06-10 08:13:15 +0000
+++ include/mysql/plugin_ftparser.h.pp 2014-07-08 10:54:47 +0000
@@ -256,6 +256,7 @@ void **thd_ha_data(const void* thd, cons
void thd_storage_lock_wait(void* thd, long long value);
int thd_tx_isolation(const void* thd);
int thd_tx_is_read_only(const void* thd);
+int thd_rpl_is_parallel(const void* thd);
int mysql_tmpfile(const char *prefix);
unsigned long thd_get_thread_id(const void* thd);
void thd_get_xid(const void* thd, MYSQL_XID *xid);
@@ -266,10 +267,6 @@ void *thd_get_ha_data(const void* thd, c
void thd_set_ha_data(void* thd, const struct handlerton *hton,
const void *ha_data);
void thd_wakeup_subsequent_commits(void* thd, int wakeup_error);
-void thd_report_wait_for(const void* thd, void *other_thd);
-int thd_need_wait_for(const void* thd);
-int thd_need_ordering_with(const void* thd, const void* other_thd);
-int thd_deadlock_victim_preference(const void* thd1, const void* thd2);
enum enum_ftparser_mode
{
MYSQL_FTPARSER_SIMPLE_MODE= 0,
=== modified file 'mysql-test/suite/perfschema/r/threads_mysql.result'
--- mysql-test/suite/perfschema/r/threads_mysql.result 2014-06-03 08:31:11 +0000
+++ mysql-test/suite/perfschema/r/threads_mysql.result 2014-07-08 10:54:47 +0000
@@ -44,16 +44,6 @@ processlist_info NULL
unified_parent_thread_id unified parent_thread_id
role NULL
instrumented YES
-name thread/sql/slave_background
-type BACKGROUND
-processlist_user NULL
-processlist_host NULL
-processlist_db NULL
-processlist_command NULL
-processlist_info NULL
-unified_parent_thread_id unified parent_thread_id
-role NULL
-instrumented YES
CREATE TEMPORARY TABLE t1 AS
SELECT thread_id FROM performance_schema.threads
WHERE name LIKE 'thread/sql%';
@@ -115,5 +105,4 @@ parent_thread_name child_thread_name
thread/sql/event_scheduler thread/sql/event_worker
thread/sql/main thread/sql/one_connection
thread/sql/main thread/sql/signal_handler
-thread/sql/main thread/sql/slave_background
thread/sql/one_connection thread/sql/event_scheduler
=== modified file 'sql/log.cc'
--- sql/log.cc 2014-06-10 08:13:15 +0000
+++ sql/log.cc 2014-07-08 10:54:47 +0000
@@ -4078,7 +4078,7 @@ int MYSQL_BIN_LOG::purge_first_log(Relay
included= false;
break;
}
- if (!included && 0 == strcmp(ir->name, rli->group_relay_log_name))
+ if (!included && !strcmp(ir->name, rli->group_relay_log_name))
break;
if (!next)
{
@@ -9332,7 +9332,7 @@ int TC_LOG_BINLOG::recover(LOG_INFO *lin
file= -1;
}
- if (0 == strcmp(linfo->log_file_name, last_log_name))
+ if (!strcmp(linfo->log_file_name, last_log_name))
break; // No more files to do
if ((file= open_binlog(&log, linfo->log_file_name, &errmsg)) < 0)
{
=== modified file 'sql/log_event.cc'
--- sql/log_event.cc 2014-06-10 08:13:15 +0000
+++ sql/log_event.cc 2014-07-08 10:54:47 +0000
@@ -7324,6 +7324,13 @@ int Xid_log_event::do_apply_event(rpl_gr
uint64 sub_id= 0;
Relay_log_info const *rli= rgi->rli;
+ /*
+ XID_EVENT works like a COMMIT statement. And it also updates the
+ mysql.gtid_slave_pos table with the GTID of the current transaction.
+
+ Therefore, it acts much like a normal SQL statement, so we need to do
+ mysql_reset_thd_for_next_command() as if starting a new statement.
+ */
mysql_reset_thd_for_next_command(thd);
/*
Record any GTID in the same transaction, so slave state is transactionally
=== modified file 'sql/mysqld.cc'
--- sql/mysqld.cc 2014-06-03 08:31:11 +0000
+++ sql/mysqld.cc 2014-07-08 10:54:47 +0000
@@ -361,7 +361,7 @@ static I_List<THD> thread_cache;
static bool binlog_format_used= false;
LEX_STRING opt_init_connect, opt_init_slave;
static mysql_cond_t COND_thread_cache, COND_flush_thread_cache;
-mysql_cond_t COND_slave_background;
+mysql_cond_t COND_slave_init;
static DYNAMIC_ARRAY all_options;
/* Global variables */
@@ -700,7 +700,7 @@ mysql_mutex_t
LOCK_crypt,
LOCK_global_system_variables,
LOCK_user_conn, LOCK_slave_list, LOCK_active_mi,
- LOCK_connection_count, LOCK_error_messages, LOCK_slave_background;
+ LOCK_connection_count, LOCK_error_messages, LOCK_slave_init;
mysql_mutex_t LOCK_stats, LOCK_global_user_client_stats,
LOCK_global_table_stats, LOCK_global_index_stats;
@@ -875,7 +875,7 @@ PSI_mutex_key key_LOCK_stats,
PSI_mutex_key key_LOCK_gtid_waiting;
PSI_mutex_key key_LOCK_prepare_ordered, key_LOCK_commit_ordered,
- key_LOCK_slave_background;
+ key_LOCK_slave_init;
PSI_mutex_key key_TABLE_SHARE_LOCK_share;
static PSI_mutex_info all_server_mutexes[]=
@@ -938,7 +938,7 @@ static PSI_mutex_info all_server_mutexes
{ &key_LOCK_error_messages, "LOCK_error_messages", PSI_FLAG_GLOBAL},
{ &key_LOCK_prepare_ordered, "LOCK_prepare_ordered", PSI_FLAG_GLOBAL},
{ &key_LOCK_commit_ordered, "LOCK_commit_ordered", PSI_FLAG_GLOBAL},
- { &key_LOCK_slave_background, "LOCK_slave_background", PSI_FLAG_GLOBAL},
+ { &key_LOCK_slave_init, "LOCK_slave_init", PSI_FLAG_GLOBAL},
{ &key_LOG_INFO_lock, "LOG_INFO::lock", 0},
{ &key_LOCK_thread_count, "LOCK_thread_count", PSI_FLAG_GLOBAL},
{ &key_LOCK_thread_cache, "LOCK_thread_cache", PSI_FLAG_GLOBAL},
@@ -993,7 +993,7 @@ PSI_cond_key key_TC_LOG_MMAP_COND_queue_
PSI_cond_key key_COND_rpl_thread_queue, key_COND_rpl_thread,
key_COND_rpl_thread_pool,
key_COND_parallel_entry, key_COND_group_commit_orderer,
- key_COND_prepare_ordered, key_COND_slave_background;
+ key_COND_prepare_ordered, key_COND_slave_init;
PSI_cond_key key_COND_wait_gtid, key_COND_gtid_ignore_duplicates;
static PSI_cond_info all_server_conds[]=
@@ -1042,7 +1042,7 @@ static PSI_cond_info all_server_conds[]=
{ &key_COND_parallel_entry, "COND_parallel_entry", 0},
{ &key_COND_group_commit_orderer, "COND_group_commit_orderer", 0},
{ &key_COND_prepare_ordered, "COND_prepare_ordered", 0},
- { &key_COND_slave_background, "COND_slave_background", 0},
+ { &key_COND_slave_init, "COND_slave_init", 0},
{ &key_COND_wait_gtid, "COND_wait_gtid", 0},
{ &key_COND_gtid_ignore_duplicates, "COND_gtid_ignore_duplicates", 0}
};
@@ -1050,7 +1050,7 @@ static PSI_cond_info all_server_conds[]=
PSI_thread_key key_thread_bootstrap, key_thread_delayed_insert,
key_thread_handle_manager, key_thread_main,
key_thread_one_connection, key_thread_signal_hand,
- key_thread_slave_background, key_rpl_parallel_thread;
+ key_thread_slave_init, key_rpl_parallel_thread;
static PSI_thread_info all_server_threads[]=
{
@@ -1076,7 +1076,7 @@ static PSI_thread_info all_server_thread
{ &key_thread_main, "main", PSI_FLAG_GLOBAL},
{ &key_thread_one_connection, "one_connection", 0},
{ &key_thread_signal_hand, "signal_handler", PSI_FLAG_GLOBAL},
- { &key_thread_slave_background, "slave_background", PSI_FLAG_GLOBAL},
+ { &key_thread_slave_init, "slave_init", PSI_FLAG_GLOBAL},
{ &key_rpl_parallel_thread, "rpl_parallel_thread", 0}
};
@@ -2162,8 +2162,8 @@ static void clean_up_mutexes()
mysql_mutex_destroy(&LOCK_prepare_ordered);
mysql_cond_destroy(&COND_prepare_ordered);
mysql_mutex_destroy(&LOCK_commit_ordered);
- mysql_mutex_destroy(&LOCK_slave_background);
- mysql_cond_destroy(&COND_slave_background);
+ mysql_mutex_destroy(&LOCK_slave_init);
+ mysql_cond_destroy(&COND_slave_init);
DBUG_VOID_RETURN;
}
@@ -4378,9 +4378,9 @@ static int init_thread_environment()
mysql_cond_init(key_COND_prepare_ordered, &COND_prepare_ordered, NULL);
mysql_mutex_init(key_LOCK_commit_ordered, &LOCK_commit_ordered,
MY_MUTEX_INIT_SLOW);
- mysql_mutex_init(key_LOCK_slave_background, &LOCK_slave_background,
+ mysql_mutex_init(key_LOCK_slave_init, &LOCK_slave_init,
MY_MUTEX_INIT_SLOW);
- mysql_cond_init(key_COND_slave_background, &COND_slave_background, NULL);
+ mysql_cond_init(key_COND_slave_init, &COND_slave_init, NULL);
#ifdef HAVE_OPENSSL
mysql_mutex_init(key_LOCK_des_key_file,
@@ -9458,8 +9458,6 @@ PSI_stage_info stage_waiting_for_room_in
PSI_stage_info stage_master_gtid_wait_primary= { 0, "Waiting in MASTER_GTID_WAIT() (primary waiter)", 0};
PSI_stage_info stage_master_gtid_wait= { 0, "Waiting in MASTER_GTID_WAIT()", 0};
PSI_stage_info stage_gtid_wait_other_connection= { 0, "Waiting for other master connection to process GTID received on multiple master connections", 0};
-PSI_stage_info stage_slave_background_process_request= { 0, "Processing requests", 0};
-PSI_stage_info stage_slave_background_wait_request= { 0, "Waiting for requests", 0};
#ifdef HAVE_PSI_INTERFACE
@@ -9579,9 +9577,7 @@ PSI_stage_info *all_server_stages[]=
& stage_waiting_to_get_readlock,
& stage_master_gtid_wait_primary,
& stage_master_gtid_wait,
- & stage_gtid_wait_other_connection,
- & stage_slave_background_process_request,
- & stage_slave_background_wait_request
+ & stage_gtid_wait_other_connection
};
PSI_socket_key key_socket_tcpip, key_socket_unix, key_socket_client_connection;
=== modified file 'sql/mysqld.h'
--- sql/mysqld.h 2014-06-03 08:31:11 +0000
+++ sql/mysqld.h 2014-07-08 10:54:47 +0000
@@ -309,8 +309,8 @@ extern PSI_cond_key key_COND_wait_gtid,
extern PSI_thread_key key_thread_bootstrap, key_thread_delayed_insert,
key_thread_handle_manager, key_thread_kill_server, key_thread_main,
- key_thread_one_connection, key_thread_signal_hand,
- key_thread_slave_background, key_rpl_parallel_thread;
+ key_thread_one_connection, key_thread_signal_hand, key_thread_slave_init,
+ key_rpl_parallel_thread;
extern PSI_file_key key_file_binlog, key_file_binlog_index, key_file_casetest,
key_file_dbopt, key_file_des_key_file, key_file_ERRMSG, key_select_to_file,
@@ -447,8 +447,6 @@ extern PSI_stage_info stage_waiting_for_
extern PSI_stage_info stage_master_gtid_wait_primary;
extern PSI_stage_info stage_master_gtid_wait;
extern PSI_stage_info stage_gtid_wait_other_connection;
-extern PSI_stage_info stage_slave_background_process_request;
-extern PSI_stage_info stage_slave_background_wait_request;
#ifdef HAVE_PSI_STATEMENT_INTERFACE
/**
@@ -512,7 +510,7 @@ extern mysql_mutex_t
LOCK_slave_list, LOCK_active_mi, LOCK_manager,
LOCK_global_system_variables, LOCK_user_conn,
LOCK_prepared_stmt_count, LOCK_error_messages, LOCK_connection_count,
- LOCK_slave_background;
+ LOCK_slave_init;
extern MYSQL_PLUGIN_IMPORT mysql_mutex_t LOCK_thread_count;
#ifdef HAVE_OPENSSL
extern mysql_mutex_t LOCK_des_key_file;
@@ -523,7 +521,7 @@ extern mysql_rwlock_t LOCK_grant, LOCK_s
extern mysql_rwlock_t LOCK_system_variables_hash;
extern mysql_cond_t COND_thread_count;
extern mysql_cond_t COND_manager;
-extern mysql_cond_t COND_slave_background;
+extern mysql_cond_t COND_slave_init;
extern int32 thread_running;
extern int32 thread_count;
extern my_atomic_rwlock_t thread_running_lock, thread_count_lock;
=== modified file 'sql/rpl_parallel.cc'
--- sql/rpl_parallel.cc 2014-06-10 08:13:15 +0000
+++ sql/rpl_parallel.cc 2014-07-08 10:54:47 +0000
@@ -240,7 +240,6 @@ convert_kill_to_deadlock_error(rpl_group
rgi->killed_for_retry)
{
thd->clear_error();
- thd->get_stmt_da()->reset_diagnostics_area();
my_error(ER_LOCK_DEADLOCK, MYF(0));
rgi->killed_for_retry= false;
thd->reset_killed();
@@ -325,7 +324,7 @@ retry_event_group(rpl_group_info *rgi, r
register_wait_for_prior_event_group_commit(rgi, entry);
mysql_mutex_unlock(&entry->LOCK_parallel_entry);
- strcpy(log_name, ir->name);
+ strmake_buf(log_name, ir->name);
if ((fd= open_binlog(&rlog, log_name, &errmsg)) <0)
{
err= 1;
=== modified file 'sql/rpl_rli.cc'
--- sql/rpl_rli.cc 2014-06-10 08:13:15 +0000
+++ sql/rpl_rli.cc 2014-07-08 10:54:47 +0000
@@ -1302,7 +1302,7 @@ Relay_log_info::alloc_inuse_relaylog(con
my_error(ER_OUTOFMEMORY, MYF(0), (int)sizeof(*ir));
return 1;
}
- strcpy(ir->name, name);
+ strmake_buf(ir->name, name);
if (!inuse_relaylog_list)
inuse_relaylog_list= ir;
@@ -1504,6 +1504,7 @@ rpl_group_info::reinit(Relay_log_info *r
trans_retries= 0;
last_event_start_time= 0;
gtid_sub_id= 0;
+ commit_id= 0;
gtid_pending= false;
worker_error= 0;
row_stmt_start_timestamp= 0;
@@ -1548,6 +1549,7 @@ event_group_new_gtid(rpl_group_info *rgi
rgi->current_gtid.server_id= gev->server_id;
rgi->current_gtid.domain_id= gev->domain_id;
rgi->current_gtid.seq_no= gev->seq_no;
+ rgi->commit_id= gev->commit_id;
rgi->gtid_pending= true;
return 0;
}
=== modified file 'sql/rpl_rli.h'
--- sql/rpl_rli.h 2014-06-10 08:13:15 +0000
+++ sql/rpl_rli.h 2014-07-08 10:54:47 +0000
@@ -164,6 +164,7 @@ class Relay_log_info : public Slave_repo
*/
inuse_relaylog *inuse_relaylog_list;
inuse_relaylog *last_inuse_relaylog;
+ /* Lock used to protect inuse_relaylog::dequeued_count */
my_atomic_rwlock_t inuse_relaylog_atomic_lock;
/*
@@ -526,6 +527,7 @@ struct rpl_group_info
*/
uint64 gtid_sub_id;
rpl_gtid current_gtid;
+ uint64 commit_id;
/*
This is used to keep transaction commit order.
We will signal this when we commit, and can register it to wait for the
=== modified file 'sql/slave.cc'
--- sql/slave.cc 2014-06-10 08:13:15 +0000
+++ sql/slave.cc 2014-07-08 10:54:47 +0000
@@ -287,22 +287,13 @@ static void init_slave_psi_keys(void)
#endif /* HAVE_PSI_INTERFACE */
-static bool slave_background_thread_running;
-static bool slave_background_thread_stop;
-static bool slave_background_thread_gtid_loaded;
-
-struct slave_background_kill_t {
- slave_background_kill_t *next;
- THD *to_kill;
-} *slave_background_kill_list;
+static bool slave_init_thread_running;
pthread_handler_t
-handle_slave_background(void *arg __attribute__((unused)))
+handle_slave_init(void *arg __attribute__((unused)))
{
THD *thd;
- PSI_stage_info old_stage;
- bool stop;
my_thread_init();
thd= new THD;
@@ -310,7 +301,7 @@ handle_slave_background(void *arg __attr
mysql_mutex_lock(&LOCK_thread_count);
thd->thread_id= thread_id++;
mysql_mutex_unlock(&LOCK_thread_count);
- thd->system_thread = SYSTEM_THREAD_SLAVE_BACKGROUND;
+ thd->system_thread = SYSTEM_THREAD_SLAVE_INIT;
thd->store_globals();
thd->security_ctx->skip_grants();
thd->set_command(COM_DAEMON);
@@ -323,126 +314,49 @@ handle_slave_background(void *arg __attr
thd->get_stmt_da()->sql_errno(),
thd->get_stmt_da()->message());
- mysql_mutex_lock(&LOCK_slave_background);
- slave_background_thread_gtid_loaded= true;
- mysql_cond_broadcast(&COND_slave_background);
-
- THD_STAGE_INFO(thd, stage_slave_background_process_request);
- do
- {
- slave_background_kill_t *kill_list;
-
- thd->ENTER_COND(&COND_slave_background, &LOCK_slave_background,
- &stage_slave_background_wait_request,
- &old_stage);
- for (;;)
- {
- stop= abort_loop || thd->killed || slave_background_thread_stop;
- kill_list= slave_background_kill_list;
- if (stop || kill_list)
- break;
- mysql_cond_wait(&COND_slave_background, &LOCK_slave_background);
- }
-
- slave_background_kill_list= NULL;
- thd->EXIT_COND(&old_stage);
-
- while (kill_list)
- {
- slave_background_kill_t *p = kill_list;
- kill_list= p->next;
-
- mysql_mutex_lock(&p->to_kill->LOCK_thd_data);
- p->to_kill->awake(KILL_CONNECTION);
- mysql_mutex_unlock(&p->to_kill->LOCK_thd_data);
- my_free(p);
- }
- mysql_mutex_lock(&LOCK_slave_background);
- } while (!stop);
-
- slave_background_thread_running= false;
- mysql_cond_broadcast(&COND_slave_background);
- mysql_mutex_unlock(&LOCK_slave_background);
-
mysql_mutex_lock(&LOCK_thread_count);
delete thd;
mysql_mutex_unlock(&LOCK_thread_count);
my_thread_end();
- return 0;
-}
-
+ mysql_mutex_lock(&LOCK_slave_init);
+ slave_init_thread_running= false;
+ mysql_cond_broadcast(&COND_slave_init);
+ mysql_mutex_unlock(&LOCK_slave_init);
-void
-slave_background_kill_request(THD *to_kill)
-{
- slave_background_kill_t *p=
- (slave_background_kill_t *)my_malloc(sizeof(*p), MYF(MY_WME));
- if (p)
- {
- p->to_kill= to_kill;
- to_kill->rgi_slave->killed_for_retry= true;
- mysql_mutex_lock(&LOCK_slave_background);
- p->next= slave_background_kill_list;
- slave_background_kill_list= p;
- mysql_mutex_unlock(&LOCK_slave_background);
- mysql_cond_signal(&COND_slave_background);
- }
+ return 0;
}
/*
- Start the slave background thread.
-
- This thread is currently used for two purposes:
+ Start the slave init thread.
- 1. To load the GTID state from mysql.gtid_slave_pos at server start; reading
- from table requires valid THD, which is otherwise not available during
- server init.
-
- 2. To kill worker thread transactions during parallel replication, when a
- storage engine attempts to take an errorneous conflicting lock that would
- cause a deadlock. Killing is done asynchroneously, as the kill may not
- be safe within the context of a callback from inside storage engine
- locking code.
+ This thread is used to load the GTID state from mysql.gtid_slave_pos at
+ server start; reading from table requires valid THD, which is otherwise not
+ available during server init.
*/
static int
-start_slave_background_thread()
+run_slave_init_thread()
{
pthread_t th;
- slave_background_thread_running= true;
- slave_background_thread_stop= false;
- slave_background_thread_gtid_loaded= false;
- if (mysql_thread_create(key_thread_slave_background,
- &th, &connection_attrib, handle_slave_background,
- NULL))
+ slave_init_thread_running= true;
+ if (mysql_thread_create(key_thread_slave_init, &th, &connection_attrib,
+ handle_slave_init, NULL))
{
sql_print_error("Failed to create thread while initialising slave");
return 1;
}
- mysql_mutex_lock(&LOCK_slave_background);
- while (!slave_background_thread_gtid_loaded)
- mysql_cond_wait(&COND_slave_background, &LOCK_slave_background);
- mysql_mutex_unlock(&LOCK_slave_background);
+ mysql_mutex_lock(&LOCK_slave_init);
+ while (!slave_init_thread_running)
+ mysql_cond_wait(&COND_slave_init, &LOCK_slave_init);
+ mysql_mutex_unlock(&LOCK_slave_init);
return 0;
}
-static void
-stop_slave_background_thread()
-{
- mysql_mutex_lock(&LOCK_slave_background);
- slave_background_thread_stop= true;
- mysql_cond_broadcast(&COND_slave_background);
- while (slave_background_thread_running)
- mysql_cond_wait(&COND_slave_background, &LOCK_slave_background);
- mysql_mutex_unlock(&LOCK_slave_background);
-}
-
-
/* Initialize slave structures */
int init_slave()
@@ -454,7 +368,7 @@ int init_slave()
init_slave_psi_keys();
#endif
- if (start_slave_background_thread())
+ if (run_slave_init_thread())
return 1;
/*
@@ -1084,9 +998,6 @@ void end_slave()
master_info_index= 0;
active_mi= 0;
mysql_mutex_unlock(&LOCK_active_mi);
-
- stop_slave_background_thread();
-
global_rpl_thread_pool.destroy();
free_all_rpl_filters();
DBUG_VOID_RETURN;
=== modified file 'sql/slave.h'
--- sql/slave.h 2014-06-10 08:13:15 +0000
+++ sql/slave.h 2014-07-08 10:54:47 +0000
@@ -238,7 +238,6 @@ pthread_handler_t handle_slave_io(void *
void slave_output_error_info(Relay_log_info *rli, THD *thd);
pthread_handler_t handle_slave_sql(void *arg);
bool net_request_file(NET* net, const char* fname);
-void slave_background_kill_request(THD *to_kill);
extern bool volatile abort_loop;
extern Master_info main_mi, *active_mi; /* active_mi for multi-master */
=== modified file 'sql/sql_class.cc'
--- sql/sql_class.cc 2014-07-04 05:44:55 +0000
+++ sql/sql_class.cc 2014-07-08 10:54:47 +0000
@@ -4211,6 +4211,24 @@ extern "C" int thd_slave_thread(const MY
return(thd->slave_thread);
}
+/* Returns true for a worker thread in parallel replication. */
+extern "C" int thd_rpl_is_parallel(const MYSQL_THD thd)
+{
+ return thd->rgi_slave && thd->rgi_slave->is_parallel_exec;
+}
+
+/*
+ This function can optionally be called to check if thd_report_wait_for()
+ needs to be called for waits done by a given transaction.
+
+ If this function returns false for a given thd, there is no need to do any
+ calls to thd_report_wait_for() on that thd.
+
+ This call is optional; it is safe to call thd_report_wait_for() in any case.
+ This call can be used to save some redundant calls to thd_report_wait_for()
+ if desired. (This is unlikely to matter much unless there are _lots_ of
+ waits to report, as the overhead of thd_report_wait_for() is small).
+*/
extern "C" int
thd_need_wait_for(const MYSQL_THD thd)
{
@@ -4224,6 +4242,31 @@ thd_need_wait_for(const MYSQL_THD thd)
return rgi->is_parallel_exec;
}
+/*
+ Used by InnoDB/XtraDB to report that one transaction THD is about to go to
+ wait for a transactional lock held by another transactions OTHER_THD.
+
+ This is used for parallel replication, where transactions are required to
+ commit in the same order on the slave as they did on the master. If the
+ transactions on the slave encounters lock conflicts on the slave that did
+ not exist on the master, this can cause deadlocks.
+
+ Normally, such conflicts will not occur, because the same conflict would
+ have prevented the two transactions from committing in parallel on the
+ master, thus preventing them from running in parallel on the slave in the
+ first place. However, it is possible in case when the optimizer chooses a
+ different plan on the slave than on the master (eg. table scan instead of
+ index scan).
+
+ InnoDB/XtraDB reports lock waits using this call. If a lock wait causes a
+ deadlock with the pre-determined commit order, we kill the later transaction,
+ and later re-try it, to resolve the deadlock.
+
+ This call need only receive reports about waits for locks that will remain
+ until the holding transaction commits. InnoDB/XtraDB auto-increment locks
+ are released earlier, and so need not be reported. (Such false positives are
+ not harmful, but could lead to unnecessary kill and retry, so best avoided).
+*/
extern "C" void
thd_report_wait_for(const MYSQL_THD thd, MYSQL_THD other_thd)
{
@@ -4254,12 +4297,51 @@ thd_report_wait_for(const MYSQL_THD thd,
cause replication to rollback (and later re-try) the other transaction,
releasing the lock for this transaction so replication can proceed.
*/
-
-#ifdef HAVE_REPLICATION
- slave_background_kill_request(other_thd);
-#endif
+ other_rgi->killed_for_retry= true;
+ mysql_mutex_lock(&other_thd->LOCK_thd_data);
+ other_thd->awake(KILL_CONNECTION);
+ mysql_mutex_unlock(&other_thd->LOCK_thd_data);
}
+/*
+ This function is called from InnoDB/XtraDB to check if the commit order of
+ two transactions has already been decided by the upper layer. This happens
+ in parallel replication, where the commit order is forced to be the same on
+ the slave as it was originally on the master.
+
+ If this function returns false, it means that such commit order will be
+ enforced. This allows the storage engine to optionally omit gap lock waits
+ or similar measures that would otherwise be needed to ensure that
+ transactions would be serialised in a way that would cause a commit order
+ that is correct for binlogging for statement-based replication.
+
+ Since transactions are only run in parallel on the slave if they ran without
+ lock conflicts on the master, normally no lock conflicts on the slave happen
+ during parallel replication. However, there are a couple of corner cases
+ where it can happen, like these secondary-index operations:
+
+ T1: INSERT INTO t1 VALUES (7, NULL);
+ T2: DELETE FROM t1 WHERE b <= 3;
+
+ T1: UPDATE t1 SET secondary=NULL WHERE primary=1
+ T2: DELETE t1 WHERE secondary <= 3
+
+ The DELETE takes a gap lock that can block the INSERT/UPDATE, but the row
+ locks set by INSERT/UPDATE do not block the DELETE. Thus, the execution
+ order of the transactions determine whether a lock conflict occurs or
+ not. Thus a lock conflict can occur on the slave where it did not on the
+ master.
+
+ If this function returns true, normal locking should be done as required by
+ the binlogging and transaction isolation level in effect. But if it returns
+ false, the correct order will be enforced anyway, and InnoDB/XtraDB can
+ avoid taking the gap lock, preventing the lock conflict.
+
+ Calling this function is just an optimisation to avoid unnecessary
+ deadlocks. If it was not used, a gap lock would be set that could eventually
+ cause a deadlock; the deadlock would be caught by thd_report_wait_for() and
+ the transaction T2 killed and rolled back (and later re-tried).
+*/
extern "C" int
thd_need_ordering_with(const MYSQL_THD thd, const MYSQL_THD other_thd)
{
@@ -4277,7 +4359,7 @@ thd_need_ordering_with(const MYSQL_THD t
return 1;
if (rgi->current_gtid.domain_id != other_rgi->current_gtid.domain_id)
return 1;
- if (rgi->commit_id != other_rgi->commit_id)
+ if (!rgi->commit_id || rgi->commit_id != other_rgi->commit_id)
return 1;
/*
These two threads are doing parallel replication within the same
@@ -4289,6 +4371,26 @@ thd_need_ordering_with(const MYSQL_THD t
}
+/*
+ If the storage engine detects a deadlock, and needs to choose a victim
+ transaction to roll back, it can call this function to ask the upper
+ server layer for which of two possible transactions is prefered to be
+ aborted and rolled back.
+
+ In parallel replication, if two transactions are running in parallel and
+ one is fixed to commit before the other, then the one that commits later
+ will be prefered as the victim - chosing the early transaction as a victim
+ will not resolve the deadlock anyway, as the later transaction still needs
+ to wait for the earlier to commit.
+
+ Otherwise, a transaction that uses only transactional tables, and can thus
+ be safely rolled back, will be prefered as a deadlock victim over a
+ transaction that also modified non-transactional (eg. MyISAM) tables.
+
+ The return value is -1 if the first transaction is prefered as a deadlock
+ victim, 1 if the second transaction is prefered, or 0 for no preference (in
+ which case the storage engine can make the choice as it prefers).
+*/
extern "C" int
thd_deadlock_victim_preference(const MYSQL_THD thd1, const MYSQL_THD thd2)
{
=== modified file 'sql/sql_class.h'
--- sql/sql_class.h 2014-06-10 08:13:15 +0000
+++ sql/sql_class.h 2014-07-08 10:54:47 +0000
@@ -1358,7 +1358,7 @@ enum enum_thread_type
SYSTEM_THREAD_EVENT_SCHEDULER= 16,
SYSTEM_THREAD_EVENT_WORKER= 32,
SYSTEM_THREAD_BINLOG_BACKGROUND= 64,
- SYSTEM_THREAD_SLAVE_BACKGROUND= 128,
+ SYSTEM_THREAD_SLAVE_INIT= 128,
};
inline char const *
=== modified file 'storage/heap/hp_write.c'
--- storage/heap/hp_write.c 2014-06-03 08:31:11 +0000
+++ storage/heap/hp_write.c 2014-07-08 10:54:47 +0000
@@ -156,7 +156,7 @@ static uchar *next_free_record_pos(HP_SH
("record file full. records: %lu max_records: %lu "
"data_length: %llu index_length: %llu "
"max_table_size: %llu",
- (unsigned long)info->records, info->max_records,
+ info->records, info->max_records,
info->data_length, info->index_length,
info->max_table_size));
my_errno=HA_ERR_RECORD_FILE_FULL;
=== modified file 'storage/innobase/handler/ha_innodb.cc'
--- storage/innobase/handler/ha_innodb.cc 2014-06-10 08:13:15 +0000
+++ storage/innobase/handler/ha_innodb.cc 2014-07-08 10:54:47 +0000
@@ -4169,13 +4169,18 @@ innobase_kill_query(
if (trx)
{
+ THD *cur = current_thd;
+ THD *owner = trx->current_lock_mutex_owner;
+
/* Cancel a pending lock request. */
- lock_mutex_enter();
+ if (owner != cur)
+ lock_mutex_enter();
trx_mutex_enter(trx);
if (trx->lock.wait_lock)
lock_cancel_waiting_and_release(trx->lock.wait_lock);
trx_mutex_exit(trx);
- lock_mutex_exit();
+ if (owner != cur)
+ lock_mutex_exit();
}
DBUG_VOID_RETURN;
=== modified file 'storage/innobase/include/trx0trx.h'
--- storage/innobase/include/trx0trx.h 2014-03-26 21:25:38 +0000
+++ storage/innobase/include/trx0trx.h 2014-07-08 10:54:47 +0000
@@ -993,6 +993,11 @@ struct trx_t{
count of tables being flushed. */
/*------------------------------*/
+ THD* current_lock_mutex_owner;
+ /*!< If this is equal to current_thd,
+ then in innobase_kill_query() we know we
+ already hold the lock_sys->mutex. */
+ /*------------------------------*/
#ifdef UNIV_DEBUG
ulint start_line; /*!< Track where it was started from */
const char* start_file; /*!< Filename where it was started */
=== modified file 'storage/innobase/lock/lock0lock.cc'
--- storage/innobase/lock/lock0lock.cc 2014-06-10 08:13:15 +0000
+++ storage/innobase/lock/lock0lock.cc 2014-07-08 10:54:47 +0000
@@ -374,6 +374,11 @@ struct lock_stack_t {
ulint heap_no; /*!< heap number if rec lock */
};
+extern "C" void thd_report_wait_for(const MYSQL_THD thd, MYSQL_THD other_thd);
+extern "C" int thd_need_wait_for(const MYSQL_THD thd);
+extern "C"
+int thd_need_ordering_with(const MYSQL_THD thd, const MYSQL_THD other_thd);
+
/** Stack to use during DFS search. Currently only a single stack is required
because there is no parallel deadlock check. This stack is protected by
the lock_sys_t::mutex. */
@@ -392,6 +397,14 @@ UNIV_INTERN mysql_pfs_key_t lock_sys_wai
#ifdef UNIV_DEBUG
UNIV_INTERN ibool lock_print_waits = FALSE;
+/* Buffer to collect THDs to report waits for. */
+struct thd_wait_reports {
+ struct thd_wait_reports *next;
+ ulint used;
+ trx_t *waitees[64];
+};
+
+
/*********************************************************************//**
Validates the lock system.
@return TRUE if ok */
@@ -1037,8 +1050,12 @@ lock_rec_has_to_wait(
the correct order so that statement-based replication
will give the correct results. Since the right order
was already determined on the master, we do not need
- to enforce it again here (and doing so could lead to
- occasional deadlocks). */
+ to enforce it again here.
+
+ Skipping the locks is not essential for correctness,
+ since in case of deadlock we will just kill the later
+ transaction and retry it. But it can save some
+ unnecessary rollbacks and retries. */
return (FALSE);
}
@@ -3814,7 +3831,8 @@ static
trx_id_t
lock_deadlock_search(
/*=================*/
- lock_deadlock_ctx_t* ctx) /*!< in/out: deadlock context */
+ lock_deadlock_ctx_t* ctx, /*!< in/out: deadlock context */
+ struct thd_wait_reports*waitee_ptr) /*!< in/out: list of waitees */
{
const lock_t* lock;
ulint heap_no;
@@ -3893,10 +3911,24 @@ lock_deadlock_search(
/* We do not need to report autoinc locks to the upper
layer. These locks are released before commit, so they can
not cause deadlocks with binlog-fixed commit order. */
- if (lock_get_type_low(lock) != LOCK_TABLE ||
- lock_get_mode(lock) != LOCK_AUTO_INC)
- thd_report_wait_for(ctx->start->mysql_thd,
- lock->trx->mysql_thd);
+ if (waitee_ptr && (lock_get_type_low(lock) != LOCK_TABLE ||
+ lock_get_mode(lock) != LOCK_AUTO_INC)) {
+ if (waitee_ptr->used == sizeof(waitee_ptr->waitees)/
+ sizeof(waitee_ptr->waitees[0])) {
+ waitee_ptr->next =
+ (struct thd_wait_reports *)
+ mem_alloc(sizeof(*waitee_ptr));
+ waitee_ptr = waitee_ptr->next;
+ if (!waitee_ptr) {
+ ctx->too_deep = TRUE;
+ return(ctx->start->id);
+ }
+ waitee_ptr->next = NULL;
+ waitee_ptr->used = 0;
+ }
+ waitee_ptr->waitees[waitee_ptr->used++] = lock->trx;
+ }
+
if (lock->trx->lock.que_state == TRX_QUE_LOCK_WAIT) {
/* Another trx ahead has requested a lock in an
@@ -3989,6 +4021,41 @@ lock_deadlock_trx_rollback(
trx_mutex_exit(trx);
}
+static void
+mysql_report_waiters(struct thd_wait_reports *waitee_buf_ptr,
+ THD *mysql_thd,
+ trx_id_t victim_trx_id)
+{
+ struct thd_wait_reports *p = waitee_buf_ptr;
+ while (p) {
+ struct thd_wait_reports *q;
+ ulint i = 0;
+
+ while (i < p->used) {
+ trx_t *w_trx = p->waitees[i];
+ /* There is no need to report waits to a trx already
+ selected as a victim. */
+ if (w_trx->id != victim_trx_id)
+ {
+ /* If thd_report_wait_for() decides to kill the
+ transaction, then we will get a call back into
+ innobase_kill_query. We mark this by setting
+ current_lock_mutex_owner, so we can avoid trying
+ to recursively take lock_sys->mutex. */
+ w_trx->current_lock_mutex_owner = mysql_thd;
+ thd_report_wait_for(mysql_thd, w_trx->mysql_thd);
+ w_trx->current_lock_mutex_owner = NULL;
+ }
+ ++i;
+ }
+ q = p->next;
+ if (p != waitee_buf_ptr)
+ mem_free(q);
+ p = q;
+ }
+}
+
+
/********************************************************************//**
Checks if a joining lock request results in a deadlock. If a deadlock is
found this function will resolve the dadlock by choosing a victim transaction
@@ -4005,12 +4072,20 @@ lock_deadlock_check_and_resolve(
const trx_t* trx) /*!< in: transaction */
{
trx_id_t victim_trx_id;
+ struct thd_wait_reports waitee_buf, *waitee_buf_ptr;
+ THD* start_mysql_thd;
ut_ad(trx != NULL);
ut_ad(lock != NULL);
ut_ad(lock_mutex_own());
assert_trx_in_list(trx);
+ start_mysql_thd = trx->mysql_thd;
+ if (start_mysql_thd && thd_need_wait_for(start_mysql_thd))
+ waitee_buf_ptr = &waitee_buf;
+ else
+ waitee_buf_ptr = NULL;
+
/* Try and resolve as many deadlocks as possible. */
do {
lock_deadlock_ctx_t ctx;
@@ -4023,7 +4098,17 @@ lock_deadlock_check_and_resolve(
ctx.wait_lock = lock;
ctx.mark_start = lock_mark_counter;
- victim_trx_id = lock_deadlock_search(&ctx);
+ if (waitee_buf_ptr) {
+ waitee_buf_ptr->next = NULL;
+ waitee_buf_ptr->used = 0;
+ }
+
+ victim_trx_id = lock_deadlock_search(&ctx, waitee_buf_ptr);
+
+ /* Report waits to upper layer, as needed. */
+ if (waitee_buf_ptr)
+ mysql_report_waiters(waitee_buf_ptr, start_mysql_thd,
+ victim_trx_id);
/* Search too deep, we rollback the joining transaction. */
if (ctx.too_deep) {
=== modified file 'storage/innobase/trx/trx0trx.cc'
--- storage/innobase/trx/trx0trx.cc 2014-06-10 08:13:15 +0000
+++ storage/innobase/trx/trx0trx.cc 2014-07-08 10:54:47 +0000
@@ -50,6 +50,9 @@ Created 3/26/1996 Heikki Tuuri
#include<set>
+extern "C"
+int thd_deadlock_victim_preference(const MYSQL_THD thd1, const MYSQL_THD thd2);
+
/** Set of table_id */
typedef std::set<table_id_t> table_id_set;
=== modified file 'storage/xtradb/handler/ha_innodb.cc'
--- storage/xtradb/handler/ha_innodb.cc 2014-06-10 08:13:15 +0000
+++ storage/xtradb/handler/ha_innodb.cc 2014-07-08 10:54:47 +0000
@@ -4650,12 +4650,15 @@ innobase_kill_connection(
DBUG_ENTER("innobase_kill_connection");
DBUG_ASSERT(hton == innodb_hton_ptr);
- lock_mutex_enter();
-
trx = thd_to_trx(thd);
if (trx)
{
+ THD *cur = current_thd;
+ THD *owner = trx->current_lock_mutex_owner;
+
+ if (owner != cur)
+ lock_mutex_enter();
trx_mutex_enter(trx);
/* Cancel a pending lock request. */
@@ -4663,10 +4666,10 @@ innobase_kill_connection(
lock_cancel_waiting_and_release(trx->lock.wait_lock);
trx_mutex_exit(trx);
+ if (owner != cur)
+ lock_mutex_exit();
}
- lock_mutex_exit();
-
DBUG_VOID_RETURN;
}
=== modified file 'storage/xtradb/include/trx0trx.h'
--- storage/xtradb/include/trx0trx.h 2014-02-26 18:21:23 +0000
+++ storage/xtradb/include/trx0trx.h 2014-07-08 10:54:47 +0000
@@ -1020,6 +1020,11 @@ struct trx_t{
count of tables being flushed. */
/*------------------------------*/
+ THD* current_lock_mutex_owner;
+ /*!< If this is equal to current_thd,
+ then in innobase_kill_query() we know we
+ already hold the lock_sys->mutex. */
+ /*------------------------------*/
#ifdef UNIV_DEBUG
ulint start_line; /*!< Track where it was started from */
const char* start_file; /*!< Filename where it was started */
=== modified file 'storage/xtradb/lock/lock0lock.cc'
--- storage/xtradb/lock/lock0lock.cc 2014-06-10 08:13:15 +0000
+++ storage/xtradb/lock/lock0lock.cc 2014-07-08 10:54:47 +0000
@@ -374,6 +374,11 @@ struct lock_stack_t {
ulint heap_no; /*!< heap number if rec lock */
};
+extern "C" void thd_report_wait_for(const MYSQL_THD thd, MYSQL_THD other_thd);
+extern "C" int thd_need_wait_for(const MYSQL_THD thd);
+extern "C"
+int thd_need_ordering_with(const MYSQL_THD thd, const MYSQL_THD other_thd);
+
/** Stack to use during DFS search. Currently only a single stack is required
because there is no parallel deadlock check. This stack is protected by
the lock_sys_t::mutex. */
@@ -392,6 +397,14 @@ UNIV_INTERN mysql_pfs_key_t lock_sys_wai
#ifdef UNIV_DEBUG
UNIV_INTERN ibool lock_print_waits = FALSE;
+/* Buffer to collect THDs to report waits for. */
+struct thd_wait_reports {
+ struct thd_wait_reports *next;
+ ulint used;
+ trx_t *waitees[64];
+};
+
+
/*********************************************************************//**
Validates the lock system.
@return TRUE if ok */
@@ -1038,8 +1051,12 @@ lock_rec_has_to_wait(
the correct order so that statement-based replication
will give the correct results. Since the right order
was already determined on the master, we do not need
- to enforce it again here (and doing so could lead to
- occasional deadlocks). */
+ to enforce it again here.
+
+ Skipping the locks is not essential for correctness,
+ since in case of deadlock we will just kill the later
+ transaction and retry it. But it can save some
+ unnecessary rollbacks and retries. */
return (FALSE);
}
@@ -3827,7 +3844,8 @@ static
trx_id_t
lock_deadlock_search(
/*=================*/
- lock_deadlock_ctx_t* ctx) /*!< in/out: deadlock context */
+ lock_deadlock_ctx_t* ctx, /*!< in/out: deadlock context */
+ struct thd_wait_reports*waitee_ptr) /*!< in/out: list of waitees */
{
const lock_t* lock;
ulint heap_no;
@@ -3906,10 +3924,24 @@ lock_deadlock_search(
/* We do not need to report autoinc locks to the upper
layer. These locks are released before commit, so they can
not cause deadlocks with binlog-fixed commit order. */
- if (lock_get_type_low(lock) != LOCK_TABLE ||
- lock_get_mode(lock) != LOCK_AUTO_INC)
- thd_report_wait_for(ctx->start->mysql_thd,
- lock->trx->mysql_thd);
+ if (waitee_ptr && (lock_get_type_low(lock) != LOCK_TABLE ||
+ lock_get_mode(lock) != LOCK_AUTO_INC)) {
+ if (waitee_ptr->used == sizeof(waitee_ptr->waitees)/
+ sizeof(waitee_ptr->waitees[0])) {
+ waitee_ptr->next =
+ (struct thd_wait_reports *)
+ mem_alloc(sizeof(*waitee_ptr));
+ waitee_ptr = waitee_ptr->next;
+ if (!waitee_ptr) {
+ ctx->too_deep = TRUE;
+ return(ctx->start->id);
+ }
+ waitee_ptr->next = NULL;
+ waitee_ptr->used = 0;
+ }
+ waitee_ptr->waitees[waitee_ptr->used++] = lock->trx;
+ }
+
if (lock->trx->lock.que_state == TRX_QUE_LOCK_WAIT) {
/* Another trx ahead has requested a lock in an
@@ -4002,6 +4034,41 @@ lock_deadlock_trx_rollback(
trx_mutex_exit(trx);
}
+static void
+mysql_report_waiters(struct thd_wait_reports *waitee_buf_ptr,
+ THD *mysql_thd,
+ trx_id_t victim_trx_id)
+{
+ struct thd_wait_reports *p = waitee_buf_ptr;
+ while (p) {
+ struct thd_wait_reports *q;
+ ulint i = 0;
+
+ while (i < p->used) {
+ trx_t *w_trx = p->waitees[i];
+ /* There is no need to report waits to a trx already
+ selected as a victim. */
+ if (w_trx->id != victim_trx_id)
+ {
+ /* If thd_report_wait_for() decides to kill the
+ transaction, then we will get a call back into
+ innobase_kill_query. We mark this by setting
+ current_lock_mutex_owner, so we can avoid trying
+ to recursively take lock_sys->mutex. */
+ w_trx->current_lock_mutex_owner = mysql_thd;
+ thd_report_wait_for(mysql_thd, w_trx->mysql_thd);
+ w_trx->current_lock_mutex_owner = NULL;
+ }
+ ++i;
+ }
+ q = p->next;
+ if (p != waitee_buf_ptr)
+ mem_free(q);
+ p = q;
+ }
+}
+
+
/********************************************************************//**
Checks if a joining lock request results in a deadlock. If a deadlock is
found this function will resolve the dadlock by choosing a victim transaction
@@ -4018,12 +4085,20 @@ lock_deadlock_check_and_resolve(
const trx_t* trx) /*!< in: transaction */
{
trx_id_t victim_trx_id;
+ struct thd_wait_reports waitee_buf, *waitee_buf_ptr;
+ THD* start_mysql_thd;
ut_ad(trx != NULL);
ut_ad(lock != NULL);
ut_ad(lock_mutex_own());
assert_trx_in_list(trx);
+ start_mysql_thd = trx->mysql_thd;
+ if (start_mysql_thd && thd_need_wait_for(start_mysql_thd))
+ waitee_buf_ptr = &waitee_buf;
+ else
+ waitee_buf_ptr = NULL;
+
/* Try and resolve as many deadlocks as possible. */
do {
lock_deadlock_ctx_t ctx;
@@ -4036,7 +4111,17 @@ lock_deadlock_check_and_resolve(
ctx.wait_lock = lock;
ctx.mark_start = lock_mark_counter;
- victim_trx_id = lock_deadlock_search(&ctx);
+ if (waitee_buf_ptr) {
+ waitee_buf_ptr->next = NULL;
+ waitee_buf_ptr->used = 0;
+ }
+
+ victim_trx_id = lock_deadlock_search(&ctx, waitee_buf_ptr);
+
+ /* Report waits to upper layer, as needed. */
+ if (waitee_buf_ptr)
+ mysql_report_waiters(waitee_buf_ptr, start_mysql_thd,
+ victim_trx_id);
/* Search too deep, we rollback the joining transaction. */
if (ctx.too_deep) {
=== modified file 'storage/xtradb/trx/trx0trx.cc'
--- storage/xtradb/trx/trx0trx.cc 2014-06-10 08:13:15 +0000
+++ storage/xtradb/trx/trx0trx.cc 2014-07-08 10:54:47 +0000
@@ -51,6 +51,9 @@ Created 3/26/1996 Heikki Tuuri
#include<set>
+extern "C"
+int thd_deadlock_victim_preference(const MYSQL_THD thd1, const MYSQL_THD thd2);
+
/** Set of table_id */
typedef std::set<table_id_t> table_id_set;
References
