randgen team mailing list archive

[John Embretsen <johnemb@xxxxxxxxx>]

Hi Philip,


Some time ago (November 2011) you added MAX_ROWS_THRESHOLD to the MySQL 
executor in the RQG. With that change, the executor would send "KILL 
query" whenever a result set contained more than MAX_ROWS_THRESHOLD rows.

revid:philips@eve-20111125094721-i0xh1uqkf1w2sjvw
"* Terminate queries that have returned more than 10K rows via an 
off-band KILL QUERY command"

After some subsequent changes to the Transformer validator's handling of 
various statuses, specifically:

revid:philips@eve-20111130081527-uhrfobttsz1zdxx4
"make handling of interrupted queries more robust"

and

revid:philips@fedora13-20111125163020-ypohd2fcs018ac3g
"properly handle STATUS_WONT_HANDLE from Transforms"


I have started seeing some weird behavior. The issue is that a 
combination of the transformers "Distinct" and "ExecuteAsPreparedTwice" 
results in STATUS_ENVIRONMENT_FAILURE. This happens because of the 
following order of events, it seems:


1) Distinct transformer removes DISTINCT from query; transformed query 
returns more than MAX_ROWS_THRESHOLD rows.
2) Executor detects the threshold being hit and sends
KILL QUERY to the server.
3) Executor/transformer continues with next transform:
PREPARE query; EXECUTE prep_stmt; EXECUTE prep_stmt;
4) "PREPARE query" is killed because of the KILL QUERY command
issued earlier.
5) "EXECUTE prep_stmt" fails because there is no prepared
statement, as the PREPARE was killed.
6) Transformer returns STATUS_ENVIRONMENT_FAILURE due to getting 
STATUS_SEMANTIC_ERROR, specifically:
"1243 Unknown prepared statement handler (%s) given to EXECUTE."



It seems to me that the "KILL QUERY" issued by the executor does not 
work as intended. It kills the next query (or statement, to be precise) 
instead of the original query.

I guess this is because the original query has already finished when the 
executor counts the number of rows, and the next query is executed with 
the same connection_id as the first one, thus matching the KILL QUERY 
command sent earlier? Note that this is all with --threads=1.

I have attached some log files where you can see this issue (master.log 
excerpt and part of RQG output). You can reproduce it using the 
following grammar contents (referred to below as envFail.yy):

query:
    SELECT DISTINCT   alias1 . `col_int_key` AS field1 FROM ( C AS 
alias1 , ( C AS alias2 , D AS alias3 ) )    ORDER BY field1, field1  ;


and the following RQG command:

perl runall.pl \
  --queries=1 \
  --seed=1 \
  --threads=1 \
  --duration=60 \
  --transformer=Distinct,ExecuteAsPreparedTwice \
  --grammar=envFail.yy \
  --basedir=$CODE_OPT \
  --vardir=$PWD/var-env-fail

where $CODE is an environment variable pointing to a valid MySQL 
codebase/installation.

Any comments / suggestion for fix?

--
John

# 2012-01-10T10:49:43 Loading Validator GenTest::Validator::Transformer.
# 2012-01-10T10:49:43 Transformer Validator will use the following Transformers: Distinct, ExecuteAsPreparedT
wice
SELECT DISTINCT alias1 . `col_int_key` AS field1 FROM ( C AS alias1 , ( C AS alias2 , D AS alias3 ) ) ORDER B
Y field1, field1;
/* GenTest::Transform::Distinct */ SELECT  alias1 . `col_int_key` AS field1 FROM ( C AS alias1 , ( C AS alias
2 , D AS alias3 ) ) ORDER BY field1, field1 /* TRANSFORM_OUTCOME_SUPERSET */ ;
# 2012-01-10T10:49:44 Query: /* GenTest::Transform::Distinct */ SELECT  alias1 . `col_int_key` AS field1 FROM
 ( C AS alias1 , ( C AS alias2 , D AS alias3 ) ) ORDER BY field1, field1 /* TRANSFORM_OUTCOME_SUPERSET */  re
turned more than MAX_ROWS_THRESHOLD (50000) rows. Killing it ...
/* GenTest::Transform::ExecuteAsPreparedTwice */ PREPARE prep_stmt_20603 FROM 'SELECT DISTINCT alias1 . `col_
int_key` AS field1 FROM ( C AS alias1 , ( C AS alias2 , D AS alias3 ) ) ORDER BY field1, field1';
# 2012-01-10T10:49:44 Query: /* GenTest::Transform::ExecuteAsPreparedTwice */ PREPARE prep_stmt_20603 FROM 'S
ELECT DISTINCT alias1 . `col_int_key` AS field1 FROM ( C AS alias1 , ( C AS alias2 , D AS alias3 ) ) ORDER BY
 field1, field1' failed: 1317 Query execution was interrupted
EXECUTE prep_stmt_20603 /* TRANSFORM_OUTCOME_UNORDERED_MATCH */;
# 2012-01-10T10:49:44 ---------- TRANSFORM ISSUE ----------
# 2012-01-10T10:49:44 Transform GenTest::Transform::ExecuteAsPreparedTwice failed with a syntactic or semanti
c error: 1243 Unknown prepared statement handler (%s) given to EXECUTE; RQG Status: STATUS_SEMANTIC_ERROR (22
)
# 2012-01-10T10:49:44 Offending query is: EXECUTE prep_stmt_20603 /* TRANSFORM_OUTCOME_UNORDERED_MATCH */;
# 2012-01-10T10:49:44 Original query is: SELECT DISTINCT alias1 . `col_int_key` AS field1 FROM ( C AS alias1 
, ( C AS alias2 , D AS alias3 ) ) ORDER BY field1, field1;
# 2012-01-10T10:49:44 ERROR: Possible syntax or semantic error caused by code in transformer GenTest::Transfo
rm::ExecuteAsPreparedTwice. Raising severity to STATUS_ENVIRONMENT_FAILURE.
# 2012-01-10T10:49:44 Query: EXECUTE prep_stmt_20603 /* TRANSFORM_OUTCOME_UNORDERED_MATCH */ failed: 1243 Unknown prepared statement handler (%s) given to EXECUTE. Further errors of this kind will be suppressed.
# 2012-01-10T10:49:49 gentest.pl exited with exit status STATUS_ENVIRONMENT_FAILURE (110)


 11 Query     SHOW FULL PROCESSLIST
 11 Quit      
 10 Query     SELECT DISTINCT alias1 . `col_int_key` AS field1 FROM ( C AS alias1 , ( C AS alias2 , D AS alias3 ) ) ORDER BY field1, field1
 10 Query     CREATE DATABASE IF NOT EXISTS transforms
 10 Query     /* GenTest::Transform::Distinct */ SELECT  alias1 . `col_int_key` AS field1 FROM ( C AS alias1 , ( C AS alias2 , D AS alias3 ) ) ORDER BY field1, field1 /* TRANSFORM_OUTCOME_SUPERSET */
 12 Connect   root@127.0.0.1 on test
 12 Query     SET SESSION autocommit=1
 12 Query     KILL QUERY 10
 12 Quit      
 10 Query     /* GenTest::Transform::ExecuteAsPreparedTwice */ PREPARE prep_stmt_19711 FROM 'SELECT DISTINCT alias1 . `col_int_key` AS field1 FROM ( C AS alias1 , ( C AS alias2 , D AS alias3 ) ) ORDER BY field1, field1'
 10 Query     EXECUTE prep_stmt_19711 /* TRANSFORM_OUTCOME_UNORDERED_MATCH */
 13 Connect   root@127.0.0.1 on test
 13 Query     SET SESSION autocommit=1
 13 Query     SHOW FULL PROCESSLIST
 13 Quit