← Back to team overview

maria-developers team mailing list archive

Updated (by Guest): Make mysqlbinlog not to output unneeded statements when using --database (38)

 

-----------------------------------------------------------------------
                              WORKLOG TASK
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TASK...........: Make mysqlbinlog not to output unneeded statements when using
		--database
CREATION DATE..: Sat, 08 Aug 2009, 12:40
SUPERVISOR.....: Monty
IMPLEMENTOR....: 
COPIES TO......: 
CATEGORY.......: Server-RawIdeaBin
TASK ID........: 38 (http://askmonty.org/worklog/?tid=38)
VERSION........: Benchmarks-3.0
STATUS.........: Un-Assigned
PRIORITY.......: 60
WORKED HOURS...: 0
ESTIMATE.......: 0 (hours remain)
ORIG. ESTIMATE.: 0

PROGRESS NOTES:

-=-=(Guest - Sun, 09 Aug 2009, 12:22)=-=-
High-Level Specification modified.
--- /tmp/wklog.38.old.20756     2009-08-09 12:22:52.000000000 +0300
+++ /tmp/wklog.38.new.20756     2009-08-09 12:22:52.000000000 +0300
@@ -1 +1,18 @@
+Monty's suggestions for fix:
+
+A way to fix this for 'most' cases are:
+
+If we do filtering (like mysqlbinlog --database='xxx') then:
+
+- In mysql_bin_log(), do a flush of the Log_event::cache() between
+  each statement.
+- Log on statement.
+- If the statement was ignored (we need a flag for this) and
+  there is something in the cache and the file position didn't change
+  (the cache didn't overflow), then reset the cache.
+
+Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single
+database is used
+- Could be fixed by having a flag to mark if something was printed
+  to the log since last commit.
 

-=-=(Guest - Sun, 09 Aug 2009, 12:20)=-=-
High Level Description modified.
--- /tmp/wklog.38.old.20618     2009-08-09 12:20:16.000000000 +0300
+++ /tmp/wklog.38.new.20618     2009-08-09 12:20:16.000000000 +0300
@@ -1,10 +1,24 @@
-This comes from MySQL BUG#23890: 
+This comes from MySQL BUG#23890 and BUG#23894: when one runs 
 
-  mysqlbinlog --database=bar N-bin.000003 
+ mysqlbinlog --database=bar binlog_file 
 
-will output all the COMMIT statements in the binary log even if it didn't print
-any statements between the COMMITs (because all the statements that were there
-were for the other databases).
+the produced SQL may contain useless sequences of commands like:
 
-The fix is trivial: in mysqlbinlog, check if we've printed anything after we've
-printed the previous commit statement.
+COMMIT; 
+COMMIT;
+COMMIT;
+...
+
+or 
+
+SET INSERT_ID=val1;
+SET INSERT_ID=val2;
+SET INSERT_ID=val3;
+...
+
+This happens because the statements between COMMIT (or SET) statements had no
+effect on the specified database and so were filtered out.  COMMIT and SET
+statements themselves are not associated with any database and were left in.
+
+Presence of redundant COMMIT or SET statements makes binlog SQL script
+unnecessarily big and it will take more client<->server roundtrips to apply it.

-=-=(Guest - Sun, 09 Aug 2009, 12:19)=-=-
Title modified.
--- /tmp/wklog.38.old.20544     2009-08-09 12:19:22.000000000 +0300
+++ /tmp/wklog.38.new.20544     2009-08-09 12:19:22.000000000 +0300
@@ -1 +1 @@
-Make mysqlbinlog not to output unneeded COMMIT statements
+Make mysqlbinlog not to output unneeded statements when using --database



DESCRIPTION:

This comes from MySQL BUG#23890 and BUG#23894: when one runs 

 mysqlbinlog --database=bar binlog_file 

the produced SQL may contain useless sequences of commands like:

COMMIT; 
COMMIT;
COMMIT;
...

or 

SET INSERT_ID=val1;
SET INSERT_ID=val2;
SET INSERT_ID=val3;
...

This happens because the statements between COMMIT (or SET) statements had no
effect on the specified database and so were filtered out.  COMMIT and SET
statements themselves are not associated with any database and were left in.

Presence of redundant COMMIT or SET statements makes binlog SQL script
unnecessarily big and it will take more client<->server roundtrips to apply it.


HIGH-LEVEL SPECIFICATION:



Monty's suggestions for fix:

A way to fix this for 'most' cases are:

If we do filtering (like mysqlbinlog --database='xxx') then:

- In mysql_bin_log(), do a flush of the Log_event::cache() between
  each statement.
- Log on statement.
- If the statement was ignored (we need a flag for this) and
  there is something in the cache and the file position didn't change
  (the cache didn't overflow), then reset the cache.

Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single
database is used
- Could be fixed by having a flag to mark if something was printed
  to the log since last commit.


ESTIMATED WORK TIME

ESTIMATED COMPLETION DATE
-----------------------------------------------------------------------
WorkLog (v3.5.9)