maria-developers team mailing list archive

[worklog-noreply@xxxxxxxxxxxx]

-----------------------------------------------------------------------
                              WORKLOG TASK
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TASK...........: MRR backport
CREATION DATE..: Thu, 26 Nov 2009, 15:19
SUPERVISOR.....: Monty
IMPLEMENTOR....: Psergey
COPIES TO......: 
CATEGORY.......: Server-Sprint
TASK ID........: 67 (http://askmonty.org/worklog/?tid=67)
VERSION........: Server-9.x
STATUS.........: Un-Assigned
PRIORITY.......: 60
WORKED HOURS...: 0
ESTIMATE.......: 0 (hours remain)
ORIG. ESTIMATE.: 0

PROGRESS NOTES:

-=-=(Guest - Thu, 26 Nov 2009, 16:52)=-=-
High-Level Specification modified.
--- /tmp/wklog.67.old.694       2009-11-26 14:52:53.000000000 +0000
+++ /tmp/wklog.67.new.694       2009-11-26 14:52:53.000000000 +0000
@@ -1 +1,66 @@
+1. Requirements
+===============
+
+We need the following:
+
+1. Latest MRR interface support, including extensions to support ICP when 
+   using BKA.
+2. Let DS-MRR support clustered primary keys (needed when using BKA).
+3. Remove conditions used for key access from the condition pushed to index
+   (ATM this manifests itself as "Using index condition" appearing where there 
+    was no "Using where". TODO: example of this?)
+4. Introduce a separate @@optimizer_switch flag for turning on/out ICP (atm it 
+   is switched on/off by @@engine_condition_pushdown)
+5. Introduce a separate @@mrr_buffer_size variable to control MRR buffer size
+   for range+MRR scans. ATM it is controlled by @@read_rnd_size flag and that
+   makes it unobvious for a number of users.
+6. Rename multi_range_read_info_const() to look like it is not a part of MRR
+   interface.
+8. Try to make MRR to be more of a module
+7. Improve MRR's cost model.
+
+2. Required actions 
+===================
+
+Roughly in the order in which it will be done:
+
+2.1 Fix DS-MRR/InnoDB bugs
+--------------------------
+We need to fix the bugs listed here:
+
+http://bugs.mysql.com/search.php?cmd=display&status=Active&tags=index_condition_pushdown
+http://bugs.mysql.com/search.php?cmd=display&status=Active&tags=mrr
+
+2.2 Backport DS-MRR code to MariaDB 5.2
+---------------------------------------
+The easiest way seems to be to to manually move the needed code from mysql-6.0
+(or whatever it's called now) to MariaDB.
+
+2.3 Introduce control variables 
+-------------------------------
+Act on items #4 and #5 from the requirements. Should be easy as 
+@@optimizer_switch is supported in 5.1 codebase.
+
+2.4 Other backport issues 
+-------------------------
+* Figure out what to do with NDB/MRR.  5.1 codebase has "old" NDB/MRR
+  implementation. mysql-6.0 (and NDB's branch) have the updated NDB/MRR 
+  but merging it into 5.1 can be very labor-intensive.
+  Will it be ok to disable NDB/MRR altogether?
+
+
+2.5 Make MRR code more of a module
+----------------------------------
+Some code in handler.cc can be moved to separate file.
+But changes in opt_range.cc can't.  
+TODO: Sort out how much we really can do here. Initial guess is not much as the
+code consists of:
+- Default MRR implementation in handler.cc
+- Changes in opt_range.cc to use MRR instead of multiple records_in_range()
+  calls. These rely on opt_range.cc's internal structures like SEL_ARG trees and 
+  so there is not much point in moving them out.
+- DS-MRR implementations which are spread over storage engines.
+and the only modularization we see is to move #1 into a separate file which
+won't achieve much.
+
 



DESCRIPTION:

Backport DS-MRR into MariaDB-5.2 codebase, also adding certain extra features to
make it more usable.


HIGH-LEVEL SPECIFICATION:



1. Requirements
===============

We need the following:

1. Latest MRR interface support, including extensions to support ICP when 
   using BKA.
2. Let DS-MRR support clustered primary keys (needed when using BKA).
3. Remove conditions used for key access from the condition pushed to index
   (ATM this manifests itself as "Using index condition" appearing where there 
    was no "Using where". TODO: example of this?)
4. Introduce a separate @@optimizer_switch flag for turning on/out ICP (atm it 
   is switched on/off by @@engine_condition_pushdown)
5. Introduce a separate @@mrr_buffer_size variable to control MRR buffer size
   for range+MRR scans. ATM it is controlled by @@read_rnd_size flag and that
   makes it unobvious for a number of users.
6. Rename multi_range_read_info_const() to look like it is not a part of MRR
   interface.
8. Try to make MRR to be more of a module
7. Improve MRR's cost model.

2. Required actions 
===================

Roughly in the order in which it will be done:

2.1 Fix DS-MRR/InnoDB bugs
--------------------------
We need to fix the bugs listed here:

http://bugs.mysql.com/search.php?cmd=display&status=Active&tags=index_condition_pushdown
http://bugs.mysql.com/search.php?cmd=display&status=Active&tags=mrr

2.2 Backport DS-MRR code to MariaDB 5.2
---------------------------------------
The easiest way seems to be to to manually move the needed code from mysql-6.0
(or whatever it's called now) to MariaDB.

2.3 Introduce control variables 
-------------------------------
Act on items #4 and #5 from the requirements. Should be easy as 
@@optimizer_switch is supported in 5.1 codebase.

2.4 Other backport issues 
-------------------------
* Figure out what to do with NDB/MRR.  5.1 codebase has "old" NDB/MRR
  implementation. mysql-6.0 (and NDB's branch) have the updated NDB/MRR 
  but merging it into 5.1 can be very labor-intensive.
  Will it be ok to disable NDB/MRR altogether?


2.5 Make MRR code more of a module
----------------------------------
Some code in handler.cc can be moved to separate file.
But changes in opt_range.cc can't.  
TODO: Sort out how much we really can do here. Initial guess is not much as the
code consists of:
- Default MRR implementation in handler.cc
- Changes in opt_range.cc to use MRR instead of multiple records_in_range()
  calls. These rely on opt_range.cc's internal structures like SEL_ARG trees and 
  so there is not much point in moving them out.
- DS-MRR implementations which are spread over storage engines.
and the only modularization we see is to move #1 into a separate file which
won't achieve much.



ESTIMATED WORK TIME

ESTIMATED COMPLETION DATE
-----------------------------------------------------------------------
WorkLog (v3.5.9)