maria-discuss team mailing list archive

[Justin Swanhart <greenlion@xxxxxxxxx>]

Questions comments are inline:

> * InnoDB: InnoDB native partitioning - so MySQL 8 InnoDB? But Monty says
there's next to no changes in InnoDB 8...
 Instant add column. New InnoDB deadlock detection (8.0). New
INFORMATION_SCHEMA table (8.0). Dedicated
 tablespace for temporary tables (in 5.7 and merged, check). Lock wait
policy (contribution)

Monty is *notorious* for low balling estimates.  His famous phrase is "it
is trivial".  Everybody knows that if Monty says it is trivial, you can add
10x the work to get it done, or more.  I am loathe to bring up CS, but it
is a perfect example.  You have to look no further than the time estimates
for the CS project.  The CS team was forced to accept an unacceptable
timeline based on poor estimates by non-CS team members, and when the team
couldn't meet arbitrary deadlines an alpha was shipped that should never
have touched customer systems, and could have damaged them.  InnoDB 8 also
includes transportable tablespaces for partitioned tables, and will likely
support native partitioning by the time 8 rolls.  Native partitioning also
entails implementing the changes to the SE.  See:
http://mysqlserverteam.com/innodb-native-partitioning-early-access/


> * Pluggable parser - MDEV-10142, PL/SQL parser, Oracle syntax support,
INTERSECT, EXCLUDE
FULL OUTER JOIN?  Table functions?

> * more for window functions (user defined window functions, MDEV-10855)
What other databases have user definable window functions?  Will the syntax
be similar?  I can't really think of a reason to need custom window
functions personally, as the existing set is very comprehensive.  What kind
of custom window function do you think people would need to write?  In most
implementations any aggregate functions can be used in a window frame
context, thus any aggregate SQL functions should satisfy requirements for
user defined window funcs.

> * more for CTEs (CTE sharing isn't in now, and MySQL 8 has it, so needs
to be done; Writable CTEs)
I assume writable CTE will have the same restrictions as updateable views?

> * "always encrypt" on the client side - Booking says there is no
particular interest in this at the moment.
Other RDBMS are rolling this out though.  Microsoft has a new ADO.NET layer
that supports end-to-end encryption from data at rest, to where it is
accessed in the client.

> * socket authentication
Can you explain this.  Is there an MDEV?

> * X Protocol/Document store - only if people have time or money will it
be done
If you want people to pay for it, perhaps you should implement it in
MaxScale instead of the server :)

>* Peter asks if there is any plans to support other languages like V8?
Supporting legacy database like Oracle PL/SQL is nice, but what about
JavaScript and stuff? Eric mentions his old work on Perl and Java external
language stored procedures. Peter says, if JSON and JavaScript are your
lingua franca, PL/SQL isn't so interesting - but there was no interest in
this amongst the folk.

PLEASE PLEASE PLEASE PLEASE IMPLEMENT WL-820.  External stored procs and
table functions.  Been sitting there for the taking for a long time.
Antony has tried to get you to get it into the server, but alas, it has
never happened, and though there continues to be wide requests for this
feature from the community, they fall on deaf ears.

ALSO: PL/SQL support requires support for ARRAYS and other contructs that
MySQL doesn't have.  Without greatly improving the SQL language support,
even basic Oracle 7 PL/SQL blocks won't work.  MySQL cursor support is kind
of lacking, and UPDATE WHERE CURRENT OF, etc, could be quite hard to
implement.
PLEASE PLEASE PLEASE add proper array support.  PLEASE.

> * Compressed binary log (from Tencent)
Compressing the binary log saves on space on the master, but it makes
seeking into binary logs much more difficult, and searching backwards
through them becomes much more difficult (which has implications for query
'rewind')

> * For ALTER TABLE show the corresponding CREATE TABLE statement as a
comment in the binary log. This is an annotation.
I guess this is to make streaming CDC easier, so that you can just stream
changes without having a copy of the DDL downstream in an idempotent manner
(updates that match no rows are inserts,etc?)

> * Counters for unsafe statements in replication (maybe in 10.2)
MySQL and MariaDB are notorious for marking statements unsafe that are
actually safe.  I'm not sure a counter would be of much use.  I think SBR
should be deprecated and ROW mode should be made the default. Any CDC
related feature (like rewind) needs full RBR.

> * Extended table map in RBR is missing
Please explain.  Is there an MDEV?

> * Galera 4, encryption of gcache
This is a pretty big bullet point in a small space.  Adding support for
Galera 4 while also making all the changes for InnoDB needs to be
coordinated carefully.

> * Fix the XA transaction bug ( MySQL has fixed it already ) - MDEV-7974
Please actually complete XA support, don't just half fix it like Oracle
did.  Add full support for XA SUSPEND and XA RESUME and allow more than one
thread to participate in a distributed transaction in the server.  This
will allow multiple transactions to share the same exact point-in-time so
that multi-threaded applications can easily maintain consistent snapshots!
This support could be exclusive to read-only transactions.

> * Indexes on expressions (this is part of virtual columns, will it not go
into 10.2? Check with Serg)
Indexes on expressions requires parser support.
create index expr_idx on some_table(a + b);
select * from some_table where a+b > 30 and a+b <= 50 -- (uses range over
expr_idx)

> * Flashback DDL MDEV-10571 (flashback DML will come in 10.2). It only
works with row based replication. Talk about what to name it. There's
already a MySQL time machine on github. Many like the name Rewind (but not
Monty). Let's do a poll on the mailing list
Okay, Flashback query is VERY complicated. A flashback query is a
materialized view.  There are two ways generally to achieve flashback:
a) materialize the query as is, and instead of rolling the query forward
incrementally, you roll it backwards.  Flexviews achieves this by computing
the query as it is now, then computing the delta from a prior point in time
until the transaction in which that computation happened. Then the delta is
played back against the query, but the MONUS of each operation is applied
which changes insertions to deletions and vice versa.  This still presents
a problem for OUTER JOIN.  No documented asynchronous refresh algorithms
exist for outer join that I'm aware of, that would work with the concept of
reading row history from serialized changes.  Flexviews uses the "rolling
join propagation" algorithm, which only works with inner joins.

b) provide a point-in-time snapshot of every table used in the query at the
desired point in time, and run the query over those tables (this is a
synchronous mechanism which supports OUTER JOIN).  The problem here is how
to provide such a table.  The most straight-forward way is to copy the
table, and then do the backwards replay for each (ideally in parallel) but
this is obviously undesirable if the query is "select * from some_big_table
join another_big_table on (...)" because you have to fully copy each table
before you can undo changes.  Otherwise you need to have the SE display old
versions, just like it does for MVCC, but it has to display versions from
binary logs, not undo logs, and this requires a lot of SE and engine
changes!

Mysql-time-machine on github is just a MySQL -> HBASE replicator.  It
relies on HBASE for flashback.  I don't think you plan to embed HBASE in
MariaDB, so adding DML flashback is a VERY BIG job.  I've been working on
Flexviews for 8 years and I just recently added flashback support to it!

If you are going to have generic flashback, you might as well commit to
incrementally refreshable materialized views.

> * Additional GIS functions to stay compatible. Also it would be good to
have a standalone GIS library (Georg suggests; wlad isn't too happy with
the suggestion). Georg suggests that calculations should use the reference
systems (Unflatten the world - MariaDB Server GIS the world looks flat)

GIS functions should use gdal, just like postgresql does.  Then you can
also add support for rasters.  Here is an example of the awesomeness of
postgresql and postgis.  It uses SQL aggregate functions, table functions,
CTE, GIS raster functions, sequences, etc:
https://github.com/greenlion/osmvox/blob/master/postgresql/combined_schema.sql

> * Query rewriting - MDEV-5561
I would like you to provide a SQL->DOM function call instead of just
providing a DOM to plugins.  This function could be exposed as a regular
item function as well so that anything in the server can parse SQL. I
suggest you implement my SQL "shim" interface and just provide some way to
get a easy to iterate over parsed data structure from the SQL, such as a
nested JSON array of objects.  A nested array is generated by
PHP-SQL-Parser and would provide a good template for such a JSON object.
https://github.com/greenlion/PHP-SQL-Parser

> * With MyRocks coming, should we drop TokuDB (and maybe even deprecate in
10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get
fixed. Peter says that bugs are fixed for customers... and there is ongoing
development to make it better
It is certainly disheartening that Percona isn't responsive to MariaDB
bugs, but I'm sure you understand that it is hard for a competitor to fix
bugs for another competitor.  MariaDB maintains a forked version of
TokuDB.  It isn't fair to expect the upstream vendor to fix bugs that your
customers are paying you to support, does it?  Perhaps you should pay a
percentage of your support fees for TokuDB issues to Percona, or come to
some other support agreement.  Perhaps YOU should make the bug fixes and
submit them to Percona. You could have just hired somebody to work on it at
MariaDB like you plan to do so for MyRocks, but you didn't.  Don't blame
Percona for your failure to properly be a steward of an open source fork!

> * ORDER BY LIMIT optimizer bugs (MDEV-8306)
Oh god, this is a downward spiral every time somebody touches it. Fixing it
correctly requires rewriting the parser, something that Oracle is
undertaking.

What about other new MySQL 8 features?  Are you getting rid of the .FRM
nightmare?  Are you going to support SET PERSIST?  etc?

-- Greenlion