maria-developers team mailing list archive

[MARK CALLAGHAN <mdcallag@xxxxxxxxx>]

Welcome back to the MySQL community Jan. It is good to see SkySQL/MariaDB
get more InnoDB expertise.


On Mon, Aug 19, 2013 at 9:32 AM, Jan Lindström <jplindst@xxxxxxxxxxx> wrote:

>  Hi,
>
> From experience working on two storage engines on MySQL and being main
> developer for the second one, and one "storage engine" on big company, I
> would say MySQL storage engine API has some legacy problems and clearly
> some missing functionality. e.g.
>
> - Some part of the storage engines are always the same, parameters are not
> fully given to functions or you really can't return proper error messages
>
> - There is interface to get e.g. row type, but still you need to hack
> handler.cc/handler.h because some values are hard coded there (show
> create table)
>
> - Storage engine API clearly shows that transaction concept has being
> introduced after the first API version was designed, still you can't really
> return errors on commit operation
>
> - Storage engine can announce some that it can do some functionality to
> upper layer, but this is very limited. Storage engines should have more
> control what is done on MySQL parser/optimiser level and what is done on
> storage engine layer. E.g. for columnar storage engine, I would love to
> have opportunity to inform MySQL that I can do join's myself with all
> conditions pushed down to attributes, but that I can't do e.g. external
> sorting.
>
> - For in-memory storage engine there should be API where MySQL data type
> and in-memory data types are exactly the same to avoid conversions,
> similarly in-memory buffer pool should contain directly usable types for
> MySQL.
>
> Thus, as a conclusion, we really would need a better and cleaner API where
> storage engine has more control.
>
> R: Jan
>
> The storage engine API has never been my favorite part of MySQL. I haven't
> written a storage engine but I used to maintain a custom storage engine and
> migrated that from 4.0 to 5.0 so I had some experience with the API.
>
>  Is there a different API that would make it easier for innovation to
> come to MySQL/MariaDB? For example there is a lot going on in the
> write-optimized database space. Until recently we had nothing for MySQL
> open-source. Now we have open source TokuDB (great news). I haven't read
> their source but I have been told they use something that was derived from
> the BerkeleyDB API.
>
>  Would the BerkeleyDB API make it easier to get new storage engines into
> MySQL? For example, we don't have an engine optimized for in-memory
> workloads. And while InnoDB will have great cache hit rates when all data
> is in-memory. It might write much more data to disk than required.
>
>  Do we need a better API or is the current code good enough? I hope to
> hear from people who have done serious storage engine work.
>
>  --
> Mark Callaghan
> mdcallag@xxxxxxxxx
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~maria-developers
> Post to     : maria-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~maria-developers
> Post to     : maria-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Mark Callaghan
mdcallag@xxxxxxxxx