← Back to team overview

maria-developers team mailing list archive

forwarded message from Kristian Köhntopp



Here is booking.com's wish list!


--- Begin Message ---
These are the notes from the flipchart, transcribed. I have not yet made a Wiki page from that.

I hope the list of attendees is correct and I did not leave anybody out - I forgot to make a list.

Please check these notes, and send me back any annotations. I will turn all of this into a wiki page, and send copies to Monty/SkySQL as well.


Bert Lindner <bert.lindner@xxxxxxxxxxx>
Eric Herman <eric.herman@xxxxxxxxxxx>
Herald van der Breggen <herald.vanderbreggen@xxxxxxxxxxx>
Kristian Köhntopp <kristian.koehntopp@xxxxxxxxxxx>
Elizabeth Mattijsen <elizabeth.mattijsen@xxxxxxxxxxx>
Luciano Rocha <luciano.rocha@xxxxxxxxxxx>
Michael Diliberto <Michael.Diliberto@xxxxxxxxxxxxx>
Nicolai Plum <nicolai.plum@xxxxxxxxxxx>
"Ruud H.G. van Tol" <ruud.vantol@xxxxxxxxxxx>
Yves Orton <yves.orton@xxxxxxxxxxx>

Guy Nir <guy.nir@xxxxxxxxxxx>
Deborah Wade <deborah.wade@xxxxxxxxxxx>
Simon J Mudd <simon.mudd@xxxxxxxxxxx>
Glenn Nadeau <glenn.nadeau@xxxxxxxxxxx>

Ivan Zoratti <ivan@xxxxxxxxxx>
Maria Luisa Raviol <mluisa.raviol@xxxxxxxxxx>
Olivier Beutels <olivier@xxxxxxxxxx>
"Michael \"Monty\" Widenius" <monty@xxxxxxxxxxxx>

Scalability Team Session

- DROP TABLE is slow -- Kris, Ruud

- Run an instance of MySQL with Profiling on, with real life traffic, forward profiling data to MySQL. Also oprofile the box. -- Monty

- Replication Scaling: Number of fsync calls in SQL_THREAD
  - Group Commit and Parallel Apply -- Monty

- Nested Transactions -- Yves
  - If anybody has a definition of Nested Transactions that is viable, I'd be glad to hear that. What happens in 'begin; begin; ...; commit; ... rollback;'? -- Kris

- Replication Synchronisation: GUID, MASTER_POS_WAIT() across 3 or more hops -- Kris

- Large new table in Replication causes lag -- Yves

- Online compressible MyISAM/Aria -- Liz

- GET_LOCK() improvements: Aquire sequentially more than one -- Yves

- Hash Join! -- Yves
  - Way ahead of you -- Monty

- More execution plan info (Waits, I/O) -- Yves
  - Performance_Schema -- Monty

- Better Explain -- Yves
  - WL Item exists -- Monty
  - What is the # of that? -- Kris

- Better Statistics for Queries -- Monty
  - Log query skeletons -- Monty
    - Yes, please. Remember we have tables with time-tailed names (_YYYYMMDD), variable length WHERE ... IN (...) lists, and variable length extended insert statements. -- Yves

- Unquoted numbers cause weird query plans -- Ruud
  - Yes, they have to, thanks to numeric casting -- Monty
    - Can we make such casts cause an error optionally, that would make that SQL easier to find in Development. -- Kris

- UTF-8 currently stalled at booking because of MEMORY tables being fixed length in query plans with 'using temporary'. -- Kris, Eric
  - variable length memory tables existed with limits, need sponsorship. -- Monty

- Some large join take ages to execute. Probably due to tmp tables to disk. -- Liz

- Using memcache as a backend. -- Liz
  - done. -- Monty

- What did that update change? (Adminlogging) -- Yves, Ruud
- Multiple record update syntax (av) -- Yves, Ruud
  - Spec these two out. -- Monty
  - insert/delete, update/select, delete/select previous values. -- Yves, Ruud

- Trailing controls w/ RBR + Statement log or Audit plugin in 5.5 -- Kris

- Federation vs. Table Copy? -- Eric
- Other ways to coalesce data from multiple masters. Multiple disjunct source replication slave. -- Eric, Nicolai

- In replication, create new tables and populate them through a second connection to speed things up. -- Ruud
- Also for ALTER TABLE? -- Ruud
  - Spec it. -- Monty

- ALTER TABLE progress report. -- Yves
  - done. -- Monty

- IN-SQL-CLONE-PROCEDURE. Setting up a slave from within the slaves command line client
- move tables physically from master to slave at disk speed, no index creation needed?
    -- Nicolai


- Monitoring. -- Aaron
  - Monyog. -- Kris, Ivan
  - Zabbix -- Aaron, Kris

- FAST ALTER TABLE, online ALTER TABLE, versioned rows in schema. -- Herald, Kris
  - Online table change, fast drop columns discussion. -- Monty, Kris

DBA Team Session

- Large joins in DW -- Debs
- Large tables joined in DW -- Debs
  - Hash Join, Join optimizations in 5.3/5.5 -- Monty

- Optimizer Issues: Histograms, Hash Join, Subqueries
  - Partly done. -- Monty

- Try out engines for fast inserts: Infobright, InfiniDB, ... -- SkySQL to Booking

  - Facebook Hack (-- SkySQL) vs. Proper Solution. -- Kris

- Setting up a profiler host for Monty. -- DBA

- Move IBD files physically. -- DBA
  - possible with XtraDB at target, xtrabackup. -- Kris

  - Security/Firewall issues due to direction of connect. -- Kris
  - mysqldumper parallel load -- Max, SimonM

- proper tablespace management -- SimonM
  - state benefits, spec it. -- Monty to SimonM

- Configuration changes require server restart (especially: Replication, InnoDB) -- DBA

- faster I_S, need to work with 1E6 tables. -- SimonM

- Grants, Roles -- Debs, SimonM
  - Spec it! -- Monty

- Views are being reparsed for every access, never cached. -- Monty, Kris, Ruud

- RBR conflict resolution SP -- Kris

- "fan-in": Multiple Masters, FederatedX: "unsharding should be easy" -- SimonM

- transaction-p: "Am I in a transaction"-Predicate as a status variable -- Kris, Monty

- Compressed Binlog? At Event level? -- SimonM

- Async Binlog Shipping: Run a SP or Shell cmd on binlog rotation. -- Kris

- Async query API, so that we can write a map-reducer at client level. -- Kris
  - To be done soon, funding from elsewhere. -- Monty

- functional indexes, indexes global to partitions. -- Kris

- Config in tables, not in files, e.g. master.info, relay,info -- DBA
  - part of 5.6 -- Monty

Internal at DBA/Booking:

- Build a snapshot management script for hourly snapshots. -- Kris
- Check out how rsync fails for cloning, fast cloning. -- Kris
- LB DB connections with F5 -- Glenn, Guy, Kris
  - Do you have partners that do this? What are there experiences? -- to SkySQL

--- End Message ---

Follow ups