maria-developers team mailing list archive

[Michael Widenius <monty@xxxxxxxxxxxx>]

Hi!

Sorry for replying this 'off-thread', but I was not by some strange
reason subscribed to the maria-developers list :(

MARK CALLAGHAN writes:

mark> One of the features listed here
mark> http://askmonty.org/wiki/index.php/MariaDB_versus_MySQL as a potential
mark> backport is support for pool-of-threads. I am not sure how safe this
mark> feature is today.
mark> 
mark> http://mysqlha.blogspot.com/2009/01/no-new-global-mutexes-and-how-to-make.html
mark> describes some of the problems that might cause a server to hang when
mark> pool-of-threads is used.

The main port of the pool-of-threads code has been in extensive use by
a high performance MySQL customer for a long time and, within their
limits, it has given them a lot of extra performance.

To be fair, the above customer is using a scheduler that is using
getport instead of libevent.  I am considering to add also getport
support to the scheduler.cc in MariaDB, even if this only gives higher
performance on Solaris.

The patch has it's problems and is not complete, but for certain
workloads and configurations it can definitely save the day.  One
example is if your application needs 200,000 connected users.

One of the biggest problem with pool-of-threads is that when you run
out of threads you are stuck as you can't see what's going on and you
can connect and kill things.

The extra-port options fixes that.  It also allows you to mix
pool-of-threads and one-thread-per-connection in an easy way.

It would have been better to make this a connection option, but
because connections are also handled by the thread pool, when you run
with pool-of-threads, this doesn't work as if all worker threads are
occupied the connection will never go trough.  Adding another pool for
connection would require so much new work that it wouldn't be safe for
MariaDB 5.1.

For MariaDB 5.2 we should consider looking at adding another thread
pool for connections and making the scheduler a connection option.

Drizzle is by default using pool-of-threads, so the code gets more
tested every day.

mark> http://bugs.mysql.com/bug.php?id=42288 describes the performance
mark> overhead of pool-of-threads. The overhead is significant for workloads
mark> with fast queries such as sysbench.

Brian/Drizzle is, as far as I know, trying to benchmark the original
getevent port code and the libevent code to find out how things really
stands.

Brian suspect that there may a problem in libevent that needs to be
fixed and he has someone looking into it.  When they or we have fixed
it, we will collaborate to ensure both projects works properly.

mark> I think that pool-of-threads needs a lot of work before it can be
mark> considered safe. We backported this to our 5.0 branch at work and have
mark> begun to search for fixes.

At least pool-of-threads is not default in MariaDB and you have to
turn it explicitely one if you want to use it;  This makes it
reasonable safe to include into MariaDB 5.1 at this point as old
user will not be affected at all by it and new users trying it can do
that as long as they are aware of the limitations.

Regards,
Monty