maria-developers team mailing list archive
Mailing list archive
Re: Some notes on pool_of_threads
>>>>> "Patrick" == Patrick Crews <gleebix@xxxxxxxxx> writes:
Patrick> Hi there,
Patrick> I've been playing around with the pool_of_threads addition and here is some
Patrick> of what I've found:
Patrick> All of this was on Mac OSX 10.5
Patrick> 1) Biggie here: Running the server with the --extra-port=NNNN option (with
Patrick> pool-of-threads set) causes the server to start then immediately stop. I'm
Patrick> still looking through logs and trace files to see if I can find anything
Patrick> meaningful. I am able to run without setting extra-port though.
Works for me and this is also tested in t/pook_of_threads.
Does the following work for you:
If yes, can can you start the server with
./mysqld --no-defaults --debug --extra-port=9999
and when it aborts, check the /tmp/mysqld.trace file for clues.
(Search from the end after 'error')
(For the above to work, you should configure MariaDB with --extra-debug).
Patrick> 2) We can set extra-max-connections when --extra-port isn't set. The limit
Patrick> is 100000, at least on my system. As I haven't been able to get the server
Patrick> running with extra-port yet, I haven't tested much beyond this.
You never need to set extra-max-connections to more than 2000.
Why trying to set extra-max-connection to 10000 ?
Normally you should set max-connections to a high number and
extra-max-connections to 100 or similar.
Patrick> 3) We can also set thread-pool-size even if thread-handling isn't set to
Patrick> pool-of-threads. It's limit appears to be < 2560 (I was running into some
Patrick> mess on my system so I haven't pinpointed it yet). The biggest thing is
Patrick> that if thread-pool-size is set too large, we just get a start and stop
Patrick> (Once again, my logs are a bit of a mess due to 'play', so I'll have more
Patrick> info on the server behavior in a bit).
thread-pool-size should normally not be higher than the number of
core * 2 you have in your system.
At startup mysqld creates 'pool-of-threads' number of threads to handle
your queries. Because of this, it's quite expected that you can't
create more than 2580 threads; You will run out of virtual memory.
This shouldn't however be a problem, as when you use pool-of-threads,
you are not likely to want more than 8-16 threads.
Patrick> 4) I found that if the number of connections is = thread-pool-size, new
Patrick> connections will terminate within 'connect-timeout' seconds (there was some
Patrick> question about whether we had to wait for the denied connection to receive a
Patrick> message until after a thread was freed).
When you use pool of threads (mysqld
--thread-handling=pool-of-threads) there shouldn't normally be any timeouts for
new connections; The timeout only happens if all threads in the pool
is doing some work.
Patrick> 5) Due to these problems (most especially #1), main.pool_of_threads.test
Patrick> fails on my machine. If I remove the --extra-port stuff, the test will run
Patrick> more than otherwise (the server dies quickly with extra-port stuff).
It's very strange.
It would be of great help if you could make a dbug trace for this.
Patrick> For extra-max-connections and thread-pool-size, do we want to allow these to
Patrick> be set even if they aren't relevant to the way the server is currently
The idea is that you can specify defaults in your .my.cnf file and
then just start mysqld with different --thread-handling options
without having to change anything else.
(This is the same way all other MySQL options works).
Patrick> Is there any specific information I can provide if anyone wants to look into
Patrick> why the server dies when I try to use --extra-port?
See above. A trace file would be very useful!
Patrick> Testing notes:
Patrick> 1) I have some changes to make to pool_of_threads.test (style, additional
Patrick> testing, etc).
Patrick> 2) We should also add some testing for extra-max-connections,
Patrick> thread-pool-size, etc.
We have tests for extra-max-connections in connect.test and
thread-pool-size is a bit hard to test, but don't know if this needs
to be explicitely tested as benchmarks will show how this works better
than anything else.
Patrick> Possibly to variables.test, though somewhere else
Patrick> would work. I've played around a bit with dynamically changing the
Patrick> variables, etc. I just can't get MTR to run the test to really polish my
Patrick> proposed changes before submitting a patch.
Anything we can help you with?