maria-developers team mailing list archive

[Michael Widenius <monty@xxxxxxxxxxxx>]

Hi!


>>>>> "Patrick" == Patrick Crews <gleebix@xxxxxxxxx> writes:

Patrick> Hi there,
Patrick> I've been playing around with the pool_of_threads addition and here is some
Patrick> of what I've found:
Patrick> All of this was on Mac OSX 10.5

Patrick> 1)  Biggie here:  Running the server with the --extra-port=NNNN option (with
Patrick> pool-of-threads set) causes the server to start then immediately stop.  I'm
Patrick> still looking through logs and trace files to see if I can find anything
Patrick> meaningful.  I am able to run without setting extra-port though.

Very strange.

Works for me and this is also tested in t/pook_of_threads.

Does the following work for you:

cd mysql-test
./mysql-test-run t/pool_of_threads

If yes, can can you start the server with

cd mysqld
./mysqld --no-defaults --debug --extra-port=9999

and when it aborts, check the /tmp/mysqld.trace file for clues.
(Search from the end after 'error')

(For the above to work, you should configure MariaDB with --extra-debug).

Patrick> 2)  We can set extra-max-connections when --extra-port isn't set.  The limit
Patrick> is 100000, at least on my system.  As I haven't been able to get the server
Patrick> running with extra-port yet, I haven't tested much beyond this.

You never need to set extra-max-connections to more than 2000.
Why trying to set extra-max-connection to 10000 ?
Normally you should set max-connections to a high number and
extra-max-connections to 100 or similar.

Patrick> 3)  We can also set thread-pool-size even if thread-handling isn't set to
Patrick> pool-of-threads.  It's limit appears to be < 2560 (I was running into some
Patrick> mess on my system so I haven't pinpointed it yet).  The biggest thing is
Patrick> that if thread-pool-size is set too large, we just get a start and stop
Patrick> (Once again, my logs are a bit of a mess due to 'play', so I'll have more
Patrick> info on the server behavior in a bit).

thread-pool-size should normally not be higher than the number of
core * 2 you have in your system.

At startup mysqld creates 'pool-of-threads' number of threads to handle
your queries.  Because of this, it's quite expected that you can't
create more than 2580 threads;  You will run out of virtual memory.

This shouldn't however be a problem, as when you use pool-of-threads,
you are not likely to want more than 8-16 threads.

Patrick> 4)  I found that if the number of connections is = thread-pool-size, new
Patrick> connections will terminate within 'connect-timeout' seconds (there was some
Patrick> question about whether we had to wait for the denied connection to receive a
Patrick> message until after a thread was freed).

When you use pool of threads (mysqld
--thread-handling=pool-of-threads) there shouldn't normally be any timeouts for
new connections;  The timeout only happens if all threads in the pool
is doing some work.

Patrick> 5)  Due to these problems (most especially #1), main.pool_of_threads.test
Patrick> fails on my machine.  If I remove the --extra-port stuff, the test will run
Patrick> more than otherwise (the server dies quickly with extra-port stuff).

It's very strange.
It would be of great help if you could make a dbug trace for this.

Patrick> For extra-max-connections and thread-pool-size, do we want to allow these to
Patrick> be set even if they aren't relevant to the way the server is currently
Patrick> running?

Yes.

The idea is that you can specify defaults in your .my.cnf file and
then just start mysqld with different --thread-handling options
without having to change anything else.

(This is the same way all other MySQL options works).

Patrick> Is there any specific information I can provide if anyone wants to look into
Patrick> why the server dies when I try to use --extra-port?

See above.  A trace file would be very useful!

Patrick> Testing notes:
Patrick> 1)  I have some changes to make to pool_of_threads.test (style, additional
Patrick> testing, etc).

Please propose!

Patrick> 2)  We should also add some testing for extra-max-connections,
Patrick> thread-pool-size, etc.

We have tests for extra-max-connections in connect.test and
pool_of_threads.test

thread-pool-size is a bit hard to test, but don't know if this needs
to be explicitely tested as benchmarks will show how this works better
than anything else.

Patrick>  Possibly to variables.test, though somewhere else
Patrick> would work.  I've played around a bit with dynamically changing the
Patrick> variables, etc.  I just can't get MTR to run the test to really polish my
Patrick> proposed changes before submitting a patch.

Anything we can help you with?

Regards,
Monty