maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

MARK CALLAGHAN <mdcallag@xxxxxxxxx> writes:

> On Fri, Sep 23, 2011 at 4:32 AM, Kristian Nielsen
> <knielsen@xxxxxxxxxxxxxxx> wrote:

>> In general, when implementing non-blocking semantics like this, there are two
>> main approaches:
>>
>> 1. Write the code in event-driven style. This means using some kind of state
>>   machine or message passing (or both) rather than normal nested
>>   calls. Whenever code needs to do I/O, or call another function that might
>>   do I/O, the current state is manually saved to some struct and control
>>   returns to the caller. When the I/O completes control must continue in the
>>   next state of the state machine or handler of completion message.
>>
>> 2. Use co-routines (also called fibers or light-weight threads or
>>   whatever). The code is written in normal style with nested calls. For
>>   operations that need to do I/O, a co-routine is spawned to run the
>>   operation; when waiting for I/O the co-routine is suspended, and resumed
>>   again when I/O completes.

> Why isn't it sufficient to do a sequence of non-blocking reads to
> buffer enough data to produce a query result and then process the
> received data? That doesn't require getcontext/setcontext.

Let's take mysql_real_query() as an example. Here is the relevant part of the
call tree:

    mysql_real_query
      mysql_send_query
        simple_command
          cli_advanced_command
            mysql_reconnect
              # ... more stuff here if want to handle auto reconnect
            net_write_command
              net_write_buff
                net_real_write
                  vio_write
                    write
      cli_read_query_result
        cli_safe_read
          my_net_read
            my_real_read
              vio_read
                read

The places where we can block are write() and read() (ignoring automatic
reconnect). vio_write() and vio_read() will return EWOULDBLOCK on a
non-blocking socket.

We can detect this in net_real_write(), save the current state (eg. local
variables) in the MYSQL struct, and return. Then we similarly modify
net_write_buff(), net_write_command(), cli_advanced_command(),
simple_command(), mysql_send_query(), my_real_read(), my_net_read(),
cli_safe_read(), cli_read_query_result(), and mysql_real_query() to detect
that we need to return due to blocking, save the local state, and return.

Then when the application calls back into mysql_real_query() after poll() has
found data ready on the socket, we further modify mysql_real_query() to
inspect the current state and proceed to call into mysql_send_query() or
cli_read_query_result() as appropriate. And similarly modify the other
functions to be able to resume from the saved state.

This is what I refered to as method (1), the event-driven code style with
state machine. It can be implemented in different ways of course, all of which
require modifying every function in the call stack.

Any bugs introduced in these modifications will affect the 99% of applications
that never use the non-blocking API. Any overhead introduced in each function,
checking what the current state is, will affect the 99% of applications not
using the non-blocking API.

Is this sufficient? Yes. In fact, one can implement the exact same API in this
way. It will be fully binary compatible, one could switch between this and the
co-routine implementation in libmysqlclient.so and the application would not
even need to be recompiled.

But compare with method (2) using co-routines. Here, the _only_ modification
we need to existing code is a single conditional in vio_write() and vio_read()
to check if we run non-blocking, and call the corresponding non-blocking I/O
routine if so.

    if (using_nonblocking)
      return async_read(socket, buf, len);
    else
      return read(socket, buf, len);

    async_read(socket, buf, len) {
      for (;;) {
        res= read(socket, buf, len);
        if (res < 0 && errno == EWOULDBLOCK) {
          wait_for_flag= MYSQL_WAIT_READ;
          yield();
        }
        else
          return res;
      }

Now the impact on the 99% blocking applications is minimal. Risk of
introducing bugs in existing code is low. Run-time overhead is low, just an
extra if () for each read() and write() syscall.

So yes, if the goal is to avoid using co-routines, it's certainly
possible. But I find the co-routine approach preferable.

Hope this helps,

 - Kristian.