maria-developers team mailing list archive

[Sumit Lakra <sumitlakradev@xxxxxxxxx>]

Hello,

JIRA - https://jira.mariadb.org/browse/MDEV-12213
Project Status -
https://docs.google.com/spreadsheets/d/1nE-qFXhwwhF0hpOX3wUlczPjlPZTK_UzBu_PyrEF4Fw/edit?usp=sharing

WIP Code Branch - https://github.com/theGodlessLakra/server/tree/numa_one
Cumulative work    -
https://github.com/theGodlessLakra/server/tree/NUMA_Support

On Sat, Sep 16, 2017 at 4:36 PM, <serg@xxxxxxxxxxx> wrote:

> Hi, Sumit!
>
> On Sep 16, Sumit Lakra wrote:
>
> > The project 'NUMA Support in MariaDB' is still incomplete and like I
> > mentioned in my mails to my mentors, I am still willing to work on it
> > and see it through to the end.
>
> Great, thanks!
>
> > Most of the minor tasks along with some important tasks are already
> > complete. The important part that remains is to work on the task-queue
> > interface between SQL and InnoDB. I was unable to implement this for
> > the NUMA case and I won't be able to complete it without some help.
> > So, do you think you can connect me with someone from the
> > MariaDB/InnoDB/MySQL team who is an expert of this interface? I will
> > need someone who has a very good understanding of the present
> > structure to help me come up with a way to implement this for the NUMA
> > case. If you do find someone who can be of help, please see to it that
> > since GSoC is over (and otherwise as well), it shouldn't be a
> > Student-Mentor kind of thing, but more like two or more developers
> > working together trying to come up with a solution to this problem.
>
> Why wouldn't you ask the question on the maria-developers@ mailing list?
> I'll try to make sure that it'll be answered.
>

The Buffer Pool has been divided into as many instances as there are NUMA
nodes. All user SQL threads accessing the same Database has been bound to
the same NUMA node.

[1] Now, they must be restricted to use the Buffer Pool Instance of the
same NUMA node for all purpose henceforth. For example, if a user SQL
thread accessing Database A is running on NUMA node 1, then all the bpages
related to Database A must be loaded into the Buffer Pool Instance existing
on that NUMA node, let's call that Buffer Pool Instance 1.

[2] The InnoDB read/write io threads are in multiples of the number of
available NUMA nodes. They have been bound to respective NUMA nodes as
well. After the above task [1] is implemented, these background threads
will need to be restricted to pick tasks from the queues associated with
the same NUMA node as they are bound to run on. This will mean that the
threads will now only be executing tasks that are associated with the
Buffer Pool Instance on the same NUMA node.

Implementing [1] and [2] above will complete the integral part of the
project 'NUMA Support in MariaDB'. DEBUG and test cases will need to be
written for the same afterwards.

However, one need not know the details of NUMA Implementation in order to
answer this. Just suppose that we had to re-structure the SQL-InnoDB
interface such that all background reading/writing tasks associated with a
particular Database is put on one queue, and the background threads are
restricted to use only one of these queues. How to implement this ?

>
> > However, if you are unable to connect me with someone who has
> > expertise in the handler interface, I am afraid I won't be able to
> > complete this project on my own. In this case, would you like me to
> > create a pull request of the work done so far (
> > https://github.com/theGodlessLakra/server/tree/NUMA_Support). Although
> > NUMA Support is incomplete, merging this code to the MariaDB code base
> > will allow other developers to continue working on this.
>
> Is it sufficiently complete to be useful as is?
>
> Regards,
> Sergei
> Chief Architect MariaDB
> and security@xxxxxxxxxxx
>



Thanks,
Sumit Lakra