maria-developers team mailing list archive

[Igor Babaev <igor@xxxxxxxxxxxx>]

Daniel, all

I would like to avoid these discussions because they take me a lot of time.

Anyway I have to participate.

The author of the original patch  Fredrik Nylander called the new key
cache structure 'segmented key cache'.
When I started working on the submitted patch I immediately realized
that the author actually used partitioning technique.
I remind you that the original patch just replaced the regular key cache
structure with the structure of the partitioned (segmented key cache)
and this new structure still was called KEY_CACHE in his code. The
number of partitions/segments was the compilation parameter of the
server (the same for all key caches). So the original patch just added a
new property to the existing key caches.

I added a possibility to partition any key cache.

I really did not see any valid arguments not to call this new type of
key caches 'partitioned key cache', or to call the new feature 'key
cache partitioning'.

I don't mind if we call the paragraph in the manual Segmented Key Cache
in acknowledgment of the original patch submitted by  Fredrik Nylander.
Yet, I would insist that Segmented Key Cache is the result of
partitioning of a regular key cache. Accordingly it consists of key
cache partitions, not segments.

Using widely the term Segmented Key Cache ultimately would require
massive renaming in the patch and I would like not to do it because the
term 'segment' would be really misleading for the implementation.

When resolving this argument we could ask:

1. the author of the original patch
2. our community members (not only PeterZ!)
3. our developers

what term they prefer 'Segmented Key Cache' or 'Partitioned Key Cache'.

To be quite honest I also should mention that MichaelR from Oracle/MySQL
introduced in 5.5 a possibility for MySQL partitions in MyISAM to use
different key caches for different partitions. Before that all
partitions could use only default key caches. This feature is 100%
orthogonal to the discussed key cache partitioning.


Daniel Bartholomew wrote:
> Hello everyone,
> 
> I'm working on the documentation for the Segmented Key Cache, one of
> the new features in MariaDB 5.2.
> 
> The documentation page will have "about", "syntax", and "example"
> sections.
> 
> I've based the "About" section on the High-Level Description from
> the worklog ( http://askmonty.org/worklog/Server-Sprint/?tid=85 ) and
> from the description in the mysys/mf_keycache.c file in the source.
> 
> Here is my first draft. I've changed the wording a bit to help
> it flow better and I've changed "partition" to "segment" in keeping with
> the official name.
> 
> ----------------------------------------------------------------------
> 
> == About Segmented Key Cache ==
> 
> A segmented key cache is a collection of structures for regular MyISAM
> key caches called key cache segments. Segmented key caches mitigate one
> of the major problem of the simple key cache: thread contention for key
> cache lock (mutex). With regular key caches, every call of a key cache
> interface function must acquire this lock. Threads compete for this
> lock even when the file and pages they have acquired shared locks for
> are in the key cache buffers.
> 
> When working with a segmented key cache any key cache interface
> function that needs only one page has to acquire the key cache lock
> only for the segment the page is assigned to. This makes the chances
> for threads not having to compete for the same key cache lock better.
> 
> Any page from a file can be placed into a buffer of only one segment.
> The number of the segment is calculated from the file number and the
> position of the page in the file, and it's always the same for the
> page. Pages are evenly distributed among segments.
> 
> The idea and the original code of the segmented key cache was provided
> by Fredrik Nylander from Stardoll.com. The code was extensively
> reworked, improved, and eventually merged into MariaDB by Igor Babaev
> from Monty Program.
> ----------------------------------------------------------------------
> 
> If there are any factual errors with the above, let me know.
> Improvements and suggestions are also welcome.
> 
> One thing I'm not happy with is this sentence:
>     "Threads compete for this lock even when the file and pages they
>     have acquired shared locks for are in the key cache buffers."
> 
> The original sentence from the worklog reads:
>     So threads compete for this lock even in the case when they have
>     acquired shared locks for the file and pages they want read from
>     are in the key cache buffers.

Ok ,I will help with the reading of the original statement adding
missing words:

So threads compete for this lock even in the case when
they have acquired shared locks for the file
and
the pages they want to read from are in the key cache buffers.

> 
> ... which is very confusing to me. If anyone has a better or more
> accurate rewording, please send it my way.
> 
> Also, there's a note in the source code (lines 5018-5023 of
> mysys/mf_keycache.c) which states:
>     Unfortunately if we use a partitioned key cache with N partitions
>     for B-tree indexes we can't say that the chances becomes N times
>     less. The fact is that any index lookup operation requires reading
>     from the root page that, for any index, is always ascribed to the
>     same partition. To resolve this problem we should have employed
>     more sophisticated mechanisms of working with root pages.
> 
> Do any of you have any opinions on whether or not this should be
> mentioned somewhere in the documentation? The only issue I have with
> mentioning it is that if I do, the first question that comes to my mind
> is: "If it is not N-times less, how much less is it?" I don't have an
> answer to that question (and math isn't one of my strengths).

I don't think it should be mentioned in the manual (at least in the
first variant of it).

Regards,
Igor.

> 
> Next up will be the "Syntax" section.
> 
> Thanks.
>