maria-discuss team mailing list archive

[MARK CALLAGHAN <mdcallag@xxxxxxxxx>]

Can you be more specific about your performance issues? We want to learn
what can be made better in MyRocks and RocksDB.

Level compaction is enabled by default. To confirm what is being used look
for this in $datadir/.rocksdb/LOG
2017/06/01-09:18:41.867035 7fba2196ea00
 Options.compaction_style: kCompactionStyleLevel

The most important options to change are rocksdb_block_cache_size and
rocksdb_max_background_compactions. There are more that can be set, but it
would be great if changing these were sufficient to resolve your problems.
When compression is enabled I set rocksdb_block_cache_size to 1/3 of RAM. I
usually set rocksdb_max_background_compactions to a value between 1/4 and
1/2 of the CPUs on the server.

What kind of storage do you have -- disk array, disk, fast SSD, less-fast
SSD (TLC NAND)?


On Thu, Jun 1, 2017 at 2:54 AM, Julien Muchembled <jm@xxxxxxxxxx> wrote:

> Hello,
>
> We have started to evaluate MyRocks 3 months ago, our DBs were initially
> empty and now they have reached 500 GB on-disk. We experience severe
> performance issues, and we have the feelings it could be better.
>
> https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide is
> interesting, especially the "Universal Compaction" section:
> « Universal compaction has a size limitation. Please be careful when your
> DB (or column family) size is over 100GB. »
>
> I don't think that write amplification is a bottleneck for our
> application, so my obvious first question is:
>  What compaction style is used in MariaDB and how to make sure that Level
> Style Compaction is used ?
>
> Also, we haven't set any 'rocksdb_*' option. The default configuration may
> not be appropriate in our case. For example, rocksdb_block_cache_size
> (default: 512MB) seems to be one of the most important options, similarly
> to innodb_buffer_pool_size and tokudb_cache_size, although I have never
> understood why (see my unanswered question at https://lists.launchpad.net/
> maria-discuss/msg02683.html ). Although the DB is much bigger than RAM
> (128 GB), RAM should be big enough to contain the active working set.
>
> Julien
>
> _______________________________________________
> Mailing list: https://launchpad.net/~maria-discuss
> Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~maria-discuss
> More help   : https://help.launchpad.net/ListHelp
>



-- 
Mark Callaghan
mdcallag@xxxxxxxxx