← Back to team overview

maria-discuss team mailing list archive

Rocksdb_no_file_closes in MariaDB MyRocks


Hi all,

I am trying to put a MariaDB MyRocks database into production and right now
I am facing problems regarding the memory usage of MyRocks. I haven't been
able to identify one specific problem but the following links contained
some useful information:



The current behavior of the mysqld process is that when putting the
database into production (an OLTPish process) on a machine with 64G RAM and
a block cache configured with 16G it starts to slowly accumulate more and
more memory until swap is used. This is obviously not suitable for

Currently I am wondering about a particular statistic, the number of open
and closed files. *Particularly I am wondering why Rocksdb_no_file_closes
is staying a zero at all time while Rocksdb_no_file_opens slowly increases
over time.* Does anybody know why that is? Could it be responsible for the
steady memory increase?

I experimented with the parameter rocksdb_max_open_files, it was set to -1,
I tried setting it to 1536, 512 and 128. The lower the value the faster
Rocksdb_no_file_opens increases (which seems right). However, at no setting
do I see any change in

Running MariaDB 10.2.18.

Here are the settings regarding rocksdb in my.cnf:

# MyRocks
plugin-load-add                    = ha_rocksdb.so
default-storage-engine                = rocksdb
default-tmp-storage-engine            = MyISAM
transaction-isolation                = READ-COMMITTED
rocksdb_unsafe_for_binlog            = 1 # enables statement based
rocksdb_datadir                    = /var/local/mysql/rocksdb
rocksdb_wal_dir                    = /var/mysql_logs/rocksdb
rocksdb_tmpdir                    = /var/mysql_logs/rocksdb

rocksdb_flush_log_at_trx_commit            = 0
rocksdb_use_direct_io_for_flush_and_compaction    = 0
rocksdb_use_direct_reads            = 0

rocksdb_max_open_files                = 512
rocksdb_max_background_jobs            = 8
rocksdb_max_total_wal_size            = 4G
rocksdb_block_size                = 64K
rocksdb_block_cache_size            = 4G
rocksdb_table_cache_numshardbits        = 6
rocksdb_new_table_reader_for_compaction_inputs    = 1
rocksdb_compaction_readahead_size        = 4M
#rocksdb_db_write_buffer_size            = 0 # max write buffer across all
column families, zero = disabled

# rate limiter
rocksdb_bytes_per_sync                = 4M
rocksdb_wal_bytes_per_sync            = 4M
rocksdb_rate_limiter_bytes_per_sec        = 80M # MB/s. Increase if you're
running on higher spec machines

# triggering compaction if there are many sequential deletes
rocksdb_compaction_sequential_deletes_count_sd    = 1
rocksdb_compaction_sequential_deletes        = 199999
rocksdb_compaction_sequential_deletes_window    = 200000

# read free replication