maria-discuss team mailing list archive

[Vladislav Vaintroub <vvaintroub@xxxxxxxxx>]

There is no kSnappyCompression on Windows, unless you managed to somehow compile it yourself.

________________________________
From: Maria-discuss <maria-discuss-bounces+vvaintroub=gmail.com@xxxxxxxxxxxxxxxxxxx> on behalf of Juan Telleria Ruiz de Aguirre <jtelleria.rproject@xxxxxxxxx>
Sent: Monday, September 10, 2018 5:09:01 PM
To: maria-discuss@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Maria-discuss] MariaDB + RocksDB: MemTables Compactation

I will introduce the following configuration settings to my.ini Configuration File in order to use Bloom Filters, and enable compression.

Any further suggestion? Thank you!


rocksdb_default_cf_options="block_based_table_factory={filter_policy=bloomfilter:10:false;};prefix_extractor=capped:4};level_compaction_dynamic_level_bytes=true;optimize_filters_for_hits=true;compaction_pri=kMinOverlappingRatio;compression=kSnappyCompression"

rocksdb_whole_key_filtering = 0 # NOTE: Default Value: ON (1). 'whole_key_filtering' (1) cannot be enabled simmultaneously with 'prefix_extractor', as the first would override the second. Therefore,
# This has been disabled in order to reduce "Bloom Filter Memory Consumption", and use lower resources.

##############################################
# block_based_table_factory
##############################################
#
# filter_policy:    "The optional FilterPolicy mechanism can be used to enable Bloom Filters, and reduce the number of disk reads substantially.
# If you're doing point lookups you definitely want to turn bloom filters on. We use bloom filters to avoid unnecessary disk reads.
# Default bits_per_key is 10, which yields ~1% false positive rate. Larger bits_per_key values (e.g.: 12) will
#      reduce false positive rate, but increase memory usage and space amplification.".
#
#
# rocksdb_whole_key_filtering:  "If set to None the rocksdb default of True is used. If True, place whole keys in the filter (not just prefixes).
# This must generally be true for gets to be efficient".
#
# prefix_extractor: "A SliceTransform object that defines key prefixes. Key prefixes are then used to perform some interesting optimizations:"
# (1) Define prefix bloom filters, which can reduce read amplification of prefix range queries (e.g., give me all keys that start with prefix XXX). This
#     usually sufficient in reducing I/Os, points (2) and (3) are usually not necessary nor common.
# (2) Use hash-map-based memtables to avoid binary search costs in memtables.
# (3) Add hash index to table files to avoid binary search costs in table files.
#
#
# **PREFIX VS. WHOLE KEY** a) "By default a hash of every whole key is added to the bloom filter. This can be disabled by setting 'rocksdb_whole_key_filtering'
#     to false.
# b) When 'prefix_extractor' is set, a Hash of the Prefix is also Added to the Bloom. Since there are less unique prefixes than unique whole keys,
#    storing only the prefixes in bloom will result into smaller blooms with the down side of having larger false positive rate.
#    Moreover the prefix blooms can be optionally also used during ::Seek whereas the whole key blooms are only used for point lookups.
#
# WARNING: If both 'whole_key_filtering' and 'prefix' are set, 'prefix' are not checked during point lookups. If 'whole_key_filtering' is set,
# this is the result of checking the bloom of the 'whole key', otherwise this is the result of checking the bloom of the 'prefix'.
#
##############################################
# Other Optimization Settings
##############################################
#
# level_compaction_dynamic_level_bytes:  "If set TRUE: Target size of the last level (num_levels-1) will always be actual size of the level.
#   And then Target_Size(Ln-1) = Target_Size(Ln) / max_bytes_for_level_multiplier. We won't
#   fill any level whose target will be lower than max_bytes_for_level_base / max_bytes_for_level_multiplier.
#   These levels will be kept empty and all L0 compaction will skip those levels and directly go to the first
#   level with valid target size.
#
# optimize_filters_for_hits: "Enable to to reduce some bloom filter block size".
#
##############################################
# Compactation & Compression
##############################################
#
# compaction_pri: "Multi-Thread Compactation Algorithm. 'kMinOverlappingRatio' is choosen, as reduces write amplification".
#
# compression: "Allows to specify the compression to use, which by default is Snappy. Snappy is lightweight compression so it usually strikes
#   a good balance between space and CPU usage".
#