maria-discuss team mailing list archive

[Juan Telleria Ruiz de Aguirre <jtelleria.rproject@xxxxxxxxx>]

I will introduce the following configuration settings to my.ini
Configuration File in order to use Bloom Filters, and enable compression.

Any further suggestion? Thank you!


rocksdb_default_cf_options="block_based_table_factory={filter_policy=bloomfilter:10:false;};prefix_extractor=capped:4};level_compaction_dynamic_level_bytes=true;optimize_filters_for_hits=true;compaction_pri=kMinOverlappingRatio;compression=kSnappyCompression"

rocksdb_whole_key_filtering = 0 # NOTE: Default Value: ON (1).
'whole_key_filtering' (1) cannot be enabled simmultaneously with
'prefix_extractor', as the first would override the second. Therefore,
# This has been disabled in order to reduce "Bloom Filter Memory
Consumption", and use lower resources.

##############################################
# block_based_table_factory
##############################################
#
# filter_policy:    "The optional FilterPolicy mechanism can be used to
enable Bloom Filters, and reduce the number of disk reads substantially.
# If you're doing point lookups you definitely want to turn bloom filters
on. We use bloom filters to avoid unnecessary disk reads.
# Default bits_per_key is 10, which yields ~1% false positive rate. Larger
bits_per_key values (e.g.: 12) will
#      reduce false positive rate, but increase memory usage and space
amplification.".
#
#
# rocksdb_whole_key_filtering:  "If set to None the rocksdb default of True
is used. If True, place whole keys in the filter (not just prefixes).
# This must generally be true for gets to be efficient".
#
# prefix_extractor: "A SliceTransform object that defines key prefixes. Key
prefixes are then used to perform some interesting optimizations:"
# (1) Define prefix bloom filters, which can reduce read amplification of
prefix range queries (e.g., give me all keys that start with prefix XXX).
This
#     usually sufficient in reducing I/Os, points (2) and (3) are usually
not necessary nor common.
# (2) Use hash-map-based memtables to avoid binary search costs in
memtables.
# (3) Add hash index to table files to avoid binary search costs in table
files.
#
#
# **PREFIX VS. WHOLE KEY** a) "By default a hash of every whole key is
added to the bloom filter. This can be disabled by setting
'rocksdb_whole_key_filtering'
#     to false.
# b) When 'prefix_extractor' is set, a Hash of the Prefix is also Added to
the Bloom. Since there are less unique prefixes than unique whole keys,
#    storing only the prefixes in bloom will result into smaller blooms
with the down side of having larger false positive rate.
#    Moreover the prefix blooms can be optionally also used during ::Seek
whereas the whole key blooms are only used for point lookups.
#
# WARNING: If both 'whole_key_filtering' and 'prefix' are set, 'prefix' are
not checked during point lookups. If 'whole_key_filtering' is set,
# this is the result of checking the bloom of the 'whole key', otherwise
this is the result of checking the bloom of the 'prefix'.
#
##############################################
# Other Optimization Settings
##############################################
#
# level_compaction_dynamic_level_bytes:  "If set TRUE: Target size of the
last level (num_levels-1) will always be actual size of the level.
#   And then Target_Size(Ln-1) = Target_Size(Ln) /
max_bytes_for_level_multiplier. We won't
#   fill any level whose target will be lower than max_bytes_for_level_base
/ max_bytes_for_level_multiplier.
#   These levels will be kept empty and all L0 compaction will skip those
levels and directly go to the first
#   level with valid target size.
#
# optimize_filters_for_hits: "Enable to to reduce some bloom filter block
size".
#
##############################################
# Compactation & Compression
##############################################
#
# compaction_pri: "Multi-Thread Compactation Algorithm.
'kMinOverlappingRatio' is choosen, as reduces write amplification".
#
# compression: "Allows to specify the compression to use, which by default
is Snappy. Snappy is lightweight compression so it usually strikes
#   a good balance between space and CPU usage".
#