org.apache.flink.contrib.streaming.state

Enum PredefinedOptions

java.lang.Object

java.lang.Enum<PredefinedOptions>

org.apache.flink.contrib.streaming.state.PredefinedOptions

All Implemented Interfaces:

Serializable, Comparable<PredefinedOptions>

public enum PredefinedOptions
extends Enum<PredefinedOptions>

The PredefinedOptions are configuration settings for the EmbeddedRocksDBStateBackend. The various pre-defined choices are configurations that have been empirically determined to be beneficial for performance under different settings.

Some of these settings are based on experiments by the Flink community, some follow guides from the RocksDB project.

All of them effectively disable the RocksDB log by default because this file would grow indefinitely and will be deleted with the TM anyway.

The PredefinedOptions are designed to cope with different situations. If some configurations should be enabled unconditionally, they are not included in any of the pre-defined options. Please check RocksDBResourceContainer.createBaseCommonDBOptions() and RocksDBResourceContainer.createBaseCommonColumnOptions() for common settings. Note that setUseFsync(false) is set by default irrespective of the PredefinedOptions setting. Because Flink does not rely on RocksDB data on disk for recovery, there is no need to sync data to stable storage.

Enum Constant Summary

Enum Constants

Enum Constant and Description
DEFAULT Default options for all settings.
FLASH_SSD_OPTIMIZED Pre-defined options for Flash SSDs.
SPINNING_DISK_OPTIMIZED Pre-defined options for regular spinning hard disks.
SPINNING_DISK_OPTIMIZED_HIGH_MEM Pre-defined options for better performance on regular spinning hard disks, at the cost of a higher memory consumption.

Method Summary

Modifier and Type	Method and Description
static PredefinedOptions	valueOf(String name) Returns the enum constant of this type with the specified name.
static PredefinedOptions[]	values() Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Detail

DEFAULT

public static final PredefinedOptions DEFAULT

Default options for all settings.

There are no specified options here.

SPINNING_DISK_OPTIMIZED

public static final PredefinedOptions SPINNING_DISK_OPTIMIZED

Pre-defined options for regular spinning hard disks.

This constant configures RocksDB with some options that lead empirically to better performance when the machines executing the system use regular spinning hard disks.

The following options are set:

• setCompactionStyle(CompactionStyle.LEVEL)
• setLevelCompactionDynamicLevelBytes(true)
• setMaxBackgroundJobs(4)
• setMaxOpenFiles(-1)

SPINNING_DISK_OPTIMIZED_HIGH_MEM

public static final PredefinedOptions SPINNING_DISK_OPTIMIZED_HIGH_MEM

Pre-defined options for better performance on regular spinning hard disks, at the cost of a higher memory consumption.

NOTE: These settings will cause RocksDB to consume a lot of memory for block caching and compactions. If you experience out-of-memory problems related to, RocksDB, consider switching back to SPINNING_DISK_OPTIMIZED.

The following options are set:

• BlockBasedTableConfig.setBlockCacheSize(256 MBytes)
• BlockBasedTableConfig.setBlockSize(128 KBytes)
• BlockBasedTableConfig.setFilterPolicy(BloomFilter( RocksDBConfigurableOptions.BLOOM_FILTER_BITS_PER_KEY, RocksDBConfigurableOptions.BLOOM_FILTER_BLOCK_BASED_MODE)
• setLevelCompactionDynamicLevelBytes(true)
• setMaxBackgroundJobs(4)
• setMaxBytesForLevelBase(1 GByte)
• setMaxOpenFiles(-1)
• setMaxWriteBufferNumber(4)
• setMinWriteBufferNumberToMerge(3)
• setTargetFileSizeBase(256 MBytes)
• setWriteBufferSize(64 MBytes)

Enabling use of a Bloom filter here is equivalent to setting RocksDBConfigurableOptions.USE_BLOOM_FILTER.

FLASH_SSD_OPTIMIZED

public static final PredefinedOptions FLASH_SSD_OPTIMIZED

Pre-defined options for Flash SSDs.

This constant configures RocksDB with some options that lead empirically to better performance when the machines executing the system use SSDs.

The following options are set:

• setMaxBackgroundJobs(4)
• setMaxOpenFiles(-1)

Method Detail

values

public static PredefinedOptions[] values()

Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:

for (PredefinedOptions c : PredefinedOptions.values())
    System.out.println(c);

Returns:

an array containing the constants of this enum type, in the order they are declared

valueOf

public static PredefinedOptions valueOf(String name)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null