pyflink.datastream.state_backend.RocksDBStateBackend

class RocksDBStateBackend(checkpoint_data_uri=None, enable_incremental_checkpointing=None, checkpoint_stream_backend=None, j_rocks_db_state_backend=None)

IMPORTANT RocksDBStateBackend is deprecated in favor of EmbeddedRocksDBStateBackend and FileSystemCheckpointStorage. This change does not affect the runtime characteristics of your Jobs and is simply an API change to help better communicate the ways Flink separates local state storage from fault tolerance. Jobs can be upgraded without loss of state. If configuring your state backend via the StreamExecutionEnvironment please make the following changes.

>> env.set_state_backend(EmbeddedRocksDBStateBackend())
>> env.get_checkpoint_config().set_checkpoint_storage("hdfs://checkpoints")

If you are configuring your state backend via the flink-conf.yaml no changes are required.

A State Backend that stores its state in RocksDB. This state backend can store very large state that exceeds memory and spills to disk.

All key/value state (including windows) is stored in the key/value index of RocksDB. For persistence against loss of machines, checkpoints take a snapshot of the RocksDB database, and persist that snapshot in a file system (by default) or another configurable state backend.

The behavior of the RocksDB instances can be parametrized by setting RocksDB Options using the methods set_predefined_options() and set_options().

Methods

get_checkpoint_backend()	Gets the state backend that this RocksDB state backend uses to persist its bytes to.
get_db_storage_paths()	Gets the configured local DB storage paths, or null, if none were configured.
get_number_of_transfering_threads()	Gets the number of threads used to transfer files while snapshotting/restoring.
get_options()	Gets the fully-qualified class name of the options factory in Java that lazily creates the RocksDB options.
get_predefined_options()	Gets the current predefined options for RocksDB.
is_incremental_checkpoints_enabled()	Gets whether incremental checkpoints are enabled for this state backend.
set_db_storage_paths(*paths)	Sets the directories in which the local RocksDB database puts its files (like SST and metadata files).
set_number_of_transfering_threads(...)	Sets the number of threads used to transfer files while snapshotting/restoring.
set_options(options_factory_class_name)	Sets org.rocksdb.Options for the RocksDB instances.
set_predefined_options(options)	Sets the predefined options for RocksDB.