pyflink.datastream.state_backend.EmbeddedRocksDBStateBackend

class EmbeddedRocksDBStateBackend(enable_incremental_checkpointing=None, j_embedded_rocks_db_state_backend=None)

A State Backend that stores its state in an embedded RocksDB instance. This state backend can store very large state that exceeds memory and spills to local disk.

All key/value state (including windows) is stored in the key/value index of RocksDB. For persistence against loss of machines, please configure a CheckpointStorage instance for the Job.

The behavior of the RocksDB instances can be parametrized by setting RocksDB Options using the methods set_predefined_options() and set_options().

Methods

get_db_storage_paths()	Gets the configured local DB storage paths, or null, if none were configured.
get_number_of_transfer_threads()	Gets the number of threads used to transfer files while snapshotting/restoring.
get_options()	Gets the fully-qualified class name of the options factory in Java that lazily creates the RocksDB options.
get_predefined_options()	Gets the current predefined options for RocksDB.
is_incremental_checkpoints_enabled()	Gets whether incremental checkpoints are enabled for this state backend.
set_db_storage_paths(*paths)	Sets the directories in which the local RocksDB database puts its files (like SST and metadata files).
set_number_of_transfer_threads(...)	Sets the number of threads used to transfer files while snapshotting/restoring.
set_options(options_factory_class_name)	Sets org.rocksdb.Options for the RocksDB instances.
set_predefined_options(options)	Sets the predefined options for RocksDB.