Package org.apache.flink.state.rocksdb

Class EmbeddedRocksDBStateBackend

    • Constructor Detail

      • EmbeddedRocksDBStateBackend

        public EmbeddedRocksDBStateBackend()
        Creates a new EmbeddedRocksDBStateBackend for storing local state.

      • EmbeddedRocksDBStateBackend

        public EmbeddedRocksDBStateBackend​(boolean enableIncrementalCheckpointing)
        Creates a new EmbeddedRocksDBStateBackend for storing local state.
        Parameters:
        enableIncrementalCheckpointing - True if incremental checkpointing is enabled.

      • EmbeddedRocksDBStateBackend

        public EmbeddedRocksDBStateBackend​(TernaryBoolean enableIncrementalCheckpointing)
        Creates a new EmbeddedRocksDBStateBackend for storing local state.
        Parameters:
        enableIncrementalCheckpointing - True if incremental checkpointing is enabled.

      • EmbeddedRocksDBStateBackend

        protected EmbeddedRocksDBStateBackend​(EmbeddedRocksDBStateBackend original,
                                      ReadableConfig config,
                                      ClassLoader classLoader)
        Private constructor that creates a re-configured copy of the state backend.
        Parameters:
        original - The state backend to re-configure.
        config - The configuration.
        classLoader - The class loader.

    • Method Detail

      • configure

        public EmbeddedRocksDBStateBackend configure​(ReadableConfig config,
                                             ClassLoader classLoader)
        Creates a copy of this state backend that uses the values defined in the configuration for fields where that were not yet specified in this state backend.
        Specified by:
        configure in interface ConfigurableStateBackend
        Parameters:
        config - The configuration.
        classLoader - The class loader.
        Returns:
        The re-configured variant of the state backend

      • getMemoryConfiguration

        public RocksDBMemoryConfiguration getMemoryConfiguration()
        Gets the memory configuration object, which offers settings to control RocksDB's memory usage.

      • setDbStoragePath

        public void setDbStoragePath​(String path)
        Sets the path where the RocksDB local database files should be stored on the local file system. Setting this path overrides the default behavior, where the files are stored across the configured temp directories.

        Passing null to this function restores the default behavior, where the configured temp directories will be used.

        Parameters:
        path - The path where the local RocksDB database files are stored.

      • setDbStoragePaths

        public void setDbStoragePaths​(String... paths)
        Sets the directories in which the local RocksDB database puts its files (like SST and metadata files). These directories do not need to be persistent, they can be ephemeral, meaning that they are lost on a machine failure, because state in RocksDB is persisted in checkpoints.

        If nothing is configured, these directories default to the TaskManager's local temporary file directories.

        Each distinct state will be stored in one path, but when the state backend creates multiple states, they will store their files on different paths.

        Passing null to this function restores the default behavior, where the configured temp directories will be used.

        Parameters:
        paths - The paths across which the local RocksDB database files will be spread.

      • getDbStoragePaths

        public String[] getDbStoragePaths()
        Gets the configured local DB storage paths, or null, if none were configured.

        Under these directories on the TaskManager, RocksDB stores its SST files and metadata files. These directories do not need to be persistent, they can be ephermeral, meaning that they are lost on a machine failure, because state in RocksDB is persisted in checkpoints.

        If nothing is configured, these directories default to the TaskManager's local temporary file directories.

      • isIncrementalCheckpointsEnabled

        public boolean isIncrementalCheckpointsEnabled()
        Gets whether incremental checkpoints are enabled for this state backend.

      • getPriorityQueueStateType

        public EmbeddedRocksDBStateBackend.PriorityQueueStateType getPriorityQueueStateType()
        Gets the type of the priority queue state. It will fallback to the default value, if it is not explicitly set.
        Returns:
        The type of the priority queue state.

      • setPriorityQueueStateType

        public void setPriorityQueueStateType​(EmbeddedRocksDBStateBackend.PriorityQueueStateType priorityQueueStateType)
        Sets the type of the priority queue state. It will fallback to the default value, if it is not explicitly set.

      • setPredefinedOptions

        public void setPredefinedOptions​(@Nonnull
                                 PredefinedOptions options)
        Sets the predefined options for RocksDB.

        If user-configured options within RocksDBConfigurableOptions is set (through config.yaml) or a user-defined options factory is set (via setRocksDBOptions(RocksDBOptionsFactory)), then the options from the factory are applied on top of the here specified predefined options and customized options.

        Parameters:
        options - The options to set (must not be null).

      • setRocksDBOptions

        public void setRocksDBOptions​(RocksDBOptionsFactory optionsFactory)
        Sets Options for the RocksDB instances. Because the options are not serializable and hold native code references, they must be specified through a factory.

        The options created by the factory here are applied on top of the pre-defined options profile selected via setPredefinedOptions(PredefinedOptions) and user-configured options from configuration set by configure(ReadableConfig, ClassLoader) with keys in RocksDBConfigurableOptions.

        Parameters:
        optionsFactory - The options factory that lazily creates the RocksDB options.

      • getNumberOfTransferThreads

        public int getNumberOfTransferThreads()
        Gets the number of threads used to transfer files while snapshotting/restoring.

      • setNumberOfTransferThreads

        public void setNumberOfTransferThreads​(int numberOfTransferThreads)
        Sets the number of threads used to transfer files while snapshotting/restoring.
        Parameters:
        numberOfTransferThreads - The number of threads used to transfer files while snapshotting/restoring.

      • setWriteBatchSize

        public void setWriteBatchSize​(long writeBatchSize)
        Sets the max batch size will be used in RocksDBWriteBatchWrapper, no positive value will disable memory size controller, just use item count controller.
        Parameters:
        writeBatchSize - The size will used to be used in RocksDBWriteBatchWrapper.