org.apache.flink.state.forst

Class ForStStateBackend

java.lang.Object

org.apache.flink.runtime.state.AbstractStateBackend

org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend

org.apache.flink.state.forst.ForStStateBackend

All Implemented Interfaces:

Serializable, ConfigurableStateBackend, StateBackend

@Experimental
public class ForStStateBackend
extends AbstractManagedMemoryStateBackend
implements ConfigurableStateBackend

A StateBackend that stores its state in a ForSt instance. This state backend can store very large state that exceeds memory even disk and spills to remote storage.

The behavior of the ForSt instances can be parametrized by setting ForSt Options using the methods setForStOptions(ForStOptionsFactory).

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.StateBackend

StateBackend.CustomInitializationMetrics, StateBackend.KeyedStateBackendParameters<K>, StateBackend.OperatorStateBackendParameters

Field Summary

Fields inherited from class org.apache.flink.runtime.state.AbstractStateBackend

latencyTrackingConfigBuilder

Constructor Summary

Constructors

Constructor and Description
ForStStateBackend() Creates a new ForStStateBackend for storing state.

Method Summary

Modifier and Type	Method and Description
ForStStateBackend	configure(ReadableConfig config, ClassLoader classLoader) Creates a copy of this state backend that uses the values defined in the configuration for fields where that were not yet specified in this state backend.
<K> ForStKeyedStateBackend<K>	createAsyncKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters) Creates a new AsyncKeyedStateBackend which supports to access keyed state asynchronously.
<K> AbstractKeyedStateBackend<K>	createKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters) Creates a new CheckpointableKeyedStateBackend that is responsible for holding keyed state and checkpointing it.
OperatorStateBackend	createOperatorStateBackend(StateBackend.OperatorStateBackendParameters parameters) Creates a new OperatorStateBackend that can be used for storing operator state.
ForStOptionsFactory	getForStOptions() Gets Options for the ForSt instances.
String[]	getLocalDbStoragePaths() Gets the configured local DB storage paths, or null, if none were configured.
void	setForStOptions(ForStOptionsFactory optionsFactory) Sets Options for the ForSt instances.
void	setLocalDbStoragePath(String path) Sets the path where the ForSt local files should be stored on the local file system.
void	setLocalDbStoragePaths(String... paths) Sets the local directories in which the ForSt database puts some files (like metadata files).
boolean	supportsAsyncKeyedStateBackend() Tells if a state backend supports the AsyncKeyedStateBackend.
String	toString()

Methods inherited from class org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend

useManagedMemory

Methods inherited from class org.apache.flink.runtime.state.AbstractStateBackend

getCompressionDecorator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.StateBackend

getName, supportsNoClaimRestoreMode, supportsSavepointFormat, useManagedMemory

Constructor Detail

ForStStateBackend

public ForStStateBackend()

Creates a new ForStStateBackend for storing state.

Method Detail

configure

public ForStStateBackend configure(ReadableConfig config,
                                   ClassLoader classLoader)

Creates a copy of this state backend that uses the values defined in the configuration for fields where that were not yet specified in this state backend.

Specified by:

configure in interface ConfigurableStateBackend

Parameters:

config - The configuration.

classLoader - The class loader.

Returns:

The re-configured variant of the state backend

supportsAsyncKeyedStateBackend

public boolean supportsAsyncKeyedStateBackend()

Description copied from interface: StateBackend

Tells if a state backend supports the AsyncKeyedStateBackend.

If a state backend supports AsyncKeyedStateBackend, it could use StateBackend.createAsyncKeyedStateBackend(KeyedStateBackendParameters) to create an async keyed state backend to access keyed state asynchronously.

Specified by:

supportsAsyncKeyedStateBackend in interface StateBackend

Returns:

If the state backend supports AsyncKeyedStateBackend.

createAsyncKeyedStateBackend

public <K> ForStKeyedStateBackend<K> createAsyncKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters)
                                                           throws IOException

Description copied from interface: StateBackend

Creates a new AsyncKeyedStateBackend which supports to access keyed state asynchronously.

Keyed State is state where each value is bound to a key.

Specified by:

createAsyncKeyedStateBackend in interface StateBackend

Type Parameters:

K - The type of the keys by which the state is organized.

Parameters:

parameters - The arguments bundle for creating AsyncKeyedStateBackend.

Returns:

The Async Keyed State Backend for the given job, operator.

Throws:

IOException

createKeyedStateBackend

public <K> AbstractKeyedStateBackend<K> createKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters)

Description copied from interface: StateBackend

Creates a new CheckpointableKeyedStateBackend that is responsible for holding keyed state and checkpointing it.

Keyed State is state where each value is bound to a key.

Specified by:

createKeyedStateBackend in interface StateBackend

Specified by:

createKeyedStateBackend in class AbstractManagedMemoryStateBackend

Type Parameters:

K - The type of the keys by which the state is organized.

Parameters:

parameters - The arguments bundle for creating CheckpointableKeyedStateBackend.

Returns:

The Keyed State Backend for the given job, operator, and key group range.

createOperatorStateBackend

public OperatorStateBackend createOperatorStateBackend(StateBackend.OperatorStateBackendParameters parameters)
                                                throws Exception

Description copied from interface: StateBackend

Creates a new OperatorStateBackend that can be used for storing operator state.

Operator state is state that is associated with parallel operator (or function) instances, rather than with keys.

Specified by:

createOperatorStateBackend in interface StateBackend

Specified by:

createOperatorStateBackend in class AbstractStateBackend

Parameters:

parameters - The arguments bundle for creating OperatorStateBackend.

Returns:

The OperatorStateBackend for operator identified by the job and operator identifier.

Throws:

Exception - This method may forward all exceptions that occur while instantiating the backend.

setLocalDbStoragePath

public void setLocalDbStoragePath(String path)

Sets the path where the ForSt local files should be stored on the local file system. Setting this path overrides the default behavior, where the files are stored across the configured temp directories.

Passing null to this function restores the default behavior, where the configured temp directories will be used.

Parameters:

path - The path where the local ForSt database files are stored.

setLocalDbStoragePaths

public void setLocalDbStoragePaths(String... paths)

Sets the local directories in which the ForSt database puts some files (like metadata files). These directories do not need to be persistent, they can be ephemeral, meaning that they are lost on a machine failure, because state in ForSt is persisted in checkpoints.

If nothing is configured, these directories default to the TaskManager's local temporary file directories.

Each distinct state will be stored in one path, but when the state backend creates multiple states, they will store their files on different paths.

Passing null to this function restores the default behavior, where the configured temp directories will be used.

Parameters:

paths - The paths across which the local ForSt database files will be spread.

getLocalDbStoragePaths

public String[] getLocalDbStoragePaths()

Gets the configured local DB storage paths, or null, if none were configured.

Under these directories on the TaskManager, ForSt stores some metadata files. These directories do not need to be persistent, they can be ephermeral, meaning that they are lost on a machine failure, because state in ForSt is persisted in checkpoints.

If nothing is configured, these directories default to the TaskManager's local temporary file directories.

setForStOptions

public void setForStOptions(ForStOptionsFactory optionsFactory)

Sets Options for the ForSt instances. Because the options are not serializable and hold native code references, they must be specified through a factory.

The options created by the factory here are applied on top of user-configured options from configuration set by configure(ReadableConfig, ClassLoader) with keys in ForStConfigurableOptions.

Parameters:

optionsFactory - The options factory that lazily creates the ForSt options.

getForStOptions

@Nullable
public ForStOptionsFactory getForStOptions()

Gets Options for the ForSt instances.

toString

public String toString()

Overrides:

toString in class Object