Class ForStStateBackend
- java.lang.Object
-
- org.apache.flink.runtime.state.AbstractStateBackend
-
- org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend
-
- org.apache.flink.state.forst.ForStStateBackend
-
- All Implemented Interfaces:
Serializable
,ConfigurableStateBackend
,StateBackend
@Experimental public class ForStStateBackend extends AbstractManagedMemoryStateBackend implements ConfigurableStateBackend
AStateBackend
that stores its state in a ForSt instance. This state backend can store very large state that exceeds memory even disk and spills to remote storage.The behavior of the ForSt instances can be parametrized by setting ForSt Options using the methods
setForStOptions(ForStOptionsFactory)
.- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
ForStStateBackend.PriorityQueueStateType
The options to chose for the type of priority queue state.-
Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.StateBackend
StateBackend.CustomInitializationMetrics, StateBackend.KeyedStateBackendParameters<K>, StateBackend.OperatorStateBackendParameters
-
-
Field Summary
-
Fields inherited from class org.apache.flink.runtime.state.AbstractStateBackend
latencyTrackingConfigBuilder
-
-
Constructor Summary
Constructors Constructor Description ForStStateBackend()
Creates a newForStStateBackend
for storing state.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description ForStStateBackend
configure(ReadableConfig config, ClassLoader classLoader)
Creates a copy of this state backend that uses the values defined in the configuration for fields where that were not yet specified in this state backend.<K> ForStKeyedStateBackend<K>
createAsyncKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters)
Creates a newAsyncKeyedStateBackend
which supports to access keyed state asynchronously.<K> AbstractKeyedStateBackend<K>
createKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters)
Creates a newCheckpointableKeyedStateBackend
that is responsible for holding keyed state and checkpointing it.OperatorStateBackend
createOperatorStateBackend(StateBackend.OperatorStateBackendParameters parameters)
Creates a newOperatorStateBackend
that can be used for storing operator state.ForStOptionsFactory
getForStOptions()
GetsOptions
for the ForSt instances.String[]
getLocalDbStoragePaths()
Gets the configured local DB storage paths, or null, if none were configured.void
setForStOptions(ForStOptionsFactory optionsFactory)
SetsOptions
for the ForSt instances.void
setLocalDbStoragePath(String path)
Sets the path where the ForSt local files should be stored on the local file system.void
setLocalDbStoragePaths(String... paths)
Sets the local directories in which the ForSt database puts some files (like metadata files).boolean
supportsAsyncKeyedStateBackend()
Tells if a state backend supports theAsyncKeyedStateBackend
.String
toString()
-
Methods inherited from class org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend
useManagedMemory
-
Methods inherited from class org.apache.flink.runtime.state.AbstractStateBackend
getCompressionDecorator
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.apache.flink.runtime.state.StateBackend
getName, supportsNoClaimRestoreMode, supportsSavepointFormat, useManagedMemory
-
-
-
-
Method Detail
-
configure
public ForStStateBackend configure(ReadableConfig config, ClassLoader classLoader)
Creates a copy of this state backend that uses the values defined in the configuration for fields where that were not yet specified in this state backend.- Specified by:
configure
in interfaceConfigurableStateBackend
- Parameters:
config
- The configuration.classLoader
- The class loader.- Returns:
- The re-configured variant of the state backend
-
supportsAsyncKeyedStateBackend
public boolean supportsAsyncKeyedStateBackend()
Description copied from interface:StateBackend
Tells if a state backend supports theAsyncKeyedStateBackend
.If a state backend supports
AsyncKeyedStateBackend
, it could useStateBackend.createAsyncKeyedStateBackend(KeyedStateBackendParameters)
to create an async keyed state backend to access keyed state asynchronously.- Specified by:
supportsAsyncKeyedStateBackend
in interfaceStateBackend
- Returns:
- If the state backend supports
AsyncKeyedStateBackend
.
-
createAsyncKeyedStateBackend
public <K> ForStKeyedStateBackend<K> createAsyncKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters) throws IOException
Description copied from interface:StateBackend
Creates a newAsyncKeyedStateBackend
which supports to access keyed state asynchronously.Keyed State is state where each value is bound to a key.
- Specified by:
createAsyncKeyedStateBackend
in interfaceStateBackend
- Type Parameters:
K
- The type of the keys by which the state is organized.- Parameters:
parameters
- The arguments bundle for creatingAsyncKeyedStateBackend
.- Returns:
- The Async Keyed State Backend for the given job, operator.
- Throws:
IOException
-
createKeyedStateBackend
public <K> AbstractKeyedStateBackend<K> createKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters) throws IOException
Description copied from interface:StateBackend
Creates a newCheckpointableKeyedStateBackend
that is responsible for holding keyed state and checkpointing it.Keyed State is state where each value is bound to a key.
- Specified by:
createKeyedStateBackend
in interfaceStateBackend
- Specified by:
createKeyedStateBackend
in classAbstractManagedMemoryStateBackend
- Type Parameters:
K
- The type of the keys by which the state is organized.- Parameters:
parameters
- The arguments bundle for creatingCheckpointableKeyedStateBackend
.- Returns:
- The Keyed State Backend for the given job, operator, and key group range.
- Throws:
IOException
-
createOperatorStateBackend
public OperatorStateBackend createOperatorStateBackend(StateBackend.OperatorStateBackendParameters parameters) throws Exception
Description copied from interface:StateBackend
Creates a newOperatorStateBackend
that can be used for storing operator state.Operator state is state that is associated with parallel operator (or function) instances, rather than with keys.
- Specified by:
createOperatorStateBackend
in interfaceStateBackend
- Specified by:
createOperatorStateBackend
in classAbstractStateBackend
- Parameters:
parameters
- The arguments bundle for creatingOperatorStateBackend
.- Returns:
- The OperatorStateBackend for operator identified by the job and operator identifier.
- Throws:
Exception
- This method may forward all exceptions that occur while instantiating the backend.
-
setLocalDbStoragePath
public void setLocalDbStoragePath(String path)
Sets the path where the ForSt local files should be stored on the local file system. Setting this path overrides the default behavior, where the files are stored across the configured temp directories.Passing
null
to this function restores the default behavior, where the configured temp directories will be used.- Parameters:
path
- The path where the local ForSt database files are stored.
-
setLocalDbStoragePaths
public void setLocalDbStoragePaths(String... paths)
Sets the local directories in which the ForSt database puts some files (like metadata files). These directories do not need to be persistent, they can be ephemeral, meaning that they are lost on a machine failure, because state in ForSt is persisted in checkpoints.If nothing is configured, these directories default to the TaskManager's local temporary file directories.
Each distinct state will be stored in one path, but when the state backend creates multiple states, they will store their files on different paths.
Passing
null
to this function restores the default behavior, where the configured temp directories will be used.- Parameters:
paths
- The paths across which the local ForSt database files will be spread.
-
getLocalDbStoragePaths
public String[] getLocalDbStoragePaths()
Gets the configured local DB storage paths, or null, if none were configured.Under these directories on the TaskManager, ForSt stores some metadata files. These directories do not need to be persistent, they can be ephermeral, meaning that they are lost on a machine failure, because state in ForSt is persisted in checkpoints.
If nothing is configured, these directories default to the TaskManager's local temporary file directories.
-
setForStOptions
public void setForStOptions(ForStOptionsFactory optionsFactory)
SetsOptions
for the ForSt instances. Because the options are not serializable and hold native code references, they must be specified through a factory.The options created by the factory here are applied on top of user-configured options from configuration set by
configure(ReadableConfig, ClassLoader)
with keys inForStConfigurableOptions
.- Parameters:
optionsFactory
- The options factory that lazily creates the ForSt options.
-
getForStOptions
@Nullable public ForStOptionsFactory getForStOptions()
GetsOptions
for the ForSt instances.
-
-