@Experimental public class ForStStateBackend extends AbstractManagedMemoryStateBackend implements ConfigurableStateBackend
StateBackend
that stores its state in a ForSt instance.
This state backend can store very large state that exceeds memory even disk and spills to remote
storage.
The behavior of the ForSt instances can be parametrized by setting ForSt Options using the
methods setForStOptions(ForStOptionsFactory)
.
StateBackend.CustomInitializationMetrics, StateBackend.KeyedStateBackendParameters<K>, StateBackend.OperatorStateBackendParameters
latencyTrackingConfigBuilder
Constructor and Description |
---|
ForStStateBackend()
Creates a new
ForStStateBackend for storing state. |
Modifier and Type | Method and Description |
---|---|
ForStStateBackend |
configure(ReadableConfig config,
ClassLoader classLoader)
Creates a copy of this state backend that uses the values defined in the configuration for
fields where that were not yet specified in this state backend.
|
<K> ForStKeyedStateBackend<K> |
createAsyncKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters)
Creates a new
AsyncKeyedStateBackend which supports to access keyed state
asynchronously. |
<K> AbstractKeyedStateBackend<K> |
createKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters)
Creates a new
CheckpointableKeyedStateBackend that is responsible for holding
keyed state and checkpointing it. |
OperatorStateBackend |
createOperatorStateBackend(StateBackend.OperatorStateBackendParameters parameters)
Creates a new
OperatorStateBackend that can be used for storing operator state. |
ForStOptionsFactory |
getForStOptions()
Gets
Options for the ForSt instances. |
String[] |
getLocalDbStoragePaths()
Gets the configured local DB storage paths, or null, if none were configured.
|
void |
setForStOptions(ForStOptionsFactory optionsFactory)
Sets
Options for the ForSt instances. |
void |
setLocalDbStoragePath(String path)
Sets the path where the ForSt local files should be stored on the local file system.
|
void |
setLocalDbStoragePaths(String... paths)
Sets the local directories in which the ForSt database puts some files (like metadata files).
|
boolean |
supportsAsyncKeyedStateBackend()
Tells if a state backend supports the
AsyncKeyedStateBackend . |
String |
toString() |
useManagedMemory
getCompressionDecorator
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
getName, supportsNoClaimRestoreMode, supportsSavepointFormat, useManagedMemory
public ForStStateBackend()
ForStStateBackend
for storing state.public ForStStateBackend configure(ReadableConfig config, ClassLoader classLoader)
configure
in interface ConfigurableStateBackend
config
- The configuration.classLoader
- The class loader.public boolean supportsAsyncKeyedStateBackend()
StateBackend
AsyncKeyedStateBackend
.
If a state backend supports AsyncKeyedStateBackend
, it could use StateBackend.createAsyncKeyedStateBackend(KeyedStateBackendParameters)
to create an async keyed state
backend to access keyed state asynchronously.
supportsAsyncKeyedStateBackend
in interface StateBackend
AsyncKeyedStateBackend
.public <K> ForStKeyedStateBackend<K> createAsyncKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters) throws IOException
StateBackend
AsyncKeyedStateBackend
which supports to access keyed state
asynchronously.
Keyed State is state where each value is bound to a key.
createAsyncKeyedStateBackend
in interface StateBackend
K
- The type of the keys by which the state is organized.parameters
- The arguments bundle for creating AsyncKeyedStateBackend
.IOException
public <K> AbstractKeyedStateBackend<K> createKeyedStateBackend(StateBackend.KeyedStateBackendParameters<K> parameters)
StateBackend
CheckpointableKeyedStateBackend
that is responsible for holding
keyed state and checkpointing it.
Keyed State is state where each value is bound to a key.
createKeyedStateBackend
in interface StateBackend
createKeyedStateBackend
in class AbstractManagedMemoryStateBackend
K
- The type of the keys by which the state is organized.parameters
- The arguments bundle for creating CheckpointableKeyedStateBackend
.public OperatorStateBackend createOperatorStateBackend(StateBackend.OperatorStateBackendParameters parameters) throws Exception
StateBackend
OperatorStateBackend
that can be used for storing operator state.
Operator state is state that is associated with parallel operator (or function) instances, rather than with keys.
createOperatorStateBackend
in interface StateBackend
createOperatorStateBackend
in class AbstractStateBackend
parameters
- The arguments bundle for creating OperatorStateBackend
.Exception
- This method may forward all exceptions that occur while instantiating the
backend.public void setLocalDbStoragePath(String path)
Passing null
to this function restores the default behavior, where the configured
temp directories will be used.
path
- The path where the local ForSt database files are stored.public void setLocalDbStoragePaths(String... paths)
If nothing is configured, these directories default to the TaskManager's local temporary file directories.
Each distinct state will be stored in one path, but when the state backend creates multiple states, they will store their files on different paths.
Passing null
to this function restores the default behavior, where the configured
temp directories will be used.
paths
- The paths across which the local ForSt database files will be spread.public String[] getLocalDbStoragePaths()
Under these directories on the TaskManager, ForSt stores some metadata files. These directories do not need to be persistent, they can be ephermeral, meaning that they are lost on a machine failure, because state in ForSt is persisted in checkpoints.
If nothing is configured, these directories default to the TaskManager's local temporary file directories.
public void setForStOptions(ForStOptionsFactory optionsFactory)
Options
for the ForSt instances. Because the options are not
serializable and hold native code references, they must be specified through a factory.
The options created by the factory here are applied on top of user-configured options from
configuration set by configure(ReadableConfig, ClassLoader)
with keys in ForStConfigurableOptions
.
optionsFactory
- The options factory that lazily creates the ForSt options.@Nullable public ForStOptionsFactory getForStOptions()
Options
for the ForSt instances.Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.