@PublicEvolving public abstract class AbstractStateBackend extends Object implements StateBackend, Serializable
StateBackend
interface.
Modifier and Type | Field and Description |
---|---|
static String |
FS_STATE_BACKEND_NAME
The shortcut configuration name for the FileSystem State backend
|
static String |
MEMORY_STATE_BACKEND_NAME
The shortcut configuration name for the MemoryState backend that checkpoints to the JobManager
|
static String |
ROCKSDB_STATE_BACKEND_NAME
The shortcut configuration name for the RocksDB State Backend
|
Constructor and Description |
---|
AbstractStateBackend() |
Modifier and Type | Method and Description |
---|---|
abstract <K> AbstractKeyedStateBackend<K> |
createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry)
Creates a new
AbstractKeyedStateBackend that is responsible for holding keyed state
and checkpointing it. |
abstract OperatorStateBackend |
createOperatorStateBackend(Environment env,
String operatorIdentifier)
Creates a new
OperatorStateBackend that can be used for storing operator state. |
abstract CheckpointStreamFactory |
createSavepointStreamFactory(JobID jobId,
String operatorIdentifier,
String targetLocation)
Creates a
CheckpointStreamFactory that can be used to create streams
that should end up in a savepoint. |
abstract CheckpointStreamFactory |
createStreamFactory(JobID jobId,
String operatorIdentifier)
Creates a
CheckpointStreamFactory that can be used to create streams
that should end up in a checkpoint. |
static StateBackend |
loadStateBackendFromConfig(Configuration config,
ClassLoader classLoader,
org.slf4j.Logger logger)
Loads the state backend from the configuration, from the parameter 'state.backend', as defined
in
CoreOptions.STATE_BACKEND . |
static StateBackend |
loadStateBackendFromConfigOrCreateDefault(Configuration config,
ClassLoader classLoader,
org.slf4j.Logger logger)
Loads the state backend from the configuration, from the parameter 'state.backend', as defined
in
CoreOptions.STATE_BACKEND . |
public static final String MEMORY_STATE_BACKEND_NAME
public static final String FS_STATE_BACKEND_NAME
public static final String ROCKSDB_STATE_BACKEND_NAME
public abstract CheckpointStreamFactory createStreamFactory(JobID jobId, String operatorIdentifier) throws IOException
StateBackend
CheckpointStreamFactory
that can be used to create streams
that should end up in a checkpoint.createStreamFactory
in interface StateBackend
jobId
- The JobID
of the job for which we are creating checkpoint streams.operatorIdentifier
- An identifier of the operator for which we create streams.IOException
public abstract CheckpointStreamFactory createSavepointStreamFactory(JobID jobId, String operatorIdentifier, @Nullable String targetLocation) throws IOException
StateBackend
CheckpointStreamFactory
that can be used to create streams
that should end up in a savepoint.
This is only called if the triggered checkpoint is a savepoint. Commonly this will return the same factory as for regular checkpoints, but maybe slightly adjusted.
createSavepointStreamFactory
in interface StateBackend
jobId
- The JobID
of the job for which we are creating checkpoint streams.operatorIdentifier
- An identifier of the operator for which we create streams.targetLocation
- An optional custom location for the savepoint stream.IOException
- Failures during stream creation are forwarded.public abstract <K> AbstractKeyedStateBackend<K> createKeyedStateBackend(Environment env, JobID jobID, String operatorIdentifier, TypeSerializer<K> keySerializer, int numberOfKeyGroups, KeyGroupRange keyGroupRange, TaskKvStateRegistry kvStateRegistry) throws IOException
StateBackend
AbstractKeyedStateBackend
that is responsible for holding keyed state
and checkpointing it.
Keyed State is state where each value is bound to a key.
createKeyedStateBackend
in interface StateBackend
K
- The type of the keys by which the state is organized.IOException
public abstract OperatorStateBackend createOperatorStateBackend(Environment env, String operatorIdentifier) throws Exception
StateBackend
OperatorStateBackend
that can be used for storing operator state.
Operator state is state that is associated with parallel operator (or function) instances, rather than with keys.
createOperatorStateBackend
in interface StateBackend
env
- The runtime environment of the executing task.operatorIdentifier
- The identifier of the operator whose state should be stored.Exception
- This method may forward all exceptions that occur while instantiating the backend.public static StateBackend loadStateBackendFromConfig(Configuration config, ClassLoader classLoader, @Nullable org.slf4j.Logger logger) throws IllegalConfigurationException, DynamicCodeLoadingException, IOException
CoreOptions.STATE_BACKEND
.
The state backends can be specified either via their shortcut name, or via the class name
of a StateBackendFactory
. If a StateBackendFactory class name is specified, the factory
is instantiated (via its zero-argument constructor) and its
StateBackendFactory.createFromConfig(Configuration)
method is called.
Recognized shortcut names are '', '', and ''.
config
- The configuration to load the state backend fromclassLoader
- The class loader that should be used to load the state backendlogger
- Optionally, a logger to log actions to (may be null)DynamicCodeLoadingException
- Thrown if a state backend factory is configured and the factory class was not
found or the factory could not be instantiatedIllegalConfigurationException
- May be thrown by the StateBackendFactory when creating / configuring the state
backend in the factoryIOException
- May be thrown by the StateBackendFactory when instantiating the state backendpublic static StateBackend loadStateBackendFromConfigOrCreateDefault(Configuration config, ClassLoader classLoader, @Nullable org.slf4j.Logger logger) throws IllegalConfigurationException, DynamicCodeLoadingException, IOException
CoreOptions.STATE_BACKEND
. If no state backend is configures, this instantiates the
default state backend (the MemoryStateBackend
).
Refer to loadStateBackendFromConfig(Configuration, ClassLoader, Logger)
for details on
how the state backend is loaded from the configuration.
config
- The configuration to load the state backend fromclassLoader
- The class loader that should be used to load the state backendlogger
- Optionally, a logger to log actions to (may be null)DynamicCodeLoadingException
- Thrown if a state backend factory is configured and the factory class was not
found or the factory could not be instantiatedIllegalConfigurationException
- May be thrown by the StateBackendFactory when creating / configuring the state
backend in the factoryIOException
- May be thrown by the StateBackendFactory when instantiating the state backendCopyright © 2014–2018 The Apache Software Foundation. All rights reserved.