public class MemoryStateBackend extends AbstractStateBackend
AbstractStateBackend
that stores all its data and checkpoints in memory and has no
capabilities to spill to disk. Checkpoints are serialized and the serialized data is
transferredFS_STATE_BACKEND_NAME, MEMORY_STATE_BACKEND_NAME, ROCKSDB_STATE_BACKEND_NAME
Constructor and Description |
---|
MemoryStateBackend()
Creates a new memory state backend that accepts states whose serialized forms are
up to the default state size (5 MB).
|
MemoryStateBackend(boolean asynchronousSnapshots)
Creates a new memory state backend that accepts states whose serialized forms are
up to the default state size (5 MB).
|
MemoryStateBackend(int maxStateSize)
Creates a new memory state backend that accepts states whose serialized forms are
up to the given number of bytes.
|
MemoryStateBackend(int maxStateSize,
boolean asynchronousSnapshots)
Creates a new memory state backend that accepts states whose serialized forms are
up to the given number of bytes.
|
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry)
Creates a new
AbstractKeyedStateBackend that is responsible for holding keyed state
and checkpointing it. |
OperatorStateBackend |
createOperatorStateBackend(Environment env,
String operatorIdentifier)
Creates a new
OperatorStateBackend that can be used for storing operator state. |
CheckpointStreamFactory |
createSavepointStreamFactory(JobID jobId,
String operatorIdentifier,
String targetLocation)
Creates a
CheckpointStreamFactory that can be used to create streams
that should end up in a savepoint. |
CheckpointStreamFactory |
createStreamFactory(JobID jobId,
String operatorIdentifier)
Creates a
CheckpointStreamFactory that can be used to create streams
that should end up in a checkpoint. |
String |
toString() |
loadStateBackendFromConfig, loadStateBackendFromConfigOrCreateDefault
public MemoryStateBackend()
public MemoryStateBackend(int maxStateSize)
maxStateSize
- The maximal size of the serialized statepublic MemoryStateBackend(boolean asynchronousSnapshots)
asynchronousSnapshots
- Switch to enable asynchronous snapshots.public MemoryStateBackend(int maxStateSize, boolean asynchronousSnapshots)
maxStateSize
- The maximal size of the serialized stateasynchronousSnapshots
- Switch to enable asynchronous snapshots.public OperatorStateBackend createOperatorStateBackend(Environment env, String operatorIdentifier) throws Exception
StateBackend
OperatorStateBackend
that can be used for storing operator state.
Operator state is state that is associated with parallel operator (or function) instances, rather than with keys.
createOperatorStateBackend
in interface StateBackend
createOperatorStateBackend
in class AbstractStateBackend
env
- The runtime environment of the executing task.operatorIdentifier
- The identifier of the operator whose state should be stored.Exception
- This method may forward all exceptions that occur while instantiating the backend.public CheckpointStreamFactory createStreamFactory(JobID jobId, String operatorIdentifier) throws IOException
StateBackend
CheckpointStreamFactory
that can be used to create streams
that should end up in a checkpoint.createStreamFactory
in interface StateBackend
createStreamFactory
in class AbstractStateBackend
jobId
- The JobID
of the job for which we are creating checkpoint streams.operatorIdentifier
- An identifier of the operator for which we create streams.IOException
public CheckpointStreamFactory createSavepointStreamFactory(JobID jobId, String operatorIdentifier, String targetLocation) throws IOException
StateBackend
CheckpointStreamFactory
that can be used to create streams
that should end up in a savepoint.
This is only called if the triggered checkpoint is a savepoint. Commonly this will return the same factory as for regular checkpoints, but maybe slightly adjusted.
createSavepointStreamFactory
in interface StateBackend
createSavepointStreamFactory
in class AbstractStateBackend
jobId
- The JobID
of the job for which we are creating checkpoint streams.operatorIdentifier
- An identifier of the operator for which we create streams.targetLocation
- An optional custom location for the savepoint stream.IOException
- Failures during stream creation are forwarded.public <K> AbstractKeyedStateBackend<K> createKeyedStateBackend(Environment env, JobID jobID, String operatorIdentifier, TypeSerializer<K> keySerializer, int numberOfKeyGroups, KeyGroupRange keyGroupRange, TaskKvStateRegistry kvStateRegistry)
StateBackend
AbstractKeyedStateBackend
that is responsible for holding keyed state
and checkpointing it.
Keyed State is state where each value is bound to a key.
createKeyedStateBackend
in interface StateBackend
createKeyedStateBackend
in class AbstractStateBackend
K
- The type of the keys by which the state is organized.Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.