@Internal public class DefaultOperatorStateBackend extends Object implements OperatorStateBackend
| Modifier and Type | Field and Description |
|---|---|
static String |
DEFAULT_OPERATOR_STATE_NAME
The default namespace for state in cases where no state name is provided
|
| Constructor and Description |
|---|
DefaultOperatorStateBackend(ExecutionConfig executionConfig,
CloseableRegistry closeStreamOnCancelRegistry,
Map<String,PartitionableListState<?>> registeredOperatorStates,
Map<String,BackendWritableBroadcastState<?,?>> registeredBroadcastStates,
Map<String,PartitionableListState<?>> accessedStatesByName,
Map<String,BackendWritableBroadcastState<?,?>> accessedBroadcastStatesByName,
SnapshotStrategyRunner<OperatorStateHandle,?> snapshotStrategyRunner) |
| Modifier and Type | Method and Description |
|---|---|
void |
close() |
void |
dispose()
Disposes the object and releases all resources.
|
<K,V> BroadcastState<K,V> |
getBroadcastState(MapStateDescriptor<K,V> stateDescriptor)
Creates (or restores) a
broadcast state. |
ExecutionConfig |
getExecutionConfig() |
<S> ListState<S> |
getListState(ListStateDescriptor<S> stateDescriptor)
Creates (or restores) a list state.
|
Set<String> |
getRegisteredBroadcastStateNames()
Returns a set with the names of all currently registered broadcast states.
|
Set<String> |
getRegisteredStateNames()
Returns a set with the names of all currently registered states.
|
<S> ListState<S> |
getUnionListState(ListStateDescriptor<S> stateDescriptor)
Creates (or restores) a list state.
|
RunnableFuture<SnapshotResult<OperatorStateHandle>> |
snapshot(long checkpointId,
long timestamp,
CheckpointStreamFactory streamFactory,
CheckpointOptions checkpointOptions)
Operation that writes a snapshot into a stream that is provided by the given
CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to
the snapshot. |
public static final String DEFAULT_OPERATOR_STATE_NAME
public DefaultOperatorStateBackend(ExecutionConfig executionConfig, CloseableRegistry closeStreamOnCancelRegistry, Map<String,PartitionableListState<?>> registeredOperatorStates, Map<String,BackendWritableBroadcastState<?,?>> registeredBroadcastStates, Map<String,PartitionableListState<?>> accessedStatesByName, Map<String,BackendWritableBroadcastState<?,?>> accessedBroadcastStatesByName, SnapshotStrategyRunner<OperatorStateHandle,?> snapshotStrategyRunner)
public ExecutionConfig getExecutionConfig()
public Set<String> getRegisteredStateNames()
OperatorStateStoregetRegisteredStateNames in interface OperatorStateStorepublic Set<String> getRegisteredBroadcastStateNames()
OperatorStateStoregetRegisteredBroadcastStateNames in interface OperatorStateStorepublic void close()
throws IOException
close in interface Closeableclose in interface AutoCloseableIOExceptionpublic void dispose()
Disposabledispose in interface OperatorStateBackenddispose in interface Disposablepublic <K,V> BroadcastState<K,V> getBroadcastState(MapStateDescriptor<K,V> stateDescriptor) throws StateMigrationException
OperatorStateStorebroadcast state. This type of state can only
be created to store the state of a BroadcastStream. Each state is registered under a
unique name. The provided serializer is used to de/serialize the state in case of
checkpointing (snapshot/restore). The returned broadcast state has key-value format.
CAUTION: the user has to guarantee that all task instances store the same elements in this type of state.
Each operator instance individually maintains and stores elements in the broadcast state. The fact that the incoming stream is a broadcast one guarantees that all instances see all the elements. Upon recovery or re-scaling, the same state is given to each of the instances. To avoid hotspots, each task reads its previous partition, and if there are more tasks (scale up), then the new instances read from the old instances in a round robin fashion. This is why each instance has to guarantee that it stores the same elements as the rest. If not, upon recovery or rescaling you may have unpredictable redistribution of the partitions, thus unpredictable results.
getBroadcastState in interface OperatorStateStoreK - The type of the keys in the broadcast state.V - The type of the values in the broadcast state.stateDescriptor - The descriptor for this state, providing a name, a serializer for the
keys and one for the values.StateMigrationExceptionpublic <S> ListState<S> getListState(ListStateDescriptor<S> stateDescriptor) throws Exception
OperatorStateStoreNote the semantic differences between an operator list state and a keyed list state (see
KeyedStateStore.getListState(ListStateDescriptor)). Under the context of operator
state, the list is a collection of state items that are independent of each other and
eligible for redistribution across operator instances in case of changed operator
parallelism. In other words, these state items are the finest granularity at which non-keyed
state can be redistributed, and should not be correlated with each other.
The redistribution scheme of this list state upon operator rescaling is a round-robin pattern, such that the logical whole state (a concatenation of all the lists of state elements previously managed by each operator before the restore) is evenly divided into as many sublists as there are parallel operators.
getListState in interface OperatorStateStoreS - The generic type of the statestateDescriptor - The descriptor for this state, providing a name and serializer.Exceptionpublic <S> ListState<S> getUnionListState(ListStateDescriptor<S> stateDescriptor) throws Exception
OperatorStateStoreNote the semantic differences between an operator list state and a keyed list state (see
KeyedStateStore.getListState(ListStateDescriptor)). Under the context of operator
state, the list is a collection of state items that are independent of each other and
eligible for redistribution across operator instances in case of changed operator
parallelism. In other words, these state items are the finest granularity at which non-keyed
state can be redistributed, and should not be correlated with each other.
The redistribution scheme of this list state upon operator rescaling is a broadcast pattern, such that the logical whole state (a concatenation of all the lists of state elements previously managed by each operator before the restore) is restored to all parallel operators so that each of them will get the union of all state items before the restore.
getUnionListState in interface OperatorStateStoreS - The generic type of the statestateDescriptor - The descriptor for this state, providing a name and serializer.Exception@Nonnull public RunnableFuture<SnapshotResult<OperatorStateHandle>> snapshot(long checkpointId, long timestamp, @Nonnull CheckpointStreamFactory streamFactory, @Nonnull CheckpointOptions checkpointOptions) throws Exception
SnapshotableCheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to
the snapshot. It is up to the implementation if the operation is performed synchronous or
asynchronous. In the later case, the returned Runnable must be executed first before
obtaining the handle.snapshot in interface Snapshotable<SnapshotResult<OperatorStateHandle>>checkpointId - The ID of the checkpoint.timestamp - The timestamp of the checkpoint.streamFactory - The factory that we can use for writing our state to streams.checkpointOptions - Options for how to perform this checkpoint.StateObject.ExceptionCopyright © 2014–2024 The Apache Software Foundation. All rights reserved.