Package | Description |
---|---|
org.apache.flink.contrib.streaming.state | |
org.apache.flink.contrib.streaming.state.restore | |
org.apache.flink.contrib.streaming.state.snapshot | |
org.apache.flink.runtime.checkpoint | |
org.apache.flink.runtime.checkpoint.metadata | |
org.apache.flink.runtime.state | |
org.apache.flink.runtime.state.filesystem | |
org.apache.flink.runtime.state.heap |
This package contains the classes for key/value state backends that store the state on the JVM
heap as objects.
|
org.apache.flink.runtime.state.memory | |
org.apache.flink.state.api.input.splits | |
org.apache.flink.streaming.api.operators |
Modifier and Type | Method and Description |
---|---|
static KeyedStateHandle |
RocksDBIncrementalCheckpointUtils.chooseTheBestStateHandleForInitial(Collection<KeyedStateHandle> restoreStateHandles,
KeyGroupRange targetKeyGroupRange)
Choose the best state handle according to the
RocksDBIncrementalCheckpointUtils.STATE_HANDLE_EVALUATOR to init the
initial db. |
Modifier and Type | Method and Description |
---|---|
RunnableFuture<SnapshotResult<KeyedStateHandle>> |
RocksDBKeyedStateBackend.snapshot(long checkpointId,
long timestamp,
CheckpointStreamFactory streamFactory,
CheckpointOptions checkpointOptions)
Triggers an asynchronous snapshot of the keyed state backend from RocksDB.
|
Modifier and Type | Method and Description |
---|---|
static KeyedStateHandle |
RocksDBIncrementalCheckpointUtils.chooseTheBestStateHandleForInitial(Collection<KeyedStateHandle> restoreStateHandles,
KeyGroupRange targetKeyGroupRange)
Choose the best state handle according to the
RocksDBIncrementalCheckpointUtils.STATE_HANDLE_EVALUATOR to init the
initial db. |
<K> AbstractKeyedStateBackend<K> |
RocksDBStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
Constructor and Description |
---|
RocksDBKeyedStateBackendBuilder(String operatorIdentifier,
ClassLoader userCodeClassLoader,
File instanceBasePath,
RocksDBResourceContainer optionsContainer,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
TaskKvStateRegistry kvStateRegistry,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
LocalRecoveryConfig localRecoveryConfig,
RocksDBStateBackend.PriorityQueueStateType priorityQueueStateType,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
StreamCompressionDecorator keyGroupCompressionDecorator,
CloseableRegistry cancelStreamRegistry) |
Modifier and Type | Field and Description |
---|---|
protected Collection<KeyedStateHandle> |
AbstractRocksDBRestoreOperation.restoreStateHandles |
Constructor and Description |
---|
AbstractRocksDBRestoreOperation(KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager) |
RocksDBFullRestoreOperation(KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> restoreStateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager,
long writeBatchSize,
PriorityQueueFlag queueRestoreEnabled) |
RocksDBIncrementalRestoreOperation(String operatorIdentifier,
KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> restoreStateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager,
long writeBatchSize,
PriorityQueueFlag queueRestoreEnabled) |
RocksDBNoneRestoreOperation(KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> restoreStateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager) |
Modifier and Type | Method and Description |
---|---|
RunnableFuture<SnapshotResult<KeyedStateHandle>> |
RocksFullSnapshotStrategy.doSnapshot(long checkpointId,
long timestamp,
CheckpointStreamFactory primaryStreamFactory,
CheckpointOptions checkpointOptions) |
protected RunnableFuture<SnapshotResult<KeyedStateHandle>> |
RocksIncrementalSnapshotStrategy.doSnapshot(long checkpointId,
long checkpointTimestamp,
CheckpointStreamFactory checkpointStreamFactory,
CheckpointOptions checkpointOptions) |
protected abstract RunnableFuture<SnapshotResult<KeyedStateHandle>> |
RocksDBSnapshotStrategyBase.doSnapshot(long checkpointId,
long timestamp,
CheckpointStreamFactory streamFactory,
CheckpointOptions checkpointOptions)
This method implements the concrete snapshot logic for a non-empty state.
|
RunnableFuture<SnapshotResult<KeyedStateHandle>> |
RocksDBSnapshotStrategyBase.snapshot(long checkpointId,
long timestamp,
CheckpointStreamFactory streamFactory,
CheckpointOptions checkpointOptions) |
Modifier and Type | Method and Description |
---|---|
KeyedStateHandle |
SubtaskState.getManagedKeyedState() |
KeyedStateHandle |
SubtaskState.getRawKeyedState() |
Modifier and Type | Method and Description |
---|---|
StateObjectCollection<KeyedStateHandle> |
PrioritizedOperatorSubtaskState.getJobManagerManagedKeyedState()
Returns the managed keyed state from the job manager, which represents the ground truth about
what this state should represent.
|
StateObjectCollection<KeyedStateHandle> |
PrioritizedOperatorSubtaskState.getJobManagerRawKeyedState()
Returns the raw keyed state from the job manager, which represents the ground truth about
what this state should represent.
|
StateObjectCollection<KeyedStateHandle> |
OperatorSubtaskState.getManagedKeyedState()
Returns a handle to the managed keyed state.
|
static List<KeyedStateHandle> |
StateAssignmentOperation.getManagedKeyedStateHandles(OperatorState operatorState,
KeyGroupRange subtaskKeyGroupRange)
Collect
managedKeyedStateHandles which have intersection with
given KeyGroupRange from operatorState . |
List<StateObjectCollection<KeyedStateHandle>> |
PrioritizedOperatorSubtaskState.getPrioritizedManagedKeyedState()
Returns an immutable list with all alternative snapshots to restore the managed keyed state,
in the order in which we should attempt to restore.
|
List<StateObjectCollection<KeyedStateHandle>> |
PrioritizedOperatorSubtaskState.getPrioritizedRawKeyedState()
Returns an immutable list with all alternative snapshots to restore the raw keyed state, in
the order in which we should attempt to restore.
|
StateObjectCollection<KeyedStateHandle> |
OperatorSubtaskState.getRawKeyedState()
Returns a handle to the raw keyed state.
|
static List<KeyedStateHandle> |
StateAssignmentOperation.getRawKeyedStateHandles(OperatorState operatorState,
KeyGroupRange subtaskKeyGroupRange)
|
Modifier and Type | Method and Description |
---|---|
static void |
StateAssignmentOperation.extractIntersectingState(Collection<? extends KeyedStateHandle> originalSubtaskStateHandles,
KeyGroupRange rangeToExtract,
List<KeyedStateHandle> extractedStateCollector)
Extracts certain key group ranges from the given state handles and adds them to the
collector.
|
static void |
StateAssignmentOperation.extractIntersectingState(Collection<? extends KeyedStateHandle> originalSubtaskStateHandles,
KeyGroupRange rangeToExtract,
List<KeyedStateHandle> extractedStateCollector)
Extracts certain key group ranges from the given state handles and adds them to the
collector.
|
static OperatorSubtaskState |
StateAssignmentOperation.operatorSubtaskStateFrom(OperatorInstanceID instanceID,
Map<OperatorInstanceID,List<OperatorStateHandle>> subManagedOperatorState,
Map<OperatorInstanceID,List<OperatorStateHandle>> subRawOperatorState,
Map<OperatorInstanceID,List<InputChannelStateHandle>> inputChannelStates,
Map<OperatorInstanceID,List<ResultSubpartitionStateHandle>> resultSubpartitionStates,
Map<OperatorInstanceID,List<KeyedStateHandle>> subManagedKeyedState,
Map<OperatorInstanceID,List<KeyedStateHandle>> subRawKeyedState) |
static OperatorSubtaskState |
StateAssignmentOperation.operatorSubtaskStateFrom(OperatorInstanceID instanceID,
Map<OperatorInstanceID,List<OperatorStateHandle>> subManagedOperatorState,
Map<OperatorInstanceID,List<OperatorStateHandle>> subRawOperatorState,
Map<OperatorInstanceID,List<InputChannelStateHandle>> inputChannelStates,
Map<OperatorInstanceID,List<ResultSubpartitionStateHandle>> resultSubpartitionStates,
Map<OperatorInstanceID,List<KeyedStateHandle>> subManagedKeyedState,
Map<OperatorInstanceID,List<KeyedStateHandle>> subRawKeyedState) |
Constructor and Description |
---|
OperatorSubtaskState(OperatorStateHandle managedOperatorState,
OperatorStateHandle rawOperatorState,
KeyedStateHandle managedKeyedState,
KeyedStateHandle rawKeyedState,
StateObjectCollection<InputChannelStateHandle> inputChannelState,
StateObjectCollection<ResultSubpartitionStateHandle> resultSubpartitionState)
For convenience because the size of the collections is typically 0 or 1.
|
SubtaskState(ChainedStateHandle<OperatorStateHandle> managedOperatorState,
ChainedStateHandle<OperatorStateHandle> rawOperatorState,
KeyedStateHandle managedKeyedState,
KeyedStateHandle rawKeyedState) |
Modifier and Type | Method and Description |
---|---|
static KeyedStateHandle |
MetadataV3Serializer.deserializeKeyedStateHandleUtil(DataInputStream dis) |
Modifier and Type | Method and Description |
---|---|
static void |
MetadataV3Serializer.serializeKeyedStateHandleUtil(KeyedStateHandle stateHandle,
DataOutputStream dos) |
Modifier and Type | Interface and Description |
---|---|
interface |
IncrementalKeyedStateHandle
Common interface to all incremental
KeyedStateHandle . |
Modifier and Type | Class and Description |
---|---|
class |
DirectoryKeyedStateHandle
This class is a keyed state handle based on a directory.
|
class |
IncrementalLocalKeyedStateHandle
State handle for local copies of
IncrementalRemoteKeyedStateHandle . |
class |
IncrementalRemoteKeyedStateHandle
The handle to states of an incremental snapshot.
|
class |
KeyGroupsStateHandle
A handle to the partitioned stream operator state after it has been checkpointed.
|
Modifier and Type | Field and Description |
---|---|
protected RunnableFuture<SnapshotResult<KeyedStateHandle>> |
StateSnapshotContextSynchronousImpl.keyedStateCheckpointClosingFuture |
protected Collection<KeyedStateHandle> |
AbstractKeyedStateBackendBuilder.restoreStateHandles |
Modifier and Type | Method and Description |
---|---|
KeyedStateHandle |
DirectoryKeyedStateHandle.getIntersection(KeyGroupRange otherKeyGroupRange) |
KeyedStateHandle |
IncrementalRemoteKeyedStateHandle.getIntersection(KeyGroupRange keyGroupRange) |
KeyedStateHandle |
KeyedStateHandle.getIntersection(KeyGroupRange keyGroupRange)
Returns a state over a range that is the intersection between this handle's key-group range
and the provided key-group range.
|
Modifier and Type | Method and Description |
---|---|
RunnableFuture<SnapshotResult<KeyedStateHandle>> |
StateSnapshotContextSynchronousImpl.getKeyedStateStreamFuture() |
static SnapshotResult<KeyedStateHandle> |
CheckpointStreamWithResultProvider.toKeyedStateHandleSnapshotResult(SnapshotResult<StreamStateHandle> snapshotResult,
KeyGroupRangeOffsets keyGroupRangeOffsets)
Helper method that takes a
SnapshotResult and a KeyGroupRangeOffsets and creates a SnapshotResult by combining
the key groups offsets with all the present stream state handles. |
Modifier and Type | Method and Description |
---|---|
abstract <K> AbstractKeyedStateBackend<K> |
AbstractStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
<K> AbstractKeyedStateBackend<K> |
StateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry)
Creates a new
AbstractKeyedStateBackend that is responsible for holding keyed
state and checkpointing it. |
Constructor and Description |
---|
AbstractKeyedStateBackendBuilder(TaskKvStateRegistry kvStateRegistry,
TypeSerializer<K> keySerializer,
ClassLoader userCodeClassLoader,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
Collection<KeyedStateHandle> stateHandles,
StreamCompressionDecorator keyGroupCompressionDecorator,
CloseableRegistry cancelStreamRegistry) |
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
FsStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
Modifier and Type | Method and Description |
---|---|
RunnableFuture<SnapshotResult<KeyedStateHandle>> |
HeapKeyedStateBackend.snapshot(long checkpointId,
long timestamp,
CheckpointStreamFactory streamFactory,
CheckpointOptions checkpointOptions) |
Constructor and Description |
---|
HeapKeyedStateBackendBuilder(TaskKvStateRegistry kvStateRegistry,
TypeSerializer<K> keySerializer,
ClassLoader userCodeClassLoader,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
Collection<KeyedStateHandle> stateHandles,
StreamCompressionDecorator keyGroupCompressionDecorator,
LocalRecoveryConfig localRecoveryConfig,
HeapPriorityQueueSetFactory priorityQueueSetFactory,
boolean asynchronousSnapshots,
CloseableRegistry cancelStreamRegistry) |
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
MemoryStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
Constructor and Description |
---|
KeyGroupRangeInputSplit(List<KeyedStateHandle> managedKeyedState,
List<KeyedStateHandle> rawKeyedState,
int numKeyGroups,
int split) |
KeyGroupRangeInputSplit(List<KeyedStateHandle> managedKeyedState,
List<KeyedStateHandle> rawKeyedState,
int numKeyGroups,
int split) |
Modifier and Type | Method and Description |
---|---|
RunnableFuture<SnapshotResult<KeyedStateHandle>> |
OperatorSnapshotFutures.getKeyedStateManagedFuture() |
RunnableFuture<SnapshotResult<KeyedStateHandle>> |
OperatorSnapshotFutures.getKeyedStateRawFuture() |
Modifier and Type | Method and Description |
---|---|
protected CloseableIterable<KeyGroupStatePartitionStreamProvider> |
StreamTaskStateInitializerImpl.rawKeyedStateInputs(Iterator<StateObjectCollection<KeyedStateHandle>> restoreStateAlternatives) |
void |
OperatorSnapshotFutures.setKeyedStateManagedFuture(RunnableFuture<SnapshotResult<KeyedStateHandle>> keyedStateManagedFuture) |
void |
OperatorSnapshotFutures.setKeyedStateRawFuture(RunnableFuture<SnapshotResult<KeyedStateHandle>> keyedStateRawFuture) |
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.