Package | Description |
---|---|
org.apache.flink.contrib.streaming.state | |
org.apache.flink.contrib.streaming.state.restore | |
org.apache.flink.contrib.streaming.state.snapshot | |
org.apache.flink.runtime.state | |
org.apache.flink.runtime.state.filesystem | |
org.apache.flink.runtime.state.heap |
This package contains the classes for key/value state backends that store the state
on the JVM heap as objects.
|
org.apache.flink.runtime.state.memory | |
org.apache.flink.streaming.api.operators | |
org.apache.flink.streaming.runtime.tasks |
This package contains classes that realize streaming tasks.
|
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
RocksDBStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
OperatorStateBackend |
RocksDBStateBackend.createOperatorStateBackend(Environment env,
String operatorIdentifier,
Collection<OperatorStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
void |
RocksDBStateDownloader.transferAllStateDataToDirectory(IncrementalRemoteKeyedStateHandle restoreStateHandle,
Path dest,
CloseableRegistry closeableRegistry)
Transfer all state data to the target directory using specified number of threads.
|
Map<StateHandleID,StreamStateHandle> |
RocksDBStateUploader.uploadFilesToCheckpointFs(Map<StateHandleID,Path> files,
CheckpointStreamFactory checkpointStreamFactory,
CloseableRegistry closeableRegistry)
Upload all the files to checkpoint fileSystem using specified number of threads.
|
Constructor and Description |
---|
RocksDBKeyedStateBackend(ClassLoader userCodeClassLoader,
File instanceBasePath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
TaskKvStateRegistry kvStateRegistry,
StateSerializerProvider<K> keySerializerProvider,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
org.rocksdb.RocksDB db,
LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
int keyGroupPrefixBytes,
CloseableRegistry cancelStreamRegistry,
StreamCompressionDecorator keyGroupCompressionDecorator,
ResourceGuard rocksDBResourceGuard,
RocksDBSnapshotStrategyBase<K> checkpointSnapshotStrategy,
RocksDBSnapshotStrategyBase<K> savepointSnapshotStrategy,
RocksDBWriteBatchWrapper writeBatchWrapper,
org.rocksdb.ColumnFamilyHandle defaultColumnFamilyHandle,
RocksDBNativeMetricMonitor nativeMetricMonitor,
org.apache.flink.contrib.streaming.state.RocksDBSerializedCompositeKeyBuilder<K> sharedRocksKeyBuilder,
PriorityQueueSetFactory priorityQueueFactory,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager) |
RocksDBKeyedStateBackendBuilder(String operatorIdentifier,
ClassLoader userCodeClassLoader,
File instanceBasePath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
TaskKvStateRegistry kvStateRegistry,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
LocalRecoveryConfig localRecoveryConfig,
RocksDBStateBackend.PriorityQueueStateType priorityQueueStateType,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
StreamCompressionDecorator keyGroupCompressionDecorator,
CloseableRegistry cancelStreamRegistry) |
Modifier and Type | Field and Description |
---|---|
protected CloseableRegistry |
AbstractRocksDBRestoreOperation.cancelStreamRegistry |
Constructor and Description |
---|
AbstractRocksDBRestoreOperation(KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager) |
RocksDBFullRestoreOperation(KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> restoreStateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager) |
RocksDBIncrementalRestoreOperation(String operatorIdentifier,
KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> restoreStateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager) |
RocksDBNoneRestoreOperation(KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
int numberOfTransferringThreads,
CloseableRegistry cancelStreamRegistry,
ClassLoader userCodeClassLoader,
Map<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
StateSerializerProvider<K> keySerializerProvider,
File instanceBasePath,
File instanceRocksDBPath,
org.rocksdb.DBOptions dbOptions,
java.util.function.Function<String,org.rocksdb.ColumnFamilyOptions> columnFamilyOptionsFactory,
RocksDBNativeMetricOptions nativeMetricOptions,
MetricGroup metricGroup,
Collection<KeyedStateHandle> restoreStateHandles,
RocksDbTtlCompactFiltersManager ttlCompactFiltersManager) |
Modifier and Type | Field and Description |
---|---|
protected CloseableRegistry |
RocksDBSnapshotStrategyBase.cancelStreamRegistry
A
CloseableRegistry that will be closed when the task is cancelled. |
Constructor and Description |
---|
RocksDBSnapshotStrategyBase(String description,
org.rocksdb.RocksDB db,
ResourceGuard rocksDBResourceGuard,
TypeSerializer<K> keySerializer,
LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
LocalRecoveryConfig localRecoveryConfig,
CloseableRegistry cancelStreamRegistry) |
RocksFullSnapshotStrategy(org.rocksdb.RocksDB db,
ResourceGuard rocksDBResourceGuard,
TypeSerializer<K> keySerializer,
LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
LocalRecoveryConfig localRecoveryConfig,
CloseableRegistry cancelStreamRegistry,
StreamCompressionDecorator keyGroupCompressionDecorator) |
RocksIncrementalSnapshotStrategy(org.rocksdb.RocksDB db,
ResourceGuard rocksDBResourceGuard,
TypeSerializer<K> keySerializer,
LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
KeyGroupRange keyGroupRange,
int keyGroupPrefixBytes,
LocalRecoveryConfig localRecoveryConfig,
CloseableRegistry cancelStreamRegistry,
File instanceBasePath,
UUID backendUID,
SortedMap<Long,Set<StateHandleID>> materializedSstFiles,
long lastCompletedCheckpointId,
int numberOfTransferingThreads) |
Modifier and Type | Field and Description |
---|---|
protected CloseableRegistry |
DefaultOperatorStateBackendBuilder.cancelStreamRegistry |
protected CloseableRegistry |
AbstractKeyedStateBackendBuilder.cancelStreamRegistry |
protected CloseableRegistry |
AbstractKeyedStateBackend.cancelStreamRegistry
Registry for all opened streams, so they can be closed if the task using this backend is closed.
|
protected CloseableRegistry |
AsyncSnapshotCallable.snapshotCloseableRegistry
Registers streams that can block in I/O during snapshot.
|
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
StateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry)
Creates a new
AbstractKeyedStateBackend that is responsible for holding keyed state
and checkpointing it. |
abstract <K> AbstractKeyedStateBackend<K> |
AbstractStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
OperatorStateBackend |
StateBackend.createOperatorStateBackend(Environment env,
String operatorIdentifier,
Collection<OperatorStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry)
Creates a new
OperatorStateBackend that can be used for storing operator state. |
abstract OperatorStateBackend |
AbstractStateBackend.createOperatorStateBackend(Environment env,
String operatorIdentifier,
Collection<OperatorStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
AsyncSnapshotCallable.AsyncSnapshotTask |
AsyncSnapshotCallable.toAsyncSnapshotFutureTask(CloseableRegistry taskRegistry)
Creates a future task from this and registers it with the given
CloseableRegistry . |
Constructor and Description |
---|
AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
StateSerializerProvider<K> keySerializerProvider,
ClassLoader userCodeClassLoader,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
CloseableRegistry cancelStreamRegistry,
StreamCompressionDecorator keyGroupCompressionDecorator) |
AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
TypeSerializer<K> keySerializer,
ClassLoader userCodeClassLoader,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
CloseableRegistry cancelStreamRegistry) |
AbstractKeyedStateBackendBuilder(TaskKvStateRegistry kvStateRegistry,
TypeSerializer<K> keySerializer,
ClassLoader userCodeClassLoader,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
Collection<KeyedStateHandle> stateHandles,
StreamCompressionDecorator keyGroupCompressionDecorator,
CloseableRegistry cancelStreamRegistry) |
DefaultOperatorStateBackend(ExecutionConfig executionConfig,
CloseableRegistry closeStreamOnCancelRegistry,
Map<String,PartitionableListState<?>> registeredOperatorStates,
Map<String,BackendWritableBroadcastState<?,?>> registeredBroadcastStates,
Map<String,PartitionableListState<?>> accessedStatesByName,
Map<String,BackendWritableBroadcastState<?,?>> accessedBroadcastStatesByName,
AbstractSnapshotStrategy<OperatorStateHandle> snapshotStrategy) |
DefaultOperatorStateBackendBuilder(ClassLoader userClassloader,
ExecutionConfig executionConfig,
boolean asynchronousSnapshots,
Collection<OperatorStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
OperatorStateRestoreOperation(CloseableRegistry closeStreamOnCancelRegistry,
ClassLoader userClassloader,
Map<String,PartitionableListState<?>> registeredOperatorStates,
Map<String,BackendWritableBroadcastState<?,?>> registeredBroadcastStates,
Collection<OperatorStateHandle> stateHandles) |
StateSnapshotContextSynchronousImpl(long checkpointId,
long checkpointTimestamp,
CheckpointStreamFactory streamFactory,
KeyGroupRange keyGroupRange,
CloseableRegistry closableRegistry) |
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
FsStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
OperatorStateBackend |
FsStateBackend.createOperatorStateBackend(Environment env,
String operatorIdentifier,
Collection<OperatorStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
Constructor and Description |
---|
HeapKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
StateSerializerProvider<K> keySerializerProvider,
ClassLoader userCodeClassLoader,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
CloseableRegistry cancelStreamRegistry,
StreamCompressionDecorator keyGroupCompressionDecorator,
Map<String,StateTable<K,?,?>> registeredKVStates,
Map<String,HeapPriorityQueueSnapshotRestoreWrapper> registeredPQStates,
LocalRecoveryConfig localRecoveryConfig,
HeapPriorityQueueSetFactory priorityQueueSetFactory,
org.apache.flink.runtime.state.heap.HeapSnapshotStrategy<K> snapshotStrategy) |
HeapKeyedStateBackendBuilder(TaskKvStateRegistry kvStateRegistry,
TypeSerializer<K> keySerializer,
ClassLoader userCodeClassLoader,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
ExecutionConfig executionConfig,
TtlTimeProvider ttlTimeProvider,
Collection<KeyedStateHandle> stateHandles,
StreamCompressionDecorator keyGroupCompressionDecorator,
LocalRecoveryConfig localRecoveryConfig,
HeapPriorityQueueSetFactory priorityQueueSetFactory,
boolean asynchronousSnapshots,
CloseableRegistry cancelStreamRegistry) |
Modifier and Type | Method and Description |
---|---|
<K> AbstractKeyedStateBackend<K> |
MemoryStateBackend.createKeyedStateBackend(Environment env,
JobID jobID,
String operatorIdentifier,
TypeSerializer<K> keySerializer,
int numberOfKeyGroups,
KeyGroupRange keyGroupRange,
TaskKvStateRegistry kvStateRegistry,
TtlTimeProvider ttlTimeProvider,
MetricGroup metricGroup,
Collection<KeyedStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
OperatorStateBackend |
MemoryStateBackend.createOperatorStateBackend(Environment env,
String operatorIdentifier,
Collection<OperatorStateHandle> stateHandles,
CloseableRegistry cancelStreamRegistry) |
Modifier and Type | Method and Description |
---|---|
protected <K> AbstractKeyedStateBackend<K> |
StreamTaskStateInitializerImpl.keyedStatedBackend(TypeSerializer<K> keySerializer,
String operatorIdentifierText,
PrioritizedOperatorSubtaskState prioritizedOperatorSubtaskStates,
CloseableRegistry backendCloseableRegistry,
MetricGroup metricGroup) |
protected OperatorStateBackend |
StreamTaskStateInitializerImpl.operatorStateBackend(String operatorIdentifierText,
PrioritizedOperatorSubtaskState prioritizedOperatorSubtaskStates,
CloseableRegistry backendCloseableRegistry) |
StreamOperatorStateContext |
StreamTaskStateInitializerImpl.streamOperatorStateContext(OperatorID operatorID,
String operatorClassName,
KeyContext keyContext,
TypeSerializer<?> keySerializer,
CloseableRegistry streamTaskCloseableRegistry,
MetricGroup metricGroup) |
StreamOperatorStateContext |
StreamTaskStateInitializer.streamOperatorStateContext(OperatorID operatorID,
String operatorClassName,
KeyContext keyContext,
TypeSerializer<?> keySerializer,
CloseableRegistry streamTaskCloseableRegistry,
MetricGroup metricGroup)
Returns the
StreamOperatorStateContext for an AbstractStreamOperator that runs in the stream
task that owns this manager. |
Constructor and Description |
---|
BackendRestorerProcedure(FunctionWithException<Collection<S>,T,Exception> instanceSupplier,
CloseableRegistry backendCloseableRegistry,
String logDescription)
Creates a new backend restorer using the given backend supplier and the closeable registry.
|
Modifier and Type | Method and Description |
---|---|
CloseableRegistry |
StreamTask.getCancelables() |
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.