Class RocksDBSnapshotStrategyBase<K,R extends SnapshotResources>
- java.lang.Object
-
- org.apache.flink.contrib.streaming.state.snapshot.RocksDBSnapshotStrategyBase<K,R>
-
- Type Parameters:
K
- type of the backend keys.
- All Implemented Interfaces:
AutoCloseable
,CheckpointListener
,SnapshotStrategy<KeyedStateHandle,RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources>
- Direct Known Subclasses:
RocksIncrementalSnapshotStrategy
,RocksNativeFullSnapshotStrategy
public abstract class RocksDBSnapshotStrategyBase<K,R extends SnapshotResources> extends Object implements CheckpointListener, SnapshotStrategy<KeyedStateHandle,RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources>, AutoCloseable
Abstract base class forSnapshotStrategy
implementations for RocksDB state backend.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources
ASnapshotResources
for native rocksdb snapshot.protected static class
RocksDBSnapshotStrategyBase.PreviousSnapshot
Previous snapshot with uploaded sst files.protected class
RocksDBSnapshotStrategyBase.RocksDBSnapshotOperation
Common operation in native rocksdb snapshot result supplier.-
Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.SnapshotStrategy
SnapshotStrategy.SnapshotResultSupplier<S extends StateObject>
-
-
Field Summary
Fields Modifier and Type Field Description protected UUID
backendUID
The state handle ids of all sst files materialized in snapshots for previous checkpoints.protected org.rocksdb.RocksDB
db
RocksDB instance from the backend.protected static RocksDBSnapshotStrategyBase.PreviousSnapshot
EMPTY_PREVIOUS_SNAPSHOT
protected File
instanceBasePath
Base path of the RocksDB instance.protected int
keyGroupPrefixBytes
Number of bytes in the key-group prefix.protected KeyGroupRange
keyGroupRange
The key-group range for the task.protected TypeSerializer<K>
keySerializer
The key serializer of the backend.protected LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo>
kvStateInformation
Key/Value state meta info from the backend.protected String
localDirectoryName
The local directory name of the current snapshot strategy.protected LocalRecoveryConfig
localRecoveryConfig
The configuration for local recovery.protected ResourceGuard
rocksDBResourceGuard
Resource guard for the RocksDB instance.
-
Constructor Summary
Constructors Constructor Description RocksDBSnapshotStrategyBase(String description, org.rocksdb.RocksDB db, ResourceGuard rocksDBResourceGuard, TypeSerializer<K> keySerializer, LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation, KeyGroupRange keyGroupRange, int keyGroupPrefixBytes, LocalRecoveryConfig localRecoveryConfig, File instanceBasePath, UUID backendUID)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected void
cleanupIncompleteSnapshot(CloseableRegistry tmpResourcesRegistry, SnapshotDirectory localBackupDirectory)
abstract void
close()
String
getDescription()
protected SnapshotResult<StreamStateHandle>
materializeMetaData(CloseableRegistry snapshotCloseableRegistry, CloseableRegistry tmpResourcesRegistry, List<StateMetaInfoSnapshot> stateMetaInfoSnapshots, long checkpointId, CheckpointStreamFactory checkpointStreamFactory)
protected SnapshotDirectory
prepareLocalSnapshotDirectory(long checkpointId)
protected abstract RocksDBSnapshotStrategyBase.PreviousSnapshot
snapshotMetaData(long checkpointId, List<StateMetaInfoSnapshot> stateMetaInfoSnapshots)
RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources
syncPrepareResources(long checkpointId)
Performs the synchronous part of the snapshot.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener
notifyCheckpointAborted, notifyCheckpointComplete
-
Methods inherited from interface org.apache.flink.runtime.state.SnapshotStrategy
asyncSnapshot
-
-
-
-
Field Detail
-
db
@Nonnull protected org.rocksdb.RocksDB db
RocksDB instance from the backend.
-
rocksDBResourceGuard
@Nonnull protected final ResourceGuard rocksDBResourceGuard
Resource guard for the RocksDB instance.
-
keySerializer
@Nonnull protected final TypeSerializer<K> keySerializer
The key serializer of the backend.
-
kvStateInformation
@Nonnull protected final LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation
Key/Value state meta info from the backend.
-
keyGroupRange
@Nonnull protected final KeyGroupRange keyGroupRange
The key-group range for the task.
-
keyGroupPrefixBytes
@Nonnegative protected final int keyGroupPrefixBytes
Number of bytes in the key-group prefix.
-
localRecoveryConfig
@Nonnull protected final LocalRecoveryConfig localRecoveryConfig
The configuration for local recovery.
-
instanceBasePath
@Nonnull protected final File instanceBasePath
Base path of the RocksDB instance.
-
localDirectoryName
protected final String localDirectoryName
The local directory name of the current snapshot strategy.
-
backendUID
@Nonnull protected final UUID backendUID
The state handle ids of all sst files materialized in snapshots for previous checkpoints.
-
EMPTY_PREVIOUS_SNAPSHOT
protected static final RocksDBSnapshotStrategyBase.PreviousSnapshot EMPTY_PREVIOUS_SNAPSHOT
-
-
Constructor Detail
-
RocksDBSnapshotStrategyBase
public RocksDBSnapshotStrategyBase(@Nonnull String description, @Nonnull org.rocksdb.RocksDB db, @Nonnull ResourceGuard rocksDBResourceGuard, @Nonnull TypeSerializer<K> keySerializer, @Nonnull LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation, @Nonnull KeyGroupRange keyGroupRange, @Nonnegative int keyGroupPrefixBytes, @Nonnull LocalRecoveryConfig localRecoveryConfig, @Nonnull File instanceBasePath, @Nonnull UUID backendUID)
-
-
Method Detail
-
getDescription
@Nonnull public String getDescription()
-
syncPrepareResources
public RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources syncPrepareResources(long checkpointId) throws Exception
Description copied from interface:SnapshotStrategy
Performs the synchronous part of the snapshot. It returns resources which can be later on used in the asynchronous part.- Specified by:
syncPrepareResources
in interfaceSnapshotStrategy<K,R extends SnapshotResources>
- Parameters:
checkpointId
- The ID of the checkpoint.- Returns:
- Resources needed to finish the snapshot.
- Throws:
Exception
-
snapshotMetaData
protected abstract RocksDBSnapshotStrategyBase.PreviousSnapshot snapshotMetaData(long checkpointId, @Nonnull List<StateMetaInfoSnapshot> stateMetaInfoSnapshots)
-
prepareLocalSnapshotDirectory
@Nonnull protected SnapshotDirectory prepareLocalSnapshotDirectory(long checkpointId) throws IOException
- Throws:
IOException
-
cleanupIncompleteSnapshot
protected void cleanupIncompleteSnapshot(@Nonnull CloseableRegistry tmpResourcesRegistry, @Nonnull SnapshotDirectory localBackupDirectory)
-
materializeMetaData
@Nonnull protected SnapshotResult<StreamStateHandle> materializeMetaData(@Nonnull CloseableRegistry snapshotCloseableRegistry, @Nonnull CloseableRegistry tmpResourcesRegistry, @Nonnull List<StateMetaInfoSnapshot> stateMetaInfoSnapshots, long checkpointId, @Nonnull CheckpointStreamFactory checkpointStreamFactory) throws Exception
- Throws:
Exception
-
close
public abstract void close() throws IOException
- Specified by:
close
in interfaceAutoCloseable
- Throws:
IOException
-
-