org.apache.flink.contrib.streaming.state.snapshot

Class RocksDBSnapshotStrategyBase<K>

java.lang.Object

org.apache.flink.runtime.state.AbstractSnapshotStrategy<KeyedStateHandle>

org.apache.flink.contrib.streaming.state.snapshot.RocksDBSnapshotStrategyBase<K>

Type Parameters:

K - type of the backend keys.

All Implemented Interfaces:

CheckpointListener, SnapshotStrategy<SnapshotResult<KeyedStateHandle>>

Direct Known Subclasses:

RocksFullSnapshotStrategy, RocksIncrementalSnapshotStrategy

public abstract class RocksDBSnapshotStrategyBase<K>
extends AbstractSnapshotStrategy<KeyedStateHandle>
implements CheckpointListener

Abstract base class for SnapshotStrategy implementations for RocksDB state backend.

Field Summary

Fields

Modifier and Type	Field and Description
protected CloseableRegistry	cancelStreamRegistry A CloseableRegistry that will be closed when the task is cancelled.
protected org.rocksdb.RocksDB	db RocksDB instance from the backend.
protected int	keyGroupPrefixBytes Number of bytes in the key-group prefix.
protected KeyGroupRange	keyGroupRange The key-group range for the task.
protected TypeSerializer<K>	keySerializer The key serializer of the backend.
protected LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo>	kvStateInformation Key/Value state meta info from the backend.
protected LocalRecoveryConfig	localRecoveryConfig The configuration for local recovery.
protected ResourceGuard	rocksDBResourceGuard Resource guard for the RocksDB instance.

Fields inherited from class org.apache.flink.runtime.state.AbstractSnapshotStrategy

description

Constructor Summary

Constructors

Constructor and Description
RocksDBSnapshotStrategyBase(String description, org.rocksdb.RocksDB db, ResourceGuard rocksDBResourceGuard, TypeSerializer<K> keySerializer, LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation, KeyGroupRange keyGroupRange, int keyGroupPrefixBytes, LocalRecoveryConfig localRecoveryConfig, CloseableRegistry cancelStreamRegistry)

Method Summary

Modifier and Type	Method and Description
protected abstract RunnableFuture<SnapshotResult<KeyedStateHandle>>	doSnapshot(long checkpointId, long timestamp, CheckpointStreamFactory streamFactory, CheckpointOptions checkpointOptions) This method implements the concrete snapshot logic for a non-empty state.
RunnableFuture<SnapshotResult<KeyedStateHandle>>	snapshot(long checkpointId, long timestamp, CheckpointStreamFactory streamFactory, CheckpointOptions checkpointOptions) Operation that writes a snapshot into a stream that is provided by the given CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to the snapshot.

Methods inherited from class org.apache.flink.runtime.state.AbstractSnapshotStrategy

logAsyncCompleted, logSyncCompleted, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Field Detail

db

@Nonnull
protected org.rocksdb.RocksDB db

RocksDB instance from the backend.

rocksDBResourceGuard

@Nonnull
protected final ResourceGuard rocksDBResourceGuard

Resource guard for the RocksDB instance.

keySerializer

@Nonnull
protected final TypeSerializer<K> keySerializer

The key serializer of the backend.

kvStateInformation

@Nonnull
protected final LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation

Key/Value state meta info from the backend.

keyGroupRange

@Nonnull
protected final KeyGroupRange keyGroupRange

The key-group range for the task.

keyGroupPrefixBytes

@Nonnegative
protected final int keyGroupPrefixBytes

Number of bytes in the key-group prefix.

localRecoveryConfig

@Nonnull
protected final LocalRecoveryConfig localRecoveryConfig

The configuration for local recovery.

cancelStreamRegistry

@Nonnull
protected final CloseableRegistry cancelStreamRegistry

A CloseableRegistry that will be closed when the task is cancelled.

Constructor Detail

RocksDBSnapshotStrategyBase

public RocksDBSnapshotStrategyBase(@Nonnull
                                   String description,
                                   @Nonnull
                                   org.rocksdb.RocksDB db,
                                   @Nonnull
                                   ResourceGuard rocksDBResourceGuard,
                                   @Nonnull
                                   TypeSerializer<K> keySerializer,
                                   @Nonnull
                                   LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
                                   @Nonnull
                                   KeyGroupRange keyGroupRange,
                                   @Nonnegative
                                   int keyGroupPrefixBytes,
                                   @Nonnull
                                   LocalRecoveryConfig localRecoveryConfig,
                                   @Nonnull
                                   CloseableRegistry cancelStreamRegistry)

Method Detail

snapshot

@Nonnull
public final RunnableFuture<SnapshotResult<KeyedStateHandle>> snapshot(long checkpointId,
                                                                                long timestamp,
                                                                                @Nonnull
                                                                                CheckpointStreamFactory streamFactory,
                                                                                @Nonnull
                                                                                CheckpointOptions checkpointOptions)
                                                                         throws Exception

Description copied from interface: SnapshotStrategy

Operation that writes a snapshot into a stream that is provided by the given CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to the snapshot. It is up to the implementation if the operation is performed synchronous or asynchronous. In the later case, the returned Runnable must be executed first before obtaining the handle.

Specified by:

snapshot in interface SnapshotStrategy<SnapshotResult<KeyedStateHandle>>

Parameters:

checkpointId - The ID of the checkpoint.

timestamp - The timestamp of the checkpoint.

streamFactory - The factory that we can use for writing our state to streams.

checkpointOptions - Options for how to perform this checkpoint.

Returns:

A runnable future that will yield a StateObject.

Throws:

Exception

doSnapshot

@Nonnull
protected abstract RunnableFuture<SnapshotResult<KeyedStateHandle>> doSnapshot(long checkpointId,
                                                                                        long timestamp,
                                                                                        CheckpointStreamFactory streamFactory,
                                                                                        CheckpointOptions checkpointOptions)
                                                                                 throws Exception

This method implements the concrete snapshot logic for a non-empty state.

Throws:

Exception