org.apache.flink.runtime.state

Interface CheckpointStorage

All Known Implementing Classes:

AbstractFsCheckpointStorage, FsCheckpointStorage, MemoryBackendCheckpointStorage

public interface CheckpointStorage

CheckpointStorage implements the durable storage of checkpoint data and metadata streams. An individual checkpoint or savepoint is stored to a CheckpointStorageLocation, created by this class.

Method Summary

Modifier and Type	Method and Description
CheckpointStreamFactory.CheckpointStateOutputStream	createTaskOwnedStateStream() Opens a stream to persist checkpoint state data that is owned strictly by tasks and not attached to the life cycle of a specific checkpoint.
boolean	hasDefaultSavepointLocation() Checks whether the storage has a default savepoint location configured.
CheckpointStorageLocation	initializeLocationForCheckpoint(long checkpointId) Initializes a storage location for new checkpoint with the given ID.
CheckpointStorageLocation	initializeLocationForSavepoint(long checkpointId, String externalLocationPointer) Initializes a storage location for new savepoint with the given ID.
CompletedCheckpointStorageLocation	resolveCheckpoint(String externalPointer) Resolves the given pointer to a checkpoint/savepoint into a checkpoint location.
CheckpointStreamFactory	resolveCheckpointStorageLocation(long checkpointId, CheckpointStorageLocationReference reference) Resolves a storage location reference into a CheckpointStreamFactory.
boolean	supportsHighlyAvailableStorage() Checks whether this backend supports highly available storage of data.

Method Detail

supportsHighlyAvailableStorage

boolean supportsHighlyAvailableStorage()

Checks whether this backend supports highly available storage of data.

Some state backends may not support highly-available durable storage, with default settings, which makes them suitable for zero-config prototyping, but not for actual production setups.

hasDefaultSavepointLocation

boolean hasDefaultSavepointLocation()

Checks whether the storage has a default savepoint location configured.

resolveCheckpoint

CompletedCheckpointStorageLocation resolveCheckpoint(String externalPointer)
                                              throws IOException

Resolves the given pointer to a checkpoint/savepoint into a checkpoint location. The location supports reading the checkpoint metadata, or disposing the checkpoint storage location.

If the state backend cannot understand the format of the pointer (for example because it was created by a different state backend) this method should throw an IOException.

Parameters:

externalPointer - The external checkpoint pointer to resolve.

Returns:

The checkpoint location handle.

Throws:

IOException - Thrown, if the state backend does not understand the pointer, or if the pointer could not be resolved due to an I/O error.

initializeLocationForCheckpoint

CheckpointStorageLocation initializeLocationForCheckpoint(long checkpointId)
                                                   throws IOException

Initializes a storage location for new checkpoint with the given ID.

The returned storage location can be used to write the checkpoint data and metadata to and to obtain the pointers for the location(s) where the actual checkpoint data should be stored.

Parameters:

checkpointId - The ID (logical timestamp) of the checkpoint that should be persisted.

Returns:

A storage location for the data and metadata of the given checkpoint.

Throws:

IOException - Thrown if the storage location cannot be initialized due to an I/O exception.

initializeLocationForSavepoint

CheckpointStorageLocation initializeLocationForSavepoint(long checkpointId,
                                                         @Nullable
                                                         String externalLocationPointer)
                                                  throws IOException

Initializes a storage location for new savepoint with the given ID.

If an external location pointer is passed, the savepoint storage location will be initialized at the location of that pointer. If the external location pointer is null, the default savepoint location will be used. If no default savepoint location is configured, this will throw an exception. Whether a default savepoint location is configured can be checked via hasDefaultSavepointLocation().

Parameters:

checkpointId - The ID (logical timestamp) of the savepoint's checkpoint.

externalLocationPointer - Optionally, a pointer to the location where the savepoint should be stored. May be null.

Returns:

A storage location for the data and metadata of the savepoint.

Throws:

IOException - Thrown if the storage location cannot be initialized due to an I/O exception.

resolveCheckpointStorageLocation

CheckpointStreamFactory resolveCheckpointStorageLocation(long checkpointId,
                                                         CheckpointStorageLocationReference reference)
                                                  throws IOException

Resolves a storage location reference into a CheckpointStreamFactory.

The reference may be the default reference, in which case the method should return the default location, taking existing configuration and checkpoint ID into account.

Parameters:

checkpointId - The ID of the checkpoint that the location is initialized for.

reference - The checkpoint location reference.

Returns:

A checkpoint storage location reflecting the reference and checkpoint ID.

Throws:

IOException - Thrown, if the storage location cannot be initialized from the reference.

createTaskOwnedStateStream

CheckpointStreamFactory.CheckpointStateOutputStream createTaskOwnedStateStream()
                                                                        throws IOException

Opens a stream to persist checkpoint state data that is owned strictly by tasks and not attached to the life cycle of a specific checkpoint.

This method should be used when the persisted data cannot be immediately dropped once the checkpoint that created it is dropped. Examples are write-ahead-logs. For those, the state can only be dropped once the data has been moved to the target system, which may sometimes take longer than one checkpoint (if the target system is temporarily unable to keep up).

The fact that the job manager does not own the life cycle of this type of state means also that it is strictly the responsibility of the tasks to handle the cleanup of this data.

Developer note: In the future, we may be able to make this a special case of "shared state", where the task re-emits the shared state reference as long as it needs to hold onto the persisted state data.

Returns:

A checkpoint state stream to the location for state owned by tasks.

Throws:

IOException - Thrown, if the stream cannot be opened.