org.apache.flink.runtime.state.filesystem

Class AbstractFileStateBackend

java.lang.Object

org.apache.flink.runtime.state.AbstractStateBackend

org.apache.flink.runtime.state.filesystem.AbstractFileStateBackend

All Implemented Interfaces:

Serializable, StateBackend

Direct Known Subclasses:

FsStateBackend, MemoryStateBackend

@PublicEvolving
public abstract class AbstractFileStateBackend
extends AbstractStateBackend

A base class for all state backends that store their metadata (and data) in files. Examples that inherit from this are the FsStateBackend, the MemoryStateBackend, or the RocksDBStateBackend.

This class takes the base checkpoint- and savepoint directory paths, but also accepts null for both of then, in which case creating externalized checkpoint is not possible, and it is not possible to create a savepoint with a default path. Null is accepted to enable implementations that only optionally support default savepoints and externalized checkpoints.

Checkpoint Layout

The state backend is configured with a base directory and persists the checkpoint data of specific checkpoints in specific subdirectories. For example, if the base directory was set to hdfs://namenode:port/flink-checkpoints/, the state backend will create a subdirectory with the job's ID that will contain the actual checkpoints: (hdfs://namenode:port/flink-checkpoints/1b080b6e710aabbef8993ab18c6de98b)

Each checkpoint individually will store all its files in a subdirectory that includes the checkpoint number, such as hdfs://namenode:port/flink-checkpoints/1b080b6e710aabbef8993ab18c6de98b/chk-17/.

Savepoint Layout

A savepoint that is set to be stored in path hdfs://namenode:port/flink-savepoints/, will create a subdirectory savepoint-jobId(0, 6)-randomDigits in which it stores all savepoint data. The random digits are added as "entropy" to avoid directory collisions.

Metadata File

A completed checkpoint writes its metadata into a file ''.

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractFileStateBackend(Path baseCheckpointPath, Path baseSavepointPath) Creates a backend with the given optional checkpoint- and savepoint base directories.
protected	AbstractFileStateBackend(Path baseCheckpointPath, Path baseSavepointPath, ReadableConfig configuration) Creates a new backend using the given checkpoint-/savepoint directories, or the values defined in the given configuration.
protected	AbstractFileStateBackend(URI baseCheckpointPath, URI baseSavepointPath) Creates a backend with the given optional checkpoint- and savepoint base directories.

Method Summary

Modifier and Type	Method and Description
Path	getCheckpointPath() Gets the checkpoint base directory.
Path	getSavepointPath() Gets the directory where savepoints are stored by default (when no custom path is given to the savepoint trigger command).
CompletedCheckpointStorageLocation	resolveCheckpoint(String pointer) Resolves the given pointer to a checkpoint/savepoint into a checkpoint location.

Methods inherited from class org.apache.flink.runtime.state.AbstractStateBackend

createKeyedStateBackend, createOperatorStateBackend, getCompressionDecorator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.StateBackend

createCheckpointStorage, createKeyedStateBackend, useManagedMemory

Constructor Detail

AbstractFileStateBackend

protected AbstractFileStateBackend(@Nullable
                                   URI baseCheckpointPath,
                                   @Nullable
                                   URI baseSavepointPath)

Creates a backend with the given optional checkpoint- and savepoint base directories.

Parameters:

baseCheckpointPath - The base directory for checkpoints, or null, if none is configured.

baseSavepointPath - The default directory for savepoints, or null, if none is set.

AbstractFileStateBackend

protected AbstractFileStateBackend(@Nullable
                                   Path baseCheckpointPath,
                                   @Nullable
                                   Path baseSavepointPath)

Creates a backend with the given optional checkpoint- and savepoint base directories.

Parameters:

baseCheckpointPath - The base directory for checkpoints, or null, if none is configured.

baseSavepointPath - The default directory for savepoints, or null, if none is set.

AbstractFileStateBackend

protected AbstractFileStateBackend(@Nullable
                                   Path baseCheckpointPath,
                                   @Nullable
                                   Path baseSavepointPath,
                                   ReadableConfig configuration)

Creates a new backend using the given checkpoint-/savepoint directories, or the values defined in the given configuration. If a checkpoint-/savepoint parameter is not null, that value takes precedence over the value in the configuration. If the configuration does not specify a value, it is possible that the checkpoint-/savepoint directories in the backend will be null.

This constructor can be used to create a backend that is based partially on a given backend and partially on a configuration.

Parameters:

baseCheckpointPath - The checkpoint base directory to use (or null).

baseSavepointPath - The default savepoint directory to use (or null).

configuration - The configuration to read values from.

Method Detail

getCheckpointPath

@Nullable
public Path getCheckpointPath()

Gets the checkpoint base directory. Jobs will create job-specific subdirectories for checkpoints within this directory. May be null, if not configured.

Returns:

The checkpoint base directory

getSavepointPath

@Nullable
public Path getSavepointPath()

Gets the directory where savepoints are stored by default (when no custom path is given to the savepoint trigger command).

Returns:

The default directory for savepoints, or null, if no default directory has been configured.

resolveCheckpoint

public CompletedCheckpointStorageLocation resolveCheckpoint(String pointer)
                                                     throws IOException

Description copied from interface: StateBackend

Resolves the given pointer to a checkpoint/savepoint into a checkpoint location. The location supports reading the checkpoint metadata, or disposing the checkpoint storage location.

If the state backend cannot understand the format of the pointer (for example because it was created by a different state backend) this method should throw an IOException.

Parameters:

pointer - The external checkpoint pointer to resolve.

Returns:

The checkpoint location handle.

Throws:

IOException - Thrown, if the state backend does not understand the pointer, or if the pointer could not be resolved due to an I/O error.