org.apache.flink.runtime.checkpoint

Class CompletedCheckpoint

java.lang.Object

org.apache.flink.runtime.checkpoint.CompletedCheckpoint

All Implemented Interfaces:

Serializable, Checkpoint

@NotThreadSafe
public class CompletedCheckpoint
extends Object
implements Serializable, Checkpoint

A CompletedCheckpoint describes a checkpoint after all required tasks acknowledged it (with their state) and that is considered successful. The CompletedCheckpoint class contains all the metadata of the checkpoint, i.e., checkpoint ID, timestamps, and the handles to all states that are part of the checkpoint.

Size the CompletedCheckpoint Instances

In most cases, the CompletedCheckpoint objects are very small, because the handles to the checkpoint states are only pointers (such as file paths). However, the some state backend implementations may choose to store some payload data directly with the metadata (for example to avoid many small files). If those thresholds are increased to large values, the memory consumption of the CompletedCheckpoint objects can be significant.

Metadata Persistence

The metadata of the CompletedCheckpoint is also persisted in an external storage system. Checkpoints have an external pointer, which points to the metadata. For example when storing a checkpoint in a file system, that pointer is the file path to the checkpoint's folder or the metadata file. For a state backend that stores metadata in database tables, the pointer could be the table name and row key. The pointer is encoded as a String.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	CompletedCheckpoint.CompletedCheckpointDiscardObject Implementation of Checkpoint.DiscardObject.

Nested classes/interfaces inherited from interface org.apache.flink.runtime.checkpoint.Checkpoint

Checkpoint.DiscardObject

Field Summary

Fields inherited from interface org.apache.flink.runtime.checkpoint.Checkpoint

NOOP_DISCARD_OBJECT

Constructor Summary

Constructors

Constructor and Description
CompletedCheckpoint(JobID job, long checkpointID, long timestamp, long completionTimestamp, Map<OperatorID,OperatorState> operatorStates, Collection<MasterState> masterHookStates, CheckpointProperties props, CompletedCheckpointStorageLocation storageLocation, CompletedCheckpointStats completedCheckpointStats)
CompletedCheckpoint(JobID job, long checkpointID, long timestamp, long completionTimestamp, Map<OperatorID,OperatorState> operatorStates, Collection<MasterState> masterHookStates, CheckpointProperties props, CompletedCheckpointStorageLocation storageLocation, CompletedCheckpointStats completedCheckpointStats, CheckpointProperties restoredProps)

Method Summary

Modifier and Type	Method and Description
static boolean	checkpointsMatch(Collection<CompletedCheckpoint> first, Collection<CompletedCheckpoint> second)
long	getCheckpointID()
long	getCompletionTimestamp()
String	getExternalPointer()
JobID	getJobId()
Collection<MasterState>	getMasterHookStates()
StreamStateHandle	getMetadataHandle()
Map<OperatorID,OperatorState>	getOperatorStates()
CheckpointProperties	getProperties()
Optional<CheckpointProperties>	getRestoredProperties()
long	getStateSize()
CompletedCheckpointStats	getStatistic()
long	getTimestamp()
Checkpoint.DiscardObject	markAsDiscarded() This method precede the Checkpoint.DiscardObject.discard() method and should be called from the CheckpointCoordinator(under the lock) while Checkpoint.DiscardObject.discard() can be called from any thread/place.
Checkpoint.DiscardObject	markAsDiscardedOnShutdown(JobStatus jobStatus)
Checkpoint.DiscardObject	markAsDiscardedOnSubsume()
void	registerSharedStatesAfterRestored(SharedStateRegistry sharedStateRegistry, RestoreMode restoreMode) Register all shared states in the given registry.
boolean	shouldBeDiscardedOnShutdown(JobStatus jobStatus)
boolean	shouldBeDiscardedOnSubsume()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CompletedCheckpoint

public CompletedCheckpoint(JobID job,
                           long checkpointID,
                           long timestamp,
                           long completionTimestamp,
                           Map<OperatorID,OperatorState> operatorStates,
                           @Nullable
                           Collection<MasterState> masterHookStates,
                           CheckpointProperties props,
                           CompletedCheckpointStorageLocation storageLocation,
                           @Nullable
                           CompletedCheckpointStats completedCheckpointStats)

CompletedCheckpoint

public CompletedCheckpoint(JobID job,
                           long checkpointID,
                           long timestamp,
                           long completionTimestamp,
                           Map<OperatorID,OperatorState> operatorStates,
                           @Nullable
                           Collection<MasterState> masterHookStates,
                           CheckpointProperties props,
                           CompletedCheckpointStorageLocation storageLocation,
                           @Nullable
                           CompletedCheckpointStats completedCheckpointStats,
                           @Nullable
                           CheckpointProperties restoredProps)

Method Detail

getJobId

public JobID getJobId()

getCheckpointID

public long getCheckpointID()

Specified by:

getCheckpointID in interface Checkpoint

getTimestamp

public long getTimestamp()

getCompletionTimestamp

public long getCompletionTimestamp()

getProperties

public CheckpointProperties getProperties()

getRestoredProperties

public Optional<CheckpointProperties> getRestoredProperties()

getOperatorStates

public Map<OperatorID,OperatorState> getOperatorStates()

getMasterHookStates

public Collection<MasterState> getMasterHookStates()

getMetadataHandle

public StreamStateHandle getMetadataHandle()

getExternalPointer

public String getExternalPointer()

getStateSize

public long getStateSize()

registerSharedStatesAfterRestored

public void registerSharedStatesAfterRestored(SharedStateRegistry sharedStateRegistry,
                                              RestoreMode restoreMode)

Register all shared states in the given registry. This method is called before the checkpoint is added into the store.

Parameters:

sharedStateRegistry - The registry where shared states are registered

restoreMode - the mode in which this checkpoint was restored from

markAsDiscarded

public Checkpoint.DiscardObject markAsDiscarded()

Description copied from interface: Checkpoint

This method precede the Checkpoint.DiscardObject.discard() method and should be called from the CheckpointCoordinator(under the lock) while Checkpoint.DiscardObject.discard() can be called from any thread/place.

Specified by:

markAsDiscarded in interface Checkpoint

markAsDiscardedOnSubsume

public Checkpoint.DiscardObject markAsDiscardedOnSubsume()

markAsDiscardedOnShutdown

public Checkpoint.DiscardObject markAsDiscardedOnShutdown(JobStatus jobStatus)

shouldBeDiscardedOnSubsume

public boolean shouldBeDiscardedOnSubsume()

shouldBeDiscardedOnShutdown

public boolean shouldBeDiscardedOnShutdown(JobStatus jobStatus)

checkpointsMatch

public static boolean checkpointsMatch(Collection<CompletedCheckpoint> first,
                                       Collection<CompletedCheckpoint> second)

getStatistic

@Nullable
public CompletedCheckpointStats getStatistic()

toString

public String toString()

Overrides:

toString in class Object