pyflink.datastream.checkpoint_config.CheckpointConfig.set_externalized_checkpoint_cleanup#

CheckpointConfig.set_externalized_checkpoint_cleanup(cleanup_mode: pyflink.datastream.checkpoint_config.ExternalizedCheckpointCleanup) → pyflink.datastream.checkpoint_config.CheckpointConfig[source]#

Sets the mode for externalized checkpoint clean-up. Externalized checkpoints will be enabled automatically unless the mode is set to ExternalizedCheckpointCleanup.NO_EXTERNALIZED_CHECKPOINTS.

Externalized checkpoints write their meta data out to persistent storage and are not automatically cleaned up when the owning job fails or is suspended (terminating with job status FAILED or SUSPENDED). In this case, you have to manually clean up the checkpoint state, both the meta data and actual program state.

The ExternalizedCheckpointCleanup mode defines how an externalized checkpoint should be cleaned up on job cancellation. If you choose to retain externalized checkpoints on cancellation you have to handle checkpoint clean-up manually when you cancel the job as well (terminating with job status CANCELED).

The target directory for externalized checkpoints is configured via org.apache.flink.configuration.CheckpointingOptions#CHECKPOINTS_DIRECTORY.

Example:

>>> config.set_externalized_checkpoint_cleanup(
...     ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION)

Parameters: cleanup_mode – Externalized checkpoint clean-up behaviour, the mode could be ExternalizedCheckpointCleanup.DELETE_ON_CANCELLATION, ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION or ExternalizedCheckpointCleanup.NO_EXTERNALIZED_CHECKPOINTS