T
- The type of records produced by the source.SplitT
- The type of splits handled by the source.@Experimental @PublicEvolving public interface ExternallyInducedSourceReader<T,SplitT extends SourceSplit> extends SourceReader<T,SplitT>
The ExternallyInducedSourceReader tells the Flink runtime that a checkpoint needs to be made by returning a checkpointId when shouldTriggerCheckpoint() is invoked.
The implementations typically works together with the SplitEnumerator which informs the external system to trigger a checkpoint. The external system also needs to forward the Checkpoint ID to the source, so the source knows which checkpoint to trigger.
Important: It is crucial that all parallel source tasks trigger their checkpoints at roughly the same time. Otherwise this leads to performance issues due to long checkpoint alignment phases or large alignment data snapshots.
Modifier and Type | Method and Description |
---|---|
Optional<Long> |
shouldTriggerCheckpoint()
A method that informs the Flink runtime whether a checkpoint should be triggered on this
Source.
|
addSplits, handleSourceEvents, isAvailable, notifyCheckpointComplete, notifyNoMoreSplits, pollNext, snapshotState, start
close
notifyCheckpointAborted
Optional<Long> shouldTriggerCheckpoint()
This method is invoked when the previous SourceReader.pollNext(ReaderOutput)
returns InputStatus.NOTHING_AVAILABLE
, to check if the source needs to be
checkpointed.
If a CheckpointId is returned, a checkpoint will be triggered on this source reader. Otherwise, Flink runtime will continue to process the records.
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.