@PublicEvolving public interface SplitEnumerator<SplitT extends SourceSplit,CheckpointT> extends AutoCloseable
SourceReader
to read. 2. assign the splits to the source reader.Modifier and Type | Method and Description |
---|---|
void |
addReader(int subtaskId)
Add a new source reader with the given subtask ID.
|
void |
addSplitsBack(List<SplitT> splits,
int subtaskId)
Add a split back to the split enumerator.
|
void |
close()
Called to close the enumerator, in case it holds on to any resources, like threads or network
connections.
|
default void |
handleSourceEvent(int subtaskId,
SourceEvent sourceEvent)
Handles a custom source event from the source reader.
|
void |
handleSplitRequest(int subtaskId,
String requesterHostname)
Handles the request for a split.
|
default void |
notifyCheckpointAborted(long checkpointId)
Called when a checkpoint is aborted.
|
default void |
notifyCheckpointComplete(long checkpointId)
We have an empty default implementation here because most source readers do not have to
implement the method.
|
CheckpointT |
snapshotState()
Checkpoints the state of this split enumerator.
|
void |
start()
Start the split enumerator.
|
void start()
The default behavior does nothing.
void handleSplitRequest(int subtaskId, @Nullable String requesterHostname)
SourceReaderContext.sendSplitRequest()
method.subtaskId
- the subtask id of the source reader who sent the source event.requesterHostname
- Optional, the hostname where the requesting task is running. This
can be used to make split assignments locality-aware.void addSplitsBack(List<SplitT> splits, int subtaskId)
SourceReader
fails and there are splits assigned to it after the last successful checkpoint.splits
- The split to add back to the enumerator for reassignment.subtaskId
- The id of the subtask to which the returned splits belong.void addReader(int subtaskId)
subtaskId
- the subtask ID of the new source reader.CheckpointT snapshotState() throws Exception
Exception
- when the snapshot cannot be taken.void close() throws IOException
close
in interface AutoCloseable
IOException
default void notifyCheckpointComplete(long checkpointId) throws Exception
Exception
default void notifyCheckpointAborted(long checkpointId)
NOTE: This method is here only in the backport to the Flink 1.11 branch. In 1.12, this default method is inherited from the CheckpointListener interface.
default void handleSourceEvent(int subtaskId, SourceEvent sourceEvent)
This method has a default implementation that does nothing, because it is only required to
be implemented by some sources, which have a custom event protocol between reader and
enumerator. The common events for reader registration and split requests are not dispatched
to this method, but rather invoke the addReader(int)
and handleSplitRequest(int, String)
methods.
subtaskId
- the subtask id of the source reader who sent the source event.sourceEvent
- the source event from the source reader.Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.