E - the element type.
SplitT - the split type.
@PublicEvolving public interface SplitReader<E,SplitT extends SourceSplit> extends AutoCloseable
|Modifier and Type
|Method and Description
Fetch elements into the blocking queue for the given splits.
Handle the split changes.
Pauses or resumes reading of individual splits readers.
Wake up the split reader in case the fetcher thread is blocking in
RecordsWithSplitIds<E> fetch() throws IOException
wakeUp() is invoked. In that case, the
implementation may either decide to return without throwing an exception, or it can just
throw an interrupted exception. In either case, this method should be reentrant, meaning that
the next fetch call should just resume from where the last fetch call was waken up or
interrupted. It is up to the implementer to either read all the records of the split or to
stop reading them at some point (for example when a given threshold is exceeded). In that
later case, when fetch is called again, the reading should restart at the record where it
left off based on the
IOException - when encountered IO errors, such as deserialization failures.
void handleSplitsChanges(SplitsChange<SplitT> splitsChanges)
For the consistency of internal state in SourceReaderBase, if an invalid split is added to
the reader (for example splits without any records), it should be put back into
RecordsWithSplitIds as finished splits so that SourceReaderBase could be able to clean up
resources created for it.
For the consistency of internal state in SourceReaderBase, if a split is removed, it
should be put back into
RecordsWithSplitIds as finished splits so that
SourceReaderBase could be able to clean up resources created for it.
splitsChanges - the split changes that the SplitReader needs to handle.
default void pauseOrResumeSplits(Collection<SplitT> splitsToPause, Collection<SplitT> splitsToResume)
Note that no other methods can be called in parallel, so it's fine to non-atomically update subscriptions. This method is simply providing connectors with more expressive APIs the opportunity to update all subscriptions at once.
This is currently used to align the watermarks of splits, if watermark alignment is used and the source reads from more than one split.
The default implementation throws an
UnsupportedOperationException where the
default implementation will be removed in future releases. To be compatible with future
releases, it is recommended to implement this method and override the default implementation.
splitsToPause - the splits to pause
splitsToResume - the splits to resume
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.