Package org.apache.flink.connector.base.source.reader.fetcher

Class SingleThreadFetcherManager<E,​SplitT extends SourceSplit>

  • @PublicEvolving
public class SingleThreadFetcherManager<E,​SplitT extends SourceSplit>
extends SplitFetcherManager<E,​SplitT>
    A Fetcher Manager with a single fetching thread (I/O thread) that handles all splits concurrently.

    This pattern is, for example, useful for connectors like File Readers, Apache Kafka Readers, etc. In the example of Kafka, there is a single thread that reads all splits (topic partitions) via the same client. In the example of the file source, there is a single thread that reads the files after another.

    • Constructor Detail

      • SingleThreadFetcherManager

        public SingleThreadFetcherManager​(Supplier<SplitReader<E,​SplitT>> splitReaderSupplier)
        Creates a new SplitFetcherManager with a single I/O threads.
        Parameters:
        splitReaderSupplier - The factory for the split reader that connects to the source system.

      • SingleThreadFetcherManager

        public SingleThreadFetcherManager​(Supplier<SplitReader<E,​SplitT>> splitReaderSupplier,
                                  Configuration configuration)
        Creates a new SplitFetcherManager with a single I/O threads.
        Parameters:
        splitReaderSupplier - The factory for the split reader that connects to the source system.
        configuration - The configuration to create the fetcher manager.

      • SingleThreadFetcherManager

        public SingleThreadFetcherManager​(Supplier<SplitReader<E,​SplitT>> splitReaderSupplier,
                                  Configuration configuration,
                                  Consumer<Collection<String>> splitFinishedHook)
        Creates a new SplitFetcherManager with a single I/O threads.
        Parameters:
        splitReaderSupplier - The factory for the split reader that connects to the source system.
        configuration - The configuration to create the fetcher manager.
        splitFinishedHook - Hook for handling finished splits in split fetchers