org.apache.flink.connector.pulsar.source.reader.fetcher

Class PulsarFetcherManagerBase<T>

java.lang.Object

org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager<PulsarMessage<T>,PulsarPartitionSplit>

org.apache.flink.connector.pulsar.source.reader.fetcher.PulsarFetcherManagerBase<T>

Type Parameters:

T - The decoded message type for flink.

Direct Known Subclasses:

PulsarOrderedFetcherManager, PulsarUnorderedFetcherManager

@Internal
public abstract class PulsarFetcherManagerBase<T>
extends SplitFetcherManager<PulsarMessage<T>,PulsarPartitionSplit>

Common fetcher manager abstraction for both ordered & unordered message.

Field Summary

Fields inherited from class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

fetchers

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	PulsarFetcherManagerBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<T>>> elementsQueue, java.util.function.Supplier<SplitReader<PulsarMessage<T>,PulsarPartitionSplit>> splitReaderSupplier) Creates a new SplitFetcherManager with multiple I/O threads.

Method Summary

Modifier and Type	Method and Description
void	addSplits(List<PulsarPartitionSplit> splitsToAdd) Override this method for supporting multiple thread fetching, one fetcher thread for one split.
protected SplitFetcher<PulsarMessage<T>,PulsarPartitionSplit>	getOrCreateFetcher(String splitId)
protected void	startFetcher(SplitFetcher<PulsarMessage<T>,PulsarPartitionSplit> fetcher)

Methods inherited from class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

checkErrors, close, createSplitFetcher, getNumAliveFetchers, maybeShutdownFinishedFetchers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PulsarFetcherManagerBase

protected PulsarFetcherManagerBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<T>>> elementsQueue,
                                   java.util.function.Supplier<SplitReader<PulsarMessage<T>,PulsarPartitionSplit>> splitReaderSupplier)

Creates a new SplitFetcherManager with multiple I/O threads.

Parameters:

elementsQueue - The queue that is used to hand over data from the I/O thread (the fetchers) to the reader (which emits the records and book-keeps the state. This must be the same queue instance that is also passed to the SourceReaderBase.

splitReaderSupplier - The factory for the split reader that connects to the source

Method Detail

addSplits

public void addSplits(List<PulsarPartitionSplit> splitsToAdd)

Override this method for supporting multiple thread fetching, one fetcher thread for one split.

Specified by:

addSplits in class SplitFetcherManager<PulsarMessage<T>,PulsarPartitionSplit>

startFetcher

protected void startFetcher(SplitFetcher<PulsarMessage<T>,PulsarPartitionSplit> fetcher)

Overrides:

startFetcher in class SplitFetcherManager<PulsarMessage<T>,PulsarPartitionSplit>

getOrCreateFetcher

protected SplitFetcher<PulsarMessage<T>,PulsarPartitionSplit> getOrCreateFetcher(String splitId)