@Internal public class KafkaSourceFetcherManager extends SingleThreadFetcherManager<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>
KafkaPartitionSplitReader
.fetchers
Constructor and Description |
---|
KafkaSourceFetcherManager(FutureCompletingBlockingQueue<RecordsWithSplitIds<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>>> elementsQueue,
java.util.function.Supplier<SplitReader<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>> splitReaderSupplier,
java.util.function.Consumer<Collection<String>> splitFinishedHook)
Creates a new SplitFetcherManager with a single I/O threads.
|
Modifier and Type | Method and Description |
---|---|
void |
commitOffsets(Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata> offsetsToCommit,
org.apache.kafka.clients.consumer.OffsetCommitCallback callback) |
addSplits, getRunningFetcher
checkErrors, close, createSplitFetcher, getNumAliveFetchers, maybeShutdownFinishedFetchers, pauseOrResumeSplits, startFetcher
public KafkaSourceFetcherManager(FutureCompletingBlockingQueue<RecordsWithSplitIds<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>>> elementsQueue, java.util.function.Supplier<SplitReader<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>> splitReaderSupplier, java.util.function.Consumer<Collection<String>> splitFinishedHook)
elementsQueue
- The queue that is used to hand over data from the I/O thread (the
fetchers) to the reader (which emits the records and book-keeps the state. This must be
the same queue instance that is also passed to the SourceReaderBase
.splitReaderSupplier
- The factory for the split reader that connects to the source
system.splitFinishedHook
- Hook for handling finished splits in split fetchers.public void commitOffsets(Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata> offsetsToCommit, org.apache.kafka.clients.consumer.OffsetCommitCallback callback)
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.