Modifier and Type | Class and Description |
---|---|
class |
RecordsBySplits<E>
An implementation of RecordsWithSplitIds to host all the records by splits.
|
Modifier and Type | Method and Description |
---|---|
RecordsWithSplitIds<E> |
SplitReader.fetch()
Fetch elements into the blocking queue for the given splits.
|
Modifier and Type | Class and Description |
---|---|
class |
FileRecords<T>
A collection of records for one file split.
|
Modifier and Type | Method and Description |
---|---|
RecordsWithSplitIds<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>> |
KafkaPartitionSplitReader.fetch() |
Constructor and Description |
---|
KafkaSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>>> elementsQueue,
KafkaSourceFetcherManager kafkaSourceFetcherManager,
RecordEmitter<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,T,KafkaPartitionSplitState> recordEmitter,
Configuration config,
SourceReaderContext context,
KafkaSourceReaderMetrics kafkaSourceReaderMetrics) |
Constructor and Description |
---|
KafkaSourceFetcherManager(FutureCompletingBlockingQueue<RecordsWithSplitIds<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>>> elementsQueue,
java.util.function.Supplier<SplitReader<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>> splitReaderSupplier,
java.util.function.Consumer<Collection<String>> splitFinishedHook)
Creates a new SplitFetcherManager with a single I/O threads.
|
Constructor and Description |
---|
PulsarFetcherManagerBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<T>>> elementsQueue,
java.util.function.Supplier<SplitReader<PulsarMessage<T>,PulsarPartitionSplit>> splitReaderSupplier)
Creates a new SplitFetcherManager with multiple I/O threads.
|
PulsarOrderedFetcherManager(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<T>>> elementsQueue,
java.util.function.Supplier<SplitReader<PulsarMessage<T>,PulsarPartitionSplit>> splitReaderSupplier) |
PulsarUnorderedFetcherManager(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<T>>> elementsQueue,
java.util.function.Supplier<SplitReader<PulsarMessage<T>,PulsarPartitionSplit>> splitReaderSupplier) |
Constructor and Description |
---|
PulsarOrderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<OUT>>> elementsQueue,
java.util.function.Supplier<PulsarOrderedPartitionSplitReader<OUT>> splitReaderSupplier,
SourceReaderContext context,
SourceConfiguration sourceConfiguration,
org.apache.pulsar.client.api.PulsarClient pulsarClient,
org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin) |
PulsarUnorderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<OUT>>> elementsQueue,
java.util.function.Supplier<PulsarUnorderedPartitionSplitReader<OUT>> splitReaderSupplier,
SourceReaderContext context,
SourceConfiguration sourceConfiguration,
org.apache.pulsar.client.api.PulsarClient pulsarClient,
org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin,
org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient coordinatorClient) |
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.