@Internal public class FlinkKafkaShuffleConsumer<T> extends FlinkKafkaConsumer<T>
SourceFunction.SourceContext<T>
DEFAULT_POLL_TIMEOUT, KEY_POLL_TIMEOUT, pollTimeout, properties
deserializer, KEY_DISABLE_METRICS, KEY_PARTITION_DISCOVERY_INTERVAL_MILLIS, LOG, MAX_NUM_PENDING_CHECKPOINTS, PARTITION_DISCOVERY_DISABLED
Modifier and Type | Method and Description |
---|---|
protected AbstractFetcher<T,?> |
createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<WatermarkStrategy<T>> watermarkStrategy,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode,
MetricGroup consumerMetricGroup,
boolean useMetrics)
Creates the fetcher that connect to the Kafka brokers, pulls data, deserialized the data, and
emits it into the data streams.
|
createPartitionDiscoverer, fetchOffsetsWithTimestamp, getIsAutoCommitEnabled
adjustAutoCommitConfig, assignTimestampsAndWatermarks, assignTimestampsAndWatermarks, assignTimestampsAndWatermarks, cancel, close, disableFilterRestoredPartitionsWithSubscribedTopics, getEnableCommitOnCheckpoints, getProducedType, initializeState, notifyCheckpointAborted, notifyCheckpointComplete, open, run, setCommitOffsetsOnCheckpoints, setStartFromEarliest, setStartFromGroupOffsets, setStartFromLatest, setStartFromSpecificOffsets, setStartFromTimestamp, snapshotState
getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
protected AbstractFetcher<T,?> createFetcher(SourceFunction.SourceContext<T> sourceContext, Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets, SerializedValue<WatermarkStrategy<T>> watermarkStrategy, StreamingRuntimeContext runtimeContext, OffsetCommitMode offsetCommitMode, MetricGroup consumerMetricGroup, boolean useMetrics) throws Exception
FlinkKafkaConsumerBase
createFetcher
in class FlinkKafkaConsumer<T>
sourceContext
- The source context to emit data to.assignedPartitionsWithInitialOffsets
- The set of partitions that this subtask should
handle, with their start offsets.watermarkStrategy
- Optional, a serialized WatermarkStrategy.runtimeContext
- The task's runtime context.Exception
- The method should forward exceptionsCopyright © 2014–2024 The Apache Software Foundation. All rights reserved.