public class KafkaConsumerThread extends Thread
KafkaConsumer
, connecting to the brokers and polling records.
The thread pushes the data into a Handover
to be picked up by the fetcher that will
deserialize and emit the records.
IMPORTANT: This thread must not be interrupted when attempting to shut it down. The Kafka consumer code was found to not always handle interrupts well, and to even deadlock in certain situations.
Implementation Note: This code is written to be reusable in later versions of the KafkaConsumer. Because Kafka is not maintaining binary compatibility, we use a "call bridge" as an indirection to the KafkaConsumer calls that change signature.
Thread.State, Thread.UncaughtExceptionHandler
MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY
Constructor and Description |
---|
KafkaConsumerThread(org.slf4j.Logger log,
Handover handover,
Properties kafkaProperties,
KafkaTopicPartitionState<org.apache.kafka.common.TopicPartition>[] subscribedPartitions,
MetricGroup kafkaMetricGroup,
KafkaConsumerCallBridge consumerCallBridge,
String threadName,
long pollTimeout,
boolean useMetrics) |
Modifier and Type | Method and Description |
---|---|
void |
run() |
void |
setOffsetsToCommit(Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata> offsetsToCommit)
Tells this thread to commit a set of offsets.
|
void |
shutdown()
Shuts this thread down, waking up the thread gracefully if blocked (without Thread.interrupt() calls).
|
activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield
public KafkaConsumerThread(org.slf4j.Logger log, Handover handover, Properties kafkaProperties, KafkaTopicPartitionState<org.apache.kafka.common.TopicPartition>[] subscribedPartitions, MetricGroup kafkaMetricGroup, KafkaConsumerCallBridge consumerCallBridge, String threadName, long pollTimeout, boolean useMetrics)
public void shutdown()
public void setOffsetsToCommit(Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata> offsetsToCommit)
Only one commit operation may be pending at any time. If the committing takes longer than the frequency with which this method is called, then some commits may be skipped due to being superseded by newer ones.
offsetsToCommit
- The offsets to commitCopyright © 2014–2017 The Apache Software Foundation. All rights reserved.