org.apache.flink.streaming.connectors.kafka.internals

Class Kafka08PartitionDiscoverer

java.lang.Object

org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer

org.apache.flink.streaming.connectors.kafka.internals.Kafka08PartitionDiscoverer

@Internal
public class Kafka08PartitionDiscoverer
extends AbstractPartitionDiscoverer

A partition discoverer that can be used to discover topics and partitions metadata from Kafka brokers via the Kafka 0.8 low-level consumer API.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer

AbstractPartitionDiscoverer.ClosedException, AbstractPartitionDiscoverer.WakeupException

Constructor Summary

Constructors

Constructor and Description
Kafka08PartitionDiscoverer(KafkaTopicsDescriptor topicsDescriptor, int indexOfThisSubtask, int numParallelSubtasks, Properties kafkaProperties)

Method Summary

Modifier and Type	Method and Description
protected void	closeConnections() Close all established connections.
List<KafkaTopicPartition>	getAllPartitionsForTopics(List<String> topics) Fetch the list of all partitions for a specific topics list from Kafka.
protected List<String>	getAllTopics() Fetch the list of all topics from Kafka.
List<KafkaTopicPartitionLeader>	getPartitionLeadersForTopics(List<String> topics) Send request to Kafka to get partitions for topics.
protected void	initializeConnections() Establish the required connections in order to fetch topics and partitions metadata.
protected void	wakeupConnections() Attempt to eagerly wakeup from blocking calls to Kafka in AbstractPartitionDiscoverer.getAllTopics() and AbstractPartitionDiscoverer.getAllPartitionsForTopics(List).

Methods inherited from class org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer

close, discoverPartitions, open, setAndCheckDiscoveredPartition, wakeup

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Kafka08PartitionDiscoverer

public Kafka08PartitionDiscoverer(KafkaTopicsDescriptor topicsDescriptor,
                                  int indexOfThisSubtask,
                                  int numParallelSubtasks,
                                  Properties kafkaProperties)

Method Detail

initializeConnections

protected void initializeConnections()

Description copied from class: AbstractPartitionDiscoverer

Establish the required connections in order to fetch topics and partitions metadata.

Specified by:

initializeConnections in class AbstractPartitionDiscoverer

getAllTopics

protected List<String> getAllTopics()

Description copied from class: AbstractPartitionDiscoverer

Fetch the list of all topics from Kafka.

Specified by:

getAllTopics in class AbstractPartitionDiscoverer

getAllPartitionsForTopics

public List<KafkaTopicPartition> getAllPartitionsForTopics(List<String> topics)

Description copied from class: AbstractPartitionDiscoverer

Fetch the list of all partitions for a specific topics list from Kafka.

Specified by:

getAllPartitionsForTopics in class AbstractPartitionDiscoverer

wakeupConnections

protected void wakeupConnections()

Description copied from class: AbstractPartitionDiscoverer

Attempt to eagerly wakeup from blocking calls to Kafka in AbstractPartitionDiscoverer.getAllTopics() and AbstractPartitionDiscoverer.getAllPartitionsForTopics(List).

If the invocation indeed results in interrupting an actual blocking Kafka call, the implementations of AbstractPartitionDiscoverer.getAllTopics() and AbstractPartitionDiscoverer.getAllPartitionsForTopics(List) are responsible of throwing a AbstractPartitionDiscoverer.WakeupException.

Specified by:

wakeupConnections in class AbstractPartitionDiscoverer

closeConnections

protected void closeConnections()
                         throws Exception

Description copied from class: AbstractPartitionDiscoverer

Close all established connections.

Specified by:

closeConnections in class AbstractPartitionDiscoverer

Throws:

Exception

getPartitionLeadersForTopics

public List<KafkaTopicPartitionLeader> getPartitionLeadersForTopics(List<String> topics)

Send request to Kafka to get partitions for topics.

Parameters:

topics - The name of the topics.