org.apache.flink.streaming.connectors.kafka.internals

Class ZookeeperOffsetHandler

java.lang.Object

org.apache.flink.streaming.connectors.kafka.internals.ZookeeperOffsetHandler

@Internal
public class ZookeeperOffsetHandler
extends Object

Handler for committing Kafka offsets to Zookeeper and to retrieve them again.

Constructor Summary

Constructors

Constructor and Description
ZookeeperOffsetHandler(Properties props)

Method Summary

Modifier and Type	Method and Description
void	close() Closes the offset handler.
Long	getCommittedOffset(KafkaTopicPartition partition)
static Long	getOffsetFromZooKeeper(org.apache.curator.framework.CuratorFramework curatorClient, String groupId, String topic, int partition)
void	prepareAndCommitOffsets(Map<KafkaTopicPartition,Long> internalOffsets) Commits offsets for Kafka partitions to ZooKeeper.
static void	setOffsetInZooKeeper(org.apache.curator.framework.CuratorFramework curatorClient, String groupId, String topic, int partition, long offset)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZookeeperOffsetHandler

public ZookeeperOffsetHandler(Properties props)

Method Detail

prepareAndCommitOffsets

public void prepareAndCommitOffsets(Map<KafkaTopicPartition,Long> internalOffsets)
                             throws Exception

Commits offsets for Kafka partitions to ZooKeeper. The given offsets to this method should be the offsets of the last processed records; this method will take care of incrementing the offsets by 1 before committing them so that the committed offsets to Zookeeper represent the next record to process.

Parameters:

internalOffsets - The internal offsets (representing last processed records) for the partitions to commit.

Throws:

Exception - The method forwards exceptions.

getCommittedOffset

public Long getCommittedOffset(KafkaTopicPartition partition)
                        throws Exception

Parameters:

partition - The partition to read offset for.

Returns:

The mapping from partition to offset.

Throws:

Exception - This method forwards exceptions.

close

public void close()
           throws IOException

Closes the offset handler.

Throws:

IOException - Thrown, if the handler cannot be closed properly.

setOffsetInZooKeeper

public static void setOffsetInZooKeeper(org.apache.curator.framework.CuratorFramework curatorClient,
                                        String groupId,
                                        String topic,
                                        int partition,
                                        long offset)
                                 throws Exception

Throws:

Exception

getOffsetFromZooKeeper

public static Long getOffsetFromZooKeeper(org.apache.curator.framework.CuratorFramework curatorClient,
                                          String groupId,
                                          String topic,
                                          int partition)
                                   throws Exception

Throws:

Exception