org.apache.flink.streaming.connectors.kinesis

Interface KinesisShardAssigner

All Superinterfaces:

Serializable

@PublicEvolving
public interface KinesisShardAssigner
extends Serializable

Utility to map Kinesis shards to Flink subtask indices. Users can implement this interface to optimize distribution of shards over subtasks. See assign(StreamShardHandle, int) for details.

Method Summary

Modifier and Type	Method and Description
int	assign(StreamShardHandle shard, int numParallelSubtasks) Returns the index of the target subtask that a specific shard should be assigned to.

Method Detail

assign

int assign(StreamShardHandle shard,
           int numParallelSubtasks)

Returns the index of the target subtask that a specific shard should be assigned to. For return values outside the subtask range, modulus operation will be applied automatically, hence it is also valid to just return a hash code.

The resulting distribution of shards should have the following contract:

• 1. Uniform distribution across subtasks
• 2. Deterministic, calls for a given shard always return same index.

The above contract is crucial and cannot be broken. Consumer subtasks rely on this contract to filter out shards that they should not subscribe to, guaranteeing that each shard of a stream will always be assigned to one subtask in a uniformly distributed manner.

Parameters:

shard - the shard to determine

numParallelSubtasks - total number of subtasks

Returns:

target index, if index falls outside of the range, modulus operation will be applied