org.apache.flink.streaming.connectors.fs.bucketing

Interface Bucketer<T>

All Superinterfaces:

Serializable

All Known Implementing Classes:

BasePathBucketer, BucketingSinkTestProgram.KeyBucketer, DateTimeBucketer

public interface Bucketer<T>
extends Serializable

A bucketer is used with a BucketingSink to put emitted elements into rolling files.

The BucketingSink can be writing to many buckets at a time, and it is responsible for managing a set of active buckets. Whenever a new element arrives it will ask the Bucketer for the bucket path the element should fall in. The Bucketer can, for example, determine buckets based on system time.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.Path	getBucketPath(Clock clock, org.apache.hadoop.fs.Path basePath, T element) Returns the Path of a bucket file.

Method Detail

getBucketPath

org.apache.hadoop.fs.Path getBucketPath(Clock clock,
                                        org.apache.hadoop.fs.Path basePath,
                                        T element)

Returns the Path of a bucket file.

Parameters:

basePath - The base path containing all the buckets.

element - The current element being processed.

Returns:

The complete Path of the bucket which the provided element should fall in. This should include the basePath and also the subtaskIndex to avoid clashes with parallel sinks.