IN- The type of input elements.
BucketID- The type of the object returned by the
getBucketId(Object, BucketAssigner.Context). This has to have a correct
#equals(Object)method. In addition, the
Pathto the created bucket will be the result of the
#toString()of this method, appended to the
basePathspecified in the file sink.
@PublicEvolving public interface BucketAssigner<IN,BucketID> extends Serializable
StreamingFileSink can be writing to many buckets at a time, and it is responsible
for managing a set of active buckets. Whenever a new element arrives it will ask the
BucketAssigner for the bucket the element should fall in. The
BucketAssigner can, for
example, determine buckets based on system time.
|Modifier and Type||Interface and Description|
Context that the
|Modifier and Type||Method and Description|
Returns the identifier of the bucket the provided element should be put into.
BucketID getBucketId(IN element, BucketAssigner.Context context)
element- The current element being processed.
context- The context used by the current bucket assigner.
base pathprovided during the initialization of the file sink.
SimpleVersionedSerializercapable of serializing/deserializing the elements of type
BucketID. That is the type of the objects returned by the
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.