@PublicEvolving public class DateTimeBucketAssigner<IN> extends Object implements BucketAssigner<IN,String>
BucketAssigner
that assigns to buckets based on current system time.
The DateTimeBucketer
will create directories of the following form: /{basePath}/{dateTimePath}/
. The basePath
is the path that was specified as a base path
when creating the StreamingFileSink
. The dateTimePath
is determined based on the current system time and the user provided format string.
DateTimeFormatter
is used to derive a date string from the current system time and the
date format string. The default format string is "yyyy-MM-dd--HH"
so the rolling files
will have a granularity of hours.
Example:
BucketAssigner bucketAssigner = new DateTimeBucketAssigner("yyyy-MM-dd--HH");
This will create for example the following bucket path: /base/1976-12-31-14/
BucketAssigner.Context
Constructor and Description |
---|
DateTimeBucketAssigner()
Creates a new
DateTimeBucketAssigner with format string "yyyy-MM-dd--HH" . |
DateTimeBucketAssigner(String formatString)
Creates a new
DateTimeBucketAssigner with the given date/time format string. |
DateTimeBucketAssigner(String formatString,
java.time.ZoneId zoneId)
Creates a new
DateTimeBucketAssigner with the given date/time format string using the
given timezone. |
DateTimeBucketAssigner(java.time.ZoneId zoneId)
Creates a new
DateTimeBucketAssigner with format string "yyyy-MM-dd--HH"
using the given timezone. |
Modifier and Type | Method and Description |
---|---|
String |
getBucketId(IN element,
BucketAssigner.Context context)
Returns the identifier of the bucket the provided element should be put into.
|
SimpleVersionedSerializer<String> |
getSerializer() |
String |
toString() |
public DateTimeBucketAssigner()
DateTimeBucketAssigner
with format string "yyyy-MM-dd--HH"
.public DateTimeBucketAssigner(String formatString)
DateTimeBucketAssigner
with the given date/time format string.formatString
- The format string that will be given to SimpleDateFormat
to
determine the bucket id.public DateTimeBucketAssigner(java.time.ZoneId zoneId)
DateTimeBucketAssigner
with format string "yyyy-MM-dd--HH"
using the given timezone.zoneId
- The timezone used to format DateTimeFormatter
for bucket id.public DateTimeBucketAssigner(String formatString, java.time.ZoneId zoneId)
DateTimeBucketAssigner
with the given date/time format string using the
given timezone.formatString
- The format string that will be given to DateTimeFormatter
to
determine the bucket path.zoneId
- The timezone used to format DateTimeFormatter
for bucket id.public String getBucketId(IN element, BucketAssigner.Context context)
BucketAssigner
getBucketId
in interface BucketAssigner<IN,String>
element
- The current element being processed.context
- The context
used by the sink
.base path
provided during the initialization of the sink
.public SimpleVersionedSerializer<String> getSerializer()
getSerializer
in interface BucketAssigner<IN,String>
SimpleVersionedSerializer
capable of serializing/deserializing the elements
of type BucketID
. That is the type of the objects returned by the BucketAssigner.getBucketId(Object, BucketAssigner.Context)
.Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.