Modifier and Type | Class and Description |
---|---|
static class |
StreamingFileSink.BucketsBuilder<IN,BucketID,T extends StreamingFileSink.BucketsBuilder<IN,BucketID,T>>
The base abstract class for the
StreamingFileSink.RowFormatBuilder and StreamingFileSink.BulkFormatBuilder . |
Modifier and Type | Class and Description |
---|---|
class |
HadoopPathBasedBulkFormatBuilder<IN,BucketID,T extends HadoopPathBasedBulkFormatBuilder<IN,BucketID,T>>
Buckets builder to create buckets that use
HadoopPathBasedPartFileWriter . |
static class |
StreamingFileSink.BulkFormatBuilder<IN,BucketID,T extends StreamingFileSink.BulkFormatBuilder<IN,BucketID,T>>
A builder for configuring the sink for bulk-encoding formats, e.g.
|
static class |
StreamingFileSink.DefaultBulkFormatBuilder<IN>
Builder for the vanilla
StreamingFileSink using a bulk format. |
static class |
StreamingFileSink.DefaultRowFormatBuilder<IN>
Builder for the vanilla
StreamingFileSink using a row format. |
static class |
StreamingFileSink.RowFormatBuilder<IN,BucketID,T extends StreamingFileSink.RowFormatBuilder<IN,BucketID,T>>
A builder for configuring the sink for row-wise encoding formats.
|
Constructor and Description |
---|
StreamingFileSink(StreamingFileSink.BucketsBuilder<IN,?,? extends StreamingFileSink.BucketsBuilder<IN,?,?>> bucketsBuilder,
long bucketCheckInterval)
Creates a new
StreamingFileSink that writes files to the given base directory with
the give buckets properties. |
Constructor and Description |
---|
StreamingFileSink(StreamingFileSink.BucketsBuilder<IN,?,? extends StreamingFileSink.BucketsBuilder<IN,?,?>> bucketsBuilder,
long bucketCheckInterval)
Creates a new
StreamingFileSink that writes files to the given base directory with
the give buckets properties. |
Modifier and Type | Method and Description |
---|---|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.compactionWriter(DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder,
FileSystemFactory fsFactory,
Path path,
CompactReader.Factory<T> readFactory,
long targetFileSize)
Create a file writer with compaction operators by input stream.
|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.writer(DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder)
Create a file writer by input stream.
|
Modifier and Type | Method and Description |
---|---|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.compactionWriter(DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder,
FileSystemFactory fsFactory,
Path path,
CompactReader.Factory<T> readFactory,
long targetFileSize)
Create a file writer with compaction operators by input stream.
|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.writer(DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder)
Create a file writer by input stream.
|
Constructor and Description |
---|
AbstractStreamingWriter(long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<IN,String,? extends StreamingFileSink.BucketsBuilder<IN,String,?>> bucketsBuilder) |
StreamingFileWriter(long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<IN,String,? extends StreamingFileSink.BucketsBuilder<IN,String,?>> bucketsBuilder) |
Constructor and Description |
---|
AbstractStreamingWriter(long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<IN,String,? extends StreamingFileSink.BucketsBuilder<IN,String,?>> bucketsBuilder) |
StreamingFileWriter(long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<IN,String,? extends StreamingFileSink.BucketsBuilder<IN,String,?>> bucketsBuilder) |
Constructor and Description |
---|
CompactFileWriter(long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder) |
Constructor and Description |
---|
CompactFileWriter(long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder) |
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.