org.apache.flink.formats.hadoop.bulk

Class HadoopPathBasedPartFileWriter<IN,BucketID>

java.lang.Object

org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter<IN,BucketID>

org.apache.flink.formats.hadoop.bulk.HadoopPathBasedPartFileWriter<IN,BucketID>

All Implemented Interfaces:

CompactingFileWriter, InProgressFileWriter<IN,BucketID>, PartFileInfo<BucketID>, RecordWiseCompactingFileWriter<IN>

public class HadoopPathBasedPartFileWriter<IN,BucketID>
extends AbstractPartFileWriter<IN,BucketID>

The part-file writer that writes to the specified hadoop path.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HadoopPathBasedPartFileWriter.HadoopPathBasedBucketWriter<IN,BucketID> Factory to create HadoopPathBasedPartFileWriter.

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter

InProgressFileWriter.InProgressFileRecoverable, InProgressFileWriter.PendingFileRecoverable

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.CompactingFileWriter

CompactingFileWriter.Type

Constructor Summary

Constructors

Constructor and Description
HadoopPathBasedPartFileWriter(BucketID bucketID, HadoopPathBasedBulkWriter<IN> writer, HadoopFileCommitter fileCommitter, long createTime)

Method Summary

Modifier and Type	Method and Description
InProgressFileWriter.PendingFileRecoverable	closeForCommit() Closes the writer and gets the InProgressFileWriter.PendingFileRecoverable of the written compacting file.
void	dispose() Dispose the part file.
long	getSize()
InProgressFileWriter.InProgressFileRecoverable	persist()
void	write(IN element, long currentTime) Write an element to the part file.

Methods inherited from class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter

getBucketId, getCreationTime, getLastUpdateTime, markWrite

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter

write

Constructor Detail

HadoopPathBasedPartFileWriter

public HadoopPathBasedPartFileWriter(BucketID bucketID,
                                     HadoopPathBasedBulkWriter<IN> writer,
                                     HadoopFileCommitter fileCommitter,
                                     long createTime)

Method Detail

write

public void write(IN element,
                  long currentTime)
           throws IOException

Description copied from interface: InProgressFileWriter

Write an element to the part file.

Parameters:

element - the element to be written.

currentTime - the writing time.

Throws:

IOException - Thrown if writing the element fails.

persist

public InProgressFileWriter.InProgressFileRecoverable persist()

Returns:

The state of the current part file.

closeForCommit

public InProgressFileWriter.PendingFileRecoverable closeForCommit()
                                                           throws IOException

Description copied from interface: CompactingFileWriter

Closes the writer and gets the InProgressFileWriter.PendingFileRecoverable of the written compacting file.

Returns:

The state of the pending part file. Bucket uses this to commit the pending file.

Throws:

IOException - Thrown if an I/O error occurs.

dispose

public void dispose()

Description copied from interface: InProgressFileWriter

Dispose the part file.

getSize

public long getSize()
             throws IOException

Returns:

The size of the currently open part file.

Throws:

IOException