org.apache.flink.formats.hadoop.bulk

Class HadoopPathBasedPartFileWriter<IN,BucketID>

java.lang.Object

org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter<IN,BucketID>

org.apache.flink.formats.hadoop.bulk.HadoopPathBasedPartFileWriter<IN,BucketID>

All Implemented Interfaces:

InProgressFileWriter<IN,BucketID>, PartFileInfo<BucketID>

public class HadoopPathBasedPartFileWriter<IN,BucketID>
extends AbstractPartFileWriter<IN,BucketID>

The part-file writer that writes to the specified hadoop path.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HadoopPathBasedPartFileWriter.HadoopPathBasedBucketWriter<IN,BucketID> Factory to create HadoopPathBasedPartFileWriter.

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter

InProgressFileWriter.InProgressFileRecoverable, InProgressFileWriter.PendingFileRecoverable

Constructor Summary

Constructors

Constructor and Description
HadoopPathBasedPartFileWriter(BucketID bucketID, HadoopPathBasedBulkWriter<IN> writer, HadoopFileCommitter fileCommitter, long createTime)

Method Summary

Modifier and Type	Method and Description
InProgressFileWriter.PendingFileRecoverable	closeForCommit()
void	dispose() Dispose the part file.
long	getSize()
InProgressFileWriter.InProgressFileRecoverable	persist()
void	write(IN element, long currentTime) Write a element to the part file.

Methods inherited from class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter

getBucketId, getCreationTime, getLastUpdateTime, markWrite

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopPathBasedPartFileWriter

public HadoopPathBasedPartFileWriter(BucketID bucketID,
                                     HadoopPathBasedBulkWriter<IN> writer,
                                     HadoopFileCommitter fileCommitter,
                                     long createTime)

Method Detail

write

public void write(IN element,
                  long currentTime)
           throws IOException

Description copied from interface: InProgressFileWriter

Write a element to the part file.

Parameters:

element - the element to be written.

currentTime - the writing time.

Throws:

IOException - Thrown if writing the element fails.

persist

public InProgressFileWriter.InProgressFileRecoverable persist()

Returns:

The state of the current part file.

closeForCommit

public InProgressFileWriter.PendingFileRecoverable closeForCommit()
                                                           throws IOException

Returns:

The state of the pending part file. Bucket uses this to commit the pending file.

Throws:

IOException - Thrown if an I/O error occurs.

dispose

public void dispose()

Description copied from interface: InProgressFileWriter

Dispose the part file.

getSize

public long getSize()
             throws IOException

Returns:

The size of the currently open part file.

Throws:

IOException