org.apache.flink.core.fs

Class CommitterFromPersistRecoverableFsDataOutputStream<RESUME_RECOVERABLE extends RecoverableWriter.ResumeRecoverable>

java.lang.Object

java.io.OutputStream

org.apache.flink.core.fs.FSDataOutputStream

org.apache.flink.core.fs.RecoverableFsDataOutputStream

org.apache.flink.core.fs.CommitterFromPersistRecoverableFsDataOutputStream<RESUME_RECOVERABLE>

Type Parameters:

RESUME_RECOVERABLE - return type of #persist()

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

Direct Known Subclasses:

BaseHadoopFsRecoverableFsDataOutputStream, LocalRecoverableFsDataOutputStream

public abstract class CommitterFromPersistRecoverableFsDataOutputStream<RESUME_RECOVERABLE extends RecoverableWriter.ResumeRecoverable>
extends RecoverableFsDataOutputStream

RecoverableFsDataOutputStream with fixed implementation of closeForCommit() that is based on using persist() to ensure durability and creates the RecoverableFsDataOutputStream.Committer from the corresponding RecoverableWriter.ResumeRecoverable.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.core.fs.RecoverableFsDataOutputStream

RecoverableFsDataOutputStream.Committer

Constructor Summary

Constructors

Constructor and Description
CommitterFromPersistRecoverableFsDataOutputStream()

Method Summary

Modifier and Type	Method and Description
RecoverableFsDataOutputStream.Committer	closeForCommit() Closes the stream, ensuring persistence of all data (similar to FSDataOutputStream.sync()).
protected abstract RecoverableFsDataOutputStream.Committer	createCommitterFromResumeRecoverable(RESUME_RECOVERABLE recoverable)
abstract RESUME_RECOVERABLE	persist() Ensures all data so far is persistent (similar to FSDataOutputStream.sync()) and returns a handle to recover the stream at the current position.

Methods inherited from class org.apache.flink.core.fs.RecoverableFsDataOutputStream

close

Methods inherited from class org.apache.flink.core.fs.FSDataOutputStream

flush, getPos, sync

Methods inherited from class java.io.OutputStream

write, write, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CommitterFromPersistRecoverableFsDataOutputStream

public CommitterFromPersistRecoverableFsDataOutputStream()

Method Detail

persist

public abstract RESUME_RECOVERABLE persist()
                                    throws IOException

Description copied from class: RecoverableFsDataOutputStream

Ensures all data so far is persistent (similar to FSDataOutputStream.sync()) and returns a handle to recover the stream at the current position.

Specified by:

persist in class RecoverableFsDataOutputStream

Throws:

IOException

See Also:

RecoverableFsDataOutputStream.persist()

createCommitterFromResumeRecoverable

protected abstract RecoverableFsDataOutputStream.Committer createCommitterFromResumeRecoverable(RESUME_RECOVERABLE recoverable)

Parameters:

recoverable - a resume recoverable to create the committer from. Typically the parameter is the return value of persist().

Returns:

the committer created from recoverable.

See Also:

RecoverableFsDataOutputStream.closeForCommit()

closeForCommit

public final RecoverableFsDataOutputStream.Committer closeForCommit()
                                                             throws IOException

Description copied from class: RecoverableFsDataOutputStream

Closes the stream, ensuring persistence of all data (similar to FSDataOutputStream.sync()). This returns a Committer that can be used to publish (make visible) the file that the stream was writing to.

Specified by:

closeForCommit in class RecoverableFsDataOutputStream

Throws:

IOException

See Also:

RecoverableFsDataOutputStream.closeForCommit()