org.apache.flink.fs.azurefs

Class AzureBlobRecoverableWriter

java.lang.Object

org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriter

org.apache.flink.fs.azurefs.AzureBlobRecoverableWriter

All Implemented Interfaces:

RecoverableWriter

public class AzureBlobRecoverableWriter
extends HadoopRecoverableWriter

Recoverable writer for AzureBlob file system.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.core.fs.RecoverableWriter

RecoverableWriter.CommitRecoverable, RecoverableWriter.ResumeRecoverable

Field Summary

Fields inherited from class org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriter

fs

Constructor Summary

Constructors

Constructor and Description
AzureBlobRecoverableWriter(org.apache.hadoop.fs.FileSystem fs) Creates a new Recoverable writer.

Method Summary

Modifier and Type	Method and Description
protected void	checkSupportedFSSchemes(org.apache.hadoop.fs.FileSystem fs)
protected RecoverableFsDataOutputStream	getRecoverableFsDataOutputStream(org.apache.hadoop.fs.Path targetFile, org.apache.hadoop.fs.Path tempFile)
RecoverableFsDataOutputStream	recover(RecoverableWriter.ResumeRecoverable recoverable) Resumes a recoverable stream consistently at the point indicated by the given ResumeRecoverable.
RecoverableFsDataOutputStream.Committer	recoverForCommit(RecoverableWriter.CommitRecoverable recoverable) Recovers a recoverable stream consistently at the point indicated by the given CommitRecoverable for finalizing and committing.

Methods inherited from class org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriter

cleanupRecoverableState, getCommitRecoverableSerializer, getResumeRecoverableSerializer, open, requiresCleanupOfRecoverableState, supportsResume

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AzureBlobRecoverableWriter

public AzureBlobRecoverableWriter(org.apache.hadoop.fs.FileSystem fs)

Creates a new Recoverable writer.

Parameters:

fs - The AzureBlob file system on which the writer operates.

Method Detail

checkSupportedFSSchemes

protected void checkSupportedFSSchemes(org.apache.hadoop.fs.FileSystem fs)

Overrides:

checkSupportedFSSchemes in class HadoopRecoverableWriter

getRecoverableFsDataOutputStream

protected RecoverableFsDataOutputStream getRecoverableFsDataOutputStream(org.apache.hadoop.fs.Path targetFile,
                                                                         org.apache.hadoop.fs.Path tempFile)
                                                                  throws IOException

Overrides:

getRecoverableFsDataOutputStream in class HadoopRecoverableWriter

Throws:

IOException

recover

public RecoverableFsDataOutputStream recover(RecoverableWriter.ResumeRecoverable recoverable)
                                      throws IOException

Description copied from interface: RecoverableWriter

Resumes a recoverable stream consistently at the point indicated by the given ResumeRecoverable. Future writes to the stream will continue / append the file as of that point.

This method is optional and whether it is supported is indicated through the RecoverableWriter.supportsResume() method.

Specified by:

recover in interface RecoverableWriter

Overrides:

recover in class HadoopRecoverableWriter

Parameters:

recoverable - The opaque handle with the recovery information.

Returns:

A recoverable stream writing to the file/object as it was at the point when the ResumeRecoverable was created.

Throws:

IOException - Thrown, if resuming fails.

recoverForCommit

public RecoverableFsDataOutputStream.Committer recoverForCommit(RecoverableWriter.CommitRecoverable recoverable)
                                                         throws IOException

Description copied from interface: RecoverableWriter

Recovers a recoverable stream consistently at the point indicated by the given CommitRecoverable for finalizing and committing. This will publish the target file with exactly the data that was written up to the point then the CommitRecoverable was created.

Specified by:

recoverForCommit in interface RecoverableWriter

Overrides:

recoverForCommit in class HadoopRecoverableWriter

Parameters:

recoverable - The opaque handle with the recovery information.

Returns:

A committer that publishes the target file.

Throws:

IOException - Thrown, if recovery fails.