Package org.apache.flink.fs.osshadoop

Class FlinkOSSFileSystem

java.lang.Object

org.apache.flink.core.fs.FileSystem

org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

org.apache.flink.fs.osshadoop.FlinkOSSFileSystem

All Implemented Interfaces:

IFileSystem

public class FlinkOSSFileSystem
extends HadoopFileSystem

Implementation of the Flink FileSystem interface for Aliyun OSS. This class implements the common behavior implemented directly by Flink and delegates common calls to an implementation of Hadoop's filesystem abstraction.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

FileSystem.FSKey, FileSystem.WriteMode

Field Summary

Fields

Modifier and Type	Field	Description
static long	MULTIPART_UPLOAD_PART_SIZE_MIN

Constructor Summary

Constructors

Constructor	Description
FlinkOSSFileSystem(org.apache.hadoop.fs.FileSystem fileSystem, long ossUploadPartSize, int maxConcurrentUploadsPerStream, String localTmpDirectory, OSSAccessor ossAccessor)

Method Summary

Modifier and Type	Method	Description
RecoverableWriter	createRecoverableWriter()	Creates a new RecoverableWriter.
String	getLocalTmpDir()

Methods inherited from class org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

create, create, createRecoverableWriter, delete, exists, getDefaultBlockSize, getFileBlockLocations, getFileStatus, getHadoopFileSystem, getHomeDirectory, getUri, getWorkingDirectory, isDistributedFS, listStatus, mkdirs, open, open, rename, toHadoopPath

Methods inherited from class org.apache.flink.core.fs.FileSystem

create, get, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initialize, initOutPathDistFS, initOutPathLocalFS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.core.fs.IFileSystem

canCopyPaths

Field Detail

MULTIPART_UPLOAD_PART_SIZE_MIN

public static final long MULTIPART_UPLOAD_PART_SIZE_MIN

See Also:

Constant Field Values

Constructor Detail

FlinkOSSFileSystem

public FlinkOSSFileSystem(org.apache.hadoop.fs.FileSystem fileSystem,
                          long ossUploadPartSize,
                          int maxConcurrentUploadsPerStream,
                          String localTmpDirectory,
                          OSSAccessor ossAccessor)

Method Detail

createRecoverableWriter

public RecoverableWriter createRecoverableWriter()
                                          throws IOException

Description copied from interface: IFileSystem

Creates a new RecoverableWriter. A recoverable writer creates streams that can persist and recover their intermediate state. Persisting and recovering intermediate state is a core building block for writing to files that span multiple checkpoints.

The returned object can act as a shared factory to open and recover multiple streams.

This method is optional on file systems and various file system implementations may not support this method, throwing an UnsupportedOperationException.

Specified by:

createRecoverableWriter in interface IFileSystem

Overrides:

createRecoverableWriter in class HadoopFileSystem

Returns:

A RecoverableWriter for this file system.

Throws:

IOException - Thrown, if the recoverable writer cannot be instantiated.

getLocalTmpDir

public String getLocalTmpDir()