org.apache.flink.fs.osshadoop

Class FlinkOSSFileSystem

java.lang.Object

org.apache.flink.core.fs.FileSystem

org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

org.apache.flink.fs.osshadoop.FlinkOSSFileSystem

public class FlinkOSSFileSystem
extends HadoopFileSystem

Implementation of the Flink FileSystem interface for Aliyun OSS. This class implements the common behavior implemented directly by Flink and delegates common calls to an implementation of Hadoop's filesystem abstraction.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

FileSystem.WriteMode

Field Summary

Fields

Modifier and Type	Field and Description
static long	MULTIPART_UPLOAD_PART_SIZE_MIN

Constructor Summary

Constructors

Constructor and Description
FlinkOSSFileSystem(org.apache.hadoop.fs.FileSystem fileSystem, long ossUploadPartSize, int maxConcurrentUploadsPerStream, String localTmpDirectory, OSSAccessor ossAccessor)

Method Summary

Modifier and Type	Method and Description
RecoverableWriter	createRecoverableWriter() Creates a new RecoverableWriter.
FileSystemKind	getKind() Gets a description of the characteristics of this file system.
String	getLocalTmpDir()

Methods inherited from class org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

create, create, delete, exists, getDefaultBlockSize, getFileBlockLocations, getFileStatus, getHadoopFileSystem, getHomeDirectory, getUri, getWorkingDirectory, isDistributedFS, listStatus, mkdirs, open, open, rename, toHadoopPath

Methods inherited from class org.apache.flink.core.fs.FileSystem

create, get, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initialize, initOutPathDistFS, initOutPathLocalFS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MULTIPART_UPLOAD_PART_SIZE_MIN

public static final long MULTIPART_UPLOAD_PART_SIZE_MIN

See Also:

Constant Field Values

Constructor Detail

FlinkOSSFileSystem

public FlinkOSSFileSystem(org.apache.hadoop.fs.FileSystem fileSystem,
                          long ossUploadPartSize,
                          int maxConcurrentUploadsPerStream,
                          String localTmpDirectory,
                          OSSAccessor ossAccessor)

Method Detail

getKind

public FileSystemKind getKind()

Description copied from class: FileSystem

Gets a description of the characteristics of this file system.

Overrides:

getKind in class HadoopFileSystem

createRecoverableWriter

public RecoverableWriter createRecoverableWriter()
                                          throws IOException

Description copied from class: FileSystem

Creates a new RecoverableWriter. A recoverable writer creates streams that can persist and recover their intermediate state. Persisting and recovering intermediate state is a core building block for writing to files that span multiple checkpoints.

The returned object can act as a shared factory to open and recover multiple streams.

This method is optional on file systems and various file system implementations may not support this method, throwing an UnsupportedOperationException.

Overrides:

createRecoverableWriter in class HadoopFileSystem

Returns:

A RecoverableWriter for this file system.

Throws:

IOException - Thrown, if the recoverable writer cannot be instantiated.

getLocalTmpDir

public String getLocalTmpDir()