org.apache.flink.fs.s3presto

Class FlinkS3PrestoFileSystem

java.lang.Object

org.apache.flink.core.fs.FileSystem

org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

org.apache.flink.fs.s3.common.FlinkS3FileSystem

org.apache.flink.fs.s3presto.FlinkS3PrestoFileSystem

All Implemented Interfaces:

EntropyInjectingFileSystem

public class FlinkS3PrestoFileSystem
extends FlinkS3FileSystem

FlinkS3PrestoFileSystem provides custom recursive deletion functionality to work around a bug in the internally used Presto file system.

https://github.com/prestodb/presto/issues/17416

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

FileSystem.WriteMode

Field Summary

Fields inherited from class org.apache.flink.fs.s3.common.FlinkS3FileSystem

S3_MULTIPART_MIN_PART_SIZE

Constructor Summary

Constructors

Constructor and Description
FlinkS3PrestoFileSystem(org.apache.hadoop.fs.FileSystem hadoopS3FileSystem, String localTmpDirectory, String entropyInjectionKey, int entropyLength, S3AccessHelper s3UploadHelper, long s3uploadPartSize, int maxConcurrentUploadsPerStream)

Method Summary

Modifier and Type	Method and Description
boolean	delete(Path path, boolean recursive) Delete a file.

Methods inherited from class org.apache.flink.fs.s3.common.FlinkS3FileSystem

createRecoverableWriter, generateEntropy, getEntropyInjectionKey, getKind, getLocalTmpDir

Methods inherited from class org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

create, create, exists, getDefaultBlockSize, getFileBlockLocations, getFileStatus, getHadoopFileSystem, getHomeDirectory, getUri, getWorkingDirectory, isDistributedFS, listStatus, mkdirs, open, open, rename, toHadoopPath

Methods inherited from class org.apache.flink.core.fs.FileSystem

create, get, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initialize, initOutPathDistFS, initOutPathLocalFS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkS3PrestoFileSystem

public FlinkS3PrestoFileSystem(org.apache.hadoop.fs.FileSystem hadoopS3FileSystem,
                               String localTmpDirectory,
                               @Nullable
                               String entropyInjectionKey,
                               int entropyLength,
                               @Nullable
                               S3AccessHelper s3UploadHelper,
                               long s3uploadPartSize,
                               int maxConcurrentUploadsPerStream)

Method Detail

delete

public boolean delete(Path path,
                      boolean recursive)
               throws IOException

Description copied from class: FileSystem

Delete a file.

Overrides:

delete in class HadoopFileSystem

Parameters:

path - the path to delete

recursive - if path is a directory and set to true, the directory is deleted else throws an exception. In case of a file the recursive can be set to either true or false

Returns:

true if delete is successful, false otherwise

Throws:

IOException