org.apache.flink.table.filesystem

Class PartitionTempFileManager

java.lang.Object

org.apache.flink.table.filesystem.PartitionTempFileManager

@Internal
public class PartitionTempFileManager
extends Object

Manage temporary files for writing files. Use special rules to organize directories for temporary files.

Temporary file directory contains the following directory parts: 1.temporary base path directory. 2.checkpoint id directory. 3.task id directory. 4.directories to specify partitioning. 5.data files. eg: /tmp/cp-1/task-0/p0=1/p1=2/fileName.

Method Summary

Modifier and Type	Method and Description
Path	createPartitionDir(String... partitions) Generate a new partition directory with partitions.
static void	deleteCheckpoint(FileSystem fs, Path basePath, long checkpointId) Delete checkpoint path.
static long[]	headCheckpoints(FileSystem fs, Path basePath, long toCpId) Returns checkpoints whose keys are less than or equal to toCpId in temporary base path.
static List<Path>	listTaskTemporaryPaths(FileSystem fs, Path basePath, long checkpointId) Returns task temporary paths in this checkpoint.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createPartitionDir

public Path createPartitionDir(String... partitions)

Generate a new partition directory with partitions.

deleteCheckpoint

public static void deleteCheckpoint(FileSystem fs,
                                    Path basePath,
                                    long checkpointId)
                             throws IOException

Delete checkpoint path.

Throws:

IOException

headCheckpoints

public static long[] headCheckpoints(FileSystem fs,
                                     Path basePath,
                                     long toCpId)
                              throws IOException

Returns checkpoints whose keys are less than or equal to toCpId in temporary base path.

Throws:

IOException

listTaskTemporaryPaths

public static List<Path> listTaskTemporaryPaths(FileSystem fs,
                                                Path basePath,
                                                long checkpointId)
                                         throws Exception

Returns task temporary paths in this checkpoint.

Throws:

Exception