org.apache.flink.table.utils

Class PartitionPathUtils

java.lang.Object

org.apache.flink.table.utils.PartitionPathUtils

@Internal
public class PartitionPathUtils
extends Object

Utils for file system.

Constructor Summary

Constructors

Constructor and Description
PartitionPathUtils()

Method Summary

Modifier and Type	Method and Description
static Object	convertStringToInternalValue(String valStr, DataType type) Restore partition value from string and type.
static LinkedHashMap<String,String>	extractPartitionSpecFromPath(Path currPath) Make partition spec from path.
static List<String>	extractPartitionValues(Path currPath) Make partition values from path.
static GenericRowData	fillPartitionValueForRecord(String[] fieldNames, DataType[] fieldTypes, int[] selectFields, List<String> partitionKeys, Path path, String defaultPartValue) Extract partition value from path and fill to record.
static String	generatePartitionPath(LinkedHashMap<String,String> partitionSpec) Make partition path from partition spec.
static FileStatus[]	listStatusWithoutHidden(FileSystem fs, Path dir) List file status without hidden files.
static List<Tuple2<LinkedHashMap<String,String>,Path>>	searchPartSpecAndPaths(FileSystem fs, Path path, int partitionNumber) Search all partitions in this path.
static String	unescapePathName(String path)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitionPathUtils

public PartitionPathUtils()

Method Detail

generatePartitionPath

public static String generatePartitionPath(LinkedHashMap<String,String> partitionSpec)

Make partition path from partition spec.

Parameters:

partitionSpec - The partition spec.

Returns:

An escaped, valid partition name.

extractPartitionValues

public static List<String> extractPartitionValues(Path currPath)

Make partition values from path.

Parameters:

currPath - partition file path.

Returns:

Sequential partition specs.

extractPartitionSpecFromPath

public static LinkedHashMap<String,String> extractPartitionSpecFromPath(Path currPath)

Make partition spec from path.

Parameters:

currPath - partition file path.

Returns:

Sequential partition specs.

unescapePathName

public static String unescapePathName(String path)

listStatusWithoutHidden

public static FileStatus[] listStatusWithoutHidden(FileSystem fs,
                                                   Path dir)
                                            throws IOException

List file status without hidden files.

Throws:

IOException

searchPartSpecAndPaths

public static List<Tuple2<LinkedHashMap<String,String>,Path>> searchPartSpecAndPaths(FileSystem fs,
                                                                                     Path path,
                                                                                     int partitionNumber)

Search all partitions in this path.

Parameters:

path - search path.

partitionNumber - partition number, it will affect path structure.

Returns:

all partition specs to its path.

fillPartitionValueForRecord

public static GenericRowData fillPartitionValueForRecord(String[] fieldNames,
                                                         DataType[] fieldTypes,
                                                         int[] selectFields,
                                                         List<String> partitionKeys,
                                                         Path path,
                                                         String defaultPartValue)

Extract partition value from path and fill to record.

Parameters:

fieldNames - record field names.

fieldTypes - record field types.

selectFields - the selected fields.

partitionKeys - the partition field names.

path - the file path that the partition located.

defaultPartValue - default value of partition field.

Returns:

the filled record.

convertStringToInternalValue

public static Object convertStringToInternalValue(String valStr,
                                                  DataType type)

Restore partition value from string and type.

Parameters:

valStr - string partition value.

type - type of partition field.

Returns:

partition value.