org.apache.flink.table.filesystem

Class PartitionLoader

java.lang.Object

org.apache.flink.table.filesystem.PartitionLoader

All Implemented Interfaces:

Closeable, AutoCloseable

@Internal
public class PartitionLoader
extends Object
implements Closeable

Loader to temporary files to final output path and meta store. According to overwrite, the loader will delete the previous data.

This provide two interface to load: 1.loadPartition(java.util.LinkedHashMap<java.lang.String, java.lang.String>, java.util.List<org.apache.flink.core.fs.Path>): load temporary partitioned files, if it is new partition, will create partition to meta store. 2.loadNonPartition(java.util.List<org.apache.flink.core.fs.Path>): just rename all files to final output path.

TODO: src and dest may be on different FS.

Constructor Summary

Constructors

Constructor and Description
PartitionLoader(boolean overwrite, FileSystem fs, TableMetaStoreFactory factory)

Method Summary

Modifier and Type	Method and Description
void	close()
void	loadNonPartition(List<Path> srcDirs) Load a non-partition files to output path.
void	loadPartition(LinkedHashMap<String,String> partSpec, List<Path> srcDirs) Load a single partition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitionLoader

public PartitionLoader(boolean overwrite,
                       FileSystem fs,
                       TableMetaStoreFactory factory)
                throws Exception

Throws:

Exception

Method Detail

loadPartition

public void loadPartition(LinkedHashMap<String,String> partSpec,
                          List<Path> srcDirs)
                   throws Exception

Load a single partition.

Throws:

Exception

loadNonPartition

public void loadNonPartition(List<Path> srcDirs)
                      throws Exception

Load a non-partition files to output path.

Throws:

Exception

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException