org.apache.flink.api.common.io

Class FilePathFilter

java.lang.Object

org.apache.flink.api.common.io.FilePathFilter

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

FilePathFilter.DefaultFilter, GlobFilePathFilter

@PublicEvolving
public abstract class FilePathFilter
extends Object
implements Serializable

The filterPath(Path) method is responsible for deciding if a path is eligible for further processing or not. This can serve to exclude temporary or partial files that are still being written.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FilePathFilter.DefaultFilter The default file path filtering method and is used if no other such function is provided.

Field Summary

Fields

Modifier and Type	Field and Description
static String	HADOOP_COPYING Name of an unfinished Hadoop file

Constructor Summary

Constructors

Constructor and Description
FilePathFilter()

Method Summary

Modifier and Type	Method and Description
static FilePathFilter	createDefaultFilter() Returns the default filter, which excludes the following files: Files starting with "_" Files starting with "." Files containing the string "_COPYING_"
abstract boolean	filterPath(Path filePath) Returns true if the filePath given is to be ignored when processing a directory, e.g.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HADOOP_COPYING

public static final String HADOOP_COPYING

Name of an unfinished Hadoop file

See Also:

Constant Field Values

Constructor Detail

FilePathFilter

public FilePathFilter()

Method Detail

filterPath

public abstract boolean filterPath(Path filePath)

Returns true if the filePath given is to be ignored when processing a directory, e.g.

 public boolean filterPaths(Path filePath) {
     return filePath.getName().startsWith(".") || filePath.getName().contains("_COPYING_");
 }
 

createDefaultFilter

public static FilePathFilter createDefaultFilter()

Returns the default filter, which excludes the following files:

• Files starting with "_"
• Files starting with "."
• Files containing the string "_COPYING_"

Returns:

The singleton instance of the default file path filter.