Package org.apache.flink.api.common.io
Class FilePathFilter
- java.lang.Object
-
- org.apache.flink.api.common.io.FilePathFilter
-
- All Implemented Interfaces:
Serializable
- Direct Known Subclasses:
FilePathFilter.DefaultFilter
,GlobFilePathFilter
@PublicEvolving public abstract class FilePathFilter extends Object implements Serializable
ThefilterPath(Path)
method is responsible for deciding if a path is eligible for further processing or not. This can serve to exclude temporary or partial files that are still being written.- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
FilePathFilter.DefaultFilter
The default file path filtering method and is used if no other such function is provided.
-
Field Summary
Fields Modifier and Type Field Description static String
HADOOP_COPYING
Name of an unfinished Hadoop file
-
Constructor Summary
Constructors Constructor Description FilePathFilter()
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description static FilePathFilter
createDefaultFilter()
Returns the default filter, which excludes the following files: Files starting with "_" Files starting with "." Files containing the string "_COPYING_"abstract boolean
filterPath(Path filePath)
Returnstrue
if thefilePath
given is to be ignored when processing a directory, e.g.
-
-
-
Field Detail
-
HADOOP_COPYING
public static final String HADOOP_COPYING
Name of an unfinished Hadoop file- See Also:
- Constant Field Values
-
-
Method Detail
-
filterPath
public abstract boolean filterPath(Path filePath)
Returnstrue
if thefilePath
given is to be ignored when processing a directory, e.g.public boolean filterPaths(Path filePath) { return filePath.getName().startsWith(".") || filePath.getName().contains("_COPYING_"); }
-
createDefaultFilter
public static FilePathFilter createDefaultFilter()
Returns the default filter, which excludes the following files:- Files starting with "_"
- Files starting with "."
- Files containing the string "_COPYING_"
- Returns:
- The singleton instance of the default file path filter.
-
-