Interface FileEnumerator
-
- All Known Subinterfaces:
DynamicFileEnumerator
- All Known Implementing Classes:
BlockSplittingRecursiveAllDirEnumerator
,BlockSplittingRecursiveEnumerator
,NonSplittingRecursiveAllDirEnumerator
,NonSplittingRecursiveEnumerator
@PublicEvolving public interface FileEnumerator
TheFileEnumerator
's task is to discover all files to be read and to split them into a set ofFileSourceSplit
.This includes possibly, path traversals, file filtering (by name or other patterns) and deciding whether to split files into multiple splits, and how to split them.
-
-
Nested Class Summary
Nested Classes Modifier and Type Interface Description static interface
FileEnumerator.Provider
Factory for theFileEnumerator
, to allow theFileEnumerator
to be eagerly initialized and to not be serializable.
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description Collection<FileSourceSplit>
enumerateSplits(Path[] paths, int minDesiredSplits)
Generates all file splits for the relevant files under the given paths.
-
-
-
Method Detail
-
enumerateSplits
Collection<FileSourceSplit> enumerateSplits(Path[] paths, int minDesiredSplits) throws IOException
Generates all file splits for the relevant files under the given paths. TheminDesiredSplits
is an optional hint indicating how many splits would be necessary to exploit parallelism properly.- Throws:
IOException
-
-