public static class CsvFileSystemFormatFactory.CsvInputFormat extends AbstractCsvInputFormat<RowData>
RowData
.FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread
csvInputStream, csvSchema
currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable
Constructor and Description |
---|
CsvInputFormat(Path[] filePaths,
DataType[] fieldTypes,
String[] fieldNames,
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvSchema csvSchema,
RowType formatRowType,
int[] selectFields,
List<String> partitionKeys,
String defaultPartValue,
long limit,
int[] csvSelectFieldToProjectFieldMapping,
int[] csvSelectFieldToCsvFieldMapping,
boolean ignoreParseErrors) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Closes the file input stream of the input format.
|
RowData |
nextRecord(RowData reuse)
Reads the next record from the input.
|
void |
open(FileInputSplit split)
Opens an input stream to the file defined in the input format.
|
boolean |
reachedEnd()
Method used to check if the end of the input is reached.
|
supportsMultiPaths
acceptFile, configure, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable, toString
closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext
public CsvInputFormat(Path[] filePaths, DataType[] fieldTypes, String[] fieldNames, org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvSchema csvSchema, RowType formatRowType, int[] selectFields, List<String> partitionKeys, String defaultPartValue, long limit, int[] csvSelectFieldToProjectFieldMapping, int[] csvSelectFieldToCsvFieldMapping, boolean ignoreParseErrors)
public void open(FileInputSplit split) throws IOException
FileInputFormat
The stream is actually opened in an asynchronous thread to make sure any interruptions to the thread working on the input format do not reach the file system.
open
in interface InputFormat<RowData,FileInputSplit>
open
in class AbstractCsvInputFormat<RowData>
split
- The split to be opened.IOException
- Thrown, if the spit could not be opened due to an I/O problem.public boolean reachedEnd() throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
IOException
- Thrown, if an I/O error occurred.public RowData nextRecord(RowData reuse) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
reuse
- Object that may be reused.IOException
- Thrown, if an I/O error occurred.public void close() throws IOException
FileInputFormat
close
in interface InputFormat<RowData,FileInputSplit>
close
in class FileInputFormat<RowData>
IOException
- Thrown, if the input could not be closed properly.Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.