org.apache.flink.formats.csv

Class AbstractCsvInputFormat<T>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<T>

org.apache.flink.formats.csv.AbstractCsvInputFormat<T>

All Implemented Interfaces:

Serializable, InputFormat<T,FileInputSplit>, InputSplitSource<FileInputSplit>

Direct Known Subclasses:

RowCsvInputFormat

public abstract class AbstractCsvInputFormat<T>
extends FileInputFormat<T>

Input format that reads csv. This abstract class is responsible for cutting the boundary of InputSplit.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields

Modifier and Type	Field and Description
protected InputStream	csvInputStream
protected org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvSchema	csvSchema

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
AbstractCsvInputFormat(Path[] filePaths, org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvSchema csvSchema)

Method Summary

Modifier and Type	Method and Description
void	open(FileInputSplit split) Opens an input stream to the file defined in the input format.
boolean	supportsMultiPaths() Override this method to supports multiple paths.

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, close, configure, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.io.InputFormat

nextRecord, reachedEnd

Field Detail

csvSchema

protected final org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvSchema csvSchema

csvInputStream

protected transient InputStream csvInputStream

Constructor Detail

AbstractCsvInputFormat

public AbstractCsvInputFormat(Path[] filePaths,
                              org.apache.flink.shaded.jackson2.com.fasterxml.jackson.dataformat.csv.CsvSchema csvSchema)

Method Detail

supportsMultiPaths

public boolean supportsMultiPaths()

Description copied from class: FileInputFormat

Override this method to supports multiple paths. When this method will be removed, all FileInputFormats have to support multiple paths.

Overrides:

supportsMultiPaths in class FileInputFormat<T>

Returns:

True if the FileInputFormat supports multiple paths, false otherwise.

open

public void open(FileInputSplit split)
          throws IOException

Description copied from class: FileInputFormat

Opens an input stream to the file defined in the input format. The stream is positioned at the beginning of the given split.

The stream is actually opened in an asynchronous thread to make sure any interruptions to the thread working on the input format do not reach the file system.

Specified by:

open in interface InputFormat<T,FileInputSplit>

Overrides:

open in class FileInputFormat<T>

Parameters:

split - The split to be opened.

Throws:

IOException - Thrown, if the spit could not be opened due to an I/O problem.