org.apache.flink.api.java.io

Class PojoCsvInputFormat<OUT>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<OT>

org.apache.flink.api.common.io.GenericCsvInputFormat<OUT>

org.apache.flink.api.java.io.CsvInputFormat<OUT>

org.apache.flink.api.java.io.PojoCsvInputFormat<OUT>

Type Parameters:

OUT - resulting POJO type

All Implemented Interfaces:

Serializable, CheckpointableInputFormat<FileInputSplit,Long>, InputFormat<OUT,FileInputSplit>, InputSplitSource<FileInputSplit>

@Internal
public class PojoCsvInputFormat<OUT>
extends CsvInputFormat<OUT>

Input format that reads csv into POJOs.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.java.io.CsvInputFormat

DEFAULT_FIELD_DELIMITER, DEFAULT_LINE_DELIMITER, parsedValues

Fields inherited from class org.apache.flink.api.common.io.GenericCsvInputFormat

commentCount, commentPrefix, fieldIncluded, invalidLineCount, lineDelimiterIsLinebreak

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

currBuffer, currLen, currOffset, RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
PojoCsvInputFormat(Path filePath, PojoTypeInfo<OUT> pojoTypeInfo)
PojoCsvInputFormat(Path filePath, PojoTypeInfo<OUT> pojoTypeInfo, boolean[] includedFieldsMask)
PojoCsvInputFormat(Path filePath, PojoTypeInfo<OUT> pojoTypeInfo, int[] includedFieldsMask)
PojoCsvInputFormat(Path filePath, PojoTypeInfo<OUT> pojoTypeInfo, String[] fieldNames)
PojoCsvInputFormat(Path filePath, PojoTypeInfo<OUT> pojoTypeInfo, String[] fieldNames, boolean[] includedFieldsMask)
PojoCsvInputFormat(Path filePath, PojoTypeInfo<OUT> pojoTypeInfo, String[] fieldNames, int[] includedFieldsMask)
PojoCsvInputFormat(Path filePath, String lineDelimiter, String fieldDelimiter, PojoTypeInfo<OUT> pojoTypeInfo)
PojoCsvInputFormat(Path filePath, String lineDelimiter, String fieldDelimiter, PojoTypeInfo<OUT> pojoTypeInfo, boolean[] includedFieldsMask)
PojoCsvInputFormat(Path filePath, String lineDelimiter, String fieldDelimiter, PojoTypeInfo<OUT> pojoTypeInfo, int[] includedFieldsMask)
PojoCsvInputFormat(Path filePath, String lineDelimiter, String fieldDelimiter, PojoTypeInfo<OUT> pojoTypeInfo, String[] fieldNames)
PojoCsvInputFormat(Path filePath, String lineDelimiter, String fieldDelimiter, PojoTypeInfo<OUT> pojoTypeInfo, String[] fieldNames, boolean[] includedFieldsMask)
PojoCsvInputFormat(Path filePath, String lineDelimiter, String fieldDelimiter, PojoTypeInfo<OUT> pojoTypeInfo, String[] fieldNames, int[] includedFieldsMask)

Method Summary

Modifier and Type	Method and Description
OUT	fillRecord(OUT reuse, Object[] parsedValues)
void	initializeSplit(FileInputSplit split, Long offset) Initialization method that is called after opening or reopening an input split.

Methods inherited from class org.apache.flink.api.java.io.CsvInputFormat

createDefaultMask, getFieldTypes, nextRecord, readRecord, toBooleanMask, toString

Methods inherited from class org.apache.flink.api.common.io.GenericCsvInputFormat

checkAndCoSort, checkForMonotonousOrder, close, enableQuotedStringParsing, getCommentPrefix, getFieldDelimiter, getFieldParsers, getGenericFieldTypes, getNumberOfFieldsTotal, getNumberOfNonNullFields, isLenient, isSkippingFirstLineAsHeader, parseRecord, setCharset, setCommentPrefix, setFieldDelimiter, setFieldsGeneric, setFieldsGeneric, setFieldTypesGeneric, setLenient, setSkipFirstLineAsHeader, skipFields, supportsMultiPaths

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

configure, getBufferSize, getCharset, getCurrentState, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadConfigParameters, loadGlobalConfigParams, open, reachedEnd, readLine, reopen, setBufferSize, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getSupportedCompressionFormats, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          PojoTypeInfo<OUT> pojoTypeInfo)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          String[] fieldNames)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          String lineDelimiter,
                          String fieldDelimiter,
                          PojoTypeInfo<OUT> pojoTypeInfo)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          String lineDelimiter,
                          String fieldDelimiter,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          String[] fieldNames)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          int[] includedFieldsMask)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          String[] fieldNames,
                          int[] includedFieldsMask)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          String lineDelimiter,
                          String fieldDelimiter,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          int[] includedFieldsMask)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          String lineDelimiter,
                          String fieldDelimiter,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          String[] fieldNames,
                          int[] includedFieldsMask)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          boolean[] includedFieldsMask)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          String[] fieldNames,
                          boolean[] includedFieldsMask)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          String lineDelimiter,
                          String fieldDelimiter,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          boolean[] includedFieldsMask)

PojoCsvInputFormat

public PojoCsvInputFormat(Path filePath,
                          String lineDelimiter,
                          String fieldDelimiter,
                          PojoTypeInfo<OUT> pojoTypeInfo,
                          String[] fieldNames,
                          boolean[] includedFieldsMask)

Method Detail

initializeSplit

public void initializeSplit(FileInputSplit split,
                            Long offset)
                     throws IOException

Description copied from class: DelimitedInputFormat

Initialization method that is called after opening or reopening an input split.

Overrides:

initializeSplit in class CsvInputFormat<OUT>

Parameters:

split - Split that was opened or reopened

offset - Checkpointed state if the split was reopened

Throws:

IOException

fillRecord

public OUT fillRecord(OUT reuse,
                      Object[] parsedValues)

Specified by:

fillRecord in class CsvInputFormat<OUT>