org.apache.flink.api.java.io

Class PrimitiveInputFormat<OT>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<OT>

org.apache.flink.api.java.io.PrimitiveInputFormat<OT>

All Implemented Interfaces:

Serializable, CheckpointableInputFormat<FileInputSplit,Long>, InputFormat<OT,FileInputSplit>, InputSplitSource<FileInputSplit>

@PublicEvolving
public class PrimitiveInputFormat<OT>
extends DelimitedInputFormat<OT>

An input format that reads single field primitive data from a given file. The difference between this and CsvInputFormat is that it won't go through Tuple1.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

currBuffer, currLen, currOffset, RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
PrimitiveInputFormat(Path filePath, Class<OT> primitiveClass)
PrimitiveInputFormat(Path filePath, String delimiter, Class<OT> primitiveClass)

Method Summary

Modifier and Type	Method and Description
protected void	initializeSplit(FileInputSplit split, Long offset) Initialization method that is called after opening or reopening an input split.
OT	readRecord(OT reuse, byte[] bytes, int offset, int numBytes) This function parses the given byte array which represents a serialized record.

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

close, configure, getBufferSize, getCharset, getCurrentState, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadConfigParameters, loadGlobalConfigParams, nextRecord, open, reachedEnd, readLine, reopen, setBufferSize, setCharset, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, supportsMultiPaths, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PrimitiveInputFormat

public PrimitiveInputFormat(Path filePath,
                            Class<OT> primitiveClass)

PrimitiveInputFormat

public PrimitiveInputFormat(Path filePath,
                            String delimiter,
                            Class<OT> primitiveClass)

Method Detail

initializeSplit

protected void initializeSplit(FileInputSplit split,
                               Long offset)
                        throws IOException

Description copied from class: DelimitedInputFormat

Initialization method that is called after opening or reopening an input split.

Overrides:

initializeSplit in class DelimitedInputFormat<OT>

Parameters:

split - Split that was opened or reopened

offset - Checkpointed state if the split was reopened

Throws:

IOException

readRecord

public OT readRecord(OT reuse,
                     byte[] bytes,
                     int offset,
                     int numBytes)
              throws IOException

Description copied from class: DelimitedInputFormat

This function parses the given byte array which represents a serialized record. The function returns a valid record or throws an IOException.

Specified by:

readRecord in class DelimitedInputFormat<OT>

Parameters:

reuse - An optionally reusable object.

bytes - Binary data of serialized records.

offset - The offset where to start to read the record data.

numBytes - The number of bytes that can be read starting at the offset position.

Returns:

Returns the read record if it was successfully deserialized.

Throws:

IOException - if the record could not be read.