org.apache.flink.api.java.io

Class PrimitiveInputFormat<OT>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<OT>

org.apache.flink.api.java.io.PrimitiveInputFormat<OT>

All Implemented Interfaces:

Serializable, CheckpointableInputFormat<FileInputSplit,Long>, InputFormat<OT,FileInputSplit>, InputSplitSource<FileInputSplit>

@PublicEvolving
public class PrimitiveInputFormat<OT>
extends DelimitedInputFormat<OT>

An input format that reads single field primitive data from a given file. The difference between this and CsvInputFormat is that it won't go through Tuple1.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
PrimitiveInputFormat(Path filePath, Class<OT> primitiveClass)
PrimitiveInputFormat(Path filePath, String delimiter, Class<OT> primitiveClass)

Method Summary

Modifier and Type	Method and Description
void	open(FileInputSplit split) Opens the given input split.
OT	readRecord(OT reuse, byte[] bytes, int offset, int numBytes) This function parses the given byte array which represents a serialized record.

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

close, configure, getBufferSize, getCharset, getCurrentState, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadConfigParameters, loadGlobalConfigParams, nextRecord, reachedEnd, readLine, reopen, setBufferSize, setCharset, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, supportsMultiPaths, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PrimitiveInputFormat

public PrimitiveInputFormat(Path filePath,
                            Class<OT> primitiveClass)

PrimitiveInputFormat

public PrimitiveInputFormat(Path filePath,
                            String delimiter,
                            Class<OT> primitiveClass)

Method Detail

open

public void open(FileInputSplit split)
          throws IOException

Description copied from class: DelimitedInputFormat

Opens the given input split. This method opens the input stream to the specified file, allocates read buffers and positions the stream at the correct position, making sure that any partial record at the beginning is skipped.

Specified by:

open in interface InputFormat<OT,FileInputSplit>

Overrides:

open in class DelimitedInputFormat<OT>

Parameters:

split - The input split to open.

Throws:

IOException - Thrown, if the spit could not be opened due to an I/O problem.

See Also:

FileInputFormat.open(org.apache.flink.core.fs.FileInputSplit)

readRecord

public OT readRecord(OT reuse,
                     byte[] bytes,
                     int offset,
                     int numBytes)
              throws IOException

Description copied from class: DelimitedInputFormat

This function parses the given byte array which represents a serialized record. The function returns a valid record or throws an IOException.

Specified by:

readRecord in class DelimitedInputFormat<OT>

Parameters:

reuse - An optionally reusable object.

bytes - Binary data of serialized records.

offset - The offset where to start to read the record data.

numBytes - The number of bytes that can be read starting at the offset position.

Returns:

Returns the read record if it was successfully deserialized.

Throws:

IOException - if the record could not be read.