org.apache.flink.api.java.io

Class TextValueInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<StringValue>

org.apache.flink.api.java.io.TextValueInputFormat

All Implemented Interfaces:

Serializable, CheckpointableInputFormat<FileInputSplit,Long>, InputFormat<StringValue,FileInputSplit>, InputSplitSource<FileInputSplit>

@PublicEvolving
public class TextValueInputFormat
extends DelimitedInputFormat<StringValue>

Input format that reads text files.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
TextValueInputFormat(Path filePath)

Method Summary

Modifier and Type	Method and Description
void	configure(Configuration parameters) Configures this input format by reading the path to the file from the configuration and the string that defines the record delimiter.
String	getCharsetName()
boolean	isSkipInvalidLines()
StringValue	readRecord(StringValue reuse, byte[] bytes, int offset, int numBytes) This function parses the given byte array which represents a serialized record.
void	setCharsetName(String charsetName)
void	setSkipInvalidLines(boolean skipInvalidLines)
boolean	supportsMultiPaths() Override this method to supports multiple paths.
String	toString()

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

close, getBufferSize, getCharset, getCurrentState, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadConfigParameters, loadGlobalConfigParams, nextRecord, open, reachedEnd, readLine, reopen, setBufferSize, setCharset, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

TextValueInputFormat

public TextValueInputFormat(Path filePath)

Method Detail

getCharsetName

public String getCharsetName()

setCharsetName

public void setCharsetName(String charsetName)

isSkipInvalidLines

public boolean isSkipInvalidLines()

setSkipInvalidLines

public void setSkipInvalidLines(boolean skipInvalidLines)

configure

public void configure(Configuration parameters)

Description copied from class: DelimitedInputFormat

Configures this input format by reading the path to the file from the configuration and the string that defines the record delimiter.

Specified by:

configure in interface InputFormat<StringValue,FileInputSplit>

Overrides:

configure in class DelimitedInputFormat<StringValue>

Parameters:

parameters - The configuration object to read the parameters from.

See Also:

InputFormat.configure(org.apache.flink.configuration.Configuration)

readRecord

public StringValue readRecord(StringValue reuse,
                              byte[] bytes,
                              int offset,
                              int numBytes)

Description copied from class: DelimitedInputFormat

This function parses the given byte array which represents a serialized record. The function returns a valid record or throws an IOException.

Specified by:

readRecord in class DelimitedInputFormat<StringValue>

Parameters:

reuse - An optionally reusable object.

bytes - Binary data of serialized records.

offset - The offset where to start to read the record data.

numBytes - The number of bytes that can be read starting at the offset position.

Returns:

Returns the read record if it was successfully deserialized.

toString

public String toString()

Overrides:

toString in class FileInputFormat<StringValue>

supportsMultiPaths

public boolean supportsMultiPaths()

Description copied from class: FileInputFormat

Override this method to supports multiple paths. When this method will be removed, all FileInputFormats have to support multiple paths.

Overrides:

supportsMultiPaths in class FileInputFormat<StringValue>

Returns:

True if the FileInputFormat supports multiple paths, false otherwise.