org.apache.flink.api.java.io

Class TextInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<String>

org.apache.flink.api.java.io.TextInputFormat

All Implemented Interfaces:

Serializable, CheckpointableInputFormat<FileInputSplit,Long>, InputFormat<String,FileInputSplit>, InputSplitSource<FileInputSplit>

@PublicEvolving
public class TextInputFormat
extends DelimitedInputFormat<String>

Input Format that reads text files. Each line results in another element.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

currBuffer, currLen, currOffset, RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
TextInputFormat(Path filePath)

Method Summary

Modifier and Type	Method and Description
void	configure(Configuration parameters) Configures this input format by reading the path to the file from the configuration and the string that defines the record delimiter.
String	getCharsetName()
String	readRecord(String reusable, byte[] bytes, int offset, int numBytes) This function parses the given byte array which represents a serialized record.
void	setCharsetName(String charsetName)
boolean	supportsMultiPaths() Override this method to supports multiple paths.
String	toString()

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

close, getBufferSize, getCharset, getCurrentState, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, initializeSplit, loadConfigParameters, loadGlobalConfigParams, nextRecord, open, reachedEnd, readLine, reopen, setBufferSize, setCharset, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

TextInputFormat

public TextInputFormat(Path filePath)

Method Detail

getCharsetName

public String getCharsetName()

setCharsetName

public void setCharsetName(String charsetName)

configure

public void configure(Configuration parameters)

Description copied from class: DelimitedInputFormat

Configures this input format by reading the path to the file from the configuration and the string that defines the record delimiter.

Specified by:

configure in interface InputFormat<String,FileInputSplit>

Overrides:

configure in class DelimitedInputFormat<String>

Parameters:

parameters - The configuration object to read the parameters from.

See Also:

InputFormat.configure(org.apache.flink.configuration.Configuration)

readRecord

public String readRecord(String reusable,
                         byte[] bytes,
                         int offset,
                         int numBytes)
                  throws IOException

Description copied from class: DelimitedInputFormat

This function parses the given byte array which represents a serialized record. The function returns a valid record or throws an IOException.

Specified by:

readRecord in class DelimitedInputFormat<String>

Parameters:

reusable - An optionally reusable object.

bytes - Binary data of serialized records.

offset - The offset where to start to read the record data.

numBytes - The number of bytes that can be read starting at the offset position.

Returns:

Returns the read record if it was successfully deserialized.

Throws:

IOException - if the record could not be read.

toString

public String toString()

Overrides:

toString in class FileInputFormat<String>

supportsMultiPaths

public boolean supportsMultiPaths()

Description copied from class: FileInputFormat

Override this method to supports multiple paths. When this method will be removed, all FileInputFormats have to support multiple paths.

Overrides:

supportsMultiPaths in class FileInputFormat<String>

Returns:

True if the FileInputFormat supports multiple paths, false otherwise.