org.apache.flink.orc

Class OrcFileSystemFormatFactory.OrcRowDataInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<RowData>

org.apache.flink.orc.OrcFileSystemFormatFactory.OrcRowDataInputFormat

All Implemented Interfaces:

Serializable, InputFormat<RowData,FileInputSplit>, InputSplitSource<FileInputSplit>

Enclosing class:

OrcFileSystemFormatFactory

public static class OrcFileSystemFormatFactory.OrcRowDataInputFormat
extends FileInputFormat<RowData>

An implementation of FileInputFormat to read RowData records from orc files.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
OrcRowDataInputFormat(Path[] paths, String[] fullFieldNames, DataType[] fullFieldTypes, int[] selectedFields, String partDefaultName, long limit, Properties properties)

Method Summary

Modifier and Type	Method and Description
void	close() Closes the file input stream of the input format.
RowData	nextRecord(RowData reuse) Reads the next record from the input.
void	open(FileInputSplit fileSplit) Opens an input stream to the file defined in the input format.
boolean	reachedEnd() Method used to check if the end of the input is reached.
boolean	supportsMultiPaths() Override this method to supports multiple paths.

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, configure, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

OrcRowDataInputFormat

public OrcRowDataInputFormat(Path[] paths,
                             String[] fullFieldNames,
                             DataType[] fullFieldTypes,
                             int[] selectedFields,
                             String partDefaultName,
                             long limit,
                             Properties properties)

Method Detail

open

public void open(FileInputSplit fileSplit)
          throws IOException

Description copied from class: FileInputFormat

Opens an input stream to the file defined in the input format. The stream is positioned at the beginning of the given split.

The stream is actually opened in an asynchronous thread to make sure any interruptions to the thread working on the input format do not reach the file system.

Specified by:

open in interface InputFormat<RowData,FileInputSplit>

Overrides:

open in class FileInputFormat<RowData>

Parameters:

fileSplit - The split to be opened.

Throws:

IOException - Thrown, if the spit could not be opened due to an I/O problem.

supportsMultiPaths

public boolean supportsMultiPaths()

Description copied from class: FileInputFormat

Override this method to supports multiple paths. When this method will be removed, all FileInputFormats have to support multiple paths.

Overrides:

supportsMultiPaths in class FileInputFormat<RowData>

Returns:

True if the FileInputFormat supports multiple paths, false otherwise.

reachedEnd

public boolean reachedEnd()
                   throws IOException

Description copied from interface: InputFormat

Method used to check if the end of the input is reached.

When this method is called, the input format it guaranteed to be opened.

Returns:

True if the end is reached, otherwise false.

Throws:

IOException - Thrown, if an I/O error occurred.

nextRecord

public RowData nextRecord(RowData reuse)

Description copied from interface: InputFormat

Reads the next record from the input.

When this method is called, the input format it guaranteed to be opened.

Parameters:

reuse - Object that may be reused.

Returns:

Read record.

close

public void close()
           throws IOException

Description copied from class: FileInputFormat

Closes the file input stream of the input format.

Specified by:

close in interface InputFormat<RowData,FileInputSplit>

Overrides:

close in class FileInputFormat<RowData>

Throws:

IOException - Thrown, if the input could not be closed properly.