org.apache.flink.orc

Class OrcRowInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<T>

org.apache.flink.orc.OrcInputFormat<Row>

org.apache.flink.orc.OrcRowInputFormat

All Implemented Interfaces:

Serializable, InputFormat<Row,FileInputSplit>, ResultTypeQueryable<Row>, InputSplitSource<FileInputSplit>

public class OrcRowInputFormat
extends OrcInputFormat<Row>
implements ResultTypeQueryable<Row>

InputFormat to read ORC files into Row.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.orc.OrcInputFormat

batchSize, conf, conjunctPredicates, reader, schema, selectedFields

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
OrcRowInputFormat(String path, String schemaString, Configuration orcConfig) Creates an OrcRowInputFormat.
OrcRowInputFormat(String path, String schemaString, Configuration orcConfig, int batchSize) Creates an OrcRowInputFormat.
OrcRowInputFormat(String path, org.apache.orc.TypeDescription orcSchema, Configuration orcConfig, int batchSize) Creates an OrcRowInputFormat.

Method Summary

Modifier and Type	Method and Description
TypeInformation<Row>	getProducedType() Gets the data type (as a TypeInformation) produced by this function or input format.
void	open(FileInputSplit fileSplit) Opens an input stream to the file defined in the input format.
void	selectFields(int... selectedFields) Selects the fields from the ORC schema that are returned by InputFormat.

Methods inherited from class org.apache.flink.orc.OrcInputFormat

addPredicate, close, closeInputFormat, nextRecord, reachedEnd, supportsMultiPaths

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, configure, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

OrcRowInputFormat

public OrcRowInputFormat(String path,
                         String schemaString,
                         Configuration orcConfig)

Creates an OrcRowInputFormat.

Parameters:

path - The path to read ORC files from.

schemaString - The schema of the ORC files as String.

orcConfig - The configuration to read the ORC files with.

OrcRowInputFormat

public OrcRowInputFormat(String path,
                         String schemaString,
                         Configuration orcConfig,
                         int batchSize)

Creates an OrcRowInputFormat.

Parameters:

path - The path to read ORC files from.

schemaString - The schema of the ORC files as String.

orcConfig - The configuration to read the ORC files with.

batchSize - The number of Row objects to read in a batch.

OrcRowInputFormat

public OrcRowInputFormat(String path,
                         org.apache.orc.TypeDescription orcSchema,
                         Configuration orcConfig,
                         int batchSize)

Creates an OrcRowInputFormat.

Parameters:

path - The path to read ORC files from.

orcSchema - The schema of the ORC files as ORC TypeDescription.

orcConfig - The configuration to read the ORC files with.

batchSize - The number of Row objects to read in a batch.

Method Detail

selectFields

public void selectFields(int... selectedFields)

Description copied from class: OrcInputFormat

Selects the fields from the ORC schema that are returned by InputFormat.

Overrides:

selectFields in class OrcInputFormat<Row>

Parameters:

selectedFields - The indices of the fields of the ORC schema that are returned by the InputFormat.

open

public void open(FileInputSplit fileSplit)
          throws IOException

Description copied from class: FileInputFormat

Opens an input stream to the file defined in the input format. The stream is positioned at the beginning of the given split.

The stream is actually opened in an asynchronous thread to make sure any interruptions to the thread working on the input format do not reach the file system.

Specified by:

open in interface InputFormat<Row,FileInputSplit>

Overrides:

open in class FileInputFormat<Row>

Parameters:

fileSplit - The split to be opened.

Throws:

IOException - Thrown, if the spit could not be opened due to an I/O problem.

getProducedType

public TypeInformation<Row> getProducedType()

Description copied from interface: ResultTypeQueryable

Gets the data type (as a TypeInformation) produced by this function or input format.

Specified by:

getProducedType in interface ResultTypeQueryable<Row>

Returns:

The data type produced by this function or input format.