org.apache.flink.connectors.hive.read

Class HiveTableFileInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<RowData>

org.apache.flink.connectors.hive.read.HiveTableFileInputFormat

All Implemented Interfaces:

Serializable, InputFormat<RowData,FileInputSplit>, InputSplitSource<FileInputSplit>

public class HiveTableFileInputFormat
extends FileInputFormat<RowData>

A FileInputFormat that wraps a HiveTableInputFormat.

We only use a HiveTableInputFormat to read the data of a FileInputSplit. `createInputSplits`, `getInputSplitAssigner` will use FileInputFormat's logic.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

FileInputFormat.FileBaseStatistics, FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
HiveTableFileInputFormat(HiveTableInputFormat inputFormat, HiveTablePartition hiveTablePartition)

Method Summary

Modifier and Type	Method and Description
void	close() Closes the file input stream of the input format.
void	closeInputFormat() Closes this InputFormat instance.
void	configure(Configuration parameters) Configures the file input format by reading the file path from the configuration.
RowData	nextRecord(RowData reuse) Reads the next record from the input.
void	open(FileInputSplit fileSplit) Opens an input stream to the file defined in the input format.
void	openInputFormat() Opens this InputFormat instance.
boolean	reachedEnd() Method used to check if the end of the input is reached.
void	setRuntimeContext(RuntimeContext t)

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, supportsMultiPaths, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

getRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HiveTableFileInputFormat

public HiveTableFileInputFormat(HiveTableInputFormat inputFormat,
                                HiveTablePartition hiveTablePartition)

Method Detail

open

public void open(FileInputSplit fileSplit)
          throws IOException

Description copied from class: FileInputFormat

Opens an input stream to the file defined in the input format. The stream is positioned at the beginning of the given split.

The stream is actually opened in an asynchronous thread to make sure any interruptions to the thread working on the input format do not reach the file system.

Specified by:

open in interface InputFormat<RowData,FileInputSplit>

Overrides:

open in class FileInputFormat<RowData>

Parameters:

fileSplit - The split to be opened.

Throws:

IOException - Thrown, if the spit could not be opened due to an I/O problem.

reachedEnd

public boolean reachedEnd()
                   throws IOException

Description copied from interface: InputFormat

Method used to check if the end of the input is reached.

When this method is called, the input format it guaranteed to be opened.

Returns:

True if the end is reached, otherwise false.

Throws:

IOException - Thrown, if an I/O error occurred.

nextRecord

public RowData nextRecord(RowData reuse)
                   throws IOException

Description copied from interface: InputFormat

Reads the next record from the input.

When this method is called, the input format it guaranteed to be opened.

Parameters:

reuse - Object that may be reused.

Returns:

Read record.

Throws:

IOException - Thrown, if an I/O error occurred.

configure

public void configure(Configuration parameters)

Description copied from class: FileInputFormat

Configures the file input format by reading the file path from the configuration.

Specified by:

configure in interface InputFormat<RowData,FileInputSplit>

Overrides:

configure in class FileInputFormat<RowData>

Parameters:

parameters - The configuration with all parameters (note: not the Flink config but the TaskConfig).

See Also:

InputFormat.configure(org.apache.flink.configuration.Configuration)

close

public void close()
           throws IOException

Description copied from class: FileInputFormat

Closes the file input stream of the input format.

Specified by:

close in interface InputFormat<RowData,FileInputSplit>

Overrides:

close in class FileInputFormat<RowData>

Throws:

IOException - Thrown, if the input could not be closed properly.

setRuntimeContext

public void setRuntimeContext(RuntimeContext t)

Overrides:

setRuntimeContext in class RichInputFormat<RowData,FileInputSplit>

openInputFormat

public void openInputFormat()
                     throws IOException

Description copied from class: RichInputFormat

Opens this InputFormat instance. This method is called once per parallel instance. Resources should be allocated in this method. (e.g. database connections, cache, etc.)

Overrides:

openInputFormat in class RichInputFormat<RowData,FileInputSplit>

Throws:

IOException - in case allocating the resources failed.

See Also:

InputFormat

closeInputFormat

public void closeInputFormat()
                      throws IOException

Description copied from class: RichInputFormat

Closes this InputFormat instance. This method is called once per parallel instance. Resources allocated during RichInputFormat.openInputFormat() should be closed in this method.

Overrides:

closeInputFormat in class RichInputFormat<RowData,FileInputSplit>

Throws:

IOException - in case closing the resources failed

See Also:

InputFormat