org.apache.flink.connectors.hive.read

Class HiveBulkFormatAdapter

java.lang.Object

org.apache.flink.connectors.hive.read.HiveBulkFormatAdapter

All Implemented Interfaces:

Serializable, ResultTypeQueryable<RowData>, BulkFormat<RowData,HiveSourceSplit>

public class HiveBulkFormatAdapter
extends Object
implements BulkFormat<RowData,HiveSourceSplit>

A BulkFormat implementation for HiveSource. This implementation delegates reading to other BulkFormat instances, because different hive partitions may need different BulkFormat to do the reading.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.connector.file.src.reader.BulkFormat

BulkFormat.Reader<T>, BulkFormat.RecordIterator<T>

Constructor Summary

Constructors

Constructor and Description
HiveBulkFormatAdapter(JobConfWrapper jobConfWrapper, List<String> partitionKeys, String[] fieldNames, DataType[] fieldTypes, String hiveVersion, RowType producedRowType, boolean useMapRedReader)

Method Summary

Modifier and Type	Method and Description
BulkFormat.Reader<RowData>	createReader(Configuration config, HiveSourceSplit split) Creates a new reader that reads from the split's path starting at the FileSourceSplit.offset() split's offset} and reads length bytes after the offset.
TypeInformation<RowData>	getProducedType() Gets the type produced by this format.
boolean	isSplittable() Checks whether this format is splittable.
BulkFormat.Reader<RowData>	restoreReader(Configuration config, HiveSourceSplit split) Creates a new reader that reads from split.path() starting at offset and reads until length bytes after the offset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HiveBulkFormatAdapter

public HiveBulkFormatAdapter(JobConfWrapper jobConfWrapper,
                             List<String> partitionKeys,
                             String[] fieldNames,
                             DataType[] fieldTypes,
                             String hiveVersion,
                             RowType producedRowType,
                             boolean useMapRedReader)

Method Detail

createReader

public BulkFormat.Reader<RowData> createReader(Configuration config,
                                               HiveSourceSplit split)
                                        throws IOException

Description copied from interface: BulkFormat

Creates a new reader that reads from the split's path starting at the FileSourceSplit.offset() split's offset} and reads length bytes after the offset.

Specified by:

createReader in interface BulkFormat<RowData,HiveSourceSplit>

Throws:

IOException

restoreReader

public BulkFormat.Reader<RowData> restoreReader(Configuration config,
                                                HiveSourceSplit split)
                                         throws IOException

Description copied from interface: BulkFormat

Creates a new reader that reads from split.path() starting at offset and reads until length bytes after the offset. A number of recordsToSkip records should be read and discarded after the offset. This is typically part of restoring a reader to a checkpointed position.

Specified by:

restoreReader in interface BulkFormat<RowData,HiveSourceSplit>

Throws:

IOException

isSplittable

public boolean isSplittable()

Description copied from interface: BulkFormat

Checks whether this format is splittable. Splittable formats allow Flink to create multiple splits per file, so that Flink can read multiple regions of the file concurrently.

See top-level JavaDocs (section "Splitting") for details.

Specified by:

isSplittable in interface BulkFormat<RowData,HiveSourceSplit>

getProducedType

public TypeInformation<RowData> getProducedType()

Description copied from interface: BulkFormat

Gets the type produced by this format. This type will be the type produced by the file source as a whole.

Specified by:

getProducedType in interface ResultTypeQueryable<RowData>

Specified by:

getProducedType in interface BulkFormat<RowData,HiveSourceSplit>

Returns:

The data type produced by this function or input format.