org.apache.flink.connectors.hive

Class HiveTableSource.HiveContinuousPartitionFetcherContext<T extends Comparable<T>>

java.lang.Object

org.apache.flink.connectors.hive.read.HivePartitionFetcherContextBase<org.apache.hadoop.hive.metastore.api.Partition>

org.apache.flink.connectors.hive.HiveTableSource.HiveContinuousPartitionFetcherContext<T>

All Implemented Interfaces:

Serializable, ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T>, PartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition>

Enclosing class:

HiveTableSource

public static class HiveTableSource.HiveContinuousPartitionFetcherContext<T extends Comparable<T>>
extends HivePartitionFetcherContextBase<org.apache.hadoop.hive.metastore.api.Partition>
implements ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T>

PartitionFetcher.Context for ContinuousPartitionFetcher.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.table.filesystem.PartitionFetcher.Context

PartitionFetcher.Context.ComparablePartitionValue<P,T extends Comparable<T>>

Field Summary

Fields inherited from class org.apache.flink.connectors.hive.read.HivePartitionFetcherContextBase

configuration, confWrapper, consumeOrder, defaultPartitionName, fieldNames, fieldTypes, hiveShim, metaStoreClient, partitionKeys, tableLocation, tablePath, tableProps, tableSd

Constructor Summary

Constructors

Constructor and Description
HiveContinuousPartitionFetcherContext(ObjectPath tablePath, HiveShim hiveShim, JobConfWrapper confWrapper, List<String> partitionKeys, DataType[] fieldTypes, String[] fieldNames, Configuration configuration, String defaultPartitionName)

Method Summary

Modifier and Type	Method and Description
void	close() close the resources of the Context, this method should call when the context do not need any more.
T	getConsumeStartOffset() Get the partition consume start offset.
long	getModificationTime(org.apache.hadoop.hive.metastore.api.Partition partition, T partitionOffset) Get the partition modified time.
Optional<org.apache.hadoop.hive.metastore.api.Partition>	getPartition(List<String> partValues) Get partition by file partition values.
ObjectPath	getTablePath() The table full path.
TypeSerializer<T>	getTypeSerializer() Get the Serializer of partition order.
HiveTablePartition	toHiveTablePartition(org.apache.hadoop.hive.metastore.api.Partition partition) Convert partition to HiveTablePartition.

Methods inherited from class org.apache.flink.connectors.hive.read.HivePartitionFetcherContextBase

getComparablePartitionValueList, open

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.filesystem.PartitionFetcher.Context

getComparablePartitionValueList, open

Constructor Detail

HiveContinuousPartitionFetcherContext

public HiveContinuousPartitionFetcherContext(ObjectPath tablePath,
                                             HiveShim hiveShim,
                                             JobConfWrapper confWrapper,
                                             List<String> partitionKeys,
                                             DataType[] fieldTypes,
                                             String[] fieldNames,
                                             Configuration configuration,
                                             String defaultPartitionName)

Method Detail

getPartition

public Optional<org.apache.hadoop.hive.metastore.api.Partition> getPartition(List<String> partValues)
                                                                      throws org.apache.thrift.TException

Description copied from interface: PartitionFetcher.Context

Get partition by file partition values.

Specified by:

getPartition in interface PartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition>

Throws:

org.apache.thrift.TException

getTablePath

public ObjectPath getTablePath()

Description copied from interface: ContinuousPartitionFetcher.Context

The table full path.

Specified by:

getTablePath in interface ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T extends Comparable<T>>

getModificationTime

public long getModificationTime(org.apache.hadoop.hive.metastore.api.Partition partition,
                                T partitionOffset)

Get the partition modified time.

the time is the the folder/file modification time in filesystem when fetched in create-time order, the time is extracted from partition name when fetched in partition-time order, the time is partion create time in metaStore when fetched in partition-name order.

toHiveTablePartition

public HiveTablePartition toHiveTablePartition(org.apache.hadoop.hive.metastore.api.Partition partition)

Convert partition to HiveTablePartition.

getTypeSerializer

public TypeSerializer<T> getTypeSerializer()

Description copied from interface: ContinuousPartitionFetcher.Context

Get the Serializer of partition order.

Specified by:

getTypeSerializer in interface ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T extends Comparable<T>>

getConsumeStartOffset

public T getConsumeStartOffset()

Description copied from interface: ContinuousPartitionFetcher.Context

Get the partition consume start offset.

Specified by:

getConsumeStartOffset in interface ContinuousPartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition,T extends Comparable<T>>

close

public void close()
           throws Exception

Description copied from interface: PartitionFetcher.Context

close the resources of the Context, this method should call when the context do not need any more.

Specified by:

close in interface PartitionFetcher.Context<org.apache.hadoop.hive.metastore.api.Partition>

Overrides:

close in class HivePartitionFetcherContextBase<org.apache.hadoop.hive.metastore.api.Partition>

Throws:

Exception