public class HiveLookupTableSource extends HiveTableSource implements LookupTableSource
Hive Table source has both lookup and continuous read ability, when it acts as continuous read source it does not have the lookup ability but can be a temporal table just like other stream sources. When it acts as bounded table, it has the lookup ability.
A common user case is use hive table as dimension table and always lookup the latest partition data, in this case, hive table source is a continuous read source but currently we implements it by LookupFunction. Because currently TableSource can not tell the downstream when the latest partition has been read finished. This is a temporarily workaround and will re-implement in the future.
HiveTableSource.HiveContinuousPartitionFetcherContext<T extends Comparable<T>>
|Constructor and Description|
|Modifier and Type||Method and Description|
Returns a provider of runtime implementation for reading the data.
applyDynamicFiltering, applyLimit, applyPartitions, applyProjection, asSummaryString, copy, getChangelogMode, getDataStream, getJobConf, getProducedTableSchema, getScanRuntimeProvider, getTableSchema, isStreamingSource, listPartitions, reportStatistics, supportsNestedProjection
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
public LookupTableSource.LookupRuntimeProvider getLookupRuntimeProvider(LookupTableSource.LookupContext context)
There exist different interfaces for runtime implementation which is why
LookupTableSource.LookupRuntimeProvider serves as the base interface.
Independent of the provider interface, a source implementation can work on either
arbitrary objects or internal data structures (see
LookupTableSource.LookupContext offers utilities by the planner for creating runtime
implementation with minimal dependencies to internal data structures.
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.