org.apache.flink.connectors.hive

Class HiveSourceBuilder

java.lang.Object

org.apache.flink.connectors.hive.HiveSourceBuilder

@PublicEvolving
public class HiveSourceBuilder
extends Object

Builder to build HiveSource instances.

Constructor Summary

Constructors

Constructor and Description
HiveSourceBuilder(org.apache.hadoop.mapred.JobConf jobConf, ReadableConfig flinkConf, ObjectPath tablePath, String hiveVersion, ResolvedCatalogTable catalogTable) Creates a builder to read a hive table.
HiveSourceBuilder(org.apache.hadoop.mapred.JobConf jobConf, ReadableConfig flinkConf, String hiveVersion, String dbName, String tableName, Map<String,String> tableOptions) Creates a builder to read a hive table.

Method Summary

Modifier and Type	Method and Description
<T> HiveSource<T>	buildWithBulkFormat(BulkFormat<T,HiveSourceSplit> bulkFormat) Builds HiveSource with custom BulkFormat.
HiveSource<RowData>	buildWithDefaultBulkFormat() Builds HiveSource with default built-in BulkFormat that returns records in type of RowData.
protected BulkFormat<RowData,HiveSourceSplit>	createDefaultBulkFormat()
HiveSourceBuilder	setDynamicFilterPartitionKeys(List<String> dynamicFilterPartitionKeys)
HiveSourceBuilder	setLimit(Long limit) Sets the maximum number of records this source should return.
HiveSourceBuilder	setPartitions(List<HiveTablePartition> partitions) Sets the partitions to read in batch mode.
HiveSourceBuilder	setProjectedFields(int[] projectedFields) Sets the indices of projected fields.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HiveSourceBuilder

public HiveSourceBuilder(@Nonnull
                         org.apache.hadoop.mapred.JobConf jobConf,
                         @Nonnull
                         ReadableConfig flinkConf,
                         @Nullable
                         String hiveVersion,
                         @Nonnull
                         String dbName,
                         @Nonnull
                         String tableName,
                         @Nonnull
                         Map<String,String> tableOptions)

Creates a builder to read a hive table.

Parameters:

jobConf - holds hive and hadoop configurations

flinkConf - holds flink configurations

hiveVersion - the version of hive in use, if it's null the version will be automatically detected

dbName - the name of the database the table belongs to

tableName - the name of the table

tableOptions - additional options needed to read the table, which take precedence over table properties stored in metastore

HiveSourceBuilder

public HiveSourceBuilder(@Nonnull
                         org.apache.hadoop.mapred.JobConf jobConf,
                         @Nonnull
                         ReadableConfig flinkConf,
                         @Nonnull
                         ObjectPath tablePath,
                         @Nullable
                         String hiveVersion,
                         @Nonnull
                         ResolvedCatalogTable catalogTable)

Creates a builder to read a hive table.

Parameters:

jobConf - holds hive and hadoop configurations

flinkConf - holds flink configurations

hiveVersion - the version of hive in use, if it's null the version will be automatically detected

tablePath - path of the table to be read

catalogTable - the table to be read

Method Detail

buildWithDefaultBulkFormat

public HiveSource<RowData> buildWithDefaultBulkFormat()

Builds HiveSource with default built-in BulkFormat that returns records in type of RowData.

buildWithBulkFormat

public <T> HiveSource<T> buildWithBulkFormat(BulkFormat<T,HiveSourceSplit> bulkFormat)

Builds HiveSource with custom BulkFormat.

setPartitions

public HiveSourceBuilder setPartitions(List<HiveTablePartition> partitions)

Sets the partitions to read in batch mode. By default, batch source reads all partitions in a hive table.

setDynamicFilterPartitionKeys

public HiveSourceBuilder setDynamicFilterPartitionKeys(List<String> dynamicFilterPartitionKeys)

setLimit

public HiveSourceBuilder setLimit(Long limit)

Sets the maximum number of records this source should return.

setProjectedFields

public HiveSourceBuilder setProjectedFields(int[] projectedFields)

Sets the indices of projected fields.

Parameters:

projectedFields - indices of the fields, starting from 0

createDefaultBulkFormat

protected BulkFormat<RowData,HiveSourceSplit> createDefaultBulkFormat()