org.apache.flink.table.catalog.hive.client

Class HiveShimV100

java.lang.Object

org.apache.flink.table.catalog.hive.client.HiveShimV100

All Implemented Interfaces:

Serializable, HiveShim

Direct Known Subclasses:

HiveShimV101

public class HiveShimV100
extends Object
implements HiveShim

Shim for Hive version 1.0.0.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	holdDDLTime
protected boolean	inheritTableSpecs
protected boolean	isAcid
protected boolean	isSkewedStoreAsSubdir

Constructor Summary

Constructors

Constructor and Description
HiveShimV100()

Method Summary

Modifier and Type	Method and Description
void	alterPartition(org.apache.hadoop.hive.metastore.IMetaStoreClient client, String databaseName, String tableName, org.apache.hadoop.hive.metastore.api.Partition partition)
void	alterTable(org.apache.hadoop.hive.metastore.IMetaStoreClient client, String databaseName, String tableName, org.apache.hadoop.hive.metastore.api.Table table) Alters a Hive table.
BulkWriter.Factory<RowData>	createOrcBulkWriterFactory(org.apache.hadoop.conf.Configuration conf, String schema, LogicalType[] fieldTypes) Create orc BulkWriter.Factory for different hive versions.
void	createTableWithConstraints(org.apache.hadoop.hive.metastore.IMetaStoreClient client, org.apache.hadoop.hive.metastore.api.Table table, org.apache.hadoop.conf.Configuration conf, UniqueConstraint pk, List<Byte> pkTraits, List<String> notNullCols, List<Byte> nnTraits) Creates a table with PK and NOT NULL constraints.
org.apache.hadoop.hive.ql.udf.generic.SimpleGenericUDAFParameterInfo	createUDAFParameterInfo(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector[] params, boolean isWindowing, boolean distinct, boolean allColumns) Creates SimpleGenericUDAFParameterInfo.
Optional<org.apache.hadoop.hive.ql.exec.FunctionInfo>	getBuiltInFunctionInfo(String name) Get a Hive built-in function by name.
Class<?>	getDateDataTypeClass() Hive Date data type class was changed in Hive 3.1.0.
Class<?>	getDateWritableClass() Gets writable class for Date type.
List<org.apache.hadoop.hive.metastore.api.FieldSchema>	getFieldsFromDeserializer(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hive.metastore.api.Table table, boolean skipConfError) Get Hive table schema from deserializer.
org.apache.hadoop.hive.metastore.IMetaStoreClient	getHiveMetastoreClient(HiveConf hiveConf) Create a Hive Metastore client based on the given HiveConf object.
Class<?>	getHiveMetaStoreUtilsClass() Get the class of Hive's HiveMetaStoreUtils as it was split from MetaStoreUtils class in Hive 3.1.0.
Class	getHiveOutputFormatClass(Class outputFormatClz) For a given OutputFormat class, get the corresponding HiveOutputFormat class.
org.apache.hadoop.hive.ql.exec.FileSinkOperator.RecordWriter	getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jobConf, Class outputFormatClz, Class<? extends org.apache.hadoop.io.Writable> outValClz, boolean isCompressed, Properties tableProps, org.apache.hadoop.fs.Path outPath) Get Hive's FileSinkOperator.RecordWriter.
Class<?>	getMetaStoreUtilsClass() Get the class of Hive's MetaStoreUtils because its package name was changed in Hive 3.1.0.
Set<String>	getNotNullColumns(org.apache.hadoop.hive.metastore.IMetaStoreClient client, org.apache.hadoop.conf.Configuration conf, String dbName, String tableName) Get the set of columns that have NOT NULL constraints.
Optional<UniqueConstraint>	getPrimaryKey(org.apache.hadoop.hive.metastore.IMetaStoreClient client, String dbName, String tableName, byte requiredTrait) Get the primary key of a Hive table and convert it to a UniqueConstraint.
Class<?>	getTimestampDataTypeClass() Hive Timestamp data type class was changed in Hive 3.1.0.
Class<?>	getTimestampWritableClass() Gets writable class for Timestamp type.
List<String>	getViews(org.apache.hadoop.hive.metastore.IMetaStoreClient client, String databaseName) Get a list of views in the given database from the given Hive Metastore client.
org.apache.hadoop.io.Writable	hivePrimitiveToWritable(Object value) Converts a Hive primitive java object to corresponding Writable object.
boolean	isDateStats(org.apache.hadoop.hive.metastore.api.ColumnStatisticsData colStatsData) Whether a Hive ColumnStatisticsData is for DATE columns.
Set<String>	listBuiltInFunctions() List names of all built-in functions.
void	loadPartition(org.apache.hadoop.hive.ql.metadata.Hive hive, org.apache.hadoop.fs.Path loadPath, String tableName, Map<String,String> partSpec, boolean isSkewedStoreAsSubdir, boolean replace, boolean isSrcLocal)
void	loadTable(org.apache.hadoop.hive.ql.metadata.Hive hive, org.apache.hadoop.fs.Path loadPath, String tableName, boolean replace, boolean isSrcLocal)
void	registerTemporaryFunction(String funcName, Class funcClass)
java.time.LocalDate	toFlinkDate(Object hiveDate) Converts a hive date instance to LocalDate which is expected by DataFormatConverter.
CatalogColumnStatisticsDataDate	toFlinkDateColStats(org.apache.hadoop.hive.metastore.api.ColumnStatisticsData hiveDateColStats) Generate Flink CatalogColumnStatisticsDataDate from Hive ColumnStatisticsData for DATE columns.
java.time.LocalDateTime	toFlinkTimestamp(Object hiveTimestamp) Converts a hive timestamp instance to LocalDateTime which is expected by DataFormatConverter.
Object	toHiveDate(Object flinkDate) Converts a Flink date instance to what's expected by Hive.
org.apache.hadoop.hive.metastore.api.ColumnStatisticsData	toHiveDateColStats(CatalogColumnStatisticsDataDate flinkDateColStats) Generate Hive ColumnStatisticsData from Flink CatalogColumnStatisticsDataDate for DATE columns.
Object	toHiveTimestamp(Object flinkTimestamp) Converts a Flink timestamp instance to what's expected by Hive.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.catalog.hive.client.HiveShim

getIntervalDayTimeTypeInfo, getIntervalYearMonthTypeInfo, isIntervalDayTimeType, isIntervalYearMonthType, isMaterializedView

Field Detail

holdDDLTime

protected final boolean holdDDLTime

See Also:

Constant Field Values

isAcid

protected final boolean isAcid

See Also:

Constant Field Values

inheritTableSpecs

protected final boolean inheritTableSpecs

See Also:

Constant Field Values

isSkewedStoreAsSubdir

protected final boolean isSkewedStoreAsSubdir

See Also:

Constant Field Values

Constructor Detail

HiveShimV100

public HiveShimV100()

Method Detail

getHiveMetastoreClient

public org.apache.hadoop.hive.metastore.IMetaStoreClient getHiveMetastoreClient(HiveConf hiveConf)

Description copied from interface: HiveShim

Create a Hive Metastore client based on the given HiveConf object.

Specified by:

getHiveMetastoreClient in interface HiveShim

Parameters:

hiveConf - HiveConf instance

Returns:

an IMetaStoreClient instance

getViews

public List<String> getViews(org.apache.hadoop.hive.metastore.IMetaStoreClient client,
                             String databaseName)
                      throws org.apache.hadoop.hive.metastore.api.UnknownDBException,
                             org.apache.thrift.TException

Description copied from interface: HiveShim

Get a list of views in the given database from the given Hive Metastore client.

Specified by:

getViews in interface HiveShim

Parameters:

client - Hive Metastore client

databaseName - the name of the database

Returns:

A list of names of the views

Throws:

org.apache.hadoop.hive.metastore.api.UnknownDBException - if the database doesn't exist

org.apache.thrift.TException - for any other generic exceptions caused by Thrift

alterTable

public void alterTable(org.apache.hadoop.hive.metastore.IMetaStoreClient client,
                       String databaseName,
                       String tableName,
                       org.apache.hadoop.hive.metastore.api.Table table)
                throws org.apache.hadoop.hive.metastore.api.InvalidOperationException,
                       org.apache.hadoop.hive.metastore.api.MetaException,
                       org.apache.thrift.TException

Description copied from interface: HiveShim

Alters a Hive table.

Specified by:

alterTable in interface HiveShim

Parameters:

client - the Hive metastore client

databaseName - the name of the database to which the table belongs

tableName - the name of the table to be altered

table - the new Hive table

Throws:

org.apache.hadoop.hive.metastore.api.InvalidOperationException

org.apache.hadoop.hive.metastore.api.MetaException

org.apache.thrift.TException

alterPartition

public void alterPartition(org.apache.hadoop.hive.metastore.IMetaStoreClient client,
                           String databaseName,
                           String tableName,
                           org.apache.hadoop.hive.metastore.api.Partition partition)
                    throws org.apache.hadoop.hive.metastore.api.InvalidOperationException,
                           org.apache.hadoop.hive.metastore.api.MetaException,
                           org.apache.thrift.TException

Specified by:

alterPartition in interface HiveShim

Throws:

org.apache.hadoop.hive.metastore.api.InvalidOperationException

org.apache.hadoop.hive.metastore.api.MetaException

org.apache.thrift.TException

createUDAFParameterInfo

public org.apache.hadoop.hive.ql.udf.generic.SimpleGenericUDAFParameterInfo createUDAFParameterInfo(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector[] params,
                                                                                                    boolean isWindowing,
                                                                                                    boolean distinct,
                                                                                                    boolean allColumns)

Description copied from interface: HiveShim

Creates SimpleGenericUDAFParameterInfo.

Specified by:

createUDAFParameterInfo in interface HiveShim

getMetaStoreUtilsClass

public Class<?> getMetaStoreUtilsClass()

Description copied from interface: HiveShim

Get the class of Hive's MetaStoreUtils because its package name was changed in Hive 3.1.0.

Specified by:

getMetaStoreUtilsClass in interface HiveShim

Returns:

MetaStoreUtils class

getHiveMetaStoreUtilsClass

public Class<?> getHiveMetaStoreUtilsClass()

Description copied from interface: HiveShim

Get the class of Hive's HiveMetaStoreUtils as it was split from MetaStoreUtils class in Hive 3.1.0.

Specified by:

getHiveMetaStoreUtilsClass in interface HiveShim

Returns:

HiveMetaStoreUtils class

getDateDataTypeClass

public Class<?> getDateDataTypeClass()

Description copied from interface: HiveShim

Hive Date data type class was changed in Hive 3.1.0.

Specified by:

getDateDataTypeClass in interface HiveShim

Returns:

Hive's Date class

getDateWritableClass

public Class<?> getDateWritableClass()

Description copied from interface: HiveShim

Gets writable class for Date type.

Specified by:

getDateWritableClass in interface HiveShim

getTimestampDataTypeClass

public Class<?> getTimestampDataTypeClass()

Description copied from interface: HiveShim

Hive Timestamp data type class was changed in Hive 3.1.0.

Specified by:

getTimestampDataTypeClass in interface HiveShim

Returns:

Hive's Timestamp class

getTimestampWritableClass

public Class<?> getTimestampWritableClass()

Description copied from interface: HiveShim

Gets writable class for Timestamp type.

Specified by:

getTimestampWritableClass in interface HiveShim

toHiveDateColStats

public org.apache.hadoop.hive.metastore.api.ColumnStatisticsData toHiveDateColStats(CatalogColumnStatisticsDataDate flinkDateColStats)

Description copied from interface: HiveShim

Generate Hive ColumnStatisticsData from Flink CatalogColumnStatisticsDataDate for DATE columns.

Specified by:

toHiveDateColStats in interface HiveShim

isDateStats

public boolean isDateStats(org.apache.hadoop.hive.metastore.api.ColumnStatisticsData colStatsData)

Description copied from interface: HiveShim

Whether a Hive ColumnStatisticsData is for DATE columns.

Specified by:

isDateStats in interface HiveShim

toFlinkDateColStats

public CatalogColumnStatisticsDataDate toFlinkDateColStats(org.apache.hadoop.hive.metastore.api.ColumnStatisticsData hiveDateColStats)

Description copied from interface: HiveShim

Generate Flink CatalogColumnStatisticsDataDate from Hive ColumnStatisticsData for DATE columns.

Specified by:

toFlinkDateColStats in interface HiveShim

getHiveRecordWriter

public org.apache.hadoop.hive.ql.exec.FileSinkOperator.RecordWriter getHiveRecordWriter(org.apache.hadoop.mapred.JobConf jobConf,
                                                                                        Class outputFormatClz,
                                                                                        Class<? extends org.apache.hadoop.io.Writable> outValClz,
                                                                                        boolean isCompressed,
                                                                                        Properties tableProps,
                                                                                        org.apache.hadoop.fs.Path outPath)

Description copied from interface: HiveShim

Get Hive's FileSinkOperator.RecordWriter.

Specified by:

getHiveRecordWriter in interface HiveShim

getHiveOutputFormatClass

public Class getHiveOutputFormatClass(Class outputFormatClz)

Description copied from interface: HiveShim

For a given OutputFormat class, get the corresponding HiveOutputFormat class.

Specified by:

getHiveOutputFormatClass in interface HiveShim

getFieldsFromDeserializer

public List<org.apache.hadoop.hive.metastore.api.FieldSchema> getFieldsFromDeserializer(org.apache.hadoop.conf.Configuration conf,
                                                                                        org.apache.hadoop.hive.metastore.api.Table table,
                                                                                        boolean skipConfError)

Description copied from interface: HiveShim

Get Hive table schema from deserializer.

Specified by:

getFieldsFromDeserializer in interface HiveShim

listBuiltInFunctions

public Set<String> listBuiltInFunctions()

Description copied from interface: HiveShim

List names of all built-in functions.

Specified by:

listBuiltInFunctions in interface HiveShim

getBuiltInFunctionInfo

public Optional<org.apache.hadoop.hive.ql.exec.FunctionInfo> getBuiltInFunctionInfo(String name)

Description copied from interface: HiveShim

Get a Hive built-in function by name.

Specified by:

getBuiltInFunctionInfo in interface HiveShim

getNotNullColumns

public Set<String> getNotNullColumns(org.apache.hadoop.hive.metastore.IMetaStoreClient client,
                                     org.apache.hadoop.conf.Configuration conf,
                                     String dbName,
                                     String tableName)

Description copied from interface: HiveShim

Get the set of columns that have NOT NULL constraints.

Specified by:

getNotNullColumns in interface HiveShim

getPrimaryKey

public Optional<UniqueConstraint> getPrimaryKey(org.apache.hadoop.hive.metastore.IMetaStoreClient client,
                                                String dbName,
                                                String tableName,
                                                byte requiredTrait)

Description copied from interface: HiveShim

Get the primary key of a Hive table and convert it to a UniqueConstraint. Return empty if the table doesn't have a primary key, or the constraint doesn't satisfy the desired trait, e.g. RELY.

Specified by:

getPrimaryKey in interface HiveShim

toHiveTimestamp

public Object toHiveTimestamp(Object flinkTimestamp)

Description copied from interface: HiveShim

Converts a Flink timestamp instance to what's expected by Hive.

Specified by:

toHiveTimestamp in interface HiveShim

toFlinkTimestamp

public java.time.LocalDateTime toFlinkTimestamp(Object hiveTimestamp)

Description copied from interface: HiveShim

Converts a hive timestamp instance to LocalDateTime which is expected by DataFormatConverter.

Specified by:

toFlinkTimestamp in interface HiveShim

toHiveDate

public Object toHiveDate(Object flinkDate)

Description copied from interface: HiveShim

Converts a Flink date instance to what's expected by Hive.

Specified by:

toHiveDate in interface HiveShim

toFlinkDate

public java.time.LocalDate toFlinkDate(Object hiveDate)

Description copied from interface: HiveShim

Converts a hive date instance to LocalDate which is expected by DataFormatConverter.

Specified by:

toFlinkDate in interface HiveShim

hivePrimitiveToWritable

public org.apache.hadoop.io.Writable hivePrimitiveToWritable(Object value)

Description copied from interface: HiveShim

Converts a Hive primitive java object to corresponding Writable object.

Specified by:

hivePrimitiveToWritable in interface HiveShim

createTableWithConstraints

public void createTableWithConstraints(org.apache.hadoop.hive.metastore.IMetaStoreClient client,
                                       org.apache.hadoop.hive.metastore.api.Table table,
                                       org.apache.hadoop.conf.Configuration conf,
                                       UniqueConstraint pk,
                                       List<Byte> pkTraits,
                                       List<String> notNullCols,
                                       List<Byte> nnTraits)

Description copied from interface: HiveShim

Creates a table with PK and NOT NULL constraints.

Specified by:

createTableWithConstraints in interface HiveShim

createOrcBulkWriterFactory

public BulkWriter.Factory<RowData> createOrcBulkWriterFactory(org.apache.hadoop.conf.Configuration conf,
                                                              String schema,
                                                              LogicalType[] fieldTypes)

Description copied from interface: HiveShim

Create orc BulkWriter.Factory for different hive versions.

Specified by:

createOrcBulkWriterFactory in interface HiveShim

registerTemporaryFunction

public void registerTemporaryFunction(String funcName,
                                      Class funcClass)

Specified by:

registerTemporaryFunction in interface HiveShim

loadPartition

public void loadPartition(org.apache.hadoop.hive.ql.metadata.Hive hive,
                          org.apache.hadoop.fs.Path loadPath,
                          String tableName,
                          Map<String,String> partSpec,
                          boolean isSkewedStoreAsSubdir,
                          boolean replace,
                          boolean isSrcLocal)

Specified by:

loadPartition in interface HiveShim

loadTable

public void loadTable(org.apache.hadoop.hive.ql.metadata.Hive hive,
                      org.apache.hadoop.fs.Path loadPath,
                      String tableName,
                      boolean replace,
                      boolean isSrcLocal)

Specified by:

loadTable in interface HiveShim